evals

Vidya

completesmokeruns/polar-dense: polar dense 4L/32H, 30000 updates, 1.8469 bpb, 3.62 M MAC/byte, 2.14e+16 FLOP

CORE 0.0436(+0.0392 vs previous)

2026-10-21 20:21 · max 50 examples/task · scorer v1 · bundle abc150c04fa3 · endpoint https://core.roland.blah.dev

TaskAccuracyCenteredΔ centerednTime
agi_eval_lsat_ar0.22000.0250-0.05005099s
arc_challenge0.28000.04000.00005089s
arc_easy0.32000.0933+0.05335077s
bigbench_cs_algorithms0.00000.00000.00005017s
bigbench_dyck_languages0.04000.0400+0.02005017s
bigbench_language_identification0.28000.2079+0.08805095s
bigbench_operators0.08000.08000.00005015s
bigbench_qa_wikidata0.00000.00000.0000507s
bigbench_repeat_copy_logic0.00000.00000.00003215s
boolq0.62000.0000+0.63165053s
commonsense_qa0.44000.30000.00005078s
copa0.50000.0000-0.0800505s
coqa0.00000.00000.00005023s
hellaswag0.36000.1467+0.106750100s
hellaswag_zeroshot0.34000.1200+0.08005023s
jeopardy0.00000.00000.00005016s
lambada_openai0.00000.00000.0000506s
openbook_qa0.30000.0667+0.0533506s
piqa0.54000.0800+0.24005043s
squad0.00000.00000.00005026s
winograd0.4800-0.0400-0.1200504s
winogrande0.4000-0.2000-0.1600505s

Every example's outcome (per-option losses, greedy matches, correctness) is stored with this attempt · deltas are against the previous smoke attempt.