evals

Vidya

completesmokeruns/polar-dense: polar dense 4L/32H, 3000 updates, 2.0490 bpb, 3.62 M MAC/byte, 2.14e+15 FLOP

CORE 0.0044(+0.0182 vs previous)

2026-10-21 10:14 · max 50 examples/task · scorer v1 · bundle abc150c04fa3 · endpoint https://core.roland.blah.dev

TaskAccuracyCenteredΔ centerednTime
agi_eval_lsat_ar0.26000.0750+0.050050104s
arc_challenge0.28000.04000.00005094s
arc_easy0.28000.0400-0.05335082s
bigbench_cs_algorithms0.00000.00000.00005018s
bigbench_dyck_languages0.02000.0200+0.02005018s
bigbench_language_identification0.20000.1199-0.022050103s
bigbench_operators0.08000.0800+0.06005016s
bigbench_qa_wikidata0.00000.00000.0000507s
bigbench_repeat_copy_logic0.00000.00000.00003214s
boolq0.3800-0.63160.00005053s
commonsense_qa0.44000.3000+0.40005083s
copa0.54000.0800-0.0800504s
coqa0.00000.00000.00005023s
hellaswag0.28000.0400+0.026750101s
hellaswag_zeroshot0.28000.0400+0.05335019s
jeopardy0.00000.00000.00005016s
lambada_openai0.00000.00000.0000507s
openbook_qa0.26000.0133+0.0267506s
piqa0.4200-0.1600-0.20005043s
squad0.00000.00000.00005028s
winograd0.54000.0800+0.0400505s
winogrande0.4800-0.0400+0.0800505s

Every example's outcome (per-option losses, greedy matches, correctness) is stored with this attempt · deltas are against the previous smoke attempt.