evals

Vidya

completesmokeruns/cpu-polar: polar dense 3L/16H, 2000 updates, 2.3583 bpb, 0.78 M MAC/byte, 1.54e+14 FLOP

CORE -0.0138

2026-10-21 08:52 · max 50 examples/task · scorer v1 · bundle abc150c04fa3 · endpoint https://core.roland.blah.dev

TaskAccuracyCenteredΔ centerednTime
agi_eval_lsat_ar0.22000.0250—5059s
arc_challenge0.28000.0400—5053s
arc_easy0.32000.0933—5046s
bigbench_cs_algorithms0.00000.0000—5013s
bigbench_dyck_languages0.00000.0000—5012s
bigbench_language_identification0.22000.1419—5059s
bigbench_operators0.02000.0200—5013s
bigbench_qa_wikidata0.00000.0000—506s
bigbench_repeat_copy_logic0.00000.0000—329s
boolq0.3800-0.6316—5035s
commonsense_qa0.1200-0.1000—5049s
copa0.58000.1600—504s
coqa0.00000.0000—5015s
hellaswag0.26000.0133—5059s
hellaswag_zeroshot0.2400-0.0133—5013s
jeopardy0.00000.0000—5012s
lambada_openai0.00000.0000—504s
openbook_qa0.2400-0.0133—504s
piqa0.52000.0400—5029s
squad0.00000.0000—5020s
winograd0.52000.0400—504s
winogrande0.4400-0.1200—504s

Every example's outcome (per-option losses, greedy matches, correctness) is stored with this attempt.