evals

Vidya

completestandardruns3k/polar-dense: polar dense 4L/32H, 3000 updates (98 MB), 2.0490 bpb, 3.62 M MAC/byte, 2.14e+15 FLOP

CORE 0.0089

2026-10-21 11:42 · max 500 examples/task · scorer v1 · bundle abc150c04fa3 · endpoint https://core.roland.blah.dev

TaskAccuracyCenteredΔ centerednTime
agi_eval_lsat_ar0.27830.0978—230510s
arc_challenge0.27400.0320—500947s
arc_easy0.31200.0827—500858s
bigbench_cs_algorithms0.00000.0000—500191s
bigbench_dyck_languages0.03400.0340—500243s
bigbench_language_identification0.23600.1595—5001146s
bigbench_operators0.06670.0667—21079s
bigbench_qa_wikidata0.00000.0000—50069s
bigbench_repeat_copy_logic0.00000.0000—3215s
boolq0.3720-0.6526—500567s
commonsense_qa0.40400.2550—500856s
copa0.56000.1200—1009s
coqa0.00000.0000—500268s
hellaswag0.2340-0.0213—5001068s
hellaswag_zeroshot0.2360-0.0187—500202s
jeopardy0.00000.0000—500208s
lambada_openai0.00000.0000—50058s
openbook_qa0.26000.0133—50066s
piqa0.4840-0.0320—500425s
squad0.00000.0000—500313s
winograd0.52010.0403—27325s
winogrande0.51000.0200—50049s

Every example's outcome (per-option losses, greedy matches, correctness) is stored with this attempt.