Vidya
completestandardruns/polar-dense: polar dense 4L/32H, 30000 updates (983 MB), 1.8469 bpb, 3.62 M MAC/byte, 2.14e+16 FLOPCORE 0.0182(+0.0093 vs previous)
2026-10-21 20:36 · max 500 examples/task · scorer v1 · bundle abc150c04fa3 · endpoint https://core.roland.blah.dev
| Task | Accuracy | Centered | Δ centered | n | Time |
|---|---|---|---|---|---|
| agi_eval_lsat_ar | 0.2130 | 0.0163 | -0.0815 | 230 | 443s |
| arc_challenge | 0.2580 | 0.0107 | -0.0213 | 500 | 979s |
| arc_easy | 0.3140 | 0.0853 | +0.0027 | 500 | 823s |
| bigbench_cs_algorithms | 0.0000 | 0.0000 | 0.0000 | 500 | 162s |
| bigbench_dyck_languages | 0.0560 | 0.0560 | +0.0220 | 500 | 194s |
| bigbench_language_identification | 0.2380 | 0.1617 | +0.0022 | 500 | 976s |
| bigbench_operators | 0.0762 | 0.0762 | +0.0095 | 210 | 66s |
| bigbench_qa_wikidata | 0.0000 | 0.0000 | 0.0000 | 500 | 65s |
| bigbench_repeat_copy_logic | 0.0000 | 0.0000 | 0.0000 | 32 | 12s |
| boolq | 0.4680 | -0.4000 | +0.2526 | 500 | 511s |
| commonsense_qa | 0.4200 | 0.2750 | +0.0200 | 500 | 794s |
| copa | 0.5300 | 0.0600 | -0.0600 | 100 | 8s |
| coqa | 0.0000 | 0.0000 | 0.0000 | 500 | 215s |
| hellaswag | 0.2520 | 0.0027 | +0.0240 | 500 | 970s |
| hellaswag_zeroshot | 0.2440 | -0.0080 | +0.0107 | 500 | 188s |
| jeopardy | 0.0000 | 0.0000 | 0.0000 | 500 | 149s |
| lambada_openai | 0.0000 | 0.0000 | 0.0000 | 500 | 50s |
| openbook_qa | 0.2460 | -0.0053 | -0.0187 | 500 | 52s |
| piqa | 0.5060 | 0.0120 | +0.0440 | 500 | 394s |
| squad | 0.0000 | 0.0000 | 0.0000 | 500 | 257s |
| winograd | 0.5128 | 0.0256 | -0.0147 | 273 | 25s |
| winogrande | 0.5160 | 0.0320 | +0.0120 | 500 | 46s |
Every example's outcome (per-option losses, greedy matches, correctness) is stored with this attempt · deltas are against the previous standard attempt.