Vidya
completesmokeruns/polar-dense: polar dense 4L/32H, 3000 updates, 2.0490 bpb, 3.62 M MAC/byte, 2.14e+15 FLOPCORE 0.0044(+0.0182 vs previous)
2026-10-21 10:14 · max 50 examples/task · scorer v1 · bundle abc150c04fa3 · endpoint https://core.roland.blah.dev
| Task | Accuracy | Centered | Δ centered | n | Time |
|---|---|---|---|---|---|
| agi_eval_lsat_ar | 0.2600 | 0.0750 | +0.0500 | 50 | 104s |
| arc_challenge | 0.2800 | 0.0400 | 0.0000 | 50 | 94s |
| arc_easy | 0.2800 | 0.0400 | -0.0533 | 50 | 82s |
| bigbench_cs_algorithms | 0.0000 | 0.0000 | 0.0000 | 50 | 18s |
| bigbench_dyck_languages | 0.0200 | 0.0200 | +0.0200 | 50 | 18s |
| bigbench_language_identification | 0.2000 | 0.1199 | -0.0220 | 50 | 103s |
| bigbench_operators | 0.0800 | 0.0800 | +0.0600 | 50 | 16s |
| bigbench_qa_wikidata | 0.0000 | 0.0000 | 0.0000 | 50 | 7s |
| bigbench_repeat_copy_logic | 0.0000 | 0.0000 | 0.0000 | 32 | 14s |
| boolq | 0.3800 | -0.6316 | 0.0000 | 50 | 53s |
| commonsense_qa | 0.4400 | 0.3000 | +0.4000 | 50 | 83s |
| copa | 0.5400 | 0.0800 | -0.0800 | 50 | 4s |
| coqa | 0.0000 | 0.0000 | 0.0000 | 50 | 23s |
| hellaswag | 0.2800 | 0.0400 | +0.0267 | 50 | 101s |
| hellaswag_zeroshot | 0.2800 | 0.0400 | +0.0533 | 50 | 19s |
| jeopardy | 0.0000 | 0.0000 | 0.0000 | 50 | 16s |
| lambada_openai | 0.0000 | 0.0000 | 0.0000 | 50 | 7s |
| openbook_qa | 0.2600 | 0.0133 | +0.0267 | 50 | 6s |
| piqa | 0.4200 | -0.1600 | -0.2000 | 50 | 43s |
| squad | 0.0000 | 0.0000 | 0.0000 | 50 | 28s |
| winograd | 0.5400 | 0.0800 | +0.0400 | 50 | 5s |
| winogrande | 0.4800 | -0.0400 | +0.0800 | 50 | 5s |
Every example's outcome (per-option losses, greedy matches, correctness) is stored with this attempt · deltas are against the previous smoke attempt.