Vidya
completesmokeruns/polar-dense: polar dense 4L/32H, 30000 updates, 1.8469 bpb, 3.62 M MAC/byte, 2.14e+16 FLOPCORE 0.0436(+0.0392 vs previous)
2026-10-21 20:21 · max 50 examples/task · scorer v1 · bundle abc150c04fa3 · endpoint https://core.roland.blah.dev
| Task | Accuracy | Centered | Δ centered | n | Time |
|---|---|---|---|---|---|
| agi_eval_lsat_ar | 0.2200 | 0.0250 | -0.0500 | 50 | 99s |
| arc_challenge | 0.2800 | 0.0400 | 0.0000 | 50 | 89s |
| arc_easy | 0.3200 | 0.0933 | +0.0533 | 50 | 77s |
| bigbench_cs_algorithms | 0.0000 | 0.0000 | 0.0000 | 50 | 17s |
| bigbench_dyck_languages | 0.0400 | 0.0400 | +0.0200 | 50 | 17s |
| bigbench_language_identification | 0.2800 | 0.2079 | +0.0880 | 50 | 95s |
| bigbench_operators | 0.0800 | 0.0800 | 0.0000 | 50 | 15s |
| bigbench_qa_wikidata | 0.0000 | 0.0000 | 0.0000 | 50 | 7s |
| bigbench_repeat_copy_logic | 0.0000 | 0.0000 | 0.0000 | 32 | 15s |
| boolq | 0.6200 | 0.0000 | +0.6316 | 50 | 53s |
| commonsense_qa | 0.4400 | 0.3000 | 0.0000 | 50 | 78s |
| copa | 0.5000 | 0.0000 | -0.0800 | 50 | 5s |
| coqa | 0.0000 | 0.0000 | 0.0000 | 50 | 23s |
| hellaswag | 0.3600 | 0.1467 | +0.1067 | 50 | 100s |
| hellaswag_zeroshot | 0.3400 | 0.1200 | +0.0800 | 50 | 23s |
| jeopardy | 0.0000 | 0.0000 | 0.0000 | 50 | 16s |
| lambada_openai | 0.0000 | 0.0000 | 0.0000 | 50 | 6s |
| openbook_qa | 0.3000 | 0.0667 | +0.0533 | 50 | 6s |
| piqa | 0.5400 | 0.0800 | +0.2400 | 50 | 43s |
| squad | 0.0000 | 0.0000 | 0.0000 | 50 | 26s |
| winograd | 0.4800 | -0.0400 | -0.1200 | 50 | 4s |
| winogrande | 0.4000 | -0.2000 | -0.1600 | 50 | 5s |
Every example's outcome (per-option losses, greedy matches, correctness) is stored with this attempt · deltas are against the previous smoke attempt.