Vidya ternary
completesmoketernary 4L/64H polar dense, 3K updates, lr 4x, 1.9818 bpb, 3.90 MB, 13.64 M MAC/byte (96% ternary adds)CORE -0.0106
2026-10-23 11:27 · max 50 examples/task · scorer v1 · bundle abc150c04fa3 · endpoint https://core.roland.blah.dev
| Task | Accuracy | Centered | Δ centered | n | Time |
|---|---|---|---|---|---|
| agi_eval_lsat_ar | 0.1800 | -0.0250 | — | 50 | 195s |
| arc_challenge | 0.3000 | 0.0667 | — | 50 | 174s |
| arc_easy | 0.2800 | 0.0400 | — | 50 | 147s |
| bigbench_cs_algorithms | 0.0000 | 0.0000 | — | 50 | 44s |
| bigbench_dyck_languages | 0.0400 | 0.0400 | — | 50 | 32s |
| bigbench_language_identification | 0.2200 | 0.1419 | — | 50 | 173s |
| bigbench_operators | 0.0800 | 0.0800 | — | 50 | 35s |
| bigbench_qa_wikidata | 0.0000 | 0.0000 | — | 50 | 12s |
| bigbench_repeat_copy_logic | 0.0000 | 0.0000 | — | 32 | 29s |
| boolq | 0.4600 | -0.4211 | — | 50 | 89s |
| commonsense_qa | 0.1600 | -0.0500 | — | 50 | 163s |
| copa | 0.5400 | 0.0800 | — | 50 | 9s |
| coqa | 0.0000 | 0.0000 | — | 50 | 39s |
| hellaswag | 0.2800 | 0.0400 | — | 50 | 186s |
| hellaswag_zeroshot | 0.2600 | 0.0133 | — | 50 | 47s |
| jeopardy | 0.0000 | 0.0000 | — | 50 | 28s |
| lambada_openai | 0.0000 | 0.0000 | — | 50 | 10s |
| openbook_qa | 0.2800 | 0.0400 | — | 50 | 13s |
| piqa | 0.4600 | -0.0800 | — | 50 | 75s |
| squad | 0.0000 | 0.0000 | — | 50 | 46s |
| winograd | 0.5000 | 0.0000 | — | 50 | 8s |
| winogrande | 0.4000 | -0.2000 | — | 50 | 10s |
Every example's outcome (per-option losses, greedy matches, correctness) is stored with this attempt.