evals

Vidya ternary

completesmoketernary 4L/64H polar dense, 3K updates, lr 4x, 1.9818 bpb, 3.90 MB, 13.64 M MAC/byte (96% ternary adds)

CORE -0.0106

2026-10-23 11:27 · max 50 examples/task · scorer v1 · bundle abc150c04fa3 · endpoint https://core.roland.blah.dev

TaskAccuracyCenteredΔ centerednTime
agi_eval_lsat_ar0.1800-0.0250—50195s
arc_challenge0.30000.0667—50174s
arc_easy0.28000.0400—50147s
bigbench_cs_algorithms0.00000.0000—5044s
bigbench_dyck_languages0.04000.0400—5032s
bigbench_language_identification0.22000.1419—50173s
bigbench_operators0.08000.0800—5035s
bigbench_qa_wikidata0.00000.0000—5012s
bigbench_repeat_copy_logic0.00000.0000—3229s
boolq0.4600-0.4211—5089s
commonsense_qa0.1600-0.0500—50163s
copa0.54000.0800—509s
coqa0.00000.0000—5039s
hellaswag0.28000.0400—50186s
hellaswag_zeroshot0.26000.0133—5047s
jeopardy0.00000.0000—5028s
lambada_openai0.00000.0000—5010s
openbook_qa0.28000.0400—5013s
piqa0.4600-0.0800—5075s
squad0.00000.0000—5046s
winograd0.50000.0000—508s
winogrande0.4000-0.2000—5010s

Every example's outcome (per-option losses, greedy matches, correctness) is stored with this attempt.