CORE 0.0796
2026-08-20 08:36 · max 20 examples/task · scorer v1 · bundle · endpoint http://127.0.0.1:7990
| Task | Accuracy | Centered | Δ centered | n | Time |
|---|---|---|---|---|---|
| agi_eval_lsat_ar | 0.1500 | -0.0625 | — | 20 | 415s |
| arc_challenge | 0.1500 | -0.1333 | — | 20 | 141s |
| arc_easy | 0.4000 | 0.2000 | — | 20 | 168s |
| bigbench_cs_algorithms | 0.6000 | 0.6000 | — | 20 | 60s |
| bigbench_dyck_languages | 0.3000 | 0.3000 | — | 20 | 79s |
| bigbench_language_identification | 0.2500 | 0.1749 | — | 20 | 345s |
| bigbench_operators | 0.0000 | 0.0000 | — | 20 | 45s |
| bigbench_qa_wikidata | 0.3000 | 0.3000 | — | 20 | 16s |
| bigbench_repeat_copy_logic | 0.0000 | 0.0000 | — | 20 | 54s |
| boolq | 0.4000 | -0.5789 | — | 20 | 162s |
| commonsense_qa | 0.2000 | 0.0000 | — | 20 | 190s |
| copa | 0.6000 | 0.2000 | — | 20 | 6s |
| coqa | 0.1500 | 0.1500 | — | 20 | 55s |
| hellaswag | 0.4500 | 0.2667 | — | 20 | 550s |
| hellaswag_zeroshot | 0.5000 | 0.3333 | — | 20 | 88s |
| jeopardy | 0.0000 | 0.0000 | — | 20 | 34s |
| lambada_openai | 0.4000 | 0.4000 | — | 20 | 12s |
| openbook_qa | 0.2500 | 0.0000 | — | 20 | 8s |
| piqa | 0.5000 | 0.0000 | — | 20 | 65s |
| squad | 0.0000 | 0.0000 | — | 20 | 117s |
| winograd | 0.4500 | -0.1000 | — | 20 | 16s |
| winogrande | 0.3500 | -0.3000 | — | 20 | 16s |
Every example's outcome (per-option losses, greedy matches, correctness) is stored with this attempt.