CORE 0.0610
2026-10-11 07:01 · max 50 examples/task · scorer v1 · bundle abc150c04fa3 · endpoint http://127.0.0.1:18045
| Task | Accuracy | Centered | Δ centered | n | Time |
|---|---|---|---|---|---|
| agi_eval_lsat_ar | 0.2400 | 0.0500 | — | 50 | 28s |
| arc_challenge | 0.2400 | -0.0133 | — | 50 | 26s |
| arc_easy | 0.3000 | 0.0667 | — | 50 | 26s |
| bigbench_cs_algorithms | 0.3800 | 0.3800 | — | 50 | 225s |
| bigbench_dyck_languages | 0.0000 | 0.0000 | — | 50 | 221s |
| bigbench_language_identification | 0.1800 | 0.0979 | — | 50 | 28s |
| bigbench_operators | 0.0000 | 0.0000 | — | 50 | 231s |
| bigbench_qa_wikidata | 0.1000 | 0.1000 | — | 50 | 220s |
| bigbench_repeat_copy_logic | 0.0000 | 0.0000 | — | 32 | 143s |
| boolq | 0.6200 | 0.0000 | — | 50 | 28s |
| commonsense_qa | 0.3400 | 0.1750 | — | 50 | 25s |
| copa | 0.6200 | 0.2400 | — | 50 | 22s |
| coqa | 0.0400 | 0.0400 | — | 50 | 222s |
| hellaswag | 0.2800 | 0.0400 | — | 50 | 29s |
| hellaswag_zeroshot | 0.2400 | -0.0133 | — | 50 | 27s |
| jeopardy | 0.0000 | 0.0000 | — | 50 | 221s |
| lambada_openai | 0.2600 | 0.2600 | — | 50 | 219s |
| openbook_qa | 0.2200 | -0.0400 | — | 50 | 23s |
| piqa | 0.5200 | 0.0400 | — | 50 | 24s |
| squad | 0.0000 | 0.0000 | — | 50 | 225s |
| winograd | 0.4800 | -0.0400 | — | 50 | 24s |
| winogrande | 0.4800 | -0.0400 | — | 50 | 24s |
Every example's outcome (per-option losses, greedy matches, correctness) is stored with this attempt.