- Score · relative 0–100
- 57.3
- Rank
- 22
- Status
- Partial 3/3
- Players generated
- 2026-10-09
- Match evidence updated
- 2026-10-24
- Results published
Why this result is partial: some games cannot be scored: their only generation attempts failed because of the test harness or the provider and cannot currently be repeated. The rank uses the available reasoning-setting results.
Scores by reasoning setting
Available settings and grouping
Available settings
Checked on 2026-10-09 · Provider documentation
How settings are grouped
- Baseline: None — tested and included
- Balanced: Medium — tested and included
- Intensive: XHigh — tested and included
Scores by game
What this result shows
Strongest included setting: XHigh (74.9).
Across included settings, the highest game score is International Draughts (70.0); the lowest is Go (40.8). Each game counts every generation attempt, a failed one as 0. These are relative scores within each game, not direct comparisons of difficulty.
Match evidence measures these retained programs. It does not establish how a fresh generation will perform. Independent-generation variation is not estimated in this release.
Test protocol and evidence limits
- Model identifier
- claude-haiku-5-5
- Provider route (catalog)
- anthropic
- Prompt identity / profile
- Not retained in this result set / Not retained in this result set
- Prompt version
- Not retained in this result set
- Generation policy
- Generation-quality admission policy (see methodology)
- Requested output-token limits
- 128000 (28 of 28 indexed attempts)
Generation evidence
- None: 8 contributing player programs across 8 games
- Medium: 8 contributing player programs across 8 games
- XHigh: 7 contributing player programs across 7 games
Counts describe contributing programs, not independent replications: repairs and resends are not new independent samples. Independent-generation lineage is not retained.
| Game | Reasoning group | Programs | Working-program strength | Raw rating | Combined record (W / D / L) |
|---|---|---|---|---|---|
| Go | Intensive | 1 working of 1 attempt | 41.5 | 1388.1 | 187 / 34 / 295 |
| Go | Balanced | 1 working of 1 attempt | 45.6 | 1426.7 | 158 / 133 / 225 |
| Go | Baseline | 1 working of 1 attempt | 35.4 | 1327.7 | 128 / 90 / 298 |
| Chess | Intensive | 1 working of 1 attempt | 70.5 | 1645.8 | 223 / 216 / 41 |
| Chess | Balanced | 1 working of 1 attempt | 63.0 | 1576.3 | 168 / 252 / 60 |
| Chess | Baseline | 1 working of 1 attempt | 42.4 | 1412.9 | 5 / 381 / 94 |
| Shogi | Balanced | 1 working of 1 attempt | 67.8 | 1600.7 | 270 / 94 / 130 |
| Shogi | Baseline | 1 working of 1 attempt | 43.1 | 1367.4 | 103 / 182 / 209 |
| Xiangqi | Intensive | 1 working of 1 attempt | 80.0 | 1753.9 | 332 / 120 / 46 |
| Xiangqi | Balanced | 1 working of 1 attempt | 67.7 | 1622.0 | 225 / 208 / 65 |
| Xiangqi | Baseline | 1 working of 1 attempt | 41.7 | 1410.7 | 40 / 319 / 139 |
| Hex | Intensive | 1 working of 1 attempt | 86.4 | 2106.4 | 453 / 0 / 77 |
| Hex | Balanced | 1 working of 1 attempt | 57.6 | 1532.8 | 296 / 0 / 234 |
| Hex | Baseline | 1 working of 1 attempt | 16.2 | 937.8 | 81 / 0 / 449 |
| International Draughts | Intensive | 1 working of 1 attempt | 91.8 | 2229.9 | 437 / 39 / 26 |
| International Draughts | Balanced | 1 working of 1 attempt | 68.9 | 1733.4 | 327 / 6 / 169 |
| International Draughts | Baseline | 1 working of 1 attempt | 49.3 | 1409.5 | 217 / 17 / 268 |
| Game of the Amazons | Intensive | 1 working of 1 attempt | 67.1 | 1674.1 | 347 / 0 / 183 |
| Game of the Amazons | Balanced | 1 working of 1 attempt | 56.9 | 1535.5 | 290 / 0 / 240 |
| Game of the Amazons | Baseline | 1 working of 1 attempt | 52.0 | 1472.9 | 263 / 0 / 267 |
| Tumbleweed | Intensive | 1 working of 1 attempt | 86.7 | 2060.0 | 458 / 1 / 79 |
| Tumbleweed | Balanced | 1 working of 1 attempt | 33.7 | 1210.7 | 42 / 251 / 245 |
| Tumbleweed | Baseline | 1 working of 1 attempt | 34.3 | 1217.0 | 33 / 275 / 230 |
Each row covers all the model's programs in one game and setting. Strength and raw rating are attempt-weighted means; under them, each program's score and the lowest and highest rating describe the programs (not a confidence interval). The combined record adds up their matches: evidence volume, not strength.
Failures and budgets
0 model-code generation failures; 1 provider/infrastructure failures excluded from the generation denominator. Runtime player faults are separate: 0 recorded one-sided faults across contributing program rows. A crash or illegal move pinned on one program also counts as that program's loss in its match record; other faults, such as timeouts, are left out of rating evidence.
Repair/resend outcomes and available costs are included. Exact retry ceilings and execution budgets are not retained in this result set; see the methodology for the public policy. Do not infer a measured setting from a current provider catalog.
Player program results
XHigh: 100% / 0% / 0% · n 7Medium: 100% / 0% / 0% · n 8None: 88% / 12% / 0% · n 8 All three benchmark settings testedGeneration cost
$0.02average estimated cost per game and reasoning setting
- Median
- $0.01
- Range
- $0.0020–$0.07
- Estimated total for recorded runs
- $0.50
- Cost data
- 24 combinations · 24 runs
- Price source
- Standard list-price estimate
- Price list
- boardgame-list-prices-2026-10-09
- Average output
- 39k tokens
- Output range
- 1.6k–128k tokens
- Runs with output-token data
- 24
How cost is measured
Comparison cost gives equal weight to each model, game, and reasoning-setting combination. Recorded generation attempts include repairs and resends when cost evidence is available; runtime compute and service operation are excluded. Cost and output-token counts use separate telemetry denominators. A list-price estimate is not actual expenditure; mixed evidence is not a uniform standard-price comparison.
Test protocol
- Combines this model's tested reasoning settings
- Player-program generation; retained protocol details are under “Test protocol and evidence limits” above