- Score · relative 0–100
- 36.2
- Rank
- 52
- Status
- Complete 3/3
- Rank change
- -1
- Players generated
- 2026-07-02 – 2026-10-09
- Match evidence updated
- 2026-10-26
- Results published
Scores by reasoning setting
Available settings and grouping
Available settings
Checked on 2026-07-21 · Provider documentation
How settings are grouped
- Baseline: Low — tested and included
- Balanced: Medium — tested and included
- Intensive: High — tested and included
Scores by game
What this result shows
Strongest included setting: Low (42.3).
Across included settings, the highest game score is Hex (48.4); the lowest is Shogi (17.0). Each game counts every generation attempt, a failed one as 0. These are relative scores within each game, not direct comparisons of difficulty.
Match evidence measures these retained programs. It does not establish how a fresh generation will perform. Independent-generation variation is not estimated in this release.
Test protocol and evidence limits
- Model identifier
- o3
- Provider route (catalog)
- openai
- Prompt identity / profile
- Not retained in this result set / Not retained in this result set
- Prompt version
- Not retained in this result set
- Generation policy
- Generation-quality admission policy (see methodology)
- Requested output-token limits
- 100000 (10 of 82 indexed attempts)
Generation evidence
- Low: 8 contributing player programs across 8 games
- Medium: 6 contributing player programs across 6 games
- High: 19 contributing player programs across 6 games
Counts describe contributing programs, not independent replications: repairs and resends are not new independent samples. Independent-generation lineage is not retained.
| Game | Reasoning group | Programs | Working-program strength | Raw rating | Combined record (W / D / L) |
|---|---|---|---|---|---|
| Go | Intensive | 3 working of 4 attempts | 60.8 program scores 66.5 · 65.2 · 50.7 | 1576.1 program range 1476–1633 | 830 / 157 / 561 |
| Go | Balanced | 1 working of 1 attempt | 37.5 | 1348.5 | 134 / 99 / 283 |
| Go | Baseline | 1 working of 1 attempt | 37.9 | 1352.4 | 164 / 43 / 309 |
| Chess | Baseline | 1 working of 1 attempt | 42.5 | 1413.7 | 7 / 378 / 95 |
| Shogi | Intensive | 1 working of 4 attempts | 36.2 | 1307.3 | 98 / 127 / 269 |
| Shogi | Baseline | 1 working of 1 attempt | 41.9 | 1357.4 | 122 / 133 / 239 |
| Xiangqi | Balanced | 1 working of 1 attempt | 41.2 | 1406.7 | 30 / 334 / 134 |
| Xiangqi | Baseline | 1 working of 1 attempt | 44.7 | 1433.7 | 81 / 266 / 151 |
| Hex | Intensive | 4 working of 4 attempts | 49.0 program scores 58.9 · 55.6 · 44.3 · 37.2 | 1425.1 program range 1275–1550 | 1006 / 0 / 1114 |
| Hex | Balanced | 1 working of 1 attempt | 44.7 | 1371.7 | 229 / 0 / 301 |
| Hex | Baseline | 1 working of 1 attempt | 51.4 | 1455.4 | 264 / 0 / 266 |
| International Draughts | Intensive | 4 working of 4 attempts | 41.5 program scores 53.5 · 47.1 · 45.4 · 20.0 | 1289.8 program range 960–1474 | 744 / 15 / 1249 |
| International Draughts | Balanced | 1 working of 1 attempt | 45.2 | 1346.7 | 190 / 29 / 283 |
| International Draughts | Baseline | 1 working of 1 attempt | 47.9 | 1387.1 | 214 / 8 / 280 |
| Game of the Amazons | Intensive | 4 working of 4 attempts | 43.3 program scores 55.8 · 45.0 · 43.0 · 29.5 | 1363.1 program range 1185–1522 | 866 / 0 / 1254 |
| Game of the Amazons | Balanced | 1 working of 1 attempt | 58.7 | 1559.0 | 300 / 0 / 230 |
| Game of the Amazons | Baseline | 1 working of 1 attempt | 39.2 | 1312.9 | 194 / 0 / 336 |
| Tumbleweed | Intensive | 3 working of 4 attempts | 47.8 program scores 74.7 · 40.0 · 28.8 | 1404.3 program range 1157–1778 | 608 / 241 / 765 |
| Tumbleweed | Balanced | 1 working of 1 attempt | 47.1 | 1356.2 | 189 / 93 / 256 |
| Tumbleweed | Baseline | 1 working of 1 attempt | 33.3 | 1206.4 | 26 / 279 / 233 |
Each row covers all the model's programs in one game and setting. Strength and raw rating are attempt-weighted means; under them, each program's score and the lowest and highest rating describe the programs (not a confidence interval). The combined record adds up their matches: evidence volume, not strength.
Failures and budgets
11 model-code generation failures; 0 provider/infrastructure failures excluded from the generation denominator. Runtime player faults are separate: 0 recorded one-sided faults across contributing program rows. A crash or illegal move pinned on one program also counts as that program's loss in its match record; other faults, such as timeouts, are left out of rating evidence.
Repair/resend outcomes and available costs are included. Exact retry ceilings and execution budgets are not retained in this result set; see the methodology for the public policy. Do not infer a measured setting from a current provider catalog.
Player program results
High: 68% / 0% / 32% · n 28Medium: 75% / 0% / 25% · n 8Low: 88% / 12% / 0% · n 8 All three benchmark settings testedMany programs failedGeneration cost
$0.06average estimated cost per game and reasoning setting
- Median
- $0.05
- Range
- $0.03–$0.11
- Known cost subtotal (incomplete or unverified charges)
- $3.66
- Cost data
- 24 combinations · 48 runs
- Price source
- Combined: Recorded attempt + Standard list-price estimate
- Price list
- boardgame-list-prices-2026-10-09
- Average output
- 7.2k tokens
- Output range
- 1.9k–16k tokens
- Runs with output-token data
- 48
How cost is measured
Comparison cost gives equal weight to each model, game, and reasoning-setting combination. Recorded generation attempts include repairs and resends when cost evidence is available; runtime compute and service operation are excluded. Cost and output-token counts use separate telemetry denominators. A list-price estimate is not actual expenditure; mixed evidence is not a uniform standard-price comparison.
Test protocol
- Combines this model's tested reasoning settings
- Player-program generation; retained protocol details are under “Test protocol and evidence limits” above