Scores by reasoning setting

None (Baseline group, tested and included)42.4

Every tested setting is included in the overall score.

Available settings and grouping

Available settings

None

Checked on 2026-07-21 · Provider documentation

How settings are grouped

Baseline: None — tested and included
Balanced: Not available for this model
Intensive: Not available for this model

Scores by game

Go61.8
Chess42.9
Shogi39.3
Xiangqi38.6
Hex13.9
International Draughts50.1
Game of the Amazons48.0
Tumbleweed44.4

Each game counts every generation attempt, a failed one as 0 (“1 of 2 failed” marks games with failed attempts). Scores are relative within each game, across the included settings; they do not compare game difficulty.

What this result shows

Strongest included setting: None (42.4).

Across included settings, the highest game score is Go (61.8); the lowest is Hex (13.9). Each game counts every generation attempt, a failed one as 0. These are relative scores within each game, not direct comparisons of difficulty.

Match evidence measures these retained programs. It does not establish how a fresh generation will perform. Independent-generation variation is not estimated in this release.

Test protocol and evidence limits

Model identifier: gpt-4.1
Provider route (catalog): openai
Prompt identity / profile: Not retained in this result set / Not retained in this result set
Prompt version: Not retained in this result set
Generation policy: Generation-quality admission policy (see methodology)
Requested output-token limits: Not retained (0 of 11 indexed attempts)

Generation evidence

None: 8 contributing player programs across 8 games

Counts describe contributing programs, not independent replications: repairs and resends are not new independent samples. Independent-generation lineage is not retained.

Game	Reasoning group	Programs	Working-program strength	Raw rating	Combined record (W / D / L)
Go	Baseline	1 working of 1 attempt	61.8	1584.5	291 / 34 / 191
Chess	Baseline	1 working of 1 attempt	42.9	1417.0	5 / 386 / 89
Shogi	Baseline	1 working of 1 attempt	39.3	1334.5	117 / 118 / 259
Xiangqi	Baseline	1 working of 1 attempt	38.6	1386.6	24 / 321 / 153
Hex	Baseline	1 working of 1 attempt	13.9	889.1	69 / 0 / 461
International Draughts	Baseline	1 working of 1 attempt	50.1	1421.6	227 / 5 / 269
Game of the Amazons	Baseline	1 working of 1 attempt	48.0	1422.3	241 / 0 / 289
Tumbleweed	Baseline	1 working of 1 attempt	44.4	1326.5	109 / 226 / 203

Each row covers all the model's programs in one game and setting. Strength and raw rating are attempt-weighted means; under them, each program's score and the lowest and highest rating describe the programs (not a confidence interval). The combined record adds up their matches: evidence volume, not strength.

Failures and budgets

0 model-code generation failures; 0 provider/infrastructure failures excluded from the generation denominator. Runtime player faults are separate: 1 recorded one-sided faults across contributing program rows. A crash or illegal move pinned on one program also counts as that program's loss in its match record; other faults, such as timeouts, are left out of rating evidence.

Repair/resend outcomes and available costs are included. Exact retry ceilings and execution budgets are not retained in this result set; see the methodology for the public policy. Do not infer a measured setting from a current provider catalog.

Player program results

None: 63% / 37% / 0% · n 8 No Overall rankIntensive not supportedBalanced not supported

Generation cost

$0.06average estimated cost per game and reasoning setting

Median: $0.04
Range: $0.03–$0.09
Known cost subtotal (incomplete or unverified charges): $0.46
Cost data: 8 combinations · 8 runs
Price source: Standard list-price estimate
Price list: boardgame-list-prices-2026-10-09
Average output: 4.4k tokens
Output range: 2.3k–7.6k tokens
Runs with output-token data: 8

How cost is measured

Comparison cost gives equal weight to each model, game, and reasoning-setting combination. Recorded generation attempts include repairs and resends when cost evidence is available; runtime compute and service operation are excluded. Cost and output-token counts use separate telemetry denominators. A list-price estimate is not actual expenditure; mixed evidence is not a uniform standard-price comparison.

Test protocol

Combines this model's tested reasoning settings
Player-program generation; retained protocol details are under “Test protocol and evidence limits” above

GPT-4.1