- Score · relative 0–100
- 8.2
- Rank
- 65
- Status
- Partial 3/3
- Players generated
- 2026-10-09
- Match evidence updated
- 2026-10-24
- Results published
Why this result is partial: generation quality is below the publication threshold; some games cannot be scored: their only generation attempts failed because of the test harness or the provider and cannot currently be repeated. The rank uses the available reasoning-setting results.
Scores by reasoning setting
Available settings and grouping
Available settings
Checked on 2026-10-09 · Provider documentation
How settings are grouped
- Baseline: None — tested and included
- Balanced: Medium — tested and included
- Intensive: XHigh — tested and included
Scores by game
What this result shows
Strongest included setting: XHigh (13.8).
Across included settings, the highest game score is International Draughts (17.2); the lowest is 0.0, shared by 3 games. Each game counts every generation attempt, a failed one as 0. These are relative scores within each game, not direct comparisons of difficulty.
Match evidence measures these retained programs. It does not establish how a fresh generation will perform. Independent-generation variation is not estimated in this release.
Test protocol and evidence limits
- Model identifier
- upstage/solar-mini4
- Provider route (catalog)
- openrouter
- Prompt identity / profile
- Not retained in this result set / Not retained in this result set
- Prompt version
- Not retained in this result set
- Generation policy
- Generation-quality admission policy (see methodology)
- Requested output-token limits
- 131072 (49 of 49 indexed attempts)
Generation evidence
- None: 1 contributing player programs across 1 games
- Medium: 2 contributing player programs across 2 games
- XHigh: 3 contributing player programs across 3 games
Counts describe contributing programs, not independent replications: repairs and resends are not new independent samples. Independent-generation lineage is not retained.
| Game | Reasoning group | Programs | Working-program strength | Raw rating | Combined record (W / D / L) |
|---|---|---|---|---|---|
| Go | Balanced | 1 working of 1 attempt | 19.3 | 1142.2 | 53 / 79 / 384 |
| Xiangqi | Baseline | 1 working of 1 attempt | 40.6 | 1401.9 | 21 / 346 / 131 |
| Hex | Intensive | 1 working of 1 attempt | 46.1 | 1388.6 | 236 / 0 / 294 |
| International Draughts | Intensive | 1 working of 1 attempt | 31.3 | 1136.5 | 133 / 7 / 362 |
| International Draughts | Balanced | 1 working of 1 attempt | 20.3 | 965.9 | 84 / 5 / 413 |
| Game of the Amazons | Intensive | 1 working of 1 attempt | 33.1 | 1234.7 | 162 / 0 / 368 |
Each row covers all the model's programs in one game and setting. Strength and raw rating are attempt-weighted means; under them, each program's score and the lowest and highest rating describe the programs (not a confidence interval). The combined record adds up their matches: evidence volume, not strength.
Failures and budgets
17 model-code generation failures; 1 provider/infrastructure failures excluded from the generation denominator. Runtime player faults are separate: 0 recorded one-sided faults across contributing program rows. A crash or illegal move pinned on one program also counts as that program's loss in its match record; other faults, such as timeouts, are left out of rating evidence.
Repair/resend outcomes and available costs are included. Exact retry ceilings and execution budgets are not retained in this result set; see the methodology for the public policy. Do not infer a measured setting from a current provider catalog.
Player program results
XHigh: 0% / 38% / 62% · n 8Medium: 14% / 14% / 72% · n 7None: 13% / 0% / 87% · n 8 All three benchmark settings testedMany programs failedGeneration cost
$0.0084average estimated cost per game and reasoning setting
- Median
- $0.0080
- Range
- $0.000640–$0.02
- Known cost subtotal (incomplete or unverified charges)
- $0.20
- Cost data
- 24 combinations · 24 runs
- Price source
- Provider-reported
- Price list
- boardgame-list-prices-2026-10-09
- Average output
- 38k tokens
- Output range
- 1.4k–112k tokens
- Runs with output-token data
- 24
How cost is measured
Comparison cost gives equal weight to each model, game, and reasoning-setting combination. Recorded generation attempts include repairs and resends when cost evidence is available; runtime compute and service operation are excluded. Cost and output-token counts use separate telemetry denominators. A list-price estimate is not actual expenditure; mixed evidence is not a uniform standard-price comparison.
Test protocol
- Combines this model's tested reasoning settings
- Player-program generation; retained protocol details are under “Test protocol and evidence limits” above