What we tried, and what we rejected

The shipped model came out of 60 proposals across 4 model families, scored on a walk-forward validation season with the locked test seasons opened once. The first 31 rounds (2026-11-07, Claude) produced what we run; of the 29 new rounds here, 4 beat production, 10 tied, 6 were worse and 4 never produced a runnable plugin, and 1 beat production on validation but lost on the locked test, so it was not shipped. A further 5 were restore-only rounds that reproduce an earlier configuration and are not counted twice. All 29 are listed below, including the ones that lost.

What runs in production

Validation RPS
0.2039
Validation log loss
0.9976
Locked test RPS
0.2051
Test log loss
1.0047

On the same test matches: results-only model RPS 0.2087, market-prior-only model 0.2071. Of 10 probability buckets, 5 are well calibrated and 2 are too thin to judge — full buckets in the calibration record.

Injury-intel shadow experiment (pre-registered, no interim result)

Before each kick-off a web-search model reads injury and lineup reports and returns a fact-and-source list. It never touches the published probabilities; it is scored alongside them on the same matches. The adoption rule was fixed before any data arrived: the 90% confidence interval of the paired log-loss difference must sit entirely on the better side, otherwise the idea is archived as a negative. Below 150 scored matches we publish no interim difference.

Recorded
100
Scored
98
Pre-registered threshold
150

52 matches to go; the verdict will appear here either way, positive or negative.

The last 29 proposals and their results

Lower validation RPS is better; production sits at 0.2039. All 60 rounds cost $59.31 (claude-fable-5-1 $24.65, gpt-5.1 $0.67, claude-fable-5-2 $8.48, gpt-6.1-sol $0.86).

Machine-readable: /data/football-model-search.json

ProposerRoundHypothesisValidation RPSTest RPSVerdict
gpt-5.131Rescale score-only goal observations so each family’s long-run training goals match its own mean of real goals.0.2039—Tied
gpt-5.132Replace the family-wide goal-level correction with a tier-aware level (tier-1 vs tier-2), blended by recent tier mix.0.2039—Tied
gpt-5.133Give the most recent match a small extra weight per refit so ratings react faster to genuine strength changes.0.2039—Tied
gpt-5.134Add a small symmetric rest-days term to the log-strength ratio.0.2040—Worse
gpt-5.135Reweight short-term form/tempo residuals to give market-observed matches slightly more effective samples.0.2040—Worse
gpt-5.136Taper the favourite stretch and rho shift in the first ten matchweeks.(plugin longer than 20,000 characters)——Contract rejection
gpt-5.137Ship the round-26 model unchanged.(plugin longer than 20,000 characters)——Contract rejection
claude-fable-5-231Opponent-adjust the fast form and tempo layers by backfitting, with the shrinkage re-derived each pass.0.2039—Tied
claude-fable-5-232Adjust the form layer for rest asymmetry.(no <plugin> block in the answer)——Contract rejection
claude-fable-5-233Scale the tier-gap correction by the priced share of the current tier-2 teams’ training evidence.0.2039—Tied
claude-fable-5-234Count the season break as one extra in-season half-life for priced rows.0.2040—Worse
claude-fable-5-235Split the market residual by league tier.(no <plugin> block in the answer)——Contract rejection
claude-fable-5-236Scale the tier-gap correction by the mean priced share of the family.0.2039—Tied
claude-fable-5-237Make the rho shift depend on match closeness.0.2039—Tied
gpt-6.1-sol38Preserve the predicted goal total when applying strength, tier and form corrections.0.2039—Tied
gpt-6.1-sol39Use only priced observations in the fast form and tempo layers, so short-window realised scores cannot add noise to unpriced fixtures.0.20380.2053Lost on test
gpt-6.1-sol40Consolidate round 39 unchanged, so no further validation-tuned parameter is added.0.2038—Better
gpt-6.1-sol41Remove only the fast tempo adjustment and keep everything else.0.2038—Better
gpt-6.1-sol42Restore the market-only fast tempo adjustment on top of round 39.0.2038—Better
gpt-6.1-sol43Remove only the fast strength-form adjustment, keeping tempo.0.2047—Worse
gpt-6.1-sol44Restore market-only fast strength form while keeping tempo.0.2038—Same as round 39
gpt-6.1-sol45Train priced rows on market-implied goals alone instead of the 75/25 mixture.0.2042—Worse
gpt-6.1-sol46Restore the 75% market / 25% compressed-score training mixture.0.2038—Same as round 39
gpt-6.1-sol47Remove blow-out compression and its compensating goal-level correction.0.2039—Tied
gpt-6.1-sol48Restore blow-out compression and its goal-level correction.0.2038—Same as round 39
gpt-6.1-sol49Remove only the manual rho offset.0.2039—Tied
gpt-6.1-sol50Restore the source-aware rho offset.0.2038—Same as round 39
gpt-6.1-sol51Remove only the tier-gap correction.0.2042—Worse
gpt-6.1-sol52Restore the source-scaled tier correction and otherwise ship round 39.0.2038—Same as round 39

Why these rounds could not win

The shipped plugin already uses every structure that paid: a 0.5 prior, source-specific time decay, reliability weighting, a tier-gap shift, blow-out squashing, a source-aware stretch and rho pair, a self-shrunk 14-day form and tempo layer, and a decompressed market residual. Adding more structure moves the score inside the noise. What is left is information — injuries, lineups, fixture congestion, rotation — which is not in historical scores or closing prices and cannot be programmed into a function. That is why it runs as a separate experiment with the pre-registered rule above.