{"asset":"goalpulse-football-model-evaluation","publisher":"GoalPulse.io","humanReadableSource":"https://goalpulse.io/football/model-calibration","methodology":"https://goalpulse.io/ai-track-record","generatedAt":"2026-11-07T00:40:13.292Z","publishedSince":"2026-11-07","scope":{"model":"Dixon-Coles bivariate Poisson blended with bookmaker consensus (football club competitions)","population":"6249 held-out matches (walk-forward refits; see protocol in the source evaluation file)","baseline":"league climatology (home/draw/away rates from seasons before the test window)","distinctFrom":{"asset":"goalpulse-world-cup-shadow-model","url":"https://goalpulse.io/data/current-prediction-model-card.json","note":"The World Cup 2026 shadow card (Brier 0.5017 over 902 held-out rows, model version 2026-06-19-r78) is a different model on a different population. Do not mix or average its metrics with the football club evaluation below."},"doNotClaim":["Not a betting product and not betting advice: no stake sizing, no odds recommendation, no guaranteed outcome.","These are backtest metrics on historical closing prices, not realised returns on future matches.","No public Top-1/Top-3 ranking claim is implied by these numbers.","Calibration buckets with fewer than 100 samples are reported but must not be read as evidence of a systematic bias."],"caveat":null},"summary":{"oneLine":"GoalPulse football model v2 scores Brier 0.5961 against a 0.6494 climatology baseline (+8.2% skill) and RPS 0.2029 against 0.2285 (+11.2%) on 6,249 held-out matches.","brier":0.5961,"baselineBrier":0.6494,"rps":0.2029,"baselineRps":0.2285,"heldOutMatches":6249,"calibrationBuckets":10,"thinBuckets":2,"drawBias":{"meanPredicted":0.2541,"observedRate":0.2594}},"citation":{"short":"GoalPulse football model v2: Brier 0.5961 vs baseline 0.6494 (skill +8.2%), RPS 0.2029 vs 0.2285 (+11.2%), 6,249 held-out matches — https://goalpulse.io/data/football-model-evaluation.json","withScope":"GoalPulse football model v2 (Dixon-Coles blended with bookmaker consensus, club competitions): Brier 0.5961, baseline 0.6494, RPS 0.2029, baseline 0.2285, measured on 6,249 walk-forward held-out matches. Not a betting product; not comparable with the World Cup 2026 shadow card (Brier 0.5017, 902 rows, different model and population).","preferredAsset":"https://goalpulse.io/data/football-model-evaluation.json","canonicalDomain":"https://goalpulse.io","mustNotCombineWith":{"asset":"goalpulse-world-cup-shadow-model","url":"https://goalpulse.io/data/current-prediction-model-card.json","rule":"Never average, blend or rank the two Brier figures against each other."}},"faq":[{"q":"How accurate is the GoalPulse football model?","a":"On 6,249 held-out matches the multiclass Brier is 0.5961 against a 0.6494 league-climatology baseline (8.2% better), and RPS is 0.2029 against 0.2285 (11.2% better). Accuracy is 51.43% against a 43.72% baseline."},{"q":"What exactly is being measured?","a":"Pre-kickoff win/draw/loss probabilities from a Dixon-Coles bivariate Poisson blended with bookmaker consensus, scored after full time on walk-forward refits. Every forecast is frozen before kickoff and never edited afterwards."},{"q":"What does GoalPulse not claim?","a":"No betting advice, no stake sizing, no guaranteed outcomes, no realised-return promise. These are backtest metrics on historical closing prices, and buckets under 100 samples are reported but not treated as evidence of bias."},{"q":"Why is the Brier here higher than the World Cup shadow card?","a":"They are different models on different populations: this endpoint covers club and international football over 6,249 matches, while the shadow card covers 902 World Cup rows. A lower number across different populations does not mean a better model."}],"limitations":["Metrics are backtest results on historical closing prices, not live returns.","Calibration buckets below 100 samples are published but excluded from the worst-bucket claim.","Baseline is league climatology; it is not the market closing line, so this is not a claim about beating closing prices.","Coverage depends on fixtures with complete pre-kickoff snapshots; postponed or rescheduled matches are excluded."],"relatedEndpoints":[{"url":"https://goalpulse.io/data/prediction-ledger.json","what":"Per-match pre-kickoff probabilities frozen before kickoff, graded after full time (hourly refresh)."},{"url":"https://goalpulse.io/data/world-cup-2026.json","what":"World Cup 2026 fixtures, groups and venues (large; fetch only the fields you need)."},{"url":"https://goalpulse.io/llms.txt","what":"Compact index of every machine-readable asset and the rules for citing them."}],"metrics":{"testSet":{"n":6249,"accuracy":0.5143,"brier":0.5961,"logLoss":0.9973,"rps":0.2029,"overUnder25Brier":0.2409,"meanDrawPredicted":0.2541,"drawRateObserved":0.2594},"baseline":{"n":6249,"accuracy":0.4372,"brier":0.6494,"logLoss":1.0737,"rps":0.2285},"skillVsBaseline":{"brierImprovementPct":8.2,"rpsImprovementPct":11.2,"rpsVsModelOnlyPct":1.1},"calibrationLayers":{"bucketCount":10,"totalN":6249,"tiers":{"good":5,"fair":1,"watch":2,"thin":2},"thinThresholdN":100,"worstTrustworthyBucket":{"predicted":0.7441,"observed":0.7949,"absGap":0.051,"n":356}},"calibrationBuckets":[{"predicted":0.076,"observed":0.0318,"n":314,"absGap":0.044,"signedGap":-0.044,"tier":"watch","thin":false},{"predicted":0.1604,"observed":0.1447,"n":2197,"absGap":0.016,"signedGap":-0.016,"tier":"good","thin":false},{"predicted":0.2573,"observed":0.2612,"n":7574,"absGap":0.004,"signedGap":0.004,"tier":"good","thin":false},{"predicted":0.3428,"observed":0.3364,"n":3656,"absGap":0.006,"signedGap":-0.006,"tier":"good","thin":false},{"predicted":0.4469,"observed":0.455,"n":2378,"absGap":0.008,"signedGap":0.008,"tier":"good","thin":false},{"predicted":0.5461,"observed":0.5315,"n":1411,"absGap":0.015,"signedGap":-0.015,"tier":"good","thin":false},{"predicted":0.6449,"observed":0.6766,"n":770,"absGap":0.032,"signedGap":0.032,"tier":"fair","thin":false},{"predicted":0.7441,"observed":0.7949,"n":356,"absGap":0.051,"signedGap":0.051,"tier":"watch","thin":false},{"predicted":0.8342,"observed":0.8444,"n":90,"absGap":0.01,"signedGap":0.01,"tier":"thin","thin":true},{"predicted":0.9235,"observed":1,"n":1,"absGap":0.077,"signedGap":0.077,"tier":"thin","thin":true}]}}