Model & methods
Expected goals (xG) is the probability that a given shot attempt becomes a goal, estimated from where and how it was taken — distance, angle, shot type, whether it followed a rebound or a rush, the strength state, and more. The baseline model is deliberately talent-blind: it scores the chance for a league-average shooter against a league-average goalie, so that a player's finishing (goals minus xG) measures their shooting talent against that neutral bar.
How it is trained
- Same-era, grouped by game. Evaluation uses K-fold cross-validation split by whole game (a shot and its rebound never straddle the split), with train and test drawn from the same era — the game changes over time, so we never train on old seasons to score new ones.
- Two prediction columns, never conflated. Out-of-fold predictions are the honest source for performance; the in-sample final model produces the canonical xg used for aggregation (uniform scoring across every shot). The tables on this site are the in-sample canonical column — labeled as such, not a held-out score.
- Segregated regimes. Empty-net and penalty shots are scored by dedicated models, so their potent goal environments never contaminate defended-net predictions.
Honest limitations
- xG is a model of chance quality, not a verdict — it is calibrated in aggregate, but any single shot's number carries uncertainty.
- Shooter/goalie identity is not in the baseline (by design — that is what makes finishing measurable). A talent-aware variant is planned separately.
- Time-on-ice is not yet aggregated, so player rates are per-game and per-shot, not per-60.
- Coordinates are arena-adjusted but imperfect; blocked-shot origins are imputed.
This build
- Model id:
nested__baseline_xg__b7bbf5c9(nested) - Season shown: 2025-26
- Displayed xg: in-sample canonical column (aggregation)
How well does the xG model score shots?
Every number in this section is out-of-fold — each shot scored by a model that never saw its game in training — over 2,344,674 shot attempts (112,235 goals) across the full corpus. This is the honest held-out report, not the in-sample column the tables use.
| Metric | Value | Reading |
|---|---|---|
| Danger AUC (goal vs save, on-net) | 0.772 | ranks real scoring chances; immune to easy blocked-shot separability |
| All-shots AUC | 0.824 | includes blocks/misses — flattering, shown for context |
| Brier score | 0.04123 | mean squared error of the probabilities (goal rate 4.8%) |
| Log-loss | 0.15743 | the training objective, held-out |
Calibration — when we say 10%, does it happen 10% of the time?
Shots grouped into ten equal-count bins by predicted xG; each point compares the bin's mean prediction to its observed goal rate. On the dotted line = perfectly calibrated. Log scales, so every decade from long-range point shots to point-blank chances is visible.
Stability by season
Discrimination (AUC) and calibration error (Brier) per season — one model, sixteen seasons, no season-specific tuning.
What the model looks at
Gain importance per cascade stage — how much each feature contributes to the splits that decide blocked? → on net? → goal? Location dominates (distance, angle, x/y); sequence context (rebounds, rush) earns its keep in the goal stage.
The game engine — how well does it call games?
The matchup engine (matchup__6a856b2b) predicts game
winners from the xG pipeline. How it works, in one breath: every team carries
ratings — opponent-adjusted, league-centered expected-goals rates per 60
(5v5 attack & defense, special teams, discipline, finishing & saving residuals),
updated game by game with a 40-game half-life and shrunk ⅓ toward average each summer. A
Poisson regression turns tonight's two rating vectors + rest/schedule
features into expected goals for each side (λ); a score grid over all
possible finals converts the λs into win, tie, and total-goals probabilities; an
OT/shootout layer settles the tie mass.
Every number below is from the walk-forward backtest: to score season N, the engine is fit only on seasons before N — it never peeks. Seasons through 2018-19 were the development window (used for tuning); the shaded holdout was never used to make a modeling choice. The Games page tracker is the third, strictest tier: predictions published before puck drop this season, scored as issued.
Season by season vs the challengers
Log-loss per season (lower is better) for the shipped Poisson engine and its three comparators: a logistic classifier on the same ratings, goals-only Elo (no xG — "did the pipeline add anything?"), and the home-rate constant (always predict the league home-win rate). The first seasons train on very little history; the engine finds its feet as the ledger grows.
| Engine (holdout, 8,510 games) | log-loss | Brier | accuracy |
|---|---|---|---|
| poisson | 0.665 | 0.236 | 58.6% |
| logistic | 0.663 | 0.236 | 59.3% |
| elo | 0.670 | 0.239 | 58.6% |
| constant | 0.691 | 0.249 | 53.6% |
Yes — the plain logistic edges the Poisson on winner log-loss (~0.002). It optimizes exactly this one number; the generative Poisson is kept as the headline because only it produces the score grid behind expected goals, totals, and the matchup maps. Both clearly beat Elo: the xG pipeline earns its keep. Interestingly, when Poisson and Elo pick different winners they split those games about 50/50 — the engine's edge is probability accuracy, not contrarian picks.
Trust the number — the confidence record
Holdout games grouped by how confident the engine said it was about the favorite. A calibrated forecaster's 60–65% games should land ~62% — and they do.
| Stated confidence | record | expected | actual |
|---|---|---|---|
| 50–55% | 1644–1600 | 52.5% | 50.7% |
| 55–60% | 1473–1034 | 57.4% | 58.8% |
| 60–65% | 1008–581 | 62.2% | 63.4% |
| 65–70% | 585–234 | 67.2% | 71.4% |
| 70%+ | 280–71 | 73.4% | 79.8% |
Goals, not just winners — the totals check
The engine's λs claim to be expected goals, so hold them to it. The curve rises decile by decile: when the engine predicts more goals, teams score more goals — the ranking signal is clean. But the whole curve sits above the dotted perfect-calibration line: predictions run about 4.4 tenths of a goal per game cold, because a model trained only on past seasons lags the NHL's rising scoring environment. That bias cancels in win probabilities (both teams shift together) — it shows up here, so we show it here.
The same story at the game level: predicted vs actual distribution of total goals (5.74 predicted vs 6.18 actual per game). Even biased cold, the matchup-specific totals still beat a climatology baseline on over/under 5.5 (log-loss 0.696 vs 0.698) — there is real signal to unlock once the scoring-environment lag and the independent-Poisson assumption (it under-counts ties) are addressed. Both fixes are queued ahead of any totals-facing feature.
Deeper reading
Living analysis reports (calibration, agreement, arena adjustment, blocked-shot imputation) are in the project repository:
- OOF-vs-final agreement — why the in-sample column is safe for aggregation
- Empirical vs model — calibration in feature space
- Model baseline — flat vs nested, danger AUC, the blocked-shot fingerprint
- Full model contract
- Game-engine contract — ratings, score model, walk-forward methodology
- Engine improvement sweep — what was tried, what won, what honestly failed