Frozen before first pitch · graded after the last out
Every game prediction is locked before the first pitch and graded after the last out, so the record could not have seen its own answer.
Cumulative accuracy of predictions frozen before first pitch. The shaded band is the 95% interval, wide early because a dozen games prove nothing, narrowing as the evidence builds. The three dashed lines are what give the number meaning.
Static model: projections built from prior seasons only, never updated in-season. The vertical axis is fixed from 40% to 75%, so the first few days, when the interval is wider than that, are clipped at the edges.
A single accuracy over every game is the wrong summary: about 40% of the schedule is a genuine coin flip that no information resolves. The useful questions are how often the model is right when it is confident, and whether its confidence is honest.
Keep only the games the model is surest about.
What the model claimed against what happened. On the dashed diagonal is perfectly calibrated; the whiskers are 95% intervals.
Each step below is a more sophisticated way to estimate how good a team is right now. The gains are real and they flatten: a 50-game window gets most of the way, a continuously updated Elo rating gets a little further, and stacking everything adds nothing. A separate test found a full run-distribution model ties the simplest log5 model almost exactly. The ceiling is the variance of a nine-inning game, not the algebra.
One number beats two. Elo on its own edges out prior-season strength plus the 50-game window, and stacking all three adds nothing. The gains are real and they are flattening.
Tested, measured, and rejected, each with the evidence. This list is the point of the project: a claim that collapses under one follow-up question is worse than a smaller claim that holds.
Player-level projections rolled into team win distributions. Over seven seasons and 210 team-seasons it does not beat simply regressing last year's run differential: it trails by 1.3% MAE, and in 2026 it lost by 21% and its 90% intervals covered only 63% of clubs. It had looked like a small win through 2025. The game-level record above is the claim this project stands behind; this one is shown because hiding it would be the opposite of the point.
| Team | Div | Wins | 90% interval | Range | Playoff | Division |
|---|
The honest baseline is not "league average." It is last season's run differential, regressed — a measure that already contains defense, baserunning and park by construction, which is exactly why it took so long to beat.
The edge is real but modest and uneven: across five seasons the model wins two, loses two and ties one. It was reported as +4.2% when the backtest covered only 2023–25, because 2023 was by far its best year. Extending to five seasons cut that to +1.8%. The RMSE gap holds up better at +5.3%, meaning large misses are avoided even where average error is close.
| Model | MAE | RMSE | vs baseline |
|---|
Every one of these looked like a finding. Each survived until it was checked for whether the model could see its own answer. The struck-through number is what it claimed; the number beside it is what was actually there.
A null is only useful if the test was fair. Each of these was designed to give the hypothesis its best chance — player-centered controls, held-out samples, the right baseline — and each still failed.
This one number drives the whole modelling stance. Plate discipline firms up roughly 4.7× faster than batting average, so it deserves far more weight early in a season — and results-based statistics deserve far less than they usually get.
Lower κ = stabilises sooner = trust the observed number earlier.