How sample size changes a rate leaderboard

1 Question

A rate leaderboard built straight from raw counts rewards small samples that happened to run hot. The mlb_shrinkage.json artifact (site module mlb_shrinkage, as_of 2026-07-24) takes three such leaderboards from a fixed 2022-2023 Statcast slice and asks what each entity's rate looks like once a Beta-binomial prior, fit from the group's own population, pulls it toward the group mean. The artifact's own headline states the result plainly: 'Small-sample rate leaders regress toward the group mean under empirical-Bayes shrinkage; the raw leaderboard overstates them' (headline). This note works through what that shrinkage does mechanically, which published entities move the most once it is applied, and what the exercise does and does not establish about any of the named players or umpires.

Three groups are covered: catcher out-of-zone strike rate (113 entities), umpire out-of-zone strike rate (102 entities), and on-base rate against left-handed pitching, restricted to hitters with at least 20 plate appearances against lefties (394 entities, groups[].floor = 'pa_vs_l>=20'). All three groups are marked descriptive_only=true in the artifact. The out-of-zone groups carry an added caution in their own labels: 'NOT a framing/called-strike rate; descriptive' (groups[].label, catcher_ooz and umpire_ooz). Nothing here is a skill claim, a park- or umpire-quality ranking, or an input to any forecast.

2 Data and definitions

Both leaderboards and the shrinkage applied to them come from one source: observation_window.corpus_id = 'statcast_fuller_v1', a per-pitch 2022-2023 Statcast pull, with observation_window.as_of = 2026-07-05T02:39:36.769430+00:00 marking when that corpus slice was captured, and the module's own as_of (site_manifest.json, mlb_shrinkage entry) = 2026-07-24T23:02:15.164907+00:00 marking when the shrinkage was fitted from it. This is not the in-game win-probability tick corpus that carries a separate, documented multi-game-label data-integrity notice elsewhere on this site; statcast_fuller_v1 is a pitch-level Statcast pull with its own fixed window (observation_window.seasons = '2022_2023'), and that in-game-corpus notice does not apply to any number in this note.

The three groups differ in size and in what they measure. Catcher out-of-zone strike rate covers 113 catchers (groups[].n_entities); umpire out-of-zone strike rate covers 102 umpires; on-base rate vs left-handed pitching covers 394 hitters who cleared the 20-plate-appearance floor. 'Out-of-zone strike' here means any pitch outside the strike zone coded a strike by any means -- called, swung at and missed, or fouled -- attributed to the catcher receiving it or the umpire calling it; the group's own label repeats that this is not a framing or called-strike-skill measure. On-base rate vs LHP is a hits-plus-walks-plus-hit-by-pitch-over-plate-appearances rate restricted to plate appearances against left-handed pitchers.

3 Method

The artifact's own method field describes a Beta-binomial prior fit per group by method of moments (method). A pooled mean m is the sum of successes over the sum of trials across every entity in the group (m = sum(k)/sum(n)). kappa, defined as alpha+beta, is solved from the Beta-Binomial variance identity: the extra spread in the raw rates beyond what binomial sampling noise alone would produce is attributed to the prior, kappa = m*(1-m) / max(Var(p_i) - mean(m*(1-m)/n_i), tiny) - 1, floored at 1.0 if the raw rates show no more spread than sampling noise alone (method). alpha = m*kappa and beta = (1-m)*kappa follow directly.

shrunk_i = (k_i + alpha) / (n_i + alpha + beta), where k_i = raw_rate_i * n_i and kappa = alpha + beta (posterior mean of a Beta(alpha, beta) prior updated by k_i successes in n_i trials)

That posterior mean has an equivalent, more legible form: shrunk_i = (1 - w_i) * raw_rate_i + w_i * pooled_mean, where w_i = kappa / (n_i + kappa). This is the artifact's own kappa (alpha+beta) divided by an entity's own n, both published fields; it is not a new measurement, only a restatement of the blend as a ratio of two cited numbers. Checking it against a published row confirms the identity: Drew Millas, n=585, raw_rate=0.212, w = 834.93/(585+834.93) = 0.588; (1-0.588)*0.212 + 0.588*0.2813 = 0.2527, matching the published shrunk_rate of 0.2528 to rounding (groups[key=catcher_ooz].biggest_regressors, name=Drew Millas).

The published kappa values are 834.93 for catchers, 2830.5 for umpires, and 226.3 for the platoon group, all well above the 1.0 floor the method applies when a group's raw rates show no more spread than binomial noise alone (method). The artifact does not state whether that floor was reached for any group here; it reports only the fitted values.

4 Results

The three fitted priors differ enough to change how hard shrinkage pulls at a given sample size: a larger kappa relative to a typical n pulls harder toward the pooled mean, a smaller one pulls less.

Umpires carry the largest kappa (2830.5) against out-of-zone strike rates that cluster near the same 0.2813 pooled_mean as catchers (kappa 834.93) -- so at an identical n, an umpire's raw rate is shrunk harder toward 0.2813 than a catcher's is. The platoon group's kappa (226.3) is far smaller than either out-of-zone group's, but its typical n (plate appearances against lefties) is also far smaller than the pitch counts behind the out-of-zone rates, so prior weight in that group still runs high for its smallest-n entities: 68.3 percent for Jesus Sanchez at n=105 (groups[key=platoon_vs_lhp].biggest_regressors, name=Jesus Sanchez).

Table 2 lists the source's 12 published biggest_regressors for catcher out-of-zone strike rate, already ordered by the artifact from largest to smallest absolute regression, where regression = raw_rate - shrunk_rate. One name, Carlos Perez, appears twice in the source's own list at two different sample sizes (n=1277 and n=2234); both rows are reproduced here exactly as published, without further disambiguation, since the source does not distinguish them.

The source's separate top_by_shrunk list -- its own published leaderboard after shrinkage -- shows where this reordering actually lands. Ranked by raw_rate alone, Austin Barnes (raw_rate 0.3049, n=6707) sits ahead of Alejandro Kirk (raw_rate 0.3048, n=10963) for the group's second spot; ranked by shrunk_rate, Kirk (0.3031) passes Barnes (0.3023), because Kirk's much larger n shrinks less. Sandy Leon's raw_rate of 0.3015 (n=2537) is the group's fifth-highest raw rate, but his shrunk_rate of 0.2965 falls to eighth, passed by three catchers with far larger samples: Travis d'Arnaud (n=11281, shrunk_rate 0.2982), Cal Raleigh (n=14504, shrunk_rate 0.2981), and Jacob Stallings (n=12090, shrunk_rate 0.2969) (groups[key=catcher_ooz].top_by_shrunk). A raw-rate leaderboard and a shrunk-rate leaderboard are not the same ordering, and the gap between them is largest exactly where sample size varies most.

Table 3 lists the umpire group's 12 published biggest_regressors. Randy Rosenberg shows the group's largest absolute regression: a raw_rate of 0.3142 at the smallest listed n (506) shrinks to 0.2863, a movement of 0.0279, larger than any other umpire's listed movement, consistent with having both the smallest n and a raw rate well above the 0.2813 pooled_mean.

Table 4 lists the platoon group's 12 published biggest_regressors. Mitch Garver's raw on-base rate against lefties is 0.4527 at n=148, the highest raw_rate in the group's own top_by_shrunk list, but his shrunk_rate of 0.3748 ranks fifth once every entity is shrunk, behind Paul Goldschmidt (n=308, shrunk_rate 0.3898), William Contreras (n=293, shrunk_rate 0.3876), Robbie Grossman (n=286, shrunk_rate 0.3812), and Yandy Diaz (n=284, shrunk_rate 0.3749) -- four hitters whose raw rates were all lower than Garver's but whose sample sizes were roughly double his, so shrinkage cost them less (groups[key=platoon_vs_lhp].top_by_shrunk). Garver's own regression of 0.0779 is the largest movement, in either direction, across all three groups' published biggest_regressors lists.

5 Robustness and what would falsify this

The direction of every listed regression is consistent with the mechanism: whenever raw_rate sits above its group's pooled_mean, regression is positive (raw pulled down); whenever it sits below, regression is negative (raw pulled up). All 12 catcher rows, all 12 umpire rows, and all 12 platoon rows in the published biggest_regressors lists satisfy this without exception. A single row breaking that pattern -- a raw rate above the pooled mean shrinking further upward, or the reverse -- would indicate either a fitting error or a group whose pooled_mean or kappa was misspecified; none does.

A second check: the published biggest_regressors are already ordered by the size of |regression|, and that ordering tracks n inversely once distance from the pooled mean is similar -- Drew Millas (n=585, the smallest n in the catcher list) has the largest catcher regression, and Chris Conroy (n=4251, near the largest n in the umpire list) has one of the smaller umpire regressions. This is what the prior-weight formula predicts, not an independent confirmation beyond the arithmetic.

• Shrinkage assumes the entities are exchangeable draws from one prior, a modeling choice, not a fact (confounds).
• The out-of-zone rate is descriptive, not a called-strike or framing skill measure, and not predictive (confounds).
• The raw-vs-shrunk gap is the point of the exercise: it shows how much a small-sample rate should be discounted, not a new 'true skill' claim (confounds).
• The window is fixed to 2022-2023; nothing here extends to any other season (confounds).

6 Limitations

This is a within-window description of a published prior and its effect on 36 listed entities, not a validated finding across seasons and not a model of any individual player's or umpire's true rate.

• This is a fixed 2022-2023 window; it is not extended to any other season and does not forecast any entity's future rate.
• The prior (pooled_mean, alpha, beta, kappa) is estimated by method of moments from the same population it shrinks, not from an externally validated or held-out prior.
• Out-of-zone strike rate is explicitly descriptive in the source's own group labels, not a framing or called-strike-skill measure.
• Only the source's published biggest_regressors and top_by_shrunk subsets (12 rows each) are shown per group; the full 113, 102, and 394-entity populations are not reproduced here.
• The reordering examples compare two published columns for named entities; they are not a significance test and do not establish that any single difference is distinguishable from sampling noise.

7 How to read this on the site

On the site, the mlb_shrinkage module page carries the full published tables this note draws from, including every biggest_regressors and top_by_shrunk row for all three groups, not only the entities singled out above. Read a shrunk_rate beside its own n and the group's kappa: the same shrunk_rate can sit behind a raw_rate that barely moved (high n) or one that moved by eight or nine percentage points (low n), and the prior-weight column above is the number that tells you which.

The count-and-sequencing companion paper on this site (count-state-and-next-pitch-mlb) works a different 2025 Statcast corpus from a different angle -- conditional pitch mix and sequencing by ball-strike count -- and shares this note's discipline of keeping a mix question and a conditional-probability question apart rather than collapsing them into one number. The related research-lab pages 'Published Shrinkage Displacement' (mlb-shrinkage-displacement) and 'MLB catcher out-of-zone strike rate selections' (mlb-catcher-out-of-zone-strike-rate) present these same source rows in the site's sortable table format for readers who want to re-derive the movement themselves.