a bayesian hierarchical model of pitch framing in major...

A Bayesian Hierarchical Model of Pitch Framing inMajor League Baseball

Sameer K. Deshpande and Abraham J. WynerStatistics Department, The Wharton School

University of Pennsylvania

1 August 2016

Introduction

• Framing: ability of a catcher to affect likelihood a taken pitch iscalled a strike

• Most estimates: top framers can save ∼ 20 – 25 runs per season ... 2- 2.5 wins above replacement!

• ESPN The Magazine: “If you have confidence in the additional 2WAR that framing would have given Lucroy, his 2014 season wouldhave been worth about $56M dollars on free agent market thisoffseason”

Deshpande & Wyner Catcher Framing JSM 2016 2 / 16

Overview

• Hierarchical Bayesian logistic regression model of called strikeprobability

• Estimate of each catcher’s effect on each umpire over and abovefactors like pitch location, count, and other pitch participants

• Translate these effects to estimates of the impact framing (i.e. runssaved) has on the game, along with natural uncertainty quantification

We use PITCHf/x data from 2011 – 2015

• Horizontal and vertical coordinates of pitch as it crosses home plate

• Main focus is on 2014 season: 300k called pitches


Model

Let y = 1 for called strike, y = 0 for ball. For umpire u:

log

(P(y = 1)

P(y = 0)

)= θu0 + θub + θuc + θup + θucount + f u(x , z)

where

• θub , θuc , θ

up : partial effect of batter b, catcher c and pitcher p on

umpire u’s log-odds of calling strike

• θucount : partial effect of count on umpire u

• f u(x , z): function of pitch location

• θu0 : intercept


Incorporating Pitch Location

Direct Parametrization:

• Each coordinate as linear predictor: f u(x , z) = θux x + θuz z

• Polar coordinates: f u(x , z) = θur r(x , z) + θuφφ

Indirect Parametrization:

1. Fit a Generalized Additive Model of called strike probability assmooth function of location:

I Uses data from 2011 – 2013I Separate GAM for each combination of batter and pitcher handedness

2. Use forecasted log-odds of called strike a linear predictor in model:

f u(x , z) = θulpˆlog-odds


Hierarchical Model

For each of the 93 umpires u1, . . . , u93

log

(P(yui = 1)

P(yui = 0)

)= xu>i Θu

Θu1 , . . . ,Θu93 |µ ∼ N(µ, σ2I

)µ ∼ N

(0, τ2I

)• σ = 1: less than 0.3% chance that one umpire would call the same

pitch strike 99% of time and other umpire calls strike 1% of time

• τ = 0.5: replacing player by baseline player unlikely to change calledstrike probability from 75% to 25%

• Model fit using Stan


Posterior Densities of Player Effects

(a) Hank Conger (b) Jonathan Lucroy


Impact of Framing

For each catcher, look at all of the called pitches he received:

• p̂: fitted probability of strike

• p̂0: fitted probability of strike with catcher replaced by baselinecatcher

• p̂ − p̂0: catcher’s “framing effect”

• Sum ρ× (p̂ − p̂0) over all called pitches received

Value of a called strike, ρ, depends on the count:


Value of a called strike on an 0 – 1 pitch

Between 2011 and 2014:

• 182,405 0 – 1 pitches taken: 140,667 balls, 41,738 called strikes

• Avg. # runs allowed in rest of inning after called ball: 0.322

• Avg. # runs allowed in rest of inning after called strike: 0.265

Conditional on an 0 – 1 pitch being taken:called strike saves ρ = 0.057 runs, on average


Estimated Runs Saved, On Average

Catcher Runs Saved (SD) 95% Interval P(> 0) N BP

Miguel Montero 25.1 (7.1) [11.3, 38.8] 0.999 8086 11.2 (8172)

Mike Zunino 19.9 (7.3) [5.4, 34.1] 0.997 7615 20.4 (7457)

Jonathan Lucroy 19.5 (8.1) [3.8, 35.3] 0.991 8398 16.4 (8241)

Rene Rivera 18.9 (5.3) [8.6, 29.2] 1.000 5091 22.5 (5182)

Hank Conger 17.6 (4.5) [8.8, 26.4] 1.000 4743 23.8 (4768)

Russell Martin 15.4 (5.9) [3.6, 27.2] 0.994 6388 14.9 (6502)

Buster Posey 15.0 (6.1) [3.1, 26.9] 0.992 6385 23.6 (6190)

Travis d’Arnaud 13.5 (6.1) [1.8, 25.7] 0.986 6573 8.8 (6276)

Brian McCann 12.9 (5.4) [2.2, 23.2] 0.992 6335 9.7 (6471)

Christian Vazquez 12.4 (3.4) [5.9, 18.9] 1.000 3198 13.7 (3370)


Spatially Aggregate Framing Effect

Is Montero really a better pitch framer than Vazquez?

• 8086 pitches vs 3198

• Results further confounded by other pitch participants, location,counts

We can integrate ρ× (p̂ − p̂0) over all batter, pitcher, umpire, count,location combinations.

• Framing analog of Spatial Aggregate Fielding Evaluation of Jensen,Shirley, and Wyner (2008)

• SAFE2: Estimate how many runs catcher saves through framing on4000 “average” pitches


SAFE2

Rank Player Mean (SD) 95% Interval P(> 0)

1. Rene Rivera 15.1 (4.4) [6.5, 23.6] 1.000

2. Hank Conger 14.7 (4.4) [6.1, 23.3] 1.000

3. Christian Vazquez 14.6 (4.9) [5.0, 24.3] 0.999

4. Miguel Montero 12.8 (3.7) [5.5, 19.9] 0.999

5. Yasmani Grandal 12.5 (4.5) [3.8, 21.4] 0.998

6. Mike Zunino 11.5 (4.1) [3.6, 19.5] 0.997

7. Martin Maldonado 11.4 (5.9) [0.1, 23.3] 0.975

8. Chris Stewart 11.1 (5.6) [0.2, 22.2] 0.977

9. Russell Martin 10.3 (4.0) [2.4, 18.0] 0.995

10. Drew Butera 10.1 (5.2) [0.1, 20.3] 0.976


Conclusions

• SAFE2 year-to-year correlation encouraging: 0.5 – 0.75

• There are some players with statistically distinguishable effects onsome umpires

• Even with these effects, out-of-sample performance similar to that ofunderlying GAM’s: non-stationarity between seasons

• Our estimate of framing’s impact similar to others, but considerableuncertainty in our estimates!


Thanks!

Fitted Called Strike Probabilities

0 – 1 pitch, Yasiel Puig, Madison Bumgarner, Buster Posey

(a) Angel Hernandez (b) Average Umpire (c) Scott Barry


Average # Runs Given Up and Value of Strike

Count Ball Strike Value of strike, ρ

0-0 0.367 (0.002) 0.305 (0.002) 0.062 (0.002)

0-1 0.322 (0.002) 0.265 (0.004) 0.057 (0.004)

0-2 0.276 (0.003) 0.178 (0.007) 0.098 (0.008)

1-0 0.427 (0.003) 0.324 (0.003) 0.103 (0.005)

1-1 0.364 (0.003) 0.280 (0.004) 0.084 (0.005)

1-2 0.302 (0.003) 0.162 (0.006) 0.140 (0.006)

2-0 0.571 (0.007) 0.370 (0.006) 0.201 (0.009)

2-1 0.468 (0.005) 0.309 (0.006) 0.159 (0.008)

2-2 0.383 (0.004) 0.165 (0.006) 0.218 (0.007)

3-0 0.786 (0.013) 0.481 (0.008) 0.305 (0.015)

3-1 0.730 (0.010) 0.403 (0.009) 0.327 (0.014)

3-2 0.706 (0.008) 0.166 (0.008) 0.540 (0.011)

Table : Standard errors in parentheses


a bayesian hierarchical model of pitch framing in major...

Documents