<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Tim Sheehan on Medium]]></title>
        <description><![CDATA[Stories by Tim Sheehan on Medium]]></description>
        <link>https://medium.com/@timsheehan_49571?source=rss-f882f0531af9------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*65D4DCr6ALJ9GgFhkupZ_w.jpeg</url>
            <title>Stories by Tim Sheehan on Medium</title>
            <link>https://medium.com/@timsheehan_49571?source=rss-f882f0531af9------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 07:06:35 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@timsheehan_49571/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Parameterizing umpire biases using MLB Statcast data. Part 2.]]></title>
            <link>https://medium.com/@timsheehan_49571/parameterizing-umpire-biases-using-mlb-statcast-data-part-2-a27a227e2981?source=rss-f882f0531af9------2</link>
            <guid isPermaLink="false">https://medium.com/p/a27a227e2981</guid>
            <category><![CDATA[mixed-effects]]></category>
            <category><![CDATA[logistic-regression]]></category>
            <category><![CDATA[umpire]]></category>
            <category><![CDATA[baseball]]></category>
            <category><![CDATA[cognitive-bias]]></category>
            <dc:creator><![CDATA[Tim Sheehan]]></dc:creator>
            <pubDate>Wed, 08 May 2024 12:53:37 GMT</pubDate>
            <atom:updated>2024-05-08T12:53:37.461Z</atom:updated>
            <content:encoded><![CDATA[<p>In a previous <a href="https://medium.com/@timsheehan_49571/parameterizing-umpire-biases-using-mlb-statcast-data-part-1-f3ce6616b8c1">post</a>, I described how the size of the strike zone changes as a function of the count in the current at-bat. This was achieved by fitting a separate strike-zone model to each subset of pitches. In this next post I want to additionally examine the influence of previous calls more explicitly and see how these biases vary across umpires.</p><p>To start I designed a logistic-regression model encoding both the count before the current pitch and the call made on the previous n=6 pitches using one-hot vectors.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hKKBn64vvMaKbzV6XDZSCw@2x.png" /><figcaption>Logistic model.<strong> X_strike</strong> corresponds to whether the nth previous pitch was called a strike, and vice-versa for <strong>X_ball</strong>. <strong>X_count</strong> corresponds to each of the 12 possible counts in a given at-bat.</figcaption></figure><p>The coefficients β are estimated by minimizing <em>cross-entropy loss </em>on all called pitches. These can be converted into probabilities of a given pitch being called a strike. Similar to my earlier analysis of <a href="https://medium.com/@timsheehan_49571/parameterizing-umpire-biases-using-mlb-statcast-data-part-1-f3ce6616b8c1">strike-zone areas</a> I find that umpires are much more likely to call a strike on an <em>0–3</em> count relative to a <em>2–0</em> count (<strong>below right</strong>). I additionally observe that umpires have a tendency to not repeat calls, calling a strike less often when they called a proceeding pitch a strike (<strong>below left</strong>). These probabilities could be driven in large part driven by pitch selection (eg. by the pitcher). I will need to take into account pitch location to determine to what extent this is due to a bias in umpire judgement.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*kNl3DQPVse12Wmm_cAxBUA@2x.png" /><figcaption>Fit parameters from logistic regression model converted into p(strike) on the current pitch. <strong>Left:</strong> p(strike) given the call on any of the previous 6 pitches; <strong>Right</strong>: p(strike) given the count before the current pitch.</figcaption></figure><p>A straightforward way to account for pitch location in our model is to utilize the strike-zone I described in a previous post. For the resulting model, I <em>simultaneously</em> fit a strike-zone <strong>and</strong> history coefficients. The output of the strike-zone model, <strong>p(strike|x.z</strong>), is an input to our logistic regression model (put through a logit transform to maintain linearity).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/306/1*xO5BaE7Ppmg-_01XxkCifA@2x.png" /><figcaption>Note that β<strong>_strikeZone</strong> is just an extra parameter in our logistic model added to <strong>z_0</strong> and additionally that the strike zone model provides <strong>p(strike|x,z)</strong> which acts as an input column of data.</figcaption></figure><p>This allows our history model to attribute changes in outcome due to both the pitcher (pitch selection) and umpire judgement. As our logistic model now includes pitch location, I can no longer directly visualize <strong>p(strike)</strong> as it is dependent on pitch location. Instead I will visualize the fit coefficients, <strong>β</strong>, along with the resulting strike zone. For this visualization, I fit a separate model to each umpire who called at least 150 games in the 2018–2022 seasons (excluding 2020, n=71).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lrM7sIa-KpFwOfZ9s3VllQ.png" /><figcaption>Model fit to individual umpires. Positive parameters correspond to a strike call being morelikely when a given condition applies. Mean and SEM shown for history parameters (error bars typically smaller than markers). <strong>Left</strong>: influence of past calls decays suggest a propensity to alternate calls. This influence decays exponentially; <strong>Center</strong>: influence of count similar to previous model not-accounting for pitch location; <strong>Right</strong>: strike-zone shows familiar shift for left/right handed batters. SD is larger due to models attempt to integrate both location and history information.</figcaption></figure><p>Here, I continue to observe a similar pattern where umpires tend to avoid making the same call repeatedly, even after accounting for the location of the current pitch. This suggests a common bias originating with the umpire’s judgement. Are more umpires susceptible to this bias then others?Condensing our history coefficients subtracting “<em>called_strike</em>” from “<em>ball</em><strong>”</strong> we can get a scalar estimate of the bias strength.<strong> Across all (n=71) umpires, every single one shows a bias to alternate calls</strong> but with varying strength. The strength of this bias correlates significantly with overall strike zone variance. This history bias thus resembles a strategy to reduce call variance by less precise umpires.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*j8m7EkXvERvGZCjqyTWFww@2x.png" /><figcaption>Correlation between umpire uncertainty (σ) and history bias terms (a propensity to show a ‘switch’ strategy). Thus less precise umpires are more likely to rely on heuristsics to make decisions.</figcaption></figure><p>Last, I looked to see if any of these terms have changed across time. I fit a separate model each year for all umpires with at least 20 games/season. Using a mixed effect model, I examined the effect of year after controlling for umpire identity. The main effects I observed:</p><ol><li>A significant reduction in <em>σ_x</em> across years (β_year=-0.002, 95% confidence interval: [-.004,.000], p=0.039) but no change in <em>σ_y</em>. This suggests umpires may be getting more precise across time.</li><li>A significant increase in <em>count-bias </em>(β_year=0.01 [0.002, 0.019], p=0.018) but no change in history-bias due to past calls.</li><li>A large increase in the average <em>strike zone area</em> for both left and right handed hitters (Left: β_year=0.036 [0.021, 0.050], p&lt;1E-6; Right: β_year=0.042 [0.027, 0.056], p&lt;1E-7). This shear strength of this effect could be the most interesting to investigate further. Note that a similar trend was suggested from a <a href="https://fivethirtyeight.com/features/umpires-are-less-blind-than-they-used-to-be/">study</a> performed on much older data.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jYpkv918gNeIpdswxIHfAg@2x.png" /><figcaption><strong>Left</strong>: σ in the x and z dimensions across year; <strong>Middle</strong>: Avg. β coefficients for both count and history across year; <strong>Right</strong>: Avg. strike zone area for left and right handed batters across year.</figcaption></figure><p>In this post, I demonstrated an exponentially decaying <em>history-bias</em> and a <em>count-bias</em> observed in umpire behavior that is independent of pitch selection. This further demonstrates that umpires use a lot more than pitch location when calling pitches and that these biases are universal across the league. The strength of these biases seem to be stronger in less precise umpires, suggesting they may emerge from a compensatory mechanism to reduce uncertainty. Lastly, we observe that several properties of umpire behavior are changing across time. This could be due to a variety of factors including the fact that umpires now get <a href="https://fivethirtyeight.com/features/umpires-are-less-blind-than-they-used-to-be/">periodic feedback</a> from StatCast data.</p><p>In a future post I plan to unpack whether these biases are perceptual (as in due to changes in how umpires physically see the pitches) or decisional/cognitive. I also plan to quantify to what extent these biases actually impact the outcome of games.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a27a227e2981" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Parameterizing umpire biases using MLB Statcast data. Part 1.]]></title>
            <link>https://medium.com/@timsheehan_49571/parameterizing-umpire-biases-using-mlb-statcast-data-part-1-f3ce6616b8c1?source=rss-f882f0531af9------2</link>
            <guid isPermaLink="false">https://medium.com/p/f3ce6616b8c1</guid>
            <category><![CDATA[baseball]]></category>
            <category><![CDATA[behavioural-biases]]></category>
            <category><![CDATA[umpire]]></category>
            <category><![CDATA[gamblers-fallacy]]></category>
            <category><![CDATA[maximum-likelihood]]></category>
            <dc:creator><![CDATA[Tim Sheehan]]></dc:creator>
            <pubDate>Thu, 02 May 2024 20:29:08 GMT</pubDate>
            <atom:updated>2024-05-02T20:29:08.681Z</atom:updated>
            <content:encoded><![CDATA[<p>Cognitive biases are an active area of research in psychology and economics. These biases can take many forms including well known examples such as <a href="http://www.stat.columbia.edu/~gelman/surveys.course/TverskyKahneman1981.pdf">framing effect</a> where equivalent data presented in positive or negative language can dramatically alter how that information is interpreted. Typically, those phenomena are studied in highly artificial lab experiments from demographically homogeneous populations (undergrad students between 18–22 years old). This shortcoming of typical psychology experiments drove my interest in studying these biases in a much more ‘real world’ environment with loads of data. Previous <a href="https://academic.oup.com/qje/article/131/3/1181/2590011">work</a> has shown umpires exhibit the “Gambler’s Fallacy” whereby they are more likely to call a ball after calling a strike (and vice-versa). In this post, I will examine a similar phenomenon in the behavior of MLB umpires and how the current count in the at-bat impacts the shape and size of the strike zone.</p><p>Dataset: Since the 2007 season, MLB has recorded and published the velocity, location, spin, and outcome of every pitch thrown. Since the exact systems used to record this information have evolved, I will focus on data since the 2016 season. I utilized the python <a href="https://github.com/jldbc/pybaseball">pybaseball</a> package to pull pitch data from the 2016–2022 seasons. The dataset was natively over three GB in memory, but I was able to reduce this down to ~100 MB by shrinking float precision and converting many variables that occupy only a few set values {team, inning, # of balls, ...} to a <em>categorical</em> variable type. For the purposes of this analysis, I will only be utilizing the x and z (height) positions of the pitch as it crosses the plate, the result of the pitch (ball, strike, hit, etc.), and the count of the current at bat.</p><p>Model: The <strong>rulebook</strong> strike zone is defined by the width of the plate and height and stance of the batter. The <strong>called</strong> strike zone can differ from this defined strike zone substantially. Umpire response (called ball or strike) changes smoothly and monotonically as a function of pitch location resembling psychometric curves studied in neuroscience and psychology. This curve can be well described by a point of subjective equality (PSE, <strong>P(strike)</strong>=0.5) and the width or uncertainty around the pitch location. These two parameters can be estimated by maximizing the likelihood of a Gaussian CDF with center point μ and uncertainty σ. The fit curve below (red) does a substantially better job of explaining calls than the “rulebook” strike zone (black).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/567/1*YcfP56eAQAEJOZ19I6UcGA.png" /><figcaption>Black dots: binned <strong>P(strike|called pitch, 1.8&#39;&lt;plate_z&lt;3.1&#39; )</strong>. Red line: Gaussian CDF fit as part of full <strong>called</strong> strike zone model. Black line: r<strong>ulebook </strong>strike zone.</figcaption></figure><p>We can extend this simple model explaining just one wall of the strike zone to a more complete description of the entire strike zone. Formally, we define</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/387/1*ECaI7Vb7CAXiIX_gO41nGg@2x.png" /></figure><p>for the 4 walls, <strong>w</strong>, of the strike zone {Left, Right, Bottom, Top}. The equations for the four walls are:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/707/1*avQ5BqIoBXFiHvuHaZDiSw@2x.png" /></figure><p>We can estimate the 6 free parameters {<strong><em>μ1, μ2, μ3, μ4, σx, σz</em></strong>} for a given set of called pitches. Our objective function is to maximize the log-likelihood of umpire calls and best fitting parameters are estimated using gradient descent. Note that we use a softplus transformation for σ values to ensure they remain non-negative. The visualization of the <strong>called</strong> strike zone for left and right handed batters is shown below (for the 2017 season). Note that for this visualization, even the “rulebook” strike zone is wider than home plate as we are using the center location of the pitch and only a part of the baseball needs to pass over home plate to be considered a strike.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1002/1*OvMmkAps3X5ocGUxAkvxRA.png" /><figcaption><strong>Called</strong> strike zone for left and right handed hitters in the 2017 season are shown in red/blue respectively. Also visualized is the width of home plate, size of baseball, and <strong>rulebook</strong> strike zone. Right handed/standing batter depicted. Note that strike zone is presented from catchers POV. Player graphic courtesy of <strong>vecteezy.com</strong>.</figcaption></figure><p>Comparing strike zones between batter handedness, it is clear pitchers are less likely to get inside and more likely to get outside calls. We can now visualize how the called strike zone changes based on various game conditions, for instance the count in the current at bat.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/223/1*TxCYJ2rdL3DR_10dMs6Nuw@2x.png" /><figcaption><strong>Called</strong> strike zone depicted for both <strong><em>0–2</em></strong> and <strong>3–0 </strong>counts for right handed batters only.</figcaption></figure><p>Alternatively, we can also collapse these strike zones to their surface area and see how this changes as a function of the number of balls/stikes in the current at bat:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/315/1*hQyQDP7EGIrOMnRBtQP8Yg@2x.png" /><figcaption>Area of strike zone (height x width; ft^2) for all possible counts.</figcaption></figure><p>These changes are not subtle! The called strike zone for right handed batters is 0.97 square feet or 33% larger on a <strong><em>3–0</em></strong> vs. an <strong><em>0–2</em></strong> count. There are many ways to contextualize this bias. On the one hand, we could emphasize how hopelessly subjective and fallible the umpires are, “shouldn’t the MLB just switch to the automated strike calling <a href="https://bleacherreport.com/articles/10079484-mlb-unlikely-to-use-automatic-ball-strike-system-during-2024-season-rob-manfred-says">system</a> used in the minor leagues already”? Alternatively we could first acknowledge that umpires are remarkably precise, with a typical σ around 60% the width of the baseball. The biases they do exhibit are in the exact direction most outside observers would prefer, by shifting their PSE slightly, they greatly reduce the chance of large impact mistakes (eg. calling a true strike a ball on a <strong><em>3–0</em></strong> count, leading to a game changing walk).</p><p>From a more statistical perspective, these biases may also be Bayes optimal. Umpires are making decisions under substantial uncertainty, pitches are fast, requiring resource limited judgements, and move in trajectories specifically meant to mislead batters and umpires alike. Umpire’s are also surrounded by agents attempting to mislead them by framing pitches (catchers) or selling that a pitch was un-hittable (batters). In the face of all this uncertainty, umpires can improve their estimate <strong><em>P(strike | x, y)</em> </strong>by using Bayesian inference to include the probability of a given pitch being a strike based on the current count.</p><p>We can get this prior probability by examining the average <strong><em>P(strike)</em></strong> of thrown pitches for a given count using a fixed model fit to all pitches. Strikingly, we observe a nearly identical pattern in where pitches are thrown based on the count relative to umpire strike zone bias. Put another way, in more hitter friendly counts (eg. <strong><em>3–0</em></strong>), umpires are more generous to pitchers in their calls AND pitchers throw pitches more likely to be called a strike, independent of this umpire bias. Thus the bias observed by umpires may actually be optimal in that it minimizes their overall error in calls. On the managers side, one could use these data points to argue for giving more hitters the <em>green light</em> in <strong><em>3–0</em></strong> counts as pitchers and umpires alike are biased against giving them the fourth ball they desire.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/315/1*N9rBgmOIcyMN8t8mer1hOA@2x.png" /><figcaption>Average model output <strong>P(strike)</strong> for model trained across all pitch counts evaluated separately for pitches thrown in each count. Note that all pitches are evaluated, not just “called” pitches used for training.</figcaption></figure><p>In this brief exploration, I used features of pitch location to model how the <strong>called</strong> strike zone differs from the <strong>rulebook</strong> strike zone and how the current count can influence this. I plan to further explore how other features -including pitch movement, umpire identity, and more- influence umpire behavior.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f3ce6616b8c1" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>