<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Michael Augustine on Medium]]></title>
        <description><![CDATA[Stories by Michael Augustine on Medium]]></description>
        <link>https://medium.com/@MichaelAugustine78?source=rss-305b60c6c4fc------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*VMHp7CKKTDMFrzNS.</url>
            <title>Stories by Michael Augustine on Medium</title>
            <link>https://medium.com/@MichaelAugustine78?source=rss-305b60c6c4fc------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 08:04:13 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@MichaelAugustine78/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[“The Real Challenge of Production ML: A Case Study”]]></title>
            <link>https://medium.com/@MichaelAugustine78/the-real-challenge-of-production-ml-a-case-study-eb4d921bb4fd?source=rss-305b60c6c4fc------2</link>
            <guid isPermaLink="false">https://medium.com/p/eb4d921bb4fd</guid>
            <category><![CDATA[sports-analytics]]></category>
            <category><![CDATA[mlops]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[mlb]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Michael Augustine]]></dc:creator>
            <pubDate>Sat, 11 Apr 2026 23:30:03 GMT</pubDate>
            <atom:updated>2026-04-11T23:36:31.612Z</atom:updated>
            <content:encoded><![CDATA[<h4>I Built an ML App to Predict MLB Pitches. Then Reality Hit. What happened when I tested my 73% accurate model on a real game — and why the “failure” taught me more than the success.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6ant5IRGPpFOCV7exKEx-A.png" /></figure><h3>The Setup</h3><p>Using Google Cloud’s Vertex AI, I trained a machine learning model on 4,000+ Tyler Glasnow (Los Angeles Dodgers) pitches from the 2024–2026 (current) season. The goal: predict his next pitch based on game situation — count, runners on base, outs, and batter handedness.</p><p>Training results looked promising:</p><ul><li>73% ROC AUC (significantly better than random)</li><li>75% accuracy on fastball predictions</li><li>Log loss improved 9% during training (1.320 → 1.202)</li></ul><p>I deployed the model and set up a Streamlit web app demo. The metrics seemed solid. Then I tested it on a real game.</p><p>That’s when things got interesting.</p><h3>Part 1: Building the Model</h3><h4>Data Collection</h4><p>I pulled pitch-by-pitch data from <a href="https://baseballsavant.mlb.com/">Baseball Savant’s Statcast system</a> — every Tyler Glasnow pitch from the 2024 to the current 2026 season (including the playoffs).</p><p>Features:</p><ul><li>balls (0-3)</li><li>strikes (0-2)</li><li>outs (0-2)</li><li>inning (1-9)</li><li>runner_1b, runner_2b, runner_3b (binary)</li><li>hitter_hand (L/R)</li></ul><p>Target:</p><ul><li>pitch_type (Four-Seam Fastball, Sinker, Curveball, Slider- four classes)</li></ul><p>Sample size: 4,235 pitches</p><h4>Training Approach</h4><p>I used Vertex AI ML Tabular Classification — Google’s automated ML platform that handles feature engineering, model selection, and hyperparameter tuning.</p><p>Configuration:</p><ul><li>Budget: 2 node hours (~$40)</li><li>Optimization: Log loss minimization</li><li>Split: 80/10/10 (train/validation/test)</li><li>Early stopping: Enabled</li></ul><p>Training took about 2.5 hours (wall-clock time, thanks to parallel processing).</p><h3>Issue #1: Data Type Confusion</h3><p><strong>The Problem:</strong> My first prediction attempts failed with a cryptic error:<br><em>400 {“error”: “Column: inning. Error: Expected string_value but got number_value. Type casting is not allowed.”}</em></p><p>AutoML had interpreted my numeric columns (inning, balls, strikes) as categorical strings, not integers.</p><p>The Resolution: Convert all input values to strings before sending predictions:<br><em># Convert numeric values to strings for AutoML<br>instance = {k: str(v) for k, v in game_situation.items()<br># Example:<br>{<br> “inning”: “5”, # Not 5<br> “balls”: “1”, # Not 1<br> “strikes”: “2”, # Not 2<br> “hitter_hand”: “R” # Already string<br>}</em></p><p>Novice mistake; AutoML Tabular infers data types during training, and you must match those types exactly during inference.</p><h3>Issue #2: Cold Start Timeout</h3><p><strong>The Problem:</strong> First predictions after deployment would timeout with:<br><em>429 Model is not yet ready for inference. Please wait <br>while model completes scale-up from zero.</em></p><p><strong>The Context:</strong> I’d configured the endpoint with scale-to-zero for cost optimization (min replicas: 0). When idle, the endpoint shuts down completely. First request after idle? 30–60 second wake-up time.</p><p><strong>The Resolution:</strong> This isn’t really fixable with scale-to-zero — it’s the trade-off. Options:</p><ol><li>Accept it (what I did — $0/hour when idle)</li><li>Set min replicas to 1 ($0.10/hour continuously, instant predictions)</li><li>Add retry logic with longer timeout</li></ol><p>For a demo project, scale-to-zero makes sense. For production with real users, you’d likely set min replicas to 1.</p><h3>Part 2: What the Model Learned</h3><p>The evaluation metrics told an interesting story about what matters in pitch selection.</p><h3>Feature Importance Analysis</h3><p>AutoML’s Shapley value analysis revealed the hierarchy of influence:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*44UtxJbrx7CwuoxZsdSdig.png" /></figure><p><strong>The takeaway:</strong> Strike count dominates everything else. This makes intuitive sense:</p><ul><li>0–2 count (pitcher ahead) → More breaking balls, can waste pitches</li><li>3–1 count (hitter ahead) → More fastballs, must throw strikes</li><li>2–2 count (even) → Mixed approach</li></ul><p>The model essentially learned the “book” on pitch sequencing from pure data.</p><h3>Per-Pitch Performance</h3><p>Not all pitch types were equally predictable:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HgkCNi6qaVTfhC2UOBNSFA.png" /></figure><p>Fastballs were most predictable — likely because they’re thrown most often in hitter’s counts (3–1, 3–2) where the pitcher needs strikes.</p><h3>Part 3: Deployment</h3><p>I deployed the model with this architecture:</p><p><strong>Infrastructure:</strong></p><ul><li>Vertex AI Endpoint (n1-standard-2 machine type)</li><li>Auto-scaling: 0–3 replicas</li><li>Scale-to-zero: Enabled (cost optimization)</li><li>Region: us-central1</li></ul><p><strong>Application:</strong></p><ul><li>Streamlit web interface (Python)</li><li>Plotly for visualizations</li><li>REST API calls to Vertex AI endpoint</li></ul><p><strong>Cost structure:</strong></p><ul><li>Idle: $0/hour (scaled to zero)</li><li>Active predictions: ~$0.10/hour</li><li>Expected monthly cost: $2–5 for occasional testing</li></ul><p>The full deployment (model → endpoint → app) took about 3 hours, including troubleshooting the data type issues.</p><h3>Part 4: The Real Test</h3><p>With the app working, I waited for Glasnow’s first start of the 2026 season.</p><p>Game context: Dodgers vs. Rangers, 10 April 2026</p><p>I tracked every pitch, comparing the model’s prediction against what Glasnow actually threw.</p><h4>The Results</h4><p>Raw accuracy: 51%<br>Adjusted accuracy: 65% (grouping FF/SI as “fastballs”)</p><h4>Initial Reaction</h4><p>My first thought: <em>That’s… not 73%.</em></p><p>Training metrics suggested I’d get ~70–75% accuracy on new data. Reality delivered 51% raw, 65% adjusted.</p><p>Was the model broken..?</p><h3>Part 5: The Real Learning — Data Drift</h3><p>I dug into the predictions vs. actuals and discovered something fascinating: Glasnow had changed.</p><h4>Pitch Mix Evolution (2024 Training Data → 2026 Reality)</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3UwleV5xi4SXUTJM0wbauA.png" /></figure><p>The model wasn’t broken, Glasnow had shifted his approach:</p><ul><li>10% fewer four-seam fastballs</li><li>10% more curveballs</li><li>8% more sinkers</li><li>8% fewer sliders</li></ul><p>Imagine a sales forecasting model trained on historical data trying to predict 2026 behavior after customer preferences shifted 10%. It would similarly fail.</p><h4>Why the Change?</h4><p>Potential explanations:</p><ol><li>Early season variance; his approach hasn’t regressed to his mean</li><li>League adjustment — Hitters were adjusting to Glasnow’s approach</li><li>Coaching strategy — Dodgers pitching staff adjusted his arsenal</li><li>Natural evolution — Elite pitchers constantly tinker</li></ol><h3>Part 6: What 51% Really Means</h3><p>Once I understood the drift, I reassessed the model’s performance.</p><h4>Context Matters</h4><p>Baseline comparisons:</p><ul><li>Random guessing: 25% (1 in 4 pitch types)</li><li>Most common pitch only: ~35% (always guess fastball)</li><li>My 2-year-old model: 51% raw, 65% adjusted</li><li>Professional scouts (with current intel): ~60–70% estimated</li></ul><p>My model beat random guessing by 2x despite working with outdated information.</p><h4>What the Model Got Right</h4><p>Looking at count-based accuracy:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*n9Ce45ap6z_EU1fDDRHsCA.png" /></figure><p>The model learned the strategy (when to throw hard vs. soft). It just had the wrong pitch mix percentages as priors.</p><h4>The FF/SI Confusion</h4><p>The jump from 51% → 65% when grouping four-seam fastballs and sinkers reveals something important:</p><p>These pitches are functionally similar:</p><ul><li>Both ~95–98 mph</li><li>Both fastballs (four-seam and sinker) strategically</li><li>Similar usage patterns (get-me-over strikes)</li><li>Differ mainly in movement (sink vs. rise)</li></ul><p>Sometimes certain pitch evaluation/identification systems can mislable pitches. Here’s a look at Glasnow’s four-seam and sinker movement profiles.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*022OEsZHXA6pPL1pDfoCyA.png" /><figcaption>Sinker (Orange), Four-Seamer (Red)</figcaption></figure><h3>Part 7: Production ML is Different</h3><h4>Training vs. Production: The Reality Gap</h4><p>In Training:</p><ul><li>✅ Static dataset</li><li>✅ Fixed distribution</li><li>✅ Clean train/test split</li><li>✅ Metrics look great</li><li>✅ “73% accuracy!”</li></ul><p>In Production:</p><ul><li>⚠️ Data distribution shifts</li><li>⚠️ Subjects evolve their behavior</li><li>⚠️ Metrics degrade over time</li><li>⚠️ Need continuous monitoring</li><li>⚠️ “Wait, why 51%?”</li></ul><h4>The Core Challenge: The World Changes</h4><p>In business applications:</p><ul><li>Customer preferences shift</li><li>Market conditions evolve</li><li>Competitive landscapes change</li><li>Economic factors fluctuate</li></ul><p>In sports:</p><ul><li>Players adjust strategies</li><li>Injuries affect capabilities</li><li>Coaching changes philosophy</li><li>Opponents adapt</li></ul><p>My model captured mostly 2024 Glasnow. It was accurate — for 2024. But by 2026, I was predicting a slightly different pitcher.</p><h3>Part 8: The Path Forward</h3><p>Understanding the drift opened up clear next steps.</p><h4>Option 1: Retrain with Current Data</h4><p>Approach: Collect Glasnow’s 2026 pitches and retrain.</p><p>Challenge: Sample size. I have ~100 pitches from one start. Need 800–1,000+ for stable performance.</p><p>Timeline: Wait for 8–10 more starts (~6–8 weeks).</p><h4>Option 2: Weighted Ensemble</h4><p>Approach: Combine historical patterns with updated pitch mix.</p><p>Implementation:</p><ul><li>Keep the game situation logic (count, runners, etc.)</li><li>Update baseline pitch percentages to 2026 rates</li><li>Weight recent data more heavily</li></ul><p>Advantage: Works with limited new data.</p><h4>Option 3: Continuous Learning</h4><p>Approach: Retrain after every 5 games, using a rolling window.</p><p>Implementation:</p><ul><li>Always train on “last 1,500 pitches”</li><li>Add new games, drop oldest</li><li>Model stays current automatically</li></ul><h3>My Plan</h3><p>I’m taking a wait-and-collect approach:</p><ol><li>Track every Glasnow start for the next 6–8 weeks</li><li>Build a test set of 800–1,000 2026 pitches</li><li>Retrain with weighted data (2024-2026, favoring recent)</li><li>Compare performance old model vs. new</li><li>Write a follow-up on the improvement (or lack thereof)</li></ol><h3>Part 9: Key Takeaways</h3><h4>1. Testing in Production is Essential</h4><p>If I’d stopped at “73% ROC AUC,” I’d have a nice-looking portfolio project. By testing on a real game, I uncovered:</p><ul><li>Data drift (10% pitch mix change)</li><li>Model decay over time</li><li>The gap between training and production metrics</li></ul><p>This makes the project 10x more valuable for demonstrating ML understanding.</p><h4>2. Failure is More Instructive Than Success</h4><p>51% felt like failure initially. But analyzing <em>why</em> taught me:</p><ul><li>How distributions shift in production</li><li>Why monitoring is critical</li><li>What model drift looks like quantitatively</li><li>How to diagnose performance degradation</li></ul><h4>3. Domain Knowledge Matters</h4><p>Understanding baseball helped me interpret results:</p><ul><li>FF/SI confusion makes sense (similar pitches)</li><li>Count-based patterns validate (matches strategy)</li><li>10% shift is huge (not just noise)</li></ul><p>ML + domain expertise &gt; ML alone.</p><h4>4. Sample Size is Critical</h4><p>4,000 training examples was enough for 73% training metrics. But to adapt to a shifted distribution, I need comparable volume of new data.</p><p>Lesson: Don’t expect miracles from small datasets when the world has changed.</p><h4>5. Production ML Requires Different Metrics</h4><p>Training metrics I tracked:</p><ul><li>ROC AUC</li><li>Log loss</li><li>Precision/Recall</li><li>Confusion matrix</li></ul><p>Production metrics I <em>should</em> track:</p><ul><li>Accuracy over time (is it degrading?)</li><li>Distribution shift (is the data changing?)</li><li>Prediction confidence (is the model getting less certain?)</li><li>When to retrain (performance threshold triggers)</li></ul><p>I’m adding these for the next iteration.</p><h3>Part 10: Broader Applications</h3><p>This pitch prediction case study applies directly to business ML problems.</p><h4>Similar Challenges in Production:</h4><p>E-commerce Recommendations:</p><ul><li>Customer preferences shift seasonally</li><li>New products enter catalog</li><li>Trends change rapidly</li><li>Model trained on 2024 Q4 won’t work perfectly in 2026 Q2</li></ul><p>Fraud Detection:</p><ul><li>Fraudsters adapt tactics</li><li>New attack vectors emerge</li><li>Transaction patterns evolve</li><li>Model decay is constant</li></ul><p>Demand Forecasting:</p><ul><li>Market conditions change</li><li>Competitor actions shift demand</li><li>Economic factors fluctuate</li><li>Historical patterns may not hold</li></ul><p>The Glasnow Lesson Applies: <em>Your model is a snapshot of a specific time. The world keeps moving.</em></p><h3>Conclusion: The Real Value</h3><p>I set out to build an AI pitch predictor. I succeeded — technically.</p><p>But the real value wasn’t the 73% training accuracy or the slick Streamlit app. It was the 51%.</p><p>That “failure” taught me:</p><ul><li>✅ Why production ML is hard</li><li>✅ How to diagnose model drift</li><li>✅ What continuous monitoring means</li><li>✅ Why retraining strategies matter</li><li>✅ How the world changing breaks static models</li></ul><h3>Resources &amp; Code</h3><p><strong>Video Overview</strong></p><p><a href="https://www.linkedin.com/feed/update/urn:li:activity:7448399349681893395/?originTrackingId=qNxCgDTwZ8cF2n4UDI2ngQ%3D%3D">#machinelearning #ai #datascience #mlops #vertexai #googlecloud #buildinpublic #techprojects #baseball #baseballanalytics #mlb #dodgers #tylerglasnow #dodgerblue #sportsanalytics #sportstech #python... | Michael Augustine</a></p><p><strong>Tech Stack:</strong></p><ul><li>Google Cloud Vertex AI (AutoML Tabular Classification)</li><li>Python (pandas, google-cloud-aiplatform)</li><li>Streamlit (web interface)</li><li>Plotly (visualizations)</li><li>Baseball Savant (data source)</li></ul><p><strong>Key Metrics:</strong></p><ul><li>Training: 73% ROC AUC, 75% fastball accuracy</li><li>Production: 51% raw accuracy, 65% adjusted</li><li>Data drift: 10% pitch mix shift (2024 → 2026)</li></ul><p><strong>GitHub:</strong> [coming soon — will include full code, data analysis, and retraining experiments]</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=eb4d921bb4fd" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[mlB Pitch Predictor: Using Machine Learning to Previse Pitch Sequencing]]></title>
            <link>https://medium.com/@MichaelAugustine78/mlb-pitch-predictor-using-machine-learning-to-previse-pitch-sequencing-c5d38b9ac97c?source=rss-305b60c6c4fc------2</link>
            <guid isPermaLink="false">https://medium.com/p/c5d38b9ac97c</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[mlb]]></category>
            <category><![CDATA[tyler-glasnow]]></category>
            <category><![CDATA[azure]]></category>
            <dc:creator><![CDATA[Michael Augustine]]></dc:creator>
            <pubDate>Wed, 22 May 2024 18:40:36 GMT</pubDate>
            <atom:updated>2024-05-23T11:08:54.888Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>INTRODUCTION</strong></h3><p>Using Azure Machine Learning Studio, I developed a tool called mlB Pitch Predictor (combining machine learning withMajor League Baseball). The tool uses several pieces of data entered by a user to make a prediction on what the next pitch thrown will be.</p><p>For my test subject, I wanted to use a pitcher who relies on three main pitches in their arsenal. Three, because too few options (most relief pitchers throw only two pitches), the model may have a much easier time with its estimate; frankly, a human could be just as accurate with their guesses. A pitcher with a large arsenal (five or more pitches) could create too many random predictions and require a large amount of data (at least three years worth of pitch history) as well as a lot of training time.</p><p>Additionally, the pitcher should have a modest level of MLB experience and whose pitch selection doesn’t deviate too far from their historical tendencies.</p><p><a href="https://en.wikipedia.org/wiki/Tyler_Glasnow">Tyler Glasnow</a> of the Los Angeles Dodgers has always been a personal favorite. More importantly, he meets most, if not all, requirements for this experiment.</p><p>Before I begin, disclosing caveats is necessary to restrain the idea that the predictor is something of a solution for, say, scouts or professional hitters. The design of this tool is being motivated by the fun it could create for fans.</p><h3><strong>LIMITATIONS</strong></h3><p>Obviously, we don’t have access to the pitchers brain; they can make adjustments on the fly as game situations change. One way this happens is by altering their approach when the original game plan goes awry.</p><p>For example, a pitcher may decide to attack a team with a heavy dose of fastballs; perhaps the pitcher throws a near-100 MPH four-seam fastball with heavy spin and a bit of cut that throws hitters off just enough. So what happens if that strategy falls apart and the hitters are making good contact? The pitcher may scale back on their fastball and start mixing in their secondary pitches more often. This can create an issue as the predictor isn’t able to account for that type of adjustment during its training; it only learns which pitch was thrown under what conditions. While aspects of human behavior can be predictable, the complexity with which our minds evaluate and re-evaluate our circumstances can become unquantifiable.</p><p>Additionally, a pitcher may not follow their historical pitch choice given the <a href="https://library.fangraphs.com/misc/re24/">base-out state</a>, inning, count, etc. Let’s say a pitcher with runners in scoring position tends to throw more breaking balls. Maybe the hitter that’s up feasts on breaking pitches; the pitcher may not follow their normal tendencies, creating an incorrect or uncertain prediction. Or maybe a hitter does really well early in the count on fastballs and the pitcher tends to be fastball-heavy early in counts; the pitcher may adjust their normal approach depending on how confident they are in the pitch.</p><p>Another detail to consider: P<a href="https://baseballsavant.mlb.com/leaderboard/statcast-park-factors?type=distance">ark Factors</a>. A stadium like the Colorado Rockies’ Coors Field in Denver, CO is in a high elevation area which creates extra carry for fly balls; environmental factors (including weather) like that can influence how a pitcher approaches their game plan. For instance, if a pitcher is starting against the Rockies in their stadium and they like to throw a lot of fastballs (maybe a slightly disconcerting amount of them ending up home runs), they may decide to throw more breaking pitches than they otherwise do.</p><p>Pitchers can also change strategies based on the effectiveness of certain pitches. Maybe a curveball isn’t working well and the pitcher decides to throw it less, or, instead of upping the use of their other pitches, they may have to adjust its design or perhaps work on a new pitch.</p><p>One other thing to consider is the umpire; this is a sneaky problem. Most umpires are fairly consistent with their strike zones though there are some that seem to have expanding or shrinking zones game to game. That affects the predictor tool because if an ump is calling balls on pitches in certain locations that tend to be strikes (based on visual data like a heat map), the pitcher might be forced to change their strategy (e.g.- throwing pitches in more dangerous locations in an attempt to get a strike called). If that sets up hitters to have better swing opportunities, it could create chaos in the pitchers game plan and force them to deviate from their typical approach.</p><p>Its also important to note that I’m testing just one MLB pitcher <a href="https://www.fangraphs.com/leaders/major-league?pos=all&amp;lg=all&amp;qual=y&amp;type=8&amp;season=2024&amp;month=0&amp;season1=2024&amp;ind=0&amp;stats=sta&amp;pagenum=1&amp;pageitems=100">out of 77 registered starters in 2024</a>; the predictor could be more accurate for some pitchers than others (a study for another day).</p><h3><strong>THE PROCESS</strong></h3><p>I decided to take all OF Glasnow’s 2023 pitching data from <a href="https://baseballsavant.mlb.com/">Baseball Savant</a> as well as his current pitch data for the 2024 season (Glasnow’s 2022 season was cut short with an injury). The data I included for training was inning, hitter handedness, balls, strikes, runners on first/second/third, and outs. That situational criteria created the more concise predictions versus including data like pitch location; that alone could be a utility unto itself.</p><p>Over in Azure Machine Learning Studio, I uploaded the data and created a split data component divided up for training (70%) and testing (30%). I then chose an algorithm for training the model on the provided data; a <a href="https://learn.microsoft.com/en-us/azure/machine-learning/component-reference/multiclass-decision-forest?view=azureml-api-2">Multiclass Decision Forest</a>.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fv8benyaLT9w&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dv8benyaLT9w&amp;image=http%3A%2F%2Fi.ytimg.com%2Fvi%2Fv8benyaLT9w%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/4d0cf29b0473bed32d7c890c35bf6b31/href">https://medium.com/media/4d0cf29b0473bed32d7c890c35bf6b31/href</a></iframe><p>The first testing and evaluation runs in Machine Learning Studio peaked at an overall accuracy of 61%. Generally speaking, a model with an accuracy score of 70% or higher is considered a good result and therefore a reliable utlility. The unquantifiable variables caveats mentioned earlier, as well as being limited to roughly 2800 rows of data, created a challenge to climb over that threshold of accuracy.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/802/1*yHVjr5pgk4udMEhmCE-ZFw.png" /></figure><p>Since fastballs make up the majority of Glasnow’s pitch selections, the true positive rate in the confusion matrix above was expected to be high. The curveball and slider rates were significantly lower as they are used less (fewer learning situations) and false positives appeared much less often than the fastballs.</p><p>Glasnow’s May 10th start was the first real world test for the predictor. He threw 99 pitches that game and the model produced an accuracy rate of 59%; a palatable result but not great.</p><p>I then decided to look back at the prediction chart and scanned through until I found a 5% or less separation between predictions and eliminated them from the overall count as they are as good as a coin flip; this occurred because, given a circumstance, there simply wasn’t enough data to provide an accurate guess. After doing that the accuracy rate increased by 2% to 61%, which lines up with the overall accuracy of the ML training.</p><p>That result was decent but I wanted a bit more accuracy and some legitimacy to the model.</p><p>Prior to Glasnow’s next start (May 16th), I went back into Machine Learning Studio and updated the dataset to include the May 10th start and kept the Multiclass Decision Forest algorithm. Following this additional training, I was hoping to see improvement with the additional data, but there was not. So, I decided to change the testing algorithm to a <a href="https://learn.microsoft.com/en-us/azure/machine-learning/component-reference/multiclass-boosted-decision-tree?view=azureml-api-2">Multiclass Boosted Decision Tree</a>. After the change, I ran several training jobs with minor tweaks to the algorithm’s parameters until I was able to achieve the best score.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*k5kEHq2PPvx2hKsbDjBfjQ.png" /><figcaption>Final parameters for v2</figcaption></figure><p>When I settled on the right settings, I saw the predictor’s accuracy jump from 61% to 66%.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IQKR9vr251J3rrQxK5V2gg.png" /><figcaption>Evaluation Metrics for MLb Pitch Predictor v2</figcaption></figure><p>Furthermore, there were significant improvements in the confusion matrix feedback.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/804/1*cZskmefSFST2WvOVqHzGQg.png" /></figure><p>After Glasnow’s outing on the 16th, I went back and ran the model on the game data. This outing produced a 73% accuracy rate.</p><p>Evaluating the predictor output showed there were 15 toss-ups (16% of pitches) which were any predictions of 50% certainty or lower, or, any predictions (right or wrong) that were within 5% of each other.</p><p>The predictor also produced a certainty rate of 90% or more on 28 of Glasnow’s 92 pitches, missing on only three of them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Sd5sRE76yY7MshKX3_-nhQ.png" /><figcaption>Predictor sheet adjusted to display 90%+ certainties.</figcaption></figure><p>The predictor performed very well considering Glasnow had a rough start that night and had to break a bit from his normal pitch distribution. For this start, he threw 5% less fastballs compared to his ratio for this season, his slider use remained steady while his curveball rate upped from a season average of 17% to 21% during this start.</p><p>I will continue to test, train, modify parameters (if necessary), and evaluate version two of the mlB Pitch Predictor with at least one additional follow up performance report somewhere before or after the MLB All-Star Break in July. For now, I’m very satisfied with how accurate the tool is. Though the 66% accuracy rate falls below the threshold for a very good model (70%), as I mentioned earlier in the “Limitations” section, this method of machine learning lends itself to greater volatility.</p><p>While the overhead would be quite large, ultimate goal for this study would be to create enough satisfactory results to validate this as an app that anyone could use on (ideally) all registered starters. Thought the MLb Pitch Predictor tool isn’t intricate enough for a team’s R&amp;D department to rely on, the fun in using it might be akin to the enjoyment some fans get filling out a score card during a game.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c5d38b9ac97c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Who Obtains the Most Assistance in Pitcher Welfare?]]></title>
            <link>https://medium.com/@MichaelAugustine78/who-obtains-the-most-assistance-in-pitcher-welfare-67157c6e325b?source=rss-305b60c6c4fc------2</link>
            <guid isPermaLink="false">https://medium.com/p/67157c6e325b</guid>
            <category><![CDATA[sabermetics]]></category>
            <category><![CDATA[sports]]></category>
            <category><![CDATA[mlb]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Michael Augustine]]></dc:creator>
            <pubDate>Thu, 01 Feb 2018 20:19:37 GMT</pubDate>
            <atom:updated>2018-02-01T20:19:37.008Z</atom:updated>
            <content:encoded><![CDATA[<p>Nobody’s perfect, especially umpires. This is the case at any level of the game. Be it softball, tee ball, or baseball, from Little League to the Big Leagues, you will have undeniably disagreed with a call that an ump has made.</p><p>Given the movement, velocity, and the newly anointed skill of <a href="http://grantland.com/features/studying-art-pitch-framing-catchers-such-francisco-cervelli-chris-stewart-jose-molina-others/">pitch framing</a>, it’s becoming more difficult for umpires to get the calls right. <a href="http://www.sportingnews.com/mlb/news/robot-umpires-electronic-strike-zone-in-mlb-pitchfx-balls-and-strikes/ggv80p4qmsu31d72sohrmbcnj">The robo ump</a> has been discussed quite a bit but I’m not sure how I feel about a machine making decisions in lieu of accepting the concept of human error. We did it for decades before instant replay was instituted.</p><p>Umpires get balls and strikes wrong a lot. It’s the way it goes. Given that understanding, I wanted to know which pitcher has in recent years been the beneficiary of favorable calls.</p><p>And, like the umpires, not all (strike zone) charts are 100% accurate; leave a <em>little</em> room for error here.</p><p>I’ve parsed data on which pitchers have had the most declared strikes that were actually out of the zone. I decided to stop at 2014 because I felt that four years of information was sufficient for the study.</p><p>First, the accumulated data.</p><p>From 2014 to 2017, the amount of pitchers with phantom strikes has been increasing at fairly high rate; the biggest leap was from 2014 to 2015 (36 pitchers).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*2VSO4DQH66yray9B.png" /></figure><p>Interestingly, the pitchers with at least 100 ‘phantom strike’ calls has actually <em>decreased</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*uz6gMIFk21jiCErS.png" /></figure><p>And, despite the jump in total pitchers involved from ’14 to ’15, the pitchers with &lt;=100 strikes called decreased at the highest rate.</p><p>Should we go tin foil hat and infer that umps are no longer favoring certain pitchers as much as they used to? Doubtful, but I’m not investigating integrity here.</p><p>So who is getting the most benefit from the perceptively visually impaired? First, I took the last four years of pitching data for our parameters. Then, I cut final the list down to a minimum of 10,000 pitches thrown. Lastly, I included only the top 20 pitchers in the group.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/336/0*3SrL4cOkXhjrW286.jpg" /></figure><p>As we can see, <a href="http://www.fangraphs.com/statss.aspx?playerid=4930&amp;position=P">Jon Lester</a> of the Chicago Cubs has been the most aided overall; 562 non-strikes in four years.</p><p>For the optically minded, here is the pitch chart of Lester’s data.</p><p>[caption id=”attachment_2199229&quot; align=”aligncenter” width=”490&quot;]</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*GR4Xg0AoSX1tGRWH.png" /></figure><p>That’s A LOT of Trix![/caption]</p><p>Now, lets see if the percent of pitches has any impact on our leader(s).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/337/0*F0n-XVwZ5N9G80O_.jpg" /></figure><p>Not a whole lot of variance, at least near the top. Lester clearly wins The MLB Umpires’ “Benefit of the Doubt Award”.</p><p>OK, so now we’ve got our man. Case closed, right?</p><p>Oh…that little caveat of ‘pitch framing’. Perhaps its that Lester has had great framing from his catchers. Let’s look into that.</p><p>For the moment, we are going to focus on Lester and his primary catcher from 2014–2016, <a href="http://www.fangraphs.com/statss.aspx?playerid=1551&amp;position=C">David Ross</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/680/0*2vmkCyFCGK8WCUOj.jpg" /></figure><p>Clearly 2014 was Lester’s most favorable year with Ross. That year, Lester ranked third in total pitches called favorably out of the zone (156) and 11th in ratio of calls (4.47).</p><p>The subsequent years with Ross are as follows:</p><p><strong><em>2015- 6th (141), 10th (4.43)</em></strong><br> <strong><em>2016- 5th (125), 7th (3.95)</em></strong></p><p>Here’s where things get a bit intriguing. Recapping 2017, things appear to fall apart completely for the Cubs in the context of pitch framing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/442/0*3smOS69YAL7CSLHq.jpg" /></figure><p>The only catcher who was able to garner a positive framing rating was <a href="http://www.fangraphs.com/statss.aspx?playerid=16478&amp;position=OF">Kyle Schwarber</a>, who caught just seven innings that year. But even his stats are far from impressive.</p><p>And how did Lester fair in terms of ‘phantom strikes’ that year? He ranked first in overall strikes called out of the zone (150) and fourth in total call ratio to pitches thrown (4.46).</p><p>He wasn’t all that far from the top under Ross, but was basically the frontman of the metrics in 2017.</p><p>Some things are hopelessly lost in the sphere of the unexplained. But, the research didn’t set out to find reasoning. In this case its more fun to be left with subjective theories. However, it’s a bit silly to think that there is actually an umpire conspiracy allowing Lester to succeed when he apparently shouldn’t.</p><p>My best guess is maybe they feel sorry for him since <a href="http://www.stack.com/a/the-psychological-issue-behind-jon-lesters-inability-to-throw-to-first-base">he can’t accurately throw the ball in the infield</a> anywhere other than to the catcher (which <em>did</em> changed a bit in 2017)?</p><p>Regardless, Lester is our guy, here; receiving a sizable edge in terms of missed calls. It will be interesting to see if this trend continues this season.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=67157c6e325b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Miguel Cabrera and the Inevitable Regression]]></title>
            <link>https://medium.com/@MichaelAugustine78/miguel-cabrera-and-the-inevitable-regression-3dc4a5ae35dd?source=rss-305b60c6c4fc------2</link>
            <guid isPermaLink="false">https://medium.com/p/3dc4a5ae35dd</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[sports]]></category>
            <category><![CDATA[baseball]]></category>
            <category><![CDATA[sabermetics]]></category>
            <category><![CDATA[mlb]]></category>
            <dc:creator><![CDATA[Michael Augustine]]></dc:creator>
            <pubDate>Thu, 01 Feb 2018 20:15:08 GMT</pubDate>
            <atom:updated>2018-02-01T20:15:08.723Z</atom:updated>
            <content:encoded><![CDATA[<p><a href="http://www.fangraphs.com/statss.aspx?playerid=1744&amp;position=1B/3B">Miguel Cabrera</a> had a tough 2017. Could his decline be due to age regression? Could it be the back problems he allegedly played through? Or, was he just plain unlucky?</p><p>Knee-jerk assumption is health issues. From <a href="https://www.si.com/author/jon-tayler">Jon Tayler</a> (Sports Illustrated):</p><blockquote><em>…it’s clear that, at age 34, his body is breaking down. On (September 24th 2017), Detroit learned that Cabrera, who had to leave Saturday’s game early with back pain, </em><a href="http://www.freep.com/story/sports/mlb/tigers/2017/09/24/detroit-tigers-miguel-cabrera/698415001/"><em>has been diagnosed with two herniated discs in his lower back</em></a><em>, with manager Brad Ausmus telling reporters that his star may not play again this year. Back issues have been a problem for Cabrera </em><a href="https://www.cbssports.com/mlb/news/tigers-miguel-cabrera-likely-to-miss-the-rest-of-the-wbc-because-of-a-back-injury/"><em>since he played for Venezuela in the World Baseball Classic back in March</em></a><em> and are the latest in a litany of aches and pains he’s dealt with since turning 30; as Ausmus put it, “This has probably slowly been developing for years.</em></blockquote><p>Baseball players break down. Some sooner (and more drastically) than others. A player with Cabrera’s skill set can regress and still be above average.</p><p>So, let’s delve into regression and luck.</p><p>A quick overview of the last four seasons for Cabrera.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/514/0*qMaRZZHy7PCyDbIU." /></figure><p>2017 was likely worse than anyone could have reasonably expected.</p><p>We’ll mostly work with Weighted On-Base Average. wOBA is a great tool that helps determine how productive a hitter has been.</p><p>Its more informative than OBP as it uses weights to determine where a hitter ended up and what he accomplished when reaching base. OBP only tells us that the batter got on base. That might be enough for others, but some of us would like a little more context; no judgement on which you prefer.</p><p>For regression sake, let’s look at Cabrera’s career wOBA against the league average in terms of age.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/620/0*4XmZDVrf9unyTmld." /></figure><p>As we can see, Cabrera had a wOBA well above average for a player his age. According to the chart data, at ages 29 and/or 32 is when wOBA seems to peak; .319 for 29 year olds, .320 for 32 year olds.</p><p>Once Cabrera hit 34, he crashed back to earth and managed a league average wOBA.</p><p>Perhaps 2017 was an anomaly; a result of bad luck? Here’s a glance at his batting average on balls put in play. His career BABIP is .344 and 2017 he posted a .292; slightly worse than league average.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/592/0*wwRVCdKBXw884rhN." /></figure><p>Cabrera’s BABIP remained steady, save for a few fluctuations, then plunged into mediocrity. So he was just unlucky…right?</p><p>We can look at this graph and see from age 19 to 23 it continually climbed, then dove down at 25. The chart follows a similar trend, with a bit more volatility, from age 26 to 32. Can we infer it will trend upward again? Since it would be quite a feat for his BABIP to go on another positive run, I’d venture to guess that, at this point, it will stabilize.</p><p>So, can we blame the injury now?</p><p>Well, we can’t measure how much his back problems effected his hitting. We can take it into consideration but we don’t know how much it was actually bothering him. He managed to play in 130 games, so its hard to say it was that much of a problem for him. I would presume, depending on the pride of the player, that as you got older you’d want to protect your body more; give it more rest. Obviously Cabrera is a tough guy as he averages something like 150 games a season. Knowing that the organization was sliding down into obscurity, maybe he felt it was his duty to keep playing for the fans.</p><p>Those are a lot of ‘maybe’s’.</p><p>Other than his rookie year, he’s never played less than 100 games each season. He had injuries in 2015; listed as day-to-day with a back soreness on September 23rd and ended up on the 15-day DL July 4th with a calf injury.</p><p>Regardless, the sharpness of the wOBA decline is what I find disconcerting. His biggest drop in wOBA occurred between the ages of 29 and 30; about a .070 drop. Then, going from age 33 to 34, it dropped .086 points. To note, the average wOBA actually increases two hundredths of a point from the 33 to 34 years.</p><p>So why did this happen? We’re going to investigate Cabrera’s wOBA versus his xwOBA for 2017.</p><p>To summarize xwOBA: Based upon the type of contact, it’s what was expected to happen versus what actually happened.</p><p><a href="http://www.fangraphs.com/statss.aspx?playerid=15640&amp;position=OF">Aaron Judge</a> drives a ball into the left field gap, under a certain launch angle and exit velocity. Let’s say he hits it into an average outfield and it drops in for a double. Alternately, <a href="http://www.fangraphs.com/statss.aspx?playerid=10155&amp;position=OF">Mike Trout</a> drives a ball under the same conditions but <a href="http://www.fangraphs.com/statss.aspx?playerid=10199&amp;position=OF">Billy Hamilton</a> is playing center field. Since Hamilton has elite speed and is a good defender, he caught the same type of hit Judge dropped between inferior defenders.</p><p>One other thing I want to point out about xwOBA. It takes speed into account. <a href="http://www.fangraphs.com/statss.aspx?playerid=1177&amp;position=1B">Albert Pujols</a> is not a fast runner; much slower than average. That being said, he’s more inclined to hit into double plays and/or unable to leg out an infield single. A ball hit with the same trajectory by other players might be beaten out.</p><p>I understand that speed is a factor in a game but given the likelihood of that ground ball being hit for an infield single, xwOBA would adjust for a player like Pujols because it would be expected that he could leg it out. That aspect could be seen as a flaw depending on your point of view.</p><p>This might be oversimplifying the concept…or making it even more confusing. And it might not be an exact science, but its pretty darn close.</p><p>Here’s a chart of comparison to other hitters who saw a variance from xwOBA to (actual) wOBA in 2017.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/285/0*_SKO2FUxvhI3BZeU." /></figure><p>We can see one thing standing out; Cabrera, by two hundredths of a point, is well ahead of the other nine in terms of the difference. It took a little bit of a dip from <a href="http://www.fangraphs.com/statss.aspx?playerid=4467&amp;position=1B/OF">Brandon Moss</a>to <a href="http://www.fangraphs.com/statss.aspx?playerid=7185&amp;position=2B">Logan Forsythe</a> but not as drastic.</p><p>Going back to 2016, <a href="http://www.fangraphs.com/statss.aspx?playerid=7399&amp;position=1B/DH">Billy Butler</a> had the biggest drop at -.058; Cabrera finished fourth with -.050 (.459 xwOBA/.409 wOBA).</p><p>So, two reasonably big differential drops over the course of two years. The caveat here is in 2016, Cabrera was much more productive; a 4.8 WAR with a 152 wRC+.</p><p>Look upon this contact visual, from ‘16-‘17, of Cabrera’s xwOBA and wOBA.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/394/0*z3lVjee9EVReZ6W6." /></figure><p>Quite a distinction in contact as well as balls in play. Yet, his launch angles remained in the same sphere, between roughly 40 and -20 degrees. Cabrera clearly isn’t having a problem with his swing. Mechanically, anyway.</p><p>Let’s move onto contact and exit velocity during the drop-off years of ’16 and ’17. In 2016 Cabrera had a total of 238 ‘good contact’ hits (barrels/solid/flares) on 9.5% of pitches seen:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/552/0*q1TQGP5R7EmCzyg1." /></figure><p>In 2017, 161 with a 7.8% ratio:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/552/0*cHTCXMBGlDjU7GOy." /></figure><p>And how about Cabrera’s exit velocity?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/216/0*yBqw02k0jNkTLzuV." /></figure><p>Was he swinging in pain? How much and to what detriment isn’t quantifiable, especially because he still managed 500-plus plate appearances. Pain to you isn’t necessarily pain to someone else.</p><p>Cabrera is on the decline. His xwOBA data makes a case for that. I’m going to infer it was simply coincidence that his injury occurred the same year. While he wasn’t hitting the ball as hard (humans do lose strength), he maintained his launch angles; something that would have changed (at least a little) if you’re burdend from back pain.</p><p>You can’t play at a high level like Cabrera did forever. Even during his not-so-great years, he was still so much better than an average player. Regression is inevitable. 2017 appears to be the year that happens for one of the best hitters the game has ever seen.</p><p><em>*Statcast data courtesy of Baseball Savant</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3dc4a5ae35dd" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>