<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Rob Mepham on Medium]]></title>
        <description><![CDATA[Stories by Rob Mepham on Medium]]></description>
        <link>https://medium.com/@robert.mepham?source=rss-745e2a58a3bf------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Iej_I_zGO74Jfv14BInOUw.png</url>
            <title>Stories by Rob Mepham on Medium</title>
            <link>https://medium.com/@robert.mepham?source=rss-745e2a58a3bf------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 10:32:53 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@robert.mepham/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[College Basketball Has Team Sheets… Now I Built Ones for College Football]]></title>
            <link>https://medium.com/@robert.mepham/college-basketball-has-team-sheets-now-i-built-ones-for-college-football-a1cf6b1ed7ec?source=rss-745e2a58a3bf------2</link>
            <guid isPermaLink="false">https://medium.com/p/a1cf6b1ed7ec</guid>
            <category><![CDATA[college-football-playoffs]]></category>
            <category><![CDATA[march-madness]]></category>
            <category><![CDATA[sports-analytics]]></category>
            <category><![CDATA[college-football]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Rob Mepham]]></dc:creator>
            <pubDate>Wed, 18 Nov 2026 14:41:01 GMT</pubDate>
            <atom:updated>2026-11-18T14:41:01.609Z</atom:updated>
            <content:encoded><![CDATA[<p>Every March, the men’s basketball committee sits down with a sheet of paper for every bubble team. The sheet lists the team’s record in four tiers of opponent quality, its rank in the NCAA’s own rating, and a column of well-known power ratings. When two teams are close, the committee can point at a number and say why one is in and one is out. You can disagree with the number. You can’t say there wasn’t one.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-E1IQ6npazS4dhdi73-Z8w.png" /></figure><p>In December, the college football committee has a ranking and a press conference.</p><p>I’ve never thought that was a good state of affairs, and next season it gets worse. The playoff is reported to be doubling to 24 teams (that deal isn’t final: more on that in a moment), so the argument that currently happens around the 10th to 14th teams will move to the 22nd to 26th. Those are the teams with the least separation between them, and the fewest big wins and bad losses to point at, and the most to lose from an argument settled by feel.</p><p>So here’s the football version of a team sheet, how it’s built, and a backtest: twelve years of committees, to see how well a number can actually order football resumes. It turns out a number can do a lot, and it also turns out the 24th spot is a harder place to be than the 12th.</p><h3>What Basketball Does</h3><p>The basketball tool is worth describing, because it’s the template. In 2018–19 the NCAA replaced the old RPI with a new rating called <a href="https://www.cbssports.com/college-basketball/news/the-ncaa-tournament-committee-will-select-and-seed-teams-using-a-new-ranking-system-after-killing-the-rpi">NET</a> and kept its <strong>quadrant </strong>system: every game is filed into one of four tiers by the opponent’s rank and where the game was played (originally a road game at the 75th-best team or better, a neutral-site game against the 50th or better, or a home game against the top 30 counted as Quadrant 1). A team’s record in each quadrant is printed on its sheet. The sheet also carries other people’s ratings (KenPom, Sagarin, KPI, ESPN’s Strength of Record, BPI, and more recently Torvik), and, according to <a href="https://www.ncaa.org/news/2025/3/5/breaking-down-the-ncaa-division-i-mens-and-womens-basketball-committees-selection-criteria.aspx">the NCAA’s own explainer</a>, <strong>Wins Above Bubble</strong> was first used by the committee in 2024–25. <a href="https://new.cbssports.com/college-basketball/news/wins-above-bubble-ncaa-tournament-selection-committee-seeding/">Reporting at the time</a> quoted the NCAA’s Dan Gavitt saying at-large selections had been more closely tied to WAB than to NET.</p><p>Two things matter about that.</p><p>First, the sheet isn’t one number, it’s a <strong>panel</strong>: a record broken down by opponent quality, a couple of resume metrics (what you’ve done) and a couple of predictive metrics (how good you are). The committee is expected to look at the difference.</p><p>Second, the sheet is <em>standardized</em>. Every bubble team gets the same one. So the argument becomes about evidence, not about whose highlight reel you saw.</p><p>Football has no equivalent. The committee publishes a ranking, and the criteria it says it weighs (conference championships, strength of schedule, head-to-head, common opponents) are applied by judgment. That’s the gap I tried to close with the <a href="https://poll-position.robert-mepham1309.workers.dev/#/resume">Resume page in Poll Position</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dhwX8_8w7V_AuvoCp_pvhQ.png" /></figure><h3>How the Football Resume is Built</h3><p>The page runs in four steps.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JDqr1VyyZgGC-o88nQh8XQ.png" /></figure><p>1. <strong>Rate every team.</strong> A resume asks what a team has <em>done</em>, given who it played, so the first thing you need is a good measure of the opponents. The rating <strong>R</strong> is expressed in points per game versus an average FBS team: 17.85 points per 1.0 of opponent-adjusted net EPA per play, plus 0.70 times a margin-based rating (margins capped at 28 points). It’s rebuilt every week from data that exists at that point, so nothing in the past is flattered by information from the future.</p><p>I tested that rating the honest way: predict the next window’s games using only the ratings after the previous one, season by season, leaving out the season being predicted. Across 6,326 FBS-versus-FBS games it picks 70.4% of winners with a log loss of 0.557. The Las Vegas closing line, on the 5,829 games where I have it, scores 0.519 and picks 74.0%. <strong>So the rating is not better than the market, and I don’t claim it is</strong>. A resume doesn’t need to forecast; it needs opponent strengths good enough to tell a hard schedule from a soft one, which is a far lower bar.</p><p>2. <strong>Price every game. </strong>For each game, the model asks how likely a <em>reference team</em> would have been to win it: the normal CDF of (the reference team’s rating, minus the opponent’s, plus 2.4 points if it’s at home, divided by 15.4, the typical scatter of game margins).</p><p>3. <strong>Add it up.</strong> Two reference teams, two numbers:</p><ul><li><strong>Strength of Record (SOR)</strong> uses an <em>average top-25 team</em> as the reference. It’s the probability that such a team would have a record at least as good as yours against the same schedule, computed exactly from the schedule’s win probabilities. Lower is better. Miami’s 8–0 comes out at 8.2%: a typical top-25 team goes 8–0 against that schedule about once in twelve tries.</li><li><strong>Wins Above Bubble (WAB)</strong> uses a <em>bubble team</em>: one that sits exactly on the cut line, between the 12th and 13th teams by rating. WAB is your actual wins minus the wins that bubble team would be expected to have against your schedule. A +1.0 means you won a game more than a marginal playoff team would have.</li></ul><p>4. <strong>Score and rank.</strong> The resume score is the average of the WAB and SOR scores (each standardized, SOR on a log scale, because it’s wildly skewed), and the resume rank sorts by it. Beside it the page shows the things a basketball team sheet would: schedule strength (average opponent rating), quality wins (wins over teams rated in the top 25) and bad losses (losses to teams outside the top 60).</p><p>Here is the whole thing for one team, game by game. It’s Tennessee, 6–2, at Week 10 (my team, so forgive the choice… I’m not perfect).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qkISo_0Chked_lUYOwhcvg.png" /></figure><p>A bubble team is expected to win 5.79 of those eight games. Tennessee won six. That’s <strong>+0.21 wins above bubble,</strong> a resume rank of 22, and a good illustration of what the number does: it doesn’t care that the win over Alabama was a nice win (a bubble team wins that one 42% of the time, so it’s worth a lot), it cares that the two losses came in games a bubble team would have won 56% of the time each. The Strength of Record, 67%, says a typical top-25 team does this well or better two-thirds of the time. Tennessee’s resume is, at Week 10, a little below the standard for a top-25 team.</p><h3>Backtest 1: Does a Number Order Resumes Like a Committee?</h3><p>The committee isn’t a formula, but it’s the only referee we have. For every season from 2014 to 2025, at the last poll before the bowls (all conference championship games played), I ordered every team on one measure at a time and asked how much of the committee’s final top 12 and top 4 it found.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*O_3meJ_6Ki1zwmXx7v0O_w.png" /></figure><ul><li><strong>The resume rank finds 84% of the committee’s top 12 and 83% of its top 4.</strong> On average it misses about 1.9 teams out of 12 each year.</li><li><strong>Wins above bubble alone does exactly as well</strong> (84% and 85%), and so does Strength of Record (83% and 83%). The three are close to the same information. That’s good to know: the composite score isn’t adding magic; WAB is the engine.</li><li><strong>Team rating alone finds 80% of the top 12 but only 63% of the top 4.</strong> At the very top, the committee isn’t just picking the best teams; it’s picking the best <em>resumes</em>.</li><li><strong>Schedule strength alone is nearly useless as an order</strong> (41% and 19%). It’s context for a resume, not a measure of one.</li><li><strong>The AP poll finds 94% of the top 12 the week before and 97% the same day.</strong></li></ul><p>That last one deserves a pause. The committee is nearly the AP poll in a trench coat. When I looked at the 78 team-seasons where the resume and the poll disagree by five spots or more, the committee’s final rank was closer to the poll’s in <strong>63 </strong>of them and closer to the resume’s in <strong>12</strong>. (Against the same-day poll it’s 65 of 65, which tells you how much the poll and the committee read each other.)</p><p>I want to be clear about what this does and doesn’t mean. It doesn’t mean the resume is wrong. It means the committee <em>is following something other than a resume</em>, which is useful to know when you read the resume page: <strong>where it differs from the poll, the resume is telling you what the committee will probably not do, not what it’s going to do.</strong></p><p>Does it help to add more ingredients? I tested this out of sample: fit a model to the committee’s ordering on all seasons but one, predict that one, and add ingredients in turn.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/392/1*ceOjm1calMbYLAKWVW6a0w.png" /></figure><p>The differences in the top-12 column are a couple of teams across twelve seasons: noise. The one that looks real is adding team rating, which lifts the top-4 hit rate from 85% to 92%, or about three more of the 48 teams. WAB does the heavy lifting; quality wins and bad losses add nothing the committee isn’t already getting from WAB.</p><h3>Backtest 2: Does a Resume Predict Anything?</h3><p>A resume is backward-looking by design. It would be foolish to pretend it forecasts, so I checked how much it does. For every postseason game between two FBS teams from 2014 to 2025 (excluding 2020), I picked the winner by each measure.</p><ul><li>The better <strong>resume rank</strong> picked 57% of winners (260 of 454 games).</li><li>The higher <strong>team rating</strong> picked 56% (254 of 454).</li><li>The <strong>AP poll</strong> (same day, games where both teams were ranked) picked 58% (110 of 191).</li><li><strong>Schedule strength</strong> alone picked 52%, which is a coin flip.</li></ul><p>Everything is a little better than a coin flip and nothing is much better than anything else. The bowl season is mostly noise (opt-outs, motivation, who showed up). But it matters for how you read the page. <strong>“Resume” and “rating” are different questions:</strong> one is who has earned it, the other is who is better. They disagree most when it matters. At Week 10, LSU is the top-rated team in the country and has the 17th-best resume. Kentucky is the opposite: 4th on the resume and 25th on rating.</p><h3>The 24-Team Cut Line</h3><p>This is the part I most wanted to look at. A 12-team field puts the cut line between the 12th and 13th teams. A 24-team field puts it between the 24th and 25th. Does the same ruler work down there?</p><p>I ran the same committee comparison in two zones: pairs of teams the committee ranked 8th to 16th (around the 12-team cut), and pairs it ranked 17th to 25th (around a 24-team cut; its ballot stops at 25).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*riVvzTCgi-gIh6jvL0wupg.png" /></figure><p><strong>Everything gets worse further down the ballot.</strong> On a pair of teams, the resume rank agrees with the committee 72% of the time near the 12-team cut and 63% near the 24-team cut. The AP poll drops from 82% to 75%. Team rating drops from 64% to 61%. Nothing, including the committee’s own favorite source, is as sure of itself at the bottom.</p><p>The bubble also gets crowded. Measure how many teams sit within three-quarters of a win above bubble of the cut. On average over these twelve seasons that is <strong>13.3 teams at the 12-team line and 18.8 at a 24-team line</strong>, and the average gap in wins above bubble between neighbors near the cut shrinks from 0.11 per rank to 0.08. More teams, closer together.</p><p>There’s a quirk in the math that makes the point for you. The “bubble team” in WAB is defined by where the cut line falls: it’s the team between the 12th and 13th by rating. If the field were 24 teams, the bubble team would be the one between the 24th and 25th. In 2026 the 12th/13th team is rated 12.4 points better than an average FBS team. The 24th/25th team is rated 10.3 points better. <strong>The difference between the 12th-best team in college football and the 24th-best is about two points of margin per game, less than a home-field advantage.</strong> Switching the bubble to 24 teams doesn’t change the order of the resumes (the rank correlation between the two versions is 1.000), but it moves the meaning of the cut: the cut line goes from +0.93 wins above bubble to +0.32, and the number of teams with a positive WAB goes from 24 to 30.</p><p>Here’s the whole team sheet, with both cut lines drawn in.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VXqDSxtcqVtYygGn8ufHpw.png" /></figure><p>At the 12-team line, the resumes of #12 Boise State (+0.97 wins above bubble) and #13 Alabama (+0.88) are almost the same. At the 24-team line it’s worse: Iowa (+0.01, one bad loss) is in, Oregon (-0.06, 5–3, with the 6th-toughest schedule and three quality wins) is out, and behind them are New Mexico (7–1, but the 137th-ranked schedule), Northwestern and Arizona. Nine teams, from Liberty at +0.39 to Arizona at -0.06, are within about 0.45 of a win of each other. Whoever the committee picks at 24 will be defended by a number that is within a rounding error of the number for the team it left out.</p><p>And this is where the <strong>AP-versus-resume gap</strong> earns its keep. The page’s biggest gaps right now: <strong>Boise State</strong> is 20th in the poll and 12th on the resume; <strong>Texas Tech</strong> is 13th and 8th; <strong>Wake Forest </strong>is 14th and 9th; <strong>Kentucky </strong>is 8th and 4th. In the other direction, <strong>Georgia </strong>is 2nd in the poll and 10th on the resume, <strong>Ohio State</strong> is 11th and 19th, <strong>Tennessee </strong>is 16th and 22nd, and <strong>South Carolina</strong> is 23rd and 32nd. UTSA (#15) and Liberty (#20) have top-25 resumes and no AP rank at all.</p><p>My own team is a clean example. Tennessee at #22 is ten spots outside the 12-team field and two spots inside a 24-team one: the same resume goes from a long shot to a team with a small cushion, which is what a doubled field does.</p><h3>What This Can’t See</h3><p>A fair accounting:</p><ul><li><strong>It can’t see injuries, quarterbacks, or how a team looked.</strong> The committee can, and says it does.</li><li><strong>Wins above bubble and Strength of Record count only whether you won,</strong> and the rating under them uses margin only through the ratings of the teams involved (it’s in the rating of the opponent and your own, capped at 28 points). A 3-point win and a 30-point win are the same result in the resume.</li><li><strong>It’s a model of opponent strength.</strong> If those ratings are off for a group of teams (my own earlier work found they flatter Group of Six teams by about 2.5 points of margin), those teams’ resumes move too.</li><li><strong>The committee’s ballot stops at 25.</strong> My comparison for the 24-team cut line can only look at pairs the committee ranks through 25th, so the zone I measured is the near side of the line, not both sides.</li><li><strong>Twelve seasons and a lot of noise at the bottom.</strong> The numbers above are averages: any single season will look different.</li></ul><p>And the 24-team format itself isn’t official. The <a href="https://www.wlox.com/2026/10/15/college-football-playoff-nearing-deal-expand-24-teams-2027-reports-say/">reports</a> from mid-October say the deal is close and describe four Power 4 champions, the best Group of Six team and 19 at-large teams, but the playoff hasn’t confirmed it, and the 22nd-to-26th figure is my reading of what a 24-team cut would be, not an official one.</p><h3>So What’s the Case for a Football Team Sheet?</h3><p>That a standardized sheet, even an imperfect one, turns arguments about vibes into arguments about evidence, and that the sheet is what you want when the evidence is thinnest. The Resume page reproduces about 84% of what the committee does from the top 12, and it’s honest about the 16% it doesn’t. It gives you a number for the 22nd-best resume. It tells you which way the poll is leaning against it. It can’t tell you who will win, only who has earned it.</p><p>Pull up any two bubble teams on the Resume page and you get a team sheet, a game-by-game breakdown, and a share card you can post. That’s more than the committee gives you, and it costs nothing.</p><p><em>Open the Resume page at </em><a href="https://poll-position.robert-mepham1309.workers.dev/#/resume"><em>Poll Positon | Resume</em></a><em>: click any team for its game-by-game sheet, and compare two teams side by side on the Playoff page’s Bubble tab. The model behind the playoff odds is explained in this article, the committee’s pattern of discounting Group of Six teams in this one, and the 24-team picture in this one.</em></p><blockquote><strong>Methodology Used in the Above Analysis</strong></blockquote><blockquote><strong>Rating R:</strong> 17.85 × adjusted net EPA per play + 0.698 × margin rating (ridge least squares, margins capped at 28, centered on FBS), in points per game vs. an average FBS team; rebuilt each week from data available then.</blockquote><blockquote><strong>Calibration (2015–2025, leave-one-season-out, 6,326 FBS-vs-FBS games):</strong> log loss 0.5573, Brier 0.1893, 70.4% picks right; closing line on 5,829 games: 0.5189 and 74.0%</blockquote><blockquote><strong>Game price:</strong> P(reference team wins) = Φ((R_ref − R_opp + 2.41·site) / 15.39); site = +1 home, 0 neutral, −1 road.</blockquote><blockquote><strong>SOR:</strong> the exact probability (Poisson-binomial over the schedule so far) that an average top-25-rated team wins at least as many games.</blockquote><blockquote><strong>WAB:</strong> wins − Σ P(bubble team wins), bubble = a team midway between the field-size-th and next-best rating (12th/13th for the page; 24th/25th for the 24-team version).</blockquote><blockquote><strong>Resume score</strong> = mean of the WAB z-score and the z-score of −log SOR; resume rank sorts by it.</blockquote><blockquote><strong>Quality wins:</strong> wins over teams rated in the top 25; bad losses: losses to teams outside the top 60.</blockquote><blockquote><strong>Committee comparison:</strong> 2014–2025, the last poll before the bowls; the committee’s final top 25 (data/cfp.json). Top-12 and top-4 recovery: share of the committee’s top 12 / top 4 inside a measure’s top 12 / top 4, over the committee’s top 25 and the resume’s top 40. Count measures (quality wins, bad losses, win percentage) tie too often to order a ballot and are shown by pairwise agreement only.</blockquote><blockquote><strong>Pairwise agreement:</strong> over pairs among the committee’s ranks 8–16 and 17–25 (432 pairs per season group over 12 seasons), the share ordered the same way by the measure (ties skipped).</blockquote><blockquote><strong>AP vs. resume:</strong> 78 team-seasons ranked by the committee where the AP poll of the week before and the resume rank differ by 5+ spots; committee rank closer to the AP: 63; closer to the resume: 12; 3 ties. Same-day poll: 65 of 65.</blockquote><blockquote><strong>Ablation:</strong> pairwise logistic regression of the committee’s order (standardized features), fitted leave-one-season-out over 2014–2025; features added in the order shown.</blockquote><blockquote><strong>Postseason picks:</strong> all postseason games between two FBS teams, 2014–2025 excluding 2020 (454 games; the AP column 191 games where both teams were ranked); one-sided binomial p-values vs. a coin flip: resume rank 0.001, rating 0.006, AP 0.021, schedule strength 0.19.</blockquote><blockquote><strong>24-team cut:</strong> WAB recomputed with the bubble at the 24th/25th team. 2026 Week 10: bubble rating 12.4 → 10.3 points; WAB cut line +0.93 → +0.32; teams with positive WAB 24 → 30; rank correlation of the two WAB versions 1.000. Crowding: teams within 0.75 WAB of the respective cut, averaged over 12 seasons (13.3 vs 18.8); neighbor gap = (WAB at rank cut−3 minus rank cut+3) ÷ 6 (0.11 vs 0.08).</blockquote><blockquote><strong>Limits:</strong> the committee ranks 25; twelve seasons; resumes do not see injuries, game control or margin beyond the rating; the 24-team format is a reported deal.</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a1cf6b1ed7ec" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why the Internet Dragged One of the Best AP Voters Through the Mud, and What It Says About Women on…]]></title>
            <link>https://medium.com/@robert.mepham/why-the-internet-dragged-one-of-the-best-ap-voters-through-the-mud-and-what-it-says-about-women-on-a523b9699bee?source=rss-745e2a58a3bf------2</link>
            <guid isPermaLink="false">https://medium.com/p/a523b9699bee</guid>
            <category><![CDATA[college-football]]></category>
            <category><![CDATA[ap-top-25]]></category>
            <category><![CDATA[women-in-sports]]></category>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[media-bias]]></category>
            <dc:creator><![CDATA[Rob Mepham]]></dc:creator>
            <pubDate>Mon, 16 Nov 2026 14:56:01 GMT</pubDate>
            <atom:updated>2026-11-16T14:56:01.674Z</atom:updated>
            <content:encoded><![CDATA[<h3>Why the Internet Dragged One of the Best AP Voters Through the Mud, and What It Says About Women on the Poll</h3><p>On September 6, 2025, South Florida went into The Swamp and beat Florida 18–16. The next day, 63 AP voters who had ranked the Gators the week before had to decide what that meant. Sixty-two of them dropped Florida, by a median of 11 spots. Haley Sawyer moved them up two, from 16th to 14th, and left South Florida unranked.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0AkiAwWiasBzi5ymsLnKsw.png" /></figure><p>That was a bad ballot. I’ll say it first and plainly, because everything I’m about to argue depends on not pretending otherwise. It was the most out-of-step thing she did all season.</p><p>What happened next is the part I want to talk about, because it is not about the ballot.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KVocgYDnYAjxhJFE7tPHgw.png" /></figure><h3>The Pile-On</h3><p>Within two days there were articles with headlines like “Calls Mount to Disqualify AP Voter” and “AP Top 25 Voter Breaks Silence On ‘Embarrassing’ Rankings Decision.” The posts read like this:</p><blockquote><em>“</em>Pull her right to vote. Immediately<em>” — @UtahFBFans, Sept. 9, 2025</em></blockquote><blockquote><em>“</em>Disqualifying. Take her vote immediately.<em>” — @Andrew_Moser_</em></blockquote><blockquote><em>“</em>She’s directly impacting the lives of hundreds of players, coaches, and staff at the schools she’s choosing to wrongfully leave out of her ballot… it DOES matter<em>” — @SentiveJ, Sept. 9, 2025</em></blockquote><p>A fellow AP voter, Brett McMurphy, said on a podcast that “somebody should lose their vote over this.” One outlet put it in the headline that a <em>“Female College Football Reporter”</em> should be “stripped of her AP Top 25 voting privileges,” under the line “Everyone Wants” it.</p><p>Look at what is being asked. Not “she got that one wrong.” Not “she should explain it.” The ask is <strong>take her vote, remove her from the poll,</strong> and for a reason the posts don’t dispute: she gave an answer on a podcast that people didn’t like. When pressed, she said she didn’t want to get into her process, that “it’s not a perfect system,” that it evens out because so many people vote, and that “it doesn’t probably matter in the end, but I’m glad that people are having a really good time with it.” That was a poor answer, I agree. It was also an answer from someone who, in the same breath, said she wakes up early after covering a game late the night before. She had a rough Sunday. She shrugged it off. People decided she was the villain of the season.</p><h3>What the rest of her ballots look like</h3><p>Now I decided to do what the mob didn’t: actually look at her voting history. Yes, all of them. I track every AP ballot in my app (<a href="https://poll-position.robert-mepham1309.workers.dev/">Poll Position</a>) and compared hers with the other voters’ for 2025 and for 2026 through Week 8.</p><p><strong>The Florida ballot was her only miss of its size all year.</strong> I counted every time a voter placed one team 12 or more spots from where the poll had it (unranked counts as 26). Over 15 regular polls, Sawyer did it once. Fifteen voters did it more often than she did. One voter did it eleven times. There were 81 such misses across the panel and she made one.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vgcCk0iPiMFSuiYNPhK3Bg.png" /></figure><p><strong>Her worst week of the year was an ordinary week for the rest of the panel.</strong> Measured as the average number of spots a ballot sits from the poll, her worst week in 2025 was 2.4. About 18% of all ballots cast that season were further from the poll than that. Over the full season, 49 of 66 voters averaged a bigger gap than she did; the app ranks her 16th of 67 for closeness to the consensus.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZQxrjFr32eTCwP5d20G9kw.png" /></figure><p><strong>And then she got good.</strong> I’ll give you the honest first half: from Week 2 to Week 9 she was below average (30th of 66 on closeness to the poll, 39th on closeness to the <em>next</em> poll, 52nd on closeness to a results-only ranking). The Florida ballot lives in that stretch. But from Week 10 on, her ballots tightened, and from Week 11 to the end of the regular season she was in the closest quarter of the panel every single week. Over the back half of 2025 she was 7th of 66 on closeness to the poll and 2nd of 66 on closeness to the <em>next</em> poll, which is the closest thing to “saw it coming” that one ballot can show.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9Wt8Qt_7qg7EbqNQ5EvGqg.png" /></figure><p>In 2026, through seven polls, she’s 3rd of 69 on closeness to the poll and 8th of 69 on closeness to the next one. Sixty-six voters have a bigger average gap than she does. She is not perfect: her Michigan ballot in Week 2 (17th, with the poll leaving them out after a 13–12 win over Western Michigan) was the highest of any voter, and it got the pile-on going again. On a measure that checks ballots against all the results so far, she’s 41st of 69, ordinary. But on “is this person paying attention to where the poll is going,” she’s near the top of the panel this year, and was in the back half of last year.</p><p>So sure, she messed up. But my contention here is that the data actually backs up the idea that she likely realized the error of her methods, and has since been one of the <em>best</em> voters the poll has had the last two seasons.</p><h3>Now Let’s Compare the Reaction to the Record</h3><p>Here is the shape of the thing. One ballot, one shrug, and a year later it follows her. When she put Michigan 17th in 2026, the coverage said “<em>alone, this ballot might’ve gone relatively unnoticed,</em>” but “<em>when combined with a similar situation from 2025, the optics are bad.</em>” Another line from the same piece: “<em>A very loud contingent of viewers does not believe Haley Sawyer cares about football.</em>”</p><p>Read that again. Not “has a ballot some people disagree with.” <em>Does not care about football.</em> A verdict about her character, based on her two worst ballots out of the two dozen or so polls she has voted in since.</p><p>Other voters were also dragged that September. Two of them posted apologies within a day or two (“I own it and I screwed up” is one) and the story moved on. I don’t think the difference is only that she didn’t apologize. A voter who makes an error is a voter who made an error. A voter who made an error and then, people decide, <em>doesn’t care</em>, is a voter who has to be removed, and a year later is still removed in the comments.</p><h3>Who Else Made Florida-sized Misses, and Who Heard About it</h3><p>Here is the test I think the argument needs. Fifteen of the 66 voters with a full 2025 season made more 12-spot misses than Sawyer did. I took their names from the ballot data and went looking, for September 2025 through March 2026, for any article whose subject was that voter’s ballot and whose tone was critical.</p><p>The result is lopsided:</p><ul><li><strong>Haley Sawyer, one big miss:</strong> at least eight dedicated critical pieces (Athlon, The Spun/Yahoo, Pro Football Network, TotalProSports, SI, two from Awful Announcing, CBS Sports), plus a fellow voter saying someone should lose their vote.</li><li><strong>Koki Riley, seven big misses and the biggest average gap of the fifteen:</strong> three dedicated pieces (Pro Football Network, Awful Announcing, SI), plus a Barstool post calling him the worst voter ever.</li><li><strong>Kirk Kenney, six big misses:</strong> one piece I could find, from a Tennessee-focused site.</li><li><strong>The other 13 voters who made more big misses than she did, including the voter who made eleven:</strong> none. Two of them appear only as a name in a list of voters who left Tennessee off their ballots.</li><li><strong>For comparison, voters with a record like hers </strong>(Brenna Greene, one big miss; Ian Kress, none) each got one roundup item in SI and posted an apology.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WMtTmMCkklj6cu3M4SzVUw.png" /></figure><p>So the attention that Haley’s ballot garnered did not in any way reflect her “skill” as an AP voter, and actually turned into what can really only be described as a runaway social media rabbit hole that brought out a pseudo mob-mentality in folks which led to some rather unfortunate statements being made without seeing the whole picture.</p><h3>What I Think This Ultimately Is</h3><p>Anyone who knows me is going to know that this is way out of left field for how I normally adress discrepencies between societal reactions to phenomena and the data that underpin said phenomena, but I think it’s about Haley’s gender, or a large part of it is, and I’m not going to hedge that.</p><p>I can’t show you a single post that says “women shouldn’t vote” as a blanket statement, and I’m not going to pretend I found one or that some of the naysayers had that malicious intent behind their individual outrage over Haley’s bad 2025 ballot. A man with far worse ballots got dragged too. What I can show you is the pattern that anyone who has watched women in sports media for even a week will recognize. The default assumption is that a woman covering football is there for the wrong reasons and doesn’t really know the sport. A mistake from her is proof of it. A mistake from anyone else is a mistake. The headline that announces “Female College Football Reporter” as the story, in a headline about a ballot, is that default assumption in plain print. The viral clip is a woman saying it doesn’t matter and the reaction is “she’s not serious about her job,” not “she’s wrong and here’s why.”</p><p>Ask the question the numbers above raise: would a voter with her record, the same one-in-fifteen miss and the same strong back half, have been told to give up the vote and kept hearing it for over a year at this point? Fifteen voters had more extreme gaffes in their ballots than she did, and none of them saw the same firestorm from the national media. From Week 11 on she was closer to the poll than three-quarters of the panel every week, and has continued that strength into the 2026 season as a top-3 pannelist, yet she still gets her name dragged through the mud despite her objectively strong performance. The numbers describe a decent voter with one very public bad day. The reaction describes something else, and the thing it describes isn’t in the ballots.</p><h3>What I’m Really Asking For</h3><p>Hold voters to the same standard. If a ballot that is 12 spots off the poll gets you removed, it should get <em>everyone</em> removed, and the voter who did it eleven times should have heard about it long ago. If it doesn’t, then the standard was never the ballot, it was a gripe that sits a lot deeper down in the psyche of people that get on social media to complain about whatever tickles their fancy that day.</p><p>Haley Sawyer deserved some criticism for one week in September of ‘25. She did not deserve to be the face of everything wrong with the AP poll. The ballots since say she took it seriously and revised her methodology, and the people who spent a year calling her unserious and calling for her job never looked at them once.</p><p><em>Look up any voter’s ballots and behavioral trends, week by week, next to the poll at </em><a href="https://poll-position.robert-mepham1309.workers.dev/"><em>Poll Position</em></a><em>.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ne30p6f-gP7HHew4e20ZYA.png" /></figure><h3>Method, Briefly</h3><p>Ballots are the AP’s published ballots: 2025 (15 regular polls plus the final) and 2026 through Week 8. “Distance from the poll” is the average number of spots a ballot is from that week’s poll per team. “Next poll” is the same measure against the following week’s poll. “Results” compares a ballot to a results-only ranking (a Massey-style rating on all games to date) relative to the other voters that week. Ranks are among voters with at least five polls in the period. The 2025 split at Week 9/10 is a judgment call made where the pattern changes; a different split would move the numbers but not the picture.</p><h3>Sources for the Quotes</h3><ul><li>Athlon Sports, Sept. 9, 2025: <a href="https://athlonsports.com/college-football/ap-voter-issues-response-as-backlash-over-college-football-rankings-intensifies">https://athlonsports.com/college-football/ap-voter-issues-response-as-backlash-over-college-football-rankings-intensifies</a></li><li>Pro Football Network, Sept. 9, 2025: <a href="https://www.profootballnetwork.com/cfb/calls-mount-disqualify-ap-voter-as-decision-to-move-florida-up-following-usf-loss-draws-wrath/">https://www.profootballnetwork.com/cfb/calls-mount-disqualify-ap-voter-as-decision-to-move-florida-up-following-usf-loss-draws-wrath/</a></li><li>TotalProSports, Sept. 11, 2025: <a href="https://www.totalprosports.com/ncaa/female-college-football-reporter-stripped-ap-top-25-voting-ridiculous-ranking/">https://www.totalprosports.com/ncaa/female-college-football-reporter-stripped-ap-top-25-voting-ridiculous-ranking/</a></li><li>Awful Announcing, Sept. 9 and 11, 2025: <a href="https://awfulannouncing.com/college-football/ap-poll-florida-usf.html">https://awfulannouncing.com/college-football/ap-poll-florida-usf.html</a> and <a href="https://awfulannouncing.com/college-football/brett-mcmurphy-blasts-ap-voter-somebody-should-lose-vote.html">https://awfulannouncing.com/college-football/brett-mcmurphy-blasts-ap-voter-somebody-should-lose-vote.html</a></li><li>SI, Sept. 15, 2025 (other voters’ apologies): <a href="https://www.si.com/fannation/college/cfb-hq/news/three-more-ap-voters-called-out-for-week-4-top-25-poll">https://www.si.com/fannation/college/cfb-hq/news/three-more-ap-voters-called-out-for-week-4-top-25-poll</a></li><li>BroBible, Sept. 2026 (Michigan ballot): <a href="https://brobible.com/sports/article/ap-poll-voter-haley-sawyer-michigan-football-ranking-controversy/">https://brobible.com/sports/article/ap-poll-voter-haley-sawyer-michigan-football-ranking-controversy/</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a523b9699bee" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I Know Who’s Going to Win The College Football Playoff This Year… And It’s Thanks to Economics]]></title>
            <link>https://medium.com/@robert.mepham/i-know-whos-going-to-win-the-college-football-playoff-this-year-and-it-s-thanks-to-economics-5f4b97db6aab?source=rss-745e2a58a3bf------2</link>
            <guid isPermaLink="false">https://medium.com/p/5f4b97db6aab</guid>
            <category><![CDATA[pareto-principle]]></category>
            <category><![CDATA[college-football-playoffs]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[college-football]]></category>
            <category><![CDATA[economics]]></category>
            <dc:creator><![CDATA[Rob Mepham]]></dc:creator>
            <pubDate>Sat, 14 Nov 2026 15:01:04 GMT</pubDate>
            <atom:updated>2026-11-14T15:01:04.148Z</atom:updated>
            <content:encoded><![CDATA[<p>Every year, around the time the committee releases its first rankings, I notice the same thing: the teams that end up winning the national championship almost never have a weak side.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Wcb_cadBwmtSQ2QC2ZhMzQ.png" /></figure><p>That’s a hunch, and hunches are cheap. In the last four seasons I could see the pattern in the data: plot every team by how good its offense is against how good its defense is, and the champion sits out in the top-right corner, on the boundary of what anybody has done. I wondered whether that was a four-year coincidence. So I did the thing that makes a hunch into a result: I tried to break it by going back as far as the data would go.</p><p>It goes back to 2007. And the pattern held for 16 of the 19 champions I could test. Here’s how I tested it, what it does and doesn’t prove, and what it says about the 2026 field at Week 10, including a few places where I think the pattern is saying something different from what you might expect.</p><h3>An Idea From Economics</h3><p>Imagine a factory that makes two goods with a fixed pile of inputs. Some combinations of the two goods are possible and some aren’t. The boundary between them, the combinations you can reach and no more, is called the <strong>production possibility frontier,</strong> and a factory sitting on it is <strong>productively efficient:</strong> it can’t make more of one good without making less of the other. A factory inside the line is leaving something on the table.</p><p>Economists have a century of tools for this. The principle is Pareto’s: a point is efficient if nothing else beats it on every dimension at once. Koopmans (1951) and Debreu (1951) turned it into the modern idea of an efficient production plan, and Farrell (1957) made it measurable: take a set of real, observed producers, and say that the efficient ones are the ones nobody else beats on every output. That is the “frontier” you get from data without assuming any equation for it. Charnes, Cooper and Rhodes (1978) gave the method its famous name,<strong> data envelopment analysis</strong>, and Deprins, Simar and Tulkens (1984) its non-convex cousin, the <strong>free disposal hull</strong>, which is exactly what I use: the efficient teams are the ones that no other team beats on both outputs.</p><p>Football fits this almost too neatly. A team produces two things: offense (expected points added per play on offense) and defense (expected points <em>prevented</em> per play). Those are two outputs. A team is efficient if no other team is better at both, and the frontier is the set of teams nobody beats on both. The distance of an inefficient team from the line has a name in the economics literature too, the <strong>directional distance function</strong> (Chambers, Chung and Färe, 1996). I call it <strong>slack</strong>: how much EPA per play a team would have to add to <em>both</em> its offense and its defense before nobody beats it on both. A team on the frontier has zero slack.</p><p>Why would champions live on it? Single elimination. A champion has to beat three or four opponents, each one good enough to have gotten there. A team with an elite offense and a shaky defense can beat the world on a good day and lose to a good team on a bad one, and the bracket doesn’t give you a replay. That’s an explanation, not a test, and I’ll try to test it directly below.</p><h3>The Test</h3><p><strong>The measurement.</strong> EPA per play measures how much each play changes the expected points on the scoreboard. I use the version the Poll Position Ratings page uses: offense and defense separately, garbage time excluded, and <strong>opponent-adjusted</strong> (a team that piles up EPA against a soft schedule gets marked down; one that does it against a hard one gets marked up, iterated like the Massey ratings, with some shrinkage toward the league mean for teams with few games). I measure it at the end of the regular season, after the conference title games and before the bowls and playoff. That’s the last moment the committee, and you, would have to pick a champion from.</p><p><strong>The data.</strong> The app has EPA for 2014 through 2025. For this article I also pulled 2007 through 2013 from CollegeFootballData, the BCS era, which gives me 19 seasons and 19 champions. Earlier than 2007 the play-by-play coverage thins out (a typical team in 2005 has about 610 plays, against about 700 in 2007 and later), so I stopped there. A word on that: the idea came from watching the recent seasons, so those seven earlier seasons are the closest thing to a clean test. I formed the hypothesis without looking at them.</p><p><strong>The result.</strong> In all seven BCS-era seasons, the champion was exactly on the frontier: LSU 2007, Florida 2008, Alabama 2009, Auburn 2010, Alabama 2011, Alabama 2012, Florida State 2013. In the playoff era it was 9 of 12. The three misses are all small: Ohio State in 2014 (slack 0.055 EPA per play), Clemson in 2016 (0.018) and Alabama in 2017 (0.007). Sixteen of nineteen, 84%, with a frontier that averages 3.5 teams out of about 127 (a bit under 3% of the field).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SuImYzgPi6i81L4bA0R7LA.png" /></figure><p>Here’s what the frontier looks like in four different seasons: the green dotted line connects the teams nobody beats on both sides of the ball, and the champion is ringed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zMM9HNh2QItr7K4DusVJ7w.png" /></figure><p>In 2024 the line connected two teams, Notre Dame and Ohio State, and the national championship game was Ohio State against Notre Dame. In 2018 and 2019 the two finalists were both on it. In 2013 there was only one team on the whole frontier, and it was Florida State.</p><h3>Is That Surprising?</h3><p>Not if you ask a stupid question. If a champion were a random FBS team, the chance of landing on a frontier that holds under 3% of the field in 16 of 19 years is on the order of 10⁻²³. That number is meaningless, because a national champion is never a random team; it’s always one of the best.</p><p>The honest question is whether the frontier tells you anything beyond <strong>“this team is very good.”</strong> Here’s a harder test. Each year, take the five teams with the best net rating (adjusted offense minus adjusted defense) and suppose the champion is picked at random from them. Because the frontier overlaps with those five, you’d expect about 10.6 of the 19 champions to be on it just by being good. The actual number is 16. The chance of that or more under the harder null is<strong> 0.6%</strong>.</p><p>But here’s the more interesting wrinkle, and I want to flag it clearly. As a <em>filter</em>, the frontier is no better than a plain net-rating cut. A cut of “the top k teams by net rating,” where k is the size of that year’s frontier, also catches 17 of 19 champions. The frontier isn’t a better detector of good teams. If it matters, it matters because of the shape: whether a team is good because it’s balanced, or because it’s brilliant at one thing.</p><h3>Does Shape Predict the Big Games?</h3><p>This is the part I care about, because it’s the direct test of the “no weak side” story. Take every College Football Playoff game from 2014 through 2025 between two teams in the field: 51 games. Pick the winner with a rule and see how often it’s right.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vIh6TFVsCCOPHzMQoZ421w.png" /></figure><ul><li><strong>The team with the smaller slack (closer to the frontier) won 31 of 40 games, 78%</strong>. Eleven games were ties (both teams on the frontier) and can’t be called by this rule.</li><li><strong>The team with the higher net rating won 36 of 51, 71%.</strong></li></ul><p>Both beat a coin flip comfortably. The better question is whether slack tells you anything the net rating doesn’t, so I fitted a simple logistic model of who wins, with the net-rating difference and the slack difference as predictors. Adding slack to net rating improves the fit (likelihood-ratio statistic 5.5 on one degree of freedom, p = 0.019). Adding net rating to slack adds essentially nothing (0.04). When the two rules disagree about a game (it only happens three times) slack was right twice.</p><p>A good deal of caution is due here. Fifty-one games is a small sample, many of the same teams appear in several of them, the model’s fit is in-sample, and the slack is measured at the end of the regular season, before the games it is being used to call. The honest summary: it’s suggestive, and it runs in the direction the “no weak side” story predicts.</p><p>What about at Week 10, which is where we are in 2026? Using Week 10 values to call those same games: slack picks 31 of 44 (70%); net rating picks 32 of 51 (63%). A weaker signal, but the same direction.</p><h3>How Early Does the Line Show Up?</h3><p>The end-of-season test is the clean one, but it’s also the least useful. You want to know by the first week of November. So I ran the same tests at Week 10 for every season.</p><p>At Week 10 the champion was on the frontier in <strong>11 of 19 </strong>seasons, and within 0.015 of it in <strong>16 of 19</strong>. The other three were far off, and then closed the gap: Ohio State in 2014 (0.078 at Week 10, 0.055 at the end), Clemson in 2016 (0.088, then 0.018), and LSU in 2019 (0.078, then exactly on it). Here is the pattern I find most striking: <strong>no champion in 19 years was between 0.015 and 0.077 from the line at Week 10.</strong> They were either basically on it or a long way off and surging.</p><p>I should be careful: the claim at Week 10 isn’t that the frontier beats net rating at picking a champion early. It doesn’t. Using the same “pick from the top five by net rating” null as before, the Week 10 frontier does no better than chance (11 champions on the line against 10.6 expected, p = 0.52).</p><p>What does seem to carry information is the <em>distance</em>. Take the 12 teams with the best net rating each year and group them by how far they sat from the line.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LpWb1IhqlortspDEpnfk3A.png" /></figure><p>At the end of the season, <strong>a top-12 team on the frontier won the title 25% of the time</strong> (16 of 63). A top-12 team within 0.02 won 4%, one between 0.02 and 0.05 won <strong>0 of 53</strong> times, and one more than 0.05 away won 1 of 60. At Week 10 the same groups are 16%, 10%, <strong>0 of 61</strong>, and 4%.</p><p>A permutation check says the empty middle band is not an accident: if the champion were picked at random from the top 12 each year, the chance of none of them landing in the 0.02–0.05 band is 0.2% at Week 10 and 0.5% at the end of the season. The honest caveat is that I chose the band boundaries after looking at the data, so treat the p-values as descriptive. Even if I’d tried twenty different cutoffs, those numbers would survive a crude correction, but that’s still not the same as having planned it.</p><h3>The 2026 Picture, Pre-Week 10</h3><p>Now to the part you scrolled down for.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EbX1xGGvbcpXrdoHHs9CLA.png" /></figure><p><strong>On the frontier</strong> (slack zero): Notre Dame, LSU, Miami and UCLA. Indiana sits at 0.0005, close enough to share a ring with LSU on the map. (These are the five teams on the Ratings page’s frontier line.)</p><p><strong>In the champion zone (within 0.015): Tennessee at 0.0065 and Alabama at 0.0135.</strong></p><p><strong>In the gap (0.02 to 0.077, where no champion has stood at Week 10):</strong> Utah 0.022, Texas Tech 0.026, Boise State 0.029, Ohio State 0.031, Wake Forest 0.033, BYU 0.034, Georgia 0.040, Ole Miss 0.043, Texas 0.048, and Kentucky 0.063.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xk5ilEdYULNB77zqGFC6KA.png" /></figure><p>I was asked, essentially, about six teams: Tennessee, Alabama, Utah, BYU, Ohio State and Georgia, with the instinct that they’re all a little off the frontier and so will falter in the big games. The data says something more careful, and I think it’s more interesting:</p><ul><li><strong>Tennessee and Alabama aren’t off the frontier in any way that history penalizes.</strong> They’re 0.0065 and 0.0135 away, inside the zone where 16 of the 19 champions stood at this point of the year. Tennessee’s gap to the line is nearly all on defense: UCLA beats it by just 0.006 on offense but 0.034 on defense. Alabama is chasing LSU, and the shortfall is 0.020 on offense and 0.014 on defense.</li><li><strong>Utah, Ohio State, BYU and Georgia are in the gap. </strong>None of the 19 champions was in that range at Week 10. Utah is 0.022 short of Notre Dame on offense and 0.027 on defense. Ohio State is behind LSU by 0.054 on offense and 0.031 on defense. BYU’s gap to Notre Dame is 0.066 on offense and 0.034 on defense. Georgia, ranked 2nd in the AP poll, is 0.040 and 0.045 short of Notre Dame. (Kentucky, 8th in the AP poll, is 0.063 off.)</li><li><strong>But the gap is a history, not a verdict.</strong> Three champions began Week 10 far from the line and closed it. Of the 67 team-seasons that finished on the frontier, 62 (93%) had been within 0.03 at Week 10, so a team 0.02 off has a real path to the line. A team 0.04 off has one that has historically been narrower.</li></ul><p>I’d also keep this in proportion. The frontier is not an input to the playoff model from the earlier article, and it measures how a team has <em>played</em>, not whether it makes the field. Tennessee is a champion-shaped team on this metric and still only 23% to make the 12-team field, with a title chance of about 1%, because it has two losses and a resume that currently ranks 22nd. The frontier says what a team looks like when it gets there. It says nothing about getting there. Article 14 has the odds.</p><h3>What this does and doesn’t show</h3><ul><li><strong>It is distance, not probability. </strong>The frontier says which teams <em>look like</em> past champions. It doesn’t say how likely a team is to win. For that you need the simulation.</li><li><strong>It’s 19 seasons. </strong>That’s a lot of EPA and not many championships. The three misses and the three surges are the whole story of the exceptions.</li><li><strong>The method has a known weakness.</strong> Frontiers built from observed units are sensitive to extreme observations: one freakishly strong team can pull the line so far out that everyone else looks inefficient. Statisticians who work with these estimators (Cazals, Florens and Simar, 2002, is the standard reference) use “partial” frontiers that ignore a few outliers. I used the plain version, and in 2007, 2012 and 2025, when six teams were on the line, you can see the opposite problem: a crowded, lopsided frontier is easy to be on.</li><li><strong>The earlier seasons are noisier.</strong> CollegeFootballData’s play-by-play for 2007–2013 is less complete than for the recent years, and I dropped teams with thin coverage (under 60% of the median team’s plays).</li><li><strong>I tried a few variants. </strong>End of season versus Week 10, a few distance bands, a net-rating comparison. The result I trust most is the simplest one: champions are on the frontier. The band results I’d call descriptive.</li></ul><p>If you’re a Tennessee fan, a Utah fan, a Georgia fan: none of this says your team can’t win it. It says that when teams have won it, they’ve looked a certain way by the first week of December, and that your team can still look that way by then.</p><p><em>Draw the frontier for any two stats, any week, and share it: the Ratings page at </em><a href="https://poll-position.robert-mepham1309.workers.dev/#/ratings"><em>Poll Position</em></a><em> has an “Efficient frontier” checkbox under the chart.</em></p><h3>References</h3><ul><li>Chambers, R. G., Chung, Y., and Färe, R. (1996). Benefit and distance functions. <em>Journal of Economic Theory</em> 70(2), 407–419.</li><li>Charnes, A., Cooper, W. W., and Rhodes, E. (1978). Measuring the efficiency of decision making units. <em>European Journal of Operational Research</em> 2(6), 429–444.</li><li>Cazals, C., Florens, J.-P., and Simar, L. (2002). Nonparametric frontier estimation: a robust approach. <em>Journal of Econometrics</em> 106(1), 1–25.</li><li>Debreu, G. (1951). The coefficient of resource utilization. <em>Econometrica</em> 19(3), 273–292.</li><li>Deprins, D., Simar, L., and Tulkens, H. (1984). Measuring labor-efficiency in post offices. In Marchand, Pestieau and Tulkens (eds.), <em>The Performance of Public Enterprises</em>, North-Holland.</li><li>Farrell, M. J. (1957). The measurement of productive efficiency. <em>Journal of the Royal Statistical Society, Series A</em> 120(3), 253–290.</li><li>Koopmans, T. C. (1951). Analysis of production as an efficient combination of activities. In Koopmans (ed.), <em>Activity Analysis of Production and Allocation</em>, Wiley.</li></ul><blockquote><strong>Methodology Used in Above Analysis</strong></blockquote><blockquote><strong>Seasons:</strong> 2007–2025 (19). EPA per play (offense and defense), garbage time excluded, from CollegeFootballData’s season-to-date advanced stats; 2014–2025 through the app’s rate packs, 2007–2013 pulled directly. Teams with fewer than 60% of the median team’s offensive plays are dropped.</blockquote><blockquote><strong>Opponent adjustment:</strong> (ridge-shrunk fixed-point iteration, shrinkK 2, 40 iterations) with each team&#39;s full regular-season schedule (FBS opponents only enter the adjustment; FCS games count toward games played).</blockquote><blockquote><strong>Checkpoints:</strong> “end of the regular season” = after conference championship games (2007–2013: through CFBD week 16; 2014–2025: the last poll before the bowls); “Week 10” = games through week 9 (the Week 10 poll).</blockquote><blockquote><strong>Frontier and slack:</strong> team i is on the frontier if no team j has both O_j &gt; O_i and D_j &lt; D_i (adjusted offense, adjusted defense allowed). Slack_i = max(0, max_j min(O_j − O_i, D_i − D_j)): the smallest ε such that adding ε to O_i and subtracting ε from D_i leaves nobody better on both.</blockquote><blockquote><strong>Champions: </strong>the winner of the last postseason game of each season (BCS title game 2007–2013; CFP final 2014–2025), cross-checked against the known champions.</blockquote><blockquote><strong>Null tests:</strong> (1) champion drawn from the whole field: tail probability from a Poisson-binomial with each year’s frontier share (3.9 × 10⁻²³), shown only as a bound; (2) champion drawn from the five best net-rated teams each year: expected 10.6 on the frontier, observed 16, tail probability 0.006 (Week 10: expected 10.6, observed 11, p = 0.52).</blockquote><blockquote><strong>Big games: </strong>every CFP game 2014–2025 with both teams in the field (the committee’s top 4 through 2023, the 12-team field in 2024 and 2025): 51 games. Rule A: smaller slack wins (ties when both are on the frontier: 11 games). Rule B: higher net rating wins. One-sided binomial tests vs a coin flip: 31 of 40 (p = 0.0003) and 36 of 51 (p = 0.002). Logistic regression with no intercept on differences (a minus b): net alone log-lik −30.2, slack alone −27.5, both −27.5; adding slack to net: LR = 5.48 (1 df, p = 0.019); adding net to slack: LR = 0.04.</blockquote><blockquote><strong>Distance bands:</strong> among the 12 best teams by net rating each year: on the line; within 0.02; 0.02–0.05; over 0.05. Chosen after looking. Permutation check: champion drawn uniformly from the top 12 each year; probability of no champion in the 0.02–0.05 band: 0.002 at Week 10, 0.005 at the end of the season.</blockquote><blockquote><strong>Drift: </strong>of 67 team-seasons that ended on the frontier, 53 had Week 10 slack ≤ 0.01, 62 ≤ 0.03, 64 ≤ 0.06.</blockquote><blockquote><strong>Limits:</strong> 19 championships; the frontier is sensitive to outliers; older play-by-play is less complete; two of the tests use bands chosen post hoc; the 2026 numbers are as of Week 10.</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5f4b97db6aab" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Who Did the College Football Playoff Committee Actually Screw?]]></title>
            <link>https://medium.com/@robert.mepham/who-did-the-college-football-playoff-committee-actually-screw-ef97006718ef?source=rss-745e2a58a3bf------2</link>
            <guid isPermaLink="false">https://medium.com/p/ef97006718ef</guid>
            <category><![CDATA[college-football-playoffs]]></category>
            <category><![CDATA[college-football]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[sports-analytics]]></category>
            <category><![CDATA[ai-applications]]></category>
            <dc:creator><![CDATA[Rob Mepham]]></dc:creator>
            <pubDate>Fri, 13 Nov 2026 14:41:01 GMT</pubDate>
            <atom:updated>2026-11-13T14:41:01.836Z</atom:updated>
            <content:encoded><![CDATA[<h3>Who Did the College Football Playoff Committee Actually Screw? I Built a Results-Only Model to Find Out.</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cztXsRI2n4sZqephMSjojw.jpeg" /></figure><p>In 2016, Western Michigan went 13–0 and the College Football Playoff committee ranked them 15th.</p><p>In 2023, Liberty went 13–0 and the committee ranked them 23rd.</p><p>Let me be clear about what those two sentences do and don’t say. Neither team was getting into the playoff. The field was <strong>four teams</strong> until 2024, and a 13–0 MAC team was never going to be one of them. A results-only ranking would have put Western Michigan 6th that year, and sixth would not have been enough either.</p><p>So this is not an article about who got snubbed from the playoff. It’s about the other thing the committee does every week: <strong>it ranks 25 teams</strong>, and almost all of those rankings have nothing to do with who plays for the title. Those rankings still matter. They decide who gets a New Year’s Six bowl, they become the “ranked opponent” every top-four team gets credit for beating, and since 2024 the top 12 <em>is</em> the field. And as a ranking, we can test them.</p><p>Poll Position has the committee’s <strong>final ranking from every season since 2014,</strong> plus a rating and a wins-above-bubble number for every team on the same day. So I built a model of what the committee <strong>normally does with results alone</strong>, and asked two questions. Did the committee pick the right four? And how does it sort everyone else?</p><p>The answers are more interesting than “the committee is biased” and less comfortable than “the committee is fine.”</p><h3>The Yardstick</h3><p>I’ll call it the results-only model. It sees exactly two things about each team on the day the committee ranked it:</p><ul><li><strong>Wins above bubble:</strong> your wins minus the wins an average bubble team would be expected to have against your schedule.</li><li><strong>Team rating:</strong> how good you are in points, from opponent-adjusted EPA per play.</li></ul><p>It does <strong>not </strong>see the AP poll, the team’s name, the conference, or last year. That matters: the AP voters see the same reputations the committee does, so a model that included the poll would simply learn the committee’s biases and hand them back.</p><p>I fitted it with a pairwise logistic regression to the committee’s final top 25 in every season, and, to keep it honest, each season is predicted from weights fitted on the <em>other</em> seasons. One extra win above bubble turns out to be worth about ten points of team rating to the committee.</p><p>Two things you need to hold on to.</p><p>1. <strong>This is a model of the committee, not of the truth.</strong> When the model and the committee disagree, either the committee saw something the data can’t (an injury, a quarterback, how a team looked) or the model is missing something. The disagreements are leads, not verdicts.</p><p>2. <strong>I left out 2020.</strong> Teams played wildly different numbers of games against wildly different schedules, and the numbers get strange (the model has Coastal Carolina, 11–0, first in the country and BYU, 10–1, sixth). Including it doesn’t change the story, and I’ll say where it moves a number, but it’s not a fair test.</p><h3>Where it Counted: The Top Four (Now Twelve)</h3><p>Start with the part that decided the title race. In the nine non-2020 seasons through 2023, the committee named four teams. In 2024 and 2025, it named twelve.</p><p>Across all eleven seasons, the results-only model’s top four contains 3<strong>9 of the committee’s 44 actual top-four spots, 89%.</strong> In seven of the nine four-team seasons, the model and the committee had e<strong>xactly the same four teams.</strong> The disagreements:</p><ul><li><strong>2018:</strong> the committee put Oklahoma 4th and Georgia 5th. The results had them the other way around (Georgia 3rd, Oklahoma 5th).</li><li><strong>2023: </strong>the committee took Texas (3rd) and Alabama (4th) over Florida State (5th, 13–0) and Ohio State (7th). The model had Florida State 3rd and Ohio State 4th. This is the famous one, and it has an explanation the data can’t see: Florida State lost quarterback Jordan Travis to a late-season injury, and the committee said that was part of its decision. (From memory, not the data: check it.)</li><li><strong>2024 and 2025,</strong> when twelve spots were in play: Miami (13th) and BYU (17th) were out, with Tennessee (7th) and Alabama (11th) in; in 2025, Utah (15th) was out and Alabama (9th) in.</li></ul><p>That’s the whole list. If you hoped to find a decade of results-only champions locked out of the four, the data doesn’t have them. For the most part, when a four-team playoff was the question, the committee agreed with the results. Disagreeing about <em>which</em> of two borderline teams gets the 4th slot is different from a hidden bias, and it shows up about once every five years.</p><h3>Where it Gets Interesting: Everyone Else</h3><p>That leaves the rest of the top 25, where the committee has fifteen or so teams to sort every year and nothing riding on the order except pride, bowls, and the “ranked win” that top-four teams get to put on their resume.</p><p>Here’s how the disagreements look when I draw the line at 12th, a yardstick for how far off a ranking was, not a claim about who’d have been in the field. Eleven seasons, fifteen teams the results-only model has in its top 12 and the committee ranked 13th or lower.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*y53ebTsvXUem8YSNUtHWLw.png" /></figure><p>None of the fifteen would have been in the results-only <em>four.</em> The best of them, Western Michigan 2016 and Alabama 2019, sit at 6th. So read this list as “teams the committee ranked well below where their results put them,” not “teams who should have made the playoff.” A few deserve a sentence.</p><p><strong>Western Michigan, 2016, 13–0, #15 → #6.</strong> The biggest gap on the list. The only unbeaten team in the country besides Alabama was ranked behind seven teams with three or more losses, including 8–4 Auburn. Sixth wouldn’t have gotten them in. But a ranking that put them 6th instead of 15th would at least have matched what they did on the field. They got a bowl bid and lost to Wisconsin 24–16.</p><p><strong>Memphis, 2019, 12–1, #17 → #7.</strong> A one-loss American champion, ten spots lower than the data would put them.</p><p><strong>Alabama, 2019, 10–2, #13 → #6.</strong> The one on this list that fans won’t want to hear. Alabama lost two games and lost its quarterback to a season-ending injury in November. That is exactly the kind of thing the committee is allowed to weigh, and the model can’t see it. (The injury is from memory, not from the data: check it.) They still won the bowl, 35–16.</p><p><strong>Liberty, 2023, 13–0, #23 → #12.</strong> Unbeaten, and ranked behind three 8–4 teams. The committee evidently discounted the schedule, and then Oregon beat them 45–6 in their bowl game. In fairness to the committee, that is one point for the eye test.</p><p><strong>Houston, 2015</strong> (12–1, finished 8th in the final AP poll) and Tulane, 2022 (11–2, finished 9th) beat Florida State and USC in their bowls. If the committee was underrating anyone, it was probably them.</p><p><strong>Miami and BYU in 2024, Utah in 2025</strong> are the only three of the fifteen that fell on the wrong side of a real 12-team cut, and the model’s complaint is about teams the committee had at 13th, 15th and 17th, not at 30th.</p><p>Now the other side of the ledger: the fifteen teams the committee ranked in the top 12 that the results-only model had outside it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5yD2I0QYUC2H-uNtIF8Cgg.png" /></figure><p>Tennessee in 2024 (ranked 7th, results say 15th) and Oklahoma State in 2016 (12th, results say 20th) are the biggest gaps, eight spots each; Baylor in 2019 (7th against 13th) and Penn State in 2018 (12th against 18th) are next, at six. Alabama appears on both lists: ranked above its results in 2024 and 2025, below them in 2019. Again, almost none of these crossed the real line: Texas and Alabama in 2023 and Tennessee and Alabama in 2024 are the few that mattered.</p><h3>Is There a Pattern, or is This Just Noise?</h3><p>The first thing to check is whether the count itself is surprising. It isn’t. Fifteen and fifteen over eleven seasons is what you’d expect from a yardstick that hits 89%. A committee is a group of humans; they won’t reproduce a formula.</p><p>So the question that matters is who the disagreements are about. Here is every team-season in the committee’s top 25 plotted two ways, committee rank across and model rank up. On the dashed line they agree.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ux7PTvqUPMjAY6QAqrAuwA.png" /></figure><p>Look at the orange dots (Group of Six). Most of them sit on the “committee harsher than results” side of the line. And the shaded zone at top right, the teams the committee ranked 13th or lower and the results put in the top 12, has five Group of Six teams among its fifteen.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IuVevvevq4lutBfz_jkmGw.png" /></figure><p>Here is the number. By league, the average distance between where the committee ranked a team and where the results alone ranked it, in spots, across the whole top 25:</p><ul><li><strong>Group of Six: +3.6 spots</strong> (95% interval +2.6 to +4.9). The committee ranks a Group of Six team about three and a half places lower than results alone would.</li><li><strong>Pac-12: −2.1</strong> (−3.1 to −1.2). Ranked more than two spots higher than results alone.</li><li><strong>Big Ten: −1.1</strong> (−1.9 to −0.3). A smaller bump, still clear of zero.</li><li><strong>SEC +0.2, Big 12 −0.1, ACC −0.3: </strong>nothing, with intervals that comfortably include zero.</li><li><strong>Independents +0.8</strong> (ten team-seasons, interval spans zero).</li></ul><p>Read that SEC line twice. The loudest complaint about the committee is that it has an SEC bias. By <em>this</em> yardstick, the SEC is ranked exactly where its results put it. The gifts and the snubs are even: five each. (Even the SEC teams on the “ranked lower than results” list, 8–4 Auburn in 2014 and 8–4 Missouri in 2018, sit at 10th and 12th by results. Neither was a four-team playoff candidate, and nobody here is saying they were.)</p><p>The stricter check is on the first number. Five of the fifteen teams the committee ranked well below their results are Group of Six programs. Group of Six teams are 12% of the committee’s top-25 team-seasons. The chance of getting five or more of fifteen from a group that makes up 12% by luck is about 3%. And the committee’s top 12 contained a Group of Six team just four times in eleven seasons. The results-only model would have had nine.</p><h3>The Objection I Take Most Seriously</h3><p>“Of course the model likes Group of Six teams. Beating up on weak schedules is how you get a 13–0 record and a high rating.”</p><p>That is right, and I’ve seen it before in this project. When I tested how my rating does on games between Group of Six and power-conference teams, it flatters the Group of Six by about 2.5 points of margin. It’s the ruler problem from the conference-bias series, and it would manufacture a Group of Six “snub” out of nothing. So I corrected for it: subtract 2.5 points from every Group of Six team’s rating and rerun everything.</p><p>The gap shrinks, from +3.6 to <strong>+2.4 spots (interval +1.3 to +3.5),</strong> and the model’s count of Group of Six teams in its top 12 falls from nine to six. It does not go away. The committee ranks Group of Six teams more than two spots lower than the results alone, even after taking a generous run at the numbers.</p><p>I also want to be careful about what this does and doesn’t show. A Group of Six discount is consistent with the committee <strong>valuing power-conference brands.</strong> It’s also consistent with the committee believing that a 13–0 record in a weaker league isn’t worth what the model thinks, and with factors neither of us can measure (some of which, like Liberty’s bowl, might be the committee being right). I can show you the pattern. I can’t show you the reason.</p><h3>Why a Ranking Bbelow the Line Matters</h3><p>If the playoff was four teams, why care whether Memphis was 7th or 17th? Because the rankings do work beyond the field.</p><p>The committee’s ranking is also its <strong>argument</strong>. When it explains why a top-four team belongs, the explanation is a list: wins over ranked opponents, wins over teams in the top 25. A rank is a currency. If the committee is generous to a team in the 8–15 range, then every top-four team that beat that team gets a quality win that its results alone wouldn’t give it. If the committee is stingy with another team, the team that beat <em>that</em> one gets less credit.</p><p>I want to be careful here. This is a hypothesis, and <strong>this analysis does not test it.</strong> What I measured is that ranks outside the four deviate from results in a pattern: Pac-12 and Big Ten teams up, Group of Six teams down. Whether that pattern was <em>used</em> to prop up a playoff team’s resume, to justify a head-to-head result that looks better against a “#10” than against the “#20” that team’s results suggest, would need a different test: take each top-four team’s ranked wins and re-price them using the results-only rank. That’s the next piece. What the data can already say is that the committee’s order below the top four is less a clean ranking of resumes than it looks, and the errors aren’t spread evenly across leagues.</p><h3>Did the committee know something?</h3><p>If the committee were seeing something real in the Group of Six-and-friends cases, the teams it ranked lower should have done worse afterward than the teams it ranked higher.</p><p>Bowl results: the fifteen teams ranked below their results went <strong>9–6</strong> in their bowl games. The fifteen ranked above went <strong>6–10.</strong> That tilts toward the model, but this is a small sample. A gap that size would show up about three times in ten from a coin flip (Fisher’s exact test p = 0.29), so I’m not calling it evidence. Bowls are also a noisy check: <em>opt-outs, matchups and motivation all change.</em></p><p>What I will say is that <strong>nothing in the post-selection record says the committee was right.</strong> If anything it points the other way.</p><h3>So What’s the Verdict?</h3><p>In three parts.</p><p><strong>The top four: mostly fine.</strong> For the four-team era, the committee’s final four are the results’ final four in seven of nine seasons, and one-spot swaps (2018) or a team with an injured quarterback (2023) explain much of the rest. If you came looking for stolen playoff spots, this isn’t it. The most important thing this model says about the committee is that it agrees with a results formula where the title was decided.</p><p><strong>Below the top four, there’s a type.</strong> The Group of Six is ranked more than two spots lower than its results, even after correcting for a schedule bias, and a Group of Six team was in the committee’s top 12 in only four of eleven seasons. Western Michigan was never getting in. But a 13–0 team ranked 15th and not 6th isn’t a harmless rounding error. It’s the same discount the 12-team format now partly patches by guaranteeing the best Group of Six champion a spot, and that matters much less in a 24-team field, which I look at in the 2026 article.</p><p><strong>Power-conference teams, a little.</strong> Tennessee 2024, Baylor 2019, Penn State 2018, and a run of Pac-12 and Big Ten teams were ranked above what results alone say, by a modest margin. The Pac-12 (−2.1) and Big Ten (−1.1) premiums are real in the numbers. What they do to a top-four team’s resume is the open question above, and a claim about <em>why</em> would need a different kind of evidence than ballots and results.</p><p>What I won’t say is that the committee is rigged, or that an 8–4 team or a 13–0 MAC team was kept out of a four-team field. A committee that agrees with a results formula 89% of the time, ranks the SEC right where the results put it, and whose lower-ranked teams went 9–6 in their bowls, is doing something close to what it says. It just applies a quiet discount to one corner of the sport, and hands a small bonus to two others.</p><p><em>Look up the resume numbers behind any team, and compare two bubble teams head to head, at </em><a href="https://poll-position.robert-mepham1309.workers.dev/"><em>Poll Position | Playoffs</em></a><em>. And if you want the model that estimates each team’s chances going forward, that’s the explainer.</em></p><blockquote><strong>Methodology Used in the Above Analysis</strong></blockquote><blockquote><strong>Seasons and timing:</strong> 2014–2025. For each season, the last poll before the bowls (all conference title games played). 2020 is left out of the lists and headline numbers (shown as hollow points in the scatter) because of unequal schedules.</blockquote><blockquote><strong>Real field size:</strong> four teams 2014–2023, twelve from 2024. The top-12 lines in the lists and the scatter are a <em>yardstick</em> for sorting error, not a playoff cut, except in 2024 and 2025.</blockquote><blockquote><strong>Committee ranking:</strong> the final ESPN-published CFP ranking of each season, top 25</blockquote><blockquote><strong>Results-only model:</strong> score = w1 · wins above bubble + w2 · team rating (opponent-adjusted EPA plus results; points). Pairwise logistic regression (a team ranked above another team within the committee’s top 25), weights fitted leave-one-season-out. The AP poll, conference, and prior-season information are not inputs. Fitted weights on all seasons: 1.54 per win above bubble, 0.155 per rating point.</blockquote><blockquote><strong>Pool:</strong> the committee’s own top 25 each season, so a “snub” here is a team ranked 13th to 25th by the committee. A strong team the committee left out of its 25 entirely is not in the data.</blockquote><blockquote><strong>Recovery: </strong>the model’s top 12 contains 88.6% of the committee’s real top 12 (86.1% including 2020); top four 88.6% (87.5% including 2020), 39 of 44 spots across the eleven non-2020 seasons. Exact same four teams in seven of nine four-team seasons (2014–2019, 2021–2023 minus 2018 and 2023).</blockquote><blockquote><strong>Disagreements at the real cut (model inside, committee outside, and the reverse):</strong> 2018 Georgia (c5, m3) / Oklahoma (c4, m5); 2023 Florida State (c5, m3), Ohio State (c7, m4) / Texas (c3, m6), Alabama (c4, m5); 2024 Miami (c13, m11), BYU (c17, m12) / Tennessee (c7, m15), Alabama (c11, m14); 2025 Utah (c15, m9) / Alabama (c9, m14). None of the fifteen top-12 “snubs” is in the model’s top five.</blockquote><blockquote><strong>Gap:</strong> committee rank minus model rank, positive = the committee ranked the team lower than results alone suggest. Group means with 95% intervals from resampling seasons (4,000 resamples). With 2020 included, Group of Six is +4.5 (2.8 to 6.5).</blockquote><blockquote><strong>Group of Six ruler correction:</strong> subtract 2.5 rating points from every Group of Six team before scoring (the flattery measured on Group of Six vs power-conference games in the voter series). Gap becomes +2.4 (1.3 to 3.5); Group of Six teams in the model’s top 12: 9 raw, 6 corrected, 4 in the committee’s.</blockquote><blockquote><strong>Luck check:</strong> Group of Six = 34 of 275 team-seasons (12.4%); five of fifteen below-results teams; the binomial probability of five or more is 0.030.</blockquote><blockquote><strong>Bowls: </strong>bowl-game results of the fifteen lower-ranked (9–6) and fifteen higher-ranked teams (6–10 over 16 games, because 2025 Alabama played two postseason games); Fisher’s exact test, two-sided, p = 0.29. A playoff game counts as a bowl.</blockquote><blockquote><strong>Not tested: </strong>whether inflated or deflated ranks of opponents changed the credit a top-four team’s wins earned. That needs re-pricing each playoff team’s ranked wins with the results-only rank.</blockquote><blockquote><strong>Limits:</strong> the model does not see injuries, quarterback changes, or head-to-head; the committee’s criteria changed over time; “Pac-12” means the league as it existed 2014–2023.</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ef97006718ef" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I Simulated the College Football Playoff 40,000 Times.]]></title>
            <link>https://medium.com/@robert.mepham/i-simulated-the-college-football-playoff-40-000-times-7c628596cd94?source=rss-745e2a58a3bf------2</link>
            <guid isPermaLink="false">https://medium.com/p/7c628596cd94</guid>
            <category><![CDATA[college-football-playoffs]]></category>
            <category><![CDATA[college-football]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[bracketology]]></category>
            <category><![CDATA[sports-analytics]]></category>
            <dc:creator><![CDATA[Rob Mepham]]></dc:creator>
            <pubDate>Wed, 11 Nov 2026 14:46:01 GMT</pubDate>
            <atom:updated>2026-11-11T14:46:01.583Z</atom:updated>
            <content:encoded><![CDATA[<h3>I Simulated the College Football Playoff 40,000 Times. Here’s How the Model Works, and Where It Was Wrong.</h3><p>Here is a number that changed how I watch this Saturday as I’m writing this. Tennessee is 6–2 and plays Kentucky on November 7. According to the model, if Kentucky wins, Tennessee only has a <strong>12%</strong> chance of making the 12-team playoff. If Tennessee wins, that goes up to <strong>29%</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bC_M4-zVywuAXBlhqbk_NQ.png" /></figure><p>One game. A 17-point swing in a team’s postseason odds, and 17 points is not a trivial shift, that’s a team’s whole season at this point. In 40,000 simulated versions of the rest of the 2026 season, Tennessee made the field in 12% of the ones where it lost to Kentucky and 29% of the ones where it won.</p><p>I’ll admit I wrote the whole thing because I wanted to watch my own team’s odds move. But a number like that is worthless unless you know where it comes from and whether it’s honest. So this article does two things: it explains how the Playoff page in <a href="https://poll-position.robert-mepham1309.workers.dev/">Poll Position</a> works, step by step, and then it audits the model against twelve years of real CFP committees. The audit found a real flaw. I fixed it. You should know about both.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vaa38o0wrdz7upSniUNwUg.png" /><figcaption><em>The Odds tab, Week 10 poll. Every column is a count over simulated seasons.</em></figcaption></figure><h3>One simulated season, in five steps</h3><p>The model is a loop. Play out the rest of the season once, see who ends up where, write it down, and do it again 40,000 times. A team’s “chance of making the field” is the share of those seasons in which it made it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jQgi9xn2UCydtYytKilC0Q.png" /></figure><p><strong>Step 1: play the games.</strong> Every unplayed game, including conference title games once ESPN lists them, is a weighted coin flip. The weight comes from a team rating (in points, built from opponent-adjusted EPA per play plus results), a home-field edge of 2.4 points, and the fact that real game margins scatter widely around their expected value (a standard deviation of about 15.4 points). If you are 7 points better than your opponent and playing at home, your win probability is the normal CDF of (7 + 2.4) ÷ 15.4, or about 73%. Take that same 7-point edge on the road and it is about 62%. Nobody is ever 100%, which is how upsets get into the model.</p><p><strong>Step 2: crown champions. </strong>Each conference’s winner comes from the simulated standings (conference win percentage, ties broken by rating). The conferences that play a title game send their top two to it.</p><p><strong>Step 3: score the committee.</strong> This is the hard part, and I’ll come back to it. The committee does not publish a formula, so the model estimates one.</p><p><strong>Step 4: pick the field.</strong> For 2026–27 the model’s default is the freshest rule I could find: the four Power 4 conference champions, the highest-ranked Group of Six team, and the rest of the 12 by committee score. Seeds go straight by score (a champion is not guaranteed a bye anymore), and the top four get byes. The Playoff page also lets you flip to the 2024 and 2025 five-champion rule, because the sources I could find disagree about the Group of Six spot (best-ranked team versus champion).</p><p><strong>Step 5: play the bracket.</strong> 5 plays 12, 6 plays 11, 7 plays 10, 8 plays 9, at the higher seed; the winners meet the top four in the quarterfinals; no reseeding; everything after round one is neutral-site. The same win probabilities, run forward.</p><h3>The part nobody can see: the committee</h3><p>If you could read the committee’s mind, every playoff model would be easy. You can’t, so I did the next best thing: I fitted one. For each season from 2014 to 2025, I took the committee’s final ranking (the one that picks the field) and asked what combination of ingredients best explains who they put above whom.</p><p>The method is a pairwise logistic regression, the same family as the Bradley-Terry model used for chess ratings: the chance the committee ranks team A above team B is the logistic function of the weighted difference between their ingredients. Three ingredients survived:</p><ul><li><strong>Wins above bubble.</strong> Your wins minus the wins an average bubble team would be expected to have against your exact schedule. It rewards beating good teams and punishes losing to bad ones, and it is the single best one-number summary of a resume.</li><li><strong>Team rating.</strong> How good you actually are, in points, from the EPA model.</li><li><strong>AP poll share.</strong> The share of the maximum possible AP points the team had in the poll N polls earlier. The committee does not look at the AP poll, but it looks at the same teams and the same games, so the poll is a decent stand-in for the reputation and eye-test pieces I can’t measure directly.</li></ul><p>I fitted one set of weights per number of polls left, because the poll is worth more the closer the end gets:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/497/1*afVpC6GMY9Nq00MO9i_YtQ.png" /></figure><p>Read the table from the bottom up: with one poll to go, the committee’s order is basically the poll’s order. Four weeks out, it is mostly resumes.</p><p>Here is how well that works, tested the honest way. Each season is predicted using weights fitted only on the <em>other eleven</em>, so the model never sees the answer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8Bja5vkMOMG56nFRwZKEPQ.png" /></figure><p>The committee model recovers <strong>84% of the real top 12 four polls out, 87% at three, 89% at two, and 93% at one.</strong> Note the last group: with one poll left, the AP poll by itself does as well (94%). That is not a failure, it’s the point: by then the committee is mostly following the same information that the AP voters are, or rather the AP voters just mirror what they believe the CFP committee will do. The model’s edge is earlier, when the poll hasn’t yet figured out what the committee will do.</p><h3>Now the audit: does “70%” really mean 70%?</h3><p>Matching the committee’s past rankings is not the same as forecasting. A forecast is a set of probabilities, and the only way to know whether they are any good is to check whether things you called 70% happen about 70% of the time.</p><p>So I ran the whole machine, not just the committee piece, on every season from 2014 to 2025. For each one I stopped the clock four, three, two and one polls before the final ranking, treated every later game as unplayed, simulated from there, and compared each team’s chance of making the field with whether it made the field the committee’s real ranking implied under the same rules. That is about 1,570 team-seasons at each of the four checkpoints. (A caveat: the 12-team format only began in 2024, so the earlier “fields” are what the committee’s ranking would have produced under the 12-team rule, not what happened.)</p><p>The first version failed.</p><p><strong>It was too sure of itself.</strong> Teams the model gave at least a 98% chance made the field only <strong>93%</strong> of the time (307 of 330). Teams it put between 50% and 90% averaged 70% in the model and made it 54% of the time. And teams it gave under 2% made the field 54 times out of 5,458, more often than you’d want from a number that small.</p><p>The reason is easy to see once you’ve seen it. My committee score was a <em>deterministic</em> function of results: same results, same ranking, every time. But a real committee isn’t a formula. It has judgment, injuries, a view on how a team looked, a reaction to a late loss. A model with no room for any of that can’t be wrong about the committee, so it’s too confident whenever results are lopsided.</p><p><strong>The fix</strong> is to give the committee some room to disagree with itself: add a random amount to each team’s score in each simulated season, drawn from a bell curve whose width I fitted to the backtest (a standard deviation of 1.4 to 2.2 points of committee score, bigger further from the end, when more can still change). Nothing about the model’s favorites changed. What changed is how often a team the score has “a hair behind” gets picked anyway.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wIt8UIrD4cUrqOZfiKcjtQ.png" /></figure><p>The pink line is the first version. The green line is the fix. With committee noise, teams given 98% or more made the field <strong>99%</strong> of the time (123 of 124); teams given under 2% made it twice in over 5,000 chances; and the 50–90% group averaged 70.7% in the model and <strong>70.2%</strong> in reality.</p><p>Two honest limits. I picked the four noise settings using the same twelve seasons I’m now scoring them on, so they get a small home-field advantage. Four numbers on about 6,300 team-checkpoints is not much to overfit, but the real test is this season and next. And the improvement is calibration, not cleverness: it makes the percentages mean what they actually say.</p><h3>What the model is not</h3><p>Here is the table I think matters most for how much to trust it. For each checkpoint I compared the simulation’s Brier score (a measure of probability error: lower is better; zero is perfect, and “everyone has the base rate” scores 0.084) against a simple lookup, “teams ranked 5–8 in the AP poll make the field X% of the time”, learned on the same data:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/424/1*zkXla-yoGL3qJ1rPXNKREg.png" /></figure><p>The simulation is marginally better four polls out and then slightly <em>worse</em> than a table that only knows the AP rank. That is the honest result, and it’s an informative one. When it comes to “what are this team’s odds right now,” the AP poll alone is nearly as good, and the lookup is a little bit in-sample-flattering. What the poll <strong>cannot</strong> tell you is what happens if one specific game goes the other way. That is what the simulation is for: Tennessee’s 12%-versus-29% is a thing the poll can’t compute.</p><p>And there are things it doesn’t know at all: injuries and quarterback changes, how a team <em>looks</em> (the committee’s “game control”), head-to-head results beyond what is in the ratings, and the finer tiebreakers inside a conference. The 12-team format also has only two seasons of history, so the odds are a model’s view and not a calibrated forecast in the weather-service sense. And the real committee’s first rankings of 2026 come out Tuesday night; the model doesn’t use them yet, and I plan to add them as an anchor once I have a few weeks of data.</p><h3>How to use it</h3><p>The Playoff page has five tabs. <strong>Odds </strong>is the table above. <strong>Seed matrix </strong>shows each team’s chance of each seed. <strong>Bubble</strong> drills into any team (the resume head-to-head lets you put two bubble teams side by side). <strong>Bracket</strong> lets you drag teams into the field and pick winners to see the title odds change. And the <strong>What-if planner</strong> is where the number I referenced for my Volunteers came from: click a result on a team’s remaining schedule, and everything below it, and the games that matter elsewhere, recomputes assuming the result you entered actually happened.</p><p>I used the model in the next two pieces I’ll be releasing. <strong>One looks at the twelve past committees and asks who a results-only model would have put in that the real one left out. The other is the 2026 picture, and what a 24-team field would do to it.</strong></p><p><em>Play with the odds, pick results and see who makes the field at </em><a href="https://poll-position.robert-mepham1309.workers.dev/"><em>Poll Position — Playoffs</em></a></p><blockquote><strong>Methodology Used in the Above Analysis</strong></blockquote><blockquote><strong>Win model: </strong>P(A beats B) = Φ((R_A − R_B + h·s) / σ), with ratings R in points from opponent-adjusted EPA plus results, home-field h = 2.41, σ = 15.39, s = +1/0/−1 for home/neutral/away. FCS opponents get a fixed rating of −22.</blockquote><blockquote><strong>Committee score:</strong> wab·(wins − expected wins of a bubble team) + R_coef·rating + ap_coef·(AP points ÷ maximum possible, N polls earlier). Fitted by pairwise logistic regression to the 2014–2025 final CFP rankings (top-25 pairs), weights per N = 1 to 4; leave-one-season-out accuracy in the first chart (models/playoff/COMMITTEE.md).</blockquote><blockquote><strong>Committee noise:</strong> score_i += σ_N · z_i with z standard normal, one draw per team per simulated season; σ = 1.4, 2.0, 2.0, 2.2 for N = 1, 2, 3, 4, chosen by log-loss on the backtest (σ grid 0 to 2.5). In-sample for the four σ values.</blockquote><blockquote><strong>Backtest:</strong> 12 seasons × 4 checkpoints, 6,000 simulated seasons each, about 1,570 team-seasons per checkpoint. Realized field = four Power 4 champions (by the title game or best conference record) + the highest-ranked Group of Six team + the highest-ranked remaining teams to 12, using the real final committee ranking. 2014–2023 fields are hypothetical (a four-team playoff existed then). Baseline: the share of teams in each AP-rank bucket that made the field, learned on the same rows.</blockquote><blockquote><strong>Simulation count for this article:</strong> 40,000 seasons at the Week 10 poll (seeded; the Playoff page defaults to 8,000).</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7c628596cd94" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Is the College Football AP Poll Best Modeled by Non-Linear Statistical Models?]]></title>
            <link>https://medium.com/@robert.mepham/is-the-college-football-ap-poll-best-modeled-by-non-linear-statistical-models-b40e16dceb19?source=rss-745e2a58a3bf------2</link>
            <guid isPermaLink="false">https://medium.com/p/b40e16dceb19</guid>
            <category><![CDATA[ap-top-25]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[college-football]]></category>
            <category><![CDATA[nonlinear-models]]></category>
            <category><![CDATA[sports-analytics]]></category>
            <dc:creator><![CDATA[Rob Mepham]]></dc:creator>
            <pubDate>Mon, 09 Nov 2026 14:31:01 GMT</pubDate>
            <atom:updated>2026-11-09T14:31:01.719Z</atom:updated>
            <content:encoded><![CDATA[<h3>Is the College Football AP Poll Best Modeled by Non-Linear Statistical Models? The Answer May Surprise You</h3><p>The production forecast behind the app’s Forecast page (“Pole Position AI”, beta 2.6) is linear almost end to end: shrunk pairwise-logistic models for each voter, then a ridge regression that stacks their output with 30 team-level features. The previous article (<a href="https://medium.com/@robert.mepham/inside-pole-position-ai-hierarchical-shrinkage-asymmetric-calibration-and-a-stacked-ridge-model-237c1cfc22e8"><strong><em>Inside Pole Position AI</em></strong></a>) walks through how it got there. This one is about the obvious next question, the one every reader with a machine-learning background asked: <strong>why not trees?</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*N7L_ZhxMfDFHsKuSlKCsgw.png" /></figure><p>We tested it properly. We wrote the protocol, the candidate models, the acceptance gates and the decision rules <em>before</em> fitting anything, tagged that document in git, and then ran it. The short version:</p><ul><li>One candidate, <strong>beta 3.0</strong> (a gradient-boosted correction on top of the beta 2.6 forecast), improved the average rank miss from 0.935 to 0.903 over 104 scored weeks: <strong>-0.032, 95% block-bootstrap interval [-0.054, -0.017]</strong>, Diebold-Mariano p &lt; 0.001, Holm-adjusted p &lt; 0.001. It passed every pre-registered gate, including an untouched 2026 holdout (-0.133 over six weeks).</li><li>The other candidate, <strong>beta 3.1</strong> (a LambdaMART learning-to-rank stack), did not beat the linear model (+0.008, interval [-0.026, +0.042]) and failed three gates.</li><li>The gain is real in the statistical sense and <strong>still not enough to ship:</strong> it is concentrated in two seasons (2020 and 2025), it vanishes if you drop those two seasons (-0.010), the hyperparameters the nested selection picks swing wildly from season to season, and six holdout weeks is six holdout weeks.</li></ul><p>So beta 2.6 stays in production. Beta 3.0 now runs in <strong>shadow mode,</strong> it is fitted and logged every week, never shown on the live site (<a href="https://poll-position.robert-mepham1309.workers.dev/#/poll">AP Top 25 · Poll Position</a>), and a review is scheduled for late April 2027, after the 2026 season is complete, with promotion rules written down today. This article explains the design, the statistics, the results with their uncertainty, and why “passes the gates” and “ship it” are different statements.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9RF90TkVqjD9AbBTcIdi5Q.png" /></figure><h3>1. Why ask the nonlinear question at all</h3><p>A linear stack can only add up main effects. If the effect of “won by 30” depends on how highly the opponent was ranked, or on whether the team is on a bye, or on whether it is rank 3 or rank 17, a ridge model sees those as separate additive terms. Trees can represent such interactions for free.</p><p>We do not have to guess whether interactions exist in this data; we can test it. Take the residual of the beta 2.6 forecast (real poll points minus forecast points, scaled) and group it by cells of two coarse factors: the team’s previous-poll rank band and the game outcome type (lost to a ranked team, won comfortably, bye, and so on). Compare a model with additive row and column effects against one with a free mean per cell. The standard two-way ANOVA interaction test is an F test on the extra sum of squares:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*47TPsHMwmUfBcd-JcbTKYQ.png" /></figure><p>On the 2015 to 2019 tuning era only (the data we were allowed to look at before registering the plan), the interaction F was <strong>2.02 on (22, 1543) degrees of freedom, p = 0.003.</strong> The additive structure explained 5.0% of residual variance and the full cell structure 7.7%. That is a small effect in absolute terms (residual variance is mostly noise) but it is a real one, and it is the quantitative reason the experiment was worth running. It also sets expectations: if only 2.7 percentage points of a residual’s variance is interaction signal, a model that converts all of it to accuracy still cannot move the headline much. This is the first of several places where the arithmetic told us in advance not to expect a transformation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iCcMHBHwDPxlGr7lLR3_5w.png" /></figure><h3>2. Designing a test that a flexible model cannot pass by luck</h3><p>The problem with testing a flexible model on a tiny sample is that flexible models have many ways to look good by chance. Our evaluation set is 104 scored weeks over eight seasons; the weeks inside a season are strongly correlated (the same teams, the same storylines). Everything in the protocol exists to keep the selection process from inflating the result.</p><p><strong>2.1 Winner’s curse.</strong> If you fit K candidate configurations and report the best one’s score, the estimate is biased upward. For K equally good candidates with estimation noise sigma, the expected maximum of the estimates grows like sigma times the square root of 2 ln K:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EdcjZ8jegeUeuEB14kyjxg.png" /></figure><p>With 24 random hyperparameter configurations per season and a standard error on the order of 0.02 to 0.03 in average miss, the selection bias is of the same size as the effect we were trying to detect. So the reported score of a candidate is never the score of its best configuration on the evaluation data.</p><p><strong>2.2 Nested walk-forward selection.</strong> For every scored season s, the hyperparameters are chosen using only the two prior seasons as an inner validation window, and the model is then refit before each poll using only information available at that time. The configuration chosen for 2023 never saw 2023. This is rolling-origin evaluation with nesting in the sense of Cawley and Talbot (2010) and Varma and Simon (2006), adapted to time series per Bergmeir and Benitez (2012) and Cerqueira, Torgo and Mozetic (2020): shuffled cross-validation would leak the future, and an un-nested search would leak the evaluation window. Each refit uses five random seeds averaged, so single-seed luck does not pick winners.</p><p><strong>2.3 The one-standard-error rule.</strong> Among configurations whose inner score is within one standard error of the best, we pick the <em>simplest</em> (fewest trees, shallowest depth):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MLIvykt0j3qj3Ed2pYrLag.png" /></figure><p>This is the Breiman-Friedman-Olshen-Stone heuristic. It is protective against over-fitting, and, as section 6 shows, it has a cost: with so little inner data it often picks very small models.</p><p><strong>2.4 Power before data.</strong> Before fitting anything we computed what the sample could detect. For paired weekly differences d_t with standard deviation sigma_d over n weeks, the minimum detectable effect at 5% two-sided significance and 80% power is</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hGhngeY3ygwWCz8cEd04yQ.png" /></figure><p>Using the weekly standard deviation of the difference between beta 2.6 and a correction layer of this type (about 0.067, measured on the tuning era), n = 104 gives an MDE near 0.018 for a correction layer and about 0.044 for a model that replaces the stack outright (sigma around 0.16). This is why the primary gate for 3.0 required an improvement of at least 0.02 with an upper confidence bound below zero, and for 3.1 at least 0.035. It also tells us that “no significant difference” for 3.1 is weak evidence of equivalence, not strong evidence of it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iP0_KrhkINVYqrajnpCdFg.png" /></figure><p><strong>2.5 Inference that respects dependence.</strong> Weekly differences are not independent: consecutive weeks share teams and narratives. A naive standard error would be too small. We use two protections. The first is a moving-block bootstrap (Kuensch, 1989) with blocks of three consecutive weeks inside one season, which preserves local dependence:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cbMzYIDYVIxjKF24Vqy3Rg.png" /></figure><p>The second is the Diebold-Mariano test of equal predictive accuracy, with the Harvey-Leybourne-Newbold small-sample correction and Student-t reference distribution:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ickZTctZw73YTBXWw_639w.png" /></figure><p>Because three candidates were registered, the primary p-values are Holm-adjusted (step-down Bonferroni, which controls the family-wise error rate and is uniformly more powerful than plain Bonferroni):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LTRjY7HDNcWmx0Gc2s7MnQ.png" /></figure><p><strong>2.6 Gates, not a single number.</strong> A lower average miss is necessary but not sufficient. The plan fixed ten gates in advance: the primary endpoint (P), accuracy on the №1 team (G1), first-place vote error (G2), no deterioration by stage of season or by era (G3), season-level consistency (G4), within-one-spot rate and points error (G5), calibration slope between 0.90 and 1.10 (G6), a direction-of-effect audit using SHAP (G7), the 2026 holdout (G8) and retraining time (O1). Operational gates O2 to O4 are only evaluated if a candidate would otherwise be promoted. The gate list, thresholds and the rule that every gate must pass were committed under the git tag plan-beta-3-v1 before the first model was trained.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XYmiWv_8TbLmDbjbowTLMA.png" /></figure><h3>3. The candidates</h3><p><strong>Beta 3.0: a boosted correction, not a replacement.</strong> The linear stack is good. Rather than throw it away, 3.0 keeps its forecast as a base and learns a correction on the residual, using gradient-boosted trees (LightGBM) in the sense of Friedman (2001): fit a sequence of small regression trees to the negative gradient of the loss, which for squared error is just the current residual.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aU7jA3eKqAVWg5Si3JrU9w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0JOga4-Z-0twh3Uh83-BrA.png" /></figure><p>The correction is applied per team per week. The features are the same 31 stack inputs plus nine extras. Two details matter for credibility. First, we impose <strong>monotone constraints</strong> on features where the direction is not in doubt (a bigger margin of victory, a higher-quality win and a higher previous-poll standing should never <em>lower</em> the forecast, holding the others fixed):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jMljU-MgpPA3mw6vivURgQ.png" /></figure><p>Gupta et al. (2016) show that such constraints regularize tree ensembles and make them easier to audit. Second, the model is evaluated against beta 2.6 on the <strong>same weeks with the same information,</strong> so the comparison is paired.</p><p><strong>Interactions versus shape.</strong> A tree of depth 1 (a stump) can only fit a function of one feature at a time; boosting stumps gives a generalised additive model, the “shape” half of the equation below. Depth 2 or more introduces interactions. We registered a nested search over depth and two ablations specifically to separate “nonlinear shape” from “interaction”:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*grt9TFEVcs8shzSVOBv8VQ.png" /></figure><p>Ablation <strong>3.0a</strong> restricts the correction to stumps (additive but nonlinear in each feature); ablation <strong>3.0b</strong> swaps boosting for ExtraTrees (bagged random trees). If the gain comes from interaction structure, 3.0a should fail to reproduce it. That is exactly what happened (section 4.4).</p><p><strong>Beta 3.1: learning to rank.</strong> Our metric is a rank error, so a ranking loss is the natural idea. LambdaMART (Burges, 2010) optimises a listwise objective by weighting each pairwise gradient by how much swapping the two items would change an information-retrieval metric:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oPz6HGivIhd0X6dPh2eBPQ.png" /></figure><p>Our setting has a mismatch worth spelling out. Retrieval metrics such as DCG@25 reward getting the <em>top</em> of the list right and discount the rest. Our score treats an error at rank 20 the same as an error at rank 2, because both are spots missed. This is the practical difference between the two objectives:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oTHhdbWdxPrMt1Ad3uyxoA.png" /></figure><p>We registered 3.1 as a <em>replacement</em> of the ridge stack (using the beta 2.6 forecast and the stack features as inputs, with rank_xendcg and lambdarank objectives both in the search), with a higher bar because it removes the stable linear backbone.</p><p><strong>Registered but not run: 3.2 and 3.3.</strong> Hierarchical nonlinear voter models (3.2) were gated on 3.0 or 3.1 showing a signal at the stack level, and an ensemble (3.3) was conditional on at least two candidates passing. Neither was run in this round; they are deferred until the stack-level question is settled, not rejected.</p><h3>4. Results</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pdMAWiMeYpt_rT6j4z89Nw.png" /></figure><p><strong>4.1 The headline.</strong> Over 104 scored weeks, beta 2.6 averages a miss of 0.935 and beta 3.0 averages 0.903. The paired difference is -0.032 with a block-bootstrap interval of [-0.054, -0.017]; the DM statistic with the HLN correction is -3.96 (p &lt; 0.001) and the Holm-adjusted p across the three registered candidates is below 0.001. The bootstrap never produced a non-negative mean in 10,000 draws. The №1 team was right in 92.3% of weeks under both models, first-place votes were predicted slightly better (11.05 vs 11.07 mean absolute error, a wash), and the calibration slope of forecast points on real points is 1.002 against 1.004 for the baseline. In other words the gain is in <em>where teams land</em>, not in headline calls.</p><p>The improvement is not uniform across the season: -0.044 in the early weeks, -0.031 mid-season and -0.022 late. That ordering is what you would expect if the trees help most when the linear model has the least history to lean on.</p><p><strong>4.2 The ablations say it is interaction, not just shape.</strong> The additive-only version (3.0a) improved by -0.002 [-0.005, +0.002]; the ExtraTrees version (3.0b) by -0.006 [-0.014, +0.001]. Neither is distinguishable from zero. Whatever 3.0 is capturing, a sum of per-feature curves does not capture it, and random-forest-style averaging does not either. Combined with the ANOVA result in section 1, the story is coherent: the extra signal lives in feature interactions, and it needs boosting to extract it.</p><p><strong>4.3 LambdaMART did not help.</strong> Beta 3.1 came in at +0.008 [-0.026, +0.042], DM p = 0.60. It failed the primary gate and the two consistency gates (stage-and-era, season-level). Its SHAP profile explains why: almost all of the model’s attribution (mean absolute 0.64 points of rank score) is the beta 2.6 forecast it was given as an input, the next feature carries 0.18, and every other feature is below 0.03. The ranking objective collapsed to “copy the stack and add noise”. The metric mismatch from section 3 is a plausible reason (the loss spends its capacity on the top of the list), but we did not run an experiment to prove that, so treat it as a hypothesis.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PRkMZKIHFkaXrhj8CymbpA.png" /></figure><p><strong>4.4 Where the gain lives.</strong> Here the honest reading starts to diverge from the headline. Per-season changes for beta 3.0 were: 2018 0.000, 2019 0.000, 2020 -0.102, 2021 -0.010, 2022 -0.009, 2023 -0.012, 2024 -0.025, 2025 -0.089. Two seasons carry most of the average. A leave-one-season-out analysis, which recomputes the pooled difference after removing season s,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/653/1*0CfVRtQAFNWK_KAsXFc0dA.png" /></figure><p>gives values between -0.022 and -0.037, so no single season produces the result. But dropping <em>both</em> 2020 and 2025 (77 weeks) leaves a difference of only <strong>-0.010.</strong> The trees improved the forecast in 38% of weeks, so the typical week is unchanged and a minority of weeks improves by a lot. That is a perfectly valid kind of model gain (rank-forecast errors are heavy-tailed), but it is a different claim from “the model is better every week”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VzFojf5sSn-nHNSaDTdI6Q.png" /></figure><p><strong>4.5 Which situations improve.</strong> Splitting weekly team-level misses by what happened to the team:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/662/1*qzQXI3a9cfN9l-4Sq4urQw.png" /></figure><p>The improvement sits where voter behavior is least mechanical: teams entering the poll and teams making large moves, and not on blowout wins or byes where the linear model already has the answer. That pattern is exactly what an interaction model should do, and it is a useful consistency check: a model fitting noise would not improve in a structured subset of situations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FsJzYOGfkFeeSrbok18cWQ.png" /></figure><p><strong>4.6 The selection process itself is unstable.</strong> The nested search picks different configurations nearly every season. For 3.0 the number of boosting rounds chosen on the inner window ranged over 25, 100, 800, 25, 25, 100, 100, 800 for 2018 through 2025, and 800 again for 2026, with the depth flipping between two and three. The two seasons that chose the large model are the two with the large gains. Beta 3.1 chose 25 rounds for 2018 to 2022 and 400, 400, 200 afterwards. A selection procedure that flips between 25 and 800 rounds is telling us something specific: the inner validation window (two seasons, about 30 weeks) cannot resolve the right model size, and the one-standard-error rule resolves ties toward the simplest option.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_jLSdRQVChYixzR5k-ROMA.png" /></figure><p><strong>4.7 A tempting number we are labelling as exploratory.</strong> Having seen that the large model is what helped in 2020 and 2025, we also fitted <strong>beta 3.0f:</strong> the same trees with one fixed configuration (depth 3, 8 leaves, learning rate 0.02, 800 rounds, L2 penalty 10, monotone constraints on) in every season. It improved by <strong>-0.083 [-0.125, -0.070],</strong> in 68% of weeks and in <em>every</em> season (2018 -0.131, 2019 -0.046, 2020 -0.102, 2021 -0.097, 2022 -0.095, 2023 -0.055, 2024 -0.063, 2025 -0.089), and by -0.180 on the 2026 holdout. If that were the registered result it would be a much stronger case.</p><p>It is not, and we say so loudly. The configuration was chosen <em>after</em> seeing the registered results, which is exactly the post-hoc choice the protocol exists to prevent (section 2.1). Its interval is optimistic by construction. We include it for one reason: it is the best available evidence that the registered 3.0 under-fit because of the selection rule, not because the idea is wrong. It could also be a property of this specific sample. Distinguishing those two readings requires data we do not have yet, which is the point of the shadow run.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*H36NieFKPopNN5erEzRoNg.png" /></figure><h3>5. Why it works: SHAP, and why we trust it only partly</h3><p>For tree ensembles, the standard attribution is the Shapley value (Lundberg and Lee, 2017), computed exactly and efficiently for trees by TreeSHAP (Lundberg et al., 2020). Each feature’s contribution to a prediction is its average marginal contribution over all feature orderings, and contributions sum to the prediction:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hfN0yP-mT7abr7S1USL0kw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rjv-X_FQ_0eu_zI2GnA9mA.png" /></figure><p>For beta 3.0, the mean absolute SHAP values (in poll points of correction) rank previous-poll points first (1.79), then margin of victory (1.37), raw margin (0.55), the prior rank fade term (0.31), the FPI-based fade term (0.19) and quality-loss (0.17). The sign structure matters more than the ranking: Spearman correlations between each feature’s value and its SHAP value are positive for previous points (+0.69), margin (+0.31), quality win (+0.58) and market surprise (+0.41), as the monotone constraints require. Two features are informative about what the trees do that the linear model does not. Bad-loss (+0.72) and quality-loss (+0.23) show <em>positive</em> SHAP relationships, meaning the trees partly undo the linear stack’s loss penalty. That is consistent with the segment table: the largest gains are on teams that fell or entered the poll, where voters’ reaction to a loss depends on context the linear stack applies uniformly.</p><p>This is the G7 audit in practice. The registered check asked whether margin, quality-win and market-surprise had non-negative SHAP direction (tolerance -0.05); 3.0 passed. The audit caveat is real, though: SHAP explains the model, not the world. A correct-looking attribution does not prove the trees found a causal mechanism, only that they are not using the features backwards.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*T-NAPtvYt9jINkm4_utr_Q.png" /></figure><h3>6. What the production linear model is actually using</h3><p>To compare fairly we also decomposed beta 2.6, the production forecast, in two layers.</p><p><strong>The team-level stack. </strong>Standardized ridge coefficients (points per one standard deviation) show a model dominated by one input, the voter-model forecast (+384.6), with a small supporting cast: its square (+36.8), previous-poll points (+27.0), win indicator (+29.7), being in the previous poll (+25.7), forecast times a fade term (+22.1), previous strength (+17.5), bye (+18.7), talent composite (+16.3), talent times fade (-15.5), prior rank times fade (-9.6) and quality win (+14.4). Because coefficients can mislead under correlation, we also measured <strong>permutation importance,</strong> the increase in average miss when a feature is shuffled, which is the model-agnostic complement:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/938/1*pnlfmU2UptY9IZbXaPQvzA.png" /></figure><p>By that measure the forecast input is worth +7.5 spots of miss if destroyed, and everything else is an order of magnitude smaller (squared forecast +0.18, win +0.18, previous points +0.12, bye +0.09, in-previous +0.09, talent +0.05, quality win +0.05). The stack is a <em>correction</em> on a strong first stage, which is also why a correction-style 3.0 is the right architecture to test.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*L11ivRB_EeShEy0UD3_ISQ.png" /></figure><p><strong>The voter layer.</strong> Across 2023 to 2025 (65,050 voter-team forecasts), the shares of feature movement in the pairwise-logistic voter models: team-level win (26.2%, weight +0.52, positive in 95% of voters), losses (13.0%, weight -3.02, negative in 100%), margin (6.6%), surprise (6.4%), market surprise (5.9%) and strength of schedule (5.2%). Voters are, in aggregate, unusually unanimous about what a loss means.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8RdfHuYeLqnhOnNINrc7_w.png" /></figure><h3>7. Why we are not promoting it</h3><p>By the letter of the protocol, beta 3.0 is promotable on offline evidence: every gate passed. We are not promoting it, for five reasons, in rough order of weight.</p><ol><li><strong>The effect is concentrated. </strong>Without 2020 and 2025 the improvement is -0.010, below the 0.02 threshold we pre-registered as meaningful. The registered result is a statement about the whole eight-season window; the question for a production model is whether next season looks like those two seasons or like the other six.</li><li><strong>The selection process is not settled. </strong>Choosing between 25 and 800 rounds based on two seasons of validation means we do not know what the “true” 3.0 is. The exploratory 3.0f suggests a better answer exists, but we cannot validate it using data we used to pick it.</li><li><strong>Six holdout weeks cannot carry a decision. </strong>The 2026 holdout (3.0: -0.133 [-0.240, -0.027]; 3.1: -0.033) is encouraging. It is also six weeks of one season, and the interval, while excluding zero, spans a factor of nine.</li><li><strong>Asymmetric costs.</strong> A promotion changes what readers see every Sunday. Staying on 2.6 for another half-season costs about 0.03 spots of average miss in the best case. A bad promotion costs trust. With no deadline forcing the issue, waiting is cheap.</li><li><strong>The season is not over.</strong> 2026 is the first season in which the model can be evaluated prospectively, with predictions logged before polls are released. By late April 2027 we will have a full season of data that neither selected nor tuned anything.</li></ol><p>None of this says the nonlinear idea is wrong. It says the evidence is of the kind that justifies <em>continuing to collect evidence</em>.</p><h3>8. Shadow mode and the pre-committed review</h3><p><strong>What shadow mode is.</strong> Every week the refresh pipeline now fits beta 3.0 and the fixed-configuration variant, scores them against the same polls as beta 2.6 and appends the forecasts to a frozen log (data/shadow3-log.json). Nothing appears on the site. The log is append-only and each entry is written when a poll is ingested, using only information available to the walk-forward replay at that point. The honest limit: entries are written when the poll is ingested, not before it is released, so they are walk-forward rather than strictly prospective.</p><p><strong>The decision rules,</strong> written down now. In late April 2027 an automated review reads the log and the 2026 results and applies five rules.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/479/1*3pebmu4Fqn3O5mxKsLLdNQ.png" /></figure><p>Outcomes are promote, continue shadow, or archive. The default is to keep beta 2.6. A backup review is scheduled two weeks later in case the first does not run. Each rule is a response to something in this article: R1 is the replication we cannot do on the same data, R2 is the concentration problem, R3 is the six-week holdout, and R5 is the SHAP audit.</p><h3>9. Limitations</h3><ul><li><strong>Sample size.</strong> 104 scored weeks over eight seasons, 15 polls per season. The effective sample is smaller because weeks within a season are correlated.</li><li><strong>Researcher degrees of freedom.</strong> The tuning era (2015 to 2019) was used to design the plan, and the registered fits were run once. Several decisions (the exploratory 3.0f, the analysis excluding 2020 and 2025) were made after seeing results, and we have labelled them.</li><li><strong>One sport, one model family.</strong> We tested boosted trees and LambdaMART. Neural networks and hierarchical nonlinear voter models were not run. Football is the only sport where this was tested.</li><li><strong>Stationarity.</strong> The 2024 expansion of the playoff to 12 teams changed voter incentives. The 2024–25 era shows -0.057, larger than the average, which could mean the trees adapt better, or could be noise.</li><li><strong>Walk-forward is not prospective.</strong> Shadow entries are made on ingestion; strictly pre-release logging would require changing the pipeline.</li></ul><h3>10. Check It Out Yourself (Coming for 2027 Season)</h3><p>These models underpin our app <a href="https://poll-position.robert-mepham1309.workers.dev/#/poll">AP Top 25 · Poll Position</a>. Version Beta2.6 is live on the site right now, with a potential deployment of the non-linear v3.0 models coming in April 2027. I’ll write another article at that time if the 2026 evidence for non-linearity in the modeling of the new-era AP Top 25 poll justifies its promotion to production. Feel free to check it out, investigate the rankings, the model, the forecasts, its residual reports, and any other ways you can use the app to make yourself a better informed CFB fan! We’re always open to feedback!</p><h3>References</h3><blockquote>Friedman, J. H. (2001). Greedy function approximation: a gradient boosting machine. <em>Annals of Statistics</em> 29(5).</blockquote><blockquote>Burges, C. J. C. (2010). From RankNet to LambdaRank to LambdaMART: an overview. Microsoft Research TR-2010–82.</blockquote><blockquote>Cawley, G. C. and Talbot, N. L. C. (2010). On over-fitting in model selection and subsequent selection bias in performance evaluation. <em>JMLR</em> 11.</blockquote><blockquote>Varma, S. and Simon, R. (2006). Bias in error estimation when using cross-validation for model selection. <em>BMC Bioinformatics</em> 7.</blockquote><blockquote>Bergmeir, C. and Benitez, J. M. (2012). On the use of cross-validation for time series predictor evaluation. <em>Information Sciences</em> 191.</blockquote><blockquote>Cerqueira, V., Torgo, L. and Mozetic, I. (2020). Evaluating time series forecasting models. <em>Machine Learning</em> 109.</blockquote><blockquote>Kuensch, H. R. (1989). The jackknife and the bootstrap for general stationary observations. <em>Annals of Statistics</em> 17(3).</blockquote><blockquote>Diebold, F. X. and Mariano, R. S. (1995). Comparing predictive accuracy. <em>J. Business and Economic Statistics</em> 13(3). Harvey, D., Leybourne, S. and Newbold, P. (1997). Testing the equality of prediction mean squared errors. <em>Int. J. Forecasting</em> 13(2).</blockquote><blockquote>Holm, S. (1979). A simple sequentially rejective multiple test procedure. <em>Scandinavian J. Statistics</em> 6(2).</blockquote><blockquote>Lundberg, S. M. and Lee, S.-I. (2017). A unified approach to interpreting model predictions. <em>NeurIPS</em>. Lundberg, S. M. et al. (2020). From local explanations to global understanding with explainable AI for trees. <em>Nature Machine Intelligence</em> 2.</blockquote><blockquote>Gupta, M. et al. (2016). Monotonic calibrated interpolated look-up tables. <em>JMLR</em> 17.</blockquote><blockquote>Lou, Y., Caruana, R., Gehrke, J. and Hooker, G. (2013). Accurate intelligible models with pairwise interactions. <em>KDD</em>.</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b40e16dceb19" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[In a 24-Team Playoff, Almost Nobody Is Out In Early-November]]></title>
            <link>https://medium.com/@robert.mepham/in-a-24-team-playoff-almost-nobody-is-out-in-early-november-a0909b90bc52?source=rss-745e2a58a3bf------2</link>
            <guid isPermaLink="false">https://medium.com/p/a0909b90bc52</guid>
            <category><![CDATA[sports-data-science]]></category>
            <category><![CDATA[college-football]]></category>
            <category><![CDATA[college-football-playoffs]]></category>
            <category><![CDATA[sports-analytics]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Rob Mepham]]></dc:creator>
            <pubDate>Sat, 07 Nov 2026 14:01:01 GMT</pubDate>
            <atom:updated>2026-11-07T14:01:01.788Z</atom:updated>
            <content:encoded><![CDATA[<h3>In a 24-Team Playoff, Almost Nobody Is Out in Early-November</h3><p>The AP poll after Week 10 reads Miami, Georgia, Utah, Notre Dame, UCLA, Indiana. Three of those six are 8–0, and another is 8–1. And then there’s LSU: 6–2, 15th in the poll, and the third most likely team in the country to win the national title.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8SgEe8ByPacppxyXr39mIQ.png" /></figure><p>That is the thing I want to look at this week. Poll Position now has a playoff model that plays out the rest of the season 40,000 times, so I can ask a better question than “who’s in the top four?”: <strong>who is actually likely to be in the field, who wins it, and what would the whole picture look like if the playoff were twice as big?</strong></p><p>The second question is not hypothetical for long. More on that below.</p><h3>The Field, Right Now</h3><p>Here are the 24 teams most likely to make the 12-team field after Week 10, 266 games from the end of the regular season. The green bar is the chance of making the 12-team field; the blue extension is the chance under a 24-team field (assumed format, explained in the second half).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AwjS7yDJb8J5mYFRvKAQMw.png" /></figure><p>A few things jump out.</p><p><strong>It’s nowhere near settled.</strong> Exactly one team, Miami at 95%, is above a 90% chance of making the 12-team field. Twenty-one teams sit between 10% and 90%. Nine weeks into a season with one real committee ranking still to come, the field is mostly undecided. That is not model hedging, it’s math: eleven teams are at 50% or better and there are only twelve spots, so the bubble is basically everyone’s problem.</p><p><strong>The unbeaten teams aren’t equal.</strong> Miami (8–0) is 95%. Utah (8–0) is 88%. UCLA (8–0) is 79%. All three are undefeated, and the model still separates them by sixteen points. It does that because it isn’t just counting losses: it asks how good each team is, who’s left on the schedule and how the committee has treated resumes like theirs.</p><p><strong>The AP poll and the model disagree about a few teams. </strong>Wake Forest (7–1) is 14th in the poll and 56% to make it. Boise State (7–1) is 20th and 45%. LSU is 15th at 6–2 and 63%, equal to Alabama, which is 9th with the same record. The model is giving LSU the benefit of its rating and its schedule. The poll is giving it the penalty of two losses.</p><h3>Who Wins it All</h3><p>The title odds are even more lopsided than the field odds, and they hold the week’s biggest surprise.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2_No0qML8n7fBSNAJULrng.png" /></figure><p>Miami is the clear favorite at <strong>18.8%,</strong> which is high for a team that still has to get through a quarterfinal and a semifinal and a final. Notre Dame is next at <strong>12.8%</strong>. And third, at <strong>12.4%</strong>, is LSU, a 6–2 team the AP voters have in 15th.</p><p>The reason is that a title is two things multiplied: the chance of making the field, and the chance of winning the bracket once you’re in. LSU’s first number is only 63%. Its second, about 20% given that it gets in, is level with Miami’s, and higher than Notre Dame’s (17%) or Alabama’s (14%), because the second number depends on how good you are, not on how many games you lost on the way.</p><h3>The Efficiency Frontier Says the Same Thing</h3><p>I added a new lens to the Ratings page recently: plot every team by how well its offense does (opponent-adjusted EPA per play, across) against how well its defense does (EPA allowed per play, up), and draw a line through the teams that nobody beats on both. Economists call it an efficient frontier. In this season it connects five teams.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AkTfXKJsst_1tx0nI77fLg.png" /></figure><p><strong>UCLA</strong> (the best defense in the country), <strong>LSU, Indiana, Notre Dame and Miami</strong> (the best offense by a wide margin) are on it. Utah, Georgia, Alabama and Kentucky, all ranked higher in the poll than at least one of those teams, are inside it.</p><p>I tested the idea on history before putting it on the page. At the last poll before the bowls, <strong>8 of the 11 national champions from 2014 to 2024</strong> were exactly on this line, and the other three had only one or two teams ahead of them on both measures. That is a pattern in eleven seasons, not a law, and the frontier is not an input to the model: I’m using it as a second opinion. But it’s a good second opinion. The frontier has LSU on it. So does the title-odds chart.</p><p>One nuance: “on the frontier” doesn’t mean “balanced.” Miami’s defense is below average (it allows 0.085 more adjusted EPA per play than a typical defense). It’s on the line because its offense is so far out in front that nobody can match it.</p><h3>The Games That Decide It</h3><p>Here are the eight games over the next three weeks that move the most field spots. “Importance” counts how many playoff places change hands between the two outcomes, then weights it by how close the game is: a 90–10 game moves things less because it rarely matters.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*z9n5_LTemDKQMicf5I01pg.png" /></figure><p><strong>Alabama-LSU on November 7 is the biggest game on the board.</strong> If Alabama wins, it is 81% to make the field and LSU 46%. If LSU wins, it flips: LSU 74%, Alabama 52%. That is a combined 57 points of probability riding on one afternoon.</p><p><strong>Georgia-Ole Miss</strong> is the same day and almost as big (Ole Miss 32% if it loses, 58% if it wins). Further down, <strong>Kentucky-Tennessee</strong> that same Saturday moves Kentucky from 44% to 67% and Tennessee from 12% to 29%.</p><p>Notice the pattern: the most important games are between teams in the middle of the table. A game between a lock and a long shot barely moves the field. A game between two 50%-ish teams moves everything.</p><h3>Now Double the Field</h3><p>On October 15, ESPN’s Pete Thamel reported that the College Football Playoff is <a href="https://www.wlox.com/2026/10/15/college-football-playoff-nearing-deal-expand-24-teams-2027-reports-say/">closing in on a deal to expand to 24 teams starting in 2027</a>, pending final approvals. <a href="https://www.si.com/college/michigan/cfp-set-to-expand-to-24-teams-opening-the-door-to-more-playoff-access-and-mismatches">SI summarized the same report</a>. Per the reports, an officialdecision on 2027 is due by December 1, but it’s all but a foregone conclusion by this point. The structure, as reported: <strong>the four Power 4 conference champions, the highest-ranked Group of Six team, and 19 at-large teams.</strong> Seeding, byes and the bracket have not been released.</p><p>I can’t tell you how 2027 will go. What I can do is run <strong>this </strong>season through that structure, leaving the rest of the model alone: same ratings, same committee score, same simulated games, and a field of 24 instead of 12. This is a thought experiment, not a forecast of next year’s playoff, and a 24-team model can’t be backtested the way the 12-team one could.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nVCcbxhTo8W_3pLxdIGW8Q.png" /></figure><p><strong>The bubble nearly doubles.</strong> Teams with between a 10% and a 90% chance go from 21 to 37. The field doubles and the number of teams still fighting for it goes up by 76%.</p><p><strong>More teams are safe, too.</strong> Teams at 90% or better go from one to <strong>eight</strong>. Teams at 50% or better go from 11 to 21. So there are more locks <em>and</em> a bigger bubble. The middle of the table is the place where seasons get decided.</p><p><strong>Every power conference roughly doubles its expected seats.</strong> In simulated terms the SEC goes from 3.8 teams in the field to 7.5, the Big Ten from 2.3 to 5.0, the Big 12 from 2.3 to 4.5, the ACC from 1.6 to 2.9.</p><p><strong>The Group of Six triples.</strong> The non-power leagues together (including the rebuilt Pac-12) go from an expected 1.1 teams, essentially the one guaranteed slot, to 3.1. That’s the finding I think is most interesting, because of what the 12 years of committee data say: the committee has consistently ranked Group of Six teams lower than their results alone suggest, and in a field of 12 that discount cost them a bid. In a field of 24, the cutoff reaches far enough down that the discount stops being fatal.</p><h3>Who This Helps</h3><p>The teams that gain the most are the ones sitting just outside the 12:</p><ul><li><strong>Texas</strong> (6–2, #10 in the AP): 38% to 82%</li><li><strong>Tennessee</strong> (6–2, #16): 23% to 66%</li><li><strong>Houston </strong>(6–2, #17): 28% to 70%</li><li><strong>Missouri </strong>(6–2, #19): 17% to 57%</li><li><strong>Ole Miss, Texas Tech, Ohio State:</strong> each a plus of roughly 35–40 points.</li></ul><p><strong>Liberty </strong>(7–1, unranked) goes from 21% to 51%. <strong>Northwestern </strong>(6–2, unranked) from 10% to 44%. Five AP-ranked teams are still below even odds at 24: Arizona 48%, Iowa 36%, Florida 33%, New Mexico 28%, South Carolina 24%.</p><h3>What it Does to the Regular Season</h3><p>The conventional argument against a 24-team field is that it drains the regular season of meaning. The model’s version of that argument is not as simple as I expected. The biggest game on the board loses a lot: Alabama-LSU is worth <strong>half </strong>as much under 24 teams (importance 33 down to 16) because both teams would very likely be in anyway. But the games in the middle keep their weight. Kentucky-Tennessee is worth about the same either way, and for Tennessee it moves the field odds from <strong>54% if it loses to 73% if it wins,</strong> against 12% and 29% now.</p><p>So the picture is: <strong>marquee games matter less, bubble games matter just as much,</strong> and the bubble is nearly twice as crowded. If you are Tennessee, the November 7 game is still the biggest game on your schedule. If you are Alabama, it isn’t, anymore.</p><h3>What I’d Watch</h3><ol><li><strong>Tuesday night: the first real committee rankings.</strong> The model doesn’t use them yet, but I’m collecting them from this week on, and the first set is a good test of the committee weights. If the committee has Wake Forest or Boise State well above where the AP does, the “AP says no, results say yes” picture gets confirmed. If it has LSU low, it is weighing the two losses more than the model does.</li><li><strong>The format details. </strong>Whether the top four conference champions or the top ranked teams get byes, and whether conference champions keep guaranteed spots, will change a lot of these numbers. I used the structure from the October 15 reports and nothing more.</li><li><strong>LSU at Alabama, Nov 7.</strong> That’s the game I’d circle. In 40,000 simulated seasons, its two possible outcomes are the single largest swing in the whole 2026 field.</li></ol><p><em>All of the numbers above are live on my new app: </em><a href="https://poll-position.robert-mepham1309.workers.dev/#/playoff"><em>Poll Position | Playoff</em></a><em> : pick a result for any game and watch every team’s odds update.</em></p><blockquote><strong>Methodology Used for the Above Analysis</strong></blockquote><blockquote><strong>Model:</strong> The Poll Position playoff simulation as of the Week 10 poll (266 unplayed regular-season and conference-title games; 4 polls before the final ranking), 40,000 simulated seasons, seed 2026. Field rule: four Power 4 champions + the highest-ranked Group of Six team + the rest by committee score. Method, audit and the committee-noise term: article 12 (docs/PLAYOFF.md).</blockquote><blockquote><strong>24-team field:</strong> the same simulation with the field size set to 24 and everything else unchanged: four Power 4 champions + the highest-ranked Group of Six team + at-large teams by committee score to 24. This follows press reports (Oct. 15, 2026) of a deal that is not final. Seeding, byes and the bracket are not modeled, so only “makes the field” is reported for 24 teams. The model has no backtest for this format.</blockquote><blockquote><strong>Group of Six here includes the five leagues outside the four Power 4 conferences:</strong> the American, Conference USA, Mountain West, Sun Belt and the 2026 Pac-12.</blockquote><blockquote><strong>Bubble / lock counts:</strong> teams with a chance strictly between 10% and 90% / at or above 90%.</blockquote><blockquote><strong>Game importance:</strong> for each unplayed game, the sum over teams of |P(makes the field | team A wins) − P(makes the field | team B wins)| ÷ 2 (field spots that change hands), times 4·p·(1−p), where p is the pre-game win probability (so a toss-up counts fully and a near-certainty barely counts). Computed from the same simulated seasons; the 24-team column uses the 24-team simulation for the same games.</blockquote><blockquote><strong>Efficiency frontier:</strong> teams not beaten on both opponent-adjusted offensive EPA per play and defensive EPA allowed per play; Indiana is within a hair of LSU on both and is shown on the line. Historical check (tools/playoff/check-frontier.js): 8 of 11 champions 2014–2024 on the frontier at the last poll before the bowls, the other 3 with one or two teams ahead on both axes.</blockquote><blockquote><strong>Not seen:</strong> injuries, quarterback changes, tiebreaker details inside a conference, anything the committee weighs beyond results and the AP poll. The first committee rankings of 2026 are not used.</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a0909b90bc52" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Inside Poll Position AI: Hierarchical Shrinkage, Asymmetric Calibration, and a Stacked Ridge Model…]]></title>
            <link>https://medium.com/@robert.mepham/inside-pole-position-ai-hierarchical-shrinkage-asymmetric-calibration-and-a-stacked-ridge-model-237c1cfc22e8?source=rss-745e2a58a3bf------2</link>
            <guid isPermaLink="false">https://medium.com/p/237c1cfc22e8</guid>
            <category><![CDATA[college-football]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[linear-models]]></category>
            <category><![CDATA[ap-top-25]]></category>
            <category><![CDATA[statistics]]></category>
            <dc:creator><![CDATA[Rob Mepham]]></dc:creator>
            <pubDate>Fri, 06 Nov 2026 14:46:01 GMT</pubDate>
            <atom:updated>2026-11-06T14:50:32.032Z</atom:updated>
            <content:encoded><![CDATA[<h3>Inside Poll Position AI: Hierarchical Shrinkage, Asymmetric Calibration, and a Stacked Ridge Model for the AP Top 25</h3><p>Every week of the season, about sixty sportswriters turn a ranking of 25 teams into one number each: points. The AP adds the ballots, and the poll comes out. Our model tries to predict that poll <em>after the weekend’s games and before the voters submit</em>. It is a small forecasting problem with a few unusual properties: the “population” is a panel of about sixty named people with persistent habits, the target is a rank aggregate rather than a score, the sample is tiny (15 polls a year), and the world is non-stationary (the playoff went from 4 to 12 teams in 2024).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XmmjW01OcDdyf1YE8UHnBw.png" /></figure><p>This article walks through how the model got from version 1.0 to 2.6: what we started with, every statistical idea we tried, what worked, what did not, and the evidence for each claim. It is deliberately technical. If you want the light version, the app ([APP_URL]) has a Forecast page where you can type scores and watch the poll move.</p><p><strong>The headline, with its error bars.</strong> Over 142 regular-season weeks (2015 to 2025), the average miss, the number of spots each of the real Top 25 teams was off, fell from 1.51 (beta 1.0) to 0.94 (beta 2.6), against 2.37 for the naive forecast “nothing changes”. Most of that came from three ideas: letting the model react to <em>this week’s</em> results (-0.32), correcting how it scales rises versus falls (-0.10), and adding a team-level stage on top of the voter-level models (-0.05 to -0.06). Several things we expected to help did not, and I will say so.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dAlnLJKIjZgV6YH9Tmt9_A.png" /></figure><h3>1. The problem and the scorecard</h3><p><strong>Setup.</strong> At poll t there are V_t voters (about 60). Voter v submits an ordered list of 25 teams. A team’s poll points are</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zVaooQ8aziYJ4nqRInhbuA.png" /></figure><p>where r is the voter’s rank for the team (unranked teams get nothing). The poll orders teams by P, breaking ties by first-place votes. We observe every voter’s ballot, which is the unusual part: it lets us model the <em>voters</em> and then aggregate, rather than model the aggregate directly.</p><p>The metric. For a forecast ranking r̂ of poll t, the headline is</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3COxB70CbpJeW96M9pqKrQ.png" /></figure><p>where the sum runs over the 25 teams in the <em>real</em> poll and a team we left out of our top 25 counts as 26th. The baseline is the previous poll carried forward under the same rule (2.37). I report within-one-spot counts, exact hits, whether №1 was right, first-place votes misallocated, and points error as well, but the average miss is the number every decision below uses.</p><p><strong>The protocol.</strong> Everything is walk-forward. A forecast of poll t+1 in season Y uses only ballots from earlier seasons and ballots up to poll t of season Y, plus results and ratings available before the poll. The harness replays all 142 regular-season weeks from 2015 to 2025 (2014 has no public ballots; the post-bowl poll is excluded). 2026 to date is a locked holdout: scored once per version, never used to pick anything. Comparisons between versions are <em>paired</em>: the difference in average miss over the same weeks, with a 95% percentile interval from a bootstrap that resamples weeks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/994/1*qWNb5ZpSyLczav_yTcwawQ.png" /></figure><p>Two honest caveats about those intervals. Weeks within a season are not independent (a team’s fortunes persist), so the intervals are somewhat optimistic. And with 142 weeks, “significant” gaps of 0.01 to 0.02 spots are within what a handful of tuned variants can produce by chance; I flag those below. For tuning we also used a five-season panel (2016, 2018, 2020, 2022, 2024; 64 weeks) because a full replay of every variant was slow. That choice turned out to matter (section 7).</p><h3>2. Where we started: beta 1.0</h3><p>Beta 1.0 models each voter separately with a Bradley-Terry style logistic regression on pairs of teams:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*USDOMEytrzPr8ylExuQZbg.png" /></figure><p>where x_i is a vector of 15 team-level features: the voter’s own last-ballot placement and preseason placement, wins, losses, average margin, quality wins (over ranked teams), bad losses (to unranked teams), indicators for the SEC, Big Ten, Big 12, ACC, independents and Group of Five, “own school” and “own conference” flags, plus a pairwise head-to-head term. The weights solve a ridge-regularized logistic problem over that voter’s ballots in the <em>current season</em> through the latest poll:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1BRKP39fT8ivvfzdJTJRQA.png" /></figure><p>Training pairs for one ballot come from (a) teams within four spots of each other, (b) teams eight apart, and © the last few ranked spots against unranked contenders, about 200 pairs per voter per week. We fit with Adam (learning rate 0.12, 120 iterations) and ridge λ = 0.004.</p><p><strong>Forecasting a ballot.</strong> A pure “score every team and sort” forecast ignores the fact that ballots are mostly last week’s ballot. So beta 1.0 anchors on it. Take the voter’s previous ballot, compute each listed team’s score w^T x at the previous poll, sort those scores, and give the k-th best score to the team the voter had at position k. Then add the <em>change</em> in score caused by the new results:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ylPWyjIF5mK-7XBKuJFZtQ.png" /></figure><p>Re-sort, apply a small head-to-head swap pass, and tally the 25-to-1 points over voters. This anchor-plus-delta design is the reason habitual voters barely move while result-driven voters move a lot, and it is the single most important structural choice in everything that follows.</p><p><strong>The diagnosis.</strong> Beta 1.0 scored 1.51. Its movement ratio, the total spots our forecast moved teams divided by the total the real poll moved them,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/689/1*2jIiyuHd8CXZbcw904P5gw.png" /></figure><p>was <strong>0.56</strong>. It under-reacted by almost half. The reason is structural: wins, losses and average margin are season totals. One more game changes them a little, the weights are cross-sectional (fit to explain who is ranked above whom), so a result’s effect on the score is tiny compared with how hard voters actually punish a loss or reward an upset.</p><h3>3. Beta 2.0: let the model see this week</h3><p>The fix was to add features that describe <em>what just happened</em> in context, and to be careful about how they enter the forecast. Thirteen new features, in four groups:</p><ul><li><strong>This week’s result:</strong> win/loss, capped margin, beat a team ranked at kickoff, lost to a ranked team, a ranked team losing to an unranked one, road win, upset win over a higher-ranked team, and a surprise term.</li><li><strong>Strength:</strong> an Elo-style margin rating (below) and strength of schedule.</li><li><strong>Prior:</strong> last season’s final AP rank.</li><li><strong>Chaos:</strong> the number of ranked teams that lost that week, interacted with a ranked loss and an upset win.</li></ul><p>The rating is an Elo variant on point margin:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/962/1*MwvjZQLE7Cc9by_txPi0uQ.png" /></figure><p>with m the margin clipped to ±28, k = 0.07, home advantage h = 2.5, and ratings multiplied by 0.7 at each season start. The “surprise” feature is the result against what the rating expected:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kDQKeX0j1M-4ExxHWTSwZQ.png" /></figure><p><strong>A subtle point that cost us a bad afternoon.</strong> The first implementation applied the anchor-plus-delta rule to every feature: add w^T(x_new minus x_base). It over-reacted (movement ratio 1.04) and got <em>worse</em> on a 2019 smoke test (1.51 vs 1.15). The reason: a “this week” feature at the previous poll describes last week’s game, and the voter’s previous ballot already priced it in. A team on a bye would “lose” last week’s win credit. The right treatment separates state features (wins, Elo, which accumulate: add the change) from event features (this week’s result: add only the new week’s value):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zqfUeYrUskrlc2jD0ulqEg.png" /></figure><p>With that fixed, 2019 went to 1.05. Removing even 30% of last week’s event effect (a decay parameter) hurt: 1.41 vs 1.26 on the tuning panel. Ablating groups on that panel (beta 1.0 scored 1.64): this-week features alone 1.37, adding chaos 1.36, strength alone 1.53, prior alone 1.62, all four <strong>1.26</strong>. A ridge sweep favoured <em>less</em> regularization than beta 1.0 used (λ = 0.004: 1.26; 0.0005: 1.25; <strong>0.001: 1.24</strong>).</p><p><strong>Result:</strong> 1.51 to 1.19 over 142 weeks, a paired change of -0.32 (95% CI -0.38 to -0.26). The movement ratio went from 0.56 to 0.94.</p><h3>4. An error analysis before building anything else</h3><p>With a model that was already good enough to be useful, we stopped adding features and asked where the remaining error lived. We replayed beta 2.0 with per-team detail (3,552 real top-25 team-weeks) and cut the error every way we could think of.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7fGUT6g7GjF7kvrubYLOLw.png" /></figure><p>What the cuts said:</p><ul><li><strong>The extremes were under-moved.</strong> Projected movement was 78% of real movement. Teams that rose four or more spots were projected 1.31 spots too low; teams that fell four or more were projected 1.20 too high.</li><li><strong>Early weeks are the hardest</strong> (1.79 in weeks 1 to 3, 0.85 in weeks 11 to 14), and voter-level gains over “same ballot as last week” were only 13% in weeks 1 to 3 against 34% later.</li><li><strong>Teams that lost were placed worst</strong> (a loss to an unranked team: average miss 1.94; to a ranked team: 2.00), and teams on a bye were put 0.87 spots too low.</li><li><strong>№1 was right 87% of the time,</strong> and in 13 of the 18 misses the previous №1 stayed №1 and the model flipped it.</li><li><strong>Not weak spots:</strong> wild weeks (many ranked losses) and the 2024 to 2025 playoff era (1.15) were not worse.</li></ul><p>That became the plan: pool data to help early weeks (2.1), fix the movement scale and the top of the poll (2.2), then deal with whatever was left.</p><h3>5. Beta 2.1: hierarchical pooling</h3><p>Every voter’s model was trained only on their own ballots from the current season. In week 3 that is about 600 pairs for a 16-dimensional problem, and it throws away ten seasons of the same person’s behavior. Beta 2.1 fits a population model and a per-voter model shrunk toward it, a standard empirical-Bayes structure:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sLwf6BUTuQQG28cEAB-3Ig.png" /></figure><p>with ρ = 0.85 decay per season, γ = 1 (2024 onward counts double, because the 12-team playoff may have changed ballot behavior), and a fade φ that makes earlier seasons matter in week 1 and vanish by midseason:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6WRIyYiYv1ZOzgZR05y-Kg.png" /></figure><p>Then each voter’s weights are pulled toward the population weights rather than toward zero:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-3t4kCeMD5KXs0GtDcr3Tw.png" /></figure><p><strong>Result: </strong>1.19 to 1.14 (paired -0.051, CI -0.083 to -0.020), but only a minority of weeks improved (46%). Early weeks went from 1.62 to 1.47. The honest finding is that pooling was <em>not</em> the big lever we expected: every setting of κ (0.005 to 1), earlier-season weight, decay and era boost landed between 1.21 and 1.27 on the panel. The voter models were not as data-starved as weeks 1 to 3 suggested. And the fade matters: without it, old rows helped weeks 1 to 3 (1.67 to 1.59 on the panel) but hurt weeks 6 to 10 (1.14 to 1.18). One side effect worth noting: №1 right slipped to 85%, as pooling pulls voters toward the crowd and makes close top-three calls noisier.</p><h3>6. Beta 2.2: the calibration that fixed the top of the poll</h3><p>The error analysis said the model moved teams 78% as far as voters did, so the obvious fix was to stretch its score changes. That makes things <em>worse</em>, and the reason is a classic one worth spelling out.</p><p><strong>Why a slope of 0.78 is not under-reaction.</strong> Suppose the forecast move m̂ is perfectly calibrated: the real move is m = m̂ + u, where u is voters’ idiosyncratic behavior, uncorrelated with m̂. Then regressing the <em>forecast</em> on the <em>real</em> move gives</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dKt1Rbu8xYuVTKiEuiyqaA.png" /></figure><p>which is below 1 even for a perfect forecast. The slope of real on forecast (the one that tests calibration) can be exactly 1 while the slope the other way is 0.78. So a uniform stretch overshoots.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_imJuWTXxMzkB55OPYR4Xg.png" /></figure><p>On the tuning panel a uniform stretch of 1.2 raised the average miss from 1.24 to 1.35, 1.4 to 1.56, 1.7 to 1.93. But rises and falls behave differently. Normalize each voter’s score change by the typical spacing between adjacent ranks on their ballot, then scale positive and negative changes separately:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fG4OKm4j90ydCg9Gg6KjfA.png" /></figure><p>The best region was broad: a₊ between 0.5 and 0.7 and a₋ between 1.1 and 1.2 were all within 0.02 of each other; shutting rises off entirely was clearly worse (1.33). We use a₊ = 0.6, a₋ = 1.2. In plain terms: <strong>voters reward a win less than a raw score model says and punish a loss more,</strong> a loss-aversion pattern. The b term is a small stickiness bonus (one rank spacing) for the sitting №1.</p><p>Because we tuned two parameters on five seasons, we checked on the six seasons that were <em>not</em> in the panel: average miss 0.955 for beta 2.2 against 1.146 for beta 2.0 and 1.405 for beta 1.0, so the gain is not a panel artifact.</p><p><strong>Result: </strong>1.14 to 1.03 (paired -0.105, CI -0.139 to -0.070). №1 right went from 85% to 94% and first-place votes misallocated from 16.5 to 10.0 per week. We also tried averaging many sampled ballots (adding noise to scores) to produce expected points instead of a hard sort: at best -0.01 for twelve times the compute. Not worth it.</p><h3>7. Beta 2.3: listwise training, and the winner’s curse</h3><p>Beta 2.1 and 2.2 train on pairs, which optimises the order of pairs rather than the ballot. A listwise alternative is the Plackett-Luce likelihood: at each ballot position, the team the voter put there must beat everything below it, plus a few unranked contenders:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hBa9WF2Tz6UnjBCKnuyRjQ.png" /></figure><p>We trained it with the same pooling and shrinkage. Every variant (position decay, shrinkage strength, head-to-head scale, 48 versus 96 lists per voter, 6 versus 12 contenders) landed at 1.11 to 1.13 on the panel, the same as the pairwise model. On all 142 weeks: <strong>1.04 vs 1.03, paired +0.011 (CI -0.012 to +0.035),</strong> no difference. Listwise training is the more principled objective, but the voter models were never limited by the loss function. We kept it as the base for the next step because it is no worse and it optimizes what we care about.</p><p>This version is also where we learned the most important statistical lesson of the project. We added outside data (ESPN’s FPI from the previous season, 247Sports recruiting classes, the transfer portal, the CFP committee’s rankings) and screened each group on the tuning panel. FPI plus recruiting plus transfers looked real: <strong>-0.029, interval -0.054 to -0.005.</strong> We then replicated the same idea on all 142 weeks: <strong>-0.002, interval -0.029 to +0.026.</strong> The panel result was a winner’s curse: choose the best of several variants on a small sample, and the winner’s estimate is biased upward. After that, anything that looked good on the panel had to survive the full replay before it earned a place.</p><h3>8. Beta 2.4: a team-level model stacked on the voter models</h3><p>Anchor-plus-delta can only move a team <em>relative to where the voters already had it</em>. That is great for dynamics but means level information (how good is this team, what did the committee say, what do the betting markets think) cannot enter. Beta 2.4 adds a second stage: a pooled ridge regression of each team’s next-poll points on the voter-model forecast and level and event features.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dhlmasYT6wxPfemvsMfx1g.png" /></figure><p>The inputs are the voter-model forecast P̂ (plus its square and an early-season interaction), last poll’s points and rank, whether the team was in the last poll, this week’s result (win/loss, margin, bye, quality win, quality loss, bad loss), Elo, and the outside signals. It is trained <strong>walk-forward on the out-of-sample forecasts the model made for every earlier week </strong>(it needs 500 candidate rows, about the first ten weeks of 2015), so it never sees the future. One more design decision: the unconstrained stack scored well on average miss but dropped №1 right from 93% to 87%, so we hold №1 as the voter models chose it and let the stack order ranks 2 to 25.</p><p><strong>Result:</strong> 1.04 (beta 2.3) to 1.00, paired -0.047 (CI -0.073 to -0.022); in early weeks -0.063 against beta 2.2 (CI -0.113 to -0.013). Ridge strength 1, 3 and 10 gave the same answer.</p><p><strong>How much of that is outside data? </strong>Little. A stack with only last season’s AP rank as its level signal scored 0.99; with CFP, FPI, recruiting and transfers 0.97 (paired -0.016, CI -0.032 to -0.001). One at a time, nothing was significant: CFP -0.008 (it helps late weeks, where the committee’s rankings exist, by -0.027, CI -0.063 to +0.007), FPI -0.001, recruiting -0.005, transfers -0.003. Early weeks did not move at all (1.27 with or without), which is evidence <em>against</em> the idea that recruiting, transfers and last year’s strength drive early-season ballots beyond what last year’s AP rank and Elo already carry. Most of the stack’s gain is the structure (level + event + forecast together), not any one data source.</p><h3>9. Beta 2.5 to 2.7: market, SP+, and returning production</h3><p>With a CollegeFootballData key we could add features that are defensible (all known before the poll being forecast) and that capture <em>expectations</em>:</p><ul><li><strong>SP+ </strong>(the previous season’s final rating), <strong>returning production</strong> (preseason share of last year’s PPA and usage that returns), and the <strong>247 talent composite,</strong> each z-scored and faded as the season arrives: z · φ(w) with φ(w) = 3/(3+w), so the signal carries a delta in the anchor-plus-delta frame. (A same-season year-end SP+ would leak the future, so it is never used; CFBD has no weekly SP+ history.)</li><li><strong>The betting market.</strong> Two features from closing lines: the result against the line (actual margin minus the market’s expected margin, an event feature like “surprise” but from the market rather than our own Elo), and a <em>market rating</em>, solved each week from every closing spread so far:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LSjGJoXi3aODb8pZ1rsO-w.png" /></figure><p>where s_g is game g’s mean spread over providers, h = 2.5, m_ig is the market margin for the pair, and r⁰ is last season’s rating decayed by 0.7. We solve it by Jacobi iteration with λ = 1.5.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*r9EcTNovGfgy7AJklhQZoA.png" /></figure><p><strong>Beta 2.5</strong> (these four groups in the voter models on top of beta 2.2): 1.00, paired -0.031 vs beta 2.2 (CI -0.052 to -0.009); early -0.051 and mid -0.041, both significant; late -0.001. The market features alone were not significant over 142 weeks (-0.017, CI -0.037 to +0.004) even though they were on the panel; SP+, returning production and talent drive the early and mid-season gain. In-season EPA per play, which we also tried, added nothing to the voter models. The fade half-life (3, 6 or 12 weeks) made no difference.</p><p><strong>Beta 2.6 </strong>(beta 2.5 plus the beta 2.4 stack, now fed the CFBD signals): 0.94 (95% CI 0.87 to 1.03), paired -0.058 against beta 2.5 and -0.089 against beta 2.2. Against beta 2.2 the early-week gain is -0.113 (CI -0.174 to -0.054).</p><p><strong>Beta 2.7</strong> adds season-to-date EPA to the stack: 0.93, paired -0.013 (CI -0.024 to -0.002). That is a small gain spread thinly (better in only 27% of weeks), needs a weekly API pull, and is inside what a few tried variants can produce by chance, so we treat it as marginal and run beta 2.6.</p><h3>10. The scoreboard</h3><p>The pipeline that ships is below: offline training, a frozen weekly log of every voter’s weights, and a per-season pack the browser applies. The stack is the last stage.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DAzmU_Bzqv85aIle7fDeug.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*T60DXP-WrVbhFLTXXOfaCA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*n_n1KLJWZ24OVmvNincGGg.png" /></figure><p>Three things to notice. <strong>First,</strong> the biggest single improvement by far was the step that gave the model access to <em>this week</em>; everything after is smaller, which is the usual shape of a modelling project. <strong>Second,</strong> by stage of the season, every version is worst early and best late, and the gap narrows with each step but does not close:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oA6jmnboJHYatwnfOPpjJg.png" /></figure><p><strong>Third,</strong> the 2026 holdout (6 weeks) scored 2.12 for beta 1.0 and 1.49 to 1.65 for versions 2.4 to 2.7. Six weeks cannot rank those four, and I would not try. Its job is to catch a catastrophic failure (a leak, a bug), and it did not.</p><h3>11. Does the model see something the voters will later see?</h3><p>A forecast that merely matches what voters do is useful. A forecast that is <em>right when it disagrees with the voters</em> would be remarkable. We can test that: for the forecast of poll t+1, compare our points with the real points, then look at polls t+2, t+3 and so on. If our number carried information the voters initially missed, later polls should move toward it. Let d₀ be our points minus the real points at t+1, and d_k how far the real points moved k polls later:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YstGCM84RnzwxTMhEovDNw.png" /></figure><p>β_k is the share of our disagreement the polls later adopted: 0 means they ignored us, 1 means they fully caught up. (The Forecast page has a Residual analysis button that shows this for any past forecast.)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OwDQNrJLM349ATX3yrIiEQ.png" /></figure><p>Across 131 forecasts (2015 to 2025): one poll later β ≈ 0%, two polls 11%, three 19%, four 30%, five 27%. On the gaps of 150 or more points, the poll moved our way 43%, 52%, 51%, 50% and 50% of the time (a coin flip is 50%), and our forecast was <em>no closer</em> than the forecast week’s own poll to where the poll ended up (154 against 138 points per team one poll later). So: the polls drift a quarter of the way toward us over a month, but the direction is a coin flip and the forecast is no better a guess at the future poll than the poll we were forecasting. The evidence does not support a claim that the model has foresight. It is a good predictor of what voters will do; it is not an oracle for what they should have done.</p><h3>12. What could be wrong with all of this</h3><ul><li><strong>Multiple comparisons.</strong> We tried roughly a hundred variants. Gaps of 0.01 to 0.02 spots among versions 2.4 to 2.7 are not reliable.</li><li><strong>Tuning set optimism.</strong> The panel flattered the outside-data screen; the full replay corrected it. Stack settings were chosen on the same 142 weeks they are scored on. The 2026 holdout is the only truly untouched sample and it is six weeks.</li><li><strong>Dependence between weeks.</strong> The bootstrap treats weeks as exchangeable. They are not.</li><li><strong>Non-stationarity.</strong> The playoff expanded in 2024; 2020 was a short season; voters turn over. The model shows no deterioration in 2024 to 2025 (0.93 vs 0.94 overall), but that is two seasons.</li><li><strong>Data.</strong> Ballots come from a secondary archive reconciled to AP totals; voter affiliations are inferred; closing lines cover about 85% of games (FCS opponents have none).</li><li><strong>What the metric rewards.</strong> Average rank miss weights a 21st-versus-25th error like a 1st-versus-5th error. The model’s top-5 behavior is much better than its tail (0.16 miss at №1, 1.88 at 21 to 25).</li></ul><h3>13. What comes next: version 3 and nonlinear models</h3><p>Everything above is linear in its features, with hand-built interactions. That was deliberate: with fifteen polls a season, the interpretable weights let us see what a voter does and find bugs like the event/state mix-up. But linear-plus-engineered-interactions is a strong assumption, and several of the leftover errors look like interactions: a loss <em>times</em> the team’s rank <em>times</em> how many ranked teams above it also lost <em>times</em> how early in the season it is. The natural next step is to test whether nonlinear models can find that structure.</p><p>The plan for version 3 and onward, in the same walk-forward harness:</p><ul><li><strong>Gradient-boosted trees</strong> on the stack’s problem (a team-level regression of next-poll points, or a pairwise/listwise ranking objective such as LambdaMART), with the same features and the voter-model forecast as an input. Trees can find thresholds and interactions that a ridge regression can only approximate.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RCgyka6V6pb2uTNp4_3lXg.png" /></figure><ul><li><strong>Monotone constraints </strong>on the features where direction is known (a bigger win margin should never lower a team), which regularizes hard in a small-data regime and keeps the model explainable.</li><li><strong>Nonlinear voter models</strong>, for example a shallow boosted model per voter, pooled hierarchically. This is the riskiest: per-voter data is tiny, so the shrinkage structure matters more than the learner.</li><li><strong>Attribution and interaction audits</strong> (SHAP, partial dependence, permutation importance by stage of season) so that every gain has an explanation we can check against the linear model’s.</li></ul><p>The success criteria are fixed in advance: a paired improvement over beta 2.6 with an interval excluding zero on the full 142 weeks, <em>not</em> just a tuning panel; no degradation in the 2024 to 2025 era; and no loss at №1. My honest prior is modest: with roughly 3,500 team-weeks over ten seasons, a boosted model is as likely to overfit as to help, and the early-season weeks, where we are weakest, are exactly where there is the least data to learn from. But it is the right experiment to run, and a null result would be informative too.</p><h3>Want to Play Around With The Model?</h3><p>I’ve recently launched <a href="https://poll-position.robert-mepham1309.workers.dev/#/poll"><strong>Poll Position</strong></a>, a free-to-use online research tool centered around the AP Top 25 poll for both College Football and College Basketball. The model discussed above has been deployed under the hood in that app within the “Forecast” tab. Want to see how any specific outcome shifts how the AP Poll is going to view a team’s body of work? The tool can do that! Want to drill down and see how Voter A is going to view a big upset vs. how Voter B views that same upset? The tool can do that too!</p><p>All of this is available at: <a href="https://poll-position.robert-mepham1309.workers.dev/#/poll"><strong>AP Top 25 · Poll Position</strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=237c1cfc22e8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Is There a Conference Bias in the AP Basketball Poll? (Spoiler: It’s Not the SEC)]]></title>
            <link>https://medium.com/@robert.mepham/is-there-a-conference-bias-in-the-ap-basketball-poll-spoiler-its-not-the-sec-3d02025dbcd8?source=rss-745e2a58a3bf------2</link>
            <guid isPermaLink="false">https://medium.com/p/3d02025dbcd8</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[predictive-modeling]]></category>
            <category><![CDATA[march-madness]]></category>
            <category><![CDATA[sports-analytics]]></category>
            <category><![CDATA[ap-top-25]]></category>
            <dc:creator><![CDATA[Rob Mepham]]></dc:creator>
            <pubDate>Wed, 04 Nov 2026 14:31:02 GMT</pubDate>
            <atom:updated>2026-11-04T14:31:02.048Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7crnMam0W9Txp2wmop5iwQ.png" /></figure><p>Let me set the scene. It’s the week of Selection Sunday, somebody in your group chat has just typed “the Big Ten is overrated, they just beat each other up,” and somebody else replies, “bro, the ACC has been propped up since the nineties.” Within about forty seconds you’ve got four conference loyalties, three outdated statistics, and one guy who thinks Syracuse never lost.</p><p>In football, the AP-poll argument has a clear villain: the SEC. I tested that one in my earlier articles. But what about hoops? There, nobody can agree on who the AP favors, which is a problem I could measure instead of argue about.</p><p>So I used every AP men’s basketball poll from 2010–11 through 2025–26 (16 seasons) and the same method as the football series: build a ranking from nothing but game results, see how many AP points a team with that results-only rank <em>typically</em> earns, and measure each conference’s gap from that. (Method in the box at the bottom.) I’ll admit I went in expecting the SEC to show up, because that’s the narrative I was carrying around. I should have known better.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GHPGiPD8W052htORJU3kwg.png" /></figure><h3>Result 1: The conference ranking</h3><p>Here are the gaps for teams in the results-only top 40: the AP points a team actually earned minus what teams with the same results-only rank typically earn (as a share of the maximum possible points).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*o81--I-LFPmLMWKaCs8JbQ.png" /></figure><ul><li><strong>Big East +5.9</strong> (95% interval 2.0 to 9.9) and Big 12 +5.1 (1.9 to 8.3) are the beneficiaries. Both intervals sit clear of zero.</li><li><strong>ACC +2.9 </strong>(-0.2 to +6.3). Leaning positive.</li><li><strong>Big Ten -0.7, Pac-12 -1.4, SEC -2.6</strong> (-5.9 to +0.9). All consistent with “treated about right,” with the SEC leaning slightly <em>under</em>-credited.</li><li>The <strong>mid-major conferences</strong> (A-10, Mountain West, WCC, Valley, C-USA) are all negative, from about -5 to -11. More on that below, because it’s the shakiest part.</li></ul><p>So the first answer to the question is: <strong>the AP basketball poll’s favorite conference isn’t the SEC. It’s the Big East, with the Big 12 right behind.</strong></p><p>The ordering is stable. The Big East is positive in 13 of the 16 seasons, and the Big 12 in 11. Dropping any single season barely moves the Big East’s number (4.9 to 6.8). The Big East and Big 12 lead under every version of the results-only rating I tried (lighter shrinkage, heavier shrinkage).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8X3A2JTAwnzBrva7kNWw-A.png" /></figure><h3>Result 2: Is it bigger or smaller than the football bias?</h3><p>Same ruler, same scale:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xlN3ZT-d8WPrM-f3HVuy0Q.png" /></figure><p>In football, the SEC runs <strong>+4.6</strong> (1.3 to 7.9) above its results-only rank, and the Group of Five sits between -6 and -14. In basketball, the top conference runs <strong>+5.9</strong>. So by the headline number, <em>basketball’s biggest conference bonus is slightly larger than football’s</em>, and it goes to a different league.</p><p>Look at the spread among power conferences, which is where “bias” arguments live:</p><ul><li><strong>Basketball: </strong>best (Big East +5.9) to worst (SEC -2.6) is <strong>8.5 points of share.</strong></li><li><strong>Football:</strong> best (SEC +4.6) to worst (ACC -1.0) is <strong>5.6 points.</strong></li></ul><p>Another way to see it is seat share. Here’s the number of Top-25 slots each conference holds in an average poll, versus the number of results-only top-25 teams it has:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bov6nGIpaJ-8rGEEjcWnAQ.png" /></figure><p>The differences are small in absolute terms. The biggest winner, the Big East, holds <strong>0.4 extra slots per poll</strong> (3.3 versus 2.9); the ACC and Big 12 about 0.3. The biggest loser is the WCC (<strong>-0.5</strong>), and the Mountain West and Valley are each down about 0.2. In football the SEC’s seat bonus is similar (+0.2). A conference “bias” in either sport is, in practice, a fraction of a spot per poll. That’s worth knowing before you rage-tweet at a voter.</p><p>Lastly, how often does the AP rank a team that the results put in the 8th-to-25th range? In basketball: the <strong>ACC 84%, Big East 82%, Big 12 82%, Big Ten 75%, SEC 71%, Pac-12 71%</strong> and <strong>mid-majors 48 to 57%</strong> (American 86%). In football the equivalent figures run SEC 84%, Big 12 75%, Pac-12 79%, ACC 64%, Big Ten 75%, with the Mountain West at 45% and the Sun Belt at 43%.</p><h3>Result 3: Is it the conference, or the program?</h3><p>This is the part I like most. If the AP truly favored the Big East as a <em>league</em>, every Big East team in the top 40 should get a boost. If it’s about specific famous programs, the boost should live in teams that finished the previous season ranked.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nM0ZpfPMfyup0CaTfZr7Gw.png" /></figure><ul><li><strong>Big East:</strong> teams that finished last season ranked: <strong>+8.7</strong>. Teams that didn’t: <strong>+2.1</strong> (interval -3.3 to +7.4).</li><li><strong>Big 12:</strong> <strong>+7.4</strong> versus <strong>+1.2</strong>.</li><li><strong>SEC:</strong> <strong>+1.6</strong> versus <strong>-5.4 </strong>(interval -9.3 to -1.3). SEC teams without a ranked pedigree are <em>under</em>-credited by about five points.</li><li><strong>Big Ten:</strong> +1.8 versus -2.9.</li><li><strong>ACC </strong>is the interesting one: <strong>+3.4 versus +3.3</strong>. Its premium is spread across the whole league and doesn’t depend on recent history.</li></ul><p>So most of the Big East’s edge is brand, not league. The programs doing the work: Notre Dame (+27 over 22 team-polls in its Big East years), Syracuse (+21), Georgetown (+14), Providence (+14), UConn (+13, over 102 team-polls) and Villanova (+10, over 156). The old 16-team Big East (just two seasons in this split) had a bigger premium (+7.6) than the current one (+4.3), but both intervals are wide, so I wouldn’t read much into the difference.</p><p>It also means “conference bias” and “blue-blood bias” are tangled together here: the Big East, Big 12 and, to a lesser degree, Big Ten and SEC all get a bump when they bring established programs and no bump when they don’t.</p><h3>The shakiest part: mid-majors</h3><p>I did say I’d flag this. In the raw numbers, power conferences run<strong> +1.7</strong> above the results-only baseline and mid-majors <strong>-5.2</strong>: a gap of about 7 points. In football the same two groups differ by about 11 points (+2.2 versus -8.9). So in raw terms, basketball is <em>kinder</em> to its mid-majors than football is.</p><p>But my results-only rating flatters mid-major teams. When I checked it against 2,877 games between mid-majors and power-conference teams, mid-majors did about 1.2 points worse than predicted (3.2 points worse for the highly rated ones). Correct for that and the picture changes: the last column of the robustness table subtracts 2.5 points from every mid-major’s rating. It flips the mid-major rows from negative to positive (with large changes in the sample), and the power ordering barely moves: <strong>Big East +1.9, Big 12 +1.6, ACC -0.1, Big Ten -3.4, SEC -4.8, Pac-12 -4.9.</strong></p><p>The corrected football picture is similar: the SEC remains the top conference (+3.0) and the Group of Five’s gap shrinks to about -2 (interval -5 to +2). Corrected spread among power conferences: <strong>basketball 6.8 points, football 5.6.</strong></p><p>I wrote the full audit in the previous article ([LINK_ARTICLE_2]); the short version is that<strong> the mid-major part of “conference bias” is smaller than the raw numbers suggest and I can’t tell you which side of zero it’s on.</strong> The power-conference part is the one I trust.</p><h3>So: is there a conference bias in the AP basketball poll?</h3><p>My answer, with the caveats:</p><ul><li><strong>Yes, a modest one, and it favors the Big East and Big 12, not the SEC.</strong> The SEC is, if anything, slightly under-credited relative to its results.</li><li><strong>It’s about as big as football’s.</strong> Among power conferences, the best-to-worst spread is about 8.5 points in basketball versus 5.6 in football (6.8 versus 5.6 after correcting for mid-major flattery). Same order of magnitude, different beneficiary.</li><li><strong>It looks more like program reputation than league loyalty</strong> (UConn, Villanova, Georgetown, Syracuse), though the ACC’s boost is the exception and is league-wide.</li><li><strong>In practice it’s small:</strong> a fraction of a poll slot per week.</li></ul><h3>Caveats (please read)</h3><ol><li><strong>Most individual conference intervals include zero.</strong> Only the Big East and Big 12 are clearly positive, and only the lowest mid-major leagues are clearly negative on the raw ruler. Treat the exact numbers as approximate.</li><li><strong>Conference names are as they were that season.</strong> The “Big East” is the 16-team league in 2010–13 and the basketball-first league after; the “Pac-12” has almost no teams after 2023–24 and “Big 12” absorbs realignment. I didn’t try to reassign teams across eras.</li><li><strong>The mid-major numbers are the weakest</strong> (see above). I’d call that part of the answer “unresolved,” not “no bias.”</li><li><strong>My results-only rating is margin-based.</strong> A team that wins close games, or doesn’t run up scores, gets underrated, and so does any league full of them.</li><li><strong>I only measure outcomes, not intent.</strong> Nothing here says why voters rank teams the way they do (regional exposure, tournament history, TV schedules). It just shows the gaps.</li><li><strong>Post hoc looks. </strong>I examined the reputation split (ranked last season or not) after seeing the conference ranking, so treat it as descriptive.</li></ol><h3>Try It Yourself</h3><p>Everything above can be explored in <a href="https://poll-position.robert-mepham1309.workers.dev/#/poll"><strong>Poll Position</strong></a>, an app that tracks the AP Top 25 week by week for football and men’s basketball, with every voter’s ballot, game-by-game AP point swings, and your own ballot:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hjTFz2-5YOuT2MjVOftATQ.png" /></figure><ul><li>See the conference in every team’s row and every poll: [APP_URL]/?sport=ncaam</li><li>Dig into how any individual voter ranks teams (home-state teams, conferences, favorites): <a href="https://poll-position.robert-mepham1309.workers.dev/#/voters"><strong>Voters · Poll Position</strong></a></li><li>Check a Big East blue-blood’s whole poll history (try UConn): <a href="https://poll-position.robert-mepham1309.workers.dev/#/team/CONN"><strong>Team · Poll Position</strong></a></li><li>Cast your own ballot and see whether <em>you’re</em> the one favoring the Big East (no judgment; okay, a little)</li></ul><blockquote><strong>Methods I Used For The Above Analyses:</strong></blockquote><blockquote><strong>Results-only rating: </strong>margin-based least-squares (Massey-style) from game results only, margin capped at 28, home advantage estimated, ridge shrinkage 0.6 (main); lighter (0.2, cap 21) and heavier (1.5, cap 35) variants for robustness. Same code as the football series.</blockquote><blockquote><strong>Gap:</strong> for each poll (6 onward, including the Final poll), the expected share of maximum AP points by results-only rank is the non-increasing average over all Division I teams (isotonic regression); a team’s gap is its actual share minus that expectation. Conference gap = mean over team-polls in the results-only top 40 (the top 25 gives a similar picture: Big East +6.6, ACC and Big 12 +5.4, SEC -3.4); 95% intervals from 1,000 team-season cluster-bootstrap draws.</blockquote><blockquote><strong>Seat share:</strong> average number of AP Top-25 slots per poll versus the number of results-only top-25 teams, over 215 basketball polls and 124 football polls.</blockquote><blockquote><strong>Reputation split:</strong> teams that finished the previous regular season in the AP Top 25 (last pre-postseason poll) versus those that did not; 2011–12 onward.</blockquote><blockquote><strong>Correction for mid-major flattery:</strong> subtract 2.5 points from every mid-major’s rating (the size of the shortfall found in the cross-game check) before ranking, then repeat. Football uses the same 2.5 for Group of Five (excluding independents).</blockquote><blockquote><strong>Football</strong>: 2014 through 2025. Basketball: 2010–11 through 2025–26.</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3d02025dbcd8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Your Team Might Actually Be Getting Screwed by the AP Poll. Here’s What the Numbers Say.]]></title>
            <link>https://medium.com/@robert.mepham/your-team-might-actually-be-getting-screwed-by-the-ap-poll-heres-what-the-numbers-say-2ad32181bc49?source=rss-745e2a58a3bf------2</link>
            <guid isPermaLink="false">https://medium.com/p/2ad32181bc49</guid>
            <category><![CDATA[football-analytics]]></category>
            <category><![CDATA[ap-poll]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[college-football]]></category>
            <category><![CDATA[sports-analytics]]></category>
            <dc:creator><![CDATA[Rob Mepham]]></dc:creator>
            <pubDate>Mon, 02 Nov 2026 14:31:01 GMT</pubDate>
            <atom:updated>2026-11-02T14:31:01.758Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*N5kt4buFUSl1sxiUn1kMHA.png" /></figure><p>Every fan base believes the same two things: <em>our</em> wins never count for enough, and <em>our</em> losses always count for too much. Let’s find out who’s right.</p><p>Every week the AP poll hands out and takes away points. Beat a ranked team and you climb. Lose and you fall. But <em>how much</em> you climb or fall is supposed to depend on who you beat, how, where, and where you were ranked to begin with.</p><p>Is it? Or do some programs get a bigger bonus for the same result, and some get docked more?</p><p><a href="https://poll-position.robert-mepham1309.workers.dev/">Poll Position</a> records the exact AP-point change for every ranked team in every game since 2014. I used that to ask:<strong> given who a team beat or lost to, how many points <em>should</em> it have gained or lost, and who consistently gets more or less?</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Di7JpViykSCHStAWOMT62g.png" /><figcaption><em>Every game in Poll Position shows the before/after AP points. The magnifying glass searches for similar games.</em></figcaption></figure><h3>The method (short version, I promise)</h3><p>I took every game played by a team that was ranked in the AP Top 25 entering the game (3,375 of them: 2,420 wins and 955 losses, 2014–2025), excluding bowls and weeks where a team’s poll window had a second game in it (so the point swing belongs to a single result). Then, separately for wins and for losses, I fit a model of the AP-point change as a function of:</p><ul><li>the team’s rank entering the game,</li><li>the margin (capped at 35),</li><li>the opponent’s AP rank,</li><li>home, away, or neutral site, and</li><li>whether the opponent was an FCS team or unranked.</li></ul><p>That gives an <em>expected</em> swing for every game. The gap between expected and actual is that game’s <strong>residual</strong>: how many points the poll gave or took beyond what a typical, comparable result earns. (The typical residual is ±90 points, on a scale where a poll of 62 voters hands out around 1,500 points to a #1 team, so a +40 average is quite meaningful.)</p><p>A team’s <strong>score </strong>is its average residual, shrunk toward zero when it has few games (an eight-game “pseudo-sample” prior so a team with three games can’t top the list). I also bootstrapped a 95% interval.</p><h3>The pampered</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*f23yvO2B53RlQytI0T7gpw.png" /></figure><p>The top of the list:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/604/1*iMXTXvJlAYuypGDnghPL4Q.png" /></figure><p>These six are the only teams whose intervals sit entirely above zero. Notice what the biggest bonuses have in common: a low-ranked team beating a top-10 team.</p><p>Some of that is a real effect the model only partly captures. A win over a team ranked #4 by a team ranked #25 moves a lot of voters at once. The model controls for rank, but not for <em>surprise</em>. Hold that thought. The top outliers support it:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/530/1*w1Ii0s5LdjGIM1MlDjmyeA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-R90WO-gXO1ysDthVlkcCQ.png" /></figure><h3>The shortchanged</h3><p>The bottom of the list is more interesting, because the pattern is different.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/569/1*2RSdwjySwc23GX927Ngk-w.png" /></figure><p><strong>To be fair to the poll, let’s be honest about this table:</strong> only <strong>Memphis </strong>and <strong>UCF </strong>have intervals that exclude zero. The others are suggestive, not proven. If you’re a Notre Dame or Oklahoma fan who feels like the poll is out to get you, the data nudges in your direction but doesn’t clinch it.</p><p>And the sources of the shortfall are different for different teams:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qKVMPjlNe7G6M36FCvyU3Q.png" /></figure><ul><li><strong>The Group of Five teams (Memphis, UCF, Boise State, Houston) are underpaid for their <em>wins</em>.</strong> Memphis’s average win earns 74 points <em>fewer</em> than a comparable win. UCF’s is 27 fewer. Their losses, on the other hand, are punished <em>less</em> than average (Memphis +13). That’s the conference-bias picture from the first article, now at the team level: the poll doesn’t believe their wins.</li><li><strong>The blue bloods (Notre Dame, Florida State, Oklahoma, Texas, USC) are punished hard for <em>losses</em>.</strong> Notre Dame’s average loss costs 77 points more than a comparable loss. Florida State’s costs 101 more. Oklahoma’s, 65 more. Texas’s, 60. Their <em>wins</em> are paid about normally.</li></ul><p>In other words, there are two different ways to get “screwed,” and they point in opposite directions: <strong>the little guys aren’t given credit for winning, and the big names aren’t given any slack for losing.</strong> When you’re expected to win, a loss is a bigger surprise, and voters react more.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PO1ul7hsMiGl5oA8XXO4rg.png" /></figure><p>The costliest individual losses in the sample all share one feature: the opponent wasn’t ranked.</p><ul><li>2025 Penn State (#7) lost 37–42 to UCLA: <strong>-1,082</strong> points vs. -453 expected.</li><li>2022 Texas A&amp;M (#6) lost 14–17 to Appalachian State: <strong>-1,096</strong> vs. -525.</li><li>2022 Notre Dame (#8) lost 21–26 to Marshall: <strong>-1,062</strong> vs. -523.</li></ul><p>Those are the largest overshoots against expectation in the sample. Nothing unfair about a top-10 team losing to an unranked opponent being punished; what stands out is that these cost more than double what a typical comparable loss does.</p><p>And the other side, losses the poll forgave:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rtTEbT4dpoiK3Pv-kynzvw.png" /></figure><p>Clemson (#3) lost 42–43 to Pittsburgh in 2016 and dropped just 104 points. Michigan (#2) lost 13–14 at Iowa that year and fell 109. Top-five teams that lose by a single point get the benefit of the doubt.</p><h3>What about the conferences?</h3><p>Averaged across all ranked games, here’s the conference-level residual (points per game, relative to comparable games): Big 12 +9, Pac-12 +6, ACC +5, Big Ten +2, SEC 0, Independents -9, Mountain West -11, American -20, MAC -34, Sun Belt -46.</p><p>The SEC sits at zero on this measure, so at the <em>game</em> level (as opposed to the <em>poll-position</em> level in the earlier articles) it isn’t paid a bonus. A reasonable reading: the SEC premium from the second article comes less from how many points each result earns and more from voters’ <em>starting</em> point: where SEC teams enter the season, and how slowly they fall. This analysis, which only looks at changes, wouldn’t detect that.</p><h3>Caveats</h3><ol><li><strong>Net weekly movement.</strong> The “AP point change” is the net change from one poll to the next. It reflects the game, plus any shift from other teams’ results.</li><li><strong>Rank-before is controlled; expectations aren’t.</strong> Preseason hype, margin vs. spread, and injury news all affect voter reactions. Betting lines are only archived in the app from 2026 on, so I couldn’t adjust for them.</li><li><strong>Small samples.</strong> Kentucky’s top spot rests on 22 games. I’ve shown intervals and only six teams on the “pampered” side clear zero; on the “shortchanged” side, only two.</li><li><strong>Era effects.</strong> Programs that were good in specific stretches may be capturing eras (a few great seasons) rather than an enduring attitude from voters.</li></ol><h3>Check your own team</h3><p>The reason I built <a href="https://poll-position.robert-mepham1309.workers.dev/"><strong>Poll Position</strong></a> is so you can check your own team, rather than take my word for any of this:</p><ul><li>Open any team, see every game’s AP-point swing, and hit the <strong>magnifying glass</strong> to compare it with the most similar games in poll history</li><li>See where your team’s voters rank them, and which individual voters are highest and lowest on them</li><li>Cast your own ballot and see whether <em>you</em> pamper your favorite team</li></ul><p>Go ahead, prove me wrong: <a href="https://poll-position.robert-mepham1309.workers.dev/">Poll Position App</a></p><blockquote>Methods I Used For The Above Analyses:</blockquote><blockquote>Sample: games with the team ranked #1–25 entering, FBS teams, poll index ≥ 2, excluding postseason and games sharing a poll window with another result; 2,420 wins / 955 losses.</blockquote><blockquote>Model: ridge OLS fit separately to wins and losses: rank-before dummies (25), capped margin, opponent-rank strength (26 − rank, 0 if unranked), away/neutral, FCS opponent (and FCS × margin), unranked opponent.</blockquote><blockquote>Team score: mean residual × n/(n+8); intervals from 500 bootstrap resamples of the team’s games; minimum 20 ranked games (54 teams qualify).</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2ad32181bc49" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>