<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Syed Ali on Medium]]></title>
        <description><![CDATA[Stories by Syed Ali on Medium]]></description>
        <link>https://medium.com/@syed_ali?source=rss-08b3001202b0------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Bnmc0Z1soQ4WPHYhO9doLA.png</url>
            <title>Stories by Syed Ali on Medium</title>
            <link>https://medium.com/@syed_ali?source=rss-08b3001202b0------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 01:49:43 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@syed_ali/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[A Chart Ban Is Only as Good as Its AI Music Detector]]></title>
            <link>https://medium.com/@syed_ali/a-chart-ban-is-only-as-good-as-its-ai-music-detector-2e4e9a0bf841?source=rss-08b3001202b0------2</link>
            <guid isPermaLink="false">https://medium.com/p/2e4e9a0bf841</guid>
            <category><![CDATA[music-industry]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[streaming]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Syed Ali]]></dc:creator>
            <pubDate>Tue, 17 Nov 2026 11:32:30 GMT</pubDate>
            <atom:updated>2026-11-17T11:32:30.589Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Cover: the headline Only as good as its detector above a steel sieve over a small podium; seven spheres have dropped into a tray, five blue and two vermilion." src="https://cdn-images-1.medium.com/max/1024/1*00NaX-Q2BxG-yJfji8XvuQ.jpeg" /><figcaption>A ban is only as good as the detector behind it. Image created by the author with AI.</figcaption></figure><p><em>An AI music detector that is right 99.8% of the time can, on illustrative numbers, still be wrong about three in ten of the tracks it flags, if the tracks it is judging are mostly human. Chart rules now depend on these tools, and none of them publishes the numbers that would show who gets excluded by mistake.</em></p><p>In August, Australia’s chart body said wholly AI generated tracks can no longer chart. In October, a streaming service said it would let listeners switch AI music off. Both rules lean on a detector, and neither says how often that detector is wrong about the tracks that matter. I have spent years asking that question of models, so I ran the arithmetic.</p><p>The question comes from an earlier job. Before Echonos I worked as a data science consultant at Deloitte, Booz Allen Hamilton and Accenture, and later ran operations as COO of Tabler. Every model I shipped in those years came with a confusion matrix: how often it flagged the right thing, how often it flagged the wrong one, and what each mistake cost. A chart rule that excludes tracks on a classifier’s word is a model with consequences. It deserves the same table.</p><p>My read is this. A chart ban is a measurement problem before it is a policy. Whether it treats real artists fairly depends less on the headline accuracy of any AI music detector than on two numbers nobody has published: how often the detector flags human work, and how rare AI tracks are among the songs that actually compete for a chart place. On illustrative numbers, which I set out below, a detector that looks excellent on a stream of uploads could still be wrong about a large share of the chart contenders it flags.</p><p>This is written for artists and managers whose release could be caught by one of these rules, for chart and label operators who will be asked to defend them, and for founders building detection tools. If you want an argument about whether AI music is art, this is the wrong essay. Nothing below is about how music is made. It is about how an exclusion is decided.</p><h3>Three Rules, Three Definitions of AI Made</h3><p>The short answer first. AI music detector accuracy, as quoted today, is one number measured on an undisclosed set of tracks. For a chart rule it says little on its own. What matters is how often the detector flags human work, and how common AI tracks are among real chart contenders. Neither figure has been published.</p><p>Start with what the rules say, because they do not say the same thing.</p><p>ARIA, Australia’s chart body, has banned wholly AI generated tracks from its charts from the chart dated Monday 31 August 2026, Music Week reported on 25 August. Recordings that use generative AI in a supporting role stay eligible if they are substantially human made and raise no manipulation concerns. ARIA can remove a track “prospectively or retrospectively”, and it has updated its disputes process so artists can contest an exclusion. ¹ The report gives no detection method and no numbers.</p><p>IFPI, which manages official charts directly in markets across the Middle East, Southeast Asia, South Africa and Latin America, published global principles on 30 July 2026. A recording “must not be included in official charts where there is reason to believe” that it fails any of six criteria. Two of them are “They are substantially human made.” and “They do not raise manipulation concerns.” IFPI says its National Groups will extend the principles to “an additional 20+ official charts programs”. ² The standard is reason to believe, not proof, and no detection method is defined.</p><p>Deezer is the third, and its rule is not about charts at all. Music Week reported on 13 October 2026 that Deezer will let listeners filter out fully AI generated tracks, with the measures introduced globally before the end of 2026. ³ As of mid November that toggle is announced, not live. It hides tracks from a listener rather than from a chart.</p><p>So the three rules use three tests. ARIA bans wholly AI tracks. IFPI excludes on reason to believe a track is not substantially human made. Deezer hides fully AI tracks from those who opt out. The same two letters cover three different thresholds, and each threshold changes what a detector is being asked to find.</p><figure><img alt="Three cards titled Same word. Three tests. ARIA: wholly AI generated, banned from the chart. IFPI: reason to believe, not substantially human made. Deezer: fully AI generated, hidden by a listener toggle, marked Announced." src="https://cdn-images-1.medium.com/max/1024/1*G9CILAO2vGRFHS3oRmnNdw.jpeg" /><figcaption>Three chart and platform rules use the same words for three different tests. Image created by the author with AI.</figcaption></figure><h3>What a 99.8% Accurate AI Music Detector Does and Does Not Tell You</h3><p>The most quoted number in this debate is Deezer’s. “The tool’s accuracy is 99.8%, a company spokesperson told TechCrunch” in January 2026. The same report says Deezer claims the tool can identify every AI generated track from major generative models such as Suno and Udio. ⁴ That is a single spokesperson quote. There is no published method, no test set, and no breakdown of the error.</p><p>That last gap matters more than it sounds. Accuracy is the share of all judgements a detector gets right, and it hides which way the mistakes go. Three plain definitions carry the rest of this essay.</p><p><strong>Sensitivity</strong> is the share of AI tracks the detector catches. A detector with 95% sensitivity misses one AI track in twenty.</p><p><strong>The false positive rate</strong> is the share of human tracks it wrongly flags as AI. At 0.2%, it flags two human tracks in every thousand.</p><p><strong>Prevalence</strong> is the share of AI tracks in the pool being judged. This is the number that changes most between an upload queue and a chart, and it is the one an accuracy figure never mentions.</p><p>Here is the test you can run on any detector claim. Ask what the accuracy would be for a detector that flags nothing at all. If one track in two hundred is AI, that useless detector is 99.5% accurate. A headline accuracy number only means something next to the prevalence it was measured at.</p><h3>Why Base Rates Decide Who Gets Excluded</h3><p>What follows is my own illustration, not a Deezer figure, and not a claim about any named company’s error rates. I assumed a detector with 95% sensitivity and a 0.2% false positive rate, then changed only the prevalence. Precision, the share of flagged tracks that really are AI, comes out as follows:</p><ul><li><strong>At 50% prevalence,</strong> precision is 99.8%. Two in a thousand flags land on human work.</li><li><strong>At 10% prevalence,</strong> precision is 98.1%.</li><li><strong>At 2% prevalence,</strong> precision is 90.6%. About one flag in ten is a human track.</li><li><strong>At 0.5% prevalence,</strong> precision is 70.5%. About three flags in ten are human tracks.</li></ul><p>The detector did not change between those lines. Only the pool did. When AI tracks are common, the false positives drown in true ones. When AI tracks are rare, the false positives are most of what is left to notice.</p><figure><img alt="Four cards titled Same detector. Different pool. As the AI share of the pool falls from 50% to 10%, 2% and 0.5%, the share of flags that are AI falls from 99.8% to 98.1%, 90.6% and 70.5%; at 0.5%, 3 in 10 flags are human. Each card&#39;s percentage counts up and its small cylinder grows in turn. Illustration with assumed 95% sensitivity and 0.2% false positive rate." src="https://cdn-images-1.medium.com/max/1024/1*FKv1MYFo57nXOMahh-2FdQ.gif" /><figcaption>The same detector judged on smaller and smaller shares of AI tracks, on illustrative numbers. Animation created by the author with AI.</figcaption></figure><p>One more detail makes the point uncomfortable. On these same assumptions, overall accuracy at 0.5% prevalence works out at about 99.8%. A detector can honestly be described as 99.8% accurate and still be wrong about roughly three in ten of the tracks it flags. That is not a hidden flaw. It is what the arithmetic does when the thing being found is rare.</p><p>Now picture a pool of 1,000 tracks competing for a chart, with five of them wholly AI. The detector catches about five. It also flags about two human tracks. Seven tracks are flagged, and two of those artists did nothing the rule forbids.</p><figure><img alt="Flowchart titled Seven flagged. Two are human. 1,000 contenders split into 5 wholly AI and 995 human; about 5 AI tracks are caught and about 2 human tracks are flagged wrongly, giving 7 flagged, 2 by mistake. Illustration with assumed 95% sensitivity and 0.2% false positive rate." src="https://cdn-images-1.medium.com/max/1024/1*_fY4ats03iibY1T7FMlIhA.jpeg" /><figcaption>In an illustrative pool of 1,000 chart contenders with five AI tracks, two of seven flags land on human work. Image created by the author with AI.</figcaption></figure><p>The upload queue looks nothing like that pool. Deezer said in April 2026 that AI tracks made up 44% of new uploads, almost 75,000 a day, while drawing 1 to 3% of total streams, and that 85% of those streams were detected as fraudulent and demonetized. ⁵ By October it reported nearly 90,000 fully AI tracks every day, more than 50% of daily deliveries on peak days. ³ These are Deezer’s own figures and have been reported at different levels over the year, so I date each one and do not combine them.</p><p>My read of those figures is that AI tracks are a large share of what arrives and a small share of what gets heard. A chart is built from what gets heard. So the prevalence that matters for a chart rule is closer to the bottom of my list than the top. Nobody outside the platforms can measure it, and that is the problem.</p><p>Spotify’s numbers show the same gap from another angle. Spotify said on 20 October 2026 that “this year alone, we’ve taken more than 50 million slop and spam tracks out of recommendations.” ⁸ That is a count of tracks taken out of recommendations, not a count of AI tracks and not a chart decision. I wrote about what that figure does and does not measure in <a href="https://medium.com/@syed_ali/what-spotifys-50-million-removed-ai-tracks-actually-measure-de05acb433cc">What Spotify’s 50 Million Removed AI Tracks Actually Measure</a>. The lesson carries over: a detector tuned on the flood will meet a very different base rate at the top of a chart.</p><h3>Labels, Detectors and Fraud Signals Are Three Different Measurement Problems</h3><p>Charts do not rely on one tool. They rely on a mix of declared labels, automated detection and fraud signals, and each part fails in its own way. Mixing them into one verdict hides where the error sits. Three gaps stand out.</p><p><strong>Gap one: a detector flags, a second step confirms, and neither step reports its error.</strong> Music Business Worldwide reported on 26 August 2026 that Luminate, the music data company, flags likely AI items with a data science system and then uses audio signature matching to confirm those flags before applying labels such as “AI Generated” and “Human”. There is a review process for artists and labels to respond to incorrect tags, and the labels were not yet visible in its CONNECT product at the time. ⁶ A two step design is sensible. It also means the error rate that matters is that of the whole chain, and that is the number missing.</p><p><strong>Gap two: a missing label is not evidence.</strong> In the same report, Luminate said an absent label does not confirm a track is human made. ⁶ That is an honest statement of the limit. It also means a chart operator cannot treat the label set as complete in either direction, so some decisions will fall back on judgement.</p><p><strong>Gap three: reason to believe has no stated threshold.</strong> IFPI’s principles exclude on reason to believe. ² That standard can combine a detector score, a label, a disclosure and a manipulation signal, which is reasonable. But no published document says how strong the reason must be, or what share of excluded tracks turned out to be human on appeal. The test exists. Its error rate does not.</p><p>The Swedish case shows how this plays out in public. Music Business Worldwide reported on 19 January 2026 that a folk pop track with more than 6.3 million Spotify plays was blocked from Sweden’s official chart. IFPI Sweden’s chief executive Ludvig Werner was quoted saying that a song mainly made with AI has no right to be on the list. The publisher said AI tools had only been used as “assisting instruments”. ⁷ That is a reported dispute between two parties, not a proven classification error, and I make no judgement on which side is right. It shows that the line between mainly AI and supporting AI is drawn case by case, without a published measure.</p><h3>Where This Thesis Breaks</h3><p>You deserve the version of this argument where I am wrong. I would not trust it otherwise.</p><p><strong>Counter one: provenance and disclosure beat classification.</strong> IFPI’s own test is reason to believe, built from labels, disclosure and manipulation signals as well as detection. The classifier is one input among several, so its standalone error rate matters less than I suggest. I agree with the premise, and it is close to my point. When the classifier is one input, its error rate still flows into the decision, and a reason to believe without a stated threshold is the part nobody can audit. More inputs make the system better. They do not make it measurable from outside.</p><p><strong>Counter two: errors are cheap to fix.</strong> ARIA has a disputes process, Luminate has a review process for incorrect tags, and the Deezer toggle is reversible by the listener. If a mistake can be undone, false positives cost little. Reversibility does lower the cost of a false positive. But a chart position is time sensitive. A track excluded in its release week and reinstated a month later has lost the week that mattered. And no chart body has published how many disputes were raised, how many were upheld, or how long they took.</p><p><strong>Counter three: the scale argues for blunt rules.</strong> If AI tracks are more than half of deliveries on peak days and 1 to 3% of streams, on Deezer’s 2026 figures, then most of them are spam or fraud, and a crude filter removes real harm. This is the counter I take most seriously, because for uploads and recommendations it is right. A blunt filter on a flood with high prevalence works, and its mistakes are few in proportion. That is exactly why the chart case is different. A chart is a small pool with high stakes, where prevalence is low and each mistake lands on a named artist.</p><p>I am not claiming any detector named here is inaccurate. I have no access to their error rates, and that is the point of the essay. I am not claiming AI tracks should be allowed to chart, or that the rules are wrong to exist. What I am claiming is narrower: a rule that excludes on a detector’s word should publish the detector’s errors on the pool it judges.</p><p>Pick whichever counter is most persuasive to you. The practical implication is the same: ask for the error rate on chart contenders, not the accuracy on uploads.</p><h3>What a Chart Body Should Publish, and What an Artist Should Ask</h3><p>None of this needs new technology. It needs four numbers that any team running a classifier already has, reported for the pool that matters.</p><ol><li><strong>Precision on chart contenders.</strong> Of the tracks flagged among those that would otherwise have charted, how many were confirmed as AI.</li><li><strong>Recall, or sensitivity.</strong> Of the AI tracks that were found later by other means, how many the detector caught first.</li><li><strong>Appeal rate.</strong> How many exclusions were disputed, as a share of all exclusions.</li><li><strong>Reversal rate.</strong> How many disputes ended with the track reinstated, and the median time to a decision.</li></ol><p>My read is that the fourth number is the most useful. A reversal rate near zero suggests either a very precise system or a dispute process nobody wins. Published alongside the appeal rate, it would tell artists which of those they are facing.</p><figure><img alt="Takeaway grid titled Four numbers a chart should publish: 1 precision, flags confirmed as AI among chart contenders; 2 recall, AI tracks the detector caught first; 3 appeal rate, exclusions that were disputed; 4 reversal rate, disputes that reinstated the track." src="https://cdn-images-1.medium.com/max/1024/1*T3mCjOxlew18Tm6IqkyxLg.jpeg" /><figcaption>Four numbers that would make a chart rule auditable. Image created by the author with AI.</figcaption></figure><p><strong>If you are an artist or manager with a release in the next ninety days,</strong> keep a dated record of how the track was made: project files, session notes, any AI tool used and for which part. Read the chart rules of every market you expect to chart in, because ARIA’s ban, IFPI’s principles and a platform’s filter are not the same test. And if a track is excluded, ask three questions in writing before you dispute it: which test it failed, what evidence triggered it, and how long the review takes.</p><p><strong>If you run a chart or a label,</strong> publish the four numbers above for each quarter, even if the counts are small. Small counts are information too.</p><p><strong>If you build detection tools,</strong> report precision at several levels of prevalence, not one accuracy figure. Your customers will apply your tool to pools you never tested.</p><h3>FAQ</h3><p><strong>How to detect AI music</strong></p><p>The methods in public use combine a trained classifier, which Deezer says can identify tracks from major generators such as Suno and Udio, with confirmation steps such as audio signature matching, which Luminate reports using, and declared labels. ⁴ ⁶ No published method gives an error rate on chart contenders.</p><p><strong>Can an AI music detector be wrong?</strong></p><p>Yes. Every classifier has a false positive rate, and its precision falls as AI tracks become rarer in the pool it judges. On my illustrative numbers, a detector that is 99.8% accurate can still be wrong about roughly three in ten of its flags when only one track in two hundred is AI.</p><p><strong>Can an AI song chart in Australia now?</strong></p><p>A wholly AI generated track cannot, from the ARIA chart dated 31 August 2026. A recording that uses generative AI in a supporting role stays eligible if it is substantially human made and raises no manipulation concerns, and exclusions can be contested through ARIA’s disputes process. ¹</p><p><strong>What does substantially human made mean for a song?</strong></p><p>No published source defines it with a number. ARIA and IFPI both use the phrase without a threshold, share or test, so in practice it is decided case by case. ¹ ²</p><h3>Who Loses If I Am Right</h3><p>If chart rules keep excluding on unpublished error rates, the people who carry the mistakes are the artists with the least power to dispute them: independent acts without a label to argue their case, working against a release week that will not come back. My guess is that they are also the artists most likely to use cheap tools for parts of the work, and so the most exposed to the blurry line between a supporting role and a wholly AI track.</p><p>Chart bodies lose something slower. A chart is a trust product. Each disputed exclusion with no published numbers behind it spends a little of that trust, and once artists believe the rule is arbitrary, the chart stops settling arguments and starts causing them. Detection vendors lose too, if their single accuracy figure ends up quoted as the answer to a question it was never measured against.</p><h3>Closing</h3><p>Two rules this year leaned on a detector, and neither said how often that detector is wrong about the tracks that matter. The arithmetic says that question is the whole question. An accuracy figure measured on a flood of uploads tells you little about a pool where AI tracks are rare and every flag carries an artist’s name.</p><p>The fix is four numbers in a quarterly note. Precision on chart contenders, recall, appeal rate and reversal rate. Until a chart body publishes them, a ban is only as good as a detector nobody can audit.</p><p><em>A detector’s accuracy describes the pool it was tested on. A chart ban is judged on the pool it excludes.</em></p><p><em>Syed Ali</em></p><p><em>Founder and CEO, Echonos</em></p><p><em>Syed Ali is cofounder of </em><a href="https://echonos.ai/"><em>Echonos</em></a><em>, an AI music video pipeline for indie artists. He was previously COO at Tabler App (1M+ users, $50K+ MRR, successful exit) and a data science consultant at Deloitte, Booz Allen Hamilton, and Accenture. Find him on LinkedIn: </em><a href="https://www.linkedin.com/in/syedali-/"><em>https://www.linkedin.com/in/syedali-/</em></a></p><h3>Sources</h3><ol><li>Music Week, 25 August 2026, ARIA bans wholly AI generated tracks from Australian charts: the changes “take effect from the ARIA Chart dated Monday, August 31”; recordings using generative AI in a supporting role remain eligible if substantially human made with no manipulation concerns; ARIA can remove a track “prospectively or retrospectively”; ARIA “has updated its disputes process so artists and their representatives can contest an exclusion”. <a href="https://www.musicweek.com/digital/read/aria-bans-ai-generated-tracks-from-australian-music-charts/094819">https://www.musicweek.com/digital/read/aria-bans-ai-generated-tracks-from-australian-music-charts/094819</a></li><li>IFPI, 30 July 2026, global principles for the eligibility of recordings developed using AI in official charts: recordings “must not be included in official charts where there is reason to believe that they do not satisfy all of the following criteria”, including “They are substantially human made.” and “They do not raise manipulation concerns.”; National Groups to extend the principles to “an additional 20+ official charts programs”. <a href="https://www.ifpi.org/ifpi-rolls-out-global-principles-for-the-eligibility-of-recordings-developed-using-ai-in-official-music-charts-worldwide/">https://www.ifpi.org/ifpi-rolls-out-global-principles-for-the-eligibility-of-recordings-developed-using-ai-in-official-music-charts-worldwide/</a></li><li>Music Week, 13 October 2026, Deezer to let users filter out AI generated music: Deezer is receiving nearly 90,000 fully AI generated tracks every day, more than 50% of total daily delivery on peak days; it has excluded more than 42 million AI generated tracks from algorithmic and editorial recommendations; “The new measures will be introduced globally before the end of 2026.” Corroborated by TechCrunch, 14 October 2026: “The feature is expected to be available before the end of the year.” <a href="https://www.musicweek.com/digital/read/deezer-to-let-users-filter-out-ai-generated-music/095235">https://www.musicweek.com/digital/read/deezer-to-let-users-filter-out-ai-generated-music/095235</a> and <a href="https://techcrunch.com/2026/10/14/deezer-will-soon-let-you-filter-out-ai-generated-music/">https://techcrunch.com/2026/10/14/deezer-will-soon-let-you-filter-out-ai-generated-music/</a></li><li>TechCrunch, 29 January 2026: “The tool’s accuracy is 99.8%, a company spokesperson told TechCrunch.” Deezer claims the tool can identify every AI generated track from major generative models like Suno and Udio. No method, precision or recall published. <a href="https://techcrunch.com/2026/01/29/deezer-makes-it-easier-for-rival-platforms-to-take-a-stance-against-ai-generated-music">https://techcrunch.com/2026/01/29/deezer-makes-it-easier-for-rival-platforms-to-take-a-stance-against-ai-generated-music</a></li><li>TechCrunch, 20 April 2026: AI generated tracks represent 44% of all new music uploaded to Deezer, almost 75,000 per day, at 1 to 3% of total streams; “85% of these streams are detected as fraudulent and demonetized by the company.” <a href="https://techcrunch.com/2026/04/20/deezer-says-44-of-songs-uploaded-to-its-platform-daily-are-ai-generated/">https://techcrunch.com/2026/04/20/deezer-says-44-of-songs-uploaded-to-its-platform-daily-are-ai-generated/</a></li><li>Music Business Worldwide, 26 August 2026, Luminate AI identification framework: “We then use audio signature matching technology to confirm those initial flags and apply AI labels”; planned AI Generated, AI and Human tags; “a dedicated review process for artists and labels to respond to incorrect tags”; an absent label does not confirm a track is human made, Luminate said; “Those labels are not visible in CONNECT yet.” <a href="https://www.musicbusinessworldwide.com/luminate-launches-ai-music-identification-framework-with-ai-generated-labels-reaching-connect-later-this-year/">https://www.musicbusinessworldwide.com/luminate-launches-ai-music-identification-framework-with-ai-generated-labels-reaching-connect-later-this-year/</a></li><li>Music Business Worldwide, 19 January 2026: a track with “over 6.3 million plays” on Spotify was blocked from Sweden’s official chart; IFPI Sweden chief executive Ludvig Werner said a song that is mainly AI generated does not have the right to be on the top list; the publisher said “AI music tools have only been used as assisting instruments.” <a href="https://www.musicbusinessworldwide.com/ai-created-track-blocked-from-swedens-official-charts-after-racking-up-millions-of-streams-on-spotify">https://www.musicbusinessworldwide.com/ai-created-track-blocked-from-swedens-official-charts-after-racking-up-millions-of-streams-on-spotify</a></li><li>Spotify Newsroom, 20 October 2026, One Year In: Putting Human Artistry First on Spotify: “This year alone, we’ve taken more than 50 million slop and spam tracks out of recommendations.” <a href="https://newsroom.spotify.com/2026-10-20/prioritizing-human-artistry/">https://newsroom.spotify.com/2026-10-20/prioritizing-human-artistry/</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2e4e9a0bf841" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Where Does Spotify’s AI Remix Money Go? Two Scenarios and One Signal]]></title>
            <link>https://medium.com/@syed_ali/where-does-spotifys-ai-remix-money-go-two-scenarios-and-one-signal-9d1170dea46e?source=rss-08b3001202b0------2</link>
            <guid isPermaLink="false">https://medium.com/p/9d1170dea46e</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[streaming]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[spotify]]></category>
            <category><![CDATA[music-industry]]></category>
            <dc:creator><![CDATA[Syed Ali]]></dc:creator>
            <pubDate>Sun, 15 Nov 2026 11:33:13 GMT</pubDate>
            <atom:updated>2026-11-15T11:33:13.663Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Cover: the headline Watch the accounting above a sea-glass channel of brass coins that forks into a large shallow pool and a separate small brass basin." src="https://cdn-images-1.medium.com/max/1024/1*nvHZCYzmgf04gy3_na9dBw.png" /><figcaption>Two places the remix money could land: inside the pool, or in a basin of its own. Image created by the author with AI.</figcaption></figure><p><em>Spotify’s AI remix tool has a reported price and no published payout. The money can either sit inside the streaming pool or beside it, and one line in Spotify’s 2027 reporting will show which.</em></p><p>In October 2025, Spotify said artists and rightsholders would choose whether to take part in its AI music tools, and that creators would be properly compensated. It has not said how much, or from which pot. The Financial Times has reported a price of four to five dollars a month and a first quarter 2027 launch; Spotify declined to comment. Two readings of where the money goes are plausible. One disclosure in 2027 will tell you which one arrived.</p><p>The Spotify AI remix tool is the first product where a streaming service sells fans the right to change a song, rather than the right to play it. That difference matters to anyone who gets paid from a royalty statement. A play is easy to account for. A remix is a new piece of audio built on someone else’s recording, and every payout system in streaming was designed before that object existed.</p><p>I come at this as a founder in AI music and, before that, as a data science consultant at Deloitte, Booz Allen Hamilton and Accenture, and later COO of Tabler. A lot of that work was reading what a company chose to report as its own line and what it chose to fold into a total. Nothing below is about how remixes sound or how they are made. It is about where the revenue gets booked.</p><p>My read is this. Price is a marketing decision and payout is an accounting decision, and almost all of the coverage so far has been about the first. Whether remix money lands inside the existing pro rata pool or in a separate tier with its own split decides who gains and who quietly pays. Nobody outside Spotify and its licensors knows yet. The good news is that the answer has a visible symptom, and it is a reporting choice rather than a press release.</p><p>This is written for artists, managers, label and distributor staff, and founders who will have to explain a royalty statement in 2027. If you want a view on whether AI remixes are good for music, this is the wrong essay.</p><h4>What Spotify has said, and what only the FT said</h4><p>The short answer first. Spotify has confirmed an AI covers and remixes tool built with licensing partners, an opt in for artists, compensation for creators and a paid add on for Premium. The price, the launch window and the status of the Sony and Warner deals come only from the Financial Times, relayed by trade press.</p><p>On 16 October 2025, Spotify’s newsroom announced the work with Sony Music, Universal Music Group, Warner Music Group, Merlin and Believe. It said artists and rightsholders “will choose if and how to participate,” and that creators would be “properly compensated for uses of their work.” ¹ It mentioned new revenue streams. It gave no percentage, no price and no financial terms.</p><p>The company’s own executives have added a little since. On the second quarter call in August 2026, Gustav Söderström said Spotify does “not need a deal with all the majors” to launch, and described the product as a paid add on for Premium subscribers, as reported by Music Business Worldwide. ⁴ Alex Norström framed the approach as three Cs: consent, credit and compensation. Again, no price and no split. Engadget, reporting the Universal deal on 21 May 2026, wrote that some of the add on revenue goes to the original artists, and that artists must opt in. ⁷</p><p>Everything more specific comes from one place. The Financial Times reported on 30 October 2026, as relayed by Music Business Worldwide, that Spotify is weighing a price of four to five dollars a month “on top of existing subscription fees” and a first quarter 2027 launch, possibly around the Grammys on 7 February. ² The same report dated the licensing deals: Universal on 21 May, Merlin on 4 August, Kobalt on 13 August and Believe on 20 October, with Sony and Warner described as still negotiating. Spotify declined to comment. Hits Daily Double and Deseret News carried the same points, but both relay the FT, so on price and timing this is a single source. ³</p><p>That leaves an odd picture. Sony and Warner were named in the 2025 announcement, yet the FT reports them as still negotiating the licences the tool needs. Both can be true: announcing that you are working together is not the same as signing terms. As of 15 November 2026, I could find no Spotify confirmation or denial of the FT’s price and date.</p><figure><img alt="Two cards. Spotify has said: artists choose to take part, creators properly compensated, a paid add on for Premium. Only the FT reported: 4 to 5 dollars a month, first quarter 2027 launch, Sony and Warner still negotiating. Footer: Spotify declined to comment on the FT report." src="https://cdn-images-1.medium.com/max/1024/1*irx9IyvZ1FJinGPQzk37_A.png" /><figcaption>What Spotify has said, set against what only the Financial Times has reported. Image created by the author with AI.</figcaption></figure><h4>Scenario A: the money stays inside the pool</h4><p>In the first reading, a remix play is treated like a stream. It sits in the same pro rata pool as everything else, and its royalties flow to the rights holders of the original recording and composition.</p><p>There is executive language that fits this reading. Speaking at a Goldman Sachs conference on 9 September 2026, Söderström said: “If one of your fan creates a remix of your song, that actually increases the share of the streaming pool that you get.” ² Hits Daily Double reported that remix plays may also count toward the original track’s charts. ³ Put those together and the pitch to an artist is clear: your fans make more versions of your song, the versions get played, and your share of the pot grows.</p><p>The mechanism is worth stating plainly, because it cuts both ways. In a pro rata pool, a rights holder’s payout is their share of payable streams multiplied by the money in the pool. The pool is set by subscribers and price. More plays do not make it bigger. So when a remix of one song takes plays, the share it adds to that song’s owners has to come from somebody else’s share.</p><p>My read is that a pool can be diluted as easily as grown. Söderström’s sentence is true for the artist whose song gets remixed. It is silent about the artist whose song does not. If the add on revenue goes into the pool, the pool does grow a little, and the question becomes whether remix plays take a larger share of it than the add on money brings in. Nobody has published the inputs to answer that.</p><p>The implication for an artist is uncomfortable but simple. In Scenario A, the winners are catalogues fans already want to remix, which tend to be the biggest ones. I took apart the arithmetic of a fixed pool in <a href="https://medium.com/@syed_ali/ai-music-uploads-are-surging-what-share-of-the-royalty-pool-is-that-1560ad37a923">an earlier piece on AI music uploads and the royalty pool</a>, and the same logic applies here, with one difference: these plays are licensed and invited, not uploaded by strangers.</p><h4>Scenario B: a separate tier with its own split</h4><p>In the second reading, the add on is its own product with its own economics. Subscribers pay the reported four to five dollars on top of Premium, and that revenue is split under terms negotiated separately from streaming royalties, between Spotify, labels, publishers and the artists who opted in.</p><p>The stated design points this way. Spotify, Engadget and the August call all describe a paid add on. ⁴ ⁷ The FT report, as relayed by MBW, says fan remix royalties would go to the artists behind the original songs, with no percentage given. ² Engadget’s description, that some of the add on revenue goes to the original artists, reads like a separate revenue share rather than a stream count. ⁷</p><p>In this reading the remix money does not touch the existing pool at all. An artist who never opts in loses nothing, because the remix plays are paid from a different pot. An artist who opts in gets a second line of income, sized by whatever split their label or distributor agreed.</p><p>The catch is that a separate tier is only separate if its payouts are reported separately. A label can receive add on money under its own contract and still pass it through to artists inside a single “streaming” line on a statement. For most artists, Scenario B on paper and Scenario A on the statement look identical. The decision that matters is not made in the press release. It is made in the reporting.</p><figure><img alt="Two scenario cards under the headline Inside the pool, or beside it. Scenario A, inside the pool: remix plays count as streams, shares move between artists. Scenario B, a separate tier: add on revenue split on its own terms, the existing pool is untouched." src="https://cdn-images-1.medium.com/max/1024/1*v2NTCmiXV6t_ipYhWvq44w.png" /><figcaption>The two readings: remix plays counted inside the pool, or add on revenue split in a tier of its own. Image created by the author with AI.</figcaption></figure><h4>The one signal to watch</h4><p>Spotify publishes one recurring document that answers this kind of question: Loud &amp; Clear, its annual report on music payouts. The 2026 edition, published on 11 March 2026, said Spotify paid “more than $11 billion” to the music industry in 2025 and “nearly $70 billion” over its history, and that “roughly half of those royalties were generated by independent artists and labels.” ⁵ Those are 2025 figures; there is no 2026 total yet.</p><p>The signal is whether the next Loud &amp; Clear, or Spotify’s earnings reporting, breaks out remix add on revenue or payouts as its own line, or leaves them inside the headline “paid to the industry” number.</p><p><strong>A separate line</strong> for add on revenue or remix payouts points to Scenario B. It means the money is tracked as its own product, and that rights holders can be shown what it earned.</p><p><strong>Silence, plus chart crediting</strong> to the original tracks, points to Scenario A. It means remix activity is being treated as streaming, and its effect on everyone else’s share is folded into a total that only goes up.</p><p>Here is the test you can run yourself in 2027. Open the next Loud &amp; Clear when it comes out and search it for the words remix, cover and add on. Then read the payout headline and ask whether it could contain remix money without saying so. If you cannot tell, that is an answer too.</p><p>My read is that silence is more likely in the first year, simply because a new product rarely gets its own line until it is material. That is exactly why the signal is useful. A company that chooses to break out a small number is telling you it wants that number seen.</p><figure><img alt="Animated decision map titled Watch the payout line. It builds left to right: first the next Loud and Clear or earnings report, then two readings (remix money as its own line; silence plus chart crediting), then the scenario each points to (B and A)." src="https://cdn-images-1.medium.com/max/1024/1*pTTKADVb7JlkEAg9nc5Byw.gif" /><figcaption>The signal: a separate payout line points to Scenario B; silence plus chart crediting points to Scenario A. Animation created by the author with AI.</figcaption></figure><h4>Why Deezer built a filter instead of a tier</h4><p>Spotify is not the only service deciding what to do with AI made music, and the contrast helps. Music Business Worldwide reported on 13 October 2026 that Deezer would let users hide fully AI generated tracks from search results, with the toggle rolling out globally before the end of 2026, according to Deezer. ⁶ Deezer said it receives about 90,000 fully AI tracks a day, that on peak days they make up more than 50% of new deliveries, and that they account for 1% to 3% of streams.</p><p>Those are different problems. Deezer is dealing with tracks nobody licensed, arriving in volume and mostly not listened to. Spotify is building tracks that rights holders licensed, made by fans who are, by definition, listening. One service is building a way to keep AI music out of view. The other is building a way to sell it.</p><p>The way I would model it, both answers protect the same thing: the share of the pool held by recordings people chose to play. Deezer protects it by filtering. Spotify, in Scenario B, would protect it by moving remix money into a separate tier. In Scenario A, Spotify would be the one service actively adding new claimants to the pool, licensed and invited ones. That is a coherent strategy. It just has a different set of winners.</p><h4>Where this thesis breaks</h4><p>You deserve the version of this argument where I am wrong. I would not trust it otherwise.</p><p><strong>Counter one: “It is a separate product from the start.”</strong> This is the strongest counter. Spotify, Engadget and the August call all describe a paid add on, so a separate tier is already the stated design, and Scenario A is the one that needs evidence. ⁴ ⁷ I take this seriously, and if I had to bet on the commercial structure, I would bet on B. My response is that a separate price does not settle where the payout is booked or how it is reported to artists. The signal is about reporting, not pricing. A product can be sold separately and accounted for as streaming.</p><p><strong>Counter two: “Pool dilution cuts against the idea that the pool grows.”</strong> If remix plays count as streams in the same pro rata pool, they take share from other tracks, and Söderström’s line describes only the winners. That is right, and it is the reason Scenario A is a risk rather than a gift. I would put it to independents this way: roughly half of Spotify’s 2025 royalties went to independent artists and labels, by Spotify’s own account. ⁵ Whether remix plays draw that half down depends on which catalogues fans remix, and nobody has published that.</p><p><strong>Counter three: “The money is too small to matter.”</strong> Hits Daily Double reported a Bernstein estimate that 7% to 8% of US subscribers could sign up by the end of 2028. ³ KeyBanc estimated in September that AI music features could add about 2% to revenue per listener, according to a summary of Spotify’s third quarter call by The AI Musicpreneur. ⁸ Both are single analyst estimates. As an illustration, not a forecast: one million subscribers paying five dollars a month is $60 million a year, about half of one percent of the $11 billion Spotify reported paying out in 2025. My response is that small flows still set precedent. The terms agreed for a new kind of use tend to become the template for the next one, and this piece is about rights and reporting, not dollars.</p><p>I am not claiming Spotify is shortchanging anyone. Its terms are not public, so I cannot judge them, and the company has said creators would be properly compensated. I am not claiming Scenario A is more likely. What I am claiming is narrower: the question that decides who gains has not been answered in public, and the price is being discussed as if it were the answer.</p><p>Pick whichever counter is most persuasive to you. The practical implication is the same: ask how the money is reported, not only what it costs.</p><h4>What to ask your distributor or label this quarter</h4><p>The deals are being signed now, and the first quarter of 2027 is the reported launch window, so the next ninety days are when these questions can still change a contract or at least get an answer on the record.</p><p><strong>If you are an artist or a manager,</strong> ask your label or distributor three things in writing. Is my catalogue opted in, and who decided? Will remix plays credit my original track on charts and in my analytics? Will add on revenue arrive as its own line on my statement, or inside streaming? Keep the answers. They are the only version of the terms you are likely to see.</p><p><strong>If you work at a label or a distributor,</strong> decide now how you would report remix income to the artists you represent if Spotify does not report it separately. Believe and Kobalt signed in the window the FT described; if you work with either, ask what reporting your deal requires. A separate line costs little to set up before launch and a lot to rebuild after.</p><p><strong>If you are a founder building on licensed music,</strong> read Spotify’s design as a precedent for your own. Consent, credit and compensation is a useful frame, but the third C is only as good as the statement that shows it. I made a related point about who gets paid in <a href="https://medium.com/@syed_ali/spotify-licensed-ai-covers-who-gets-paid-and-what-nobody-has-disclosed-190cb304538f">an earlier piece on Spotify’s licensed AI covers</a>. Build the reporting before you build the pricing page.</p><figure><img alt="Takeaway card titled Three questions for your label: 1, is my catalogue opted in, and who decided? 2, will remix plays credit my original track? 3, will add on revenue arrive as its own line? Footer: ask in writing this quarter." src="https://cdn-images-1.medium.com/max/1024/1*R44FFd_SZDcG0TqAsw26fA.png" /><figcaption>Three questions to put to your label or distributor in writing this quarter. Image created by the author with AI.</figcaption></figure><h4>FAQ</h4><p><strong>Is Spotify launching an AI remix tool?</strong></p><p>Spotify has confirmed it is building AI covers and remixes tools with licensing partners. The Financial Times reported, as relayed by Music Business Worldwide on 30 October 2026, a first quarter 2027 launch. Spotify declined to comment.</p><p><strong>How much will Spotify’s AI remix feature cost?</strong></p><p>Only the Financial Times has reported a price, of four to five dollars a month on top of a Premium subscription. Spotify has not confirmed it.</p><p><strong>Do artists get paid when a fan remixes their song on Spotify?</strong></p><p>Spotify says creators would be properly compensated, and that artists choose whether to take part. No rate or revenue share has been disclosed.</p><p><strong>Which labels have signed with Spotify for AI remixes?</strong></p><p>According to the FT report relayed by MBW, Universal, Merlin, Kobalt and Believe signed between May and October 2026, while Sony and Warner were reported as still negotiating.</p><h4>Who loses if I am right</h4><p>If remix money stays inside the pool and is reported as streaming, the artists who lose are the ones whose songs fans do not remix. Their share of a fixed pool shrinks a little each time a popular catalogue gains remix plays, and their statement gives them no way to see it. Independent artists without a direct relationship with Spotify are the most exposed, because they are the least able to ask for a separate line.</p><p>Analysts lose too, more quietly. A remix product folded into the headline payout makes year on year growth harder to read, because part of it would be a new product rather than more listening. And Spotify carries a slower cost. A company that asks rightsholders to opt in on trust will eventually be asked to show them the numbers.</p><h4>Closing</h4><p>Spotify has told us the tool exists, that artists can choose, and that creators would be paid. It has not told us from which pot. The FT has told us a price, and a price answers the least important question.</p><p>The remix money will sit somewhere, and in 2027 a payout report will either name it or absorb it. If you want to know which scenario arrived, do not watch the launch. Watch the accounting.</p><p><em>A price tells you what fans pay. Only the payout line tells you who got paid.</em></p><p><em>Syed Ali</em></p><p><em>Founder and CEO, Echonos</em></p><p><em>Syed Ali is cofounder of </em><a href="https://echonos.ai/"><em>Echonos</em></a><em>, an AI music video pipeline for indie artists. He was previously COO at Tabler App (1M+ users, $50K+ MRR, successful exit) and a data science consultant at Deloitte, Booz Allen Hamilton, and Accenture. Find him on LinkedIn: </em><a href="https://www.linkedin.com/in/syedali-/"><em>https://www.linkedin.com/in/syedali-/</em></a></p><h4>Sources</h4><p>1. Spotify Newsroom, 16 October 2025: partners Sony Music Group, Universal Music Group, Warner Music Group, Merlin and Believe; “artists and rightsholders will choose if and how to participate”; creators “properly compensated for uses of their work.” <a href="https://newsroom.spotify.com/2025-10-16/artist-first-ai-music-spotify-collaboration/">https://newsroom.spotify.com/2025-10-16/artist-first-ai-music-spotify-collaboration/</a></p><p>2. Music Business Worldwide, 30 October 2026, relaying a Financial Times report: first quarter 2027 launch, possibly around the 7 February 2027 Grammys; price of “$4 to $5 per month, on top of existing subscription fees”; deals with UMG (21 May 2026), Merlin (4 August), Kobalt (13 August) and Believe (20 October); Sony and Warner “still negotiating”; Spotify declined to comment; Gustav Söderström at a Goldman Sachs conference, 9 September 2026: “If one of your fan creates a remix of your song, that actually increases the share of the streaming pool that you get.” <a href="https://musicbusinessworldwide.com/spotify-plans-q1-2027-launch-for-ai-covers-and-remixes-tool-weighing-4-5-monthly-price-report/">https://musicbusinessworldwide.com/spotify-plans-q1-2027-launch-for-ai-covers-and-remixes-tool-weighing-4-5-monthly-price-report/</a></p><p>3. Hits Daily Double, 30 October 2026, relaying the same FT report: remix plays may count toward the original track’s charts; Bernstein analyst Ian Moore estimates 7% to 8% of US subscribers could sign up by the end of 2028. <a href="https://www.hitsdailydouble.com/news/streaming/spotify-ai-music-remix-service-2026-10-30">https://www.hitsdailydouble.com/news/streaming/spotify-ai-music-remix-service-2026-10-30</a></p><p>4. Music Business Worldwide, 4 August 2026, on Spotify’s second quarter call: Söderström, “we do not need a deal with all the majors”; a paid add on for Premium; Alex Norström’s “3 Cs” of consent, credit and compensation. <a href="https://musicbusinessworldwide.com/spotify-says-it-doesnt-need-deals-with-all-the-major-labels-to-launch-its-ai-covers-and-remixes-tool">https://musicbusinessworldwide.com/spotify-says-it-doesnt-need-deals-with-all-the-major-labels-to-launch-its-ai-covers-and-remixes-tool</a></p><p>5. Spotify Newsroom, 11 March 2026, Loud &amp; Clear highlights: Spotify paid “more than $11 billion” in 2025 and “nearly $70 billion” to date; “roughly half of those royalties were generated by independent artists and labels.” <a href="https://newsroom.spotify.com/2026-03-11/loud-and-clear-music-economics-highlights/">https://newsroom.spotify.com/2026-03-11/loud-and-clear-music-economics-highlights/</a></p><p>6. Music Business Worldwide, 13 October 2026, on Deezer: a toggle to hide fully AI generated tracks from search, rolling out globally before the end of 2026; about 90,000 fully AI tracks delivered a day; more than 50% of new deliveries on peak days; 1% to 3% of streams. <a href="https://www.musicbusinessworldwide.com/deezer-to-let-users-block-ai-generated-music-from-search-results/">https://www.musicbusinessworldwide.com/deezer-to-let-users-block-ai-generated-music-from-search-results/</a></p><p>7. Engadget, 21 May 2026, on the Universal deal: a paid Premium add on, with some of its revenue going to the original artists; artists must opt in. <a href="https://www.engadget.com/2178670/">https://www.engadget.com/2178670/</a></p><p>8. The AI Musicpreneur, summary of Spotify’s third quarter 2026 call: Norström declined to discuss terms for 2027 add on pricing in label renewals; KeyBanc estimated in September that AI music features could add about 2% to revenue per listener. <a href="https://www.aimusicpreneur.com/ai-music-news/spotify-q3-2026-earnings-call-7-ai-takeaways/">https://www.aimusicpreneur.com/ai-music-news/spotify-q3-2026-earnings-call-7-ai-takeaways/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9d1170dea46e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How Long Does Your Seed Round Actually Have to Last?]]></title>
            <link>https://medium.com/@syed_ali/how-long-does-your-seed-round-actually-have-to-last-7cd6259f4187?source=rss-08b3001202b0------2</link>
            <guid isPermaLink="false">https://medium.com/p/7cd6259f4187</guid>
            <category><![CDATA[fundraising]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[entrepreneurship]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[venture-capital]]></category>
            <dc:creator><![CDATA[Syed Ali]]></dc:creator>
            <pubDate>Fri, 13 Nov 2026 11:39:51 GMT</pubDate>
            <atom:updated>2026-11-13T11:39:51.619Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Editorial cover reading Size runway to the wait, Why a seed round should last 1,000 days: two oak planks reach from a clay cliff over a gap; the short one stops halfway, the long one reaches the far cliff." src="https://cdn-images-1.medium.com/max/1024/1*woY0zKTkfwU_aTwAHyGACg.png" /><figcaption>A short plan ends halfway across the wait; a long plan reaches the other side. Image created by the author with AI.</figcaption></figure><p><em>Startup runway at seed should be sized to the wait between rounds, not to a milestone. On Carta&#39;s data the slow tail now runs three years or longer. Here is the arithmetic, the three levers that move it, and a check to run every month.</em></p><p>Most seed stage plans I see size runway to a milestone: eighteen months, a product launch, a revenue target. The data on how long founders then wait for the next round says the plan should be sized to the wait. Carta reported in October 2025 that 39% of startups took three or more years to raise a Series A after their prior round.</p><p>The short answer first. A startup runway is cash divided by net monthly burn. At seed, plan it against the reported gap between rounds rather than against a milestone, which on Carta&#39;s 2025 figures means planning for something close to 1,000 days, plus the months the raise itself takes.</p><p>My background is capital markets and corporate strategy. I built models for clients at Deloitte, Booz Allen Hamilton and Accenture, then ran operations as COO at Tabler, and I now run Echonos. The habit those seats leave you with is simple: a plan is only as good as its slow case.</p><p>So the argument here is narrow. An eighteen month plan is a bet that your company lands in the fast part of the distribution. The reported data says most companies do not. Build the plan for the slow tail, and treat a fast round as upside rather than as the base case.</p><p>This is written for founders between pre seed and seed who are deciding how fast to hire with the money in the bank, including AI startups whose inference bill moves with usage. It is not for founders holding a Series A term sheet, and it is only partly for founders who never plan to raise again, although the monthly check at the end works for them too.</p><h4>What the round timing data says</h4><p>Start with the middle of the distribution. Carta reported in February 2025 that in 2024 the median company raising a Series A did so 2.1 years after its seed round. That is about twenty five months. The median company, in other words, had already blown through an eighteen month plan by more than half a year before its next primary round closed.</p><p>Now the tails. Across 3,000 or more US startups on Carta, Peter Walker reported in October 2025 that, so far that year, only 15% got to the next round in less than a year, 24% raised their A in 1.5 to 2 years after the prior round, and 39% raised it three or more years after. By my arithmetic the remaining fifth or so sit in the buckets between those. The single largest group is the slowest one.</p><figure><img alt="Animation of three cards: 15% under 1 year, 24% 1.5 to 2 years, 39% 3 or more years. Each number counts up from zero and is underlined while its clay stack grows, ending on the tallest stack for 3 or more years. Source line: Carta, October 2025, 3,000+ US startups." src="https://cdn-images-1.medium.com/max/1024/1*TgKDwIM9TMKOzH5nqsE1xA.gif" /><figcaption>Time from seed to Series A on Carta in 2025: the largest group is the slowest. Animation created by the author with AI.</figcaption></figure><p>SaaStr&#39;s Jason Lemkin, writing about the same Carta dataset (3,365 US startups that raised a Series A between Q1 2018 and Q3 2025), put the change over time in one line. In Q3 2025, 23% raised within 1.5 to 2 years, down from 41% in Q3 2019, and 39% took three or more years, up from 19% in Q3 2019. The 24% and 23% figures differ because they cover different periods. They are cuts of one dataset, not two independent checks.</p><p>The rule of thumb was already under pressure three years ago. In October 2023 Carta published a note titled &quot;18 to 24 Months of Runway May Not Be Enough Anymore&quot;, reporting that the average startup was taking over two years to raise a Series A after a priced seed. The tail has grown since. I made a related point about funding headlines in <a href="https://medium.com/@syed_ali/the-159b-headline-is-not-your-seed-market-7f5352e60f04">an earlier piece on what the seed market headline hides</a>: the aggregate number rarely describes the company reading it.</p><p>One more figure matters for planning. Carta noted in February 2025 that about 32 to 36 percent of companies end up raising bridge capital before their next primary round, which it described as typically expensive. Roughly one company in three goes back to its existing investors to buy time. That is what an undersized plan costs when the slow case arrives.</p><p>A caveat that belongs here, not in a footnote: every load bearing timing figure in this piece rests on Carta, reported across several of its own pages and cut again by SaaStr. I looked for an independent cut of seed to A timing and did not find one I could quote, so read these as one large dataset rather than as consensus.</p><h4>Startup runway is a distribution, not a number</h4><p>The definition is short. Runway in months is cash on hand divided by net monthly burn, where net burn is cash out minus cash in. A company with $3 million and a net burn of $100,000 a month has thirty months.</p><p>That number answers the wrong question on its own. What a founder needs is the date the next round has to close, and that date sits earlier than the date the cash runs out by the length of the raise. I assume six months from first meeting to money in the bank. That is my working assumption, not a sourced figure, and you should replace it with your own experience.</p><p>Put the two together and an eighteen month plan looks different. Six months of raising means the process starts at month twelve. On Carta&#39;s 2025 figures, only 15% of companies raised their next round in under a year. The 39% who took three years or more would run out of cash at roughly the halfway point of their wait unless a bridge filled the gap.</p><p>Carta&#39;s own guidance, in the October 2025 post, is blunt: &quot;If you&#39;re not planning on making your seed round cash last for 1,000 days (2.7 years), you should!&quot; A thousand days is about thirty three months. That figure is Carta&#39;s, and I think it is the right anchor because it sits near the slow end of the reported distribution without assuming the very worst case.</p><p>Here is the test. Write down the month your cash hits zero at current net burn. Subtract your raise window. If the result is earlier than month twenty five, the 2024 median, your plan only works if you beat the median company. Nothing wrong with planning to beat it. Just know that you are.</p><h4>A worked example (author calculation, labeled illustrative)</h4><p>Every number in this section is mine and illustrative. Swap in your own.</p><p>Carta noted in February 2025 that the median seed round on its platform is something like $3.5 million. Spread over 1,000 days, about thirty three months, that allows an average net burn of roughly $106,000 a month. Over thirty months it is about $117,000. Over eighteen months it is about $194,000. A team that sizes runway to eighteen months is allowing itself to spend nearly twice as fast every month as a team sizing to 1,000 days.</p><figure><img alt="Worked example titled Same seed, three spend rates: a 3.5M dollar median seed divided by months of runway gives the allowed monthly net burn, 194K dollars a month over 18 months, 117K over 30 months and 106K over 1,000 days. Illustrative author calculation." src="https://cdn-images-1.medium.com/max/1024/1*8Y0Y9fWSqLOpr3VCYyYA6A.png" /><figcaption>The same $3.5M median seed allows very different monthly burn depending on the runway target. Image created by the author with AI.</figcaption></figure><p>Burn is never flat, so here is a shape closer to a real seed company:</p><p>Months one to six: the founding team builds, net burn $70,000 a month. Cash used: $420,000.</p><p>Month seven onward: three hires at a fully loaded $15,000 a month each lift net burn to $115,000.</p><p>Remaining cash at month six: $3.08 million, which lasts about 26.8 months at $115,000.</p><p>Total runway comes out near 32.8 months, almost exactly 1,000 days. With a six month raise window, the raise has to start by about month twenty seven. Add a three month cushion for a slow process and the real start date is month twenty four, two years in. No revenue is assumed, no interest on cash, and no bridge.</p><h4>The three burn levers a founder can pull</h4><p>Three things move the end date. They do not move it equally.</p><p>Hiring pace. Delay those three hires by one quarter and you save three people for three months at $15,000 each, $135,000 in total. At $115,000 a month, that buys about 1.2 months, roughly five weeks. Timing alone is a smaller lever than most founders expect. The level matters more: make it two hires instead of three, burn falls to $100,000, and the $3.08 million lasts 30.8 months instead of 26.8. Four extra months, from one fewer seat.</p><p>Variable costs, and for AI startups, inference. Most seed burn is payroll, which you control. Inference is different, because the unit price is set by a model provider and the volume is set by your users. Suppose usage grows faster than revenue and the inference bill runs $20,000 a month above plan from month seven. Burn becomes $135,000, the remaining cash lasts about 22.8 months, and the company loses roughly four months of runway without hiring anyone. For an AI product the operational answer is to track inference cost per active user every month, and to treat any free usage as a burn line with a cap rather than as marketing.</p><p>The date of first revenue or milestone. Revenue reduces net burn directly. Say the company reaches $20,000 in monthly revenue at month eighteen. By then it has spent $1.8 million, and the remaining $1.7 million at a net burn of $95,000 lasts about 17.9 months. Total runway moves from 32.8 to about 35.9 months, three extra months. Pulling first revenue forward by a quarter is worth more than delaying a quarter of hiring, which is the comparison most founders never run.</p><figure><img alt="Four cards titled What moves the end date: delaying 3 hires one quarter adds 1.2 months, two hires instead of three adds 4 months, inference 20K dollars over plan removes 4 months, and 20K dollars of revenue at month 18 adds 3 months. Illustrative inputs, base case 3.5M dollars lasts 32.8 months." src="https://cdn-images-1.medium.com/max/1024/1*TVSrUDBQKKRg2zJa8kmEiA.png" /><figcaption>Illustrative levers: how hiring, inference spend and first revenue move the end date. Image created by the author with AI.</figcaption></figure><h4>Counterposition: Where this thesis breaks</h4><p>Three counters deserve a real answer.</p><p><strong>The data only sees the companies that made it</strong></p><p>Carta says this itself. The October 2023 note reads: &quot;Obviously this data can only show companies that actually raised their next round.&quot; The figures also cover US companies on Carta, not every startup. The real distribution of how long founders try before they raise, or stop trying, is unknown from this data.</p><p>My response: survivorship cuts toward the slow case, not away from it. Companies that never raised do not appear, so the true picture for a company starting today is unlikely to be faster than the published one. I cannot say how much the full market differs, and the piece should not pretend the dataset is the whole market.</p><p><strong>Not every company is aiming at a Series A</strong></p><p>A founder planning to reach profitability, or to raise one small round and stop, does not need to size runway to a Series A at all. Fair. The model still applies; the milestone simply becomes break even instead of a priced round. Cash divided by net burn, minus the time to reach break even, is the same arithmetic, and the monthly check below is identical.</p><p><strong>Speed is itself a signal</strong></p><p>This is the counter I take most seriously. Investors read speed. A company that raises its A quickly is telling the market something about traction, and a team that plans for 1,000 days can drift into moving like a team with 1,000 days. There is a real risk that a long plan becomes a slow company.</p><p>My response: planning for 1,000 days is not a plan to be slow. It is protection for the case where the fast path does not arrive, and on Carta&#39;s figures that is most cases. The aggressive team can still set an eighteen month milestone internally. What changes is the spend rate attached to it.</p><p>I am not claiming every seed company should spend $106,000 a month, and I am not claiming 2027 will look like 2025. Two scenarios fit from here. In one, round gaps keep lengthening and 1,000 days becomes the floor. In the other, the A market loosens and the fast bucket grows back toward its 2019 share. The signal that tells you which is arriving is the next Carta quarterly seed to A distribution, specifically the share taking three years or more. Watch that one number.</p><h4>A monthly check</h4><p>Three numbers, reviewed on the same day each month:</p><p>Months of runway at current net burn. Cash divided by the average net burn of the last three months, not the budget.</p><p>Months to the next milestone. Your honest estimate of when the thing that justifies the next round will be true.</p><p>The date the raise must start. The zero cash date, minus your raise window, minus a cushion.</p><p>The decision rule: if months to the milestone is larger than runway minus the raise window, something changes this month. Cut a line of burn, shrink the milestone, or start the raise conversations early. Waiting a month never improves that comparison.</p><p>A spreadsheet layout you can copy is one row per month with five columns: month, cash at month end, net burn, trailing three month average burn, and the computed raise start date. The last column is the one to watch. When it moves earlier two months in a row, you have a problem the board should hear about from you first. I made a similar argument in <a href="https://medium.com/@syed_ali/your-north-star-metric-is-probably-the-wrong-single-number-c05c69416f8d">a piece on north star metrics</a>: one headline number hides the tree underneath it, and runway is a headline number.</p><figure><img alt="Checklist titled Three numbers, every month: 1 months of runway at current net burn, 2 months to the next milestone, 3 the date the raise must start, above the rule Milestone later than runway minus raise window? Act this month." src="https://cdn-images-1.medium.com/max/1024/1*YSGK1nVvXYn0N16E8mGwkQ.png" /><figcaption>Three numbers to review every month. Image created by the author with AI.</figcaption></figure><h4>Closing</h4><p>The plans I see are sized to a milestone. The reported data says the money has to last through the wait, and the wait, for the largest group of companies in Carta&#39;s 2025 data, is three years or more. The milestone still matters. It just does not set the budget.</p><p>One move before next Friday: put your three numbers in a sheet, compute the date your raise must start, and compare it with month twenty four. If your date comes first, decide this month which lever you will pull.</p><p>Plan for the wait. Treat the fast round as a gift, not a forecast.</p><p><em>Syed Ali</em></p><p><em>Founder and CEO, Echonos</em></p><p><em>Syed Ali is cofounder of </em><a href="https://echonos.ai/"><em>Echonos</em></a><em>, an AI music video pipeline for indie artists. He was previously COO at Tabler App (1M+ users, $50K+ MRR, successful exit) and a data science consultant at Deloitte, Booz Allen Hamilton, and Accenture. Find him on LinkedIn: </em><a href="https://www.linkedin.com/in/syedali-/"><em>https://www.linkedin.com/in/syedali-/</em></a></p><h4>Sources</h4><ol><li>Carta (Peter Walker), &quot;Ignore Headlines About Startups Raising A Rounds in Six Months&quot;, 28 October 2025. Sample: &quot;3,000+ startups on Carta (all US companies)&quot;. &quot;24% of startups raised their A in 1.5 to 2 years after the prior round&quot; (range written out); &quot;39% of startups raised their A 3+ years after the prior round.&quot;; &quot;only 15% got to the next round in less than a year.&quot;; &quot;If you&#39;re not planning on making your seed round cash last for 1,000 days (2.7 years), you should!&quot; <a href="https://carta.com/data/linkedin-seed-to-series-a-time-ignore-headlines/">https://carta.com/data/linkedin-seed-to-series-a-time-ignore-headlines/</a></li><li>2. Carta (Peter Walker), &quot;Founders Should Plan for Much Longer Between Rounds&quot;, 6 February 2025: &quot;in 2024 the median company raising a Series A did so 2.1 years after their seed round&quot;; &quot;About 32 to 36% of companies will end up raising bridge capital before their next primary round (which is typically expensive)&quot; (range written out); &quot;the median Seed round on Carta these days is something like $3.5M&quot;. <a href="https://carta.com/data/linkedin-time-between-rounds-lengthened-2025/">https://carta.com/data/linkedin-time-between-rounds-lengthened-2025/</a></li><li>3. SaaStr (Jason Lemkin), analysis of Carta data, published November 2025: &quot;Carta just analyzed 3,365 US startups that raised a Series A between Q1 2018 and Q3 2025.&quot; Q3 2025: 23% raised within 1.5 to 2 years (41% in Q3 2019); 39% took three or more years (19% in Q3 2019). Same dataset as source 1. <a href="https://www.saastr.com/your-seed-round-now-needs-to-last-3-years-what-3365-startups-tell-us-about-the-new-series-a-timeline">https://www.saastr.com/your-seed-round-now-needs-to-last-3-years-what-3365-startups-tell-us-about-the-new-series-a-timeline</a></li><li>4. Carta (Peter Walker), &quot;18 to 24 Months of Runway May Not Be Enough Anymore&quot;, 27 October 2023: &quot;The average startup now takes over 2 years to raise a Series A after their priced Seed round.&quot;; &quot;Obviously this data can only show companies that actually raised their next round&quot;. <a href="https://carta.com/data/linkedin-startup-runway-18-24-months-not-enough/">https://carta.com/data/linkedin-startup-runway-18-24-months-not-enough/</a></li><li>5. Worked example, burn levers and monthly check: author calculations, illustrative. The six month raise window is an author assumption with no source.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7cd6259f4187" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Price of One Second: Two Scenarios for AI Video Through 2027]]></title>
            <link>https://medium.com/@syed_ali/the-price-of-one-second-two-scenarios-for-ai-video-through-2027-807b4acd6b02?source=rss-08b3001202b0------2</link>
            <guid isPermaLink="false">https://medium.com/p/807b4acd6b02</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[pricing-strategy]]></category>
            <category><![CDATA[creator-economy]]></category>
            <category><![CDATA[generative-ai]]></category>
            <dc:creator><![CDATA[Syed Ali]]></dc:creator>
            <pubDate>Wed, 11 Nov 2026 11:40:52 GMT</pubDate>
            <atom:updated>2026-11-11T11:40:52.176Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Editorial cover reading THE PRICE OF ONE SECOND with the subhead Two scenarios through 2027, one signal to watch; a frosted glass cube sits where a teal staircase steps down and a sienna plank stays level." src="https://cdn-images-1.medium.com/max/1024/1*97S-hBc3f3LJEoHviyM6vA.png" /><figcaption>One price per second, two possible paths through 2027. Image created by the author with AI.</figcaption></figure><p><em>AI video generation pricing is now quoted per second, and the cheap end keeps getting cheaper while the top tiers hold. Two futures fit the evidence through 2027. One observable signal tells you which one is arriving.</em></p><p>In AI video, the cost of goods sold is now a number per second. One major provider cut that number by almost half in a single year, while GPU rental prices rose in the same stretch. Both facts are true, and they point in opposite directions. The useful question is which one wins by 2027.</p><p>The short answer on AI video generation pricing, as of 11 November 2026: API list prices run from about three cents to about 78 cents per second of output, a spread of roughly 26 times, according to the UGC Copilot price index of 1 October 2026 and the provider pages I checked against it. Resolution, audio and model tier decide where a clip lands.</p><p>The two facts in that opening are on the record. The Decoder reported on 9 September 2025 that Google cut Veo 3 video with audio from $0.75 to $0.40 per second, a 46.7% cut for that one model. In April 2026, KuCoin’s news desk, citing Ornn’s Compute Price Index, reported that the spot rental price of an Nvidia Blackwell GPU rose from $2.75 to $4.08 an hour in two months, a 48% increase. One line on the cost sheet fell. The input underneath it rose.</p><p>Before Echonos I spent years as a data science consultant at Deloitte, Booz Allen Hamilton and Accenture, and later ran operations as COO of Tabler, so my habit with any price is to ask what it is a price of and who sets it. Nothing here is about how to make a good video; it is about a cost line and the margins that sit on top of it. My read is this: the cheap tiers and the premium tiers are now moving separately, and the question for 2027 is whether the standard tier follows the cheap end down or the premium end sideways.</p><p>This piece is written for founders who resell generated video inside a product, and for creator businesses that buy it by the second and price their work per clip. If you want this week’s cheapest API, a comparison site will serve you better.</p><h3>The number that matters</h3><p>A per second price is the creator’s cost of goods sold. Everything else on the invoice is fixed or small; the second of output is the unit that scales with the work.</p><p>Per second also beats per clip for comparison, because providers sell clips of different lengths. Luma’s API page bills video in five second blocks and lists a Ray3.2 clip by block, while Runway quotes credits per second and Google’s Vertex page quotes a price “/ 1 count”. Put each of them on a per second basis and they can finally sit in the same column.</p><p>But per second has a trap. It is a list price for one generation, and a creator does not buy generations. A creator buys usable clips. If a usable clip takes four attempts, the real cost of goods is four times the list price, and that multiple is invisible on every pricing page I read.</p><p>So hold two numbers at once. The list price per second is published and moves in public. The cost per usable second depends on your prompts, your model choice and your standard for done, so only you can measure it. Most of this piece is about the first number; the action at the end is about the second.</p><h3>AI video generation pricing today</h3><p>Here are the list prices per second, with each source and the date it was read. Prices change monthly, so treat every line as a snapshot.</p><ul><li><strong>Google Veo 3.1 Lite, video with audio:</strong> $0.05 at 720p, $0.08 at 1080p (Vertex AI pricing page, 31 October and 11 November 2026)</li><li><strong>Google Veo 3.1 Fast, video with audio:</strong> $0.10 at 720p, $0.12 at 1080p, $0.30 at 4K (Vertex AI pricing page, same dates)</li><li><strong>Google Veo 3.1 Standard, video with audio:</strong> $0.40 at 720p and 1080p, $0.60 at 4K (Vertex AI pricing page, same dates)</li><li><strong>Runway gen4_turbo:</strong> 5 credits, so $0.05 (Runway API docs, $0.01 per credit, 11 November 2026)</li><li><strong>Runway gen4.5:</strong> 12 credits, so $0.12 (Runway API docs, same date)</li><li><strong>Luma Ray3.2:</strong> about $0.03 at 540p, $0.06 at 720p, $0.24 at 1080p (Luma API page: 0.15, 0.30 and 1.20 per five second block)</li><li><strong>Kling 3.0, 720p with audio:</strong> $0.126 (UGC Copilot index, 1 October 2026)</li><li><strong>Seedance 2.0 on BytePlus:</strong> $0.152 at 720p, $0.778 at 4K (UGC Copilot index, 1 October 2026)</li><li><strong>Seedance 2.0 Fast on BytePlus:</strong> $0.12 at 720p (UGC Copilot index, 1 October 2026)</li></ul><p>Three labels matter here. The Google page quotes “/ 1 count”, and the UGC Copilot index treats a count as one second of video; I follow the index and say so. The Luma page prices in credits and lists 0.15, 0.30 and 1.20 in its FAQ without naming a currency; the index reads those as dollars, and so do I. And the Kling and Seedance lines come through the index only, because those provider pages render in the browser and did not return price text to my research fetch. Treat those three lines as one third party’s reading.</p><p>The same index records one exit. It states that OpenAI’s Sora API “was retired on September 24, 2026”. That removes one supplier from the list. It does not, on its own, tell anyone what video costs to make.</p><p>Now the arithmetic a creator cares about. A thirty second clip at $0.05 a second is $1.50 before retries. The same thirty seconds on Veo 3.1 Standard is $12. At the 4K Seedance rate it is $23.34. Same length, same buyer: eight times from Google’s cheapest tier with audio to its standard tier, and more than fifteen times to the 4K top of the index.</p><figure><img alt="Three cards titled One clip, three prices: Veo 3.1 Lite 720p costs $1.50 at $0.05 a second, Veo 3.1 Standard $12 at $0.40 a second, Seedance 2.0 4K $23.34 at $0.778 a second. Each price counts up in turn while its column grows from the base." src="https://cdn-images-1.medium.com/max/1024/1*6D6rdEtTQ-9JxHQOveyjtA.gif" /><figcaption>The same thirty second clip at three list prices, before retries. Animation created by the author with AI.</figcaption></figure><h3>What a list price hides</h3><p>Three gaps sit between the list and a real cost model, and most of the pricing coverage I read skips all three.</p><p>The first is the audio toggle. In the UGC Copilot index, Kling 3.0 at 720p is $0.084 a second with audio off and $0.126 with audio on, so sound alone moves the price by half again. A cost model that ignores whether audio is included can be wrong by 50% before anything else goes wrong.</p><p>The second is the billing block. When a provider bills in five second blocks, a seven second clip is billed as ten. For short form work, rounding can be a bigger line than the price difference between two providers, and it never shows up in a per second headline.</p><p>The third is the compute signal itself. The 48% Blackwell rise is one index reported through one news desk. KuCoin’s desk also carried a piece on 8 April 2026 noting that four GPU index providers on Bloomberg terminals “offer quotes differing by more than $2”, and a later item on 27 May 2026 reporting that an Ornn H100 compute product “pulled back by roughly 3% to 9%” after its May high. GPU prices are moving, and the readings are noisy. A video price model built on one GPU print is a model of the noise.</p><figure><img alt="List titled Three costs the headline skips: audio toggle, $0.084 to $0.126 a second for Kling 3.0 at 720p; billing block, 7 seconds billed as 10; retries, 4 attempts at 4x the list price, marked illustrative." src="https://cdn-images-1.medium.com/max/1024/1*_p75ycIgKDjDaFqYGNwwEQ.png" /><figcaption>Three costs that a per second list price leaves out. Image created by the author with AI.</figcaption></figure><h3>Scenario A: toward commodity</h3><p>In the first scenario, the cheap end of the list keeps falling and pulls the standard tier down behind it by 2027.</p><p>The evidence for it is the history of these prices. Veo 3 with audio went from $0.75 to $0.40 a second in September 2025, and Veo 3 Fast went from $0.40 to $0.15 in the same update, per The Decoder. A year later the Vertex page lists a Veo 3.1 Lite tier at $0.05 a second at 720p. Runway’s gen4_turbo sits at the same $0.05. Seedance has a Fast tier priced below its standard model in the index. Most providers in the list now sell a cheaper tier or resolution, and the cheap rates cluster around five cents.</p><p>The mechanism, in my interpretation, is that cheap tiers act as loss leaders for the rest of the menu. I have no provider margin data and I am not claiming any tier is sold below cost; I could not find a single published margin figure. The shape is familiar from other software markets: the low tier wins the developer, and the provider earns on volume and on the upgrades that follow.</p><p>In this scenario, a creator business wins first. At $1.50 for thirty seconds, several variants per brief are affordable, and the cost of goods stops deciding whether a project is profitable. I wrote about the same two scenario structure for language models in <a href="https://medium.com/@syed_ali/ai-inference-cost-and-gross-margin-two-scenarios-for-2027-and-the-signal-that-picks-one-9a3db02292c5">AI inference cost and gross margin</a>; video is the same question with a bigger unit.</p><h3>Scenario B: flat or rising</h3><p>In the second scenario, the cheap tiers stay cheap and stay narrow, while the tiers a professional buyer needs hold their price or rise through 2027.</p><p>The evidence is in the same list. The Veo 3.1 Standard tier is $0.40 a second, which is exactly where Veo 3 landed after the September 2025 cut, more than a year earlier. The 4K price is $0.60. Seedance 2.0 at 4K is $0.778 a second in the index, the top of the whole range. The big cut I could document happened once, on one model, and the standard rate today sits where that cut left it.</p><p>The mechanism here, as I read it, is supply. If GPU rental keeps rising the way the April 2026 Ornn print showed, providers have a reason to keep premium output scarce and priced, and to steer volume toward the smaller models that are cheaper to serve. In that world, the Lite tier is real but it is a different product, and the price of a clip a client would accept barely moves.</p><p>In this scenario, a founder who priced a product on the assumption of falling costs is the one exposed. The margin plan said the cost of goods would halve again. It did not, and the price promised to customers has already been set.</p><figure><img alt="Two cards titled Two futures, one tier to watch: Scenario A, toward commodity, cheap tiers fall and the standard tier follows down; Scenario B, flat or rising, cheap tiers stay narrow and the standard tier holds or rises. A band reads The signal: the standard tier price at constant quality." src="https://cdn-images-1.medium.com/max/1024/1*yoY93OIU5uqusUiJMsLQ5A.png" /><figcaption>Two scenarios for AI video pricing and the one signal that separates them. Image created by the author with AI.</figcaption></figure><h3>Where this thesis breaks</h3><p>You deserve the version of this argument where I am wrong. Here are the three counters I take seriously.</p><p><strong>“Per second list price is the wrong unit, because retries dominate.”</strong> This is the strongest one, and I concede most of it. If a usable clip takes four attempts on one model and one attempt on another, the cheaper list price can be the more expensive clip. I could not find a published, verifiable retry rate I would cite. My response is to fold the retry multiple into the signal: a list price cut that comes with a higher retry rate is not a real cut, and you can only see that from your own logs.</p><p><strong>“List prices are subsidized and will not last.”</strong> Maybe. I do not know, and nobody outside the providers does, because no provider margin data was found in this research. I am not claiming any named provider prices below cost. If the cheap tiers are subsidized and the subsidy ends, that is Scenario B arriving through a different door, and the same signal catches it.</p><p><strong>“The Sora retirement proves AI video is uneconomic.”</strong> It does not prove that. The UGC Copilot index records that OpenAI retired the Sora API on 24 September 2026. That is one company’s decision about one product. One exit is a data point about a supplier, not a cost curve for an industry.</p><p>I am not claiming prices will fall, and I am not claiming they will rise. I am not saying which scenario I find more likely, because the honest answer is that the evidence fits both. Pick whichever counter position is most persuasive to you. The practical implication is the same: watch the standard tier, and measure your own cost per usable second.</p><h3>The one signal</h3><p>Everything above comes down to one observation you can make once a month: what happens to a provider’s standard quality tier.</p><p>Not a new Lite tier: both scenarios predict more of those. Watch the tier a paying client would accept, held at constant quality.</p><p><strong>Signals Scenario A:</strong> a provider cuts the price of an existing standard tier, such as Veo 3.1 Standard at $0.40 a second, with the model and resolution unchanged. One cut is a data point. Two providers within a quarter is a trend.</p><p><strong>Signals Scenario B:</strong> a standard tier rises in price, gets a lower rate limit, moves behind a waitlist, or is quietly replaced by a successor at the same or higher price.</p><p><strong>How to run the check:</strong> pick three providers you actually use. On the first working day of each month, record the per second price of their standard tier at the resolution you deliver, with the date and the page it came from. Add your own retry multiple from last month’s logs. Twelve rows will tell you more than any forecast, including this one.</p><figure><img alt="Four step takeaway titled Watch the standard tier: pick three providers you use, record the standard tier price per second, add your retry multiple, then a cut means Scenario A and a rise, rate limit or waitlist means Scenario B. Footer: First working day of every month." src="https://cdn-images-1.medium.com/max/1024/1*JgALbsWXgNJ1mHjUXSHDIg.png" /><figcaption>The monthly check: watch the standard tier. Image created by the author with AI.</figcaption></figure><h3>What to do before your next pricing review</h3><p>If you resell generated video inside a product, write your margin model with two cost lines, one per scenario, and set your customer price so the product still earns at the flat case. A price you can only defend under Scenario A is a bet, and you do not have to make it.</p><p>If you run a creator business and buy video by the second, start logging attempts per usable clip this week. In thirty days you will know your real cost per usable second, which is the only number in this piece that is yours alone.</p><h3>Who loses in each scenario</h3><p>If Scenario A lands, the losers are the businesses whose only edge was access to cheap generation. When thirty seconds of video costs $1.50 for everyone, cost is not a moat, and the advantage moves to taste, distribution and the customer relationship.</p><p>If Scenario B lands, the losers are the founders who priced on a falling curve. Their unit economics were written for a cost of goods that did not arrive, and their customers have already been told the price.</p><p>In both, the buyer who measured cost per usable second and watched the standard tier is the one who is not surprised.</p><h3>Closing</h3><p>The cost of goods in AI video is a number per second, and that number is now really two numbers: a cheap tier that keeps falling and a standard tier that sits today at $0.40, where the September 2025 cut left it. The headline cut was real. So was the rise in compute underneath it.</p><p>Which one wins by 2027 will not be announced. It will show up first as a quiet change to a standard tier on a pricing page, months before anyone writes the trend piece. Check the standard tier every month, and you will see it first.</p><p><em>Syed Ali</em></p><p><em>Founder and CEO, Echonos</em></p><p><em>Syed Ali is cofounder of </em><a href="https://echonos.ai/"><em>Echonos</em></a><em>, an AI music video pipeline for indie artists. He was previously COO at Tabler App (1M+ users, $50K+ MRR, successful exit) and a data science consultant at Deloitte, Booz Allen Hamilton, and Accenture. Find him on LinkedIn: </em><a href="https://www.linkedin.com/in/syedali-/"><em>https://www.linkedin.com/in/syedali-/</em></a></p><h3>Sources</h3><ol><li>Google Cloud, Vertex AI generative AI pricing, read 31 October and 11 November 2026. Veo 3.1 Lite, video with audio: “720p $0.05 / 1 count 1080p $0.08 / 1 count”; Veo 3.1 Fast: “720p $0.10 / 1 count 1080p $0.12 / 1 count 4k $0.30 / 1 count”; Veo 3.1 Standard: “720p, 1080p $0.40 / 1 count 4k $0.60 / 1 count”. <a href="https://cloud.google.com/vertex-ai/generative-ai/pricing">https://cloud.google.com/vertex-ai/generative-ai/pricing</a></li><li>Runway API pricing, read 11 November 2026: gen4_turbo “5 credits per second”; gen4.5 “12 credits per second”; “Credits can be purchased for $0.01 per credit in the developer portal for a project.” <a href="https://docs.dev.runwayml.com/guides/pricing/">https://docs.dev.runwayml.com/guides/pricing/</a></li><li>Luma AI API pricing, read 11 November 2026: Ray3.2 text or image to video, 50, 100 and 400 credits per five second clip at 540p, 720p and 1080p; FAQ “540p = 0.15, 720p = 0.30, 1080p = 1.20”; video generations are billed per five second block. <a href="https://lumalabs.ai/api/pricing">https://lumalabs.ai/api/pricing</a></li><li>UGC Copilot, AI video model price index, updated 1 October 2026: “As of October 1, 2026, per second API list prices for AI video models run from $0.03 to $0.778” (hyphen removed); “OpenAI’s Sora API was retired on September 24, 2026”; Kling 3.0 official API, 720p, $0.084 audio off and $0.126 audio on; Seedance 2.0 on BytePlus ModelArk, $0.152 at 720p and $0.778 at 4K; Seedance 2.0 Fast, $0.12 at 720p. <a href="https://ugccopilot.ai/research/ai-video-model-price-index/">https://ugccopilot.ai/research/ai-video-model-price-index/</a></li><li>The Decoder, 9 September 2025: “The standard Veo 3 model with audio now costs $0.40 per second, down from $0.75.” Veo 3 Fast “is now $0.15 per second for video with audio, compared to the previous $0.40.” <a href="https://the-decoder.com/google-veo-3-adds-1080p-916-video-and-drops-prices-by-half/">https://the-decoder.com/google-veo-3-adds-1080p-916-video-and-drops-prices-by-half/</a></li><li>KuCoin News Flash, 13 April 2026, citing Ornn’s Compute Price Index: the Blackwell spot price “has risen to $4.08 per hour, up from $2.75 two months ago”, a 48% increase. Aggregator report of a single index. <a href="https://www.kucoin.com/news/flash/nvidia-gpu-rental-prices-surge-48-in-two-months-amid-ai-industry-s-worst-compute-shortage-in-five-years">https://www.kucoin.com/news/flash/nvidia-gpu-rental-prices-surge-48-in-two-months-amid-ai-industry-s-worst-compute-shortage-in-five-years</a></li><li>KuCoin News Flash (TechFlow), 8 April 2026: “four index providers on Bloomberg terminals offer quotes differing by more than $2, with varying directions and rhythms.” <a href="https://www.kucoin.com/news/flash/gpu-pricing-disarray-four-indices-diverge-market-lacks-consensus">https://www.kucoin.com/news/flash/gpu-pricing-disarray-four-indices-diverge-market-lacks-consensus</a></li><li>KuCoin News Flash, 27 May 2026, on Ornn’s H100 compute product after its May high: “pulled back by roughly 3% to 9%”. <a href="https://www.kucoin.com/news/flash/nvidia-h100-gpu-rental-prices-drop-after-may-surge">https://www.kucoin.com/news/flash/nvidia-h100-gpu-rental-prices-drop-after-may-surge</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=807b4acd6b02" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why Does Prompt Caching Save Less Than the Pricing Page Says?]]></title>
            <link>https://medium.com/@syed_ali/why-does-prompt-caching-save-less-than-the-pricing-page-says-f10667bdfc18?source=rss-08b3001202b0------2</link>
            <guid isPermaLink="false">https://medium.com/p/f10667bdfc18</guid>
            <category><![CDATA[cost-optimization]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[prompt-engineering]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Syed Ali]]></dc:creator>
            <pubDate>Mon, 09 Nov 2026 11:30:49 GMT</pubDate>
            <atom:updated>2026-11-09T11:30:49.437Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Cover reading Prompt caching, Your invoice is the average, with the subhead The write premium, the expiry window, and your real hit rate: a clear hourglass beside a stack of thin aluminium tiles resting on one thick orange tile." src="https://cdn-images-1.medium.com/max/1024/1*s2-YWZ4jxQiGoIa3RB3fRg.jpeg" /><figcaption>The headline price describes the best request, not the bill. Image created by the author with AI.</figcaption></figure><p><em>Prompt caching is a real discount on a narrow slice of your bill. What you keep depends on the write premium, the expiry window and a hit rate measured in dollars.</em></p><p>The pricing page says a cached token costs a fraction of a normal one. That is true, and it is still not the number on your invoice. Three other numbers decide the gap: the write premium, the expiry window, and your real hit rate. Many teams never put all three in the same spreadsheet.</p><p>Prompt caching saves money only when reuse inside the cache lifetime outweighs the write premium. On a busy, stable prefix that can be most of your input bill. On sparse traffic with a short expiry it can cost more than sending the same tokens uncached. Arithmetic decides which one you have.</p><p>My background is data science consulting at Deloitte, Booz Allen Hamilton and Accenture, and the habit that stuck is refusing a unit price until I have seen the denominator. A read multiplier is a unit price. My read is that you should treat it as a ceiling you approach and never reach.</p><p>Three numbers sit between that ceiling and your bill.</p><p><strong>The write premium</strong> is what you pay the first time a prefix enters the cache. It is above the normal input price, and it buys nothing until something reads it.</p><p><strong>The expiry window</strong> is how long a prefix survives without a hit. Miss the window and you pay the write premium again.</p><p><strong>The real hit rate</strong> is the one measured in dollars, not tokens. The two are easy to confuse, and they can sit a long way apart.</p><h4>What the pricing pages promise in November 2026</h4><p>Every figure in this section comes from provider documentation as I read it on 9 November 2026. These pages change often, so check them before you model anything.</p><p><strong>Anthropic&#39;s documentation</strong> states that cache writes with a five minute lifetime cost 1.25 times the base input price, writes with a one hour lifetime cost 2 times, and reads cost 0.1 times, with footnoted exceptions of 0.05 and 0.025 on some newer models. The default lifetime is five minutes. Minimum cacheable length runs from 512 to 4,096 tokens depending on the model, and shorter prompts are &quot;processed without caching, and no error is returned.&quot; That last clause deserves more attention than it gets. A prefix one token too short fails silently.</p><p><strong>OpenAI&#39;s guide</strong> lists a minimum of 1,024 tokens for GPT-5.6 and later, and cached input &quot;discounted up to 95%.&quot; On those newer models the default 30 minute retention is written at 1.25 times the uncached input rate and the 12 hour option at 2.5 times. The guide calls these total write rates, not surcharges on top of ordinary input. Reads are 0.1 times on most of those models.</p><p><strong>Google&#39;s documentation</strong> says implicit caching is on by default for Gemini 2.5 and newer, with a minimum of 4,096 tokens on the newer models. One row of its pricing page, Gemini 3.5 Flash on the standard tier, lists $1.50 per million input tokens, $0.15 cached, and a storage price of $1.00 per million tokens per hour. Treat that as one row on one page.</p><p>The storage line changes the shape of the problem, because it is rent. On that row each read saves $1.35 per million tokens, and each hour of holding the cache costs $1.00 per million. By my arithmetic you need about 0.74 reads per hour, or one every 81 minutes, just to cover the rent.</p><p>No provider is hiding any of this. It is all published. It simply lives in several rows, and the headline only quotes one of them.</p><h4>Prompt caching break even math</h4><p>Derive this yourself instead of borrowing a blog&#39;s number, because published break even figures use different definitions of a reuse and they do not agree.</p><p>Let w be the write multiplier, r the read multiplier, and n the number of reuses after the first request. Normalise the base input price to 1. Without caching, one request plus n reuses costs 1 + n. With caching, the same traffic costs w + n × r. Caching wins when w + n × r is less than 1 + n, which rearranges to n greater than (w minus 1) divided by (1 minus r).</p><p>Run it for the three write prices above, each with a 0.1 read:</p><p><strong>Write at 1.25:</strong> n must exceed 0.28, so one reuse pays. You spend 1.35 instead of 2, which is 32.5% off.</p><p><strong>Write at 2:</strong> n must exceed 1.11, so you need two reuses. One reuse costs 2.1 against 2, a small loss. Two reuses cost 2.2 against 3, which is 26.7% off.</p><p><strong>Write at 2.5:</strong> n must exceed 1.67, so again two reuses. They cost 2.7 against 3, which is 10% off.</p><p>Now hold the write at 1.25 and let reuse grow. One reuse saves 32.5%. Two save 51.7%. Nine save 78.5%. Ninety nine save 88.9%. The 90% on the pricing page is the limit as n goes to infinity, and no prefix you ship will live that long.</p><figure><img alt="Three cards titled How many reuses pay back the write. Write at 1.25: one reuse, 32.5% off. Write at 2: two reuses, 26.7% off. Write at 2.5: two reuses, 10% off. Each percentage counts up from zero, gets underlined, and a small stack of tiles grows from its base." src="https://cdn-images-1.medium.com/max/1024/1*VxCNul0jzYXpNoMpIfi-ug.gif" /><figcaption>How many reuses pay back the write at three write prices, each with a 0.1 read. Animation created by the author with AI.</figcaption></figure><p>That is the first gap, and it is the small one.</p><h4>Where the discount leaks</h4><p><strong>Leak one is expiry.</strong> The formula above assumes every reuse arrives in time. Firecrawl put the failure plainly on 18 October 2026: &quot;A 5-minute cache on requests that arrive more than 5 minutes apart is a loss.&quot; Each request misses, writes again at 1.25 times, and you pay 25% more than with no caching at all.</p><p>Real traffic sits between those extremes, so model it. Let p be the share of requests that arrive inside the lifetime of the previous one. Average cost per request is (1 minus p) × w + p × r, and caching breaks even when p equals (w minus 1) divided by (w minus r). At a 1.25 write and a 0.1 read, that is a p of 21.7%. At a 2 write it is 52.6%.</p><p>If arrivals are random with a mean gap g and a lifetime T, then p is 1 minus e to the power of minus T divided by g. That assumption is mine, and bursty traffic will do better inside a burst and worse between them. It assumes a hit resets the clock, which is how Firecrawl describes the five minute cache refreshing itself. Here is what it gives on a 0.1 read:</p><p><strong>Mean gap of 2 minutes, five minute lifetime at 1.25:</strong> 80.6% saved.</p><p><strong>Mean gap of 10 minutes, five minute lifetime at 1.25:</strong> 20.3% saved.</p><p><strong>Mean gap of 10 minutes, one hour lifetime at 2:</strong> 89.5% saved.</p><p><strong>Mean gap of 60 minutes, five minute lifetime at 1.25:</strong> 15.8% more expensive than no cache.</p><p><strong>Mean gap of 60 minutes, one hour lifetime at 2:</strong> 20.1% saved.</p><p>The pricier write wins by a wide margin once traffic thins out. Under this model the five minute cache stops paying at a mean gap of about 20 minutes, and the one hour cache at about 80.</p><figure><img alt="Grid titled What expiry does to the discount. Mean gap of 10 minutes: five minute lifetime at 1.25 saves 20.3%, one hour lifetime at 2 saves 89.5%. Mean gap of 60 minutes: five minute lifetime is 15.8% more expensive, one hour lifetime saves 20.1%. Illustrative, random arrivals, 0.1 read." src="https://cdn-images-1.medium.com/max/1024/1*Mf6t0XS7odScu5CxKOM7fQ.jpeg" /><figcaption>The same request gaps against two cache lifetimes, under my random arrival assumption. Image created by the author with AI.</figcaption></figure><p><strong>Leak two is dynamic content inside the prefix.</strong> A timestamp, a user name or a shuffled tool list near the top changes the bytes, and everything after the change stops matching. Nothing errors. You just pay to write a new entry on each request.</p><p><strong>Leak three is routing.</strong> A cache belongs to a provider and, in practice, to a model, so a router that spreads traffic thins the reuse on every path. I wrote about measuring that separately in <a href="https://medium.com/@syed_ali/llm-routing-saves-less-than-the-papers-claim-here-is-how-to-measure-your-own-router-c4e06aafd50f">LLM Routing Saves Less Than the Papers Claim</a>. The two savings do not add up, because each one eats the other&#39;s denominator.</p><h4>How to measure your real hit rate</h4><p>Every major API already returns what you need. Anthropic&#39;s usage block has cache_read_input_tokens, cache_creation_input_tokens and input_tokens, and its documentation says total input is the sum of the three. OpenAI&#39;s has cached_tokens and cache_write_tokens under input_tokens_details. Google&#39;s documentation names usage.total_cached_tokens.</p><p>Call those R for reads, W for writes and U for uncached. Then compute two numbers side by side.</p><p>Token hit rate is R divided by (R + W + U). Dollar savings is 1 minus (R × r + W × w + U) divided by (R + W + U).</p><p>Take an illustrative workload of 90 read tokens and 10 written tokens at a 1.25 write and a 0.1 read. The token hit rate is 90%. The cost is 9 for reads plus 12.5 for writes, so 21.5 against 100, or 78.5% saved. Writes are one tenth of the tokens and 58% of the cache spend.</p><figure><img alt="Two cards titled A token hit rate flatters you. Token hit rate 90%, from 90 read tokens and 10 written tokens. Dollar savings 78.5%, reads 9 plus writes 12.5 against 100. A band reads Writes are one tenth of the tokens and 58% of the cache spend." src="https://cdn-images-1.medium.com/max/1024/1*qlgSxKsSigUzwV3LE4qPUg.jpeg" /><figcaption>The same illustrative workload measured in tokens and in dollars. Image created by the author with AI.</figcaption></figure><p>The published numbers show the same pattern. Firecrawl, in its October piece, cites an August 2026 analysis of 63,880 billed Claude Code requests with an aggregate hit rate of 96.59%, where writes carried 76% of the cache input cost in a typical session. That is a secondary citation of one analysis. ProjectDiscovery reported in April 2026 that its hit rate went from 7% to 84% and that caching saved 59% against the same token volume at full input rates. That is one team&#39;s result. My read of the two together is that a token hit rate flatters you, and the distance between it and dollar savings is the number worth watching.</p><p>One more slice before you tell finance anything. The discount only touches cached input. If your prefix is 80% of input tokens, input is 40% of the bill, and you save 78.5% on the prefix, then the bill falls by about 25%. Those inputs are illustrative. Put in your own.</p><h4>Where this thesis breaks: when not to cache, and other caveats</h4><p><strong>Counter one: reads are 90% to 97.5% off, so caching always wins.</strong> On dense traffic with a stable prefix this is close to right, and the numbers above say so. If your gaps are short, turn it on and go back to work. The read price is still only half the formula, and it says nothing about what happens when requests stop arriving in time.</p><p><strong>Counter two: providers cache automatically, so do not engineer it.</strong> This is the one I take most seriously, because automatic caching removes real work, and where there is no write premium the downside mostly disappears. But automatic means no code change. It does not mean no design. ProjectDiscovery&#39;s move from 7% to 84% came from restructuring prompts, and an automatic cache cannot reorder your prefix for you.</p><p><strong>Counter three: semantic caching or batch APIs beat this.</strong> My read, and I have not researched it for this piece, is that they solve different problems and can stack. A semantic cache skips the model call. A batch API trades latency for price. Prompt caching does neither.</p><p>I am not claiming caching is a bad deal, and I am not claiming any provider misstates its prices. The discount is real and the pages say what they say. I am claiming the headline multiplier is the wrong input for a forecast.</p><p>There are cases where none of this applies. If your monthly input spend is small, an afternoon of instrumentation costs more than the leak. If your prefix sits under the provider&#39;s minimum, the question never arises.</p><p>The multipliers will also move, so hold this article&#39;s numbers loosely. One scenario is that write premiums spread to more models and more lifetimes, which makes the math above more important. The other is that competition absorbs them, which leaves expiry and prefix design as the remaining levers. The signal is whether the next revision of each pricing page adds a write premium or removes one.</p><h4>A decision checklist</h4><p>The thresholds here are my starting points, not benchmarks. Adjust them to your traffic.</p><p><strong>Check the minimum.</strong> Confirm the stable prefix is above your provider&#39;s cacheable length for that exact model.</p><p><strong>Clean the prefix.</strong> Move system text, tools and examples to the top and in a fixed order. Timestamps and user fields go after the last cache boundary.</p><p><strong>Pick the lifetime with your own gaps.</strong> Pull a week of request timestamps per prefix, compute p for each lifetime you can buy, and price both.</p><p><strong>Count your paths.</strong> List each distinct pair of prefix and model. Many pairs with thin traffic means routing is eating your cache.</p><p><strong>Log three token classes per request.</strong> Reads, writes and uncached, with the multipliers on the same row.</p><p><strong>Alert in dollars.</strong> I would alert when dollar savings falls ten points below its trailing four week average, or when writes pass half of cache spend.</p><p>The spreadsheet needs eight input columns: date, model, input price per million, w, r, R, W and U. It needs four computed ones: token hit rate, dollars paid, dollars at full input price, and savings percent. One row per model per day is enough.</p><figure><img alt="Checklist titled A decision checklist: Check the minimum, Clean the prefix, Pick the lifetime with your own gaps, Count your paths, Log three token classes per request, Alert in dollars, beside a small aluminium hourglass." src="https://cdn-images-1.medium.com/max/1024/1*khpmeNfU-8xoXiqeHzcehg.jpeg" /><figcaption>Six checks to run before you trust a cache discount. Image created by the author with AI.</figcaption></figure><h4>The move for this week</h4><p>Before next Friday, pull seven days of usage fields and put token hit rate and dollar savings in adjacent columns. If they are close, you are done and can stop reading articles about this. If they are far apart, the writes are your bill, and the lifetime is the first thing to reprice.</p><p>The pricing page quotes the best request you will ever send. Your invoice is the average of all the others.</p><p><em>Syed Ali</em></p><p><em>Founder and CEO, Echonos</em></p><p><em>Syed Ali is cofounder of </em><a href="https://echonos.ai/"><em>Echonos</em></a><em>, an AI music video pipeline for indie artists. He was previously COO at Tabler App (1M+ users, $50K+ MRR, successful exit) and a data science consultant at Deloitte, Booz Allen Hamilton, and Accenture. Find him on LinkedIn: </em><a href="https://www.linkedin.com/in/syedali-/"><em>https://www.linkedin.com/in/syedali-/</em></a></p><h4>Sources</h4><ol><li>Anthropic documentation, prompt caching, read 9 November 2026: cache writes at 1.25 times the base input price for a five minute lifetime and 2 times for a one hour lifetime; reads at 0.1 times with per model exceptions; default lifetime five minutes. <a href="https://platform.claude.com/docs/en/build-with-claude/prompt-caching">https://platform.claude.com/docs/en/build-with-claude/prompt-caching</a></li><li>2. OpenAI documentation, prompt caching, read 9 November 2026: minimum cacheable length of 1,024 tokens on the newer GPT models; cached input &quot;discounted up to 95%&quot;; default retention written at 1.25 times the uncached input rate, 12 hour retention at 2.5 times. <a href="https://developers.openai.com/api/docs/guides/prompt-caching">https://developers.openai.com/api/docs/guides/prompt-caching</a></li><li>3. Google Gemini API pricing, read 9 November 2026, Gemini 3.5 Flash standard tier: input $1.50, context caching $0.15, storage &quot;$1.00 / 1,000,000 tokens per hour&quot;. <a href="https://ai.google.dev/gemini-api/docs/pricing">https://ai.google.dev/gemini-api/docs/pricing</a></li><li>4. Google Gemini API, context caching, read 9 November 2026: &quot;Implicit caching is enabled by default for all Gemini 2.5 and newer models.&quot; <a href="https://ai.google.dev/gemini-api/docs/caching">https://ai.google.dev/gemini-api/docs/caching</a></li><li>5. Firecrawl, 18 October 2026: a short cache on requests spaced beyond its lifetime is a loss, per the quote in the body. Also cites an August 2026 analysis of 63,880 billed Claude Code requests: aggregate hit rate 96.59%, writes 76% of cache input cost in a typical session. <a href="https://www.firecrawl.dev/blog/prompt-caching">https://www.firecrawl.dev/blog/prompt-caching</a></li><li>6. ProjectDiscovery, 10 April 2026, &quot;How We Cut LLM Costs by 59% With Prompt Caching&quot;: &quot;Overall, caching saved 59% on LLM costs compared to what the same token volume would have cost at full input rates&quot;; cache hit rate 7% to 84%. <a href="https://projectdiscovery.io/blog/how-we-cut-llm-cost-with-prompt-caching">https://projectdiscovery.io/blog/how-we-cut-llm-cost-with-prompt-caching</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f10667bdfc18" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI Music Uploads Are Surging. What Share of the Royalty Pool Is That?]]></title>
            <link>https://medium.com/@syed_ali/ai-music-uploads-are-surging-what-share-of-the-royalty-pool-is-that-1560ad37a923?source=rss-08b3001202b0------2</link>
            <guid isPermaLink="false">https://medium.com/p/1560ad37a923</guid>
            <category><![CDATA[streaming]]></category>
            <category><![CDATA[royalty]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[music-industry]]></category>
            <category><![CDATA[data-analysis]]></category>
            <dc:creator><![CDATA[Syed Ali]]></dc:creator>
            <pubDate>Sat, 07 Nov 2026 11:33:36 GMT</pubDate>
            <atom:updated>2026-11-07T11:33:36.642Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Editorial cover reading Ask for the share, not the count, with the subhead Every royalty pool is a fraction: a round brass tray cut into many wedges with one thin teal wedge, beside a tall tower of teal glass tiles." src="https://cdn-images-1.medium.com/max/1024/1*ygXLNSf0hoWbjMI8cw_Q5g.png" /><figcaption>Uploads are a count; royalties are a share of a fixed pool. Image created by the author with AI.</figcaption></figure><p><em>Streaming royalties are paid as a fraction of a pool. The AI upload figures of 2026 describe how much music arrives, and nobody has published the fraction that decides who gets paid less.</em></p><p>On Universal Music Group’s third quarter call, an executive said AI uploads had grown from about ten thousand a day to nearly one hundred thousand, and that current safeguards are not sufficient. I read that as a founder in this space, and the first thing I looked for was the denominator. Every royalty pool is a fraction, and I could not find the fraction.</p><p>Streaming royalties under a pro rata model are paid out of a pool, so the question is never how many tracks arrive. It is what share of the paid listening they take. That share is the number I wanted. Neither the call summary nor the coverage around it gave one.</p><p>The habit of asking for it comes from an earlier job. Before Echonos I spent years as a data science consultant at Deloitte, Booz Allen Hamilton and Accenture, and later ran operations as COO of Tabler. Much of that work came down to one dull question: what is this headline number divided by? Nothing below is about how music is made. It is about how a royalty share is built, and who absorbs it when it moves.</p><p>My read is this. The AI upload numbers of 2026 are supply numbers, and royalty exposure is a demand number: the share of payable streams that AI tracks take from a pool that does not grow because more tracks arrive. On the evidence published so far, that share is small, smaller than the upload headline suggests, and impossible to check from outside. And an anti dilution clause does not make the AI share vanish from the pool. My working assumption is that it decides who absorbs it.</p><p>This is written for analysts who model music companies, founders building in AI music, and label or distributor operators who will be asked about this on their next call. If you want a view on whether AI music is good art, this is the wrong essay.</p><h4>Supply Is Not Demand</h4><p>The short answer first. AI music dilutes streaming royalties only to the extent AI tracks take a share of the payable streams in a pro rata pool. Upload counts do not measure that share. On Deezer’s own figures, fully AI tracks are a large part of what arrives and a small part of what gets played.</p><p>Music Business Worldwide reported on 21 July 2026 that Deezer was receiving nearly 90,000 fully AI generated tracks every day in June, up from 75,000 a day, or 44% of daily deliveries, in April. On peak days that content made up more than 50% of all new music delivered to the service. The same report carried the other half of the picture: fully AI music accounts for between 1% and 3% of all streams on the platform, according to Deezer. ³</p><p>Why the gap? Uploading costs almost nothing, and listening is capped by the hours in a day. Supply can rise tenfold while the number of people pressing play barely moves. So a model that scales royalty dilution with upload counts is modelling the wrong line.</p><figure><img alt="Two cards titled Supply is not demand. Uploads: more than 50% of new music delivered on peak days, shown with a stack of teal tiles. Listening: 1% to 3% of all streams, shown with a single flat tile. Deezer figures reported by Music Business Worldwide, July 2026, different periods." src="https://cdn-images-1.medium.com/max/1024/1*HYaj3UTqsnqhf62x9ugN9w.png" /><figcaption>On Deezer’s figures, AI tracks are a large share of deliveries and a small share of streams. Image created by the author with AI.</figcaption></figure><p>The UMG figure has its own problem. The call summary published on Yahoo Finance, written by GuruFocus, attributes the ten thousand to nearly one hundred thousand rise to chief digital officer Michael Nash. ¹ It does not say whose uploads, over what period, or on which platforms. Deezer’s 90,000 is one service. The two figures may describe different things, and I would not stack them.</p><p>The vintages differ too. Deezer’s fraud figure, which I come to below, is explicitly for 2025, and the stream share was not reported for the same June window as the uploads. Dividing one into the other as if they described a single month would produce a precise looking number with no meaning.</p><h4>How Streaming Royalties Get Diluted</h4><p>For a finance reader, the mechanism fits in one paragraph. A subscription service takes in revenue for a period, sets aside the portion owed to rights holders, and divides that pool by the total payable streams. Each rights holder receives its share of streams multiplied by the pool. The pool is set by subscribers and price, not by how many tracks sit in the catalog.</p><p>That makes dilution simple arithmetic. If AI tracks take 2% of payable streams, every other rights holder’s share falls by roughly 2%, all else equal. This is the thing that does not change: every new claimant on a fixed pool is paid by everyone already in it.</p><figure><img alt="Four step flow titled How streaming royalties get diluted: subscription revenue, royalty pool for rights holders, divided by payable streams, each holder gets its share of streams. Below: Every new claimant is paid by everyone already in it." src="https://cdn-images-1.medium.com/max/1024/1*2wQR1u5_BqyFtJbmjdLSgg.png" /><figcaption>Under a pro rata pool, every new claimant is paid by everyone already in it. Image created by the author with AI.</figcaption></figure><p>UMG says it has negotiated against exactly this. According to the same call summary, Nash outlined four key contractual protections secured with streaming services, including provisions so that fully AI generated music “doesn’t dilute artist royalties.” ¹ The summary also records him saying the steps taken so far are not sufficient. No rates, mechanics or counterparties are public.</p><p>Without the terms, I can only list the shapes such a clause could take, and none of these is something UMG has described. AI streams could be removed from the denominator used to compute UMG’s share. There could be a floor under the per stream value paid to UMG’s artists. Or AI tracks could be paid from a separate pool. In each shape, unless the platform funds the difference itself, the AI share is still taken out of somebody’s payout. The clause moves the cost. My read is that it does not remove it.</p><h4>A Back of Envelope Exposure</h4><p>What follows is my own illustration, not a UMG or platform number, and it rests on one strong assumption: a single pro rata pool, with any AI share taken evenly from every rights holder in proportion to their streams. Real contracts are struck service by service and market by market, often with minimums, so treat this as a sizing exercise.</p><p>Music Business Worldwide reported on 29 October 2026 that UMG generated revenues of EUR 3.394 billion (USD 3.90 billion) in the third quarter, and that subscription streaming revenues grew 19.2% year on year at constant currency to EUR 1.422 billion. ² The Yahoo Finance summary adds that growth excluding the Downtown acquisition was 9.5%, so the base itself mixes an acquired business with the organic one. ¹</p><p>On that base, my back of envelope runs like this:</p><p><strong>Each 1% of payable streams</strong> that moves to unprotected AI tracks is about EUR 14.2 million of UMG subscription revenue a quarter (1% of EUR 1.422 billion).</p><p><strong>At 3%,</strong> the top of Deezer’s range, it is about EUR 42.7 million a quarter.</p><p><strong>Annualised</strong> on a flat quarter, that is roughly EUR 57 million and EUR 171 million.</p><figure><img alt="Illustrative sketch on a base of UMG subscription streaming revenue of EUR 1.422 billion a quarter: 1% of payable streams is EUR 14.2M a quarter and 3% is EUR 42.7M a quarter. Each figure counts up from zero, gets underlined, and a stack of brass coins grows beneath it, one card after the other. Author calculation assuming a single pro rata pool." src="https://cdn-images-1.medium.com/max/1024/1*AMtI7Bqf_0m_g4Sk3WSOcA.gif" /><figcaption>My back of envelope: what 1% and 3% of payable streams would mean for UMG’s subscription line each quarter. Animation created by the author with AI.</figcaption></figure><p>Against a EUR 1.4 billion quarterly line, the 1% case is small. My guess is that the 3% case is the kind of number a label negotiates a clause to avoid, which hints at what the people across the table thought was plausible. The implication is that the share matters far more than the count: a tenfold rise in uploads moves this model only if it moves the payable share.</p><h4>The Fraud Overlay</h4><p>Part of the AI share is not listening at all. In the same July report, Deezer said up to 85% of the streams generated by fully AI generated tracks were fraudulent in 2025. ³ Streaming fraud, in the usual sense of the term, means plays generated to collect payouts rather than people listening.</p><p>Here the arithmetic gets interesting, and again this is my illustration. If every fraudulent stream were caught and excluded from payment, the payable AI share would be 15% of the 1% to 3% range, roughly 0.15% to 0.45% of streams. That “if” carries the whole question. I wrote about the same missing denominator in <a href="https://medium.com/@syed_ali/what-spotifys-50-million-removed-ai-tracks-actually-measure-de05acb433cc">Spotify’s 50 million removed AI tracks</a> last month, and the pattern repeats here: a count is published, the base is not.</p><p>Three gaps sit between the numbers we have and the one we need.</p><p><strong>Gap one: share of streams versus share of payable streams.</strong> Deezer’s 1% to 3% describes streams. Royalties follow payable streams, after filtering. Nobody has published the second figure, and the difference between them could be most of the first.</p><p><strong>Gap two: what happens to a caught stream.</strong> A fraud rate tells you how many plays were flagged. It does not tell you whether their money went back into the pool, was withheld, or was paid and later clawed back. Each outcome leaves the denominator in a different place.</p><p><strong>Gap three: one platform’s data.</strong> The 1% to 3% and the 85% are Deezer’s alone. Other services set their own rules. TechCrunch reported on 14 October 2026 that YouTube Music’s new AI labels come with the line “AI involvement alone doesn’t make a track ineligible for monetization.” ⁴ A labelled track can still be a paid track, so the payable share on one service tells you little about another.</p><h4>Metrics to Ask For</h4><p>If you want to know whether AI music is moving streaming royalties, these four numbers would settle it. Each is a disclosure only the platforms and labels can make.</p><p>1. <strong>AI share of streams,</strong> by platform, for the same period as the upload figures.</p><p>2. <strong>AI share of payable streams,</strong> after fraud filtering. This is the denominator.</p><p>3. <strong>Detected fraud rate on AI tracks,</strong> and what happens to the money attached to those streams.</p><p>4. <strong>AI licensing revenue,</strong> reported as its own line rather than folded into subscription growth.</p><figure><img alt="Takeaway list titled Four numbers to ask for: 1 AI share of streams, 2 AI share of payable streams, 3 detected fraud rate on AI tracks, 4 AI licensing revenue on its own line, beside a brass tray with one thin teal wedge. Footer: Ask for the share, not the count." src="https://cdn-images-1.medium.com/max/1024/1*1XSgDDrf9i9n-AnLiPyMnQ.png" /><figcaption>The four disclosures that would settle whether AI music is moving royalties. Image created by the author with AI.</figcaption></figure><p>With those missing, the honest way to frame the next year is two scenarios and one signal.</p><p><strong>Scenario one: the share stays small.</strong> The payable AI share stays under 1%, fraud filters keep most automated plays out, and the protections hold. On my sketch the exposure is EUR 14 million a quarter or less for a company of UMG’s size, absorbed inside normal growth. In this world, the upload surge is a moderation cost story, not a royalty story.</p><p><strong>Scenario two: the share climbs.</strong> Uploads keep rising, more services follow policies that keep labelled AI tracks payable, and the payable share moves toward 3%. On the same sketch that is around EUR 43 million a quarter, and the terms of every anti dilution clause become material to anyone modelling the sector.</p><p><strong>The signal: a separate disclosure.</strong> Watch whether the next round of label earnings reports, starting with UMG’s full year results, shows AI share of payable streams or AI licensing revenue as its own line. My read is that a company with a reassuring number has a reason to publish it. Silence does not prove scenario two, but it should keep an analyst from assuming scenario one.</p><h4>Where This Thesis Breaks</h4><p>You deserve the version of this argument where I am wrong. I would not trust it otherwise.</p><p><strong>Counter one: “1% to 3% is noise.”</strong> This is the strongest counter, and the one I take most seriously. On a EUR 1.422 billion quarter, 1% is a rounding error next to 19.2% growth. My response is that someone negotiated a protection, so someone priced the risk as worth a clause. The share is also Deezer’s own figure from a single service, and uploads are climbing fast, from 75,000 a day in April to nearly 90,000 in June on Deezer’s count, while nobody has published how the stream share moved over the same months. Run both scenarios and decide which one you believe.</p><p><strong>Counter two: “Licensing offsets dilution.”</strong> Warner Music Group’s CEO Robert Kyncl said on the company’s 5 August 2026 call that he expects AI licensing deals “to contribute materially to our subscription streaming revenue growth starting in fiscal ‘27,” as reported by The AI Musicpreneur. ⁵ That may prove true. No rate or split has been published, so the offset is unproven until a disclosure shows it. And licensing income flows to the rights holders who signed the deals, which are not necessarily the ones whose share was diluted.</p><p><strong>Counter three: “Most AI uploads are fraud and get filtered anyway.”</strong> Partly right, on Deezer’s 85% figure. But filtering has a cost, the money attached to caught streams has to go somewhere, and policies differ, as YouTube’s labels show. The payable share depends on the filter, which is exactly why the filter’s results need to be disclosed.</p><p>I am not claiming AI tracks are draining label revenue today. I am not claiming UMG’s protections are weak; their terms are not public, so I cannot judge them. What I am claiming is narrower: the number that would settle the question has not been published, and the upload count is being read as if it were that number.</p><p>Pick whichever counter is most persuasive to you. The practical implication is the same: ask for the share, not the count.</p><h4>What to Do Before the Next Earnings Call</h4><p>The next ninety days cover the start of the next reporting cycle, which is the window where these questions can be asked on the record.</p><p><strong>If you model music companies,</strong> write the four metrics above as questions before the next UMG or Warner call, and rebuild any dilution assumption on payable share rather than uploads. Run the 1% and 3% cases on the company’s own subscription line so you know what a disclosure would change.</p><p><strong>If you run a label or a distributor,</strong> ask each streaming service in writing whether flagged AI streams are excluded from the payable denominator, and whether a protection negotiated by another rights holder changes your share of the pool. Keep the answers.</p><p><strong>If you build AI music products,</strong> assume your next distribution partner will ask what share of your users’ plays survive a fraud filter. Start tracking that number now, before someone else estimates it for you.</p><h4>Who Loses If I Am Right</h4><p>If anti dilution clauses move the AI share rather than remove it, the rights holders without such clauses carry more of it. That means smaller labels and independent artists, who often cannot negotiate terms with a streaming service directly. Their payouts could fall by a share nobody has disclosed, through a mechanism nobody has described.</p><p>Analysts lose too, more quietly. Anyone who sized the risk off upload counts has a model that overstates exposure in the good scenario and has no way to detect the bad one. And the services carry a slower cost: a pool that cannot show who it pays is asking for trust it will eventually have to earn with numbers.</p><h4>Closing</h4><p>Every royalty pool is a fraction. This year the industry was handed a dramatic numerator, a roughly tenfold rise in daily AI uploads, and no denominator to divide it by. On the figures that do exist, the payable share looks small. Small is not the same as measured.</p><p>The fix needs no new technology. A share of streams, a share of payable streams, a fraud rate with an outcome, and a licensing line. Until those appear, the upload count is a story about supply, and royalties have never been paid on supply.</p><p><em>An upload count tells you how much arrived. Only the payable share tells you who paid for it.</em></p><p><em>Syed Ali</em></p><p><em>Founder and CEO, Echonos</em></p><p><em>Syed Ali is cofounder of </em><a href="https://echonos.ai/"><em>Echonos</em></a><em>, an AI music video pipeline for indie artists. He was previously COO at Tabler App (1M+ users, $50K+ MRR, successful exit) and a data science consultant at Deloitte, Booz Allen Hamilton, and Accenture. Find him on LinkedIn: </em><a href="https://www.linkedin.com/in/syedali-/"><em>https://www.linkedin.com/in/syedali-/</em></a></p><h4>Sources</h4><p>1. Yahoo Finance (GuruFocus call highlights), 29 October 2026, UMG third quarter 2026 earnings call: chief digital officer Michael Nash “outlined four key contractual protections secured with DSPs,” including provisions ensuring fully AI generated music “doesn’t dilute artist royalties”; “he stressed these steps are not sufficient, noting AI uploads have surged from 10,000 to nearly 100,000 per day.” Subscription streaming revenue grew 19.2%, or 9.5% excluding Downtown. <a href="https://finance.yahoo.com/markets/stocks/articles/universal-music-group-nv-umgnf-051143970.html">https://finance.yahoo.com/markets/stocks/articles/universal-music-group-nv-umgnf-051143970.html</a></p><p>2. Music Business Worldwide, 29 October 2026: UMG “generated revenues of EUR €3.394 billion (USD $3.90B) across all of its divisions”; “subscription streaming revenues grew 19.2% YoY at constant currency to reach €1.422 billion ($1.64bn).” <a href="https://www.musicbusinessworldwide.com/universal-music-group-generated-3-90b-in-q3-up-11-7-yoy-driven-by-mrs-green-apple-olivia-rodrigo-stray-kids-ariana-grande-and-the-beatles/">https://www.musicbusinessworldwide.com/universal-music-group-generated-3-90b-in-q3-up-11-7-yoy-driven-by-mrs-green-apple-olivia-rodrigo-stray-kids-ariana-grande-and-the-beatles/</a></p><p>3. Music Business Worldwide, 21 July 2026, reporting Deezer’s figures: nearly 90,000 fully AI generated tracks received every day in June, up from 75,000 a day, or 44% of daily deliveries, in April; “On peak days, that content made up more than 50% of all new music delivered to the service”; fully AI music at between 1% and 3% of all streams on the platform; up to 85% of the streams generated by fully AI generated tracks fraudulent in 2025. <a href="https://www.musicbusinessworldwide.com/90000-ai-tracks-flood-deezer-daily-passing-half-of-new-music-uploads-for-the-first-time/">https://www.musicbusinessworldwide.com/90000-ai-tracks-flood-deezer-daily-passing-half-of-new-music-uploads-for-the-first-time/</a></p><p>4. TechCrunch, 14 October 2026, on YouTube Music’s AI labels: “AI involvement alone doesn’t make a track ineligible for monetization.” <a href="https://techcrunch.com/2026/10/14/youtube-music-launches-new-labels-as-ai-generated-music-floods-streaming-platforms/">https://techcrunch.com/2026/10/14/youtube-music-launches-new-labels-as-ai-generated-music-floods-streaming-platforms/</a></p><p>5. The AI Musicpreneur, 7 August 2026, on Warner Music Group’s fiscal third quarter call of 5 August 2026, CEO Robert Kyncl: AI “licensing deals to contribute materially to our subscription streaming revenue growth starting in fiscal ‘27.” <a href="https://www.aimusicpreneur.com/ai-music-news/warner-music-ai-licensing-revenue-fiscal-2027/">https://www.aimusicpreneur.com/ai-music-news/warner-music-ai-licensing-revenue-fiscal-2027/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1560ad37a923" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A/B Testing With 500 Users: What You Can Actually Detect (and What to Decide on Judgment)]]></title>
            <link>https://medium.com/@syed_ali/a-b-testing-with-500-users-what-you-can-actually-detect-and-what-to-decide-on-judgment-bc9705a2edba?source=rss-08b3001202b0------2</link>
            <guid isPermaLink="false">https://medium.com/p/bc9705a2edba</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[ab-testing]]></category>
            <category><![CDATA[product-management]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Syed Ali]]></dc:creator>
            <pubDate>Sun, 01 Nov 2026 11:26:11 GMT</pubDate>
            <atom:updated>2026-11-01T11:26:11.568Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Cover: headline Every test has a floor, over a frosted green glass sieve that lets many small clay pebbles fall through while two large rust stones stay caught." src="https://cdn-images-1.medium.com/max/1024/1*ecUQ_HADPcYyKSmcqmXZdQ.png" /><figcaption>Every test has a floor: below it, small lifts fall straight through. Image created by the author with AI.</figcaption></figure><p><em>AB testing sample size for startups with hundreds of users decides what a test can see before it runs. At 500 users per arm, modest lifts are invisible, and the wins that do clear the bar are usually inflated.</em></p><p>With 500 users in each arm and a 10 percent baseline conversion rate, an A/B test can only reliably detect a lift of about 60 percent. That is not a typo, and it is the number most startup experiment advice leaves out. The guides were written for sites with a million visitors, and most AI products I see ship weekly to cohorts of a few hundred. Before you read a result from a test that small, it helps to know what it was ever able to see.</p><p>That is the sample size problem. Every test has a floor, and below it the dashboard still prints a winner.</p><p>Before Echonos I worked as a data science consultant at Deloitte, Booz Allen Hamilton and Accenture, and later ran operations as COO at Tabler. The habit those years left me with is unglamorous. Before I trust a result, I ask what the test could have detected if the idea had worked. When the honest answer is “only a miracle”, the result is a coin flip with a confidence interval drawn around it.</p><p>This piece is for founders, product managers and data people who test on hundreds or low thousands of users. With a million weekly visitors and a platform team, you can skip to the caveats. With small cohorts, read on, because the arithmetic decides more of your roadmap than you think.</p><p>It helps to separate three questions a test can answer, because small teams blur them.</p><p><strong>The first question is detection.</strong> Did the test have enough users to notice an effect of the size you care about? This is set before launch by the baseline, the sample and the lift.</p><p><strong>The second question is size.</strong> How big is the effect? A test can detect something and still report its size badly.</p><p><strong>The third question is sign.</strong> Is the effect even positive? At very low power a “significant” result can point the wrong way.</p><p>The common confusion is between the first two. Teams treat “it was significant” as proof that the measured lift is the real lift. At small samples those are different claims.</p><h4>The only number that matters: the lift 500 users can detect</h4><p>The short answer: with a 10 percent baseline, 80 percent power and a 5 percent significance level, you need about 3,841 users in each arm to detect a 20 percent relative lift. With 500 per arm, the smallest lift you can reliably detect is about 60 percent. Smaller lifts are mostly invisible.</p><p>That figure is the minimum detectable effect, and it is the one number worth computing before any test. I reran it for this piece with the standard two sided, two proportion z test. The formula, per arm:</p><p>n = (z(1 − α/2) × √(2 × p̄ × (1 − p̄)) + z(power) × √(p1 × (1 − p1) + p2 × (1 − p2)))² ÷ (p2 − p1)²</p><p>Here p1 is the baseline, p2 is the baseline times one plus the relative lift, and p̄ is their average. With α at 0.05 and power at 0.80, the two z values are 1.96 and 0.84.</p><p>The figures below work as a calculator for most startup cases: users needed in each arm, by baseline, for each relative lift in brackets.</p><p>2% baseline: 315,206 (5%), 80,682 (10%), 21,109 (20%), 9,798 (30%), 3,826 (50%)</p><p>5% baseline: 122,124 (5%), 31,234 (10%), 8,158 (20%), 3,780 (30%), 1,471 (50%)</p><p>10% baseline: 57,763 (5%), 14,751 (10%), 3,841 (20%), 1,774 (30%), 686 (50%)</p><p>20% baseline: 25,583 (5%), 6,510 (10%), 1,683 (20%), 772 (30%), 294 (50%)</p><p>30% baseline: 14,856 (5%), 3,763 (10%), 963 (20%), 437 (30%), 163 (50%)</p><p>Read it the other way and it gets uncomfortable. At 500 users per arm the detectable relative lift is about 92% on a 5 percent baseline, 60% on a 10 percent baseline and 38% on a 20 percent baseline. At 200 per arm it is about 160%, 100% and 61%. Even at 1,000 per arm it is about 62%, 41% and 26%.</p><figure><img alt="Three cards at 500 users per arm: the smallest detectable lift is 92% on a 5% baseline, 60% on a 10% baseline and 38% on a 20% baseline. Each number counts up from zero, gets underlined and a small clay block grows under it, one card after another." src="https://cdn-images-1.medium.com/max/1024/1*BuP5GUljdoUa6GlFLgplSA.gif" /><figcaption>At 500 users per arm, the smallest detectable lift is 92%, 60% and 38% at 5%, 10% and 20% baselines. Animation created by the author with AI.</figcaption></figure><p>The mechanism is simple. Noise shrinks with the square root of the sample, so halving the lift you want to detect roughly quadruples the users you need. Lower baselines are worse, because the signal is a small difference between two small numbers.</p><p>Published guidance agrees from bigger sites. Tomasz Tunguz wrote in 2016 that a 20 percent change on a 3 percent baseline at 90 percent confidence needs about 12,000 users per variation; my recompute gives about 11,000 to 15,000 depending on power. Georgi Georgiev at Analytics Toolkit wrote in 2019 that a standard setup “needs roughly 50,000 to 500,000 users.”</p><h4>The A/B test sample size formula, in code</h4><p>Short enough to paste into a notebook, and it uses only the Python standard library:</p><pre>from math import sqrt, ceil<br>from statistics import NormalDist<br><br>def users_per_arm(baseline, relative_lift, alpha=0.05, power=0.80):<br>    &quot;&quot;&quot;Two sided, two proportion z test. Returns users needed in EACH arm.&quot;&quot;&quot;<br>    z_a = NormalDist().inv_cdf(1 - alpha / 2)<br>    z_b = NormalDist().inv_cdf(power)<br>    p1 = baseline<br>    p2 = baseline * (1 + relative_lift)<br>    pbar = (p1 + p2) / 2<br>    top = z_a * sqrt(2 * pbar * (1 - pbar)) + z_b * sqrt(p1 * (1 - p1) + p2 * (1 - p2))<br>    return ceil(top ** 2 / (p2 - p1) ** 2)<br><br>def detectable_lift(baseline, users, alpha=0.05, power=0.80):<br>    &quot;&quot;&quot;Smallest relative lift this many users per arm can reliably detect.&quot;&quot;&quot;<br>    lo, hi = 0.0001, 1 / baseline - 1.0001<br>    for _ in range(100):<br>        mid = (lo + hi) / 2<br>        if users_per_arm(baseline, mid, alpha, power) &gt; users:<br>            lo = mid<br>        else:<br>            hi = mid<br>    return hi<br><br>if __name__ == &quot;__main__&quot;:<br>    print(users_per_arm(0.10, 0.20))          # 3841<br>    for n in (200, 500, 1000):<br>        print(n, f&quot;{detectable_lift(0.10, n):.0%}&quot;)  # 100%, 59%, 41%</pre><p>Run detectable_lift with your real cohort before you design the test. If it prints a number bigger than any lift your idea could plausibly produce, the test cannot answer your question.</p><h4>Why a small sample win is usually 2 to 4 times too big</h4><p>A test that is underpowered does not just miss real effects. When it does flag one, it overstates it. Andrew Gelman and John Carlin called this the Type M, or magnitude, error in a 2014 paper, and wrote that “problems with the exaggeration ratio start to arise when power is less than 0.5.” In their worked example with power of 0.06, the expected exaggeration factor is 9.7 and the chance of getting the sign wrong is 24%.</p><p>The mechanism is a filter. At low power, only runs where noise pushed the result upward clear the bar. Keep only those and you have kept the lucky ones.</p><p>I simulated it to see the size on startup numbers. The setup: a 10 percent baseline, a real improvement of a fixed size, 200,000 simulated tests per row, and only the significant wins kept, the way a team that ships winners would see them.</p><p>+1 point at 500 users per arm: power 7.5%, reported lift 4.7 times the true lift</p><p>+2 points at 300 users per arm: power 12%, reported lift 3.1 times the true lift</p><p>+2 points at 500 users per arm: power 17%, reported lift 2.5 times the true lift</p><p>+2 points at 1,000 users per arm: power 30%, reported lift 1.8 times the true lift</p><p>These are simulation results under those assumptions, not measurements from any company. A +2 point win on a 500 user test typically shows up as about +5 points. Build next quarter’s forecast on +5 and the miss is baked in.</p><figure><img alt="Table from a simulation at a 10% baseline keeping only significant wins: a +1 point lift at 500 users per arm is reported 4.7x too big; +2 points at 300, 500 and 1,000 users run 3.1x, 2.5x and 1.8x." src="https://cdn-images-1.medium.com/max/1024/1*lc1QYTn0qcaWxeKLFd2Hrw.png" /><figcaption>In the simulation, significant wins from small tests report 1.8 to 4.7 times the true lift. Image created by the author with AI.</figcaption></figure><p>Now add how often ideas work at all. Ron Kohavi and Stefan Thomke wrote in Harvard Business Review in 2017 that “at Google and Bing, only about 10% to 20% of experiments generate positive results,” and that at Microsoft about a third are positive, a third neutral and a third negative. My read: startup ideas are unlikely to be better than Google’s, so a 10 to 20 percent hit rate is a fair working assumption. It is an assumption, not a measured startup win rate.</p><p>Put that together and you get the share of significant results that are real, the positive predictive value. If 10 percent of your ideas truly work and your test has 30 percent power, about 40 percent of your significant results are real. At a 20 percent hit rate and 30 percent power it is about 60 percent. At a 10 percent hit rate and 10 percent power, which is where a 300 user test often sits, it falls to about 18 percent. Most of the “wins” in that last case are noise.</p><h4>Peeking: 5 percent becomes 19 percent in ten looks</h4><p>The second failure is not about size at all. It is about checking the dashboard and stopping when it turns green.</p><p>I ran an A/A simulation, two identical arms with no real difference, on a 10 percent baseline with 2,000 users per arm and 20,000 runs. Each run stopped at the first look where the z statistic passed 1.96. The false positive rate came out at 5.1% with one look, 8.2% with two, 14.0% with five, 19.3% with ten, 25.0% with twenty and 37.9% with a hundred. A test designed for a 5 percent error rate ends up at nearly one in five if you check it ten times.</p><figure><img alt="Timeline from an A/A simulation with no real difference and 2,000 users per arm: the false positive rate is 5.1% with 1 look, 8.2% with 2, 14.0% with 5, 19.3% with 10 and 25.0% with 20." src="https://cdn-images-1.medium.com/max/1024/1*lxybgyDUl9mInftAnz3SVQ.png" /><figcaption>With no real difference, checking ten times turns a 5% error rate into 19.3%. Image created by the author with AI.</figcaption></figure><p>Evan Miller showed in 2010 that stopping at the first significant result in his example gives a false positive rate of 26.1%, “more than five times what you probably thought the significance level was.” Ramesh Johari, Leo Pekelis and David Walsh wrote in their paper on always valid inference that with continuous monitoring, “Type I error can easily increase fivefold.” On a nominal 5 percent, fivefold is about 25 percent.</p><p>Small teams are more exposed, because a small test runs for weeks and someone looks every morning. Multiple metrics make it worse. Track five metrics and the chance that at least one crosses the line by luck is about 22.6%; with ten it is about 40.1%; with twenty, about 64.2%. That is just one minus 0.95 to the power of the number of metrics, assuming they are independent.</p><p>The implication is procedural. Pick the sample size and one primary metric, and agree in writing that nobody acts before the sample is reached.</p><h4>Bayesian and sequential methods: what they fix and what they do not</h4><p>The usual answer to all of this is “use Bayesian testing, it handles small samples and lets you peek.” Two recent pieces push back.</p><p>Spotify Engineering wrote in September 2026 that “Bayesian and frequentist frameworks are much closer than the debate suggests.” The same post notes that most teams run the platform default, usually a flat prior, and “a flat prior provides no shrinkage.” A posterior with no shrinkage reports the same inflated winner as the frequentist estimate, so the Type M problem stays where it was.</p><p>Alex Molas tested the peeking claim directly in October 2025. In his A/A simulation with a fixed posterior threshold, peeking after every 100 observations pushed the false positive rate to 80%. A posterior checked often has the same problem as a p value checked often.</p><p>Sequential methods do fix peeking. The always valid approach from Johari and colleagues keeps the error rate no matter how often you look. My understanding of the trade: you pay with a wider interval at any given sample, so early stopping becomes honest while the sample stays small.</p><p>The method that does add real sensitivity is variance reduction. Alex Deng, Ya Xu, Ron Kohavi and Toby Walker described CUPED in 2013 and reported that on Bing it can “reduce variance by about 50%,” effectively the same power with half the users. The catch is in the method: it uses data from the pre experiment period on the same users. A new product, or a feature shown mostly to new signups, often has no such history to draw on.</p><p>So the scorecard is short. Sequential tests fix peeking. Informative priors and CUPED buy some sensitivity if you have real prior data. None of them turns 500 users into 5,000.</p><h4>Where this thesis breaks: the honest caveats</h4><p>Three objections deserve a real answer, and one of them is partly right.</p><p><strong>“Bayesian lets small teams peek freely.”</strong> It does not, at least not with the defaults most teams run. Molas measured about 80 percent false positives from peeking every 100 observations under a fixed threshold, and Spotify Engineering’s point about flat priors means the estimate is no less inflated. An informative prior built from real past tests is different, and if you have one, use it.</p><p><strong>“Any directional data beats none, so ship the winner of a 300 user test.”</strong> This is the one I take most seriously, because directional data does have value when the alternative is pure opinion. The problem is what the direction is worth. At 12 to 17 percent power, my simulation shows the reported lift running about 2.5 to 3 times too high, and a small share of significant results point the wrong way entirely. Directional data is fine as one input. It is a poor reason to stop thinking.</p><p><strong>“Startups that run A/B tests grow faster, so test everything.”</strong> Rembrand Koning, Sharique Hasan and Aaron Chatterji studied 35,262 startups and found that “just under 18% of firms use an A/B testing tool,” and that adopters “introduce new products at a 9% to 18% higher rate.” The study is observational, its performance measure is page views rather than revenue, and it covers startups with enough web traffic to test on. It does not show that a 300 user test helps a company that cannot reach the sample.</p><p>I am not claiming small tests are useless. A test can still catch a broken checkout, a crash or a huge effect, and those are worth catching. I am also not claiming judgment beats data in general. The claim is narrower: when the sample cannot see the effect you care about, the test result should not be what decides.</p><p>The caveats on my own argument are specific. If your idea plausibly moves the metric by 50 percent or more, a 500 user test can see it, so run it. With a high baseline, say 30 percent activation, the figures above are far kinder. And when the action is risky or hard to reverse, a small test still works as a harm check, as long as you read “we did not detect harm” with the caution GrowthBook quotes Kohavi on: a result that is not statistically significant “does not prove zero effect or safety.”</p><h4>A kill rule: when to stop testing and ship on judgment</h4><p>What follows is my recommendation, not a published standard. It borrows Kohavi’s advice, as summarised by GrowthBook in August 2026, to define a decision rule before you test:</p><ul><li><strong>The lift that would change your decision.</strong> Not the lift you hope for. The smallest one that would make you ship.</li><li><strong>Your real cohort.</strong> How many users per arm you can get in four weeks, at the traffic you actually have.</li><li><strong>The detectable lift for that cohort.</strong> Run detectable_lift. If it is above the lift in the first line, stop here.</li><li><strong>One primary metric and a fixed end date.</strong> No peeking before the end, no swapping metrics after.</li><li><strong>If you stop, decide on judgment and say so.</strong> Write down the reasoning, the qualitative evidence and the metric you will watch after launch.</li><li><strong>Shrink any small sample win in your forecasts.</strong> Divide a significant lift from an underpowered test by two or three before it goes into a plan.</li><li><strong>Keep the record.</strong> Log the prediction and the outcome, so over a year you learn your own hit rate instead of borrowing Google’s.</li></ul><p>The rule kills a lot of tests, and that is the point. A test that cannot detect anything costs weeks and then hands you a number you will be tempted to believe. Shipping on judgment, written down and checked after launch, is at least honest.</p><figure><img alt="Decision map titled Decide before you test: the lift that would change your decision leads to the detectable lift for your real cohort; if detectable is smaller, run the test with one metric and a fixed end date; if bigger, ship on judgment and write it down." src="https://cdn-images-1.medium.com/max/1024/1*PvyORNni5OhwhbpR-FVZDQ.png" /><figcaption>The kill rule: compare the lift that matters with the lift your cohort can detect before you test. Image created by the author with AI.</figcaption></figure><p>Once you can detect a lift, the next question is which metric deserves the test. I wrote about that in <a href="https://medium.com/@syed_ali/your-north-star-metric-is-probably-the-wrong-single-number-c05c69416f8d">Your North Star Metric Is Probably the Wrong Single Number</a>, and the same logic applies: pick the metric that moves fast enough for your sample to see.</p><h4>FAQ: AB testing sample size</h4><p><strong>How to calculate sample size for ab test?</strong></p><p>Pick your baseline conversion rate, the smallest relative lift that would change your decision, a significance level (usually 5 percent) and power (usually 80 percent). Plug them into the formula or the code above. The result is users per arm.</p><p><strong>How to calculate ab test sample size?</strong></p><p>The same way, and the input people skip is the lift. Use the smallest lift worth acting on, not the one you expect. A 10 percent baseline needs about 3,841 users per arm for a 20 percent lift and about 14,751 for a 10 percent lift.</p><p><strong>How to determine sample size for ab test?</strong></p><p>Start from the users you can actually get. Compute the detectable lift for that cohort and compare it with the lift your idea could plausibly produce. If the detectable lift is bigger, the test should not run.</p><p><strong>How to decide sample size in ab testing?</strong></p><p>Decide it before the test starts and never change it after you look. Stopping when the result looks good inflates the false positive rate, to about 19 percent after ten looks in my simulation.</p><h4>The takeaway</h4><p>With 500 users in each arm, an A/B test can see a 60 percent lift and very little else. The fix is not a fancier statistics engine. It is knowing what your test can see before you run it, and deciding the rest on judgment you write down.</p><p>Before next Friday, run detectable_lift on the test you are about to launch. If the number is bigger than your idea, ship it or kill it, and skip the test.</p><p><em>Syed Ali</em></p><p><em>Founder and CEO, Echonos</em></p><p><em>Syed Ali is cofounder of </em><a href="https://echonos.ai/"><em>Echonos</em></a><em>, an AI music video pipeline for indie artists. He was previously COO at Tabler App (1M+ users, $50K+ MRR, successful exit) and a data science consultant at Deloitte, Booz Allen Hamilton, and Accenture. Find him on LinkedIn: </em><a href="https://www.linkedin.com/in/syedali-/"><em>https://www.linkedin.com/in/syedali-/</em></a></p><h4>Sources</h4><p>1. Ramesh Johari, Leo Pekelis and David Walsh, “Always Valid Inference: Continuous Monitoring of A/B Tests”, arXiv 1512.04922 (v3, 2019): with continuous monitoring, “Type I error can easily increase fivefold.” The roughly 25 percent figure in this article is arithmetic on a nominal 5 percent. <a href="https://arxiv.org/pdf/1512.04922">https://arxiv.org/pdf/1512.04922</a></p><p>2. Evan Miller, “How Not To Run an A/B Test”, 18 April 2010: stopping at the first significant result gives a 26.1% false positive rate in his example, “more than five times what you probably thought the significance level was.” <a href="https://www.evanmiller.org/how-not-to-run-an-ab-test.html">https://www.evanmiller.org/how-not-to-run-an-ab-test.html</a></p><p>3. Ron Kohavi and Stefan Thomke, “The Surprising Power of Online Experiments”, Harvard Business Review, September to October 2017: “At Google and Bing, only about 10% to 20% of experiments generate positive results”; at Microsoft one third positive, one third neutral, one third negative. <a href="https://web-docs.stern.nyu.edu/executive/The%20Surprising%20Power%20of%20Online%20Experiments.pdf">https://web-docs.stern.nyu.edu/executive/The%20Surprising%20Power%20of%20Online%20Experiments.pdf</a></p><p>4. Andrew Gelman and John Carlin, “Beyond Power Calculations: Assessing Type S (Sign) and Type M (Magnitude) Errors”, Perspectives on Psychological Science, 2014: “Problems with the exaggeration ratio start to arise when power is less than 0.5”; at power 0.06, Type S error probability 24% and expected exaggeration factor 9.7. <a href="https://sites.stat.columbia.edu/gelman/research/published/PPS551642_REV2.pdf">https://sites.stat.columbia.edu/gelman/research/published/PPS551642_REV2.pdf</a></p><p>5. Alex Deng, Ya Xu, Ron Kohavi and Toby Walker, “Improving the Sensitivity of Online Controlled Experiments by Utilizing Pre-Experiment Data” (CUPED), WSDM 2013: “we can reduce variance by about 50%” on Bing, using data from the pre experiment period. <a href="https://exp-platform.com/cuped/">https://exp-platform.com/cuped/</a></p><p>6. Spotify Engineering, “Why Spotify is not using Bayesian A/B testing”, September 2026: “Bayesian and frequentist frameworks are much closer than the debate suggests”; “A flat prior provides no shrinkage.” <a href="https://engineering.atspotify.com/2026/9/why-spotify-is-not-using-bayesian-a-b-testing/">https://engineering.atspotify.com/2026/9/why-spotify-is-not-using-bayesian-a-b-testing/</a></p><p>7. Alex Molas, “Bayesian A/B testing is not immune to peeking”, 30 October 2025: peeking after every 100 observations, “the false positive rate increases to 80%.” <a href="https://www.alexmolas.com/2025/10/30/bayesian-ab-test-peeking.html">https://www.alexmolas.com/2025/10/30/bayesian-ab-test-peeking.html</a></p><p>8. GrowthBook, “When not to run a test: product leader takeaways”, August 2026: Ronny Kohavi, “Do not test without a decision rule”; “A result that is not statistically significant does not prove zero effect or safety.” <a href="https://www.growthbook.io/insights/when-not-to-run-test-product-leader-takeaways">https://www.growthbook.io/insights/when-not-to-run-test-product-leader-takeaways</a></p><p>9. Rembrand Koning, Sharique Hasan and Aaron Chatterji, “Experimentation and Startup Performance: Evidence from A/B Testing”, HBS working paper, 2021: 35,262 startups; “just under 18% of firms use an A/B testing tool”; adopters “introduce new products at a 9% to 18% higher rate”; observational, performance measured in page views. <a href="https://www.hbs.edu/ris/Publication%20Files/AB_Testing_R_R_08b97538-ed3f-413e-bc38-c239b175d868.pdf">https://www.hbs.edu/ris/Publication%20Files/AB_Testing_R_R_08b97538-ed3f-413e-bc38-c239b175d868.pdf</a></p><p>10. Georgi Georgiev, Analytics Toolkit blog, 4 March 2019: a standard setup with a 2 to 5 percent baseline and 5 to 10 percent minimum detectable effect “needs roughly 50,000 to 500,000 users.” <a href="https://blog.analytics-toolkit.com/?p=1071">https://blog.analytics-toolkit.com/?p=1071</a></p><p>11. Tomasz Tunguz, “Is A/B Testing A Good Idea for SaaS Startups?”, 16 May 2016: a 20 percent change on a 3 percent baseline at 90 percent confidence needs about 12,000 users per variation. <a href="https://www.tomtunguz.com/ab-testing-saas/">https://www.tomtunguz.com/ab-testing-saas/</a></p><p>12. Author’s calculations and simulations, run 1 November 2026 with Python (scipy and numpy): sample size table and minimum detectable lifts from the two sided two proportion z test (alpha 0.05, power 0.80); A/A peeking simulation (10% baseline, 2,000 users per arm, 20,000 runs); winner’s curse simulation (10% baseline, 200,000 runs per row, significant wins only); positive predictive value = w × power ÷ (w × power + (1 − w) × 0.05).</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bc9705a2edba" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The $159B Headline Is Not Your Seed Market]]></title>
            <link>https://medium.com/@syed_ali/the-159b-headline-is-not-your-seed-market-7f5352e60f04?source=rss-08b3001202b0------2</link>
            <guid isPermaLink="false">https://medium.com/p/7f5352e60f04</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[fundraising]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[venture-capital]]></category>
            <category><![CDATA[entrepreneurship]]></category>
            <dc:creator><![CDATA[Syed Ali]]></dc:creator>
            <pubDate>Sat, 31 Oct 2026 11:27:34 GMT</pubDate>
            <atom:updated>2026-10-31T11:27:34.488Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Editorial cover reading Plan with the denominator. A very tall column of damson stone discs stands beside a long low line of small pink and teal discs, with the subhead More dollars, fewer deals in the 2026 seed market." src="https://cdn-images-1.medium.com/max/1024/1*EgVNQIg9g_B5ewB3eBfCoA.png" /><figcaption>More dollars, fewer deals: the 2026 seed market sits far below the headline. Image created by the author with AI.</figcaption></figure><p><em>If you are raising a seed round in 2026, the record funding headline mostly describes 27 companies. The seed data underneath it shows more dollars reaching fewer founders, and a typical term sheet that looks far more ordinary than the headline.</em></p><p>The headline from Crunchbase’s third quarter report is 159 billion dollars of global venture funding, with AI taking 64 percent. If you are raising a seed round, that number is mostly about someone else. Crunchbase counted a record 27 rounds of one billion dollars or more in the quarter, and global seed funding was 13 billion. I lead capital markets and investor relations at my company, and I read this kind of report the way I used to read market sizing for clients: by asking who is actually in the denominator.</p><p>Those clients were at Deloitte, Booz Allen Hamilton and Accenture, and later I ran operations as COO at Tabler App. The habit both jobs left me with is the same. A total tells you how big the pool is. It says nothing about how many people drink from it, or how deep the water is where you happen to be standing.</p><p>So the claim in this piece is narrow. The 2026 funding headline is real, and it is built mostly at the very top of the market. Underneath it, seed dollars went up while the number of seed deals went down, and the median terms a typical founder sees look far more ordinary than the headline. My read is that the typical seed founder is raising in a longer tail market than the news suggests, and should measure their own process against that market rather than against the total.</p><p>This is written for founders and operators who are raising, or expect to raise, a seed or pre seed round in the next year. It is not financial advice, and it will not tell you what to raise or at what valuation. It is a way to read the data before the data reads you.</p><h4>What Crunchbase actually counted in Q3 2026</h4><p>A seed round is usually the first priced or convertible round a startup raises from professional investors, after friends, family or angels and before a Series A. It pays for proving that a product has real users. In 2026 the gap between the funding headline and the seed market is wide enough to matter for how you plan one.</p><p>Crunchbase News reported on October 5, 2026 that global venture funding “totaled $159 billion in Q3 2026 with close to 6,000 startups funded.” Startups in AI “across the stack raised $102 billion, or 64% of global venture capital.” Those are the numbers most coverage repeated, and then stopped.</p><p>The same report has two lines that change the reading. First, a record 27 companies raised rounds of one billion dollars or more, and “around a third of all global venture capital raised in Q3 went to those 27 companies.” Second, “global seed funding totaled $13 billion in Q3,” and of that, 2.6 billion dollars went to seed rounds of 100 million dollars and over. That is about a fifth of all seed dollars, in rounds most founders will never see.</p><figure><img alt="Animated diagram titled The total is not the seed pool. Three cards count up in turn to 159 billion dollars of global venture funding in Q3 2026, 13 billion dollars of global seed funding, and 2.6 billion dollars in seed rounds of 100 million dollars and over, while a small stone object grows under each number. A band reads: About a third of all global venture went to 27 companies." src="https://cdn-images-1.medium.com/max/1024/1*wsq-pmF_z-CJff6KooJ-sw.gif" /><figcaption>The Q3 2026 total, the seed pool inside it, and the largest seed rounds, per Crunchbase. Animation created by the author with AI.</figcaption></figure><p>The mechanism is simple once you write it down. A quarterly total is a sum, and sums are dominated by their largest terms. Roughly 6,000 startups shared the quarter, and 27 of them took about a third. Pull those out and the rest of the market is a much smaller pool with a much flatter shape.</p><p>The implication is the first thing I would change in how a founder talks about the market. When someone says AI is taking most of the money, ask which money. At the very top, yes. In your seed round, the relevant pool is the 13 billion, minus the rounds of 100 million and over, split across a few thousand companies.</p><h4>Dollars up, deals down: the seed round split</h4><p>The clearest version of this split is in Crunchbase’s first quarter report, published April 1, 2026. Seed funding totaled 12 billion dollars, “up 31% year over year, though the increase was entirely due to larger rounds, with deal counts falling 30% year over year to 3,800.”</p><p>Read that sentence twice. More seed money, and roughly a third fewer seed deals, in the same quarter. The total went up because the average cheque got bigger, not because more founders were funded. In the same quarter, Crunchbase counted four companies raising 65 percent of all global venture investment, which is why the top line looked like a record.</p><p>The US data for the full year says the same thing in a different way. Gené Teare at Crunchbase News reported on March 26, 2026 that US seed funding totaled 19.4 billion dollars in 2025, and that large deals drove the increase: “51% in seed deals $10 million and over, compared to a third in 2024.” Half of US seed dollars now sit in rounds of 10 million dollars or more.</p><figure><img alt="Diagram titled Dollars up, deals down. Seed dollars were up 31 percent to 12 billion dollars in Q1 2026 while seed deals were down 30 percent to 3,800, year over year, per Crunchbase. A band reads: 2025 US seed: 51 percent of dollars in rounds of 10 million dollars and over, versus a third in 2024." src="https://cdn-images-1.medium.com/max/1024/1*waPcwsHf6ciomgUwHWcO2g.png" /><figcaption>Seed dollars rose while seed deals fell in Q1 2026, per Crunchbase. Image created by the author with AI.</figcaption></figure><p>One caution belongs here. Crunchbase notes that its seed figures lag, and that seed totals tend to rise after a quarter closes as more rounds are reported. I did not find a Q3 2026 seed deal count in the primary report, so I am not claiming deal counts fell again in Q3. The first quarter is the clean comparison, and it points one way.</p><p>The implication for an average seed round is uncomfortable but useful. The “seed market” in the news is increasingly a market for a few very large seed rounds. If your round is the more ordinary size, your market got smaller in count even while the headline said it grew.</p><h4>Why the typical seed term looks ordinary</h4><p>If concentration were the whole story, you would expect the typical round to look transformed too. It does not, and that is the second half of the picture.</p><p>The pre seed data is the most direct. Carta’s State of Pre Seed report for Q2 2026, published August 13, 2026, counted 3.19 billion dollars raised by US startups on Carta across more than 11,500 pre seed instruments (SAFEs and convertible notes). A year earlier, the same quarter saw 3.22 billion dollars across 14,825 instruments. Dollars were roughly flat. Instruments fell by about a fifth. And “the average instrument size came out to $276,000,” a 27 percent increase on a year earlier. Carta says these Q2 figures will be revised upward as more data comes in, so treat the count as a floor.</p><p>The seed medians I could find come through a secondary source, so I flag them as such. Causo, summarising Carta’s six month benchmark published in July 2026, says “the median seed founder is raising about $4.1 million at a $24.3 million valuation and giving up 18%.” A second Causo guide, citing the PitchBook and NVCA Venture Monitor for Q1 2026, puts the median seed pre money valuation at 18.4 million dollars and the median seed deal at 3.0 million. These are different datasets and different periods, and I could not check either against the original report. They are indicative, not precise.</p><p>Even with that caveat, the shape is clear. A median of a few million dollars, giving up somewhere near a fifth of the company, is a familiar seed term. Nothing in these numbers looks like the 64 percent headline. The headline describes the top of the market, and the typical term sheet sits a long way below it.</p><p>The implication is the one I find most useful. Prepare for the median, not the mean. A founder who reads the record totals and expects better terms is reading the wrong row of the table.</p><figure><img alt="Diagram titled Prepare for the median, not the mean. The headline card shows 64 percent, the AI share of global venture in Q3 2026. The median seed founder card shows about 4.1 million dollars raised, a 24.3 million dollar valuation and 18 percent given up. A band reads: AI premium at the median seed round: under 4 percent." src="https://cdn-images-1.medium.com/max/1024/1*aCoB6GYNZYi_L-DfKRDBhw.png" /><figcaption>The headline share next to the median seed term, from secondary summaries. Image created by the author with AI.</figcaption></figure><h4>What investors say they want instead of a model premium</h4><p>The other thing the headline invites is the belief that an AI label is a pricing premium in itself. The medians do not support that at the typical seed round. In the same secondary summary of the PitchBook and NVCA data, the median seed pre money valuation was 18.7 million dollars for AI companies against 18.0 million for non AI companies. That is a gap of under 4 percent. Real, and small.</p><p>What investors say they want instead is worth reading in their own words, with the reminder that this is one investor’s view. CRV, in a July 2026 note on AI startup funding, wrote that “investors now require a structural advantage independent of model performance.” The same note warned that “the median time between seed close and Series A close has stretched to 20 months, so founders who plan their runway around a 12-month timeline to Series A are setting themselves up for a cash crunch.” Its own FAQ ties that to Carta data showing 616 days from seed to Series A in Q2 2025.</p><p>My read of the two statements together is that the bar moved from what you build on to what survives when the model underneath changes. A product whose advantage disappears when a better model ships is priced like it. A product with distribution, data or workflow that holds up regardless of which model it calls is what this investor says it is paying for.</p><p>The second line matters more for planning than for pitching. Twenty months from seed to Series A is a planning assumption, and it is far longer than the twelve month plan CRV warns against. That number changes how long the money has to last, which matters more than the valuation it was raised at.</p><h4>Where this thesis breaks</h4><p>You deserve the version of this argument where I am wrong. Here are three, stated as strongly as I can.</p><p><strong>Counter one: a rising tide lifts seed</strong></p><p>Seed dollars rose 31 percent year over year in Q1 2026, per Crunchbase. More money in seed is good for seed founders, whatever the shape.</p><p>My response is that the same sentence carries the counterweight. Deal counts fell 30 percent in that quarter, and 51 percent of 2025 US seed dollars were already in rounds of 10 million and over. Dollars went up while the number of founders funded went down. A rising tide that reaches fewer boats is a different tide.</p><p><strong>Counter two: seed is fine, and medians are rising</strong></p><p>Partly true. The average pre seed instrument hit a record 276,000 dollars, and the medians I found are healthy by any historical standard.</p><p>My response is that this is the point, not a rebuttal. The medians look ordinary. The headline looks extraordinary. Both can be true, and a founder who plans with the median and not the headline is doing exactly what the data supports. I am not claiming seed is broken. I am claiming the total is the wrong number to plan with.</p><p><strong>Counter three: AI founders get a premium, so AI is the only game</strong></p><p>The AI share of venture dollars is 64 percent, and the US data from PitchBook and NVCA shows how much of the money sits in the largest deals. Axios, reporting the Q2 2026 Venture Monitor on July 9, 2026, said “over 81% of the H1 2026 dollars” went to deals of 100 million dollars or more, out of 412.7 billion dollars raised. SiliconANGLE, reporting the same release, put that share at 87.5 percent. The two reports disagree, neither explains why, and I have not seen the primary release, so I say over 80 percent.</p><p>My response is that both figures describe the very largest rounds. At the median seed round, the AI premium in the secondary summary is under 4 percent. AI is where the dollars are. It is not, at the median, where the price is.</p><p>The counter I take most seriously is the second. If medians keep rising and seed deal counts recover, the longer tail I am describing could shrink back within a year, and this piece will read as a snapshot. That is why the next section is two scenarios and a signal, not a forecast. Pick whichever counter is most persuasive to you. The practical implication is the same: plan with the median, and measure your own process.</p><h4>Two scenarios and one signal to watch</h4><p>These are scenarios, not predictions, and neither comes with advice on what to raise.</p><p><strong>Scenario one: concentration persists.</strong> Mega rounds keep absorbing a third or more of each quarter, larger seed rounds keep taking a growing share of seed dollars, and seed deal counts keep falling. In this world the typical seed founder competes for a slowly shrinking number of cheques, and runway assumptions get longer.</p><p><strong>Scenario two: concentration fades.</strong> The largest late stage rounds slow, some of that capital looks for earlier entry points, and seed deal counts recover even if dollars stay flat. In this world the median term stays where it is and more founders get funded at it.</p><p>The one signal I would watch is the share of seed dollars in rounds of 10 million and over. Crunchbase put it at 51 percent for US seed in 2025, against about a third in 2024. If the next annual figure rises again, scenario one is landing. If it falls back toward a third, scenario two is. The next Crunchbase seed deal count is the faster, noisier version of the same signal.</p><h4>What to measure about your own raise</h4><p>None of this tells you what to raise. It does tell you what to measure, and those measures are more useful than any market total. I wrote about choosing the right metrics before revenue in <a href="https://medium.com/@syed_ali/your-north-star-metric-is-probably-the-wrong-single-number-c05c69416f8d">Your North Star Metric Is Probably the Wrong Single Number</a>, and the same discipline applies to a raise.</p><ol><li><strong>Your comparison set.</strong> Write down the rounds of your size and stage you can actually find. Compare against the median from those, not the quarterly total.</li><li><strong>Your runway assumption.</strong> Write the number of months you expect between closing seed and closing a Series A. If it is under twenty, write down why you expect to beat the median CRV cites.</li><li><strong>Your conversion funnel.</strong> Count first meetings, second meetings and term sheets, week by week. A raise is a funnel, and a funnel you do not count is a funnel you cannot fix.</li><li><strong>Your durability answer.</strong> Write one paragraph on what happens to your product if a better model ships next quarter. That is the question at least one investor says it is asking.</li></ol><p>Here is the test for this week. Put those four lines on one page before your next investor meeting. If you cannot fill in the first two, you are planning against the headline.</p><figure><img alt="Checklist titled Four lines on one page, for before your next investor meeting: 1 your comparison set, rounds of your size and stage; 2 your runway assumption, seed to Series A median 20 months; 3 your conversion funnel, meetings and term sheets week by week; 4 your durability answer, if a better model ships next quarter. Footer: Plan with the denominator. Not financial advice." src="https://cdn-images-1.medium.com/max/1024/1*cVWlPLhubUcf-p8GHJtS9Q.png" /><figcaption>Four lines to write down before your next investor meeting. Image created by the author with AI.</figcaption></figure><h4>FAQ</h4><p><strong>What is a seed round?</strong></p><p>A seed round is the first round a startup raises from professional investors, usually after angels and before a Series A, to prove the product has real users and a path to revenue. In 2026 the median seed round is a few million dollars, per secondary summaries of Carta and PitchBook data.</p><p><strong>What is seed round?</strong></p><p>It is the same thing: early outside capital, often raised on a SAFE, a convertible note or a priced equity round, used to reach the milestones a Series A investor will look for.</p><p><strong>How to raise a seed round?</strong></p><p>Start with a realistic comparison set and runway assumption, then run the raise as a counted funnel of meetings and term sheets. Prepare an answer on durability, since at least one investor says it now wants an advantage that holds independent of model performance.</p><p><strong>What is a pre seed round?</strong></p><p>A pre seed round comes before seed and is often raised on SAFEs or convertible notes. Carta reported an average pre seed instrument of 276,000 dollars in Q2 2026, a record for the past four years.</p><p><strong>What is the average seed round size?</strong></p><p>It depends on which average. Secondary summaries put the median seed round at about 3 to 4 million dollars in 2026. The mean is much higher, because a small number of very large seed rounds pulls it up.</p><h4>The takeaway</h4><p>The headline from Crunchbase’s third quarter is accurate, and it is mostly about 27 companies. Your seed round sits in a different market: more dollars, fewer deals, and a median term that looks ordinary next to the headline.</p><p>Read the total for the news. Plan with the denominator.</p><p><em>Syed Ali</em></p><p><em>Founder and CEO, Echonos</em></p><p><em>Syed Ali is cofounder of </em><a href="https://echonos.ai/"><em>Echonos</em></a><em>, an AI music video pipeline for indie artists. He was previously COO at Tabler App (1M+ users, $50K+ MRR, successful exit) and a data science consultant at Deloitte, Booz Allen Hamilton, and Accenture. Find him on LinkedIn: </em><a href="https://www.linkedin.com/in/syedali-/"><em>https://www.linkedin.com/in/syedali-/</em></a></p><h4>Sources</h4><ol><li>Crunchbase News, Gené Teare, October 5, 2026: “Global venture funding totaled $159 billion in Q3 2026 with close to 6,000 startups funded”; “raised $102 billion, or 64% of global venture capital”; “27 companies raised such sizable rounds in Q3 2026” (an all time record); “Around a third of all global venture capital raised in Q3 went to those 27 companies”; “Global seed funding totaled $13 billion in Q3. Of that, $2.6 billion went to seed rounds of $100 million and over.” <a href="https://news.crunchbase.com/venture/q3-2026-global-startup-funding-ai-billion-dollar-rounds-exits-data/">https://news.crunchbase.com/venture/q3-2026-global-startup-funding-ai-billion-dollar-rounds-exits-data/</a></li><li>Crunchbase News, Gené Teare, April 1, 2026: “Seed funding totaled $12 billion, up 31% year over year, though the increase was entirely due to larger rounds, with deal counts falling 30% year over year to 3,800”; four companies raised “65% of global venture investment in the quarter.” <a href="https://news.crunchbase.com/venture/record-breaking-funding-ai-global-q1-2026/">https://news.crunchbase.com/venture/record-breaking-funding-ai-global-q1-2026/</a></li><li>Crunchbase News, Gené Teare, March 26, 2026: “U.S. seed funding totaled $19.4 billion in 2025”; “51% in seed deals $10 million and over, compared to a third in 2024.” <a href="https://news.crunchbase.com/venture/seed-funding-skewing-larger-ai-competitive-data">https://news.crunchbase.com/venture/seed-funding-skewing-larger-ai-competitive-data</a></li><li>Axios, July 9, 2026, reporting the PitchBook and NVCA Q2 2026 Venture Monitor: “over 81% of the H1 2026 dollars going to deals of $100 million or more”; US companies raised $412.7 billion between January and June. <a href="https://axios.com/2026/07/09/venture-capital-record-year">https://axios.com/2026/07/09/venture-capital-record-year</a></li><li>SiliconANGLE, Duncan Riley, July 9, 2026: “Rounds of $100 million or more accounted for 87.5% of everything deployed in the half.” <a href="https://siliconangle.com/2026/07/09/pitchbook-us-venture-funding-hits-412-7b-first-half-ai-deals-dominate/">https://siliconangle.com/2026/07/09/pitchbook-us-venture-funding-hits-412-7b-first-half-ai-deals-dominate/</a></li><li>Carta, State of Pre Seed Q2 2026, Hamza Shad, August 13, 2026: $3.19 billion across more than 11,500 instruments in Q2 2026 versus $3.22 billion across 14,825 a year earlier; “the average instrument size came out to $276,000.” <a href="https://carta.com/data/state-of-pre-seed-q2-2026/">https://carta.com/data/state-of-pre-seed-q2-2026/</a></li><li>Causo (secondary, summarising Carta), updated October 7, 2026: “The median seed founder is raising about $4.1 million at a $24.3 million valuation and giving up 18%.” <a href="https://hub.causo.ai/guides/seed-funding-monthly-brief">https://hub.causo.ai/guides/seed-funding-monthly-brief</a></li><li>Causo (secondary, summarising the PitchBook and NVCA Q1 2026 Venture Monitor), reviewed October 7, 2026: median seed pre money $18.4M (AI $18.7M, non AI $18.0M); median seed deal value $3.0M. <a href="https://hub.causo.ai/guides/h1-2026-state-of-seed-report">https://hub.causo.ai/guides/h1-2026-state-of-seed-report</a></li><li>CRV, July 16, 2026: “Investors now require a structural advantage independent of model performance”; “The median time between seed close and Series A close has stretched to 20 months.” <a href="https://crv.com/content/ai-startup-funding">https://crv.com/content/ai-startup-funding</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7f5352e60f04" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Your RAG Eval Set Is Grading Your Own Question Generator]]></title>
            <link>https://medium.com/@syed_ali/your-rag-eval-set-is-grading-your-own-question-generator-7e8aa6629aa8?source=rss-08b3001202b0------2</link>
            <guid isPermaLink="false">https://medium.com/p/7e8aa6629aa8</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Syed Ali]]></dc:creator>
            <pubDate>Thu, 29 Oct 2026 11:38:37 GMT</pubDate>
            <atom:updated>2026-10-29T11:38:37.986Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Editorial cover with the headline Synthetic scores are not launch scores and the subhead Hit@5 fell 29 to 37 points on real questions. A blue key fits perfectly into a lock cut into a stack of aluminium plates, while a short clear acrylic key with an orange tip lies beside it and does not match." src="https://cdn-images-1.medium.com/max/1024/1*QM-GODSJsEsQ_26hfUY-XQ.png" /><figcaption>A key cut from the documents fits their lock; the questions users bring do not. Image created by the author with AI.</figcaption></figure><p><em>RAG evaluation on synthetic questions ranks retrievers but not readiness. The questions your pipeline writes for itself are longer, cleaner and easier than the ones your users type, so the score you launch on is a score for a user who does not exist.</em></p><p>A retrieval benchmark can be honest and still tell you nothing about your users. In a paper submitted to arXiv in September 2026, hit rate at five fell by 29 to 37 percentage points when the same retriever setups met real questions instead of synthetic ones. The synthetic questions averaged 15.7 words. The real ones averaged 6.8. As a former data science consultant I have seen this failure in many costumes: the test set was written by the same process that built the system, so the system passes. The paper is Filip Kucia and Barbara Gawlik’s RAG evaluation of one faculty information system.</p><p>Those consulting years were at Deloitte, Booz Allen Hamilton and Accenture. The habit they left me with is boring and useful. Before I trust a score, I ask who wrote the test. When the answer is “the same team, with the same tools, from the same documents”, the score is a measure of internal consistency. It is not a forecast of how the system will behave in front of a stranger.</p><p>This piece is for people who ship retrieval augmented generation in production and decide launches on an eval dashboard. If your RAG feature is an internal demo with ten users who all read the docs, your synthetic set is probably fine and you can stop here. If real people type into the box, read on, because the gap is measurable and the audit takes a week.</p><h4>What a synthetic eval set actually measures</h4><p>A synthetic eval set measures whether your retriever can find the passage that a language model was looking at when it wrote the question. That is a real skill. It is also a narrower one than “can my system answer my users”, because the question was born inside the document it is supposed to find.</p><p>The usual recipe for how to create a RAG evaluation dataset from documents goes like this. You chunk your corpus, hand each chunk to a model, and ask for a question that the chunk answers. The model is good at this. It writes a full sentence, uses the document’s own vocabulary, names the entity, and asks exactly one thing. You store the chunk as the gold answer and repeat a few thousand times.</p><p>It helps to separate three kinds of eval question, because teams mix them and then argue past each other.</p><p><strong>The first kind is generated.</strong> A model writes questions from your chunks. It is cheap, it scales to thousands, and it covers every corner of the corpus. Its questions are long, specific and answerable by construction.</p><p><strong>The second kind is hand written.</strong> Someone on the team writes questions they think users will ask. Better than generated, still written by a person who knows where the answers live.</p><p><strong>The third kind is logged.</strong> Real queries from real users, with a gold document attached afterwards by a human. Short, vague, misspelled, sometimes unanswerable. This is the only kind that measures the distribution you will actually serve.</p><p>The common confusion is between the first and the third. A generated question shares vocabulary with its gold chunk almost by definition, so lexical overlap does a lot of the retrieval work. A user who types “exam retake rules” shares very little vocabulary with a policy paragraph titled “Conditions for repeated assessment”. Same information need, very different retrieval problem.</p><h4>The RAG evaluation gap, with numbers</h4><p>RAG evaluation on synthetic questions overstated retrieval quality by 29 to 37 points of hit rate in the clearest measurement I have found. Kucia and Gawlik, at Warsaw University of Technology, tested three retriever configurations on a university faculty chatbot. They generated 1,851 synthetic questions with Google’s Gemini Notebook, the tool formerly called NotebookLM, in its default setup. Then they collected 322 authentic queries from 89 survey respondents.</p><p>The Hit@5 results, synthetic against authentic, per the paper:</p><ul><li>ChromaDB, dense retrieval: 0.820 against 0.527</li><li>Qdrant, dense retrieval: 0.849 against 0.545</li><li>Qdrant, hybrid retrieval: 0.896 against 0.527</li></ul><p>MRR@10 moved the same way, from 0.662, 0.687 and 0.709 down to 0.400, 0.413 and 0.400. The authors summarise it as a drop of 29 to 37 percentage points in Hit@5 across all configurations.</p><figure><img alt="Animated diagram of three cards. In each card the synthetic Hit@5 counts up (0.820, 0.849, 0.896), then the authentic Hit@5 counts up (0.527, 0.545, 0.527), then a pill rises with the drop: minus 29, 30 and 37 points." src="https://cdn-images-1.medium.com/max/1024/1*3Kv9LakcwXdovtWnvtGbjA.gif" /><figcaption>Hit@5 for three retriever setups on synthetic and authentic queries, per Kucia and Gawlik (arXiv, September 2026). Animation created by the author with AI.</figcaption></figure><p>The mechanism is in two other numbers from the same paper. Synthetic questions averaged 15.7 words and authentic ones 6.8. Synthetic questions pointed to 165 unique sources and authentic ones to 53. So the generator wrote long questions spread across three times as many documents, while real people wrote short questions bunched on a few documents they actually needed. My read of the mechanism: long questions carry more matching terms, and wider coverage dilutes the hard cases. Both push the synthetic score up.</p><figure><img alt="Two columns compare synthetic and real questions: synthetic questions average 15.7 words and point to 165 unique sources, real questions average 6.8 words and point to 53. A band reads: Long questions reuse the document words. Real users do not." src="https://cdn-images-1.medium.com/max/1024/1*ouk7-rgCo666SUo7FCFJjQ.png" /><figcaption>Generated questions were longer and spread over three times as many sources as real ones. Image created by the author with AI.</figcaption></figure><p>The implication is about thresholds, not just accuracy. If your launch rule is “Hit@5 of at least 0.85”, the hybrid configuration clears it on synthetic data at 0.896 and misses it by more than 30 points on authentic data. The go or no go decision flips.</p><p>A second measurement points the same direction, with a caveat on who published it. A Chroma technical report from April 2025, by Kelly Hong, Anton Troynikov, Jeff Huber and Morgan McGuire, compared generated queries with real ground truth from WandBot, a Weights and Biases support bot. OpenAI’s small embedding model scored a recall@10 of 0.439 on the real queries and 0.530 on generated queries built with context and examples. Chroma builds a vector database and the report uses one real dataset, so I treat it as supporting evidence, not independent confirmation.</p><p>The gap matters most for the RAG evaluation metrics that teams put in launch decks: absolute hit rate and recall. Those are the numbers that inflate.</p><h4>Where synthetic sets still work, and where they fail</h4><p>Here is the concession, and it is a real one. For choosing between retrievers, synthetic sets mostly agree with human ones.</p><p>Jonas van Elburg, Peter van der Putten and Maarten Marx asked “Can we Evaluate RAGs with Synthetic Data?” in an arXiv paper from August 2025. Across four datasets, two of them proprietary, they found moderate to high agreement between synthetic and human benchmarks when ranking retriever configurations. The Chroma report found the same pattern: generated queries “generally capture the relative performance differences among embedding models”, with one swap on NDCG@10 where the scores were close.</p><p>So if your question is “is retriever A better than retriever B on my corpus”, a synthetic set is a reasonable first answer. Two places break it.</p><p>The first is generator ranking. When van Elburg and colleagues ranked the answer generating models instead of the retrievers, the mean Kendall tau between synthetic and human rankings was 0.29 on SQuAD, 0.05 on ASQA, minus 0.07 on Launchpad and minus 0.33 on Sales. A tau near zero means the synthetic ranking tells you nothing about the human one. A negative tau means it points the wrong way. The authors put it plainly: several combinations of metric and dataset yield low or even negative Kendall tau values.</p><p>The second is the RAGAS style metric Context Precision, which had a mean tau of 0.08 between human and synthetic benchmarks in the same study. If your RAGAS evaluation dashboard leans on Context Precision to compare configurations, the synthetic version of that number is close to noise as a ranking signal.</p><p>My read of the two papers together is simple. Synthetic sets are a decent ruler for comparing retrievers and a poor ruler for absolute quality and for anything downstream of retrieval. I wrote about the downstream half earlier this month from the judge’s side, in <a href="https://medium.com/@syed_ali/the-llm-judge-agrees-with-humans-85-of-the-time-here-is-what-that-number-hides-e62435cae925">The LLM Judge Agrees With Humans 85% of the Time</a>. This piece is about the other end of the same pipeline: who wrote the questions.</p><figure><img alt="Verdict board titled Ranks retrievers. Not readiness. Mostly holds: retriever ranking. Breaks: absolute hit rate, generator ranking with tau from 0.29 to minus 0.33, and Context Precision with tau 0.08. A strip notes the synthetic winner, Qdrant hybrid, took 2.70 s per query versus 0.33 s." src="https://cdn-images-1.medium.com/max/1024/1*d8fBZ06u4yS-mDNj-hGxvQ.png" /><figcaption>Synthetic sets mostly agree on retriever ranking and break on absolute scores, generator ranking and Context Precision. Image created by the author with AI.</figcaption></figure><h4>How the wrong eval set picked a retriever 8 times slower</h4><p>Mostly preserved is not always preserved, and the Kucia and Gawlik paper has the case that costs money.</p><p>On synthetic questions, Qdrant hybrid was the clear winner at 0.896 Hit@5. On authentic questions it tied for last at 0.527, behind Qdrant dense at 0.545. The paper’s latency table lists the hybrid setup at 2.70 seconds per query against 0.33 seconds for ChromaDB dense, and the authors describe the hybrid configuration as costing “up to 8× the latency of the fastest configuration we tested.” A team optimising on the synthetic set would have shipped the slowest configuration for no measured gain on real traffic.</p><p>Why would hybrid look so good on generated questions? Hybrid retrieval adds a sparse, keyword matching component to the dense embedding search. My read, and it is an interpretation rather than something the paper isolates, is that sparse matching rewards exactly what a generator produces: long questions that reuse the document’s own terms. A 15.7 word question lifted from a chunk has plenty of exact terms to match. A 6.8 word question from a student who does not know the official vocabulary has few. The component that helped most on the test set was the one whose advantage came from how the test set was written.</p><p>That is the general shape of the failure. When the eval set and the system share a bias, the eval set rewards the parts of the system that exploit the bias. You do not see it in the score. You see it in the latency bill and in user complaints.</p><h4>Where this thesis breaks: the honest caveats</h4><p>Three objections deserve a real answer, not a nod.</p><p><strong>Synthetic sets rank systems fine, so who cares about absolute scores.</strong> Partly true, and I conceded it above. Van Elburg and the Chroma report both find retriever rankings mostly hold. But generator rankings did not hold, Context Precision ranked at a tau of 0.08, and the Kucia and Gawlik case shows the top pick changing to a configuration with roughly eight times the latency. Ranking holds often. Often is not the same as reliably, and you do not know in advance which case you are in.</p><p><strong>Absolute scores do not matter, only relative ones.</strong> They matter the moment a threshold enters the room. Launch gates, SLAs and “good enough to ship” calls are absolute. A fall from about 0.85 to about 0.53 is the difference between shipping and not shipping, and no amount of correct relative ordering rescues a wrong go or no go call.</p><p><strong>Real query sets are small and noisy, so skip them.</strong> This is the strongest objection and I agree with half of it. The 322 authentic queries in the paper came from 89 people, so they are not independent: one heavy user can tilt the score. They came from a survey, and the authors say themselves that they “are not necessarily a complete proxy for live traffic.” And the whole result is one faculty system. My answer is to keep both sets, not to pick one. Use synthetic for coverage across the corpus and for cheap retriever comparisons. Use real queries for the launch number, with confidence intervals resampled by user so the noise is visible instead of hidden.</p><p>I am not claiming synthetic eval sets are useless. I am not claiming every RAG system loses more than a third of its hit rate in production; this is one study of one deployment, and the size of the gap in yours is an empirical question. I am claiming that a synthetic score is a measurement of your question generator as much as your retriever, and that the launch decision should rest on something your users wrote.</p><h4>A 300 query audit you can run this week</h4><p>Here is how to evaluate RAG systems against this risk in about a week of part time work. It describes what to compute. Your numbers will be your own.</p><ul><li><strong>Generate a synthetic set</strong> of a few hundred to a few thousand questions with a fixed model and a fixed prompt, and save both so the set is reproducible.</li><li><strong>Collect 150 to 300 real queries</strong> from logs or a short user survey, and have someone attach the gold document or documents to each one. Keep the user ID.</li><li><strong>Score every configuration on both sets</strong> with hit@1, hit@3, hit@5, hit@10 and MRR@10. Report the drop per configuration, plus average query length and the count of unique gold sources on each side.</li><li><strong>Put a 95% interval on the real scores</strong> by resampling users, not queries, so one prolific user cannot fake precision.</li><li><strong>Compare the two rankings</strong> with Kendall tau, check whether the winner changes, and look up the winner’s latency before you call it a win.</li></ul><figure><img alt="Five numbered step cards: generate a fixed synthetic set, collect 150 to 300 real queries, score both with hit@k and MRR@10, intervals resampled by user, compare rankings and latency. A band reads: Let the users grade you." src="https://cdn-images-1.medium.com/max/1024/1*69AOViJ_YhqEAEA5h_IQrg.png" /><figcaption>The five step audit to run before a launch call. Image created by the author with AI.</figcaption></figure><p>The code below covers the parts people usually skip: the interval resampled by user and the ranking comparison. It needs only Python and SciPy.</p><pre>import random<br>from scipy.stats import kendalltau<br>def hit_at_k(ranked_ids, gold_ids, k):<br>    return 1.0 if set(ranked_ids[:k]) &amp; set(gold_ids) else 0.0<br>def score(results, k=5):<br>    &quot;&quot;&quot;results: list of dicts with keys user, ranked, gold.&quot;&quot;&quot;<br>    return sum(hit_at_k(r[&quot;ranked&quot;], r[&quot;gold&quot;], k) for r in results) / len(results)<br>def bootstrap_by_user(results, k=5, n=2000, seed=7):<br>    &quot;&quot;&quot;95% interval for hit@k, resampling whole users, not single queries.&quot;&quot;&quot;<br>    rng = random.Random(seed)<br>    by_user = {}<br>    for r in results:<br>        by_user.setdefault(r[&quot;user&quot;], []).append(r)<br>    users = list(by_user)<br>    stats = []<br>    for _ in range(n):<br>        picked = rng.choices(users, k=len(users))<br>        sample = [q for u in picked for q in by_user[u]]<br>        stats.append(score(sample, k))<br>    stats.sort()<br>    return stats[int(0.025 * n)], stats[int(0.975 * n) - 1]<br>def ranking_agreement(synthetic, real):<br>    &quot;&quot;&quot;Kendall tau between two dicts mapping configuration name to score.&quot;&quot;&quot;<br>    names = sorted(synthetic)<br>    tau, _ = kendalltau([synthetic[c] for c in names], [real[c] for c in names])<br>    return tau<br>if __name__ == &quot;__main__&quot;:<br>    demo = [<br>        {&quot;user&quot;: &quot;u1&quot;, &quot;ranked&quot;: [&quot;d3&quot;, &quot;d1&quot;, &quot;d7&quot;], &quot;gold&quot;: [&quot;d1&quot;]},<br>        {&quot;user&quot;: &quot;u1&quot;, &quot;ranked&quot;: [&quot;d2&quot;, &quot;d5&quot;], &quot;gold&quot;: [&quot;d9&quot;]},<br>        {&quot;user&quot;: &quot;u2&quot;, &quot;ranked&quot;: [&quot;d9&quot;, &quot;d4&quot;], &quot;gold&quot;: [&quot;d4&quot;]},<br>        {&quot;user&quot;: &quot;u3&quot;, &quot;ranked&quot;: [&quot;d8&quot;], &quot;gold&quot;: [&quot;d8&quot;]},<br>    ]<br>    print(&quot;hit@5&quot;, score(demo), &quot;interval&quot;, bootstrap_by_user(demo, n=500))<br>    synthetic = {&quot;chroma_dense&quot;: 0.820, &quot;qdrant_dense&quot;: 0.849, &quot;qdrant_hybrid&quot;: 0.896}<br>    real = {&quot;chroma_dense&quot;: 0.527, &quot;qdrant_dense&quot;: 0.545, &quot;qdrant_hybrid&quot;: 0.527}<br>    print(&quot;tau&quot;, ranking_agreement(synthetic, real))</pre><p>The demo feeds in the paper’s own Hit@5 figures. On those three configurations the Kendall tau between the synthetic and authentic rankings comes out at 0.0, which is my calculation from the published table, not a number the authors report. Three configurations is far too few for a tau to mean much on its own. It is still a fair picture of the problem: the synthetic ranking carried no usable information about the authentic one.</p><p>If the drop in your audit is small and the ranking holds, good. You have earned the right to iterate on synthetic data for a while. If the drop is large or the winner changes, you have found the problem before your users did.</p><p>Looking ahead, I see two ways this goes. In one, teams adopt paired sets as a default, synthetic for coverage and real for launch calls, and published RAG benchmarks start reporting both. In the other, teams keep shipping on synthetic scores because they are cheap and always available. The signal I would watch is whether the popular eval tools start asking for a sample of real queries by default. I would not bet on either scenario yet.</p><h4>FAQ</h4><p><strong>How to evaluate RAG?</strong></p><p>Evaluate retrieval and generation separately. For retrieval, measure hit@k and MRR on a set of questions with known gold documents. For generation, check answer correctness and faithfulness to the retrieved passages. Use real user queries for any number that decides a launch.</p><p><strong>How to evaluate RAG agents?</strong></p><p>The same retrieval checks apply to each retrieval step an agent takes, but you also need task level outcomes: did the agent finish the task, and how many steps and tool calls did it use. Agent queries are often reformulated by the agent itself, so log the query that actually hit the retriever, not only the user’s original message.</p><p><strong>How to evaluate RAG systems?</strong></p><p>Build two eval sets, one generated from your documents and one from real queries. Score each configuration on both, report the gap and the query length on each side, put intervals on the real scores resampled by user, and compare rankings with Kendall tau before you trust a winner.</p><p><strong>How to create a RAG evaluation dataset from documents?</strong></p><p>Chunk the corpus, prompt a model to write one question per chunk that the chunk answers, and store the chunk as the gold label. Fix the model and the prompt so the set is reproducible. Then remember what it measures: questions written from inside the documents, not questions your users will type.</p><p><strong>How to evaluate a RAG?</strong></p><p>Start with a small real query set, even 150 questions, because it is the only measure of the traffic you will serve. Add a larger synthetic set for coverage and cheap comparisons, and never let the synthetic score alone set a launch threshold.</p><h4>The takeaway</h4><p>A retrieval benchmark can be honest and still tell you nothing about your users. The fix is not a better generator. It is a second test set written by someone who did not build the system.</p><p>Before your next launch call, collect 150 real queries and score them. Let the users grade you.</p><p><em>Syed Ali</em></p><p><em>Founder and CEO, Echonos</em></p><p><em>Syed Ali is cofounder of </em><a href="https://echonos.ai/"><em>Echonos</em></a><em>, an AI music video pipeline for indie artists. He was previously COO at Tabler App (1M+ users, $50K+ MRR, successful exit) and a data science consultant at Deloitte, Booz Allen Hamilton, and Accenture. Find him on LinkedIn: </em><a href="https://www.linkedin.com/in/syedali-/"><em>https://www.linkedin.com/in/syedali-/</em></a></p><h4>Sources</h4><ol><li>Filip J. Kucia and Barbara M. Gawlik, “Beyond Benchmark Scores: How Synthetic and Authentic Query Distributions Diverge in RAG Evaluation”, arXiv 2609.14579, submitted September 2026: Hit@5 synthetic 0.820, 0.849, 0.896 against authentic 0.527, 0.545, 0.527 (ChromaDB dense, Qdrant dense, Qdrant hybrid); Hit@5 dropping 29 to 37 percentage points across all configurations; synthetic questions 15.7 words against authentic 6.8; 165 against 53 unique sources; 322 authentic queries from 89 respondents; Qdrant hybrid 2.70 s per query against 0.33 s for ChromaDB dense, “up to 8× the latency of the fastest configuration we tested”; authentic queries “are not necessarily a complete proxy for live traffic.” <a href="https://arxiv.org/html/2609.14579v1">https://arxiv.org/html/2609.14579v1</a></li><li>Jonas van Elburg, Peter van der Putten and Maarten Marx, “Can we Evaluate RAGs with Synthetic Data?”, arXiv 2508.11758, August 2025: several metric and dataset combinations yield low or even negative Kendall tau values; mean tau for generator rankings 0.29 (SQuAD), 0.05 (ASQA), minus 0.07 (Launchpad), minus 0.33 (Sales); Context Precision mean tau 0.08; moderate to high agreement for retriever rankings. <a href="https://arxiv.org/html/2508.11758v1">https://arxiv.org/html/2508.11758v1</a></li><li>Kelly Hong, Anton Troynikov, Jeff Huber and Morgan McGuire, “Generative Benchmarking”, Chroma technical report (vendor), 7 April 2025: WandBot recall@10 for OpenAI’s small embedding model 0.439 on ground truth queries against 0.530 on generated queries; generated queries “generally capture the relative performance differences among embedding models.” <a href="https://www.trychroma.com/research/generative-benchmarking">https://www.trychroma.com/research/generative-benchmarking</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7e8aa6629aa8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What Spotify’s 50 Million Removed AI Tracks Actually Measure]]></title>
            <link>https://medium.com/@syed_ali/what-spotifys-50-million-removed-ai-tracks-actually-measure-de05acb433cc?source=rss-08b3001202b0------2</link>
            <guid isPermaLink="false">https://medium.com/p/de05acb433cc</guid>
            <category><![CDATA[music-industry]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[spotify]]></category>
            <category><![CDATA[streaming]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Syed Ali]]></dc:creator>
            <pubDate>Tue, 27 Oct 2026 17:06:18 GMT</pubDate>
            <atom:updated>2026-10-27T17:06:18.815Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Editorial cover reading 50 MILLION, NO DENOMINATOR, with a tall stack of brass discs on a sea glass fraction bar above an empty slot where the denominator should be." src="https://cdn-images-1.medium.com/max/1024/1*sv_B4UCGnum2lHxl76sE-A.png" /><figcaption>A big count with nothing underneath it. Image created by the author with AI.</figcaption></figure><p><em>AI music detection claims, read as a data scientist. Spotify and Deezer each published a big count in October 2026, and neither published the denominator, the definition or the error rate that would turn a count into a measurement.</em></p><p>Fifty million is a large number, and it has nothing underneath it. When Spotify said on October 20 that it had stopped recommending more than 50 million tracks it considers AI slop or spam, it gave a count with no denominator, no split between AI and ordinary spam, and no error rate. I run an AI music video company, so I have a stake in how this story lands. I read the announcement the way I used to read a client’s model validation: what was counted, what was left out, and who pays when the classifier is wrong.</p><p>That habit has a source. Before Echonos I spent years as a data science consultant at Deloitte, Booz Allen Hamilton and Accenture, and later ran operations as COO of Tabler. A large part of that work was sitting across from a model owner and asking the dull questions: what was the base, where was the threshold, how often was it wrong, and who checked. Nothing below is about how music is made. It is about how a number is built.</p><p>My read is simple. The two biggest AI music detection announcements of the month report how busy the filters are, and say almost nothing about how accurate they are. A count of demoted tracks tells you a system is running. Whether it is right is a separate question, and the people who carry the cost of a wrong call are the ones the count never mentions.</p><p>This is written for the people on the receiving end of the next platform policy: artist managers, small label operators, distributor teams and anyone building AI music products. If you want a verdict on whether AI music is art, this is the wrong essay.</p><h4>Fifty million with no denominator</h4><p>The short answer first. Read every AI music detection claim as three numbers, not one: the count, the base it is drawn from, and the error rate. A platform that publishes only the first has told you how busy its filter is. It has told you nothing about how often the filter is wrong.</p><p>The Next Web reported on October 20, 2026 that Spotify said it had taken more than 50 million AI slop and spam tracks out of its recommendations this year. ¹ The same report adds that Spotify did not say how many of those tracks were made with AI and how many were other kinds of spam. Gizmodo, the same day, carried Spotify’s description of the method: several signals read together, including listener engagement and feedback, upload patterns, metadata and outside AI detection tools, because “no single signal can catch all AI slop or spam.” ²</p><p>Three numbers are missing, and each one changes what the headline means:</p><ul><li><strong>Catalog size.</strong> Fifty million out of what? Spotify did not give the number of tracks that passed through the same filter in the same period, so nobody outside the company can compute a share.</li><li><strong>AI share.</strong> The count bundles AI tracks with ordinary spam. An obvious spam upload and a fully generated song land in the same total.</li><li><strong>Error rate.</strong> There is no figure for how many of the 50 million were human tracks caught by mistake, and no count of reversals.</li></ul><figure><img alt="Diagram titled ONE NUMBER, THREE MISSING. Left card: WHAT SPOTIFY GAVE, 50 MILLION+ tracks taken out of recommendations, October 2026. Right, three cards marked NOT PUBLISHED: CATALOG SIZE, Out of how many?; AI SHARE, AI tracks or ordinary spam?; ERROR RATE, How many human tracks caught by mistake?" src="https://cdn-images-1.medium.com/max/1024/1*8noDM4E2Pf1K_cLqcuhu-A.png" /><figcaption>The one number Spotify gave, and the three it did not. Image created by the author with AI.</figcaption></figure><p>Coverage does not even agree on what was counted. Variety’s report describes the 50 million as AI generated tracks, while The Next Web and Gizmodo describe slop and spam together, and The Next Web stresses that the split is unknown. ¹ ² ⁴ I rely on the cautious version here, because two outlets carry it and because the bolder version claims a split Spotify has not published.</p><p>Why would a platform publish a count and stop there? Counts are cheap to produce and safe to release. A denominator invites a percentage, a percentage invites a comparison with last year, and an error rate invites an appeals queue. My read is that the 50 million figure was built to show effort, and as a measure of effort it is fine.</p><p>The implication for anyone who manages releases is narrower and more annoying. You cannot estimate your own risk from this announcement.</p><h4>Removed from recommendations is not removed</h4><p>The wording matters more than the number. The Next Web put it plainly: “The songs are still on the service, but Spotify’s algorithms and editors no longer suggest them.” ¹ Variety adds that the tracks can still be found through active search; Gizmodo does not say either way. ² ⁴</p><p>That makes four different actions worth separating, because headlines blur them:</p><ul><li><strong>Demoted.</strong> The track stays up and can be played, but recommendation and editorial surfaces stop pushing it. This is what the 50 million describes.</li><li><strong>Deleted.</strong> The track is taken down. Spotify separately reported deleting more than 75 million spammy tracks for serious or repeated violations. As aimusicpreneur points out, Spotify calls these spammy tracks and does not say how many involved AI. ³ The two figures should not be added together, and the 75 million should not be described as AI music.</li><li><strong>Tagged.</strong> A label tells the listener something about the track, with or without any change in reach.</li><li><strong>Filtered by the listener.</strong> This is Deezer’s approach. Music Business Worldwide reported on October 13, 2026 that Deezer will offer an on/off toggle that lets users exclude fully AI generated music from their search results, rolling out globally before the end of 2026. ⁵ TechCrunch reported the next day that the feature is “expected to be available before the end of the year,” so for now it is announced, not live. ⁷</li></ul><figure><img alt="Diagram titled REMOVED IS NOT REMOVED. Four cards build in from left to right: DEMOTED, stays up, stops being recommended; DELETED, taken down; TAGGED, labelled for the listener; FILTERED, the listener switches it off. Then a band wipes in: Same classifier underneath. No published error rate." src="https://cdn-images-1.medium.com/max/1024/1*GKHN_bHMk5pYLe-j1E9E2g.gif" /><figcaption>Four different actions, one classifier underneath. Animation created by the author with AI.</figcaption></figure><p>These four actions land on different people. Demotion costs the uploader reach and costs the listener nothing they can see. Deletion costs the uploader the track. A tag or a toggle hands the decision to the listener.</p><p>Every one of them runs on the same upstream decision: a classifier, or a stack of signals, deciding what a track is. What changes between platforms is who pays when that decision is wrong. This is the thing that does not change: none of the four comes with a published error rate for the decision underneath it.</p><h4>What the detection claims leave out</h4><p>Deezer has disclosed far more than Spotify, and it deserves credit for that. Music Business Worldwide reported on October 13, 2026 that Deezer receives nearly 90,000 fully AI generated tracks every day, and that they generate only 1% to 3% of streams. ⁵ In its October 19 analysis, MBW reported that Deezer says its tool is 99.8% accurate, may miss about two AI tracks in every 1,000, and wrongly flags fewer than one human made track in every 10,000. MBW also notes that these figures are Deezer’s own and have not been independently audited. ⁶</p><p>That is a real attempt at a measurement. It still leaves three gaps.</p><p><strong>Gap one: “fully AI” has no fixed edge.</strong> MBW quotes Deezer’s help page defining AI content as music fully created by AI generation tools rather than through traditional human composition and production. The same page also lists content with significant AI elements as a tagging trigger. ⁶ A false positive rate only means something against a stable definition. If the boundary moves between full and significant, the rate is measured against a moving target.</p><p><strong>Gap two: the rate has no audit.</strong> Fewer than one in 10,000 is a claim about whatever human catalog the detector was tested on. We do not know what that test set contained, whether it included tracks with AI assisted parts, or how the rate holds when a generated track is edited afterwards. My read is that the number may well be right. It is also unverifiable from outside, which is a different problem from being wrong.</p><p><strong>Gap three: appeals exist, appeal data does not.</strong> According to MBW, an artist who thinks a track was wrongly tagged appeals through the distributor, which escalates to Deezer for manual review, and “if the distributor does not respond within five business days, artists can go to the company directly.” Deezer has not published how many appeals it receives or how many succeed. ⁶</p><p>That third gap is the cheapest to close. The share of tags overturned on appeal is a floor on the real error rate, measured on live tracks instead of a test set. Neither Spotify nor Deezer has published one.</p><h4>Who the filter actually touches</h4><p>Put the reported numbers next to each other and three groups appear. This breakdown is my read, built from the sources above, and the boundaries between groups are exactly where the missing data sits.</p><p><strong>Bulk upload farms.</strong> Deezer’s own figures describe them: tens of thousands of fully generated uploads a day that collect a small share of listening. ⁵ For this group a filter is close to free to apply and close to harmless to get wrong, because almost nobody is listening. Detection is plausibly winning here. I wrote about the fraud side of the same flood in <a href="https://medium.com/@syed_ali/what-an-18-month-sentence-does-and-does-not-fix-in-ai-music-streaming-fraud-6bd8261a3b50">What an 18 Month Sentence Does and Does Not Fix in AI Music Streaming Fraud</a>; recommendation filters are a different instrument aimed at a related target.</p><p><strong>The ambiguous middle.</strong> Human artists who use AI on part of a track, and small independents whose production sits near the line. This is where a shifting definition and an unaudited rate matter most, and where the announcements say least. There is no published rate of how often these tracks are demoted or tagged. Public reports of wrongly flagged artists exist, but they are anecdotes, and anecdotes are not a rate in either direction.</p><p><strong>Licensed AI.</strong> On the August 5, 2026 earnings call, Warner Music Group CEO Robert Kyncl said WMG expected its “licensing deals to contribute materially to our subscription streaming revenue growth starting in fiscal ‘27,” as Music Business Worldwide reported. ⁸ No dollar figure was given. My read is that a licensed AI track does not need to beat a classifier at all. It arrives with contracts that tell the platform what it is. Its protection is commercial, and detection never enters into it. I looked at who gets paid in one such arrangement in <a href="https://medium.com/@syed_ali/spotify-licensed-ai-covers-who-gets-paid-and-what-nobody-has-disclosed-190cb304538f">Spotify Licensed AI Covers: Who Gets Paid, and What Nobody Has Disclosed</a>.</p><figure><img alt="Diagram titled THREE GROUPS, ONE MISSING RATE. BULK UPLOAD FARMS: tens of thousands of uploads a day, a small share of listening, detection plausibly winning. THE AMBIGUOUS MIDDLE: human artists using AI on part of a track, no published rate. LICENSED AI: arrives with contracts that say what it is, protected by contracts." src="https://cdn-images-1.medium.com/max/1024/1*-7EJbkeJL5a7C_mEnWSxMg.png" /><figcaption>Three groups, and the one rate nobody has published. Image created by the author with AI.</figcaption></figure><p>Where this goes is not something I would bet on. Two scenarios:</p><ul><li><strong>Filters tighten.</strong> Platforms keep demoting and tagging, definitions widen from fully to substantially AI, and licensed AI becomes the main legal route for AI music to reach recommendations.</li><li><strong>Errors force disclosure.</strong> Complaints from wrongly tagged artists, carried by distributors and the trade press, grow loud enough that one platform publishes appeal counts and error rates on live tracks, and others follow.</li></ul><p>The signal is concrete. Watch whether Spotify or Deezer publishes an error rate measured on live tracks, or an appeals count, in its next update. If neither does, the first scenario is the one arriving.</p><h4>Where this thesis breaks</h4><p>You deserve the version of this argument where I am wrong. There are three, and I find each of them plausible.</p><p><strong>First, the filters may simply be working.</strong> Variety quoted Spotify saying “we’ve seen a marked decrease in listener concerns about this content showing up in recommendations.” ⁴ Gizmodo reported that Spotify has verified more than 300,000 human artists who account for over 99% of what listeners search for. ² If the filter mostly catches farms nobody plays, the missing denominator is a reporting gap and nothing worse. My response: the evidence is Spotify’s own, and the metric is listener complaints. Listeners are not the people who get demoted. A fall in complaints is a decent measure of the listening experience and silent on the borderline cases.</p><p><strong>Second, detection may be good enough.</strong> Deezer’s figure of fewer than one wrong flag in every 10,000 human tracks is very low. ⁶ My response: at nearly 90,000 fully generated uploads a day, the ambiguous middle still matters, the figure is self reported, and robustness to edited tracks has not been shown in public. I will not turn that rate into a count of wrongly flagged artists, because I do not have the denominator for that either. The standard I am asking of the platforms applies to me.</p><p><strong>Third, listeners may not care.</strong> A Deezer commissioned Ipsos survey of 9,000 people in eight countries, run in October 2025 and reported by CBS News via AFP on November 12, 2025, found 97% could not tell fully AI music from human music, while 80% wanted fully AI music clearly labelled. ⁹ TechCrunch cites the same 2025 survey as finding that around 45% of Deezer customers wanted a way to remove AI music from their feeds. ⁷ My response: both can be true at once. Listeners who cannot hear the difference can still want a label. The useful question was never whether listeners care. It is who bears the cost when the label is wrong.</p><p>I am not claiming that Spotify or Deezer has wrongly flagged any particular artist. I am not claiming their detectors are bad. I am claiming that nobody outside the two companies can currently tell.</p><p>The first counter is the one I take most seriously. If the filters overwhelmingly hit farms, my concern is about a small group. I hold the position anyway, because the size of that group is precisely the number nobody has published.</p><h4>Five questions to ask of the next AI music announcement</h4><p>The next announcement will arrive within weeks. Before you forward it, or change a release plan because of it, run it through five questions:</p><ol><li><strong>Denominator.</strong> Fifty million out of how many? A count without a base is a measure of activity.</li><li><strong>Definition.</strong> Fully AI, substantially AI, or spam? If the announcement mixes them, the number answers none of the three.</li><li><strong>Error rate.</strong> Measured on which tracks, by whom, and on a test set or on live catalog?</li><li><strong>Appeal path.</strong> Who does an artist appeal to, how long does it take, and how many appeals succeed?</li><li><strong>Who audits.</strong> Has anyone without a stake in the result checked any of the figures?</li></ol><figure><img alt="Checklist titled BEFORE YOU FORWARD THE NEXT ONE: 1 DENOMINATOR, Out of how many?; 2 DEFINITION, Fully AI, substantially AI, or spam?; 3 ERROR RATE, Measured on which tracks, by whom?; 4 APPEAL PATH, Who, how long, how many succeed?; 5 WHO AUDITS, Checked by anyone without a stake?" src="https://cdn-images-1.medium.com/max/1024/1*YvjAv6dTBblTD3WvEJ1EvA.png" /><figcaption>Five questions to run on the next announcement. Image created by the author with AI.</figcaption></figure><p>If you manage artists, do one thing before the end of this month: ask your distributor, in writing, how an AI tag or a recommendation removal would reach you and how long an appeal takes. Keep the answer. If you run a distributor, publish that path before your artists have to ask.</p><h4>FAQ</h4><p><strong>how to detect ai music</strong></p><p>For listeners, mostly you cannot by ear: in the Deezer commissioned Ipsos survey, 97% of 9,000 respondents could not tell fully AI music from human music. ⁹ Platforms use classifiers combined with other signals; Spotify says it reads listener engagement, upload patterns, metadata and outside AI detection tools together. ²</p><p><strong>can youtube detect ai music</strong></p><p>The sources behind this piece do not cover YouTube’s detection, so I will not guess at its accuracy. The five questions above apply to any YouTube announcement in the same way.</p><p><strong>how does deezer detect ai music</strong></p><p>Deezer uses its own detection tool, which it says is 99.8% accurate with fewer than one wrong flag in every 10,000 human tracks. Those figures are Deezer’s own and unaudited, per Music Business Worldwide. Wrongly tagged artists appeal through their distributor. ⁶</p><p><strong>how to detect ai generated music</strong></p><p>There is no public tool in these sources with an independently audited error rate. For your own catalog, the more reliable route is provenance: keep a record of which parts of each release used AI, so you can answer a tag with evidence.</p><h4>Who loses if I am right</h4><p>The artists in the ambiguous middle lose first, particularly those whose distributor is slow to answer, because their only route to a correction runs through someone else’s inbox. Small distributors lose next, because they inherit appeal work the platforms have not sized.</p><p>The platforms lose something too, more slowly. A filter whose error rate is private asks for trust, and trust spent on a count is hard to earn back once the first well documented mistake reaches the press. Readers of headlines lose the least visibly. They come away believing 50 million AI songs were deleted, which is not what anyone reported.</p><h4>Closing</h4><p>Fifty million is still a large number. It tells us Spotify’s filter was busy in 2026, and Deezer’s figures tell us its detector is confident. Neither tells us how often either one is wrong, or who finds out when it is.</p><p>The fix is not exotic. A base, a definition, an error rate on live tracks, an appeals count. Only the platforms can produce those numbers, and so far they have published the one that shows effort.</p><p><em>A count tells you how much a filter did. Only an error rate tells you whether it should have.</em></p><p><em>Syed Ali</em></p><p><em>Founder and CEO, Echonos</em></p><p><em>Syed Ali is cofounder of </em><a href="https://echonos.ai/"><em>Echonos</em></a><em>, an AI music video pipeline for indie artists. He was previously COO at Tabler App (1M+ users, $50K+ MRR, successful exit) and a data science consultant at Deloitte, Booz Allen Hamilton, and Accenture. Find him on LinkedIn: </em><a href="https://www.linkedin.com/in/syedali-/"><em>https://www.linkedin.com/in/syedali-/</em></a></p><h4>Sources</h4><ol><li>The Next Web, 20 October 2026: “The songs are still on the service, but Spotify’s algorithms and editors no longer suggest them.” Also reports Spotify did not say how many of the 50 million tracks were made with AI. <a href="https://thenextweb.com/news/spotify-50-million-ai-slop-tracks-recommendations">https://thenextweb.com/news/spotify-50-million-ai-slop-tracks-recommendations</a></li><li>Gizmodo, 20 October 2026: “no single signal can catch all AI slop or spam.” Also reports more than 300,000 verified human artists accounting for over 99% of searches. <a href="https://gizmodo.com/spotify-purged-over-50-million-ai-slop-and-spam-songs-from-user-recommendations-2000828403">https://gizmodo.com/spotify-purged-over-50-million-ai-slop-and-spam-songs-from-user-recommendations-2000828403</a></li><li>aimusicpreneur, 21 October 2026, quoting Spotify’s update: “AI detection technology in particular has limitations, and it will only get harder as generative AI evolves.” Reports more than 75 million spammy tracks deleted, a figure Spotify does not attribute to AI. <a href="https://www.aimusicpreneur.com/ai-music-news/spotify-50-million-ai-slop-tracks-hidden-75-million-removed-2026/">https://www.aimusicpreneur.com/ai-music-news/spotify-50-million-ai-slop-tracks-hidden-75-million-removed-2026/</a></li><li>Variety, 20 October 2026, quoting Spotify: “we’ve seen a marked decrease in listener concerns about this content showing up in recommendations.” <a href="https://au.variety.com/2026/music/news/spotify-removes-50-million-ai-slop-tracks-recommendations-41607/">https://au.variety.com/2026/music/news/spotify-removes-50-million-ai-slop-tracks-recommendations-41607/</a></li><li>Music Business Worldwide, 13 October 2026: Deezer receives nearly 90,000 fully AI generated tracks every day, which generate only 1% to 3% of streams; search toggle rolling out before the end of 2026. <a href="https://www.musicbusinessworldwide.com/deezer-to-let-users-block-ai-generated-music-from-search-results/">https://www.musicbusinessworldwide.com/deezer-to-let-users-block-ai-generated-music-from-search-results/</a></li><li>Music Business Worldwide, 19 October 2026: Deezer says its tool is 99.8% accurate and wrongly flags fewer than one human made track in every 10,000 (Deezer figures, not independently audited); appeals run through distributors. <a href="https://www.musicbusinessworldwide.com/90000-synthetic-tracks-a-day-why-deezer-is-handing-users-the-ultimate-ai-kill-switch/">https://www.musicbusinessworldwide.com/90000-synthetic-tracks-a-day-why-deezer-is-handing-users-the-ultimate-ai-kill-switch/</a></li><li>TechCrunch, 14 October 2026: “The feature is expected to be available before the end of the year.” Cites a 2025 Deezer and Ipsos survey: around 45% of Deezer customers wanted a way to remove AI music from their feeds. <a href="https://techcrunch.com/2026/10/14/deezer-will-soon-let-you-filter-out-ai-generated-music/">https://techcrunch.com/2026/10/14/deezer-will-soon-let-you-filter-out-ai-generated-music/</a></li><li>Music Business Worldwide, August 2026, WMG fiscal Q3 call of 5 August 2026, Robert Kyncl: “licensing deals to contribute materially to our subscription streaming revenue growth starting in fiscal ‘27.” <a href="https://musicbusinessworldwide.com/warner-music-expects-ai-music-deals-to-boost-subscription-streaming-revenue-from-late-2026">https://musicbusinessworldwide.com/warner-music-expects-ai-music-deals-to-boost-subscription-streaming-revenue-from-late-2026</a></li><li>CBS News (AFP), 12 November 2025: Deezer commissioned Ipsos survey of 9,000 people in eight countries, October 2025; 97% could not distinguish fully AI music from human music; 80% wanted fully AI music clearly labelled. <a href="https://www.cbsnews.com/amp/miami/news/ai-generated-music-real-thing-cant-tell">https://www.cbsnews.com/amp/miami/news/ai-generated-music-real-thing-cant-tell</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=de05acb433cc" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>