<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Muhammed Rasin on Medium]]></title>
        <description><![CDATA[Stories by Muhammed Rasin on Medium]]></description>
        <link>https://medium.com/@rasinbinabdulla?source=rss-db506407dbe0------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*GxbraYEDkcCXb3uCnONSfQ.png</url>
            <title>Stories by Muhammed Rasin on Medium</title>
            <link>https://medium.com/@rasinbinabdulla?source=rss-db506407dbe0------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 23:21:09 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@rasinbinabdulla/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Between real and right: ten months building Misata]]></title>
            <link>https://medium.com/@rasinbinabdulla/between-real-and-right-ten-months-building-misata-78ff8a20f07f?source=rss-db506407dbe0------2</link>
            <guid isPermaLink="false">https://medium.com/p/78ff8a20f07f</guid>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[synthetic-data]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[database]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Muhammed Rasin]]></dc:creator>
            <pubDate>Sun, 25 Oct 2026 15:25:44 GMT</pubDate>
            <atom:updated>2026-10-25T15:25:44.129Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*p6uGKI4khGXv7inLGAWylA.png" /></figure><p>Making data that hits your exact numbers turns out to be a problem computer scientists have studied since 2007. Here’s what ten months inside it taught me.</p><h3>Why not just ask Claude?</h3><p>Every pitch I give ends with the same question, usually asked kindly. Why not just ask Claude?</p><p>It’s a fair question. Ask Claude for a SaaS company’s data and in under a minute you have a script that prints customers, subscriptions and invoices, with names that sound like people and dates that sound like a calendar. It looks like a business.</p><p>Now add one sentence. Say monthly recurring revenue grows from $50,000 to $200,000 over two years. Say 2% of customers leave each month, except in the third quarter of 2025, when 5% do. Run the script again and add up the invoices. You’ll land somewhere near the story. You won’t land on it, and nothing in the output will tell you which months drifted or by how much.</p><p>That sentence is where Misata lives, in the distance between near and exact. I’ve spent ten months in that distance, and only last month did I find out how far down it goes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_iAoL7v2R2K8AAvixlL7SQ.png" /></figure><h3>A Sudoku, not a spreadsheet</h3><p>When I started, I thought I was building a better fake-data generator. Faker with more realistic names, maybe a few curves on top.</p><p>It turned out to be a Sudoku. Every number you state is a rule the rows must obey. Revenue in March is $112,400. Churn in August is 5%. Enterprise plans are 45% of revenue. Every invoice belongs to a real subscription, and nobody is billed after they cancel. Each rule on its own is easy. The trouble starts when they share the same rows. Change one invoice to fix March and you may break the enterprise share, or bill a customer who already left.</p><p>Sudoku has a useful property here. Solving one is hard, and checking a filled grid takes a minute. Computer scientists have proved that general Sudoku belongs to a class of problems with no known fast method for solving every case. Data that has to meet many stated numbers at once belongs to the same family. Checking it, on the other hand, is just running the SQL and comparing.</p><p>The solver took most of the engineering. The checker is the part I’m proudest of.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*d4eWc-neC8EU13pjGwsA0Q.png" /></figure><h3>Between real and right</h3><p>There are two ways to fail at this, and they pull in opposite directions.</p><p>One is data that is right and looks fake. Every month’s revenue lands to the cent, and every customer buys at 3 pm on a Tuesday, pays the same price, and leaves on the first of the month. A sales engineer can’t put that in front of a buyer. Anyone who has looked at real data for a week can see through it.</p><p>The other is data that looks real and is wrong. Shoppers buy in the evening, a few loyal customers carry half the orders, December is busier than February. Then you look closer. Nudge an order from 11:40 pm on the 31st to 12:10 am, a perfectly realistic shift, and it lands in the next month. Nudge enough of them and March’s total is no longer March’s. Make the customers who churn look unhappy before they go, and the churn rate you promised in August is off by a customer.</p><p>Every time I pushed the engine toward real, something exact broke. Every time I held exact, something started to look fake. Most of these ten months happened in that gap, fixing one side without tearing the other, one rule at a time. I’m still in between. I think that’s where the product is.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*M3cjSrywJXoo9nBct--yZg.png" /></figure><h3>Research I didn’t know I was continuing</h3><p>A month ago I learned that this problem has a name. In 2007, database researchers in Zurich published a paper called <em>Reverse Query Processing</em>. A normal database takes your data and a query and gives you an answer. They asked the opposite question: given the query and the answer you want, can you build the data? I’m building this without a team of database researchers beside me. Most of what I know about the field I learned in the last month, from papers written before I started.</p><p>A small field grew from that paper. QAGen built test databases where each step of a query returns a chosen number of rows. Microsoft researchers wrote a system that turns stated counts into a solvable equation. A group at the Indian Institute of Science built Hydra, which regenerates a client’s database from the shape of its queries. Others followed, up to papers published this year.</p><p>The field also proved how hard the problem is. For arbitrary SQL there is no general method at all, which mathematicians showed decades ago for the logic underneath SQL. Restricted versions, like three-way tables whose totals are fixed, are as hard as the Sudoku family.</p><p>Almost all of that work was built to test database engines, so their speed in the lab matches production. As far as I can find, none of it became something a sales engineer or an analytics team could pick up and use. One more line of work came closer to what I care about. In 2020, researchers grading text-to-SQL systems started building small databases designed to catch queries that look right and return the wrong answer. Misata’s query checker does a version of that.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qD84fXM2DJaGY-qtQpsItQ.png" /></figure><h3>What Misata promises</h3><p>Because solving is hard and checking is easy, I split the promise in two.</p><p>The first half is about the proof, and it has to be perfect. Every dataset comes with a certificate listing each number you stated, the month it applies to, the value in the rows, and the SQL that reads it back. Anyone can run that SQL in any database and get the same answer. If the certificate says your March revenue is $112,400, it is $112,400 to the cent. I test this on 360 sealed requests the engine has never seen, written before any build. Across those 360, it certified zero stated numbers wrongly.</p><p>The second half is about solving, and it can’t be perfect. Some combinations of numbers can’t all hold at once, and some can but the engine won’t find the way. When that happens Misata tells you which number it couldn’t hold and why, then builds the rest. It never substitutes a number you didn’t ask for. Most of my work now goes into raising how often it finds a way.</p><p>The proof also works in the other direction. Once you know the true answer, you can check someone else’s answer. Point the checker at a dashboard query and it tells you whether the number is right. When it isn’t, it tells you what the query computed instead, for example voided invoices counted as revenue, or months taken from the invoice date when they should come from the billing period.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ths5Uw8mEv2tOgO3npvPfQ.png" /></figure><h3>Why I’m building the product this way</h3><p>For two months, Misata Studio was open to anyone. You signed up, typed what you wanted, and got a dataset with its certificate. Forty-three people came in. They explored, built datasets, came back, built more.</p><p>Watching them taught me more than any plan had. People came for data that looked real. Very few of them needed it to be exact, and the few who did needed someone to walk them through the proof before it meant anything. A sign-up page can’t do that. It doesn’t learn, either. Each dataset I built alongside someone taught the engine something new, and each anonymous build taught it almost nothing. Self-serve, for this product and at this stage, wasn’t a business. It was a demo with a login.</p><p>Then the platform that hosted the backend had a security incident. I turned the backend off the same night and rotated everything it held. Studio has been paused since, with a notice on every page.</p><p>I’m using the pause. Misata was accepted into Anthropic’s Claude for Startups program, and the next version runs on Claude models, built and measured against a bank of requests it has never seen. That work is in progress. When it comes back, it comes back to people I can work with directly, one dataset at a time.</p><p>The same reasoning covers the rest. There’s no Product Hunt launch, because a launch brings dozens of people who want fake data fast, and Faker already serves them well, for free. There’s no fundraise yet, because money would let me hire faster without changing how many requests the engine can solve this month, and that is where the work is. Neither is a vow. When the engine clears its own bar, the answers may change.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-usOazcmXUGW0M8PZM5FGw.png" /></figure><h3>A short list of people</h3><p>When I pitch Misata, I can usually watch the moment it slips away. People hear “synthetic data” and picture everyone who has ever needed a test table. That’s millions of people, and most of them are served fine by a free library. Then they hear “exact” and can’t picture who would pay for it.</p><p>The list is short. At the top is the sales engineer whose demo has to show revenue climbing to the number in the pitch deck, with churn spiking in the quarter the story says it does. Beside them are the teams shipping AI analysts and dashboards built from a prompt. They can’t tell whether their product’s answer is wrong until they know the true one, and they would much rather find out before a customer does.</p><p>There aren’t many of them. Each one has felt the problem, though, usually in front of a customer, and none of them can solve it with a prompt. I’d rather build for a short list that needs this badly than a long one that would like it a little. It makes for a harder pitch and a better product.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pDp00nYa6Y96YvV2SlqxDg.png" /></figure><h3>Someone else noticed</h3><p>I found the name for my problem by accident. An engineer building a toolkit for modelling business domains needed test data that respects the rules in those models. They studied Misata’s open-source library file by file and wrote a long analysis of it in their project’s repository, which I came across later.</p><p>The analysis placed Misata in the reverse query processing line and named the 2007 papers. It was also fair about the weak spots. Its sharpest point was that hitting one stated total is easy: Misata’s own benchmark ships a 40-line baseline that does it. The work is in holding many numbers and rules together, and in a proof someone else can re-run. The author put the lasting value plainly: “the defensible asset is adoption of the certificate as evidence.”</p><p>It was strange and good to read my own project explained by a stranger more clearly than I had explained it. It’s also why I’m writing this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*S__2xbljUBt7wZEurpFvjg.png" /></figure><h3>If your numbers have to be right</h3><p>If you build demos, your data has to tell the story your buyer expects, down to the month. If you ship an AI analyst or dashboards from prompts, you need to know the true answer before you can tell whether yours is wrong. Both are the same problem, and both are what the ten months went into.</p><p>Send me one dashboard, or the SQL behind it. I’ll send back a dataset where its numbers hold exactly, the answer key with the SQL for each one, and the wrong queries it is most likely to be confused with. You keep all of it, whatever you decide after.</p><p>I’m at rasin@misata.studio.</p><h3>Further reading</h3><ul><li>Binnig, Kossmann and Lo, <em>Reverse Query Processing</em>, ICDE 2007 (<a href="https://vldb.org/dblp/db/conf/icde/icde2007.html">DBLP listing</a>, <a href="https://dsl.cds.iisc.ac.in/~course/TIDS/papers/rqp.pdf">paper</a>)</li><li>Binnig, Kossmann, Lo and Özsu, <em>QAGen: Generating Query-Aware Test Databases</em>, SIGMOD 2007 (<a href="https://se.ifi.uni-heidelberg.de/fileadmin/pdf/publications/2007_QAGen_Sigmod.pdf">paper</a>)</li><li>Arasu, Kaushik and Li, <em>Data Generation using Declarative Constraints</em>, SIGMOD 2011 (<a href="https://people.iiis.tsinghua.edu.cn/~jianli/paper/mod311arasu.pdf">paper</a>)</li><li>Sanghi, Ahmed and Haritsa, <em>Projection-Compliant Database Generation</em>, PVLDB 2022, from the Hydra group (<a href="https://www.vldb.org/pvldb/vol15/p998-sanghi.pdf">paper</a>)</li><li>Zhong, Yu and Klein, <em>Semantic Evaluation for Text-to-SQL with Distilled Test Suites</em>, EMNLP 2020 (<a href="https://arxiv.org/abs/2010.02840">paper</a>)</li><li><em>DBRepro: Automated Database Synthesis via a Hybrid Constraint-Solving Approach for Reproducing Slow Queries</em>, 2026 (<a href="https://arxiv.org/pdf/2608.27822">paper</a>)</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=78ff8a20f07f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I checked the numbers in 80 Google AI Mode answers against the pages they cite]]></title>
            <link>https://medium.com/@rasinbinabdulla/i-checked-the-numbers-in-80-google-ai-mode-answers-against-the-pages-they-cite-0f62558542d2?source=rss-db506407dbe0------2</link>
            <guid isPermaLink="false">https://medium.com/p/0f62558542d2</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[serpapi]]></category>
            <category><![CDATA[google]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[web-scraping]]></category>
            <dc:creator><![CDATA[Muhammed Rasin]]></dc:creator>
            <pubDate>Sun, 11 Oct 2026 16:01:05 GMT</pubDate>
            <atom:updated>2026-10-11T16:01:05.436Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UjZ79-mRBbJHKFs6Lr28zQ.png" /></figure><p>Google’s AI Mode cites its sources, and SerpApi’s Google AI Mode API returns those citations as data: each paragraph and list item arrives with the indexes of the references it draws on. That allows a narrow question to be asked of every statement. Is each number it states on a page it cites?</p><p>I asked it of 80 answers. Most cited figures were where the citation said they would be. More than a third of the statements with figures cited nothing. And when my checker reported a figure missing, it was usually wrong, which turned out to be the most useful thing I learned.</p><h3>What the API returns</h3><p>A search returns text_blocks and references. The docs describe several kinds of block. My 80 answers used three, paragraphs, lists and a single table, and those are what the parser reads:</p><pre>for block in answer[&quot;text_blocks&quot;]:<br>    if block[&quot;type&quot;] == &quot;paragraph&quot;:<br>        yield block[&quot;snippet&quot;], block.get(&quot;reference_indexes&quot;, [])<br>    elif block[&quot;type&quot;] == &quot;list&quot;:<br>        for item in block[&quot;list&quot;]:<br>            yield item[&quot;snippet&quot;], item.get(&quot;reference_indexes&quot;, [])<br>    elif block[&quot;type&quot;] == &quot;table&quot;:<br>        head, *rows = block[&quot;table&quot;]<br>        for row in rows:<br>            yield &quot;; &quot;.join(f&quot;{h}: {c}&quot; for h, c in zip(head, row)), block.get(&quot;reference_indexes&quot;, [])</pre><p>List items carry their own citations, so one list can mix cited and uncited items. A table’s citations, when it has any, sit on the whole block.</p><h3>How the check works</h3><p>The checker has no model in it. For each statement it takes every number, and every date precise to a month, and looks for it in the pages the statement cites: each reference’s title and snippet, and the page’s main text, fetched and extracted with trafilatura. A number counts as found if it appears as written, as the same value (4.0% for “4 percent”), converted, or rounded:</p><pre># &quot;25 basis points&quot; also reads as 0.25 percentage points; &quot;146.39 crore&quot; as 1,463,900,000<br>values = {value * (scale or 1)} | ({value / 100} if bps else set())</pre><p>A date counts if the page’s text gives it. A date that matches only the page’s own date, or the day of the search, is reported separately. Bare years are skipped, since every page has several.</p><p>I added this as footnote audit to <a href="https://github.com/rasinmuhammed/footnote">Footnote</a>, the search-grounding tool I had already built on SerpApi. One answer, trimmed:</p><pre>$ footnote audit &quot;What is the latest stable release of Rust?&quot;<br>The latest stable release of Rust is Rust 1.99.0, which was released on October 1, 2026. [1]<br>    ✓ 1.99.0   ✓ October 1, 2026<br>- Beta: Rust 1.100.0 is currently in beta. (no citation)<br>    · 1.100.0 (in [4])<br>- Nightly: Rust 1.01.0 represents the active nightly development channel. [4]<br>    ✗ 1.01.0</pre><p>There is no Rust 1.01.0, and the page cited for it, releases.rs, doesn’t mention one.</p><h3>80 answers</h3><p>I used the 20 questions I had evaluated Footnote on and 60 new ones, 10 each on central banks and the economy, software releases, sports, science and health, policy, and business. That came to 80 SerpApi searches on 11 October 2026. Each answer’s cited pages were fetched straight after its search, apart from 3 pilot answers fetched about 45 minutes later. 171 of the 205 pages could be read.</p><p>Uncited statements. Of 429 statements, 273 stated a figure, and 105 of those (38%) carried no citation of their own, in 42 of the 80 answers. The support was usually somewhere in the answer, though: 168 of the 183 uncited figures appeared in at least one of its references. Matching by value is loose, but it suggests AI Mode often has a source for a figure and leaves the statement unlinked.</p><p>Cited figures. 322 figures sat in statements that cite something. With the pages read, the checker found 276 as written or as the same value, 3 converted and 4 rounded, and flagged 39. From snippets alone it would have flagged 115. AI Mode’s snippets run to about 160 characters, so most of what a snippet check flags is outside the excerpt.</p><h3>Checking the flags by hand</h3><p>I checked all 39 flags against the live pages, in a random order and without knowing which kind of flag each was.</p><p>26 of the figures were on a cited page. 9 weren’t, 3 appeared only rounded, and 1 I couldn’t settle. When my checker said a figure was missing, then, it was right 12 times in 38.</p><p>All 26 misses came down to what the script could read:</p><ul><li>15 cited a page it couldn’t open. Every bls.gov page AI Mode cited, 6 in all, answered HTTP 403, as did India’s income tax site, the Senate’s Joint Economic Committee and a few others. Two cited a Bank of Japan PDF.</li><li>5 were in the page’s HTML but outside the main text the extractor kept, like “Latest Release: 1.37.1” on kubernetes.io’s releases page.</li><li>6 weren’t in the HTML at all: live quotes on Google Finance (3) and Worldometer (1), and figures from statements citing YouTube videos (2).</li></ul><p>My fetcher announces itself rather than posing as a browser. I’d keep it that way, but it means some pages Google reads are closed to it, and a checker can’t judge a page it can’t open.</p><h3>What the real gaps looked like</h3><p>The 9 figures no cited page states include that Rust “1.01.0”, a gold futures price that reached the answer as $[$4, 216.30]$ (garbled LaTeX), and Everest&#39;s height given as 29,031 feet, which neither cited page gives.</p><p>That’s 9 of 322 cited figures, and it should be read as a minimum. I didn’t hand-check the figures the checker found, and a number can turn up on a long page by chance.</p><h3>Footnote’s own checker on AI Mode</h3><p>I also ran Footnote’s original checker, unchanged, over AI Mode’s statements on my 20 questions. It reads only snippets, and it marked 7 of 58 cited statements supported. The open models I built it with scored between 60 of 74 and 114 of 120.</p><p>The comparison mostly measures the checker. Footnote’s model writes short claims from the snippets it’s shown, so snippets are the right thing to check them against. AI Mode writes longer statements from whole pages, of which a snippet is a small sample. With the pages read, the audit flagged 11 of AI Mode’s 77 cited figures on the same questions.</p><h3>Notes on SerpApi’s AI Mode API</h3><p>Where these touch documented behaviour, I’ve linked the <a href="https://serpapi.com/google-ai-mode-api">docs</a>.</p><ul><li>Read citations per list item. A list item carries its own reference_indexes, so a list can mix cited and uncited items. The one table I saw had none, though the docs&#39; table example does.</li><li>References have no date field. The docs’ example shows a date in front of the snippet (“Sep 18, 2025 — …”) and a #:~:text= passage in the link. None of my 206 references had either, so don&#39;t rely on them.</li><li>Expect LaTeX. 7 statements contained LaTeX commands, such as $8,848\text{ meters}$.</li><li>Expect offers. Answers often end with “If you’d like, I can…”, sometimes with a list under it. Filter them out before checking anything.</li><li>The two formats agree. reconstructed_markdown placed its [n] markers exactly where the JSON put reference_indexes in all 508 blocks I could match, so either is safe to parse.</li><li>Allow a few seconds per answer. By search_metadata.total_time_taken, AI Mode searches took a median of 6.7 seconds and at most 19.2.</li><li>Cache both layers. SerpApi <a href="https://serpapi.com/search-api">caches identical searches</a> for an hour, and cached searches are free. I also cache responses and fetched pages on disk, failures included, so every rerun audits the same text.</li></ul><h3>Next steps</h3><p>The first is to read pages the way a browser does, PDFs included, since that’s where most of the 26 misses came from, though some sites will refuse a script whatever it calls itself. After that I’d hand-check a sample of the figures the checker found, to see how often a number turns up on a page by chance. And I’d like to rerun the same 80 questions over the coming months, because AI Mode’s answers will change, and so will the pages it cites.</p><h3>Try it</h3><p>The code, the 80 answers, my hand labels and the diagnosis are at <a href="https://github.com/rasinmuhammed/footnote">github.com/rasinmuhammed/footnote</a>. It needs SERPAPI_API_KEY in a .env file.</p><pre>git clone https://github.com/rasinmuhammed/footnote &amp;&amp; cd footnote &amp;&amp; pip install -e &quot;.[audit]&quot;<br>footnote audit &quot;What did the latest US Federal Reserve meeting decide about interest rates?&quot;</pre><p>To build the core of it yourself, the <a href="https://github.com/rasinmuhammed/footnote/tree/main/tutorial">tutorial</a> does it in about 70 lines of Python, and runs in <a href="https://colab.research.google.com/github/rasinmuhammed/footnote/blob/main/tutorial/check_ai_mode.ipynb">Colab</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0f62558542d2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Hidden Toll Booth Sitting Inside Every AI Product]]></title>
            <link>https://pub.towardsai.net/the-hidden-toll-booth-sitting-inside-every-ai-product-7b4f948c0336?source=rss-db506407dbe0------2</link>
            <guid isPermaLink="false">https://medium.com/p/7b4f948c0336</guid>
            <category><![CDATA[synthetic-data]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Muhammed Rasin]]></dc:creator>
            <pubDate>Wed, 30 Sep 2026 00:01:03 GMT</pubDate>
            <atom:updated>2026-09-30T00:01:03.003Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*A0_kDCp_y25-m384oaY0ew.png" /></figure><p>A few weeks ago I was scrolling through Freelancer.com out of habit more than anything else, and I stopped on a listing that I have not been able to stop thinking about since. The project was titled “Synthetic Dataset and Generator for Waste Hauling Company”. 90 people had already bid on it. The average bid sat at just over 2000$, and the client’s own posted range ran from 1500–3000$.</p><p>Read the brief itself and it gets more interesting, not less. The client did not want real company data, and said so explicitly, no real customer, vendor, or employee data would be used or accepted anywhere in the project. What they wanted instead was a reproducible synthetic dataset for a completely fictitious solid waste hauling company, covering twenty four consecutive months of operations, built for internal staff training, for testing a reporting and business intelligence sandbox, and for practicing month end close procedures. The scope had two layers that had to reconcile with each other on shared keys, a financial layer built as a populated QuickBooks company file with general ledger, accounts receivable, accounts payable, payroll and fixed assets, and an operational layer underneath it made of the service, tonnage, container and route data that the financials were supposed to be derived from. The brief was blunt about what would and would not be accepted. A general ledger on its own would not be accepted. A hand keyed spreadsheet would not be accepted either, because the client did not just want a filled in file, they wanted a generator, something they could rerun, extend across a longer period, or reparametrize without hiring anyone again.</p><p>Sit with that for a second. This is not a request for a trained model or a finished product. It is a request for a made up dataset describing a company that does not exist, serving no purpose other than training, testing, and practice, and the market rate for producing it clusters somewhere around two thousand dollars, with ninety separate freelancers competing for the chance to build it by hand in Python and QuickBooks over three to four weeks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0neimK2ZXS-UIYMmelNpuw.png" /></figure><h3>The part of the AI boom nobody puts in the keynote</h3><p>Everyone talks about frontier models and benchmark scores and whatever the next trillion parameter announcement happens to be. Almost nobody talks about the extremely unglamorous, extremely expensive toll booth sitting between having an idea for a model and actually having data to train it on. That toll booth is staffed, in practice, by humans running scripts, and it charges enterprise rates for something that should, by rights, cost close to nothing.</p><p>I went looking for more examples once that waste hauling listing got under my skin, and once I started looking I could not stop finding them. Data labeling firms with page long service menus. Boutique consultancies advertising custom synthetic data pipelines with day rate consultants attached to the offer. Platforms that let you configure a schema and click generate, and then quietly bill you like you had hired a small team to do it by hand. I started asking around about what was actually happening behind some of these polished interfaces while the clock ran on a generation job. The honest answer, once I pushed past the marketing language, was usually a person. A sales engineer by title, but functionally someone sitting at a desk running a script somebody else had written a couple of years earlier, adjusting a handful of parameters by hand, and emailing the result back as a spreadsheet. Even with LLM assisted generation, while the surface level problems in this got solved, the high level complex dataset generation is not solved or is too expensive.</p><p>To understand why this got so expensive, it helps to understand what synthetic data generation actually looked like for most of the last decade, before anyone outside a research lab had ever typed a prompt into a large language model. If you needed training data and did not already have it, because it did not exist yet, because it was too sensitive to touch, or because there simply was not enough of it, your options were grim in a fairly consistent way. You could hire annotators to write examples by hand, which is slow, expensive, and stops scaling the moment you need more than a few thousand rows. You could scrape something adjacent to what you actually needed and hope the underlying distribution was close enough to be useful, which it usually was not. Or you could write a generation pipeline yourself, a script full of templates, sampling rules, and hardcoded domain logic, refined over weeks by someone senior enough to know what realistic data in that domain was even supposed to look like.</p><p>That third option is where most of this industry actually lived, and it explains the pricing that survives today. Somebody built a bespoke pipeline once, at genuine engineering cost, weeks of a senior person’s time, real domain expertise baked directly into the rules. Once that pipeline existed, running it again cost almost nothing. But the price the client paid never came down to reflect that, because what was actually being sold was never a single generation run. It was access to a scarce, brittle, hand tuned artifact that only a small number of people on the planet knew how to operate or safely modify. Scarcity plus opacity is a pricing strategy whether or not the person charging for it would ever describe it that way out loud.</p><p>Then large language models arrived and, on paper, blew this whole arrangement wide open. Anyone could type a request for five hundred examples of almost anything and get something usable back within seconds. Problem solved, you would think.</p><p>Not quite, and the waste hauling project is a good illustration of exactly why not. Calling a general purpose language model in a loop, row by row, to build a large structured dataset with strict internal consistency requirements is astonishingly wasteful once you actually try it at scale. You are paying premium per token rates for a reasoning engine built for open ended conversation to repeatedly solve the same narrow structural puzzle, over and over, with no memory of having already solved it the last nine thousand times. Do this across the volume a real dataset requires, with validation passes, deduplication, and formatting layered on top, and the bill grows fast, often far faster than anyone budgeting the project expected going in.</p><p>So the toll booth simply moved rather than disappeared. Instead of a hand built pipeline being the scarce resource, compute and prompt engineering expertise became the scarce resource instead. You end up with the worst features of both eras stacked on top of each other, the pricing structure inherited from the old scarcity model, now justified with a straight face by the very real compute costs of the new brute force model. Nobody sat down and actually separated what it costs to produce data of a given quality from what buyers have simply always been charged for this category of work. The number just persisted, carried forward by inertia and by the fact that most buyers had no independent way to check whether it was fair.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Rpdtf0-JAP9Yry6FyvYnjQ.png" /></figure><h3>The obvious thing that somehow still needs saying</h3><p>Here is the obvious thing, and it is strange that it still needs stating plainly. Synthetic data generation is not one task. It is several tasks stacked on top of each other, and only one of them genuinely needs a large, expensive, general purpose model doing real thinking.</p><p>Something needs to understand what the requester actually wants. Look again at that waste hauling brief and notice how much of it is not about the data itself but about the shape of the problem, the domain, the edge cases that would make a dataset genuinely useful versus merely plausible looking, and the hard constraints, in this case that the financial and operational layers must reconcile against each other on shared keys or the whole thing is worthless. That is a reasoning problem. It rewards a capable model doing real interpretive work, the same way a skilled data engineer would ask several rounds of clarifying questions before writing a single line of a pipeline.</p><p>But once that shape is actually understood, producing the rows themselves, potentially tens of thousands of them across twenty four months of operations with two layers that need to stay consistent with each other, is a different kind of problem entirely. It does not need a frontier model reasoning from first principles on every single row. It needs an efficient generation process, informed once by the initial understanding of the problem, that can then scale without its cost scaling linearly alongside it. That is much closer to classical structured generation and constraint satisfaction than it is to open ended language modeling, and treating both halves of the job as identical, paying frontier model prices across an entire pipeline end to end, is where nearly all of the waste in this industry actually lives.</p><p>I could not find anyone who had built that distinction directly into a product. Everyone I encountered while poking around this space was offering one of two things, either a manual consulting service dressed up behind a web form, exactly like that waste hauling listing on Freelancer, or a thin wrapper around a language model API that racks up token costs with no regard for the fact that most of that spend is buying redundant reasoning the tenth time it answers a question it already settled the first time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yr2fAyjwmQRxuCOHnKk6Lg.png" /></figure><h3>Where Misata Studio fits into this</h3><p>That gap is the reason I started building Misata Studio, and I want to be upfront about where it stands right now. It is early. It is a first attempt at this problem, not a finished cathedral, and I know it. But the core idea I am testing is exactly the separation I have been describing. Use the expensive reasoning where reasoning is genuinely needed, to understand the shape and intent behind a dataset the way you would want a thoughtful data engineer to, and then hand that understanding off to a leaner, more agentic generation workflow to actually do the producing, instead of dragging the entire pipeline, from understanding the brief down to writing the last row, through a traditional Python scripting process or a brute force call to a language model on every single unit of output.</p><p>The cost difference has not been subtle. Where a project like the waste hauling one clusters around two thousand dollars in freelancer bids, and where a straightforward generation run elsewhere can run over 41$ for output that takes 10mins to produce, I have been able to produce comparable structured output for well under a 1$. Not by cutting corners on internal consistency or realism, but by refusing to pay frontier model prices for work that does not actually require frontier model reasoning applied fresh to every row.</p><p>What I want, longer term, is to fully separate synthetic data generation from scratch, built on that kind of agentic workflow, from the traditional way this has always been done, hand rolled Python scripts and ML methodology applied manually project by project by whoever happens to win the bid. I think data storytelling deserves to be treated as an actual craft rather than invisible plumbing that happens to be priced like consulting. Misata Studio is my first baby step in that direction, nothing more grand than that yet. I am open tospending this stage talking to as many people as will give me fifteen minutes, people posting projects exactly like the waste hauling one, people building fraud detection models, people who just need ten thousand clean rows to unblock a deadline on a Tuesday, and every one of those conversations I hope will be changing something about what I think this should eventually become. The goal is not to announce a finished product today. It is to keep exploring real use cases with real people until what we build actually earns the description of a world class product, rather than just claiming it.</p><p>If any part of this sounds like a problem you have personally run into, staring at a quote that did not add up, or writing another throwaway generation script late at night because nothing off the shelf actually fit your case, I would genuinely like to hear about it. Not as a pitch. As two people comparing notes on a corner of this industry that has been quietly overpriced for a lot longer than it should have been.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7b4f948c0336" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/the-hidden-toll-booth-sitting-inside-every-ai-product-7b4f948c0336">The Hidden Toll Booth Sitting Inside Every AI Product</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why Every Good Demo Starts With Data, Not a Storyboard]]></title>
            <link>https://medium.com/@rasinbinabdulla/why-every-good-demo-starts-with-data-not-a-storyboard-9d9acd416f40?source=rss-db506407dbe0------2</link>
            <guid isPermaLink="false">https://medium.com/p/9d9acd416f40</guid>
            <category><![CDATA[demo-engineers]]></category>
            <category><![CDATA[presales]]></category>
            <category><![CDATA[data-consulting]]></category>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[sales-engineering]]></category>
            <dc:creator><![CDATA[Muhammed Rasin]]></dc:creator>
            <pubDate>Thu, 27 Aug 2026 04:36:40 GMT</pubDate>
            <atom:updated>2026-08-27T04:36:40.265Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NdrP67GyPr3d3_NfUQCOPw.png" /></figure><h3>Why demo environments break on slide four, what a decade of demo tooling quietly missed, and a pre-flight audit you can run in ten minutes before your next call.</h3><p>I spend most of my time building an open-source engine that generates synthetic relational data, which means I spend a lot of time reading about how other people solve the problem I work on. A few weeks ago I went down a rabbit hole on demo data specifically, mostly because I wanted to know whether the pain I kept hearing about was real or whether I was doing the thing founders do where you find evidence for the thing you already wanted to build. I came out more convinced than when I went in, and not for the reasons I expected. I also came out with a checklist I now run before any demo, which is the part of this article you can use today regardless of what you think of the rest.</p><p>The line that stopped me came from TestBox’s write-up of their last hundred-plus conversations with sales and GTM leaders. Buried in a section about maintenance costs, an unnamed sales leader describes what it takes to keep their product demos honest: “The expense of maintaining these demo experiences is a real burden. I’ve allocated two, well-paid, FTEs that I’ve had to carve off for it and I really wish I could use them elsewhere”. Two full-time engineers, both presumably capable of building the actual product, assigned instead to stopping the demo environment from telling lies. The speaker is anonymous, which is how it was published, and TestBox sells in this market so it is vendor research rather than a neutral study. I still take it seriously, because it is an admission against interest from the person signing those two salaries. Nobody exaggerates that number upward.</p><p>The same analysis found demo data has a shelf life “measured in days, not months”, with one team putting it as “after just one or two days, we’re already falling behind.” Another leader, describing the configuration work behind a single tailored demo, said the fastest person on their team still needs over eight hours. Not the average person. The best one.</p><h4>The half of the problem that got solved</h4><p>The demo automation industry is not short of money, talent, or good products. Saleo, Navattic, Storylane, Reprise, Demostack, Supademo, Consensus and a dozen more have built things that genuinely work, and if what you sell is a CRM or a project tracker, go buy one of them and stop reading. They made a real problem substantially smaller.</p><p>The way most of them work, underneath the packaging differences, is by intercepting what renders on screen and substituting demo values into it. Your dashboard shows four point two million dollars because the platform painted four point two million dollars over whatever number was actually in the database. You record your product once, overlay a guided path, swap the logos for the prospect’s, and ship a personalised walkthrough without writing code or standing up an environment. For software where the thing being evaluated is the screen, this is exactly the right architecture. The buyer’s experience is visual, so a visual layer is sufficient, and building a parallel data environment to achieve the same effect would be enormous waste.</p><p>The criticism that follows is not that these tools are bad. It is that they solved one half of a problem so thoroughly that the industry stopped noticing there was another half.</p><h4>The half that didn’t</h4><p>Consider what happens when the thing you sell is not a screen.</p><p>You sell a data warehouse. Or a transformation layer, an ingestion pipeline, a reverse-ETL tool, or an agent that reads records, decides something, and writes records back. Your buyer is a data engineer or an analytics lead, and the first thing they do in your demo is ask you to run a query they wrote. Not click a button you pre-recorded. A query, possibly one they have used to sanity-check vendors for three years.</p><p>There is nothing to paint. You cannot overlay a pixel onto a `GROUP BY`. The data is not the backdrop of the demo, it is the subject of the demo, and the moment the buyer joins two tables or drills into the detail behind a summary figure, any illusion built at the presentation layer evaporates. Some products are evaluated at the surface and some are evaluated in the substance, and the entire demo tooling industry is built for the first kind.</p><p>This matters more every quarter, because the fastest growing category of software right now is agents, and an agent is the most substance-shaped product that exists. It reads records, it decides, it writes. You cannot demonstrate that with a screenshot. You need real rows for it to act on, and afterwards you need to prove that what it did was correct, which means you needed to know what correct was before it started.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*R3yzw3dQ8teG_-5kqyfzVw.png" /></figure><h4>Why every workaround costs something</h4><p>Nobody in this industry is lazy. Sales engineers are among the most resourceful people in any software company, which is why the persistence of this problem is interesting rather than embarrassing. The two-FTE answer exists because all five alternatives are worse than they look.</p><p>The obvious move is to use the customer’s real data, and most teams try it first. It is also where deals sit down and rest. Before production data moves near your environment there is a data processing agreement and a security review, and custom DPA negotiations extend sales cycles by four to twelve weeks on average. That is not a delay in the demo, it is a delay in the deal, arriving exactly when the buyer’s enthusiasm is highest and most perishable. Masking a copy is the natural next thought, and the privacy-first tools do it well, but masking requires having the real data first, so it inherits the entire legal delay it was supposed to remove. You bought a compliance story, not a time-to-first-demo story, and those get conflated constantly in vendor marketing.</p><p>Then there is the sample data your platform already ships. Snowflake and Databricks both bundle TPC-H, and I would encourage anyone who has not looked at it recently to go and look. It is an order-management benchmark that database engines have competed on for decades: customers, orders, line items, parts, suppliers, nations, regions. It is genuinely excellent at measuring query engine performance under a standardised workload. It is also a 1990s parts catalogue, and if you are demoing to a hospital network or a freight company, everyone in the room knows within ninety seconds that they are looking at canned data.</p><p>Which leaves generating it, where most engineers land first because it feels obvious. Faker, Mockaroo, a script someone wrote two years ago. These tools fill columns one at a time, and that sentence contains the whole problem: a column-at-a-time generator has no idea what the column beside it says, let alone what the table across the join says. Your customers table has a signup date and your orders table has an order date, and nothing prevents an order arriving eight months before the customer existed. So you build it by hand in SQL, which works beautifully exactly once, and that is the trap. Then the prospect asks what the Nordics look like, or whether that Q3 dip shows up in the cost data too, and you are at your desk at eleven at night rewriting inserts before a nine o’clock call. Multiply by every deal and every account executive an SE supports, and you arrive quite logically at two full-time engineers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oYbWb8dEiR0iuxwDLA9x_w.png" /></figure><h4>The thing I had not seen anyone say</h4><p>It took me a while to work out what separates good demo data from bad, because “realistic” turns out to be the wrong word entirely. Realistic is what everyone asks for and it is not what anybody needs.</p><p>A demo is not a dataset. It is an argument.</p><p>When a sales engineer stands up in front of a room they are making one specific claim to a person deciding whether to spend money. The claim usually sounds like: your on-time delivery is quietly degrading and you cannot see it, and this product would have caught it in July. Everything in the demo exists to support that assertion. The dashboard is not there to look plausible, it is there to make the case.</p><p>Which means the degradation has to actually be in the data. Not on average, not approximately, but sitting in Q3 where you promised, visible in the region you named, consistent with every other table the prospect clicks into, and still true when they filter it three ways. The rows have to contain the argument. If they do not, you are not demoing your product, you are narrating over noise and hoping nobody checks.</p><p>This is why the problem is so durable and why it does not respond to obvious fixes. Faker fails because it has no concept of an aggregate at all; there is no mechanism by which you could ask it to make anything sum to anything. Masking fails because production data already has whatever story it has, and that story is usually either boring or actively unhelpful, since most businesses are not conveniently degrading in exactly the dimension your feature addresses. TPC-H fails because its story was fixed decades ago. Hand-written SQL is the only approach that can produce an argument, which is why people keep doing it despite the cost, and it fails on the second request rather than the first.</p><h4>Six queries to run before your next demo</h4><p>Here is the part you can use today, whatever you make of everything above. These are the checks I run against any demo environment before it goes in front of anyone. They take about ten minutes, they work on any warehouse with minor dialect adjustments, and between them they catch most of the ways a demo dies in public. Adjust the table and column names to yours.</p><p><strong>One: orphaned foreign keys</strong>. The single most common failure, and the one that produces an empty panel mid-demo with no explanation.</p><pre><br>SELECT COUNT(*) AS orphans<br>FROM fact_orders f<br>LEFT JOIN dim_customer c ON f.customer_key = c.customer_key<br>WHERE f.customer_key IS NOT NULL<br> AND c.customer_key IS NULL;<br></pre><p>Run it for every foreign key you have, not just the one you plan to show. The prospect will click the one you did not check. Anything above zero is a demo that will break the moment someone drills in.</p><p><strong>Two: roll-ups that do not reconcile</strong>. If your order total and the sum of its line items disagree, any finance-adjacent buyer will spot it, and they will lose trust in everything else on the screen.</p><pre>SELECT o.order_id,<br> o.total_amount,<br> SUM(li.line_amount) AS line_sum,<br> o.total_amount - SUM(li.line_amount) AS drift<br>FROM orders o<br>JOIN order_lines li ON li.order_id = o.order_id<br>GROUP BY o.order_id, o.total_amount<br>HAVING ABS(o.total_amount - SUM(li.line_amount)) &gt; 0.01<br>LIMIT 20;</pre><p><strong>Three: events that happen before they can</strong>. Orders before signup, shipments before orders, payments before invoices. Column-at-a-time generators produce these constantly because each date was drawn independently.</p><pre>SELECT COUNT(*) AS impossible_rows<br>FROM orders o<br>JOIN customers c ON c.customer_id = o.customer_id<br>WHERE o.ordered_at &lt; c.signed_up_at;</pre><p><strong>Four: how stale the environment already is.</strong> Run this every single time. Data ages while you are not looking, and a dashboard whose most recent activity is from six weeks ago tells the buyer they are looking at a museum.</p><pre>SELECT MAX(ordered_at) AS most_recent,<br> CURRENT_DATE - CAST(MAX(ordered_at) AS DATE) AS days_behind<br>FROM orders;</pre><p><strong>Five: the uniformity tell.</strong> This is the one almost nobody checks and it is the fastest giveaway to anyone who works with real data daily. Genuine business distributions are long-tailed: most customers order once or twice, a few order constantly. Generated data is usually suspiciously flat.</p><pre>SELECT order_count, COUNT(*) AS customers<br>FROM (<br> SELECT customer_id, COUNT(*) AS order_count<br> FROM orders<br> GROUP BY customer_id<br>) t<br>GROUP BY order_count<br>ORDER BY order_count;</pre><p>If every customer has between two and four orders, you have a tell. Real data has someone with one order and someone with four hundred.</p><p><strong>Six: does your argument actually survive a filter?</strong> This is the important one, and it is the check that follows directly from treating a demo as an argument. Whatever claim your demo makes, write the query that would disprove it, sliced the way a curious prospect would slice it.</p><pre>SELECT r.region_name,<br> DATE_TRUNC(&#39;quarter&#39;, f.shipped_at) AS qtr,<br> AVG(CASE WHEN f.delivered_at &lt;= f.promised_at THEN 1.0 ELSE 0.0 END) AS on_time_rate<br>FROM fact_shipments f<br>JOIN dim_region r ON r.region_key = f.region_key<br>GROUP BY r.region_name, DATE_TRUNC(&#39;quarter&#39;, f.shipped_at)<br>ORDER BY r.region_name, qtr;</pre><p>If your story is that on-time delivery degrades in Q3 and the Nordics is worst, this query has to show that, cleanly, without you narrating around it. Most demo data fails here even when it passes the first five checks, because nothing in the generation process ever knew what the story was supposed to be.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*ZAwQAWyaS8G_HbkBnWf23A.png" /></figure><h4>A fix for staleness that costs almost nothing</h4><p>The single highest-value change most teams can make is to stop storing absolute dates in demo data and start storing offsets. Instead of a fixed `2025–03–14`, store the number of days before the reference point, and resolve dates at load time against `CURRENT_DATE`. A view is usually enough:</p><pre>CREATE OR REPLACE VIEW demo_orders AS<br>SELECT order_id,<br> customer_id,<br> CURRENT_DATE - days_ago AS ordered_at,<br> CURRENT_DATE - days_ago + delivery_lag_days AS delivered_at,<br> amount<br>FROM orders_seed;</pre><p>Your seed data holds `days_ago` and `delivery_lag_days` rather than timestamps, so the environment is permanently current and the intervals between related events stay intact. This does not fix incoherent data, it only fixes stale data, but staleness is the failure with the shortest fuse and this defuses it once rather than every Monday morning.</p><h4>What actually good would look like</h4><p>Running those checks tells you whether your data is broken. It does not tell you how to build data that passes them by construction, and that gap is where the real work sits.</p><p>Three properties, unusual enough together that I understand why nobody has assembled them. The numbers have to be declared rather than emergent, so you can state that the year totals 87 percent, that Q3 breaches the SLA, and that the Nordics is worst, and get rows that satisfy all three exactly. Not close, exactly, because the demo you rehearsed on Tuesday needs to be the demo you give on Thursday. It has to hold up wherever the buyer clicks, which means every one of those six queries returns clean without anyone having gone and fixed it by hand. And it has to come with a proof you can read, not an assurance that it is fine but a check you can run yourself before it goes in front of a seven-figure prospect.</p><p>That last requirement matters more than it seems, and I think it explains why people keep hand-building despite the cost. It is not stubbornness or not-invented-here. It is that they need to be certain, and certainty is the one thing a black box cannot hand you. If a tool generates your demo data and you cannot verify what it did, you will check it manually anyway, at which point the tool saved you nothing.</p><h4>What I am building, and what I do not know yet</h4><p>For about 8 months I have been building an engine around a single idea: you declare what has to be true, and it solves the rows to satisfy that exactly, then proves it did. It is open source, deliberately, because the value proposition depends on you being able to read how a declared aggregate is actually solved rather than taking my word for it. It is free, it runs locally, and if the six queries above found problems in your environment you can use it today without talking to me.</p><p>On top of that I have started building something aimed squarely at this problem: describe the prospect in prose, get a proposed warehouse schema for their vertical, declare what has to be true including the specific thing your demo turns on, and have it generated, verified, and loaded with a certificate attached. Then revise it by asking, the way you would ask a colleague: make the Nordics worse, add a carrier that runs late thirty percent of the time. The other declarations stay where you put them, which is the part hand-written SQL can never do. That layer is not finished and I would rather say so than run a countdown timer over vapour.</p><p>Two things I genuinely do not know, and if you demo data products I would value your view on either. Which vertical is worth doing properly first, because getting that wrong costs weeks. And where the sharper pain actually sits: with vendor sales engineers prepping for a Thursday call, or with consultancies and systems integrators who have to show a working solution to win the work but cannot get the client’s data until after they have won it. That second group has a genuine chicken-and-egg problem nobody seems to be solving at all, and I keep changing my mind about which is the better place to start.</p><p>If you have ever spent an evening writing insert statements so a chart would bend the right way before a morning call, I wrote this after reading about people like you, and I would like to know how you actually handle it. Including, and maybe especially, if the answer is a script you are not proud of.</p><p><em>I build </em><a href="https://www.misata.studio/"><em>Misata</em></a><em>, an </em><a href="https://github.com/rasinmuhammed"><em>open-source engine</em></a><em> for generating relational data that satisfies declared outcomes exactly. The longer technical write-up, with every figure linked to its original source, is at </em><a href="https://www.misata.studio/demo-engineering).*"><em>https://www.misata.studio/demo-engineering.</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9d9acd416f40" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Half the Answer Keys in Text-to-SQL Benchmarks Are Wrong.]]></title>
            <link>https://pub.towardsai.net/half-the-answer-keys-in-text-to-sql-benchmarks-are-wrong-9ab82ca2e7fa?source=rss-db506407dbe0------2</link>
            <guid isPermaLink="false">https://medium.com/p/9ab82ca2e7fa</guid>
            <category><![CDATA[data-agents]]></category>
            <category><![CDATA[benchmark]]></category>
            <category><![CDATA[text-to-sql]]></category>
            <category><![CDATA[evaluation]]></category>
            <category><![CDATA[synthetic-data]]></category>
            <dc:creator><![CDATA[Muhammed Rasin]]></dc:creator>
            <pubDate>Sat, 11 Jul 2026 18:01:00 GMT</pubDate>
            <atom:updated>2026-07-11T18:01:00.914Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cwpsmWT-bRC1Gzik_ZuDeg.png" /></figure><h3>Half the Answer Keys in Text-to-SQL Benchmarks Are Wrong. So I Generated the Database From the Answer Key.</h3><h4>A VLDB 2026 audit found that most text-to-SQL answer keys are wrong; so I tried inverting the construction entirely.</h4><p>Earlier this year a team at UIUC audited the benchmarks the text-to-SQL field ranks itself on. The numbers in the paper (<a href="https://arxiv.org/abs/2601.08778">Jin et al., VLDB 2026</a>) stopped me cold: 52.8% of BIRD Mini-Dev annotations and 62.8% of Spider 2.0-Snow annotations are wrong. Not ambiguous. Wrong, as adjudicated by human SQL experts after an agent flagged them. When the authors corrected just 100 examples and re-evaluated 16 open-source agents, relative performance swung by as much as 31% and leaderboard positions moved by up to 9 places. Every team that picked a data agent off those leaderboards was, to a measurable degree, choosing based on noise. I build synthetic data tooling for a living, and my whole library is organized around one guarantee: declared numbers hold exactly in generated data. Reading that paper, it was hard to miss that the two problems are mirror images. Benchmarks struggle to attach correct answers to existing databases. My engine attaches a database to declared answers. So I spent this week building the inversion, and this post walks through the mechanism, the verification, and the parts that did not survive their own verification, because those are the most instructive.</p><h3>Why annotation-after-the-fact fails structurally</h3><p>The error rates are not annotator laziness. Look at how every benchmark in this lineage gets built. A database exists first, scraped or donated or assembled. Then question writers read schemas they did not design and write natural-language questions. Then annotators produce the gold SQL and expected answers by interrogating data full of null conventions, encoding quirks, and domain definitions they can only infer. The UIUC audit’s taxonomy shows exactly where that leaks. Semantic mismatches between question and SQL, like a strict inequality in the question compiled as an inclusive BETWEEN. Schema misunderstandings: missed aggregations, wrong joins, a filter on the wrong status column. Domain knowledge errors, their example being an annotator treating K-12 as equivalent to grades 1 through 12. And genuinely ambiguous questions with several defensible readings. Strip away the details and all of these share one structural property: they can only exist because the answer was derived <em>after</em> the data, by a human reading it. The answer key is a claim about the database, and claims can be wrong.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4VE1ndsHHtCtT32vMtM-4Q.png" /></figure><h3>Inverting the construction</h3><p>So make the answer key not a claim but a constraint. Declare the ground truth first, as a specification: exact monthly revenue for calendar 2025, a fraud rate rising from 2% to 3.5%, two foreign-key relationships that must have zero orphans. Then generate a database that satisfies the specification. Questions are derived from the declaration. The expected answer for “total order amount in November 2025” is not something anyone computed by reading rows. It is $750,000.00 because the data was constructed so that it would be. The generation math matters here, so briefly: for each declared period the engine allocates a row count from the target and the average transaction value, draws per-row amounts from a lognormal around that average, then rescales the period so it sums to the target exactly. Two stages, allocation then scaling, no learned model anywhere in the path, fully deterministic given a seed. The method is written up in <a href="https://arxiv.org/abs/2606.08736">the Misata paper</a>; the engine is open source. The property that makes it usable for evaluation is that the aggregate guarantee survives all the way to files on disk: modern float serialization round-trips exactly, so the CSV a grader reads carries the same cent-exact sums the generator produced.</p><h3>Generators don’t get to grade their own homework</h3><p>Here is the part I consider non-negotiable, because “the generator says the generator is right” should convince nobody. After the CSVs are written, every candidate question’s gold SQL is executed by DuckDB against those files, the ones that actually ship, and the observed answer is compared with the declared one. DuckDB shares no code with my engine. The two systems agree on nothing except the files. Each shipped question looks like this, from the actual pack:</p><pre>{<br> &quot;id&quot;: &quot;q011&quot;,<br> &quot;question&quot;: &quot;In the orders table, what is the total of amount during November 2025 (rows whose timestamp is on or after 2025–11–01 and strictly before 2025–12–01)? Give a number rounded to 2 decimal places.&quot;,<br> &quot;gold_sql&quot;: &quot;SELECT ROUND(SUM(\&quot;amount\&quot;), 2) FROM \&quot;orders\&quot; WHERE \&quot;order_date\&quot; &gt;= TIMESTAMP &#39;2025–11–01 00:00:00&#39; AND \&quot;order_date\&quot; &lt; TIMESTAMP &#39;2025–12–01 00:00:00&#39;&quot;,<br> &quot;expected_answer&quot;: 750000.0,<br> &quot;answer_type&quot;: &quot;number&quot;,<br> &quot;round_decimals&quot;: 2<br>}</pre><p>Notice the question states its window as half-open and names its rounding. That is deliberate. Most of what the audit classified as ambiguity comes from questions that leave boundaries and output format to the reader’s imagination. When you author the question from the declaration instead of from the data, you can simply say what you mean. The demonstration pack is public: <a href="https://github.com/rasinmuhammed/evalpacks">github.com/rasinmuhammed/evalpacks</a>. A 73,629-row orders table plus customers and products, 35 shipped questions, and a certificate recording the DuckDB version, the seed, the spec hash, and every per-question verification result. Re-checking it needs one dependency and about thirty seconds:</p><pre>pip install duckdb<br>python pack/verify.py<br># 35/35 verified exactly</pre><p>Edit one amount in one CSV and the script exits nonzero. That is the entire trust model. There is no leaderboard to take on faith, just a check anyone can run against the artifact itself.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JYAN84GasmJb7cvGlIb7lQ.png" /></figure><h3>The questions that did not survive</h3><p>The pack shipped 35 questions. The spec produced 40 candidates. The other five are my favorite part of the artifact. Declared rates collide with integer arithmetic. January was declared at a 2% fraud rate, and the January allocation came out to 4,824 orders. Two percent of 4,824 is 96.48 orders, and you cannot flag half an order. The achievable neighbors are 96 flagged rows, which is 1.99%, or 97, which is 2.01%. The verification gate measured 1.99% against a declared 2.00% and refused to ship the question.</p><p>Same story for four other months:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*octjEGjjJ3Zd4fZI7vRatg.png" /></figure><p>Seven other monthly rate anchors happened to be exactly achievable at four decimal places, and those shipped. All five failures are recorded in the manifest with their observed values, next to the full generation spec. I want to be precise about what this section is, because a reader could mistake it for an admission. A benchmark that publishes what it refused to include, with the measured reason, is making a stronger claim than one that ships everything: it is telling you the shipping bar is exact verification and showing you the bar rejecting things. The alternative, quietly shipping a question whose true answer is 1.99% with an answer key that says 2%, is precisely the failure mode the UIUC paper documented at scale.</p><h3>Regeneration, or why contamination stops mattering</h3><p>BIRD and Spider are in every frontier model’s training corpus by now. That is not an accusation, it is arithmetic: benchmark questions get indexed, discussed, and absorbed. Static benchmarks depreciate. An evalpack is a spec plus a seed, so it regenerates. Change the seed and you get a different database that satisfies the same declared answers. From my own run while writing this post:</p><pre>seed 20260710: November total = 750,000.00 over 73,629 orders<br>seed 99: November total = 750,000.00 over 73,629 orders<br>shared order_ids between the two databases: 544</pre><p>Two databases, essentially disjoint rows (the 544 shared IDs are random collisions in a ten-million-value ID space), identical declared answers, and the seed-99 pack re-verifies 35/35 on its own certificate. If you suspect a model has seen your eval data, you rotate the seed and the answer key survives. No annotation cycle, no re-adjudication. There is a second contamination defense hiding in plain sight: every entity in the data is plausible fiction. No model can answer “which month had the highest revenue” from memorized world knowledge about a retailer that does not exist. It has to query. For evaluation data, fictional values are not a compromise. They are the point.</p><h3>Rewards for RL, where wrong labels hurt twice</h3><p>If you train data agents with reinforcement learning, ground-truth noise is worse than in evaluation, because the policy optimizes toward the noise. The large synthetic SQL corpora used for training measure around 86% full correctness by their own published human evals, and RL on a 14%-wrong reward signal is a strange thing to spend GPUs on. A verified evalpack is, from an RL perspective, a reward function with a certificate: exact-match rewards whose correctness was checked by an independent engine before training started, generable at whatever scale and difficulty the spec dials in. Join depth, aggregation complexity, and temporal structure are parameters, not accidents of whichever database was available. I think this is where answer-key-first construction ends up mattering most, and it is the part I would most like to compare notes on with people building agent environments.</p><h3>Design choices a skeptic should ask about</h3><p>Three decisions define the current scope, and each was made with eyes open. Question families cover declared aggregates, per-period counts, rates, extrema, and referential integrity, not arbitrary SQL. That is what the spec language can declare today. The right way to extend coverage is to extend the declaration language, keeping the property that every shipped answer traces to a declared constraint, rather than annotating free-form questions and reintroducing the original disease. Natural-language ambiguity is reduced, not eliminated. Authoring questions from the declaration lets every question state its window, its rounding, and its tie-breaking rule explicitly, which removes the largest documented ambiguity sources. Wording is still wording, and I will not claim a theorem where I only have a discipline. And this complements human-authored benchmarks rather than replacing them. Real-world schema messiness, dialect coverage, the long tail of enterprise weirdness: existing benchmarks carry that, and correcting them, as the UIUC team is doing, stays essential. Answer-key-first construction owns the complementary territory: the error classes where annotation-after-the-fact structurally fails, at whatever scale you need, contamination-free by rotation.</p><h3>Try it</h3><p>Everything is MIT licensed. The pack, certificate, and verifier are at <a href="https://github.com/rasinmuhammed/evalpacks">github.com/rasinmuhammed/evalpacks</a>. Building your own takes one install and one command:</p><pre>pip install &#39;misata[evalpack]&#39;<br>misata evalpack - config your_schema.yaml -o your_pack - seed 42</pre><p>The declaration language, integrity proofs, and the rest of the engine are documented at <a href="https://misata.studio/docs">misata.studio/docs</a>, and there is a no-code version at <a href="https://misata.studio/">misata.studio</a> if you want to go from a plain-English description to a generated database without writing the schema by hand. If you build evaluations for data agents, or RL environments that contain databases, and this construction is either useful to you or broken in a way I have not noticed, I genuinely want to hear it. Open an issue on the repo. The fastest way to make the answer key trustworthy is to keep trying to break it.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9ab82ca2e7fa" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/half-the-answer-keys-in-text-to-sql-benchmarks-are-wrong-9ab82ca2e7fa">Half the Answer Keys in Text-to-SQL Benchmarks Are Wrong.</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Synthetic Data for Analytics: Demo Numbers You Can Prove]]></title>
            <link>https://medium.com/@rasinbinabdulla/synthetic-data-for-analytics-demo-numbers-you-can-prove-84ae358d6208?source=rss-db506407dbe0------2</link>
            <guid isPermaLink="false">https://medium.com/p/84ae358d6208</guid>
            <category><![CDATA[saas]]></category>
            <category><![CDATA[synthetic-data]]></category>
            <category><![CDATA[private-equity]]></category>
            <category><![CDATA[data-analytics]]></category>
            <category><![CDATA[business-intelligence]]></category>
            <dc:creator><![CDATA[Muhammed Rasin]]></dc:creator>
            <pubDate>Mon, 22 Jun 2026 07:26:17 GMT</pubDate>
            <atom:updated>2026-06-22T07:26:17.259Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PGSTkNscxOjB2W285Rcsqw.jpeg" /></figure><p><em>Analytics, BI, and private equity teams sell trust in numbers but build and demo on data they cannot verify. Synthetic data with a known answer, proven through your pipeline, fixes that.</em></p><p><strong>You sell trust in numbers. You demo on numbers you can’t trust.</strong></p><p><em>I set out to build something valuable for you: Synthetic data with a known answer, for analytics, BI, and private equity teams who live or die by the number on the screen.</em></p><p>Every analytics company is in the business of vouching for numbers. The dashboard will say churn is 3.2 percent. The report generated will say revenue grew 41 percent. The model says this account is likely to expand next quarter. People look at those numbers and then they do things: they fire a campaign, they pull a budget, they call a customer, they tell a board. The entire product, whatever the logo on it, is a promise that the number on the screen is the right number.</p><p>But when you walk into that first meeting with your dashboard filled with data, will you have a realistic demo that simulates the challenges, opportunities, and outcomes your target organisation actually faces? What happens when you’re pitching to a prospect whose data is confidential and inaccessible? How do you showcase the true value of your product when you have no meaningful story to tell and no realistic scenario to demonstrate its impact?</p><p>This is the quiet contradiction at the center of the analytics business, and once you see it you cannot unsee it. The data your developers use to build the dashboard is not the data the dashboard is for. The data you use to demo the product is borrowed, masked, scrubbed, or invented. The data you test the metric layer against has no known answer, so when the metric layer returns a number, you have no way to say whether it is correct. You can only say it ran.</p><p>There is a way out of this, and it is not the obvious one. It is not more realistic fake data. It is data whose answer you decided in advance. And that is what we are talking about.</p><p><strong>The data whose answer you already know</strong></p><p>Start with how data normally arrives. It shows up first, carrying its own truths, and your job is to discover them. You profile it, you chart it, you write tests that hope to catch it misbehaving. The data is the authority. You are the one asking it questions and hoping the answers are stable.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*W9grrmP3Kdo9MV5Vpnp_Ww.jpeg" /></figure><p>Now turn that around. Instead of receiving data and discovering its truth, you state a truth and let the data assemble itself to honor it. You say, monthly recurring revenue rises from 50,000 dollars in January to 200,000 by December. You say, churn is 7 percent in Q1 and falls to 4 percent after the new onboarding flow ships in Q3. You say, every customer’s lifetime value equals the sum of their actual invoices. Then a tool generates thousands of individual rows, customers and subscriptions and invoices and events, whose aggregates land on exactly those targets, with every foreign key resolving and every rollup reconciling.</p><p>This is what I have been building with <a href="https://misata.studio/">Misata</a>(https://github.com/rasinmuhammed/misata), an open source Python library, and I want to be honest that I have a stake in the idea. But the idea stands on its own whatever tool you reach for. The shift is small to describe and large in consequence: you write the answer key first, then generate the data around it. For an analytics company, that single property quietly solves a handful of problems the industry has always solved badly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*z3tGijWkDhyBGLV2fexMfg.jpeg" /></figure><p><strong>Problem one: a demo has to tell a story, and real data refuses to</strong></p><p>Watch a great product demo and notice what the data is doing. The churn chart dips right after the feature the rep is describing. The revenue line has a satisfying Black Friday spike. The cohort table shows retention improving down and to the right. The story the salesperson tells with their voice is the same story the data tells on the screen, and the alignment is what makes the demo land.</p><p>Now think about where that data came from. There are three usual options and all three are bad.</p><p>The first is anonymized production data from a friendly customer. It is real, which is the selling point, but real data is off message. It has the customer’s actual messiness, their seasonal quirks, their one enterprise account that distorts every average. It tells the customer’s story, not the story you need to tell. And it carries risk, because masked data has a way of remembering things you wish it had forgotten.</p><p>The second is a pile of random fake data from a mock-data generator. It is safe and it is fast, but it has no narrative. The churn line is flat noise. The revenue chart wanders. There is no spike, no dip, no improvement after Q3, because random data has no opinion about any of that. You cannot demo a turnaround on data that never turned.</p><p>The third is hand-built data, lovingly crafted by some engineer over days, which works beautifully for exactly one demo until a prospect asks to see a different segment and the whole illusion falls apart, because the cross-table relationships were never real and the moment you slice it differently the numbers stop reconciling.</p><p>Declared synthetic data removes the tradeoff. You describe the story you want the data to tell, and the rows arrange themselves to tell it, across every table, with the relationships intact so the prospect can slice it any way they like.</p><pre>import misata<br><br>from misata import from_dict_schema<br><br>schema = from_dict_schema({<br>  # the story: churn starts high and falls after the Q3 onboarding revamp<br>  &quot;__rate_curves__&quot;: [{<br>    &quot;table&quot;: &quot;subscriptions&quot;,<br>    &quot;column&quot;: &quot;churned&quot;,<br>    &quot;time_column&quot;: &quot;period&quot;,<br>    &quot;time_unit&quot;: &quot;month&quot;,<br>    &quot;true_value&quot;: True,<br>    &quot;rate_points&quot;: [<br>      {&quot;period&quot;: &quot;2025–01&quot;, &quot;rate&quot;: 0.071},<br>      {&quot;period&quot;: &quot;2025–06&quot;, &quot;rate&quot;: 0.068},<br>      {&quot;period&quot;: &quot;2025–09&quot;, &quot;rate&quot;: 0.042},<br>      {&quot;period&quot;: &quot;2025–12&quot;, &quot;rate&quot;: 0.038},<br>      ],<br>    }],<br><br>  # the story: revenue climbs through the year with a Q4 push<br>  &quot;__outcome_curves__&quot;: [{<br>    &quot;table&quot;: &quot;invoices&quot;,<br>    &quot;column&quot;: &quot;amount&quot;,<br>    &quot;time_column&quot;: &quot;issued_at&quot;,<br>    &quot;time_unit&quot;: &quot;month&quot;,<br>    &quot;value_mode&quot;: &quot;absolute&quot;,<br>    &quot;start_date&quot;: &quot;2025–01–01&quot;,<br>    &quot;avg_transaction_value&quot;: 180.0,<br>    &quot;curve_points&quot;: [<br>      {&quot;month&quot;: 1, &quot;target_value&quot;: 50_000.0},<br>      {&quot;month&quot;: 6, &quot;target_value&quot;: 110_000.0},<br>      {&quot;month&quot;: 12, &quot;target_value&quot;: 200_000.0},<br>      ],<br>    }],<br>   &quot;customers&quot;: {<br>    &quot;__rows__&quot;: 1200,<br>    &quot;id&quot;: {&quot;type&quot;: &quot;integer&quot;, &quot;primary_key&quot;: True},<br>    &quot;name&quot;: {&quot;type&quot;: &quot;string&quot;, &quot;text_type&quot;: &quot;name&quot;},<br>    &quot;plan&quot;: {&quot;type&quot;: &quot;categorical&quot;, &quot;choices&quot;: [&quot;starter&quot;, &quot;pro&quot;, &quot;enterprise&quot;]},<br>   },<br>   &quot;subscriptions&quot;: {<br>    &quot;__rows__&quot;: 1500,<br>    &quot;id&quot;: {&quot;type&quot;: &quot;integer&quot;, &quot;primary_key&quot;: True},<br>    &quot;customer_id&quot;: {&quot;type&quot;: &quot;integer&quot;, &quot;foreign_key&quot;: {&quot;table&quot;: &quot;customers&quot;, &quot;column&quot;: &quot;id&quot;}},<br>    &quot;period&quot;: {&quot;type&quot;: &quot;date&quot;},<br>    &quot;churned&quot;: {&quot;type&quot;: &quot;boolean&quot;},<br>   },<br>   &quot;invoices&quot;: {<br>    &quot;__rows__&quot;: 9000,<br>    &quot;id&quot;: {&quot;type&quot;: &quot;integer&quot;, &quot;primary_key&quot;: True},<br>    &quot;customer_id&quot;: {&quot;type&quot;: &quot;integer&quot;, &quot;foreign_key&quot;: {&quot;table&quot;: &quot;customers&quot;, &quot;column&quot;: &quot;id&quot;}},<br>    &quot;amount&quot;: {&quot;type&quot;: &quot;float&quot;, &quot;min&quot;: 20, &quot;max&quot;: 4000},<br>    &quot;issued_at&quot;: {&quot;type&quot;: &quot;date&quot;},<br>  },<br>}, seed=42)<br><br>tables = misata.generate_from_schema(schema)</pre><p>The churn line falls after Q3 because you said it should. The revenue line climbs to 200,000 in December because you said it should. Every invoice belongs to a customer who exists, every subscription too, and when the prospect asks to filter to enterprise accounts the numbers still hold, because the relationships were generated, not faked. You can ship the same demo to your whole sales team and every rep tells the same true story, because the story is baked into the data with a fixed seed and it regenerates identically forever.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pTwd1RL3fq14SX95hgS0xg.jpeg" /></figure><p><strong>Problem two: you cannot QA a metric you cannot check</strong></p><p>This is the one that should keep analytics teams up at night, and it is the one declared data solves most completely.</p><p>Your product computes a number. Monthly active users, net revenue retention, attribution by channel, whatever your product is famous for. You want to know it is correct. So you run it against some test data and it returns 3.2 percent. Is that right?</p><p>You have no idea. You do not know the true value in your test data, so the output is unfalsifiable. The test confirms the query executed. It cannot confirm the query is correct. A join that silently drops half the rows, a filter that double counts, a timezone bug that smears events across day boundaries, none of these will fail your test, because your test has nothing to compare against. It is checking that the machine produced a number, not that it produced the right number.</p><p>Now generate the test data with the answer decided in advance. You declared net revenue retention should be 112 percent for the 2024 cohort. You run your metric layer. If it returns 111.8 percent, your logic is sound: the cohort assignment held, the expansion and contraction netted correctly, the currency conversion behaved. If it returns 96 percent, something in your computation is wrong, and you caught it on a branch, on a laptop, before it ever shipped to a customer who would have made a decision on a false number.</p><pre>import pandas as pd<br><br># you declared December revenue must total exactly 200,000<br>inv = tables[&quot;invoices&quot;].copy()<br>inv[&quot;month&quot;] = pd.to_datetime(inv[&quot;issued_at&quot;]).dt.month<br><br>december = inv.loc[inv[&quot;month&quot;] == 12, &quot;amount&quot;].sum()<br><br># this is your product&#39;s metric layer, whatever it is, run against known-answer data<br>assert abs(december - 200_000) &lt; 0.01, f&quot;metric layer is wrong: got {december}&quot;</pre><p>That assertion is the difference between a test that says “it ran” and a test that says “it is correct.” It is the difference between shipping a metric you hope is right and shipping a metric you have proven is right against a ground truth you authored yourself. For a company whose product is trust in numbers, that is not a nice-to-have. It is the foundation under the thing you sell.</p><p>There is a deeper version of this for teams that care. Generate a known-answer dataset, run it through your full transformation stack, and assert that the declared metric survives every layer. Each generation can also emit a proof report covering referential integrity, row counts, constraints, and reproducibility, so the ground truth is not just in your head, it is an artifact you can store in CI and diff across releases.</p><pre>oracle = misata.build_oracle_report(tables, schema, seed=schema.seed)<br>assert oracle[&quot;passed&quot;] # FK integrity, row counts, constraints, determinism</pre><p>Wire that into your pipeline and your metric correctness becomes a thing that fails loudly on a pull request, instead of a thing a customer discovers in a board meeting.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_mzQZXpFU-4xWOlUAMTSSA.jpeg" /></figure><p><strong>Problem three: proving your product works on data you will never be allowed to see</strong></p><p>This is the version with the most money attached to it, and it is the one existing tools cannot touch.</p><p>Say your product does not just display numbers, it finds them. You sell a platform that surfaces the KPI that matters: the churn driver, the margin leak, the expansion signal nobody noticed. Or you are a consultancy, or a value-creation team inside a private equity firm, and the whole pitch is that you point your product at a portfolio company’s data and tell them the thing they did not know about their own business.</p><p>Now try to demo that. The prospect’s data is confidential. You will not see it before the deal, and often not after. So you reach for synthetic data to stand in for it, and here the difference between the two kinds of synthetic data becomes the difference between a demo that proves something and a demo that proves nothing.</p><p>If you use an imitation tool, the kind that learns from a sample and reproduces its shape, you get data that looks plausible and contains no answer. There is no planted churn driver, because nobody planted one. There is no margin leak to find, because the tool was copying distributions, not designing a business. So when you run it through your product, your product finds nothing in particular, because there was nothing in particular to find. You have not proven your product works. You have proven it runs.</p><p>Declared data is built the other way, and that is exactly what this demo needs. You design the business. You decide this company has EBITDA expansion concentrated in one segment, churn that hides a retention problem in a single cohort, headcount growing faster than revenue in one division. You plant the story, with real numbers, across customers and contracts and invoices and headcount, all reconciling. Then you point your product at it. If your product is as good as you say, it surfaces exactly the lever you planted, and the prospect watches it happen on data that is safe to share because it belongs to no one.</p><p>That is a demo that proves the product, not one that shows the interface.</p><p>But there is a harder bar, and it is the one that separates a real claim from a hopeful one. Does the scripted story survive the pipeline? The demo is worthless if the data only carries the right answer in the raw extract and loses it the moment it runs through Alteryx, dbt, a warehouse model, or whatever crunches it before your product ever sees it. Joins drop rows. Rollups re-aggregate. Time gets re-bucketed. If the planted KPI does not survive all of that, your product reads a number that no longer matches the story you meant to tell, and the demo dies in front of the buyer.</p><p>This is where outcome conformance earns its name, and where most generators quietly fail. A generator that simply writes the right total into a summary column has put the answer somewhere a pipeline will overwrite. Misata puts it where a pipeline cannot: every foreign key resolves, every parent summary reconciles to its children to the cent, every monthly curve is exact rather than approximate, so the answer was never sitting in a field waiting to be recomputed. It was distributed truthfully across the rows the whole time. When your pipeline groups, joins, and rolls the data back up, the planted answer is still standing on the other side. The story survives the crunch.</p><p>I will be straight about where this stands today, because this audience can smell aspiration from across the room. The primitives that make this possible, exact curves, per-period rates, cross-table rollups, referential integrity, and constraints, exist and work now, and a determined team can hand-script a portfolio scenario from them today. The turnkey layer, the one where you say “a PE-backed B2B SaaS company with EBITDA expansion and a failed Q3 pricing test” and get the whole reconciled world in a single line, is what we are building toward. The value is identified, the foundation is in place, and the convenience on top is the work in progress. I would rather tell you that than have you find out in a demo.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tAF2zRi_ltBGRM0a5x9WUQ.jpeg" /></figure><p><strong>Problem four: the cold start, when there is no data at all</strong></p><p>Every analytics company begins with nothing. No customer, no data, and yet a product to build and a demo to give. The dashboard has to exist before the first account signs. The model has to be trained before there is anything real to train it on. The investor deck needs a screenshot that looks like a thriving business using your tool, months before a thriving business does.</p><p>This is the moment most teams reach for whatever fake data they can assemble, and it shows. The early demos look thin because the data is thin. The first model is trained on noise because noise was all there was.</p><p>Declared synthetic data covers the cold start honestly. You decide what a healthy account looks like inside your product, the engagement curve, the feature adoption, the expansion pattern, and you generate a whole population of them. Because you set every dial, you know what every downstream number is supposed to say, which means the dashboard you build against it can be checked the same way you would check it in production. You are not just filling space. You are building against a known world.</p><p><strong>But I could just write a Python script</strong></p><p>Of course you could. Anyone who knows pandas can write a script that spits out a CSV with the numbers a pitch needs. The question was never whether you can write one. It is how many you are willing to write, and who keeps them alive after you do.</p><p>Think about what the second script costs. The first prospect was a SaaS company, so you wrote net revenue retention and cohorts and expansion revenue. The next is a fintech, and now you need fraud rates, chargebacks, account balances. After that comes an insurer who wants loss ratios and claims reserves, then a retailer who wants GMV, returns, and a Black Friday spike. Five sectors, five scripts, and not one of them is just numbers. Each one carries foreign keys that have to resolve, parent totals that have to reconcile with their child rows, dates that have to fall in a sane order, distributions that have to survive a skeptical reviewer, and a planted answer that has to live through your pipeline. That is not a snippet. It is a small unowned product, and it starts to rot the moment your real schema shifts underneath it.</p><p>Now hand those scripts to a solutions engineer who is brilliant at demos and was never hired to be a data engineer. Watch a column that got renamed three sprints ago quietly break the fintech demo live, in front of the buyer, because nobody kept the script in step with the product. The cost of the script was the afternoon you spent writing it. The cost of the scripts is everything after: the drift, the rebuilds, the single person who understands them, the demo that fails at the worst possible moment.</p><p>Declaring the scenario instead of coding it changes the unit of work. You stop writing generators and start writing specifications. A scenario becomes a short, readable file that states what must be true, the revenue curve, the churn rate, the relationships between the tables, and the engine turns it into rows. It carries across sectors because the vocabulary stays the same even when the story does not. It is reviewable, because a colleague can read the spec and see the claim. It is version controlled, because it is text. And it survives a schema change, because you fix one declaration instead of hunting through twenty scripts.</p><p>That is where the return lives. A solutions team that burned days per vertical building and rebuilding demo data spends minutes. A metric bug that would have reached a customer gets caught on a branch. A value-creation thesis gets rehearsed on a safe, reconciling replica before anyone touches the real company. None of that is measured in lines of code saved. It is measured in deals you can demo on the day they appear, in numbers you never had to walk back, in trust you did not have to rebuild.</p><p>I am building Misata in the open because the fastest way to make someone believe an idea is to let them hold it. But I will say what I actually think. The full version of this, the one a private equity firm or a serious analytics vendor needs, is larger than an open source library. It is a governed system: scenario libraries a whole team shares, approval and audit trails for what gets generated and shown, connectors into the pipelines you already run, and turnkey domain packs that turn one sentence into a reconciled world. The library is how you learn the value. A system like that is what the value is worth. And the ROI at that level is not an afternoon saved, it is every demo that fell apart, every number that had to be retracted, and every deal that stalled because you could not show the thing working, all of it simply stopping.</p><p><strong>Where this is the wrong tool, said plainly</strong></p><p>I would rather lose you here than oversell you, because the honest boundary is part of why this works.</p><p>Declared data will not find a bug you did not think to model. If your attribution logic mishandles a channel you never put in the schema, synthetic data built from a clean spec will be innocent of that case, and your product will meet it for the first time in production. Use generated data to prove your logic against known inputs, not to discover the unknown ones. For surfacing the weird real cases, nothing replaces eventually testing against real systems.</p><p>It does not replace an integration test against your actual ingestion connectors. It exercises your computation, not your plumbing.</p><p>And if your need is a privacy-safe statistical twin of a specific customer’s real dataset, that is a different job, the imitation job, where heavier generative synthesizers have spent years. Declared data starts from intent, not from a real table. Some teams want both, and that is fine. Just know which one you are reaching for and why.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YbXx_GEGDLHiTi21H-RFOw.jpeg" /></figure><p><strong>Early on purpose, building toward a solid core</strong></p><p>I will not pretend this is finished, because the people who would get the most from it can tell when a roadmap is being sold to them as if it were a product.</p><p>Here is the honest map. Misata today is strong where it matters most and earliest, in the guarantees. Exact outcome curves, per-period rates, cross-table rollups that reconcile, referential integrity, constraints that hold on every row, locale-accurate values, and the same seed reproducing the same world byte for byte. Those are the hard, unglamorous things that have to be exactly true for anything else to mean anything, and they work now.</p><p>It is early where the convenience lives. The one-line domain scenarios, the turnkey private capital and insurance and fintech worlds, the layer that reads a plain request and routes it to the right guarantees, those are being built, in that order, deliberately. Get the things that must be exactly true exactly true first. Earn the bigger promises only as the engine can carry them without bluffing.</p><p>And there is a reason it is open that goes past generosity. Every real use case teaches the core a scenario it should have made trivial. Every sector someone tries to pitch on it, every pipeline someone runs it through, surfaces the next thing worth guaranteeing. We identified the value early. The scenarios that prove out the rest of it will arrive faster from people using it in the wild than from me imagining them alone. If you see the gap I see, the most useful thing you can do is point the tool at your hardest case and tell me exactly where it bends.</p><p>What we currently need is real adoption, real feedbacks and support for improving the Natural Language to data workflow.</p><p><strong>The thing analytics companies are actually selling</strong></p><p>Come back to the dashboard that says 3.2 percent. The reason that number is hard to trust during development is not that your engineers are careless. It is that nothing in the test could have told them it was wrong. The data had no opinion about the right answer, so the test could only confirm the machine was running, not that it was right.</p><p>What changes, when you generate data on purpose, is that the data starts to carry an opinion. You write down what should be true. The rows arrange themselves around it. And then your product has to earn that truth back out the other end, through every join and filter and rollup, and hand you the number you started with. The test can finally fail for the right reasons. The demo can finally tell the story you mean. The cold start can finally be built against a world you understand.</p><p>For most software, that is a convenience. For an analytics company, it is the whole job. You are in the business of vouching for numbers. It is worth being able to vouch for the ones you build on first.</p><p>If you want to try it, Misata is `pip install misata`, the source and runnable examples are on <a href="https://github.com/rasinmuhammed/misata">GitHub</a>, and the exact-aggregate mechanism behind the revenue curves is written up in an <a href="https://arxiv.org/abs/2606.08736v1">arXiv preprint</a>. Whatever you use, give the data an answer before you ask the product to find it.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=84ae358d6208" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Everyone is suddenly behind an AI strategy and is blind about the AI Architecture]]></title>
            <link>https://medium.com/@rasinbinabdulla/everyone-is-suddenly-behind-an-ai-strategy-and-is-blind-about-the-ai-architecture-27327cf6eb73?source=rss-db506407dbe0------2</link>
            <guid isPermaLink="false">https://medium.com/p/27327cf6eb73</guid>
            <category><![CDATA[cost-optimization]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[enterprise-technology]]></category>
            <dc:creator><![CDATA[Muhammed Rasin]]></dc:creator>
            <pubDate>Fri, 15 May 2026 06:33:38 GMT</pubDate>
            <atom:updated>2026-05-15T06:33:38.881Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8W4-WfK8w6idbO90LYVK2A.jpeg" /></figure><p>On AI Token Wall, the architecture decision most enterprises got/are getting wrong, and about where all the money’s going.</p><p>One million BERT inference requests on a g4dn.xlarge GPU instance: $0.20. Ten million: $2.00. This is a number from an AWS machine learning benchmark.</p><p>Now here is the same workload, same task, routed through commercial APIs instead. The famous and trending AI models from Anthropic or OpenAI or Google would atleast cost a 1000$.</p><p>The task in all cases is identical. Intent classification. Categorical routing. Structured metadata extraction. The kind of thing an enterprise system does millions of times a day. You are paying, depending on which API you picked, anywhere from 26,000% to 358,000% more than you would pay to run it yourself. And AI experts in linkedin gets their posts trending talking about this but the truth is most companies ignore these facts.</p><h3>How we got here</h3><p>The first wave of enterprise AI integration was not about architecture. It was about access. The API was two lines of code. The demo looked extraordinary. The product shipped. Leadership approved. The invoice arrived later.</p><p>What never happened, in most cases, was someone sitting down to ask: what are we actually asking this model to do?</p><p>Large language models are remarkable for specific things. Zero-shot abstract reasoning. Complex multi-step synthesis. Open-ended generation where the output cannot be fully defined in advance. These are real, hard-won capabilities. They justify the cost when the problem actually demands them.</p><p>The problem is that most enterprise AI workloads are not those problems. They are classification problems. Routing problems. Extraction problems. Problems that have been solved, cheaply and reliably, by deterministic NLP tools for years. But because LLMs can also handle them, and because nobody audited the architecture after the initial integration, they are now consuming the majority of enterprise AI spend. Talking about NLP and machine learning models feels old-school to some AI leaders.</p><p>The waste is invisible inside the category. Both the $7,000 run and the $2 run show up as “AI infrastructure costs”. No alert fires. No dashboard flags it. The money just goes.</p><h3>The costs that never appear on the invoice</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Z6IqFZL4cbamDGN5niOLBg.jpeg" /></figure><p>Per-token billing is only the visible part of the problem.</p><p><strong>Latency.</strong> The fastest commercial models require 200 to 500 milliseconds just for Time-To-First-Token. A locally run spaCy NLP pipeline processes a sentence in approximately one millisecond. A properly provisioned BERT model holds P99 latency under 10ms. For any real-time customer-facing system, a consistent 300ms floor is not a performance detail. It is a degraded product. Users feel it even when they cannot name it.</p><p><strong>Rate limits.</strong> Commercial APIs enforce hard ceilings on tokens per minute and requests per minute. When systems hit natural traffic spikes or try to process historical data backlogs, they get HTTP 429 errors. Engineering teams then spend weeks building async queues, backoff logic, and circuit breakers just to work around limits they are already paying to access. That work has a cost. It never appears on the API bill.</p><p><strong>Prompt inflation.</strong> LLM APIs are stateless. Every call carries the full system prompt, conversational history, few-shot examples, and context payload. In a RAG pipeline, an application might consume 5,000 input tokens to produce a 5-token classification output. A 1,000-to-1 input-to-output ratio. On a task that never needed a generative model. This inflation is structural, continuous, and invisible until someone actually looks at the token logs.</p><p><strong>Data sovereignty.</strong> Every proprietary document, customer PII record, and internal financial metric sent to a third-party API is a compliance liability and a breach exposure. Regulated industries know this acutely. Self-hosted models eliminate the exposure entirely. No contract achieves equivalent protection.</p><p>These are not engineering concerns tucked away in a Jira board. Latency affects revenue per session. Rate limits cap throughput and therefore growth. Prompt inflation makes every feature built on top of the AI layer structurally more expensive. Data sovereignty creates legal drag that compounds over time. All of them grow as the AI footprint grows.</p><h3>The actual benchmark, because the numbers matter</h3><p>The AWS result is worth understanding in full because it is not a theoretical estimate. It is a documented, reproducible benchmark from AWS’s own infrastructure team.</p><p>A g4dn.xlarge instance, one NVIDIA T4 GPU, running BERT-base at sequence length 128, processes between 1,639 and 1,794 sentences per second with P99 latency under 10ms. At that rate, a single instance handles approximately 6.45 million documents per hour.</p><p>Ten million documents takes 1.55 hours. On-demand pricing for a g4dn.xlarge in US-East-1 is $0.526 per hour. Total cost: roughly $0.81. AWS’s own benchmark confirms the linear relationship directly: one million BERT inference requests cost exactly $0.20, so ten million cost exactly $2.00.</p><p>On Spot Instances, that hourly rate drops to approximately $0.16. Platforms like TrueFoundry let you run inference directly on your own EKS and EC2 clusters with zero service markup and automated scale-to-zero for idle resources.</p><p>The local option is not just cheaper. It is faster, by one to two orders of magnitude.</p><p>These numbers describe classification workloads specifically. For tasks that genuinely require a frontier model, the calculus is different. That is the whole point. The cost catastrophe happens when you do not separate the two categories and apply frontier model economics to everything.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LFuRXVF1cqSpQoeWRzCaKg.jpeg" /></figure><h3>The architecture that actually fixes this</h3><p>The solution is not abandoning LLMs. It is building a routing layer that stops them from handling work they were never meant to handle.</p><p>The framework that holds up in production is a tiered triage system. Every incoming request gets intercepted, assessed by complexity, and sent to the least expensive resource capable of handling it. The frontier model is the last door, not the first.</p><p><strong>Tier one: Deterministic heuristics.</strong> Regular expressions and static string matching. High-frequency, predictable queries like “what are your business hours” get resolved by a hash map in microseconds. No inference cost. No latency. This tier consistently handles more traffic than teams expect once they actually implement it.</p><p><strong>Tier two: Semantic routing.</strong> When exact matching fails, bi-encoder models project text into vector space and compare cosine similarity against predefined intent clusters. Libraries like semantic-router by Aurelio AI handle this using dense encoders from HuggingFace, sparse encoders like BM25 or TF-IDF, and multimodal encoders. The vector index lives in local memory or a fast database like Pinecone or Qdrant. The routing decision is pure math. No generation involved.</p><p><strong>Tier three: Local Small Language Models.</strong> Queries needing contextual understanding or light generative capability go to internally hosted SLMs: Llama-3–8B, Mistral, fine-tuned DistilBERT. Complex enough for domain-specific tasks, lean enough to run efficiently on internal infrastructure, and they never touch a public endpoint.</p><p><strong>Tier four: Frontier LLM.</strong> Multi-step reasoning. Zero-shot abstract inference. Complex synthesis where the output structure cannot be predetermined. In a well-built system, this tier handles somewhere between 5 and 15 percent of traffic. In most LLM-first architectures today, it handles everything.</p><p>The triage layer also changes the economics of RAG pipelines specifically. Before retrieval, a local spaCy NER pipeline strips conversational noise from the query so the embedding model focuses on what actually matters. At the routing stage, the semantic router can detect that a request for live account data should skip the vector database entirely and go straight to a SQL call. After retrieval, a local SLM can compress and rank the retrieved chunks before they reach the expensive model, cutting input token volume and reducing the attention degradation that long contexts cause.</p><p>None of this is exotic. BERT has been publicly available since 2018. spaCy is mature and well-documented. The architecture is not new. What is new is the urgency of implementing it, because the cost of not doing so is now showing up in quarterly earnings reports.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F_JcjTkCdNzZmuRVbXJjMw.jpeg" /></figure><h3>Four tasks that are draining your AI budget right now</h3><p>When you audit what frontier models are being asked to do across a typical enterprise system, four task types consistently account for most of the unnecessary spend.</p><p><strong>PII masking.</strong> The common approach sends raw documents to a frontier model and asks it to return sanitised text. This is expensive, slow, and contains an irony that should bother every CISO on the call: you are trying to protect sensitive data by first transmitting it, completely unmasked, to a third-party server over the internet.</p><p>Microsoft Presidio solves this locally. An Analyzer applies deterministic regex and checksum logic for structured PII like emails, phone numbers, and financial codes. It integrates lightweight spaCy or Stanza NER for contextual entities, so it correctly distinguishes “Washington” the surname from “Washington” the location. Deployed as proxy middleware through something like LiteLLM, Presidio sanitises payloads at the infrastructure edge before they hit any external network. The latency overhead is under 5%. The per-token cost beyond basic container infrastructure is effectively zero.</p><p><strong>Intent classification and chatbot routing.</strong> Whether a user wants to talk to billing or technical support is a categorisation problem. It is not a generative problem. Using a model capable of producing complex code to answer that question is not a feature. It is a misallocation.</p><p>Compromise.js, a lightweight rule-based JavaScript NLP library, processes sentences and extracts intent at approximately one millisecond per sentence. Its entire lexicon compresses to roughly 35 kilobytes. It runs in the browser or server-side via Node.js, needs no network hop, and carries no per-query cost beyond the compute already running. For queries that need deeper semantic matching, a local embedding model with cosine similarity routing gets there in milliseconds without touching an external API.</p><p><strong>Financial sentiment analysis.</strong> Quantitative finance teams processing thousands of market news articles, earnings transcripts, and analyst reports daily for entity-level sentiment face a real dilemma with frontier APIs. These models perform well on Bloomberg datasets in zero-shot settings. But rate limits, token generation constraints, and cost structures make high-frequency large-scale ingestion economically unviable for real-time algorithmic trading pipelines.</p><p>Domain-specific open-source models, FinBERT, DistilFinRoBERTa, FinBERT-Tone, run locally on GPU instances. They process thousands of documents per minute without API-imposed ceilings. They produce deterministic structured outputs ready for downstream quantitative systems. The cost difference relative to frontier APIs is multiple orders of magnitude.</p><p><strong>E-commerce taxonomy and style tagging.</strong> Passing millions of product SKUs through a premium API to extract colors, materials, dimensions, and aesthetic categories is expensive at any meaningful catalog scale. Wayfair’s engineering team has documented this pattern in their own infrastructure work. The hybrid approach is not complicated: spaCy NER handles deterministic extraction of structured attributes from product titles. Fine-tuned open-source models via vLLM handle complex stylistic inference locally on GPU clusters. The frontier model processes what neither tier can handle. Not the entire catalog.</p><h3>AT&amp;T ran into the wall and built a door through it</h3><p>AT&amp;T processes 5 million customer support interactions per year. The initial prototype used GPT-4 to transcribe, categorise, and extract insights across 80 distinct categories from recorded call center data. The outputs were solid. The system identified areas to benefit 50,000 customers annually.</p><p>Then the architecture hit its ceiling.</p><p>The data volume generated exorbitant compounding API costs. Processing speed was fundamentally unscalable: approximately 15 hours of continuous API batch processing just to categorise a single day of call center interactions. Routing proprietary customer telecommunications data through an external vendor created real privacy and regulatory exposure.</p><p>AT&amp;T rebuilt around what they called a Master-Agent strategy. They distilled the capabilities of the large frontier models into fine-tuned Small Language Models: Mistral, Microsoft’s Phi series, H2O Danube. These SLMs were trained on AT&amp;T’s own internal data and hosted entirely on internal infrastructure, no public internet involved.</p><p>The outcomes: monthly AI operating costs cut by 90%. Latency and response speeds improved by 70%. Token throughput capacity tripled compared to the API architecture.</p><p>Andy Markus, AT&amp;T’s Chief Data and AI Officer, said publicly that the domain-specific small models proved just as accurate as the frontier LLMs for their specific tasks. The trillion-parameter model was not wrong. It was unnecessary. AT&amp;T had been paying for capabilities the task never required, at scale, continuously, until they stopped and looked at what was actually happening.</p><p>This is the thing that makes the AT&amp;T case important. The 90% cost reduction is not a miracle of optimisation. It is what happens when you match tool to task. The gap between frontier model costs and local compute costs is so wide that correcting the mismatch produces results that look dramatic. They are not dramatic. They are just what normal engineering looks like when applied to the right question.</p><h3>Klarna: what happens when there is no triage layer at all</h3><p>In early 2024, Klarna launched an AI customer service assistant built directly with OpenAI. The initial numbers were extraordinary. The system handled 2.3 million customer chats in its first month, the equivalent workload of 700 full-time human agents. A 67% automation rate. Average resolution times down from 11 minutes to under 2 minutes. Projected to deliver nearly $40 million in annual cost avoidance.</p><p>The press treated it as proof that the human operational layer in enterprise customer service was over.</p><p>By early 2025, customer satisfaction scores had dropped 22%. Klarna posted a $99 million net loss in Q1. The CEO publicly acknowledged that chasing raw efficiency had gutted service quality. He described the outputs as “generic” and incapable of handling scenarios that required specific, deterministic judgment. The company was forced into a rapid reversal: rehiring staff, reassigning internal engineers to stabilise the customer experience.</p><p>The failure was not AI. The failure was architecture. A monolithic generative model deployed as a black box across an entire operational layer with no routing logic, no deterministic guardrails, no task-specific handling for the queries that needed something different. The system was optimised for volume and produced high-speed, high-confidence mediocrity on everything that required nuance.</p><p>For a CFO, the Klarna trajectory is a useful calculation. A projected $40 million in savings became a $99 million quarterly loss driven partly by customer churn and brand damage. Short-term headcount reduction masked the longer-term cost of eroding customer trust.</p><p>For a CTO, the lesson is architectural. Removing deterministic systems and replacing them with a single generative endpoint eliminates every safeguard those systems provided: predictable outputs, explainable decisions, controlled escalation paths, domain-specific handling.</p><p>For a CEO, the Klarna story is a warning about what gets measured and when. Automation rate and resolution time looked good in month one. Customer satisfaction, which is what actually drove long-term revenue, was not in the initial reporting frame. The announcement came before the architecture had been stress-tested against the full distribution of real customer interactions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OSZ_OrWno2BWbJ6VDD6fnA.jpeg" /></figure><h3>What Stripe did instead</h3><p>Stripe moved away from manual feature engineering and rule-based fraud detection toward internally trained models built on tens of billions of their own transaction tokens. By treating financial transactions as a sequence of tokens, their internal models learned to identify latent patterns that manual rules never captured: clustering behaviors from specific card issuers, linked email address patterns, behavioral sequences that only emerge from scale.</p><p>Attack detection rates for large users moved from 59% to 97%.</p><p>Stripe did not route transaction data to a general-purpose frontier API. They built domain-specific models on domain-specific data, hosted internally, optimised for their specific problem. The performance came from the specialisation. The feasibility came from the internal hosting. You cannot do either of those things with a third-party API dependency at the center of your architecture.</p><h3>Three decisions that are creating and sustaining this problem</h3><p>When I look at where the Token Wall actually originates in organizations, it tends to come from the same three places.</p><p><strong>Approving AI spend based on capability rather than fit.</strong> The evaluation frame most organizations use when choosing AI infrastructure is: can this model do the task? The correct frame is: is this the least expensive model that can do this task at the required quality level? Those two questions produce very different procurement decisions. The first one leads to frontier models handling classification workloads. The second one leads to BERT.</p><p><strong>Treating AI integration as a product decision rather than an architectural one.</strong> LLM API integration tends to get approved at the product level because the demos are compelling and the vendor relationships are clean. The architectural questions, how traffic gets routed, what percentage of requests actually require generative capability, how costs scale with volume, are engineering concerns that enter the conversation late, usually after the patterns are already embedded.</p><p><strong>Measuring AI ROI on output quality without measuring cost per unit of output.</strong> Automation rate, resolution time, accuracy on test sets. These are necessary metrics and they are not sufficient. A system that achieves 95% accuracy at $0.02 per query is not the same business outcome as a system that achieves 95% accuracy at $0.000002 per query, even if the first number looks fine in isolation. At enterprise scale, the denominator is everything.</p><h3>What this actually looks like going forward</h3><p>The first wave of enterprise AI was about access. Can we do this at all?</p><p>The second wave, which is where we are right now, is about architecture. Are we doing this the right way?</p><p>For most organizations, the honest answer is no. Not because the technology is wrong but because the assumptions from the first wave, use the most capable model available, integrate fast, audit later, have been carried forward unchallenged into a much more expensive operating environment.</p><p>The companies building systems that will hold up are the ones asking, before every integration decision: what is the actual complexity of this task, and what is the least expensive tool that can handle it at the required quality level?</p><p>They are building triage layers before they build features. They are auditing token logs to understand what percentage of their frontier model traffic is genuinely doing frontier-model-level work. They are fine-tuning smaller models on their own data for the tasks that repeat at volume. And they are treating the frontier LLM as a precision instrument for hard problems, not as the default answer to every text processing question.</p><p>The AT&amp;T result, 90% reduction in operating costs with improved latency and higher throughput, is not an exceptional outcome. It is what correct architecture produces. The gap between frontier API costs and local compute costs is large enough that closing the mismatch looks dramatic from the outside. It is not dramatic. It is just the natural result of asking, for the first time, whether the tool actually fits the task.</p><p>The Token Wall is not fundamentally a technical problem. The technical solutions have existed for years. It is a decision problem. And the decision it is waiting for is someone in the room finally asking why a classification task is going to a trillion-parameter model in the first place.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=27327cf6eb73" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building the Best Synthetic Data Generator in Python for 2026: Why I Am Building Misata and How to…]]></title>
            <link>https://pub.towardsai.net/building-the-best-synthetic-data-generator-in-python-for-2026-why-i-am-building-misata-and-how-to-3cedd65242ed?source=rss-db506407dbe0------2</link>
            <guid isPermaLink="false">https://medium.com/p/3cedd65242ed</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[synthetic-data]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Muhammed Rasin]]></dc:creator>
            <pubDate>Sun, 12 Apr 2026 22:01:01 GMT</pubDate>
            <atom:updated>2026-04-12T22:01:01.440Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yzGsCQXJeeljBpiQA5hnCA.png" /></figure><h3>Building the Best Synthetic Data Generator in Python for 2026: Why I Am Building Misata and How to Use It</h3><h4>An honest attempt to break into Synthetic Data Generation in the LLM Era</h4><p>If you’ve ever needed realistic fake data for a dev database, or a product demo, or for testing your analytics dashboard for a niche you still doesn’t have data for; you might already know how painful this is.</p><p>You either spend days hand-crafting a script, wrestle with libraries that need real data to generate fake data, or ask an LLM to produce a CSV and watch it hallucinate 50 rows before giving up.</p><p>I know this pain personally. My first project at my first job was writing a Python script to generate synthetic data for a real use case. I worked on that script for two months. Every single day. Hand-coding for loops, managing foreign key consistency, attending stand-up calls that would change the needs and requirements almost every week. Two months. One script. I was just into a job and that experience turned into a trauma.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/831/1*MY8YCPuTpBaSBALRyhkESg.png" /></figure><p>The problem wasn’t hard. It was just <em>unsolved</em>.</p><p>That’s why I am building <strong>Misata</strong> an open-source Python library for generating realistic, multi-table synthetic datasets from a plain English description. No real data required. No ML training. One sentence in, thousands of rows out.</p><p>This post covers what Misata does, how to use it, why it’s different from everything else out there, and where I’m taking it.</p><h3>The state of synthetic data generation in 2026</h3><p>Before getting into how Misata works, it’s worth being honest about the current landscape: because the tools that exist are genuinely good at <em>some</em> things.</p><p><strong>Faker</strong> is the go-to for generating individual fake values. Names, emails, phone numbers, addresses; it handles all of that well. But it has no concept of a <em>dataset</em>. It generates one row’s worth of values at a time, with no awareness of relationships between tables. If you want 20,000 transactions that reference valid customer IDs with a 2% fraud rate and a log-normal amount distribution, you’re writing all of that yourself.</p><p><strong>SDV (Synthetic Data Vault)</strong> is impressive engineering. It learns the statistical structure of your real data and generates synthetic copies that preserve it. The problem: you need real data to train it. When the whole point is that you <em>don’t have</em> real data; which is a very common situation when you’re prototyping, building a dev environment, or training a model on GDPR-restricted data; SDV can’t help you.</p><p><strong>LLMs</strong> feel like the obvious answer until you try them. Ask any major model to generate a 10,000-row dataset and you’ll hit context limits, truncation, or hallucinated CSV structure. Even when they work, you’re burning tokens at a rate that makes large-scale generation economically absurd. And critically, LLMs have no mechanism to guarantee that your monthly revenue totals hit $320k in June; they’ll get close-ish at best.</p><p>The gap: there is no tool that takes your <em>intent</em> as input and produces statistically calibrated, relationally consistent data as output. That’s what Misata is built to fill.</p><h3>Installing Misata</h3><pre>pip install misata pandas numpy</pre><p>That’s it. No ML dependencies, no database required to get started, no API key for the core functionality.</p><h3>The basics: generating your first dataset</h3><p>The simplest way to use Misata is the generate() function. Pass it a plain English description and a random seed:</p><pre>import misata<br>tables = misata.generate(<br>    &quot;A SaaS company with 2000 users.&quot;,<br>    seed=42,<br>)<br>print(tables.keys())<br># dict_keys([&#39;users&#39;, &#39;subscriptions&#39;])<br>print(tables[&#39;subscriptions&#39;].head())</pre><p>You get back a dictionary of Pandas DataFrames, one per table, with realistic column names, types, and values. But don’t use these simple prompts though as it invalidates our whole point.</p><p>The seed parameter makes generation fully deterministic. Same description, same seed, same dataset every time. This matters for reproducible research, stable CI test fixtures, and sharing datasets with collaborators.</p><h3>Pinning business targets</h3><p>Here’s where Misata does something no other library does: it lets you describe specific business dynamics, and the generated data will satisfy them exactly.</p><pre>tables = misata.generate(<br>    &quot;A SaaS company with 2000 users. &quot;<br>    &quot;MRR rises from 80k in January to 320k in June, &quot;<br>    &quot;drops to 180k in August due to churn, &quot;<br>    &quot;then recovers to 400k in December.&quot;,<br>    seed=42,<br>)</pre><p>When you sum the monthly MRR from the generated rows:</p><pre>Jan    $80,000   ✓<br>Feb   $128,000   ✓<br>Mar   $176,000   ✓<br>Apr   $224,000   ✓<br>May   $272,000   ✓<br>Jun   $320,000   ✓<br>Jul   $250,000   ✓<br>Aug   $180,000   ✓   &lt;- churn dip, as described<br>Sep   $235,000   ✓<br>Oct   $290,000   ✓<br>Nov   $345,000   ✓<br>Dec   $400,000   ✓</pre><p>Not approximate. Exact. The individual rows still follow a log-normal distribution (median MRR $126, mean $150, p90 $291); because that’s what real SaaS revenue looks like at the row level. But the monthly totals are treated as hard constraints. Misata pins the aggregates first, then samples rows that satisfy them.</p><p>This is the key architectural difference from tools that just try to produce data that <em>looks</em> reasonable. GROUP BY on Misata data gives you what you asked for.</p><h3>Calibrated domain distributions</h3><p>The part that cost me two months to research and hand-code in my first job is what Misata ships pre-calibrated for. Seven domains, each with distribution priors tuned to real-world statistics.</p><h3>Fintech</h3><pre>tables = misata.generate(<br>    &quot;A fintech company with 2000 customers and banking transactions.&quot;,<br>    seed=42,<br>)<br>transactions = tables[&quot;transactions&quot;]<br>print(f&quot;Fraud rate: {transactions[&#39;is_fraud&#39;].mean() * 100:.2f}%&quot;)<br># Fraud rate: 2.00%</pre><p>The real-world baseline for card fraud is ~2%. That’s what you get; not a random number, a calibrated one. 400 fraudulent transactions out of 20,000, distributed across accounts in a way that reflects how fraud actually clusters.</p><p>Credit scores follow the actual US distribution:</p><pre>mean:   679   (real US average: 680–720)<br>std:     80   (real range: 70–90)<br>min:    328<br>max:    850</pre><p>Account balances are log-normal because real bank balances are:</p><pre>median     $1,976<br>mean       $6,128<br>p90       $14,260<br>p99       $62,565</pre><p>Why does this matter? If you train a fraud detection model on uniformly distributed balances, you’re teaching it that everyone has the same balance. Real fraud disproportionately targets high-value accounts. A model trained on flat data will miss exactly the cases you most need it to catch.</p><h3>Healthcare</h3><pre>tables = misata.generate(<br>    &quot;A hospital with 500 patients and doctors.&quot;,<br>    seed=42,<br>)</pre><p>Blood type frequencies match actual ABO/Rh epidemiology:</p><pre>Blood type    Generated    Real-world<br>O+               37.9%        38.0%   ✓<br>A+               33.9%        34.0%   ✓<br>B+                9.6%         9.0%   ✓<br>AB+               3.0%         3.0%   ✓<br>O-                6.5%         7.0%   ✓<br>A-                6.1%         6.0%   ✓<br>B-                2.0%         2.0%   ✓<br>AB-               0.9%         1.0%   ✓</pre><p>All eight types within 0.6% of real frequencies. Patient ages center on 45 with standard deviation 18, matching a chronic-care population. Nobody configured this; it’s what the healthcare domain prior knows.</p><h3>Ecommerce</h3><pre>schema = misata.parse(<br>    &quot;An ecommerce store with 5000 customers and orders. &quot;<br>    &quot;Revenue grows from 100k in January to 300k in November, &quot;<br>    &quot;then 350k in December.&quot;,<br>    rows=5000,<br>)<br>tables = misata.generate_from_schema(schema)</pre><p>Product categories follow Zipf’s law, because real shopping behavior does:</p><pre>electronics      47.1%<br>clothing         20.0%<br>home &amp; garden    12.3%<br>sports            8.7%<br>books             6.5%<br>beauty            5.5%</pre><p>Uniform would give ~17% per category. Real shopping isn’t uniform. Neither is Misata.</p><p>Order statuses at realistic rates:</p><pre>completed    71.5%<br>shipped      12.4%<br>pending       8.2%<br>returned      5.0%<br>cancelled     3.0%</pre><p>Real e-commerce return rates run 8–10%. Recommendation systems, inventory models, and return prediction pipelines all need this to be right to produce meaningful results.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TYlB2vjKEqtGy7PmKcM9Pg.png" /></figure><h3>Multi-table generation with guaranteed referential integrity</h3><p>Misata generates entire relational schemas, not just single tables. And it guarantees that foreign keys are always valid; not by checking after the fact, but by generating in the correct dependency order.</p><pre>tables = misata.generate(<br>    &quot;A fintech company with 2000 customers and banking transactions.&quot;,<br>    seed=42,<br>)<br>customers    = tables[&quot;customers&quot;]     # 2,000 rows<br>accounts     = tables[&quot;accounts&quot;]      # 2,600 rows<br>transactions = tables[&quot;transactions&quot;]  # 20,000 rows</pre><pre># Verify FK integrity<br>orphan_accounts = (~accounts[&quot;customer_id&quot;].isin(customers[&quot;customer_id&quot;])).sum()<br>orphan_txns     = (~transactions[&quot;account_id&quot;].isin(accounts[&quot;account_id&quot;])).sum()</pre><pre>print(orphan_accounts)  # 0<br>print(orphan_txns)      # 0</pre><p>Tables generate in topological dependency order. Parents first, children sample from the completed parent pool. Orphan keys are architecturally impossible.</p><p>This matters because referential integrity errors in test data produce silent false negatives. Your pipeline looks like it handles JOINs correctly, until it meets real data.</p><h3>The two-step flow: inspect before you generate</h3><p>For more control, separate the parsing step from the generation step:</p><pre>schema = misata.parse(&quot;A hospital with 500 patients and doctors.&quot;)<br>print(schema.summary())</pre><pre>Schema: Healthcare Dataset<br>Domain: healthcare<br>Tables (3)<br>  doctors         25 rows    [doctor_id, first_name, last_name, specialty, years_experience]<br>  patients       500 rows    [patient_id, name, age, gender, blood_type, registered_at]<br>  appointments  1500 rows    [appointment_id, patient_id, doctor_id, type, duration_minutes]</pre><pre>Relationships (2)<br>  patients.patient_id  -&gt; appointments.patient_id<br>  doctors.doctor_id    -&gt; appointments.doctor_id</pre><p>You can inspect the schema, adjust row counts, add columns, or change relationship cardinality before committing to generation. The schema object is also serializable; useful for teams where a data engineer defines the schema and developers generate against it, versioned in git.</p><h3>Seeding databases directly</h3><p>Generated DataFrames go directly into any database via SQLAlchemy:</p><pre>from misata import seed_database<br><br>tables = misata.generate(&quot;A SaaS company with 1000 users.&quot;, seed=42)<br>report = seed_database(tables, &quot;postgresql://user:pass@localhost/mydb&quot;, create=True)<br>print(report.total_rows)  # 12,400</pre><p>Or from the CLI: useful in Makefile targets and docker-compose setups:</p><pre># PostgreSQL<br>misata generate \<br>  --story &quot;A SaaS company with 1000 users&quot; \<br>  --db-url postgresql://user:pass@localhost/mydb \<br>  --db-create --db-truncate</pre><pre># SQLite (local dev)<br>misata generate \<br>  --story &quot;A SaaS company with 1000 users&quot; \<br>  --db-url sqlite:///./dev.db \<br>  --db-create --db-truncate</pre><p>A new developer runs make seed-db and has a working, realistic dataset in their environment in under 10 seconds.</p><h3>LLM-powered generation for custom domains</h3><p>The rule-based parser covers seven domains. For anything outside that; B2B marketplace vendor tiers, clinical trial schemas, commodity trading systems — the LLM backend handles schema inference:</p><pre>from misata import LLMSchemaGenerator</pre><pre>gen = LLMSchemaGenerator(provider=&quot;groq&quot;)  # or openai, ollama<br>schema = gen.generate_from_story(<br>    &quot;A B2B marketplace with vendor tiers, SLA contracts, and quarterly invoices&quot;<br>)<br>tables = misata.generate_from_schema(schema)</pre><p>Requires GROQ_API_KEY or OPENAI_API_KEY. The LLM infers schema, column types, and row count ratios from your description; you get back the same DataFrames as the rule-based path.</p><h3>How Misata compares to Faker and SDV</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6RW4vAAyln-S0w9M9rQCBw.png" /></figure><p>The core distinction: SDV is a synthetic data <em>replication</em> tool, it mirrors real data. Misata is a synthetic data <em>generation</em> tool, it generates from intent. They solve different problems. If you have real data and want a privacy-safe copy, use SDV. If you’re starting from scratch, use Misata. There are various other tools getting traction in the industry too, but when you truely know the aspiration behind what Misata should do, then this would be a novel one.</p><h3>Common use cases</h3><p><strong>Seeding development and staging environments.</strong> Every developer joining a team hits the same wall: the dev database is empty. Misata seeds it with realistic, relationally consistent data in seconds.</p><p><strong>Product demos.</strong> Sales teams demoing analytics dashboards need data that tells a plausible story and not random numbers. Misata generates internally consistent business narratives on demand.</p><p><strong>ETL and pipeline testing.</strong> Calibrated distributions naturally produce the p99 outlier values that expose pipeline brittleness, the cases hand-crafted test data systematically misses.</p><p><strong>Reproducible research benchmarks.</strong> Share a description and a seed. Anyone can reproduce your exact dataset with a single Python call.</p><h3>What’s coming</h3><p>Misata is under active development. The current version is the foundation, the goal is to be the most sophisticated intent-driven synthetic data tool that exists.</p><p>The roadmap includes broader domain coverage, time-series generation with configurable seasonality and anomalies, natural language column constraints (“30% of enterprise accounts have balances above $50k”), a schema DSL for version-controlled dataset definitions, and better LLM schema inference for unusual entity types.</p><p>If you’ve ever spent two months on a synthetic data script, you know exactly what this is trying to solve. And if you haven’t, now you don’t have to.</p><h3>Get started</h3><pre>pip install misata pandas numpy</pre><p>The fastest path is the Colab notebook, no install required, runs in under a minute:</p><p><a href="https://colab.research.google.com/github/rasinmuhammed/misata/blob/main/notebooks/quickstart.ipynb">Open the quickstart notebook →</a></p><p>Or run the included examples locally:</p><pre>python examples/saas_revenue_curve.py<br>python examples/fintech_fraud_detection.py<br>python examples/healthcare_multi_table.py<br>python examples/ecommerce_seasonal.py</pre><p>All produce full verified output in under 3 seconds.</p><p><strong>GitHub:</strong> <a href="https://github.com/rasinmuhammed/misata">github.com/rasinmuhammed/misata</a> <br><strong>PyPI:</strong> <a href="https://pypi.org/project/misata/">pypi.org/project/misata</a> <br><strong>Docs:</strong> <a href="https://github.com/rasinmuhammed/misata/blob/main/QUICKSTART.md">QUICKSTART.md</a></p><p><em>MIT licensed. Contributions, issues, and feedback are very welcome, this is early days and the direction will be shaped by what people actually need.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3cedd65242ed" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/building-the-best-synthetic-data-generator-in-python-for-2026-why-i-am-building-misata-and-how-to-3cedd65242ed">Building the Best Synthetic Data Generator in Python for 2026: Why I Am Building Misata and How to…</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Not everyone must ‘Pivot to AI’ to save their jobs]]></title>
            <link>https://medium.com/@rasinbinabdulla/not-everyone-must-pivot-to-ai-to-save-their-jobs-3edcc50953e4?source=rss-db506407dbe0------2</link>
            <guid isPermaLink="false">https://medium.com/p/3edcc50953e4</guid>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[writing]]></category>
            <category><![CDATA[future-of-work]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[creativity]]></category>
            <dc:creator><![CDATA[Muhammed Rasin]]></dc:creator>
            <pubDate>Fri, 03 Apr 2026 08:51:10 GMT</pubDate>
            <atom:updated>2026-04-03T08:51:10.877Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2xrFtVS-JMc0zcjnSt33LQ.png" /></figure><p><em>On FOMO, the quiet death of original voice, and why staying stubbornly human might be the best career move you make this decade.</em></p><p>You can tell now. Within a paragraph, sometimes within a sentence. There’s the slightly too smooth transition. The bullet points that arrive uninvited. The way every LinkedIn post seems to open with “In today’s rapidly evolving landscape”. The em dashes that show up not because a writer paused mid-thought, but because a model learned that em dashes perform intelligence. We have arrived, with uncomfortable speed, at a world where most written content reads like it came from the same invisible hand.</p><p>That hand is not malicious. It is efficient. And it is, I want to argue along with the world, slowly killing something we have not yet found the language to mourn.</p><p>I feel it most when I scroll past a Qatar Airways ad on Instagram. Glossy, too artificial, completely soulless. There is nothing wrong with it in any measurable sense. Every pixel is where it should be. But the thing that used to exist in good advertising, that small tension between a human art director and a human copywriter, the argument that produces an unexpected metaphor, the slight imperfection that says a real person cared enough to struggle with this, it’s gone. Replaced by coherence without character.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IGHTAzDuNfYLfBUt0cpSTg.png" /></figure><h3>The numbers, because the numbers are interesting</h3><p>A 2026 study by researcher Joe Youngblood surveying over 4,000 Americans found that people are getting genuinely good at spotting the machine. A separate Hookline poll of 1,000 consumers found that 82.1% say they notice AI-written content regularly, with adults aged 22 to 34 hitting 88.4% detection rates. And 40% of those consumers say they actively penalise brands they catch using AI for front-facing creative work.</p><p>74% of new web pages contain AI-generated content. 82% of U.S. consumers say they notice AI content. 40% penalise brands they discover using AI creatively. Top 10% of human creatives outperform every AI tested.</p><p>In January 2026, a research team at the Université de Montréal, working alongside Yoshua Bengio, one of the people who built the foundations of modern deep learning, published what is probably the most rigorous direct comparison of human and AI creativity done so far. They tested over 100,000 people against nine leading AI systems. The headline finding was that AI can now beat the average human on certain creativity tests. But the more important finding, the one that barely made the news cycle, is that the most creative half of human participants outperformed every single AI system tested. The top 10% left the models behind by a wide margin. And there is, so far, no clear path for the machines to close that gap.</p><p>The machines have reached average. They have not reached exceptional.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_zqQl9kKjv-zeZz-PkrfSg.png" /></figure><h3>This has happened before</h3><p>At some point in the early 2010s, the design world collectively decided that ornament was sin. Flat design arrived, Swiss grids became gospel, serifs were abandoned, and every interface in the world converged on the same white space and Helvetica. It was clean. It was efficient. And in that efficiency, a great deal of joy was quietly disposed of. The hand-lettered sign above the old apothecary. The decorative border on the book spine. The pleasure of intricacy as a signal that a human being cared enough to spend time on something nobody required them to spend time on.</p><p>What AI is doing to writing is the same thing. A second great flattening, not of visual complexity this time but of voice. The idiosyncrasy that signals a real mind behind the text. The willingness to write a sentence that doesn’t quite land correctly, because it is the honest sentence. The three words that don’t belong together and somehow say the thing more precisely than anything grammatically correct could manage.</p><p>An AI will never write something that surprises itself. That is not a criticism. It is a description of how the architecture works. It maximises for coherence. It minimises surprise. Consumed at scale, across every LinkedIn post and marketing email and company blog, the result is a grey verbal fog. Technically adequate. Experientially numbing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/864/1*L1xrtcl0Xl0XPt9Jd6QQXQ.png" /></figure><h3>The FOMO that is eating people alive</h3><p>I want to say something directly, because I think this particular anxiety is causing real damage to a generation of creative people. We need to ground all the people who spread this FOMO on online platforms like LinkedIn and instagram. As far as I noticed, the most dangerous are the institutions who wants to spread this narrative to sell their courses.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*29rXeC1NAN2SpNYjWiDe0w.png" /><figcaption>Perhaps, the most hated guy in all of Youtube.</figcaption></figure><p>The dominant story of 2025 and 2026 has been: learn AI or be left behind. It has the force of received wisdom now. Attend any conference, open any newsletter from a venture capitalist, talk to any recruiter, and you will hear some version of that sentence. Everyone must pivot. Everyone must upskill. Everyone must build something with an LLM before breakfast or they are already obsolete.</p><p>Some of this is true and useful. Knowing how to work with these tools is genuinely valuable. I use them. I would be dishonest to say otherwise.</p><p>But the FOMO this has manufactured, the specific anxiety that unless you are shipping AI-powered features you are already falling behind, is doing something cruel to people whose actual gifts lie elsewhere. To the writer made to feel that her craft is a deprecated skill. To the illustrator told his work is being automated. To the strategist, the teacher, the journalist who wakes up each morning to another wave of content about how their profession is being disrupted.</p><p>What the research actually says is this: the more the internet fills with AI-generated content, the more valuable original human creative work becomes. This is not optimism. It is basic economics. Scarcity creates value. And the irreducible, non-replicable voice of a specific person who has lived a specific life, is becoming scarce at exactly the moment when average output is becoming infinitely abundant.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/864/1*f9cjTlDUIQUOU1pGNMeIFQ.png" /></figure><h3>The attention problem underneath all of this</h3><p>There is a harder problem that the pivot-to-AI conversation almost entirely ignores.</p><p>The same digital environment flooding us with AI content has, over the past decade, been quietly degrading our capacity to produce anything else. The short-form video. The notification. The algorithmically optimised feed designed to extract engagement at the cost of depth. We have built, without quite meaning to, a machine for the destruction of the sustained attention that all genuine creative work requires.</p><p>You cannot write something original while part of your brain is waiting for the next notification. You cannot develop a distinctive voice while your reading diet consists entirely of content engineered for maximum immediate engagement. The brain that produces good writing, or good design, or good strategy, is a brain that has been allowed to be bored. To wander. To sit with a problem long enough that something unexpected comes up from somewhere below the surface.</p><p>That brain is getting harder to maintain. And when people say they feel like they have lost their creativity, or that they cannot write the way they used to, they are not imagining it. Something has been eroded. The good news is that erosion is reversible. The frustrating news is that reversing it requires doing things that feel almost perversely counterproductive right now: reading long things slowly, sitting without a screen, writing badly on purpose until something real comes through.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cKGVuTSbCuxXt_3CnY8dqw.png" /></figure><h3>What to actually do with any of this</h3><p>This is not an anti-AI piece. I want to be precise about what I am actually arguing. And honestly accept that I have used AI in writing this article to articulate my points better and to improve my vocabulary. But this was never “give me a viral article for medium” and then a Ctrl+C with a Ctrl+V. This was carefully written and corrected and optimised.</p><p>Use these tools for what they are genuinely good at. Speed, synthesis, volume, variation, first drafts, translation between formats, the thirty things on your list that need doing but do not need your specific fingerprint. These are real gifts. Be grateful for them.</p><p>But do not let the existence of those gifts convince you that the things which require your specific fingerprint have become worthless. They have not. They have become rarer. And in a market flooded with undifferentiated content, rarity is the only real premium left.</p><p>The research on how humans and AI differ on creative tasks found something worth sitting with: AI tends to produce the same safe ideas repeatedly, while humans naturally vary. The model optimises for the likely. The person, specifically the well-read, genuinely curious person who has built the habit of original thought, tends toward the surprising.</p><p>That tendency toward the surprising is not something you either have or do not have. It is a skill. It develops through reading widely and slowly, through writing regularly and badly, through the uncomfortable practice of sitting with your own observations long enough to form an actual opinion rather than reaching immediately for the consensus view.</p><p>Invest in your communication. Invest in your ability to walk into a room and make people feel something with a sentence. Invest in the capacity to tell a story that has your voice in it. Your cadences. Your metaphors. Your specific way of noticing things. These are not soft skills. In a world awash with adequate, they are quietly becoming the hardest skills of all to find and the only things people are still willing to pay a serious premium for.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8MTHPQm1g7m6IvuRT2pFIQ.png" /></figure><h3>A note on patience, because this part is genuinely hard</h3><p>Building a distinctive creative voice takes years. Learning to use an AI tool well takes weeks. In the short term, the returns on the AI investment are faster, more measurable, and more legible to a hiring manager. I am not pretending the economics make this an obvious choice.</p><p>But “right for right now” and “right for the next decade” are different calculations. And the people who will look back on 2026 with the most satisfaction, I think, will be the ones who used the efficiency gains from these tools to buy themselves more time for the slow, unglamorous, deeply human work of developing something that cannot be generated.</p><p>The world does not need more content. It has more content than it has ever had, more arriving every second than any person could read in several lifetimes. What it is hungry for, what it has always been hungry for and what no model has yet found a way to manufacture, is the feeling of encountering a real person who actually thought something. Who struggled with the words. Who cared whether they were true.</p><p>That is not going away. Protect it.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3edcc50953e4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I am not a great Coder. But I think I understand what Tech actually needs right now]]></title>
            <link>https://medium.com/@rasinbinabdulla/i-am-not-a-great-coder-but-i-think-i-understand-what-tech-actually-needs-right-now-7a936669c008?source=rss-db506407dbe0------2</link>
            <guid isPermaLink="false">https://medium.com/p/7a936669c008</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[careers]]></category>
            <category><![CDATA[technology]]></category>
            <dc:creator><![CDATA[Muhammed Rasin]]></dc:creator>
            <pubDate>Wed, 25 Mar 2026 07:01:07 GMT</pubDate>
            <atom:updated>2026-03-25T07:01:07.237Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/864/1*Y-T4aY_acnpS6De7A4POVw.png" /></figure><p><em>A note to myself and everyone in tech trying to figure it out amongst the AI panic.</em></p><p>To be brutally honest, I am on a journey of figuring out what is the meaning of life and a career at the same time, and I am not sure which one is harder. To some people, I am late. To another bunch, I am lost. And some others casually write me off as a career suicider, which is a word I did not know existed until people started using it about people who pause to figure things out. But for the past one year I have been observing, reading, listening, exploring, and trying to connect the dots on what it actually means to be a person in tech right now, in this particular moment, when everything that used to be true about this field is being renegotiated in real time. This is not a piece written from the other side of that figuring out. I am in the middle of it. But I have been paying close attention, and I have things to say.</p><p>I want to be upfront about who I am before I say any of it, because the context matters. I rejected the traditional path. I chose a degree in Computer Science without any burning passion for it, which is perhaps not the recommended way to enter a field that tends to reward obsession. I was a chronic GPT-aholic, which sounds funny until you understand what it actually cost me, which was the productive confusion of learning to code the hard way. There is something that happens when you struggle with a problem long enough that you build real understanding, a kind of slow burning clarity that stays with you. I outsourced that confusion too early and too often, and I paid for it in depth. I am terrible at memorising syntaxes on command. I panic in interviews when someone asks me to solve a problem on a whiteboard in a domain I have never personally cared about. My mind is a rebel. It goes deep only on the things it is personally involved in and refuses to perform on the things it is not. This is, historically, not a great trait for the way the industry filters people in.</p><p>And yet. The more this field has resisted accommodating me, the more I have fallen in love with it. Not with the credential game, not with the interview grind, not with the prestige hierarchy of company names and job titles. With the actual thing underneath all of that, which is the act of building something, understanding a problem so well that you can construct a solution to it, and then watching that solution actually work in the world. AI helped me get there, paradoxically. Where it failed me in the shortcuts it enabled, it turned around and helped me learn architectures and systems and concepts in depth, in the way my brain actually needs to learn things, which is through understanding the why before the how. The journey is still in motion. But I am moving.</p><p>I am writing this because I think the moment we are in is more interesting and more important than the noise around it suggests. The discourse about AI taking over and the layoffs and the collapse of the traditional tech career is real, but it is being interpreted mostly through the wrong lens. People are asking whether there will be jobs. The better question is what kind of thinking is going to matter, and why, and what that means for anyone who is trying to build a life in this field. That is what I want to think through here, as honestly as I can, drawing on a year of reading and observing and listening to founders and watching products live and die and trying to understand what separates the ones that mattered from the ones that did not.</p><p>This content is not AI slop. The grammar has been tidied up, but the thinking is mine, and it comes from genuine engagement with the question over a sustained period of time. It is exactly what an insignificant yet ambitious person in tech wants to pour into the internet from his heart.</p><h3>How we got here, and why the Old Model made sense at the time</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YLiEis3IIQTDgwMIzIdepA.png" /></figure><p>To understand what is changing, it helps to understand why things worked the way they did for so long. The software industry, for most of its commercial history, had a clean and effective division of labor. Business people figured out what to build and why. Product managers translated that into specifications. Developers implemented those specifications in code. This was not laziness or short-sightedness on anyone’s part. It was a rational response to the constraints of the time.</p><p>Building software was hard in a very specific and unforgiving way. Every layer of the stack required deep, specialised knowledge. The gap between someone who could write production-grade code and someone who could not was enormous, and that gap took years to close. In that environment, it made complete sense to hire people who were exceptionally good at implementation and let other people worry about the strategy. The specialist model was efficient. You needed people who could go deep on a narrow set of problems, and the system rewarded them accordingly.</p><p>The economics also made this work in a way that is easy to forget now. Venture capital was cheap. Growth was the primary metric. Companies could afford to build things that did not work very well, iterate slowly, waste engineering cycles on features nobody wanted, and still come out fine because the market was expanding fast enough to cover a lot of sins. The cost of a bad architectural decision or a misunderstood requirement was real but manageable. You just hired more people and fixed it later.</p><p>That world is gone. And the transition out of it has been brutal for people whose entire professional identity was built around skills that made perfect sense in that context and make considerably less sense in this one.</p><h3>What actually happend when the money got expensive</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Pd8T2371tSBQLkAVN0Oi8A.png" /></figure><p>The inflection point is not hard to locate. When interest rates rose and the era of essentially free capital ended, the calculus around engineering headcount changed overnight. Suddenly every team had to justify its existence in terms of actual value delivered, not potential value promised. Suddenly the question being asked in every boardroom and budget meeting was not how many engineers we have but what those engineers are actually producing and whether we could produce the same output with fewer of them.</p><p>This is when the layoffs started, and the pattern of who got laid off is more instructive than the raw numbers. Between 2023 and 2025, the tech industry laid off over half a million workers globally. The companies doing the cutting were not struggling startups. They were Google, Meta, Amazon, Microsoft. The ones who were supposed to be the finish line. And the roles that disappeared first were the ones furthest from the problem. The developers who were three layers removed from the user, who had never spoken to a customer, who could not tell you why the feature they were building existed, who were executing tickets written by someone else based on requirements gathered by someone else. These people were not incompetent. Many of them were technically excellent. But their value was contingent on an organisational structure that could afford to exist, and when that structure compressed, they were the ones who fell out.</p><p>At the same time, something was happening on the tooling side that made this compression possible in a way it never was before. The cost of execution was collapsing. Language models got good enough to write functional code. Infrastructure became increasingly automated. The things that used to require a team of six could increasingly be done by a team of two, or one, if that person was using the right tools and thinking clearly about what they were actually trying to accomplish.</p><p>This combination, economic pressure driving compression and technological capability enabling it, created the situation we are now in. A situation where the scarce resource in software development is no longer the ability to write code. It is the ability to figure out what code is worth writing, why, for whom, and with what tradeoffs.</p><h3>The tiles and prestigious company names are “vanishing”</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/864/1*-_6a3uQcWH6leNENcVdMxA.png" /></figure><p>I want to dwell on this for a moment because I think it is more significant than it seems. The era of the high-salaried fancy white collar job at a prestigious company being the definition of success in tech is not just economically over. It is philosophically over. The thing it was measuring, which was your ability to get through a specific kind of filtering process and perform in a specific kind of structured environment, is becoming less relevant to what actually matters in the field.</p><p>The giants, the ones who cracked the game early, are obviously still playing it well. But for the people who deliberately chose development as a career path and built their identity around that path, the challenge now is real. What the last decade figured out, slowly and painfully, is what was actually missing from the way development was practiced. The missing part was not technical. It was orientation. It was the difference between a developer who is pointed at a problem and a developer who finds the problem themselves. Between a developer who asks how and a developer who first asks why. Between someone who measures success by whether the code ships and someone who measures it by whether the code matters.</p><p>The tech stack game is over. That is not hyperbole, it is just an observation about where the differentiation has moved. Knowing React vs Vue vs Angular is not a meaningful competitive advantage anymore, not because those skills do not matter but because they are table stakes. What is not table stakes, what is still genuinely scarce, is the ability to solve the right problem in the right way for the right reasons, to learn whatever it takes to do that, to keep growing and challenging yourself in the direction of the actual outcome rather than the implementation of the assigned task. That is what is being rewarded now. And it is a fundamentally different orientation from the one the industry spent decades training people to have.</p><h3>The Problem with how the industry trained Developers</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gqDGzSqymNBif4FuaiLG3w.png" /></figure><p>Here is something that does not get said enough. The software industry spent decades selecting for a very specific cognitive profile and calling it engineering talent. The profile looked like this: exceptional ability to hold complex logical structures in working memory, strong pattern recognition within formal systems, comfort with ambiguity at the technical level, and a general preference for problems that have correct answers over problems that have better or worse answers depending on context.</p><p>These are genuinely valuable traits. The people who have them built the internet. But the selection pressure for these traits came with a cost that we are only now really feeling, which is that it systematically undervalued a different set of cognitive traits. The ability to sit with a messy human problem and resist the urge to immediately formalise it. The ability to listen to what someone says they need and hear what they actually need, which are often not the same thing. The ability to think about systems in terms of incentives and behaviors rather than just inputs and outputs. The ability to communicate across the gap between technical and non-technical thinking without condescending in either direction.</p><p>These traits were never irrelevant. But for a long time you could have a very successful career in software without them, because the system was structured in a way that put other people between you and the parts of the problem where those traits mattered. Product managers handled the user empathy. Business analysts handled the requirements. Project managers handled the communication. Developers handled the code.</p><p>The compression of that structure means those intermediate layers are thinner now, and in some organisations essentially gone. Which means the developer is much closer to the raw problem, and the traits that were optional are becoming necessary. The developer who cannot talk to a stakeholder, who cannot understand a business model, who cannot ask why a feature exists and evaluate whether the answer makes sense, who cannot make a judgment call about architectural tradeoffs without someone handing them a decision matrix, that developer is in a more precarious position than they have ever been. Not because they are a bad developer in the old sense but because the old sense is no longer the whole job.</p><h3>Requirements gathering is now a developer skill</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aLJpvuBdHes9NKRSGnqFxA.png" /></figure><p>This is one of the things that excited me when I started seeing it clearly, because it is one of the things I am actually drawn to naturally and never had a name for. The work of understanding what someone actually needs, asking the questions that expose the hidden assumptions, finding the places where the stated requirement and the actual goal are in tension, this has historically been the province of project managers and business analysts. As those roles thin out or get absorbed into product and engineering, the developer needs to be able to do this work. Not as a side job but as a core part of the role.</p><p>Requirements gathering done well is one of the most intellectually interesting things in software. It is essentially investigative work. You are trying to build a precise model of a problem from the imprecise and often contradictory testimony of the people who have the problem. Users rarely know exactly what they need. They know what they are frustrated by, what they have tried, what would make their day easier. The gap between that raw material and a well-specified requirement is significant, and crossing that gap requires judgment, empathy, and analytical rigour in roughly equal measure.</p><p>The developer who can do this well is doing something that has enormous leverage. Every degree of clarity you add at the requirements stage saves multiples of that effort in implementation. Every assumption you surface before you write code is an assumption you do not have to rip out of the codebase later. The communication between teams, between stakeholders, between builders and clients, this is not administrative overhead sitting on top of the real work. It is the real work, or at least inseparable from it. And it has come down from the project manager level to the developer level, not as a burden but as an expansion of what the role actually is.</p><h3>Business should have been in every Computer Science Degree</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/864/1*f4a-iBYfR289fSSOnJt2Ug.png" /></figure><p>I will say this as directly as I can. Business thinking is not a soft skill. It is not a nice to have that some well rounded developers pick up on the side. It is increasingly a prerequisite for making good technical decisions, and the fact that it is absent from most CS curriculum is a significant and ongoing failure that the industry is paying for in misaligned products and bad architectural choices made by people who did not have the conceptual tools to make them well.</p><p>Architectural decisions are business decisions. They are not neutral technical choices that can be evaluated purely on engineering grounds. The decision to build a monolith or a microservices architecture is a statement about your team structure, your scaling assumptions, your deployment complexity, and your organisational maturity. The decision to build a feature vs buy a solution or integrate an API is a statement about your core competency and where you want to invest engineering time. The decision to prioritise performance vs development speed is a statement about what your users actually care about vs what you assume they care about.</p><p>Marketing and SEO are being integrated into development in ways that would have seemed strange ten years ago and now just seem obvious when you think about it. The developer who understands why a product page needs to be structured the way it is, what the user intent is that a search query reflects, how the technical architecture of a site affects its discoverability, is making better decisions at the implementation level than one who has no framework for any of that. Go-to-market strategy is being integrated into the development process because the gap between building and shipping and actually reaching people has collapsed in ways that mean the person building the thing increasingly needs to understand the whole journey the thing needs to take.</p><p>This is genuinely exciting to me. I have spent the past year reading about product development, observing the lifecycle of B2B and D2C businesses and services, listening to founders and technical teams talk about what actually breaks and what actually works. I have been watching how the same idea executed with different distribution strategies produces wildly different outcomes. How a technically inferior product with a clear customer obsession beats a technically superior one with a confused value proposition almost every time. These are things that are not in the computer science syllabus but they are absolutely things that determine whether what you build actually matters.</p><h3>The Dopamine has to come from the Problem, not the Code</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/864/1*qXRRD6BoIkWVM21YiqT6fQ.png" /></figure><p>This is the part that feels most personal to me, and also the part that I think separates developers who are going to thrive in the next decade from the ones who are going to struggle with the transition we are in the middle of.</p><p>For a long time, the craft of coding was itself the reward. Writing elegant code, solving an interesting algorithmic problem, getting a system to work that was not working before, these things produced genuine satisfaction independent of whether they were solving anything that mattered in the world. And that orientation, intrinsic satisfaction from the technical work itself, was functional and even admirable in a context where someone else was making sure the technical work was pointed at real problems.</p><p>What is being asked for now is a different source of motivation. The developer who is going to matter is the one who finds their dopamine in the problem, not just in the code. Who is genuinely obsessed with the thing they are building, what it is supposed to do, why it matters, who it is for, whether it is actually working in the way it needs to work. Who reads about the domain they are building in not because they have to but because they cannot stop thinking about the problem. Who talks to users not as a process step but because they are genuinely curious about the gap between what they built and what the person actually needed.</p><p>This is not something you can fake for long. The developer who is doing this out of obligation rather than genuine interest will eventually exhaust themselves on it. But the developer who is genuinely oriented this way, who has found a problem or a domain that they are actually captivated by, that person has a kind of fuel that does not run out, and it shows in the quality of the judgment they bring to the work.</p><p>I am still figuring out where exactly this lives for me. But I know that the thing I am drawn to is being at the heart of it, not just in the implementation but in the whole arc of a product, from the problem to the build to the iteration to measuring the impact to talking to the users to understanding why it did or did not work. Being the middleman who carries the product, not just the person who wrote part of it. That is not just a role preference, it is increasingly what the role of a developer actually is, and that is the part that makes me excited to be in this field at this particular and confusing moment.</p><h3>What Obsession with the Problem actually looks like in practice</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2wDbeYh1u5l25FuqjS7I1w.png" /></figure><p>There is a version of this that gets described in a way that sounds exhausting and vaguely cult-like. The founder who thinks about the product in the shower and dreams in user stories and considers work-life balance a failure of passion. That is not what this is about, and conflating genuine intellectual engagement with self-destructive overwork is a humiliation to both.</p><p>What problem obsession actually looks like, in practice, is simpler and more sustainable. It is the developer who, before they touch a single line of code, spends real time with the problem. Who reads about the domain they are building in. Who talks to the people who have the problem, not to extract requirements but to understand the texture of the frustration, the history of attempted solutions, the context in which this problem exists in someone’s actual life or workflow. Who comes back to that understanding repeatedly as they build, using it as a reference point when making decisions about what to prioritise and what to cut.</p><p>It looks like someone who is genuinely curious about whether the thing they built actually worked, in the sense of solving the problem, not just in the sense of shipping without errors. Who treats the gap between what they built and what the user needed as an interesting puzzle rather than the user’s fault for not understanding the product correctly. Who is willing to throw out work that is technically impressive but solving the wrong problem, because the attachment is to the problem, not to the solution.</p><p>It looks like someone who reads things adjacent to their technical work. Business books and domain literature and research papers and interviews with founders and post-mortems of products that failed and analyses of products that succeeded. Not because someone told them to, but because the problem is interesting enough that understanding it from multiple angles feels natural. This kind of horizontal curiosity, the willingness to cross disciplines and bring back what is relevant, is one of the most valuable and least cultivated traits in technical education.</p><h3>Architectural thinking has come down to Every Level</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_HebB-nZmZzzvS_WoGCtSw.png" /></figure><p>One of the more interesting and underappreciated shifts in modern software development is the way that architectural thinking has moved down the organisational hierarchy. In older, larger organisations, architecture was the province of senior architects whose entire job was to think about structure and systems. Everyone else implemented what the architects designed. This model produced some genuinely good architectural thinking, but it also produced enormous disconnect, because the people making the architectural decisions were often far from the code, and the people writing the code had no ownership of the structural choices.</p><p>What is happening now, partly because of smaller teams and partly because of the pace of change making top-down architectural governance too slow, is that architectural judgment is becoming a skill required at every level. Junior developers are making choices with architectural implications every time they decide how to structure a module, how to name things, what to couple and what to separate, when to reach for an abstraction and when to leave things concrete. These choices accumulate. They either build a system that can change easily or a system that resists change.</p><p>Architectural thinking is not about knowing patterns and being able to name them. It is about understanding why certain structures solve certain problems better than others, what the tradeoffs are, and how those tradeoffs interact with the specific context you are in. You cannot make good architectural decisions without understanding the trajectory of what you are building. How fast will it need to scale? Who will be maintaining it? What are the most likely ways the requirements will change? What is the cost of being wrong? These questions cannot be answered from inside the codebase. They require understanding the business context, the team context, and the history of decisions that got you to this point.</p><h3>What AI actually changes and what it does not</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7EewNWExmPVo6hRPj-F72Q.png" /></figure><p>The honest picture of AI and software development is more nuanced and more interesting than either of the dominant narratives. AI is extremely good at certain things that used to be core developer skills. Writing boilerplate. Implementing well-understood algorithms. Translating a clear specification into working code. Debugging code that fails in patterned ways. These things can be done in a fraction of the time with AI assistance, and the quality is often comparable or better.</p><p>What AI is not good at, and what reflects a genuine limitation rather than just a current capability gap, is judgment. Understanding what problem is worth solving. Knowing when a technically correct solution is wrong for the context. Catching the places where the requirements are ambiguous in a way that will cause problems downstream. Making the architectural choice that is appropriate given the constraints and uncertainties of the specific situation. Recognizing when a user’s stated problem is not their actual problem. These are things that require understanding of context, history, incentives, and human behavior in ways that current AI systems are genuinely bad at.</p><p>The practical implication is that AI amplifies clarity. If you understand the problem well and can articulate what you are trying to build precisely, AI makes you dramatically more productive. If you are fuzzy on the problem, AI gives you very fast and confident-sounding implementation of the wrong thing, which is worse than slow implementation of the wrong thing because it is harder to recognise and correct. The developer who can use AI well is the developer who thinks clearly about problems before touching tools, who uses AI for execution and retains judgment for strategy.</p><p>This is why the builder mindset is not threatened by AI but made more important by it. As execution gets cheaper, the value shifts further toward judgment, problem understanding, and architectural thinking. The developers who internalise this will find AI to be a genuine superpower. The ones who do not will find it to be a faster way to produce output that nobody needed.</p><h3>What this all adds upto</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ySa3fhO1oDTntJ-RZRYmsA.png" /></figure><p>The developer who is going to matter in the coming years is not the one who knows the most frameworks or can pass the most technically demanding interview. It is the one who can sit with a real problem, understand it deeply, build something that addresses it well, communicate clearly about what they built and why, make good architectural decisions with incomplete information, and stay oriented toward the outcome rather than the implementation throughout the whole process.</p><p>This is not a mythological genius. It is someone who has cultivated a particular set of habits and orientations. Who has chosen to understand more than their immediate technical domain. Who has stayed curious about business and users and systems and communication not out of obligation but because of genuine interest in how the world actually works. Someone who wants to be in the trench, not just as a builder but as the person who carries the product from the idea to the impact and understands every part of that journey.</p><p>I am not there yet. I am still figuring out how to crack into exactly this kind of role. But I am more convinced than I have ever been that this is not just another role. It is the mandatory mindset for every developer who wants to be relevant in 2026 and beyond. And the interesting thing about this particular moment is that it is genuinely open in a way it was not before. The traditional markers are losing their predictive value. What is gaining value is harder to credential and harder to fake. It requires actually understanding problems. Actually building things that work. Actually making good judgment calls. Actually caring about whether the thing you built did what it was supposed to do in the world.</p><p>The tech stack game is over. The architect level game is just beginning. And it is, despite everything, exciting to be playing it.</p><p><em>This is not written from the other side of figuring it out. It is written from the middle of it, by someone who is insignificant and ambitious in equal measure, pouring it into the internet from the heart.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7a936669c008" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>