<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Rahul Powar on Medium]]></title>
        <description><![CDATA[Stories by Rahul Powar on Medium]]></description>
        <link>https://medium.com/@rahulpowar?source=rss-376557eaabb------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*FR0QSUx3nDBitG9GZq-CVw.jpeg</url>
            <title>Stories by Rahul Powar on Medium</title>
            <link>https://medium.com/@rahulpowar?source=rss-376557eaabb------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 19:12:09 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@rahulpowar/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Being Right About AI Could Still Cost You a Decade]]></title>
            <link>https://rahulpowar.medium.com/being-right-about-ai-could-still-cost-you-a-decade-324cf5d533f8?source=rss-376557eaabb------2</link>
            <guid isPermaLink="false">https://medium.com/p/324cf5d533f8</guid>
            <category><![CDATA[investing]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Rahul Powar]]></dc:creator>
            <pubDate>Mon, 31 Aug 2026 13:30:02 GMT</pubDate>
            <atom:updated>2026-08-31T13:30:02.873Z</atom:updated>
            <content:encoded><![CDATA[<h4>Why recursive machine intelligence makes this capital buildout different from railway mania — and no less dangerous to price badly</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oBXdUVQUXcf2yj97txiHSA.png" /><figcaption>Steaming ahead</figcaption></figure><h3>Being right cost a decade</h3><p>I am instinctively optimistic about AI. That is useful in my work as a technologist and potentially expensive in my life as an investor.</p><p>The two disciplines reward different kinds of timing. A technologist is supposed to see change before it becomes obvious and start building while the market still looks uncertain. An investor can correctly identify the future, choose the eventual winner and still lose an uncomfortable amount of money by arriving at the wrong price.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FwTiYaWFP59Q%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DwTiYaWFP59Q&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FwTiYaWFP59Q%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/1a7e35885cf344f581305fda310a13b0/href">https://medium.com/media/1a7e35885cf344f581305fda310a13b0/href</a></iframe><p><a href="https://www.youtube.com/watch?v=wTiYaWFP59Q">Patrick Boyle’s video about the consequences of a possible AI crash</a> left me in the awkward position of agreeing with almost everything he said while still feeling that something important was missing. Near the end, he makes an observation that stayed with me: an investor who bought Amazon at the peak of the dot-com bubble had correctly identified one of the most important companies that would ever exist. Amazon survived, the internet was real and online commerce did change the world. The stock still fell by roughly 90%, and the investor then had to wait roughly a decade to recover.</p><blockquote>Being right cost them a decade.</blockquote><p>I find this uncomfortable because I occupy both sides of the argument. I have spent much of my professional life building and investing in technology. Seeing the future is useful only if you are also right about the company, the price and the timing. The same instinct behaves differently in those two settings. For a technologist, being early is often the point: you build before the market is fully formed because, by the time everyone agrees the future has arrived, much of the interesting work has already been done. For an investor, being too early is often indistinguishable from being wrong. Capital has a cost, assets depreciate, companies run out of money, and markets can take much longer to recognise reality than an investor can remain solvent or patient.</p><p>Boyle is unquestionably right about the central lesson: a transformative technology does not protect the people financing it from overpaying. But I am less sure that AI behaves economically like the railways, or even like the internet.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/735/1*G4IRLM6BWvNJwD8Ullr3Wg.jpeg" /><figcaption>BIS Annual Economic Report 2026</figcaption></figure><h3>The useful discomfort of two minds</h3><p>My native error is unlikely to be underestimating AI, partly because I have seen this pattern before. At Shazam, I helped build one of the first apps available when Apple opened the App Store in July 2008. Mobile hardware and networks were already improving quickly, but most people still understood the phone principally as a device for calls and text messages. Working on the launch made the deeper shift clear: the phone was becoming a true computing platform, with software distribution, data connectivity and payments built into the system. The eventual winners and business models were still unknown, but the direction was not.</p><p>I felt something similar when I first used the closed GPT-3 beta in July 2020. The system was crude by current standards, but the experience was unmistakable. It felt like the early industrialisation of intelligence: a general cognitive capability that could be summoned through language, supplied at scale and improved through computation.</p><p>At the time, that view sounded hopelessly premature to most people. Then ChatGPT arrived and made it feel obvious.</p><p>That experience creates its own bias. Once you have watched something move from strange research preview to ordinary consumer behaviour in a few years, it becomes tempting to assume the rest of the curve will arrive on the same schedule — and that recognising it early is enough to make money from it.</p><p>As an investor, I know that recognising the direction of travel is not enough. The technologist in me looks at capability, while the investor has to look at capture. Who owns the scarce part of the system? How durable is that scarcity? How quickly will the asset be superseded? Who receives the productivity gain: the laboratory that paid for it, the competitor that copies it, the customer who gets a lower price, or society at large?</p><p>These questions matter in every technology boom. What makes AI unusual is that the capital being deployed may not merely serve future demand. It may help create the technology that creates that demand. That possibility is easiest to see if we ground the discussion in where the frontier actually stands as I write this, on 31 August 2026.</p><h3>What we know about Astra</h3><p><a href="https://openai.com/index/gpt-5-6/">OpenAI made GPT-5.6 generally available on 9 July</a>. The company says its researchers use the model to diagnose failures, optimise training systems, run experiments and improve other models. On 1 August, OpenAI named Astra its <a href="https://openai.com/index/ten-advances-in-mathematics/">“next major model”</a> and said an internal version had produced ten results addressing longstanding problems in mathematics and theoretical computer science.</p><p>Later that month, <a href="https://time.com/article/2026/08/26/openai-sam-altman-interview/">TIME reported</a> seeing sixteen Astra agents collaborate on a research-level mathematics problem. OpenAI chief scientist Jakub Pachocki also said Astra had passed an internal benchmark for an automated AI research intern: given an idea, it could implement and run the experiment, or reproduce work that had previously occupied a researcher for a week.</p><p>The investment argument does not depend on Astra shipping next week, on its eventual name or on any particular benchmark. It depends on whether machine intelligence is entering the process that produces the next increment of machine intelligence. On OpenAI’s account, it already is. Complete autonomy is unnecessary for the economic loop to matter; it begins when models let researchers test more ideas, run them faster and shorten the development cycle.</p><p>That is where the railway comparison starts to bend.</p><h3>What the railway comparison misses</h3><p>The railway mania analogy is powerful because its main lesson is true. During Britain’s Railway Mania of the 1840s, investors poured extraordinary sums into a technology that was plainly real and genuinely transformative. Some lines were built on implausible projections. Shareholders lost fortunes. The tracks remained, later travellers benefited and the economy inherited useful infrastructure from people who had paid too much to build it. That sounds uncomfortably familiar.</p><p>But the idea that railways merely waited for pre-existing passengers is too simple. Railways carried the coal, iron, machinery and workers needed to build more railways. They lowered the cost of the industrial economy around them. Branches fed traffic into trunk routes. New connections created settlements and industries rather than merely serving those already there.</p><p>Shildon is a useful example. Before the Stockton and Darlington Railway, it was a hamlet of around 100 people. The railway established engineering works there, and by 1851 the population had grown beyond 2,500. It became a locomotive-building town. The railway did not discover a large queue of passengers waiting politely in a field. It helped create the activity that justified its existence. <a href="https://historicengland.org.uk/whats-new/research/exploring-new-shildon-englands-first-modern-railway-town/">Historic England tells that history here</a>.</p><p>Railways were therefore self-amplifying in a real sense. So were machine tools, which helped manufacture better machines, and electrical grids, which powered the factories that made electrical equipment.</p><p>The difference with AI is not the existence of feedback. It is the directness, breadth and speed of it. A railway could carry the steel for the next railway; it could not reason about how the next locomotive should be designed. Rail fed back through logistics, metallurgy, finance and settlement over years or decades. AI can feed informational output back into software, model architecture, chip layout, research and infrastructure operation in weeks or months. It is not the first technology to improve the conditions of its own production. It may be the first general-purpose infrastructure whose principal output — cognition — can improve the infrastructure itself across such a broad range of activities and at software speed.</p><h3>The internet came closer</h3><p>The internet is a closer comparison because it also contained a powerful recursive loop. Better networks enabled better development tools, distributed collaboration, open-source software, search and cloud computing. Those products made the network more useful, which attracted more users and investment, which supported better products.</p><p>But much of the internet’s value still waited on changes in human behaviour. People had to become comfortable entering a credit-card number into a browser. Companies had to rebuild distribution and supply chains. Consumers had to learn to buy books, holidays and eventually almost everything else online. Billions of people had to decide that publishing their personal lives to strangers was a reasonable use of an afternoon.</p><p>Jeff Bezos understood the technological feedback even in 1999. In that year’s <a href="https://s2.q4cdn.com/299287126/files/doc_financials/annual/Shareholderletter99.pdf">Amazon shareholder letter</a>, he described the company as unusually fortunate because the foundational technology beneath it improved every day. “That is not normal,” he wrote. People are slow; organisations are often slower.</p><p>The fibre buildout ran far ahead of that adoption. After the dot-com boom, long-haul capacity sat unused while many of the companies that installed it failed. Yet even here, “the internet was overbuilt” conceals an important distinction. The fibre itself could remain useful for decades. The transmission equipment attached to it became obsolete much faster. A <a href="https://www.federalreserve.gov/boarddocs/speeches/2001/20010718/default.htm">Federal Reserve discussion of the capital overhang in 2001</a> made exactly this distinction between durable fibre and rapidly changing network equipment. That distinction matters for AI. A grid connection, power-generation asset or well-located data-centre site may remain valuable through several computing generations; a particular accelerator, cooling configuration or model-specific cluster may not. “AI infrastructure” therefore does not have one depreciation curve.</p><h3>Capital that can work on capital</h3><p>The strongest argument for today’s buildout is not simply that businesses will eventually buy a great deal of AI. It is that compute deployed before conventional demand catches up can be used inside the process of research, engineering and optimisation. The capacity does not necessarily have to sit in the dark waiting for passengers; it can help make the next unit of capacity more productive. We can already see early versions of this loop.</p><p>Google DeepMind says <a href="https://deepmind.google/blog/how-alphachip-transformed-computer-chip-design/">AlphaChip has been used in the layout of several generations of Google’s TPU accelerators</a>: chips built to run AI were partly designed using AI. OpenAI says GPT-5.6 Sol rewrote production kernels, helped run hundreds of experiments and monitored training processes. According to the company, that work <a href="https://openai.com/index/advancing-the-price-performance-frontier-with-gpt-5-6/">reduced end-to-end serving costs by 20% and increased token-generation efficiency by more than 15%</a>.</p><p>The important threshold is not complete human independence. The economic feedback begins much earlier. If machine intelligence allows the same researchers to test more ideas, write better systems, find more efficient kernels, design better accelerators and shorten the development cycle, then capital is already improving the productivity of future capital. Economists sometimes describe this as an <a href="https://www.aeaweb.org/articles?id=10.1257/pandp.20261034">invention in the method of invention</a>. It means part of the return from today’s compute may arrive not as immediate application revenue but as a faster and cheaper path to the next generation of capability. That makes investing ahead of visible adoption more defensible, but not self-justifying.</p><h3>The loop can eat the returns</h3><p>The same recursive mechanism that supports the bull case can make the investment case more dangerous. If each model generation helps produce the next one faster, the useful life of the current generation may shrink. If models improve chip design and inference efficiency, less hardware may be required for a given unit of useful work. Lower costs may cause total usage to explode, but a rapidly growing market can still deliver falling value per unit. If the frontier diffuses quickly, competitors can benefit from research they did not finance.</p><p>Since the start of 2026, Epoch AI estimates that the strongest open-weight models have lagged the closed frontier by an average of <a href="https://epoch.ai/data-insights/open-closed-eci-gap">around four months</a>. That is extraordinary for the spread of capability. It is less obviously attractive for the margins of whoever paid to establish the frontier. The pioneer funds the research; followers reproduce, distil and optimise it; customers receive more intelligence for less money. This can be a wonderful outcome for the economy and a mediocre one for the investor.</p><p>The race also creates a problem of collective behaviour. Each large laboratory or hyperscaler can rationally conclude that it must keep spending because falling behind may be fatal. Yet if all of them make the same calculation, they can collectively build more capacity than the eventual revenues justify. A recent <a href="https://www.bis.org/publications/working-paper-1367-ai-investment-race">Bank for International Settlements working paper</a> models precisely this dynamic. Its conservative calibration estimates investment at roughly 50% above the socially efficient level, with greater excess where demand is less responsive. That is a model, not a prediction, but the mechanism is familiar: the private prize for winning can drive every participant to spend more than the industry will collectively earn. AI can therefore be a rational race and a poor trade at the same time.</p><p>This is why rapid recursive progress does not answer the investor’s question. It sharpens it. The relevant question is no longer merely whether the capital produces something valuable. I think much of it will. The question is whether that value accrues to the company financing it, the company arriving one generation later, customers through lower prices, or society through productivity gains no single company can hold onto. The social return and the return on a particular security are not the same thing.</p><h3>Three clocks</h3><p>There is a tempting way to describe the difference between AI and earlier revolutions: the internet waited for people, while AI can increasingly operate on its own clock. I think that is partly true. Machine-generated code, experiments and research can move without waiting for millions of consumers to change their habits. As the systems become more capable, more of the improvement loop can run at machine speed. That could compress the period between infrastructure investment and useful output in a way no previous capital buildout managed.</p><p>But an investor still lives on several clocks at once: a model can arrive in the laboratory, in a product and in the income statement on three very different dates. Internal capability can advance before anyone outside the laboratory can test it. Public deployment can be delayed by product work, safety requirements or infrastructure. Commercial value arrives only when the capability becomes reliable and useful enough that somebody pays for it — and when the supplier can retain enough of that payment to earn an attractive return on the capital already spent.</p><p>Energy infrastructure, fabrication plants, transformers and grid connections obey physical construction schedules. Companies have financing costs and debt maturities. Hardware depreciates. Organisations must decide what to deploy, what to trust and who is accountable when it fails. Even an economy full of agents ultimately has to produce something that people value or allocate a genuinely scarce resource more effectively. Agents generating reports for other agents is activity; it is not automatically value. Recursive AI may speed up many of these processes. It does not repeal price, scarcity or time.</p><p>In fact, faster technical progress can increase timing risk. A six-month delay may contain several model generations. Waiting brings better and cheaper hardware; moving early may secure scarce energy and expertise. The correct choice depends on which layer of the system you own. The technology can operate increasingly on its own clock; a security still represents cash flows received on ours.</p><h3>Where I land</h3><p>I do not know whether the current AI boom will crash. Neither does anyone else. I do think Boyle’s historical warning survives intact. A technology can be real, useful and world-changing while the people who financed a particular phase of it lose enormous amounts of money. Amazon’s eventual success did not reimburse the owners of Pets.com, or make buying Amazon at any price painless.</p><p>But the analogy needs one addition: AI is not merely another destination for capital. It is becoming an input into the process by which we decide how to deploy capital, design systems and produce the next generation of AI. That recursive quality gives some ahead-of-demand investment genuine productive value and may shorten the journey from apparent excess capacity to useful capacity. It may also shorten the life of the assets we are buying, spread the resulting capability to competitors and push the price of intelligence towards the cost of energy and computation.</p><p>So I can believe that AI infrastructure is rational to build before conventional business adoption catches up while also believing that some of today’s builders are financing the systems that will make their own assets obsolete. I can be right about the technology and wrong about the company, right about the company and wrong about the price, or right about the destination and still arrive too early.</p><p>The technologist in me wants exposure to the slope. The investor has to keep asking who captures it, for how long, and how much of it is already reflected in the price. AI may increasingly improve the machinery of intelligence on its own clock.</p><p>My capital still compounds on mine.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=324cf5d533f8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What Should Children Learn When Machines Can Think?]]></title>
            <link>https://rahulpowar.medium.com/what-should-children-learn-when-machines-can-think-68a4b8f7cd3d?source=rss-376557eaabb------2</link>
            <guid isPermaLink="false">https://medium.com/p/68a4b8f7cd3d</guid>
            <category><![CDATA[future-of-work]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[education]]></category>
            <dc:creator><![CDATA[Rahul Powar]]></dc:creator>
            <pubDate>Thu, 25 Jun 2026 11:57:24 GMT</pubDate>
            <atom:updated>2026-06-25T11:57:24.806Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>I have had a front-row seat to two of the most consequential and contested technological revolutions of our time: the smartphone era and now the age of AI. Both have expanded human possibility. Both have been blamed, often fairly, for weakening parts of us we once took for granted. So I am often asked the same question: how do I think about my own preteen son, and the future he is growing up into?</em></h4><h4><em>These are the considered thoughts of a technologist and a parent who is not a professional educator, but who has had a few more years than most to think about what is coming.</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ETqooG3MWO1YTYOugaSX_A.png" /></figure><p>On 25 Jul 2020 at 00:35 UTC, some friends at OpenAI gave me access to the closed beta of the GPT 3 playground.</p><p>It is hard to describe now how strange that moment felt. The interface was crude and the model was the raw completion engine at the heart of what eventually became today’s LLM and yet the experience was unmistakable. I remember thinking: I did not expect to see this in my lifetime.</p><p>Not as some narrow machine learning system doing one impressive thing in a lab. I was used to those having built and invested in NLU/NLP technologies for close to a decade at that point. This felt like the first visible instance of something more general: intelligence becoming industrialised.</p><p>By that I mean intelligence that could be summoned on demand, at scale, by normal people, at marginal cost, through language. A colleague, tutor, editor, analyst, programmer, researcher, sparring partner; not fully any of those things, but weirdly adjacent to all of them.</p><p>I tried to explain what I had seen and almost nobody understood what I was talking about. Now that is not a criticism. Why would they have? The thing sounded abstract and a bit geeky. It was easier to file it away as another impressive technology demo, one more thing that people at the edge of the industry were overexcited about right alongside their distributed ledgers and non fungible tokens.</p><p>Then ChatGPT happened.</p><p>In some ways, the frightening thing is how recently that was. The consumerisation of this technology is still new, and yet it already feels culturally old. It has somehow both not changed the world enough and changed the world too much. Offices still have meetings. Schools still assign essays. Children still sit exams.</p><p>Yet something more fundamental has moved, and the impact zone is unevenly distributed.</p><p><a href="https://www.profgmedia.com/p/what-ai-is-doing-to-school">A recent Prof G Media piece</a> on AI in schools captured a version of this shift: teachers warning that writing, attention, and critical thinking are deteriorating in American classrooms, with students increasingly able to outsource the hard parts of learning to machines.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*G17jM-3LaaG3gRGjV9abHA.png" /><figcaption>If you line up iPhone and ChatGPT launch dates, the inversion of the slope is stark</figcaption></figure><p>The part that stayed with me was not simply that students were using AI. That is inevitable. It was the possibility that AI can make students appear more capable while leaving them less able to perform when the tool is removed. In one example cited in the piece, students using AI performed better during practice, but worse when they later had to work without it. That is the nightmare version of augmentation: competence that only exists while the machine is in the room.</p><p>Of all the bleak pieces I have read about technology, I found this one more concerning than most. Not because I am anti-technology. Quite the opposite. I am an optimistic technologist, which makes this more awkward rather than less. I have spent much of my life at the front line of two seismic shifts: smartphones and now AI. I continue to believe both are, on balance, extraordinary forces for good when used well.</p><p>But that phrase — <em>when used well</em> — is carrying almost the entire argument.</p><p>For adults, <em>used well</em> is already difficult. It requires judgement, restraint, context, and the ability to notice when a tool is helping you think versus quietly replacing the act of thinking. For children, those are precisely the capacities still under construction.</p><p>In the context of my son’s development, the issue is not whether AI is good or bad in some abstract sense. It is what happens when a tool that can remove cognitive friction arrives before a child has built the muscles that friction is meant to develop.</p><h3>The awkward position of being excited and afraid</h3><p>I do not come to this as someone who thinks the internet was a mistake, or that children should be raised in a museum of pre-digital virtue.</p><p>AI is already one of the most important tools in my own working life. Because of the role I have, and because I am lucky enough to work with a strong team that handles much of the day-to-day hard graft, increasingly with AI themselves, my own use of AI is not mostly about getting tasks done faster.</p><p>For me, AI is a collaborator in thought.</p><p>That distinction matters. I use it to articulate ideas that are still messy. I use it to argue with myself. I ask it for counterpoints, uncomfortable interpretations, possible blind spots, and better framings. Most of my thoughts begin as most human thoughts do: biased, partial, emotional, contradictory, and underdeveloped. The act of improving them usually requires contact with another intelligence.</p><p>That is not new. People have always used conversation to think. A good colleague, teacher, editor, friend, rival, or even opponent can sharpen an idea in a way private reflection often cannot. What is new is that this kind of friction is now available on demand.</p><p>I can debate an idea at 11pm. I can ask for the strongest version of the case against my own position. I can force a vague instinct into a clearer argument. I can test whether I am fooling myself. I can move from fog to structure faster than I could before. I never expected to have that in my lifetime. I now find it hard to imagine working without it.</p><p>So my concern is not that AI makes thinking impossible. My own experience is the opposite. Used well, it can make thinking more demanding, more explicit, and more rigorous.</p><p>The problem is that the same tool that sharpens the thinking of an adult who already has some agency, context, and self-discipline may weaken a child who has not yet built those muscles.</p><p>While AI can be a tutor, collaborator, simulator, editor, critic, and creative amplifier, it can also be a machine for avoiding the very struggle through which learning happens.</p><h3>The decline did not start with AI</h3><p>It would be too easy, and probably wrong, to blame all of this on AI. The decline in attention, reading stamina, numeracy, and long-form reasoning appears to predate consumer AI. Smartphones, social media, short-form video, pandemic disruption, weakened educational accountability, and a broader culture of distraction all seem to have contributed.</p><p>AI arrives into that environment.</p><p>That is what makes it dangerous, it offers an elegant escape route from difficulty at exactly the moment children may have less capacity to tolerate difficulty.</p><p>A child no longer has to sit with the blank page. They do not have to hold the shape of an argument in their head. They do not have to read the awkward paragraph three times until it starts to make sense. They do not have to feel the discomfort of not knowing.</p><p>They can paste, prompt, receive, and move on.</p><p>This is why I find the phrase <em>cognitive offloading</em> slightly inadequate. Humans have always offloaded cognition. Calculators are cognitive offloading. Search engines are cognitive offloading. A calendar, a map, a spreadsheet, a library — all of these reduce mental load, and we rightly celebrate them for the new heights they allowed us to reach.</p><p>But the more useful distinction is not between tools that offload cognition and tools that do not. It is between tools that are augmentative and tools that are substitutive.</p><p>An augmentative tool keeps the child inside the act of thinking. A calculator can accelerate arithmetic, but the child still has to understand what problem they are solving. A search engine can retrieve information, but the child still has to read, compare, interpret, and decide what matters. A spreadsheet can extend analysis, but only if the user understands the model they are building.</p><p>A substitutive tool does something different. It replaces the very cognitive act the child is supposed to be practising. This is where AI can cross the line from offloading into surrender. An LLM does not merely reduce part of the effort. Used badly, it can produce the whole thing: the structure, the tone, the supporting points, the conclusion, even the synthetic appearance of reflection. It can make it look as though thinking has happened when, in the only place that matters for learning, it has not.</p><p>That is qualitatively different.</p><p>The same tool can sit on either side of the line. If AI asks questions, gives hints, challenges assumptions, tests understanding, or critiques a child’s first attempt, it is augmentative. If it writes the essay, solves the problem, forms the opinion, or supplies the answer before the child has struggled with the question, it is substitutive.</p><p>If a child uses AI after struggling, it can accelerate learning. If they use it before struggling, it may replace learning.</p><h3>The luxury of banning the future</h3><p>There is a growing group of parents, especially among educated Western parents, who respond to all of this by restricting technology heavily or banning it outright. Sometimes this is inspired by stories about major technology founders who supposedly raised their children with little or no access to screens.</p><p>I understand the instinct.</p><p>Childhood should not be entirely mediated by glass. Attention is fragile. Social media companies have not earned our trust. A child’s mind deserves protection from products explicitly optimised to capture it.</p><p>But I am wary of turning elite low-tech parenting into a universal principle.</p><p>Many of the people giving that advice are captains of enormous amounts of capital. Their children may inherit networks, assets, education, and social insulation that make the practical consequences of technological under-preparation much less severe.</p><p>That is not the reality for most children on the planet.</p><p>For a child without that cushion, a blanket ban may be less an act of protection than an act of delayed exposure. It may preserve a certain kind of childhood for a while, but it may also leave them less prepared for a world in which working with machine intelligence becomes basic economic literacy.</p><p>The question is not simply: should children use AI?</p><p>The better question is: what kind of relationship with intelligence, effort, agency, and judgement are we helping them form?</p><p>Because there are two bad outcomes.</p><p>One is raising children who are dependent on machines.</p><p>The other is raising children who are unable to use the most powerful tools of their generation because the adults around them protected them from the technology without preparing them to live with it.</p><h3>1. Friction before acceleration</h3><p>The first principle, for me, is friction before acceleration.</p><p>Children need to struggle with problems. Not because suffering is noble, but because struggle is part of how the mind builds structure.</p><p>When a child works through a maths problem, writes a paragraph, debugs code, reads something difficult, or tries to explain an idea in their own words, the visible output is only part of the point. The real value is the internal work: forming a model, testing it, failing, adjusting, remembering, trying again.</p><p>AI can remove that friction beautifully, which is exactly why it must be introduced carefully.</p><p>A child should not reach for AI before they have first formed a rough attempt of their own. The attempt can be bad. In fact, it probably should be. Bad first drafts are evidence that a mind is actually doing something.</p><p>The machine should enter after the child has something to compare, defend, revise, or question.</p><p>“Write this essay for me” is surrender while “Here is my argument; where is it weak?” is learning.</p><p>It is the distinction between using AI to stay in the work and using AI to escape the work.</p><h3>2. Creation over consumption</h3><p>The second principle is creation over consumption.</p><p>This is where I think much of the debate around children and AI becomes too narrow. The question is not simply whether a child used AI. The question is whether the AI allowed them to <strong>do more thinking than they otherwise could</strong>, or less.</p><p>Used well, AI can let a child attempt things that would previously have been far beyond their technical ability. A child who cannot yet properly code can build a small game. A child who has only a rough grasp of physics can simulate an orbit, a bridge, or a bouncing ball. A child with an idea for a story can explore character, structure, dialogue, setting, and tone with a level of feedback that would normally require a patient adult sitting beside them for hours.</p><p>That is not a curiosity. In fact, it may be one of the most exciting educational possibilities of the technology.</p><p>For most of history, children have been limited not only by their imagination, but by the execution gap between what they could conceive and what they could actually make. AI narrows that gap, at least for tasks that exist mostly in the digital domain (for now). It gives them scaffolding. It lets them prototype before they have mastered every underlying technique. It allows them to reach for something slightly beyond themselves.</p><p>That can be deeply augmentative, but only if the child remains the author of the intent.</p><p>The thinking muscle is still being exercised if the child is deciding what they want to build, describing how it should work, testing whether it behaves as expected, noticing what is wrong, asking better questions, rejecting bad outputs, and iterating towards something that feels like theirs.</p><p>In that case, AI is not replacing the child’s agency. It is expanding the range over which that agency can operate.</p><p>A child building a game with AI still has to ask: what is the game? What makes it fun? What happens when the player loses? Why is this bug happening? Does this feel too easy? What would make it better? The machine may help with the code, but the child is still learning design, logic, causality, taste, debugging, and persistence.</p><p>That seems meaningfully different from a child asking AI to complete a worksheet, write an essay, or form an opinion before they have made any serious attempt of their own.</p><p>The line is not between human-only work and AI-assisted work. That line is already gone. The more useful line is between <strong>agency-preserving</strong> use and <strong>agency-collapsing</strong> use.</p><p>Writing is another example. We often treat writing as communication, but it is also cognition. You do not always know what you think before you write. You discover it through the act of trying to say it. If the machine writes the essay, the child may get the product without undergoing the transformation.</p><p>But the same principle applies everywhere. If AI removes the need to think, it is substitutive. If it lets a child think at a higher level, attempt something more ambitious, and stay engaged with the consequences of their choices, it is augmentative.</p><p>That is what I want for my child. Not a machine that makes his work disappear. A machine that helps him reach further than his current skills would normally allow, while still requiring him to care, choose, test, and understand.</p><h3>3. Metacognition is the skill underneath the skills</h3><p>The third principle is metacognition: children need to learn to observe their own thinking while it is happening.</p><p>That has always mattered, but it matters much more when they are learning alongside alien intelligences such as AI. The machine does not just answer questions. It can subtly change what question the child thinks they asked, what parts of the problem feel important, and whether they believe they understand something at all.</p><p>So the child needs a second process running in their head:</p><p><em>Do I actually understand this?</em></p><p><em>Where might it be wrong?</em></p><p><em>What assumption is it making?</em></p><p><em>What question should I have asked instead?</em></p><p>AI makes these questions more important, not less. The fluency of the machine creates an illusion of understanding. A well-written answer can make the user feel smarter than they are. That is dangerous for adults and catastrophic for children.</p><p>The future skill is not merely prompt engineering. That phrase was already too narrow when I first heard it invented live in 2020. The deeper skill is learning how to remain the author of your own thinking while using a machine that can think with you.</p><p>A child needs to understand that the machine is neither an oracle nor a toy.</p><p>It is a system with a shape. It has representations, strengths, weaknesses, defaults, blind spots, and compute constraints. It can be brilliant in one moment and strangely stupid in the next — in that sense, perhaps a rather accurate reflection of its creators. It can expose a hidden assumption in your reasoning and then miss an obvious physical constraint in the real world.</p><p>The point of metacognition is not to make children distrust the machine. It is to stop them from disappearing into it.</p><p>It is the internal voice that says: wait, I do not understand this yet. Wait, this sounds plausible but may be wrong. Wait, the machine has solved a different problem from the one I meant to ask. Wait, this answer is not mine.</p><p>Which brings me to the sceptics.</p><h3>The strawberry, the car wash, and the wrong lesson</h3><p>There remains a minority of people who take comfort in examples of AI failure.</p><p>They point to cases where a model fails at something trivial for a human and conclude that the whole project is somehow fraudulent. The famous version was the strawberry test: ask the model how many “r”s are in the word “strawberry”, and earlier systems would sometimes get it wrong. This became much less interesting once models were pushed to reason more explicitly about the task.</p><p>More recently there was the car wash example: “I want to wash my car. The car wash is 50 metres away. Should I walk or drive?” Some models, especially smaller ones or those running with low reasoning effort, answered that walking was healthier or more environmentally friendly, missing the rather important point that the car itself needs to arrive at the car wash.</p><p>These examples are interesting, but not in the way some AI sceptics intend.</p><p>They do not show that the systems are useless. They show that the systems are machines, with particular representations, defaults, and failure modes. Humans have their own versions of this. Intelligent people routinely fail the bat-and-ball problem because an intuitive answer arrives faster than the correct one. We miscount letters in simple sentences because we process language as meaning, not as raw character data. We miss obvious details when our attention has been pulled towards the wrong feature of a problem.</p><p>The analogy is not that AI systems think like humans. They clearly do not. The point is that failure on a simple task is not, by itself, proof that no intelligence is present. Often it is proof that the intelligence, human or machine, is looking at the problem through the wrong representation.</p><p>The strawberry example is partly about tokenisation. The model is not necessarily seeing the word as a neat sequence of letters in the way we might expect it does. The car wash example is about a surface heuristic overriding an implicit constraint. The model sees “short distance” and reaches for “walk”, unless it is pushed to reason more carefully about the goal.</p><p>In both cases, the failure is embarrassing. It is also explainable.</p><p>A child who sees AI as an oracle will be misled by it. A child who sees it as a toy will underestimate it. The right relationship is more demanding: this is a machine that turns a quantum of energy into a roughly scalable quantum of intelligence. More compute, better models, better prompting, better tools, and more time to think often produce better answers. Specifically on the car question, any frontier model as of summer 2026 with a thinking budget greater than or equal to ChatGPT 5.5 Thinking or Opus 4.8 will catch the implicit constraint: the point of going to the car wash is usually to bring the car with you. People do not, as a rule, wander down to the car wash on foot for the ambience.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*05zdiYkbMl68ncBbEMQ5_Q.png" /></figure><p>If users do not understand this relationship between cost of compute and intelligence, they misunderstand the tool in their hands.</p><p>AI literacy cannot simply mean knowing which app to open. It has to mean knowing when to ask the machine, when to struggle alone, when to demand a counterargument, when to verify, when to give it more context, when to increase the reasoning effort, and when to ignore the answer entirely.</p><p>But the question of what children should learn cannot stop at homework. If AI can increasingly perform the tasks by which the education system has traditionally measured competence, then we have to ask a larger question: what remains worth learning when machines can outperform us?</p><h3>We have exceeded ourselves before</h3><p>The mechanisation of human activity is not new.</p><p>We have repeatedly built machines that outperform us, and yet the exceeded human activity often remains meaningful.</p><p>We still celebrate Usain Bolt. We do this even though millions of people commute every day in machines that could outpace the fastest human who has ever lived. We have had these cars for more than 100 years. If tomorrow a humanoid robot ran faster than Bolt, that would be technically interesting but philosophically unsurprising.</p><p>The point of sprinting was never that humans were the fastest objects we could construct.</p><p>The point is what it means for a human being, under human constraints, to approach the edge of human performance.</p><p>The same is true in intellectual domains. Chess engines surpassed the best humans decades ago. Chess did not die. If anything, chess and chess influencers seem more alive than at any point in my lifetime.</p><p>Go followed a similar path. Strictly speaking, Go has not been “solved” in the formal mathematical sense, but machine play surpassed elite human intuition and taught human players new ways to think about the game. AlphaGo did not make Go meaningless. It revealed patterns humans had not fully seen.</p><p>The machine’s superiority did not extinguish the human practice. This is important because one lazy conclusion about AI is that if a machine can do something better than us, the human version no longer matters.</p><p>However, it is also not enough to comfort ourselves by saying humans will keep doing human things. Some activities survive mechanisation because they are sports, games, arts, rituals, disciplines, or sources of identity.</p><p>Work is harder.</p><h3>The uncomfortable question about knowledge work</h3><p>A large amount of modern knowledge work is not especially beloved.</p><p>This is the misshapen spanner I want to throw into the works. Perhaps some of our panic around AI is really panic about preserving a particular industrial model of white-collar work that was already not working especially well for many people.</p><p>We have built vast systems of meetings, performance management, documents, presentations, emails, status games, compliance rituals, shallow analysis, and performative busyness. Some of it is valuable. Some of it is necessary. Some of it is well-paid drudgery dressed up as professional purpose.</p><p>If AI threatens that, the right response cannot simply be to defend every existing task as sacred or bemoan substitution.</p><p>At the same time, work is not just a set of tasks. It is income, identity, structure, dignity, status, and social belonging. So we cannot be glib about its disruption either.</p><p>This is where countries like Japan become interesting. Decades of stagnation, ageing demographics, and a shrinking workforce give the rest of us a preview of pressures that are likely to appear more widely across advanced societies. China’s “lying flat” movement is different in cause and culture, but it points to another version of the same unease: younger people questioning whether the old bargain of relentless striving is worth it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*xJBeAB29gkkfCulUEFlS3g.jpeg" /><figcaption>Tang Ping people</figcaption></figure><p>AI is arriving into a world where the relationship between work, meaning, productivity, and social mobility was already under strain.</p><p>So when we talk about children, perhaps the question is not only: how do we prepare them for jobs?</p><p>It is also: how do we prepare them for a world in which the meaning of work itself may be renegotiated?</p><p>That is a much harder question. And having dragged you this far through my own attempt to think it through, I should admit that I do not have a complete answer.</p><p>But I do have a place I would start.</p><h3>What remains human?</h3><p>So what are the things that, as far as we can see today, make human contribution meaningful?</p><p>I would start with agency.</p><p>Today’s models are extraordinarily powerful when given a clear goal, useful context, and a direction worth pursuing. In that sense, they resemble many capable humans. They can do astonishing work when well-directed. They are much less useful when the objective itself is confused.</p><p>Knowing what is worth solving remains scarce.</p><p>So does knowing why it is worth solving. So does choosing the constraints. So does understanding the second order consequences. So does deciding what good looks like in a messy environment where the answer is not in the training data because the situation is new.</p><p>This is where metacognition and systems thinking matter.</p><p>There are also technical reasons not to assume the current models are the end state. Context windows are larger than they were, but holding all the fine detail of a vast, living system in a useful way remains difficult. The underlying transformer architecture has scaling behaviours that make long-context work expensive. Today’s widely deployed models are still largely fixed at deployment, with memory and tools bolted on rather than the durable, embodied, cumulative learning of a person.</p><p>We are clearly not at the end game.</p><p>People like Yann LeCun may be right that deeper machine intelligence needs different architectures: world models, planning, persistent memory, intrinsic objectives, and a more grounded understanding of reality. It is not obvious that AGI, whatever we eventually decide that word means, emerges simply by adding more parameters and making architectural optimisations.</p><p>But we should be careful not to turn today’s model limitations into a comforting permanent moat.</p><p>The models will get better. The tools will get better. Agents will become more capable. Memory will improve. Multimodal systems will become more grounded. Robotics will close some of the gap between language and action. A lot of things that feel like uniquely human advantages today may not feel that way for long.</p><p>We are already seeing AI systems contribute to areas like mathematics in ways that would have seemed extraordinary very recently. At the same time, many of those results come from systems built behind closed doors, with unclear architecture, unclear compute cost, and unclear generalisability.</p><p>The honest position is that machine capability is rising quickly, but agency still matters.</p><p>And it may matter more, not less, as capability becomes abundant.</p><h3>The school system may be solving the wrong problem</h3><p>Even if we agree that the future needs more metacognition, systems thinking, judgement, curiosity, creativity, and agency, it is not clear that our current education system is particularly good at producing those things.</p><p>Much of school is still organised around answer production.</p><p>Read this. Remember this. Complete this. Submit this. Sit this exam. Move to the next module.</p><p>That model made sense in a world where access to information was scarce, calculation was expensive, and producing competent written output was a meaningful proxy for knowledge work. But when a machine can produce plausible answers instantly, answer production becomes a weaker signal.</p><p>This does not mean knowledge no longer matters. Quite the opposite. You cannot have judgement without knowledge. You cannot think critically about something you know nothing about. You cannot direct a machine well if you lack a model of the domain.</p><p>But the purpose of knowledge changes.</p><p>The child is not learning facts because facts are unavailable elsewhere. The child is learning facts because facts become the internal structure that allows them to reason, notice errors, ask better questions, and resist nonsense.</p><h3>The adult fantasy of the AI tutor</h3><p>A lot of optimistic AI education talk imagines the child using AI as a Socratic tutor.</p><p>This is possible. It is also not the default.</p><p>The default use case for a tired child with an unfinished assignment and a friends group on Discord is not Socrates. It is autocomplete for effort.</p><p>If we want children to use AI as a tutor, we need to teach and constrain it as a tutor. The machine should ask questions before giving answers. It should offer hints before solutions. It should require the student to explain their reasoning. It should adapt to the child’s current understanding without allowing them to bypass understanding entirely.</p><p>In schools, this may mean redesigning assignments around process rather than just output. Drafts, oral defences, in-class writing, worked reasoning, and reflective comparison between a student’s attempt and the machine’s critique may become more important.</p><p>Detection is unlikely to be the foundation of a serious policy. AI detectors will be unreliable, gamed, or unfair often enough to create their own problems. A school culture built around accusation and evasion is not a learning culture.</p><p>The better path is to change the work.</p><h3>What I would want my child to learn</h3><p>For my own child, I do not think the answer is prohibition. Nor is it unrestricted access.</p><p>I want him to understand the machine. I want him to use it. I want him to be fluent in the tools that will shape his world. But I also want him to know the difference between help and dependence.</p><p>I want him to feel the productive discomfort of not knowing.</p><p>I want him to read things that are longer than his current attention span.</p><p>I want him to write badly before he writes well.</p><p>I want him to solve problems slowly before he solves them quickly.</p><p>I want him to experience the satisfaction of understanding something that initially resisted him.</p><p>Then I want him to use AI to go further: the quality of the question increasingly determines the quality of the power you can summon.</p><p>To test his thinking. To find gaps. To simulate opposing views. To build things. To explore ideas he could not have reached alone. To learn faster without becoming shallower. To become more capable without becoming less himself.</p><p>It requires curiosity. Taste. Moral seriousness. Domain knowledge. Patience. The ability to notice when something feels wrong. The confidence to disagree with a fluent machine. The humility to accept correction. The imagination to ask for something that does not yet exist.</p><h3>What problems are worth solving?</h3><p>So perhaps the best thing we can do is not teach children to compete with machines at being machines. We should not organise childhood around preserving every task that AI might automate. Nor should we abandon the disciplines that form the mind simply because a machine can now imitate their outputs.</p><p>We need a more careful answer.</p><p>Children should learn how to think before they outsource thinking.</p><p>They should learn how to create before they automate creation.</p><p>They should learn how to judge before they delegate judgement.</p><p>They should learn how to ask better questions before they are surrounded by infinite answers.</p><p>And above all, they should learn how to decide what is worth solving.</p><p>Because that may remain the most human question.</p><p>Not permanently, necessarily. Not because the models will never improve. Not because there is some simple technical line that machines can never cross.</p><p>But because a life is not only a sequence of tasks. A society is not only a resource optimisation function. A child is not only a future worker.</p><p>The point of education is not to produce a slightly slower version of an AI system. It is to produce a person with enough knowledge, discipline, curiosity, and agency to use power wisely.</p><p>That was true before AI.</p><p>It is much more urgent now.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=68a4b8f7cd3d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Five Civilizations AI Could Create]]></title>
            <link>https://rahulpowar.medium.com/five-civilizations-ai-could-create-4fb877ccf019?source=rss-376557eaabb------2</link>
            <guid isPermaLink="false">https://medium.com/p/4fb877ccf019</guid>
            <category><![CDATA[ai-governance]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Rahul Powar]]></dc:creator>
            <pubDate>Tue, 25 Nov 2025 18:00:30 GMT</pubDate>
            <atom:updated>2025-11-25T18:00:30.432Z</atom:updated>
            <content:encoded><![CDATA[<blockquote><strong>TL;DR</strong> — AI automates knowledge work while population growth flatlines, forcing a shift from scale to efficiency. The outcome depends on two axes: centralization (who owns compute) × access (does abundance reach everyone). At least five civilizations could emerge from this matrix, from corporate oligopoly to post-scarcity communism. The deciding factor isn’t AI itself, but compute ownership, data governance, and whether redistribution happens before solidarity becomes impossible.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*a0xCukxKxg0xqeHGmDjbcQ.jpeg" /><figcaption>Nano 🍌 Pro thinks someone approved this for construction.</figcaption></figure><p>AI is the first technology that seriously challenges the link between labour and production. Many arguments, my own included, point out that we’ve seen versions of this disruption before, during prior industrial and automation revolutions. Fears of mass obsolescence never fully materialised because new roles appeared and the overall economic pie expanded.</p><p>This time, though, I do think it’s different. AI is automating the knowledge work required to supervise and coordinate the machines of the last revolution. At the same time, populations across the developed world are flattening, placing asymptotic limits on consumption and on the consumption-led growth models that have dominated for decades. The net effect is that organisations and the societies they sit within, will tend to use these new tools to optimise for efficiency rather than scale.</p><p>New moonshots, like expanding human presence beyond Earth, could reignite speculative growth in the same way that the “New World” did centuries ago. But physics will likely keep this from becoming routine in my lifetime.</p><p>So what is the realistic path for today’s AI? To be clear, I’m not factoring in any AGI step change that may or may not emerge from more compute or architectural breakthroughs.</p><p>As I argued a year ago, what we have today — through integration and business-process transformation — drives the marginal cost of “intelligence,” and therefore of many goods and services, asymptotically toward zero. I emphasise <em>intelligence</em> because it’s still unclear whether our current models can truly innovate: that is, generate something genuinely new and valuable. You can argue both sides. Even if you exclude the deluge of AI slop, we now have examples of new science, materials, and mathematics produced with AI’s help. But it’s clearly not mechanically churning out breakthroughs that propel us forward by millennia, and transformers may not be the architecture to get us there.</p><p>Still, only a minority of human “knowledge work” day jobs are measured by this yardstick of frontier innovation and is irrelevant when thinking about automation at scale.</p><p>The second-order effects, however, are profound and far harder to predict. Take one of our sacred cows: capitalism. Strip it down while retaining the needed nuance and you get: <strong>wages</strong> → <strong>purchasing power</strong> → <strong>demand</strong> → <strong>profit</strong>.</p><p>If AI replaces labour at scale, the wage–labour system becomes structurally unstable. Marx expected capitalism to erode its own foundations by devaluing labour; AI accelerates that logic. Does AI adoption, then, imply a Marxist trajectory? One could fuel that argument further by noting that much of what the state performs in socialism — administrative coordination — is precisely the sort of function that AI could automate. That makes a minimal-state, fully coordinated society more technologically plausible than before.</p><p>But the counterargument is just as strong. AI rewards scale, compute concentration, data monopolies, and surveillance advantages. This creates powerful incentives for mega-firms or states to dominate entire sectors. Instead of stateless communism, you get techno-feudalism: a handful of compute landlords and a vast population of digital serfs. AI plus data supercharges behavioural prediction, strengthening platform monopolies, political manipulation, and social control. Historically, populations organised when exploitation became visible; AI can make exploitation invisible and personalised, undermining solidarity before it can form. Economically, automation without redistribution could generate widespread precarity, with the gains from AI locked behind gates accessible only to capital owners. In such a world, labour becomes irrelevant but capital remains dominant.</p><h3>The synthesis: which direction is more plausible?</h3><p>While it is clear that no one knows where this leads us, here are a few scenarios with conditions that might favor them. Below are 4 end states (and 1 bonus one), where they sit, and <strong>what may have to be true</strong> for that world to emerge.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KwIitVWB1HHHQg9xLiFGhg.jpeg" /><figcaption>Pick your adventure. Nano 🍌 Pro</figcaption></figure><h3>Corporate AI Oligopoly: <strong>Centralized + Partially abundant</strong></h3><p>In many ways, this is the most likely scenario as it is a linear progression of where we are today.</p><ul><li><strong>High centralization: </strong>Few corporations dominate AI.</li><li><strong>Middling access: </strong>Society gains some abundance (cheap AI services), but access remains tiered or subscription-based.</li></ul><h4>Conditions that create it</h4><ul><li>Regulatory capture: rules written by incumbents.</li><li>Strong IP enforcement on training data + compute concentration.</li><li>Closed models outcompete open ones.</li><li>Public is satisfied with convenience over autonomy.</li><li>AI productivity is real, but profits primarily accrue to shareholders.</li></ul><h3>Techno-Feudalism: Centralized + Restricted</h3><p>This could happen if supply becomes more constrained while free market economics are allowed to dominate. Global access could reduce if fewer players had access to the resources required to build state-of-the-art models and have the energy to run it.</p><ul><li><strong>High centralization:</strong> compute, models, and data controlled by a few mega-actors.</li><li><strong>Low access:</strong> AI productivity is locked behind contracts, APIs, and walled-garden ecosystems.</li></ul><h4>Conditions that create it</h4><ul><li>Closed-source frontier models dominate.</li><li>Polarization undermines democratic oversight.</li><li>Nation-states embrace social credit–like systems or platform governance takes over.</li><li>Compute becomes scarce (energy limits, export controls, geo-political chokepoints).</li><li>AI automates labor without redistribution, so people depend on platform patrons for income.</li></ul><h3>AI-Assisted Democratic Socialism: Centralized + Abundant</h3><p>Scandinavia but with near AGI-level administrative automation.</p><ul><li><strong>High centralization: </strong>State or regulated consortiums control foundational models, but distribute benefits broadly.</li><li><strong>Abundant access: </strong>Access is abundant because society deliberately redistributes the automation surplus.</li></ul><h4>Conditions that create it</h4><ul><li>Universal basic services (UBS) funded by AI-driven productivity.</li><li>State ensures open access to compute or public models.</li><li>Strong antitrust breaks up megacorp AI concentrations.</li><li>Healthcare, transport, education become AI-automated and cheap.</li><li>Politics shifts toward “AI welfare state” thinking.</li></ul><h3>Libertarian AI Abundance: <strong>Decentralized + Abundant</strong></h3><p>Crypto ethos meets cheap AGI — everyone has their own AI swarm. In many ways this is the most exciting for a technologist. We can imagine a world where AI hardware becomes cheap and energy efficient (photonics, model innovations, non-von Neumann architectures or some combination of all of the above).</p><ul><li><strong>Low centralization: </strong>Many small players run powerful models locally.</li><li><strong>Abundant access: </strong>Compute and models are cheap, open-source, widely available. Innovation happens at the edge.</li></ul><h4>Conditions that create it</h4><ul><li>Open-source AI <em>wins</em>: efficient models, cheap quantization, powerful on-device inference.</li><li>Breakthroughs in low-power silicon (neuromorphic, photonic, etc.).</li><li>Regulators fail to lock down frontier models; or society rejects regulation.</li><li>Encryption and distributed compute (e.g., GPU swarms) become normal.</li><li>Cultural preference for individual autonomy.</li></ul><h3>Marxist Post-Scarcity Communism: <strong>Decentralized + Fully abundant</strong></h3><p>This requires not just AI, but material abundance + social cooperation + political decentralization. This most likely requires abundant energy so is dependent on one or many of: cheap storage for renewables, mass market thorium reactors, fusion at scale or some magic that is yet to be invented. It is the rarest trajectory but may eventually happen after a transition to one of the above.</p><ul><li>Production is automated end-to-end.</li><li>Scarcity collapses for most goods.</li><li>No meaningful class distinction: capital/labor divide dissolves.</li></ul><h4>Conditions that create it</h4><ul><li>AI handles virtually all necessary labor: agriculture, manufacturing, administration and routine knowledge work.</li><li>Energy abundance (fusion, cheap renewables, etc.).</li><li>Compute decentralization — models can run anywhere, cheaply.</li><li>Culture shifts to cooperation because survival is no longer linked to labor.</li><li>State apparatus shrinks because logistics, planning, and arbitration are automated.</li></ul><h3>What really drives direction?</h3><p>This admittedly simplistic world view compresses the world into two forces:</p><ol><li><strong>Who owns the AI?</strong></li><li><strong>Does AI generate abundance for everyone or only for owners?</strong></li></ol><p>The most realistic near-term direction is <strong>Corporate AI Oligopoly</strong>, with slippage toward <strong>Techno-Feudalism</strong> if nation-states harden surveillance infrastructure. <strong>AI Democratic Socialism</strong> is the political counter-force to prevent the two above but if it does happen, it is unlikely to be a globally consistent blueprint which limits impact. <strong>Libertarian AI Abundance</strong> becomes more plausible if open-source models + cheap edge silicon keep improving faster than frontier models. <strong>#5 requires a multi-decade convergence of technology + culture </strong>and remains uncomfortable for my organic brain wired for entrepreneurship and personal achievement<strong>.</strong></p><p>So the deciding factor isn’t AI — it’s:</p><ul><li>How compute is owned and paid for</li><li>How data is governed</li><li>How society redistributes AI’s gains</li><li>Whether people can still organize politically in an algorithmically-mediated world</li></ul><p>AI creates the technological conditions for <strong>post-scarcity</strong>, but also creates the tools for <strong>total control</strong>. Which one wins is still in play.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4fb877ccf019" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Hidden Cost of “Hello”: Why Every Token in Your LLM Stack Matters]]></title>
            <link>https://rahulpowar.medium.com/the-hidden-cost-of-hello-why-every-token-in-your-llm-stack-matters-762819125946?source=rss-376557eaabb------2</link>
            <guid isPermaLink="false">https://medium.com/p/762819125946</guid>
            <category><![CDATA[optimization]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Rahul Powar]]></dc:creator>
            <pubDate>Thu, 30 Oct 2025 09:37:34 GMT</pubDate>
            <atom:updated>2025-10-30T09:37:34.982Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZD0NWw5KHyCtpKKn9bmNtg.png" /><figcaption>Fun fact, you can double the cost of hello for ChatGPT by shouting “HELLO” instead of a polite “hello” and triple it with DeepSeek (though DeepSeek tokens are famously cheaper).</figcaption></figure><blockquote>TL;DR. Tokens are the new bottleneck. As agents shift from human prompts to tool-calling, every field you return shows up on your invoice. I make the case for LLM-aware API design and show how <a href="https://redsift.com/">Red Sift</a> cut tokens by 84% in one example with zero quality loss. Redesign your schemas for faster agents and smaller bills.</blockquote><p>Here’s an unsettling and <a href="https://futurism.com/altman-please-thanks-chatgpt">widely reported truth</a>: every time someone types “Hello” to ChatGPT, it costs real money. Not much — fractions of a cent — but multiply that greeting across hundreds of millions of users, factor in the quadratic scaling of transformer architecture, and suddenly those two innocuous syllables represent &gt;$10M of dollars in aggregate compute costs.</p><p>Every word is transformed into a token in a modern LLM architecture and this token is mapped into embeddings, long sequences of (usually) floating-point numbers before any AI magic can happen. If this is all new to you, <a href="https://www.youtube.com/watch?v=wjZofJX0v4M">@3blue1brown</a> has an excellent introduction on his YouTube channel.</p><p>Every token, processed through dozens of neural network layers, demands energy, silicon, and cooling infrastructure. Here, I argue that we are at an inflection point where this funny observation about our social conventions and how we interact with AI is about to take on a whole new reality.</p><h3>From Human Words to Machine Exhaust</h3><p>For decades, the internet was predominantly human-to-human communication. We wrote emails, blogs, social media posts. LLMs trained on this corpus, learning the patterns of human language. But something fundamental has shifted, and I’ve been watching it happen in real-time.</p><p><strong>Human-generated content is becoming the minority of what LLMs consume.</strong></p><p>In this instance, I am not focused on the issue of AI slop training the next generation of AI (slop). ML systems should not <strong>eat</strong> their own output just as mammals should not <strong>consume brains, </strong>as my <a href="https://www.garymcgraw.com/">dear friend Gary McGraw</a> puts it and this remains a major issue. My comment is about test-time compute: inputs into <strong>AI systems are going to be dominated by reasoning on and producing structured data</strong>, not college kids prompting their homework away.</p><p>We are transitioning to a world where LLMs will spend most of their cycles processing the data exhaust of the digital world: API responses, telemetry streams, automation workflows, and tool invocations. The rise of Model Context Protocol (MCP), LangChain, and agentic systems means LLMs will spend more time reading and writing structured data for machine consumption, not just human prose for us.</p><p>And the lingua franca of this machine-to-machine world? JSON. God help us all.</p><h3>JSON: The Unavoidable Standard</h3><p>Why JSON and not protobuf, Thrift, or some other binary format? I’ve had this debate countless times, and the answer lies in its unique position at the intersection of multiple constraints:</p><p><strong>Technical Advantages:</strong></p><ul><li>Web-native: Built into every browser and web framework — if you are a dev who was born into the world of the browser (and most of you are), this is your default setting even if you rebranded from <em>full-stack </em>to <em>LLM engineer</em>.</li><li>Human-readable: Debug without special tools.</li><li>LLM-aligned: Textual format closer to training corpus.</li></ul><p>Yes, YAML might be marginally more compact syntactically. Binary formats would be smaller still. But both JSON and YAML share the same fundamental characteristic when viewed through the lens of LLM tokenization: <strong>well-designed structures are self-documenting but they’re verbose, and that verbosity compounds at scale</strong>.</p><h3>What is a ‘token’ and why does that impact anything?</h3><p>Before we go further, let me explain what I mean by “token” — because if you’re not deep in the LLM world and you did not watch the video I linked above, this might sound like cryptocurrency talk.</p><p><strong>A token is the fundamental unit of text that an LLM processes.</strong> Think of it as the atomic particle of language in AI systems. It’s not quite a character, not quite a word, but something in between that optimizes for the statistical patterns the model learned during training.</p><h3>The Mechanics of Tokenization</h3><p>Here’s where it gets interesting. When you send text to an LLM, it doesn’t see words the way you or I do. Instead, a tokenizer breaks your input into chunks:</p><p><strong>Real-World Examples:</strong></p><ul><li>&quot;hello&quot; → 1 token</li><li>&quot;authentication&quot; → 3 tokens (auth, ent, ication)</li><li>&quot;JSON&quot; → 1 token in most models, sometimes 2</li><li>&quot;user_authentication_token&quot; → 5 tokens in GPT-4, 6 in Claude</li></ul><p>The pattern isn’t arbitrary. Tokenizers are trained on massive corpora, learning which text chunks appear frequently enough to deserve their own token. Common words get single tokens. Technical terms might split across multiple. This is why the same text can tokenize differently across models — they were trained on different datasets with different statistical patterns. Language distributions on the training corpus could have an impact, e.g. a model that is trained on large non-latin documents may need to preserve more of its token budget for glyphs and words in other characters.</p><p>Consider this curious example I discovered while optimizing Red Sift’s security assessment payloads: <a href="https://tokentamer.app/guide/llm-json-optimization?tokenizer-text=authority#tokenizer-playground">the word “authority” tokenizes as two tokens in DeepSeek but one token in GPT-4</a>. If that field appears in millions of API responses per day, the cumulative token overhead would be double if we used DeepSeek (we do not). Multiply these micro-inefficiencies across every field name, every nested structure, every redundant key, and you start to see the problem.</p><h3>Why This Matters: The Economics of Intelligence</h3><p>Every token you send to an LLM costs money. Not metaphorically — literally. Here’s the current pricing landscape I’m seeing:</p><p><strong>GPT-4 Turbo (October 2025):</strong></p><ul><li>Input: $10 per million tokens (~$0.00001 per token)</li><li>Output: $30 per million tokens (~$0.00003 per token)</li></ul><p><strong>Claude Sonnet 4:</strong></p><ul><li>Input: $3 per million tokens</li><li>Output: $15 per million tokens</li></ul><p><strong>DeepSeek:</strong></p><ul><li>Input: $0.27 per million tokens</li><li>Output: $1.10 per million tokens</li></ul><p>These seem tiny, right? Let me show you why they’re not.</p><h3>Case Study: The Compounding Effect at Scale</h3><p>Recently, I analyzed a production API at Red Sift that generates security assessment reports. Each report is a JSON structure describing vulnerabilities, compliance status, and remediation steps. This is not dissimilar to what every business today is doing, using existing APIs designed for a traditional use case and exposing it to an LLM workflow.</p><p><strong>Initial Assessment:</strong></p><ul><li>Average response size: 47KB of JSON</li><li>Token count: 12,847 tokens (GPT-4 tokenization)</li><li>Let’s assume a cost of $10 per million tokens (~$0.00001 per token)</li><li>Cost per invocation: $0.12847 input (to process the query) + some assumed output token cost. As we are not optimizing that in this discussion, let us ignore it.</li><li><strong>Total: $0.12847 per tool call</strong></li></ul><p>While this still doesn’t sound like much, here’s the multiplication:</p><p><strong>Monthly Reality:</strong></p><ul><li>Volume: 100,000 assessments/month</li><li>Monthly cost: $12,847</li></ul><p>The annual projection is <strong>$154,164 </strong>for this single API endpoint. Now we need to multiply across our entire tool invocation surface. Every MCP call. Every function result. Every structured output. And this is just us, one vendor in a sea of services.</p><h3>The Hidden Costs: Compute and Latency</h3><p>But token count doesn’t just affect your inference bill — it fundamentally impacts performance.</p><p><strong>Key Performance Characteristics:</strong></p><p><strong>Processing Complexity:</strong> LLMs use self-attention mechanisms where computational cost scales quadratically with input length. Double your tokens? Quadruple your compute (roughly — this varies by architecture).</p><p><strong>Inference Latency:</strong> More tokens means more sequential processing steps. At scale, this translates to:</p><ul><li>Longer time-to-first-token (TTFT)</li><li>Extended total generation time</li><li>Higher queue wait times under load</li><li>Worse user experience</li></ul><p><strong>Context Window Competition:</strong> Every token in your tool response consumes precious context window space. That’s space that could hold conversation history, additional examples, or richer instructions. For us, this caps the number of tools we can call and the amount of insight we can reason on in parallel.</p><h3>The Energy Dimension</h3><p>Here’s what really annoyed me: tokens don’t just cost money and time — they cost energy.</p><p><a href="https://epoch.ai/gradient-updates/how-much-energy-does-chatgpt-use">According to recent Epoch AI research, processing a single GPT-4 query with 500 output tokens consumes approximately</a> <strong>0.3 watt-hours of energy</strong>. Scale that:</p><ul><li>1 million queries = 300 kWh</li><li>1 billion queries = 300,000 kWh = 300 MWh</li><li>At the scale of ChatGPT a few years ago (~100M daily queries) = 30 MWh per day</li></ul><p>For context, that’s enough to power about 1,000 average US homes for a day. From queries alone. Factor in the input processing, the infrastructure overhead, the cooling systems, and you start to understand why, according to recent IEA projections, global data center electricity consumption will more than double to 945 TWh by 2030, driven primarily by AI workloads. <strong>That’s roughly equivalent to Japan’s entire annual electricity consumption.</strong></p><p>This 0.3 Wh estimate is based on conservative assumptions, the model being estimated on is quite old and all the latest models use some form of test-time compute e.g. chain-of-thought or the like. Crucially, this blows up as the amount of input tokens goes up.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2_EPoFHxjjZ571BlFL8QLg.png" /><figcaption>Notice the quadratic difference in energy use at “Maximum-input query” vs. a “Typical ChatGPT query”</figcaption></figure><p>This graph shows the quadratic reality of modern transformers catching up to us.</p><h3>The Optimization Imperative</h3><p>This is why token efficiency matters:</p><p><strong>Economic Impact:</strong> Direct correlation to operating costs at scale<br><strong>Performance Impact:</strong> Affects latency, throughput, and user experience<br><strong>Environmental Impact:</strong> Every wasted token is wasted energy<br><strong>Competitive Impact:</strong> In a world where energy becomes the bottleneck, efficiency becomes strategic advantage</p><p>However, there are very few tools that let engineers see the impact of their API design on MCP tool calling efficiency, so I* made one.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P1r_Zsv9DJfyuI4okcZ4MA.png" /><figcaption>Our first real world optimization using Token Tamer dropped the token count of our inputs by 84% while having no impact on model performance. *I made this over many hours with OpenAI Codex. It likely amortized the tokens spent making it in the first optimization task.</figcaption></figure><p>When I <a href="https://tokentamer.app/">built Token Tamer</a>, I wanted to make these hidden costs visible. Because once you see them, you can’t unsee them. Every verbose field name, every redundant structure, every unnecessary nesting level — they all add up.</p><p>And in the age of ubiquitous AI, what adds up eventually matters.</p><h3>The Energy Reality: Numbers That Should Concern You</h3><p>It is becoming apparent that AI is likely to be limited by the supply chain, not technology innovation. How many fabs, how many data centers, but critically, how much energy is available for training and inference.</p><ul><li><strong>127% global growth</strong> in data center energy consumption by 2030</li><li><strong>130% increase in US</strong> data center power demand</li><li><strong>169% growth in China</strong> where energy is treated as a “solved problem”</li><li><strong>1.5% → 3%</strong> of global electricity consumption shift</li></ul><p><strong>China is dramatically outpacing the US in energy infrastructure development for AI.</strong></p><p>I’ve been following reports from AI researchers who toured China’s data center landscape. Their assessment? Energy availability there is treated as “a solved problem.” China adds more electricity capacity annually than Germany’s entire consumption. State-owned utilities. Advance planning. Rapid buildout.</p><p>Meanwhile, the US struggles with:</p><ul><li>Grid bottlenecks and aging infrastructure</li><li>Permitting delays spanning years</li><li>Local opposition to new data centers</li><li>Utility companies unable to scale fast enough</li></ul><p>Goldman Sachs has warned that the power grid could become a “critical bottleneck” choking AI growth. Europe lags even further behind.</p><p>In this context, token efficiency is a drop in the ocean but it is a trivial fix compared to every other challenge on the docket. <strong>We are competing in a world where compute and energy are becoming geopolitically scarce resources</strong>.</p><h3>Enter Token Tamer: A Micro-optimization Tool Born from Necessity</h3><p>I’ve wanted this tool since the GPT-3 playground days. Not another JSON formatter or prettifier — those exist and you will have used one. I needed a <strong>semantic analyzer for JSON structures</strong> specifically designed for the LLM tool-calling era.</p><p>Something that would help me understand the hidden costs embedded in API schemas and tool invocations. Something that would quantify what I’d been tuning intuitively for years.</p><p>So I built it. With an LLM, actually — a recursive beauty that captures this moment: AI helping build tools to make AI more efficient.</p><p><strong>What Token Tamer Does:</strong></p><p>Token Tamer reveals the anatomy of your JSON’s token economics:</p><ul><li><strong>Token density analysis</strong>: Which fields contribute actual semantic value vs. bureaucratic overhead</li><li><strong>Model-specific tokenization</strong>: How your schema performs across GPT-4, Claude, DeepSeek, Llama</li><li><strong>Naming efficiency</strong>: Quantifying how field name choices translate to token costs</li><li><strong>Redundancy detection</strong>: Where your schema repeats information unnecessarily</li><li><strong>Cumulative impact modeling</strong>: What design decisions cost at scale</li></ul><p><strong>Technical Architecture:</strong></p><p>Here’s something important: <strong>Token Tamer runs entirely in your browser</strong>. No server uploads. No API calls. Your data never leaves your machine.</p><p>I made this choice deliberately:</p><ol><li><strong>Privacy by design</strong>: Your potentially sensitive API schemas stay local</li><li><strong>Zero latency</strong>: No network round-trips for analysis</li><li><strong>No infrastructure costs</strong>: Sustainable operation at scale</li><li><strong>Transparent operation</strong>: Open the browser console and see exactly what’s happening</li></ol><p>The tool uses WebAssembly-compiled tokenizers for performance, processing even large JSON structures in milliseconds. All tokenization happens client-side using the same algorithms as the production models.</p><h3>Real-World Impact: Red Sift Case Study</h3><p>At <a href="https://redsift.com/">Red Sift</a>, we generate large-scale programmatic security assessment structures — complex JSON payloads that power automated posture analyses. These documents can be hundreds of kilobytes, with deeply nested structures describing security configurations, vulnerabilities, compliance status, and remediation recommendations.</p><p>Before Token Tamer, I tuned these payloads manually and intuitively. I’d look at a 50KB JSON response and think, “This feels bloated.” But feelings don’t translate to tickets, and tickets don’t translate to engineering time without data.</p><p>Token Tamer gave me that data for a representative API call.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SbnzwZb3zEXPjmPHhiRItQ.png" /><figcaption>Rules dynamically create patches that turn a vanilla API response into one optimized for LLM tool use.</figcaption></figure><p><strong>Specific Optimization:</strong></p><ul><li>Original sample data: 32,291 tokens (Gemma-3 tokens; comparable to other modern tokenizations)</li><li>Over 30 mapping rules created to rename, collapse or supresse fields. This eliminated redundant prefixes, bloat due to APIs designed for completeness vs. specific utility and various pre-LLM era design choices.</li><li>After optimization: 5,266 tokens</li><li><strong>84% reduction</strong> in input token costs per invocation</li></ul><p>Multiply that across thousands of assessments per day, and the savings become material. More importantly, the tool identified patterns I hadn’t noticed — systematic inefficiencies that compounded across our entire API surface.</p><h3>From Code Quality to Planetary Stewardship</h3><p>Good API design has always mattered for maintainability, performance, and developer experience. But in the age of ubiquitous LLM tool invocation, it takes on new dimensions.</p><p>Every redundant field in your JSON schema is:</p><p><strong>Economic waste</strong>: Burning money on unnecessary token processing.<br><strong>Computational waste</strong>: Cycles that could be spent on actual intelligence.<br><strong>Environmental waste</strong>: Energy consumed that accelerates data center emissions.</p><p>When you’re designing an API that will be called billions of times — by humans, by automation, by AI agents orchestrating other AI agents — the difference between a thoughtful schema and a bloated one isn’t just technical debt. It’s <strong>carbon debt</strong>.</p><h3>The Questions Ahead</h3><p>Token Tamer is a small tool addressing a specific problem. But it points toward larger questions:</p><p><strong>1. What new conventions will emerge for LLM-optimized tool calling and data encoding?</strong></p><p>Should we develop JSON subsets or alternatives specifically designed for machine comprehension? Can we create dialects that preserve human readability while dramatically reducing token overhead? This is the most obvious area for improvement, and approaches <a href="https://github.com/johannschopplich/toon">such as Token-Oriented Object Notation (TOON)</a> point to a simple path forward. Helpfully for us, these encoding changes are additive and easily chainable on top of Token Tamer optimizations.</p><pre>gpt-5-nano<br>→ toon         ███████████████████░  96.1% (148/154)<br>  csv          ██████████████████░░  90.3% (139/154)<br>  yaml         ██████████████████░░  89.0% (137/154)<br>  json         ██████████████████░░  87.7% (135/154)<br>  xml          █████████████████░░░  83.8% (129/154)<br><br>claude-haiku-4-5-20251001<br>  yaml         ██████████░░░░░░░░░░  49.4% (76/154)<br>→ toon         ██████████░░░░░░░░░░  48.1% (74/154)<br>  csv          ██████████░░░░░░░░░░  48.1% (74/154)<br>  json         █████████░░░░░░░░░░░  47.4% (73/154)<br>  xml          █████████░░░░░░░░░░░  46.8% (72/154)<br><br>gemini-2.5-flash<br>  csv          ██████████████████░░  87.7% (135/154)<br>  xml          █████████████████░░░  85.1% (131/154)<br>→ toon         █████████████████░░░  83.8% (129/154)<br>  json         ████████████████░░░░  78.6% (121/154)<br>  yaml         ███████████████░░░░░  76.6% (118/154)<br><br>grok-4-fast-non-reasoning<br>→ toon         ██████████░░░░░░░░░░  48.7% (75/154)<br>  json         ██████████░░░░░░░░░░  48.1% (74/154)<br>  xml          █████████░░░░░░░░░░░  47.4% (73/154)<br>  yaml         █████████░░░░░░░░░░░  46.8% (72/154)<br>  csv          █████████░░░░░░░░░░░  45.5% (70/154)</pre><p>TOON claims 30% to 60% savings in tokens compared to JSON and even YAML while slightly improving retrieval accuracy vs. JSON on benchmarked models. I have also found it more pleasant to view as an organic brain. It is unlikely to be the only innovation, as the flexibility of LLMs should, in principle, allow them to support any reasonably consistent data structure format without needing internet-scale standardization.</p><p><strong>2. How might optimization feedback into model training pipelines?</strong></p><p>Could we train models that are better at understanding compressed or optimized schemas, creating a virtuous cycle? What would LLM-native protocols look like?</p><p><strong>3. How will AI-native APIs evolve as we move from human-to-AI to AI-to-AI ecosystems?</strong></p><p>When both the producer and consumer of an API are machines, the constraints change fundamentally. Human readability becomes less critical. What does that future look like?</p><p><strong>4. Will a shift away from transformers and attention make this an irrelevant problem?</strong></p><p>While I hope so, I suspect the rules of the universe will require us to always spend energy to do work. Doing less work will always be the most reliable way to save energy.</p><h3>Efficiency as Ethics</h3><p>Every developer making schema design decisions, every architect choosing data formats, every platform optimizing (or not optimizing) their APIs — you’re all making choices that compound. Those choices aggregate into the global compute budget for intelligence, both artificial and collective.</p><p>Token Tamer exists to make those costs visible. To turn intuition into measurement. To help us build the next generation of AI infrastructure with both power and conscience.</p><p>Because in the end, every “Hello” counts.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=762819125946" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[DeepSeek Triggers $1 Trillion Slump but Paves a Bigger Future for AI]]></title>
            <link>https://rahulpowar.medium.com/deepseek-triggers-1-trillion-slump-but-paves-a-bigger-future-for-ai-2d745bddb826?source=rss-376557eaabb------2</link>
            <guid isPermaLink="false">https://medium.com/p/2d745bddb826</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[reasoning]]></category>
            <category><![CDATA[stock-market]]></category>
            <dc:creator><![CDATA[Rahul Powar]]></dc:creator>
            <pubDate>Mon, 27 Jan 2025 15:55:19 GMT</pubDate>
            <atom:updated>2025-01-27T15:55:19.872Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uJFr0iy1bte4JEcYGdtoVg.jpeg" /><figcaption>Out of the blue. By recraft.ai</figcaption></figure><p>Even individuals who have no interest in AI woke up to a <a href="https://www.independent.co.uk/business/deepseek-ai-nvidia-meta-google-share-price-news-b2686909.html">rude shock</a> as the share prices and indexes that have any exposure to the business of AI took an eye watering tumble this Monday. This is really down to the acknowledgement of what a <a href="https://www.deepseek.com/">relatively unknown Chinese lab has done</a> —they released a fully open source model that gives all the best US AI models a run for their money, quite literally. They burst on the scene with a few phenomenal innovations.</p><ol><li>They released a model in the true spirit of open source i.e. provided detailed information about how they built it and the lessons they learned in addition to releasing the source code. <a href="https://www.marketingaiinstitute.com/blog/google-ai-memo">Google famously had an internal memo leaked in 2023</a> “We have no moat, and neither does OpenAI,…” — DeepSeek demonstrated this last week.</li><li>They also released a reasoning model DeepSeek-R1 that seems to be competitive with OpenAIs flagship o1. The current frontier arms race is all about reasoning and OpenAI is quite secretive about how they actually do this, much to the disappointment of the community. A chinese lab that is both matching performance and talking about how they do it is big news.</li><li>They allegedly did this for less than $6m with minimal investment in training infrastructure as China is frozen out of the latest hardware that is available to the rest of the world due to trade restrictions. If more can be done with a lot less, this of course calls into question the market valuations and future prospects of everyone in the supply chain.</li></ol><p>Clearly, the third point has probably most impacted the stock market this morning and while it’s a reasonable knee-jerk reaction, I believe this is a strong positive for the field and ultimately, a strong positive for the market for AI compute in the medium term.</p><p>In full disclosure, I am personally exposed to the market movement being invested in the Index funds that make up a significant majority of the markets in question and being long on individual stocks such as nVidia and TSMC.</p><p>While there is short term pain, I believe DeepSeek has shown a new way forward. For most organizations using AI in applied fields, AI today has boiled down to using the AI APIs for well known closed models such as GPT-4o or Sonnet hosted by OpenAI/Microsoft Azure and Anthropic/Google respectively. There are unique prompting strategies, RAG or function calling customizations but the models are essentially the same. This release shows that much cheaper Open Source models can be created, distilled and deployed to better match the needs of the use cases. This critically makes model training and fine tuning a much more approachable and economically attractive option for the many kinds of technology business out there and will I believe drive up the demand for compute over the medium term. Also, we will have to wait and see how the frontier evolves in 2025 with a number of new models set to redefine the art of the possible this year such as o3 from OpenAI.</p><h3>Running this model</h3><p>Another piece of news that got overshadowed by DeepSeek was the release of <a href="https://lastexam.ai/">Humanity’s Last Exam</a> (HLE) by the Center of AI Safety. This was released as AI models are getting too good at the public benchmarks. As a result, it’s hard to distinguish leading model performance and we need a new baseline. HLE collects about 3,000 challenging questions and most models struggle. Good old GPT-4o has a baseline accuracy of 3.3%. o1 comes out at 9.1%. Before people get too excited about the natural superiority of humans, it is instructive to take a look at the questions.</p><p>These questions are crazy hard. It’s not published what an average human would come out at but I would be amazed if they beat either of the 2 top reasoning models today. I am not sure I can.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/549/1*8oEMQZLWfSLTgvIUiWEbNw.png" /><figcaption>DeepSeek on top for text tasks</figcaption></figure><p>Notably, DeepSeek-R1 is at 9.4%. However the model is not multi-modal i.e. it can’t interpret images which some of the questions require so the comparison to o1 at 9.1% is not apples-to-apples but that really is an impressive showing. This made me curious, how well would this actually perform on local hardware. One of the really impressive innovations with DeepSeek-R1 was actually in the small model space — <a href="https://huggingface.co/deepseek-ai/DeepSeek-R1#deepseek-r1-distill-models">they took the large model and ‘distilled’ it down to smaller models</a>. A large slow model taught a smaller model how to perform like it and Qwen, one of my favourite small models, was taught how to perform like the big DeepSeek-v3 model. This distilled model is small enough to run fast and locally on an nVidia 4090, a high end consumer GPU from 2 years ago. This is exactly what I did. This will likely also work well on an Apple M4 with more than 32GB of system RAM.</p><p>Assuming you have some form of local inference server, running it is as simple as:</p><pre>$ ollama run deepseek-r1:32b</pre><p>Then you can go question hunting, the Hugging Face site for Humanity’s Last Exam had a parquet file for the public data set.</p><p><a href="https://huggingface.co/datasets/cais/hle">cais/hle · Datasets at Hugging Face</a></p><p>I love duckdb which allows for local manipulation and querying of parquet files among other things. We can use that to find a few questions and give DeepSeek a test.</p><pre>$ brew install duckdb<br>$ duckdb # launch the duckdb shell<br>.mode line<br>SELECT question, answer<br>  FROM &#39;test-00000-of-00001.parquet&#39;<br>  WHERE question LIKE &#39;Which%&#39;;</pre><p>I chose a few questions that specifically tested reasoning as opposed to fact recall. It’s not reasonable to assume that a small distilled model is going to recall obscure facts. For example, one of the questions in the HLE dataset is <em>Who was the judge of Bártfa (now Bardejov) in 1461?</em> This does not strike me as a great model benchmark as a real knowledge system that is expected to answer this type of question would clearly rely on search or some form of database. Instead the one below seems like a good start. I have pasted the chain of thought and the answer from deepseek-r1:32b verbatim. There is a separator between the question and the model answer if you want to pit your organic neurons vs the artificial ones executing on my graphics card.</p><p><strong>QUESTION:</strong> Which animal might be able to hear human muscle twitches?</p><p>Answer Choices:<br>A. Dog<br>B. Bat<br>C. Herring<br>D. Whale<br>E. Human</p><blockquote>&lt;think&gt;<br>Okay, so I’m trying to figure out which animal can hear human muscle twitches. The options are dog, bat, herring, whale, and human. Let’s break this down step by step.</blockquote><blockquote>First, I need to understand what human muscle twitches sound like. When our muscles twitch, especially when we’re relaxed or sleeping, they make very faint sounds that humans can’t usually hear. These sounds might be in the range of low frequencies because <br>muscles aren’t making loud noises.</blockquote><blockquote>Next, let’s consider each animal:</blockquote><blockquote>1. **Dog**: Dogs have a broader hearing range than humans, up to about 45 kHz. However, human muscle twitches are probably too quiet and low-pitched for even dogs to hear distinctly.</blockquote><blockquote>2. **Bat**: Bats use echolocation and can detect high-frequency sounds up to around 100 kHz. Muscle twitches are likely too low in frequency and volume for bats to notice, especially since they’re more attuned to echoes from their ultrasonic calls.</blockquote><blockquote>3. **Herring**: Herring communicate using sounds in the range of about 50–120 Hz. Human muscle twitches might fall within this range, making it possible for herring to detect them if close enough.</blockquote><blockquote>4. **Whale**: Whales can hear very low-frequency sounds, down to as low as 7 Hz or even lower. Since human muscle twitches produce low-frequency vibrations, whales could potentially hear these sounds, especially in the water where sound travels well.</blockquote><blockquote>5. **Human**: Humans can’t typically hear their own muscle twitches because these sounds are too faint and often fall below our hearing threshold, which starts around 20 Hz.</blockquote><blockquote>Considering all this, both herring and whales have the potential to detect human muscle twitches due to their ability to hear low-frequency sounds. However, whales are more specialized in detecting such low frequencies compared to herring.<br>&lt;/think&gt;</blockquote><blockquote>Based on the analysis of each animal’s hearing capabilities relative to the frequency and volume of human muscle twitches, the most likely candidate is:</blockquote><blockquote>**Whale**</blockquote><blockquote>Whales possess an extraordinary ability to detect very low-frequency sounds, which makes them capable of sensing the faint, low-pitched vibrations caused by human muscle twitches.</blockquote><p>For the uninitiated, the text between &lt;think&gt;…&lt;/think&gt; is the model reasoning at work, it’s the working behind the scenes that a Chain-of-thought model does to reason about the question. Some questions don’t trigger it if they are simple in which case the model behaves like a vanilla LLM e.g. GPT-4o. If you have used o1, something like this is happening behind the scenes though what is actually going on is not revealed to the end user as transparently. The text after the &lt;/think&gt; tag is the answer, i.e. what you would see if you used o1 or a similar model.</p><p>At any rate, it finally ends and proposes <strong>Whale</strong> which is the correct answer as per the data set.</p><h3>A failed example</h3><p>There are of course many where it can’t reason well enough to solve them though that is also depressingly true of my organic brain. I found it instructive that the model <em>thinking</em> for those questions actually taught me about higher dimensional math, linguistics and physics. I did manage to find an example where it failed in an unreasonable way.</p><p><strong>QUESTION: </strong>Who is the mythological character in this poem, where is it situated, and what form is the poem written in?</p><p>The half-dark hides a man with eyes like voles,<br>purblind, unsmiling. If thoughts were riverbanks<br>he would have felt them split and break one endless<br>loveless night, and found his memories swirled<br>to mud; all washed-up wildness, clagged and silted.<br>That girl, a slight figure lost in light,<br>slipped out towards the sun to tease the soil,<br>make roots. And he was left to hawk and spit<br>alone where men spin jukebox coins on graves.<br>The devil’s luck. It’s closing time, but still<br>he dreams of messages, that shush of feathers;<br>feral postcards from his fucked up underworld.</p><p>Answer Choices:<br>A. Hades, Bar, sonnet<br>B. Orpheus, Central Park, iambic pentameter<br>C. Persephone, Arcade, iambic pentameter<br>D. Hades, Arcade, sonnet <br>E. Hades, Underworld, sonnet<br>F. Orpheus, Riverbank, sonnet</p><blockquote>&lt;think&gt;<br>Okay, so I’ve got this question about a poem, and I need to figure out which mythological character is in it, where the setting is, and what form the poem takes. The answer choices are from A to F, each pairing a different character with a location and a <br>poetic form.</blockquote><blockquote>First, let me read through the poem carefully to get a sense of its content and imagery. The first line mentions “The half-dark hides a man with eyes like voles.” Voles are small rodents, so maybe this man has small or beady eyes? He’s in a half-dark place, <br>which could suggest somewhere dimly lit or twilight.</blockquote><blockquote>Next, it says he’s “purblind,” meaning partially blind or having poor vision. He doesn’t smile, and if thoughts were riverbanks, they would split and break on an endless loveless night. That makes me think of someone who is in a place where time feels <br>endless, maybe stuck in some kind of limbo or darkness.</blockquote><blockquote>His memories are swirled to mud, washed-up wildness, clagged and silted. This imagery sounds like something from a riverbank or perhaps an underworld with muddy banks. Then there’s mention of a girl who is a slight figure lost in light, slipping out towards <br>the sun to tease the soil and make roots. So she’s moving towards the sun, possibly leaving the darkness.</blockquote><blockquote>The man is left alone where men spin jukebox coins on graves. Jukeboxes are usually in bars or arcades, so that could place him in a bar or an arcade setting. “Closing time” suggests it’s night, and he’s still there. He dreams of messages, the shush of <br>feathers; feral postcards from his fucked up underworld.</blockquote><blockquote>Putting this together, the man is in a dark, possibly underground place (underworld), and the girl might be Persephone who goes to the sun, while the man is Hades, left alone in the underworld. The setting where men spin coins on graves could be an arcade, as <br>arcades have games with coin inserts.</blockquote><blockquote>Looking at the answer choices: E says Hades, Underworld, sonnet; D says Hades, Arcade, sonnet. The poem has 12 lines, so if it’s a sonnet, which is typically 14 lines, maybe it’s an incomplete one or structured differently. Alternatively, perhaps it’s another <br>form.</blockquote><blockquote>Wait, the poem starts with two-line stanzas, but each line seems to be in iambic pentameter? Let me check: “The half-dark hides a man with eyes like voles,” that’s 10 syllables, so yes, iambic pentameter. So maybe it’s written in that form.</blockquote><blockquote>Looking back at the answer choices, B is Orpheus in Central Park, which doesn’t seem to fit because the poem mentions an underworld and Hades. C is Persephone in an arcade, but the main character seems male (Hades). D and E have Hades: D says Arcade, sonnet; <br>E says Underworld, sonnet.</blockquote><blockquote>The mention of graves and jukebox coins on them makes me think more of an arcade setting rather than the actual underworld. Also, “devil’s luck” could tie into the idea of being in a place where fate is involved, like an arcade with games of chance.</blockquote><blockquote>Wait, but Hades is usually associated with the underworld, so maybe it’s set there, and the poem is structured as a sonnet? The poem seems to have 12 lines, which isn’t typical for a sonnet (which has 14), unless it’s a variant. Alternatively, perhaps it’s <br>not a traditional sonnet but uses iambic pentameter.</blockquote><blockquote>Alternatively, maybe the setting is an arcade that represents Hades’ realm metaphorically. But I’m leaning towards E because Hades is in the underworld, and the poem’s imagery fits that better than an arcade. The girl going to the sun could be Persephone <br>leaving him for a while each year.</blockquote><blockquote>Wait, but if it’s set in the Underworld, then E makes sense: Hades, Underworld, sonnet. But the mention of jukebox coins on graves might point more to a real-world setting like an arcade or bar, which would make D plausible. Hmm, this is confusing.</blockquote><blockquote>I think I’ll go with E because the imagery of mud, riverbanks, and the underworld elements are stronger in suggesting Hades’ realm rather than a modern place. The poem’s structure seems to be in iambic pentameter, so maybe it’s a sonnet despite having fewer <br>lines.<br>&lt;/think&gt;</blockquote><blockquote>The mythological character in the poem is **Hades**, situated in the **Underworld**. The poem is written in **sonnet** form.</blockquote><blockquote>Answer: E. Hades, Underworld, sonnet</blockquote><p>The correct answer is:</p><p>A. Hades, <strong>Bar</strong>, sonnet.</p><p>It is interesting that the model considered that:</p><p><em>…the mention of jukebox coins on graves might point more to a real-world setting like an arcade or bar…</em></p><p>as it reasoned about the question but missed the line <strong>It’s closing time</strong> something I imagine an attentive human would spot.</p><p>Interesting that o1-pro also gives me the same answer as R1 reasoning that: <br><em>Although the poem uses modern touches (“jukebox coins,” “closing time”), it’s a contemporary reimagining of Hades’ domain</em> <br>thus dismissing it away.</p><p>This question is intriguing as it seems to want to lead the model down the path, filling its activation layers with the context of hell and the underworld and successfully traps it. I wonder if the larger DeepSeek models would perform better here-remember this 32B model is running in seconds on a GPU in my home.</p><h3>In summary</h3><p>DeepSeek’s unexpected emergence has led to a <strong>$1 trillion</strong> slump in AI-linked share valuations, shaking confidence in big-name tech. Yet, the lab’s open source stance hints at a democratized future where smaller firms can feasibly build — or distill — high-performing models. As seen in DeepSeek-R1’s chain-of-thought demonstrations, transparency in AI design can be as disruptive as performance alone. With demanding benchmarks like Humanity’s Last Exam now in play, the global AI race is sure to intensify, but those shocks could ultimately yield a more innovative and widespread adoption of AI technologies. I am excited to see open source, wherever it comes from, take a seat at the table.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2d745bddb826" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Quantum Computing, why now?]]></title>
            <link>https://rahulpowar.medium.com/quantum-computing-why-now-28a10db383bb?source=rss-376557eaabb------2</link>
            <guid isPermaLink="false">https://medium.com/p/28a10db383bb</guid>
            <category><![CDATA[cryptography]]></category>
            <category><![CDATA[quantum-computing]]></category>
            <category><![CDATA[cybersecurity]]></category>
            <dc:creator><![CDATA[Rahul Powar]]></dc:creator>
            <pubDate>Wed, 15 Nov 2023 17:35:10 GMT</pubDate>
            <atom:updated>2023-11-15T17:35:10.384Z</atom:updated>
            <content:encoded><![CDATA[<p>Quantum computing is a buzzword that has been making the rounds in tech circles for a while now. But what does it mean for the world of cryptography today?</p><figure><img alt="ChatGPT claims “The color palette and design aim to convey the advanced and mysterious nature of quantum computing.”" src="https://cdn-images-1.medium.com/max/1024/1*LUrlCRyugMjVQQuNJzlh2g.png" /><figcaption>ChatGPT / DALL-E generated. Overlap of QC and GenAI? 🤔</figcaption></figure><h4>A new computing paradigm</h4><p>Quantum computing is not just a faster version of our current computing systems; it’s an entirely new paradigm. Unlike classical computers that use bits (0s and 1s) to process information, quantum computers use quantum bits or <em>qubits</em>. This allows them to perform some complex calculations exponentially faster than classical computers. However, that does not make them generally useful as per our current understanding so they are most likely to be used as a supplement to traditional computing in the future.</p><h4>Inevitable but not yet useful</h4><p>Quantum computing is more a question of “when” than “if.” The state of the art today is still in its infancy, with companies like IBM and Google making incremental progress. However, we’re far from having a quantum computer that can outperform classical computers in practical tasks as the utility is largely a function of the number of entangled qubits in a quantum computer. Entangling qubits required hard physical problems to be solved and this is still largely the realm of R&amp;D and new physics — think lasers and super cooled helium, not hoodies and laptops.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*Hu_NRetaiQcpljoD1EibDA.jpeg" /><figcaption><strong>Google’s Sycamore quantum computer, 2021</strong></figcaption></figure><h4>The science is still young</h4><p>The computer science behind quantum computing is not yet well-understood. Researchers are still grappling with how to build efficient algorithms, error correction methods alongside the hardware to support these quantum systems. As a traditional programmer, if you have <a href="https://learn.microsoft.com/en-us/azure/quantum/tutorial-qdk-intro-to-katas">ever tried Q#</a>, you would recognise just how unusual programming with entanglement is.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3iAyBsMcJD_0XvBwqfnmwQ.png" /><figcaption>Full stack, meet Q#</figcaption></figure><h4>The Cryptography Conundrum: Shor’s Algorithm</h4><p>One of the most significant concerns about quantum computing is its potential to break asymmetric cryptography. Shor’s algorithm, developed by mathematician <a href="https://www.youtube.com/watch?v=6qD9XElTpCE">Peter Shor</a>, can factor large numbers exponentially faster than the best-known algorithms running on a classical computer. This poses a threat to cryptographic systems like RSA, which rely on the difficulty of factoring large composite numbers.</p><h4>Shor’s Algorithm for the normal person</h4><p>If you want to build an intuition for what new type of problem solving quantum computation might herald, trying to understand Shor’s Algorithm is a great place to start.</p><p>Imagine you have a massive jigsaw puzzle. Solving it with a classical computer is like having one person work on it slowly, piece by piece. Shor’s algorithm, when run on a quantum computer, is like having an army of people simultaneously working on different sections of the puzzle, sharing information instantaneously. This is achieved by the quantum computer because all the qubits are entangled so they can be in many states at the same time and elegant math can tease out the correct answer by performing a clever collapse of the system state. For a more accurate and in depth explanation, I present you with 17 minutes of minutephysics.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FlvTqbM5Dq4Q%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DlvTqbM5Dq4Q&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FlvTqbM5Dq4Q%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/4a3aabd6b3696b7fa2f24f00d31fb15b/href">https://medium.com/media/4a3aabd6b3696b7fa2f24f00d31fb15b/href</a></iframe><h4>Quantum Cryptography vs. Quantum-Safe Cryptography</h4><p>While quantum cryptography uses the principles of quantum mechanics to secure a communication channel, quantum-safe cryptography aims to develop cryptographic algorithms that are secure against the computational approaches of quantum computing. The former is more of a niche application, while the latter is a broader approach to securing data in a post-quantum world and is typically being designed for use on traditional i.e. todays, computers.</p><h4>The NIST initiative and why today matters</h4><p>The National Institute of Standards and Technology (NIST) is already in the process of standardising new cryptographic algorithms that would be secure in a quantum world. The idea is to prepare for the future by securing our secrets today, even if practical quantum computers are still on the horizon.</p><p>The key reason is that secrets encrypted today using an approach vulnerable to Shor’s approach or its successors could be attacked in the future if someone has a record of the encrypted secrets. This may not be critical in some cases e.g. cryptographic signatures, but they could be applied to unravel data like state secrets that may still be relevant at a future date if an attacker has preserved the encrypted and currently unknowable data from today.</p><p>Of course, it is entirely possible that these new approaches such as Lattice-based cryptography may have as yet undiscovered quantum (or traditional) attacks but this is not so different from the operating assumption that factoring large primes <em>seems</em> to be a hard problem for a classical computers. The fact that this was never mathematically provable did not stop the utility of current approaches so one could argue that progress is being made.</p><h4>In conclusion</h4><p>Quantum computing is an inevitable advancement that poses both challenges and opportunities. While it threatens current cryptographic systems, it also pushes us to innovate and develop more secure methods of protecting information. The time to act is now, not when quantum computers become a practical reality.</p><p>By understanding both the potential and the limitations of this emerging technology, we can better prepare for a future where quantum computing is the norm rather than the exception.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=28a10db383bb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Run ChatGPT… on your Gaming PC? Part 2]]></title>
            <link>https://rahulpowar.medium.com/run-chatgpt-on-your-gaming-pc-part-2-32e98ab507c9?source=rss-376557eaabb------2</link>
            <guid isPermaLink="false">https://medium.com/p/32e98ab507c9</guid>
            <category><![CDATA[generative-ai]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Rahul Powar]]></dc:creator>
            <pubDate>Tue, 14 Feb 2023 14:20:47 GMT</pubDate>
            <atom:updated>2023-02-14T16:12:24.720Z</atom:updated>
            <content:encoded><![CDATA[<p><a href="https://rahulpowar.medium.com/run-chatgpt-on-your-gaming-pc-part-1-87c9ac345cf8">In Part 1</a>, I provided a backstory to my desire to run large language models on relatively inexpensive local hardware. In this part, let’s drop the chat and get to code. I outline some basic instructions and snippets of code that lets us install and spin up a model that can spit out near real time completions on a typical 2022 spec gaming PC.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1RGHQw1Tl78b_i25NVfJRg.png" /><figcaption>Sample Q/A output.</figcaption></figure><h3>Prerequisites</h3><p>To make this work, you need a reasonably current NVIDIA Graphics card, a functional Python installation that can use it and some comfort with a shell terminal of your choice. Specifically:</p><p>a) A NVIDIA graphics card (GPU) with at least 8GB of VRAM so we can load a decent model. Unfortunately, most computational use of GPUs relies on CUDA which is supported on NVIDIA cards. 8GB should be easy as it has been standard on them for some time and if your PC is current and has a dedicated GPU, you probably have this.</p><p>b) Python 3.9. While version 3.10 is current as of this writing, it is not well supported by some of the libraries we need to install to get this going. If you don’t have a Python install, on Windows you can <a href="https://apps.microsoft.com/store/detail/python-39/9P7QFQMJRFP7">install 3.9 from the Microsoft Store</a> or any other way you want. On Linux, getting the specific version may need an <a href="https://github.com/pyenv/pyenv">environment manager like pyenv</a>. If you are running Linux with a serious GPU, you likely know what to do for your distribution.</p><p>c) CUDA allows applications like deep learning to use your GPU. You need version 11.7 as that is the version the next package we need currently supports. <a href="https://developer.nvidia.com/cuda-11-7-0-download-archive">Install it from here</a>.</p><p>d) PyTorch is a machine learning framework. In principle we can run RWKV-LM on other frameworks such as Tensorflow, PyTorch is quite flexible and relatively easy to setup. In your terminal of choice, run</p><pre>pip3 install torch torchvision torchaudio \<br>  --extra-index-url https://download.pytorch.org/whl/cu117</pre><p>e) To make this easy, we are going to use a wrapper <a href="https://github.com/harrisonvanderbyl/rwkvstic">RWKVSTIC</a>. To get it working, we need to install a few more Python packages:</p><pre>pip3 install inquirer tqdm transformers scipy</pre><p>then the package itself.</p><pre>pip3 install rwkvstic</pre><p>f) Next we need to pick a pre-trained model we want to use. This is where we need to make some choices based on the hardware we have available and the performance we want. <a href="https://huggingface.co/BlinkDL/rwkv-4-pile-14b/resolve/main/RWKV-4-Pile-14B-20230204-7324.pth">https://huggingface.co/BlinkDL</a> has a list of models with a different number of parameters. e.g. <br><strong>rwkv-4-pile-14b</strong> is a RWKV-4 model (the one we want) trained on <a href="https://pile.eleuther.ai/">the Pile</a> with 14 billion parameters. As this is in active development and training, you should check each model for the current best checkpoint. As of writing, let’s try the <a href="https://huggingface.co/BlinkDL/rwkv-4-pile-3b/blob/main/RWKV-4-Pile-3B-20221110-ctx4096.pth">10–11–2022 checkpoint</a>. Download this and let’s write a simple app to load it and make our first text generation.</p><h3>RWKVSTIC</h3><p>While you wait, you will note this is a 6GB download. These are the weights we need to load into VRAM and is directly related to the hardware we need to run this live. More parameters is in principle is more capable model and the package rwkvstic provides flexibility when it comes to VRAM usage but they usually come at a cost— you can:</p><p>a) Split the load across multiple GPUs (if you have them)</p><p>b) Stream the model from system RAM (slow, in my testing 30x slower)</p><p>c) Run at half resolution (odd results at the time of writing)</p><p>This is the current requirement if running at 16 bit resolution which is a reasonable balance.</p><pre>Model | bf16/fp16<br>------|----------<br>14B   | 28GB  <br>7B    | 14GB <br>3B    | 6GB  <br>1.5B  | 3GB</pre><p>As you can see, 3 billion is safe for most current GPUs though if you have a high-end GPU, you can try the 7 billion model. Unfortunately, all consumer cards as of Feb 2023 cap out at 24GB of VRAM which means you need to use one of the alternatives approaches above, run 2 cards, or wait for the rumoured NVIDIA TITAN Ada release in a few months with 48GB of VRAM.</p><h3>qa.py</h3><pre>import torch<br>from rwkvstic.load import RWKV<br>from rwkvstic.agnostic.backends import TORCH<br><br># this is the dtype used for trivial operations, <br># such as vector-&gt;vector operations and is the dtype that will <br># determine the accuracy of the model<br>runtimedtype = torch.float32 # torch.float64, torch.bfloat16<br><br># this is the dtype used for matrix-vector operations, <br># and is the dtype that will determine the performance <br># and memory/VRAM usage of the model. The alternatives<br># will use more memory but may provide better results<br>dtype = torch.bfloat16 # alternatives are torch.float32, torch.float64<br><br># use your GPU, CPU forward passes are about 30x - 100x slower<br>useGPU = True<br><br># load the checkpoint we downloaded<br>model = RWKV(&quot;./RWKV-4-Pile-3B-20221110-ctx4096.pth&quot;, mode=TORCH, useGPU=useGPU, runtimedtype=runtimedtype, dtype=dtype)<br><br># configure this as a Q/A bot<br>question = &quot;Who is Tim Cook?&quot;<br><br># tune the parameters<br>number = 200 # the number of tokens to generate<br><br>stopStrings = [&quot;\n\n&quot;] # When read, the model will stop generating output<br>stopTokens = [0] # when the model has generated any of these tokens, it will stop generating output<br><br># we want a factual bot so let&#39;s tone down the temperature to get a bland but hopefully correct answer<br>temp = 0.4 # the temperature of the model, higher values will result in more random output, lower values will result in more predictable output<br>top_p = 0.8 # the top_p of the model<br><br># context is the magic that sets up the forward pass<br>model.loadContext(newctx=f&quot;Q: {question}\n\nA:&quot;)<br>answer = model.forward(number=number, stopStrings=stopStrings, stopTokens=stopTokens, temp=temp, top_p_usual=top_p)<br><br>print(&quot;Q:&quot;, question)<br>print(&quot;A:&quot;, answer[&quot;output&quot;]) # the generated output</pre><pre>$ python3 qa.py<br>init RWKVOPS, from super<br>100%|████████████████████| 582/582 [00:00&lt;00:00, 2578.22it/s]<br>100%|████████████████████| 11/11 [01:50&lt;00:00, 10.04s/it]<br>Q: Who is Tim Cook?<br>A:  Apple&#39;s CEO.</pre><h3>Do I really need a GPU?</h3><p>Note, you can run this on a CPU backend by changing the <strong>useGPU</strong> boolean to False but prepare to wait — a forward pass on a i7–6850K CPU @ 3.60GHz takes around 5 minutes, expect similar times when running on the Apple M1 Max.</p><h3>Measuring performance</h3><p>While it is running, you can view the resource usage on your system to get a feel for the requirements of the model. There are a number of ways to do this but keeping it in Python land, we can use the <a href="https://pypi.org/project/Glances/">glances</a> module with an optional <a href="https://pypi.org/project/py3nvml/">py3nvml</a> package that gives us GPU usage.</p><pre>$ pip3 install glances windows-curses py3nvml<br>$ python3 -m glances</pre><h3>Next</h3><p>While this is 17 lines of real code, most is boilerplate and configuration. By manipulating the loaded model, the <strong>temp</strong> and the context passed to <strong>model.loadContext</strong>, you can get these 17 lines of code to emit many reasonable and useful things beyond a Q/A bot. E.g. you can make a classifier or a code generator.</p><p>This entire project is currently in active development and APIs and checkpoints are updated on a daily basis. It does however give the first hint of a viable open alternative to the state-of-the art models that are deployed behind closed doors.</p><p>What a time to be alive.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=32e98ab507c9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Run ChatGPT… on your Gaming PC? Part 1]]></title>
            <link>https://rahulpowar.medium.com/run-chatgpt-on-your-gaming-pc-part-1-87c9ac345cf8?source=rss-376557eaabb------2</link>
            <guid isPermaLink="false">https://medium.com/p/87c9ac345cf8</guid>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Rahul Powar]]></dc:creator>
            <pubDate>Sat, 11 Feb 2023 21:15:20 GMT</pubDate>
            <atom:updated>2023-02-14T14:21:33.587Z</atom:updated>
            <content:encoded><![CDATA[<p>It’s click bait, I apologize. OpenAIs models are not in the public domain so while they are now easier to use and more available than ever before, this is not something you can ‘download’ and ‘run’. There are reasons why this would be desirable that I will get to later. While you can’t do this with ChatGPT, there seems to be something else on the horizon that may open up options for state of the art generative language models.</p><p>First, a bit of background. I was lucky enough to get access to the closed beta of GPT-3 on the 25th of July 2020 and it blew my mind. I spent many nights discovering the subtleties of what is now called prompt engineering and as API access was unmetered, I experimented with every use case I could think of. I created a bot that used tricks with dynamic temperature estimation powered by GPT-3 (I called it “GPT-3 Inception” in my user group feedback session) and put a Slack interface on it. It produced some of the most entertaining exchanges I have ever had with a machine, dynamically switching between fact and humor.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*u4dfnQuKD2pGqA-5aMVeaQ.png" /><figcaption>Tell me about particle physics, but make me laugh because it’s 2am. A 2020 vintage.</figcaption></figure><p>As expected, GPT-3 was made commercially available and my toys would now bust my monthly token credits in a few interactions. I was busy with my day job running <a href="https://redsift.com/">my SaaS business</a> so I reluctantly turned my bot off and starting using GPT-3 for some more specific NLP tasks.</p><p>GitHub Copilot launched later that packaged a fine tuned model for developers helping them write code faster. It was incredible but only for those who wrote code which remains a vanishingly small part of the population. Then ChatGPT arrived and blew up. It’s a GPT-3.5 with some tuning that makes it less general purpose but a great out of the box Q/A solution. Many of our human problems can be solved through chat so that limitation is not a huge issue for its general usefulness and it made a large number of people realize just how useful AI can be. For the more thoughtful among us, what this next industrial revolution means for knowledge work and the way western society and related economic activity is structured today. But back to technology, while its utility is shocking to people who have never touched modern AI directly, it’s not really conceptually new and more practically does not let me resurrect my personal AI.</p><p>There is a profound benefit to open innovation and the ability for the curious and the capable to lift the hood tinker. After GPT-3, a number of open source massive language models arrived on the scene including <a href="https://bigscience.huggingface.co/blog/bloom">BLOOM</a>, <a href="https://opt.alpa.ai/">OPT</a> from Meta.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fwi0M2J4uE5I%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dwi0M2J4uE5I&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fwi0M2J4uE5I%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/80c229f7a66322b703037b5103ff5c8b/href">https://medium.com/media/80c229f7a66322b703037b5103ff5c8b/href</a></iframe><p>These new models showed promise but there in is a new problem. Even if you know enough to spin up these models and run them, you need serious compute resources at your disposal. Your current gen laptop is not going to cut it. Many people know that these models are very large. That refers to the number of ‘parameters’ these models use to encode all of human knowledge. All 4 of them are in the order of 170 billion parameters. There are smaller models available but they do not offer the incredible performance of the full size models. There are 2 implications that stem from this very large size:</p><p>a) They are incredibly expensive to train. When GPT-3 came out, it was trained in a dataset that didn’t know about COVID because it had not happened yet. It took a while to retrain the model because it likely cost <a href="https://lambdalabs.com/blog/demystifying-gpt-3">around $4.5m to train the mode</a>l at the time. More recently, the team behind BLOOM estimated it cost €3M. It’s great the BLOOM and Meta foot the bill but this huge size leads to a second problem.</p><p>b) Inference, that is actually using the model, also requires a lot of capacity. The way these models work, they need to be loaded into memory and GPUs, specialized graphics processors normally used for games, need to perform billions of matrix operations to generate the text responses we finally see. GPUs operate on their own dedicated memory in most PCs and a consumer GPU is way too small to hold these 170 billion parameters. If I want to run my own BLOOM or OPT model, I need to get hold of a datacenter GPU that comes with tons of memory for just this task. State of the art would be a NVIDIA A100 with 80GB of memory. Unfortunately that is $50,000 for a GPU. In addition, 80GB is not enough, I will need around 8 of them and a server to host that many high bandwidth devices. Let’s say a personal <strong>capex of half a million dollars</strong> if I really want it <a href="https://medium.com/@rahulpowar/breaking-the-gigabit-75c14d34b88b">in my server rack</a> and am willing to cut a few corners. I could of course spin them up in the cloud, assuming I can get these A100 80GB (I hear they are only available on Azure), but at that point I am going to be paying more than a metered GPT-3.</p><p>One might reasonably ask why I would want to do this. There are 3 themes to this. Curiosity, Security and Performance.</p><p>Like many people, I get great satisfaction from being able to pop the hood on things that look like magic and understand them. At a more specific level, if I want to build a real AI for use by myself and my family, I don’t want our data to leave my network. We don’t have a retail smart speaker in the house for this reason. It also holds the potential for the next step up in human/machine interaction. Local models with dedicated computation can turn results around faster, if you want a natural conversation you need speed of response. These models can also be chained together to do more sophisticated things. If I want to live in the future, I got to get me one of those.</p><p>At some point, new GPUs or <a href="https://www.youtube.com/watch?v=5tmGKTNW8DQ">dedicated novel architectures</a> to solve this will emerge. But for the impatient, there is a new kid on the block that side steps this issue. Enter <a href="https://github.com/BlinkDL/RWKV-LM">RWKV-LM</a>.</p><p>The models discussed are massive because of their design. They are all <a href="https://blogs.nvidia.com/blog/2022/03/25/what-is-a-transformer-model/">transformers</a> which is the current hotness in deep learning. Great performance, but prohibitive cost in this natural language use case.</p><p>There is however an older style of generative AI — the Recurrent Neural Networks. RWKV-LM brings this older architecture up to date and carries along the benefit of size. It’s a LOT smaller. The largest model trained today is 14 billion parameters and it’s pretty impressive.</p><p><a href="https://www.youtube.com/shorts/4QXNoFMcv5A">RNN Language Model outperforms GPT and other transformers!</a></p><p>Now the comparison above is against lower end transformers so there isn’t an expectation that it is going to beat state-of-the-art. However, at 14 billion parameters, this can start to fit on a consumer grade high end PC a.k.a. my gaming PC. Did it work? 100%.</p><h3>Short form Q&amp;A.</h3><blockquote>Q: What is the meaning of life?</blockquote><blockquote>A: That’s a hard one to answer.</blockquote><blockquote>Q: What’s the secret to a good marriage?</blockquote><blockquote>A: To do your own thing.</blockquote><blockquote>Q: How do you get rid of a headache?</blockquote><blockquote>A: By getting up and moving around.</blockquote><blockquote>Q: What is the most important thing in life?</blockquote><blockquote>A: Relationships.</blockquote><h3>Ideation.</h3><blockquote><strong>Prompt:</strong> Here’s a short cyberpunk sci-fi adventure story.</blockquote><blockquote>The Story:</blockquote><blockquote><strong>RWKV-LM: </strong>Our World Is Not the Whole Story</blockquote><blockquote>Artist Brian decided to escape from his artificial world.</blockquote><blockquote>This short story explores the idea that our world is only one of many realities in which we live. This world is not the whole story.</blockquote><blockquote>I decided to explore the idea that our reality is not the whole story. I am not suggesting that this story is a serious reflection of our world. It is, instead, a lighthearted fantasy in which Brian tries to escape the artificial world he lives in.</blockquote><blockquote>I have always been fascinated by the idea that our world is only one of many realities that make up the universe. I think that this is a concept that is universal to our species.</blockquote><p>The smaller 7 billion model is available online on <a href="https://huggingface.co/spaces/Hazzzardous/RWKV-Instruct">Hugging Face</a>.</p><p>Want to run it on your own PC? Python instructions <a href="https://rahulpowar.medium.com/run-chatgpt-on-your-gaming-pc-part-2-32e98ab507c9">in Part 2</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=87c9ac345cf8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Breaking the Gigabit]]></title>
            <link>https://rahulpowar.medium.com/breaking-the-gigabit-75c14d34b88b?source=rss-376557eaabb------2</link>
            <guid isPermaLink="false">https://medium.com/p/75c14d34b88b</guid>
            <category><![CDATA[performance-testing]]></category>
            <category><![CDATA[networking]]></category>
            <category><![CDATA[4k-video]]></category>
            <category><![CDATA[gigabit-switch]]></category>
            <dc:creator><![CDATA[Rahul Powar]]></dc:creator>
            <pubDate>Sun, 14 Mar 2021 17:12:35 GMT</pubDate>
            <atom:updated>2021-03-14T17:24:57.798Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>What the options for editing 4K video on a new M1 powered Mac mini in a domestic environment? Does WiFi6 and a remote NAS provide enough bandwidth and what are the other options? How fast is 10Gb and do we really need it? TLDR; at the end.</blockquote><figure><img alt="Home rack" src="https://cdn-images-1.medium.com/max/1024/1*rlGqQI-8EQ2ipKfNe3WutQ.jpeg" /><figcaption>I know it’s messy. In my defence I do software.</figcaption></figure><p>I recently upgraded a number of networking components in my home network and realised I was probably close to peak network, at least in a domestic environment.</p><p>The key components are a pair of new <a href="https://store.ui.com/collections/unifi-network-access-points/products/unifi-6-long-range-access-point">UniFi 6 LR access points </a>with 4x4 MU-MIMO for my home WiFi, a UniFi 6 XG Switch that support 10Gb on copper and a custom Linux PC based network attached storage connected into this stack with a Intel X520 based 10Gb PCIe running Cat 7 via a SFP+adaptor. There is of course a bunch of other UniFi kit (yes, big fan) but for the purpose of this blog post, it is out of the signal chain.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dLR-bqFkZoS-uNuPLrrIIA.jpeg" /><figcaption>Probably as good as WiFi gets in 2021.</figcaption></figure><p>I also acquired a new M1 Mac mini which is one of the first desktop options from Apple to support WiFi 6 and I happened to wanted to edit some 4K video I took on my last pre COVID-19 vacation. I was posed with a fairly simple question; I have too many gigabytes of video to load onto my tiny Mac mini’s hard drive so how should I go about accessing these files for editing? Can I load them on my 10Gb connected NAS? Will it run at ‘full speed’ if I wire in the Mac mini? Or should I just plug in a Thunderbolt 3 hard drive as most people do?</p><p>First some expectations. For the scope of 4K video editing, the OS file system cache can’t help you much, there is too much data if you are scrubbing past many gigabytes of junk video. However, a lot of access is sequential or in large blocks. So we care about the real world sequential read performance of the underlying storage if we want to set some targets for the network access overlay. Spinning disks tap out at around 200MB/s to 300MB/s, a good SSD at 1000MB/s, a good M2 at 3000MB/s and state of the art Optanes can push a bit more.</p><p>All of these are <strong>bytes per second</strong> so if we translate that into networking speak, we need roughly 8 times as many bits per second to not bottleneck data rates before we layer on the other overheads and provisos. It’s apparent that we can’t get full speed on the fastest drives without going past 10Gbps but it’s not practical as very few people are stuffing their home NAS with Optanes. And if you are, you don’t need to read this post.</p><p>First let’s benchmark the disks involved. While we could do something elaborate, we really just want a sense of how fast these disks can read locally.</p><p>Let’s just time a copy of a 4K file to /dev/null. Let’s use a sample file GX010889.MP4 which is 4000Mb. On Linux, we can test the sequential read transfer rate of the disk by clearing the system cache and copying the file to null so let’s run this on the NAS itself.</p><blockquote>sudo sh -c “/usr/bin/echo 3 &gt; /proc/sys/vm/drop_caches”</blockquote><blockquote>time cp “./2019 Video PP — Monsuar 1 2/GX010889.MP4” /dev/null</blockquote><p>On the server, sitting on a 8TB spinning disk this takes 16.83s ± 0.3s implying a read rate of 239MB/s. Mostly what we expected from a high end spinning disk.</p><p>On the same server’s boot SSD, this takes 7.72s ± 0.003s so a solid read rate of 519 MB/s. Now this is slow but this NAS has a few old parts including this SSD which is now about 10 years old.</p><p><em>Note: If we let Linux cache the files, these numbers are closer to 8GB/second but let’s assume system cache is not realistically helpful for large video reads. This is why we purge them with /proc/sys/vm/drop_caches before the test.</em></p><p>Let’s look at the M1 Mac Mini itself. Zap the cache on macOS with…</p><blockquote>sudo purge</blockquote><p>… and time the same file. It’s 1.27s ± 0.07s. No surprise, this IO is faster than 10 year old hardware at 3152MB/s.</p><p>For reference, this beats the SSD on a MacBook Pro 2018 at 2927MB/s by a bit.</p><p>But let’s go back to the goal. No doubt the modern PCIe attached SSDs will be fast but they are also tiny even when upgraded. In the real world, users will be plugin in a storage device via one of the ports so what does that look like?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*9Tvt_f0NJ6w3cBlIawM73g.jpeg" /><figcaption>What these files were originally sitting on</figcaption></figure><p>This LaCie would be common and is what I carry around.</p><p>Under the cool industrial design, it holds 5TB on a 2.5 inch 5400 RPM Seagate spinning disk. With a Thunderbolt 3 interface, this LaCie attached drive takes 28.65s ± 0.01s for the file. This is basically a slow spinning disk so 140MB/s is actually expected and very close to the advertised sequential data rate.</p><p>Not, let’s bring on the network. The M1’s are the first in Apple’s range to support 802.11ax i.e. WiFi 6. A fast bandwidth test can be done with iperf3 by running <em>iperf3 -s</em> on the NAS and <em>iperf3 -c [nas] </em>on the Mac.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/973/1*0P9LeEdnVP85DUkf8S3g1A.png" /><figcaption>WiFi 6 connection at -56dBm signal</figcaption></figure><p>First sign of a bottleneck. This M1 is connected to one of the few WiFi 6 access points out there and it is tapping out at 437 Mb/s. Note this is <strong>bits (small b)</strong> so taking this at face value, the maximum transfer rate from a hard drive funneled through this network would be 55MB/s, probably closer to 50MB/s as jumbo frames aren’t an option for WiFi dropping the data transfer efficiency. This would be about 1/3 of the Thunderbolt drive’s measure performance. A previous generation 802.11ac client running the same test at similar signal strength hits 369 Mb/s i.e. an anticipated 42MB/s peak effective data transfer rate.</p><p>Let’s take a look at wired. On a physical link, we can software tweak the frame size. Switching to jumbo frames should lift the useful data rate from a <a href="https://www.gigabit-wireless.com/gigabit-wireless/actual-maximum-throughput-gigabit-ethernet/">94% theoretical maximum to a 99% theoretical max</a> so let’s ensure all devices between the M1 and the NAS are enabled for 9000 MTU frames. After tweaking adaptor settings we can verify they work by testing a large <em>ping</em> packet.</p><p>First make sure you can send large packets by tweaking a setting on macOS.</p><blockquote>sudo sysctl -w net.inet.raw.maxdgram=16384</blockquote><p>Then, we can issue a ping with a large payload.</p><blockquote>ping -D -s 8184 [nas]</blockquote><p>If that goes out without a “ping: sendto: Message too long”, large packets are supported between your host and the server. Testing the built in wired adaptor on the Mac give us 1 Gb/s performance as we would expect.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/985/1*5gPi4fYA09e6bleaEN3_zg.png" /><figcaption>iperf3 via the in built 1Gb port on a M1 Mac mini</figcaption></figure><p>But running the math, this does not seem enough for my disks. Even at 99% effective data rates with jumbo frames, this is basically a limit of 123MB/s and slightly slower than the LaCie external disk.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/1*scQzm6kOb4o11okd4uioFQ.jpeg" /><figcaption>Runs hotter than the Mac Mini. No joke.</figcaption></figure><p>If we want to go faster, we need a bit more hardware. The Mac mini does not support 10Gb networking but Thunderbolt 3 has more than enough bandwidth for it so let’s introduce a Sabrent 10Gb adaptor. Plugged in and configured for Jumbo frames, this is likely as fast as we can get to the data on the NAS.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/978/1*PVHQn1mK_CW17ljRElEaEA.png" /><figcaption>iperf3 via the Thunderbolt 10Gb adaptor on a M1 Mac mini</figcaption></figure><p>Looking at the numbers, we are at the limit of home networking. For reference, this is now <strong>22x</strong> the raw speed of WiFI 6 in my setup. So finally, does this translate into full speed access of the drive? We can simply mount the NAS drives on the Mac via the various connection options we now have, purge the caches on both hosts and run the copy tests.</p><p>Now with everything in table form:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Y1CG3qUV8_OlRK5xScOP7w.png" /><figcaption>We can consider the 140MB/s from the external LaCie hard drive a minimum speed requirement</figcaption></figure><p>Mostly yes. The Seagate disk is the key benchmark as that is where the data is really going to sit for me and likely anyone using this type of configuration. Full speed, or near full speed access is what we want and only the 10Gb adaptor gets us there on a Mac mini.</p><p>The SSD benchmark is a reference and shows us that both WiFi 6 and the in built gigabit network on the Mac mini is the limiting factor. On the SSD, the 10Gb connection tells a different story, it is noticeably slower across the network share than on the NAS itself and does not reach full potential even though there should be enough bandwidth. This seems to be entirely down to samba and the single core performance of the CPU. smbd was sitting at 95% serving the request. Muti channel samba could likely help but 10Gb switch ports are precious and I don’t really need to share faster than a spinning disk.</p><h3>What does it all mean?</h3><ol><li>WiFi, even the latest access points and clients in 2021 are nowhere near wired Gigabit performance in a real deployment. If your are relying on remote work with a NAS storing large files on any kind of drive less than 10 years old, this will be a bottleneck. As an aside, the WiFi 6 data rate limits in my test setup are in the order of the consumer internet data rates if you have fibre to the home or some other high bandwidth internet service. If you want to fully utilize your 1Gb internet, looks like you will have to run wires.</li><li>Wired Gigabit with jumbo frames is within shooting distance of an external spinning disk plugged into the Thunderbolt 3 port on an Mac mini M1. In numbers, this means it is ~20% slower than the slowest locally attached option.</li><li>10Gb with an external Thunderbolt 3 adaptor for the Mac mini gives you practically full performance, remotely, from even the fastest spinning disks. This costs, but is better than plugging slower portable HDDs into the Mac for editing in every way. If your NAS is Linux and using samba, you need a fast CPU to keep up if you plan on using SSDs in your NAS and want close to full performance.</li><li>You likely need 40Gb if you want to use the latest drive tech remotely at anywhere near locally attached speed. However, these drives are rarely available beyond 1TB and most motherboard don’t have support for more than a couple of M2s.</li><li>Comparing the benchmarks for these components that span over a 10 year window, we might be reaching local network bandwidth and data transfer limits with current approaches. Wireless will have to get about 20x faster than it is today to compete with copper wires. So apart from wired 10Gb becoming more commonplace over the next 5 to 10 years, it is likely that these data rates remain fairly flat across wireless, wired, spinning disk, SSD and M2 technologies. If you want speed, get your house wired up with Cat 7 cables and rest easy. I will update this prediction in 2031.</li></ol><p>It seems you, my only reader, made it to the end. If you are curious about what I was editing on my NAS, take a look at one of the most beautiful underwater habitats in the world, <a href="https://en.wikipedia.org/wiki/Raja_Ampat_Islands">Raja Ampat</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fYmGSw6FJezaZlScJN1UKg.jpeg" /><figcaption>Let’s break up some boring tech talk with a bucket list destination</figcaption></figure><p>While Raja Ampat is a rare protected reserve, across the world our seas are dying. We need to raise awareness of what we have and how easily we can lose it all. If you want to support these unique habitats, make a donation to <a href="https://oceanconservancy.org/">https://oceanconservancy.org</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=75c14d34b88b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Do I ‘have’ DMARC?]]></title>
            <link>https://medium.com/postmasters/do-i-have-dmarc-7024cfba27e5?source=rss-376557eaabb------2</link>
            <guid isPermaLink="false">https://medium.com/p/7024cfba27e5</guid>
            <category><![CDATA[email]]></category>
            <category><![CDATA[cybersecurity]]></category>
            <category><![CDATA[deliverability]]></category>
            <category><![CDATA[marketing]]></category>
            <category><![CDATA[dmarc]]></category>
            <dc:creator><![CDATA[Rahul Powar]]></dc:creator>
            <pubDate>Fri, 10 Nov 2017 11:51:51 GMT</pubDate>
            <atom:updated>2025-06-17T16:49:18.742Z</atom:updated>
            <content:encoded><![CDATA[<h4>DMARC is a widely supported, standards based approach to email security. However that statement can also often cause confusion as it can suggest that organisations just get it by default. In this post we look at what it really means to ‘have’ DMARC.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CMxI-q0DAMtcF-VGs10G0Q.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/photos/i--IN3cvEjg?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Evan Dennis</a> on <a href="https://unsplash.com/?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p><a href="https://redsift.com/guides/email-security-guide/dmarc">DMARC</a> is a widely adopted security standard that provides protocol level security. But what does that mean for you? Do you automatically have it? Maybe you have heard that Microsoft, Google or your Cisco appliance supports DMARC so do you, as a business, need to do anything? Don’t you just ‘have’ DMARC?</p><h4>Well, not quite.</h4><p>The DMARC standard has 2 parts, one for receivers and another for senders. The widely supported part is the receivers, practically all email receivers such as Microsoft, Google, Yahoo etc implement this part of it.</p><p>You can think about it in terms of HTTPS, the little green security lock that you see on all good web properties. Your browser supports it but if your bank does not enable it, it provides no benefit when you use your browser to access your account details.</p><p>To turn on DMARC for your organisation, you need to advertise that you are ready for it by advertising it on your DNS. This is the only way to turn on DMARC for your email as it is what all the mail receivers around the world are checking today. It is also the magic that tools like<a href="https://redsift.com/tools/investigate"> our DMARC checker use to test your current configuration</a>.</p><p>So you find a DMARC generator online, follow the recommendations and modify your DNS with the appropriate incantations. Now you have a magic piece of text that looks a little like:</p><p>v=DMARC1; p=none; rua=mailto:me@mydomain.com; ruf=mailto:me@mydomain.com; sp=none; ri=86400</p><p>… advertising to the world that you have adopted DMARC. Surely you have it now!</p><h4>Well, not quite.</h4><p>Your inbox at me@mydomain.com will now be spammed by servers all around the world with email traffic reports for mydomain.com — yet you have not actually asked them to do much with your chosen DMARC record. All the attacks that DMARC is supposed to protected against are still going through.</p><p>You are however in a fairly common situation. Out of the 52,000 websites that advertise DMARC today, 2/3 of them are stuck in this state. You have logs but logs don’t authenticate your email or stop impersonation. In the DMARC record, the magic is p=none — you are advertising a policy that says <em>do nothing if my email does not authenticate</em> (apart from log it). It is like having a firewall that allows all traffic in and out but gives you a log of the activity.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P8E7Qc-W6NWJwEmzovwqYQ.png" /><figcaption>Email from facebook.com is authenticated, no ifs and buts.</figcaption></figure><p>To really benefit from all the internet infrastructure that already exists around you, you need to move to an enforcing policy i.e. p=quarantine or p=reject. While you could just turn it on with a simple edit to the policy, most likely your email will just stop working and you will rapidly jump back to p=none. This is because all those <a href="https://blog.ondmarc.com/why-marketeers-need-dmarc-dd8c2e98da3f">vaguely anecdotal delivery problems</a> you may have heard about, marketing emails that get low deliverability scores, proposals that have gone to spam etc suddenly become hard failures. In your post-DMARC existence, there is <strong>no</strong> leeway, all those emails go <strong>nowhere</strong>.</p><blockquote>This is the real task at hand and an inbox of XML attachments will not get you far. Hidden amongst millions of DMARC data points, any one of those reports could contain a critical piece of your infrastructure that is failing to authenticate and not getting through.</blockquote><p>This is why tools such as ours exist and why most organisations of any scale end up using them to successfully implement and maintain their global email authentication layer.</p><p>There are also unfortunate limitations in the standard that make deployments challenging such as the limits of the foundational DNS based authentication technologies. Next generation solutions such as our service OnDMARC solve these problems so you can successfully deploy DMARC irrespective of your level of infrastructure complexity.</p><p>A happy and incredibly clear ROI positive <a href="http://www.computerweekly.com/opinion/Dmarc-email-validation-were-doing-it-all-wrong">benefit of authenticated email is that deliverability goes up</a> as the receiving server can now trust the email you actually send. <a href="https://support.google.com/mail/answer/81126">Google has been recommending this</a> for some time and as we move to a more hostile landscape, email authentication via DMARC is a big part of the fight back.</p><p>In summary:</p><ul><li>DMARC is protocol level protection but that does not mean it is ON for you by default.</li><li>Turning it ON will involve more than a single DNS change. <a href="https://ondmarc.com/?utm_source=rahul_have_dmarc&amp;utm_medium=blog">You want to be using the best tool available for your needs</a>.</li><li>If your email is not DMARC enabled, you are doing email wrong. Your organisation is not enjoying the level of visibility, security and deliverability it should be.</li></ul><p>It is worth mentioning that while all serious email security solutions implement DMARC but there are a few holdouts. <a href="https://www.gartner.com/doc/3256818/fighting-phishing-protect-brand">Gartner required DMARC support</a> in 2016 to rate providers <strong>leaders</strong> in their sector but legacy providers still exist. If your company is using outdated technology in this fast moving landscape, you are exposing yourself to unnecessary risk. At <a href="https://redsift.com/">Red Sift</a>, we partner with organizations of every size to solve the DMARC puzzle. <a href="https://redsift.com/contact">Talk to us</a> and we can provide you more information given your specific infrastructure.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7024cfba27e5" width="1" height="1" alt=""><hr><p><a href="https://medium.com/postmasters/do-i-have-dmarc-7024cfba27e5">Do I ‘have’ DMARC?</a> was originally published in <a href="https://medium.com/postmasters">Postmasters</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>