<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Passiona Cottee on Medium]]></title>
        <description><![CDATA[Stories by Passiona Cottee on Medium]]></description>
        <link>https://medium.com/@passiona?source=rss-5991475a1180------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*BTG1HKRANCZWwlvvdY3Kgw.jpeg</url>
            <title>Stories by Passiona Cottee on Medium</title>
            <link>https://medium.com/@passiona?source=rss-5991475a1180------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 01:51:03 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@passiona/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[1000 monkeys with 1000 typewriters]]></title>
            <link>https://medium.com/@passiona/1000-monkeys-with-1000-typewriters-2ec9a1156c97?source=rss-5991475a1180------2</link>
            <guid isPermaLink="false">https://medium.com/p/2ec9a1156c97</guid>
            <category><![CDATA[responsible-ai]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[ai-governance]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <dc:creator><![CDATA[Passiona Cottee]]></dc:creator>
            <pubDate>Tue, 07 Jul 2026 20:39:55 GMT</pubDate>
            <atom:updated>2026-07-12T18:21:39.912Z</atom:updated>
            <content:encoded><![CDATA[<p>A surprising diLeMMa uniting software engineers, lawyers, policymakers AND BIG TECH.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1aeQAuNS155iP3TnlB1eMg.jpeg" /><figcaption>“The Blurst of Times”, <em>The Simpsons, Season 4, Episode 17 (Mark Kirkland, 1993), </em>© The Walt Disney Company (owners of the <a href="https://simpsons.fandom.com/wiki/20th_Century_Fox_Television">20th Television</a> copyright) and Gracie Films.</figcaption></figure><p>AI governance has a dirty little secret keeping your best engineers up at night that you won’t find solutions for in global consultancies’ glossy agentic AI guardrail playbooks.</p><p>But they’re not alone. You’ve seen it in your feeds already. All the aggregation platforms (linkedin, medium, facebook) are rife with those posts, articles and ads that just don’t seem <em>right</em>. Content is being produced faster than ever and at volumes greater than humans can review. The uncanny valley is everywhere. AI slop is not just killing your favourite platforms it’s challenging organisational hierarchies as we know them.</p><p>By tracing how we got to here, the breadcrumbs emerge that can lead us out. So let’s start by unpacking this maelstrom.</p><h3>The perfect storm</h3><p><strong>Generative AI </strong>brought issues of authenticity and originality into everyday work. And whilst deep fakes had been around for over a decade, <a href="https://openai.com/index/chatgpt/">Wednesday 30 November, 2022</a> changed everything by putting large language models (LLMs) freely into our hands.</p><p><strong>Agentic AI</strong> scaled up the problem. And whilst bots had been helping us for years, Gates’ blog post on <a href="https://www.gatesnotes.com/ai-agents">Friday 10 November, 2023</a> heralded the age of agentic AI. Agents turbo-charge generative AI into multi-tasking monsters (<em>aherm, I mean</em>) systems capable of taking sequences of real-world actions (browsing the web, executing code, sending emails, interacting with external services) in pursuit of a goal, with limited or no human oversight at each step. The certainty of the command line era is over — they will take your prompt<em> on boar</em>d, but you know, whatevs, here’s what I’ve come back with… d’oh!</p><p>Driving both, <strong>Frontier models </strong>(the best-in-breed paid generative multimodal-models — think: chatGPT, Gemini, Claude) have subsequently crystallised a degree of State capture that would make Adam Smith roll in his grave. It is easy to see why. They instantly produce the written word in any style — meaning an equal pace of development across written software code and any other style of written content. And our largest corporations are all in. Peer-reviewed research (aka science) backs this trend, <a href="https://hai.stanford.edu/ai-definitions/what-are-scaling-laws">scaling laws</a> explain why there are just no long-term payoffs in training your own. We are all strapped into the product release cycles of a few major players (OpenAI, Google, Anthropic). The implications for regulated industry use are significant (<a href="https://opensourcinglaw.github.io/ai.law/04-AI_Incarnations.html#43-foundation-models-and-the-governance-crisis">AI governance of foundation models is a topic of its own</a>, so for now let’s note it).</p><p>For individuals, the productivity benefits are enormous. The brief period in history where humans had to write or code for themselves is coming to an end (<a href="https://www.youtube.com/watch?v=7QcYJ4us1PQ">Gen-x</a> shed <a href="https://www.economist.com/finance-and-economics/2025/05/08/why-gen-x-is-the-real-loser-generation">another tear</a>). But for organisations that utilise any form of software coding or writing in their products or services, it means they are facing an unfamiliar dilemma. Agentic systems challenge every existing assumption about accountability in AI.</p><p>Now, what’s the problem <em>exactly?</em> Using these tools to build anything with a front-end — websites, graphics and videos — are easily testable. You give Claude a mock-up or design still of a website with buttons etc, Claude builds AND it is plain <em>on the face of it</em> to see, touch and otherwise confirm if it works.</p><p>But for all the chats that lawyers, policymakers and journalists are using to co draft their briefs, research papers and articles; the 000’s of agent-LLMs the software engineers are deploying to write their code; it is in the volume of the content that lies the blessing and the tyranny.</p><p>So, here’s the rub — how do we evaluate the quality of the output when there is no front-end? The economics of the scale of creation needs an equal rate of scaling in reviewing or governance of the same.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/979/1*fmUQPt8oiATDm2RtgJgfrQ.jpeg" /><figcaption>“The Blurst of Times”, <em>The Simpsons, Season 4, Episode 17 (Mark Kirkland, 1993), </em>© The Walt Disney Company (owners of the <a href="https://simpsons.fandom.com/wiki/20th_Century_Fox_Television">20th Television</a> copyright) and Gracie Films.</figcaption></figure><p>If we look to how some of the globe’s largest software companies are responding, we see attempts are still in their infancy. Pouring petrol on the fire, some are even are measuring agentic AI impacts by number of PRs or # of lines of code they generate.</p><p>I’ve even seen some of our best and brightest in tailspins tweaking armies of consumer-facing chat agents to support products and services whilst simultaneously not swearing, not discriminating, not being incompetent at their actual task, not breaching privacy, not breaching confidentiality, not breaking industry-specific regulations they are operating in, not being unempathetic to one of the world’s most multicultural customer bases, all within expected response times comparable to their human alternatives. But these corporate actors are suffering from the same capture as State actors. No one dares say this tech is simply <em>not fit for purpose.</em></p><p>Meanwhile, peer-reviewed science says the <a href="https://www.nature.com/articles/s42256-026-01255-3">human brain is the most efficient computational device on a fixed energy budget</a>.</p><h3>A way forward</h3><p>As a cross-disciplinary professional (DataSci+Law) the call to arms is clear. After all, <a href="https://www.smithsonianmag.com/arts-culture/a-tribute-to-a-great-artist-steve-jobs-99783256/">Jobs did it by applying typesetting to revolutionise desktop computing</a>. Agents <a href="https://www.apolloresearch.ai/science/metagaming-matters-for-training-evaluation-and-oversight/">metagame</a> which means all the consultancy advice in the world on fixed guardrails won’t work nor will an occasional eval. <em>Ipso facto</em> let’s manage them like humans. And there’s plenty of approaches to apply in this space.</p><p>To make a humble start let’s combine the:</p><ul><li>precisional clarity of the formal role separation of the likes found in<em> ye olde</em> hard-copy publishing houses (researchers, copy writers, sub-editors, editors, fact-checkers, cutters, proofers)</li><li>unadulterable transparency of <a href="https://github.com/OpenSourcingLaw/ai.law">github version control</a></li></ul><p>into a 1000-agent publishing pipeline with a closed feedback loop: producer agents write, policing agents enforce standards — in this starting case a <a href="https://www.stylemanual.gov.au/">style manual</a>, but designed to enforce any standard — and evaluator agents score whatever matters to the organisation: compliance, cost, sales, even wellbeing outcomes. Those scores rewrite the next round of instructions. It’s the industry’s ‘PRs and lines of code’ problem, solved with an actual governance layer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*j3YyYFVVZXY5oYv_aOGTsQ.png" /><figcaption>Claude response to the author’s cowork sessions throughout June and July, 2026</figcaption></figure><p>But before I lose you in the details: the motivation, the goal, is — (<em>what’s the opposite of ‘ironically’? errrgh, don’t make me type it, *shudders*</em>) synergistically —<strong><em> </em>outcomes</strong><em>. </em>Meaning this experiment is not limited in scope to the 2x approaches above. Plenty more to try, not least of which are being explored by people working in this space longer than myself (see also <a href="https://www.amazon.com/Output-Outcome-Operating-Model-Age/dp/1966280238">Output to Outcome</a> + <a href="https://a.co/d/0ap44BgG">Defensive Tool Design</a>).</p><p>Similarly, this harness is designed for the type of role definition needed to support the specified output to achieve the desired outcome(s). This exemplar, taking a book as its output has adopted publishing type roles. By analogy, maturing a separation of roles to improve outcomes in the production of software code or policy writing is the intended lift+shift.</p><p>Staging the plan:<br><strong>Stage 1.</strong> Now — prove the concept with 3 Claude Cowork agents. <br>Comprised of 2x writers (1x consultant and 1x legal researcher)and 1x editor policing the style guide (status: complete).</p><p><strong>Stage 2</strong>. Next — get to ~10 agents.<br>Adding additional writers and policing roles, including a sub-editor that can rewrite the contributing agents copy beyond the scope of stage 1 editor’s remit of enforcing the style manual. Still Cowork manual setup (status: in progress).</p><p><strong>Stage 3</strong>. At scale — ~1000 agents<br>The full 1000 monkeys. Switching to Claude Code headless mode + an orchestration script to achieve the scale.</p><p><strong>How it’s going.</strong> The <a href="https://opensourcinglaw.github.io/ai.law/01-Introduction.html">book is live</a>, judge for yourself. Though beware, the end-to-end cohesiveness the sub-editor will bring (in stage 2) is missing — evident in non-sequential defining of acronyms and a few conceptual overlaps between chapters.</p><p><strong>[Spoiler alert] </strong>The lawyer immediately changed all the ‘data scientist accessible’ headings into impenetrable legalese in the pre-baseline run. These had to be rolled back (this is <em>exactly </em>why GITHUB versioning has been selected) and their instructions updated before re-running baseline.</p><p>Link to Stage 1 results write-up — incoming! Plus, what the architecture <strong>actually</strong> looked like for that round, just so you don’t waste your time correcting it in the comments in the meantime → <em>hold that thought!</em></p><h3>A little more on the meth</h3><p>Roles relative to outcomes — too often we prompt to achieve an output. Aside from fundamentally flying in the face of inferential engines (<a href="https://opensourcinglaw.github.io/ai.law/01-Introduction.html#1-introduction-a-world-remade-by-inference">check the docs</a>), this approach seeks to explore if <em>roles </em>better align with outcomes. In this stage 1, the measured outcome is to reduce % of AI-detected work in outputs, in other words: to eliminate AI slop in AI writing whilst minimising human supervision to do the same. Energy/cost consumption is the next most tangible real-world impact of this technology (<a href="https://www.brockovichdatacenter.com/#map-section">Erin Brokovich, I’m looking at you</a>) so we will step into that outcome measurement in stage 2 execution but will likely not be fully realised until stage 3.</p><p>Kicking the tyres — for the &lt;s&gt;haters&lt;/s&gt; critics</p><p>How is this different from the <a href="https://ghuntley.com/ralph/">wiggum loop</a>?<br>Early peer review of this approach came back with:</p><blockquote>What makes this different from the classic Ralph Wiggum loop and more like the karpathy loop is that this is not a binary yes/no … It is a score…<br>What makes it different to a karpathy loop is that this is not for model training but document improvement…<br>So better advice/more realistic prose/more critical feedback/[better software code] whatever we are optimising for</blockquote><p>Thanks for reading!</p><p><strong><em>Disclaimers:</em></strong><em><br>This is my first Claude co-work project!<br>The opinions I express in this article are my own and are not representative of my employer. All work has been performed entirely in my own time and on my own devices.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2ec9a1156c97" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[We need to spend more time talking about data science failures]]></title>
            <link>https://becominghuman.ai/we-need-to-spend-more-time-talking-about-data-science-failures-4f53040b529e?source=rss-5991475a1180------2</link>
            <guid isPermaLink="false">https://medium.com/p/4f53040b529e</guid>
            <category><![CDATA[fail]]></category>
            <category><![CDATA[government-services]]></category>
            <category><![CDATA[government-administration]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Passiona Cottee]]></dc:creator>
            <pubDate>Fri, 29 Dec 2017 10:50:10 GMT</pubDate>
            <atom:updated>2017-12-30T18:54:57.154Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aBG_gdHLlJeIvpAIuehfOA.jpeg" /></figure><p>In <em>Star Wars: The Last Jedi</em>, Yoda reminds us, “the greatest teacher, failure is”. As a student of data science, sporting a ‘data science’ google alert, my news feed brims with the boasts of data science research applications. There’s very few disclaimers and even fewer failures (a phenomena that has not escaped <a href="https://www.forbes.com/sites/kalevleetaru/2017/07/30/from-subject-to-soundbite-results-to-hype-communicating-data-science/#311d512c6c35">this guy</a>, either, although <a href="http://aiweirdness.com/post/160776374467/new-paint-colors-invented-by-neural-network">this gal</a> is a delightful exception). Having spent almost two years applying data science in the domain of the administration of public services, I’m going to put out there three lessons no-one should learn the hard way. Because I did. And it would be a waste.</p><p>First, a little primer on <em>how</em> machines learn. They learn much like children do. Take the example of a picture book of animals, labelled with their names. We show the child the picture and say the animal’s name. We repeat the process until a child sees, for example, a cat on the street <em>without </em>the label, is able to recognise it as a cat, and exclaims “cat!”. In the class of machine learning referred to as supervised learning, the picture book with the labels, is the training set of data. And the cat on the street is the test set of data. Machine learning accuracy is measured by the rate at which it sees an animal without the label <em>and</em> recognises it correctly.</p><p><strong>Lesson 1: Context (there is none)</strong></p><p>In machine learning there is no context <em>beyond </em>what is shown to the machine. In the example of our child and the picture book, imagine the book <em>only </em>contains pictures of cats. It is entirely possible that a child, out in the world, having spied a four-legged furry thing with two ears and tail, exclaims “cat” when they are actually looking at a dog. The same applies to machine learning algorithms. It knows no external truths.<a href="https://www.theatlantic.com/technology/archive/2017/05/when-a-robot-names-a-new-color-of-paint/527421/"> And here is a really very *funny* experiment that illustrates this.</a></p><p>In the domain of the administration of public services, the context that is missing usually is policy. These administrative datasets typically contain that ‘<em>person a</em>’ received ‘<em>service b</em>’ on ‘<em>date c</em>’. There is no field denoting that this took place during ‘<em>policy setting d</em>’. It’s important because a change in policy could be material on the criteria for persons receiving ‘<em>service b</em>’. It’s also important because that’s the first thing a policy person is going to say. And, if your’e not ready for it, it can sink your work.</p><p>Why? How it plays out in the data is a whole lot of ‘<em>service b</em>’s administered up until the date of commencement of ‘<em>policy setting d</em>’. Followed by a substantial decline in ‘<em>service b</em>’ after that date (for example). In the absence of policy information, the machine learning algorithm is looking for explanations in the data it can see to account for that decline. Anything it finds will not be valid. It will call a dog a cat. And you’ve set it up for failure.</p><p>It’s not hard to engineer features representing policy. Is_policy_setting_D becomes a binary (or boolean) field added to the dates, as ‘1’ (or ‘true’) for the dates it was in force. Do that. The policy person will ask. And you will be ready.</p><p><strong>Lesson 2: Correlation not causation</strong></p><p>This one seems obvious. There are plenty of articles warning of the dangers of <a href="http://www.di.ens.fr/users/longo/files/BigData-Calude-LongoAug21.pdf"><em>how easy it is to find correlations in big data.</em></a><em> </em>What could I possibly add? It has to do with the context within which you are performing your work, who your audience is and what you’re ultimately producing. To illustrate, let’s say you’ve performed a proof-of-concept prediction model on a government administrative dataset. You’ve reported on what you did, including your results — the model’s accuracy and predictive features.</p><p>Enter: the policy people. They read your report and…</p><p>Policy people are used to reading reports that can inform new policies. They want to formulate policies that will maximise positive outcomes for citizens whilst maintaining fiscal responsibility. They want to know which service or combination of services <em>caused </em>better outcomes (and services, the absence of which, caused worse outcomes). Their questions have causation at their very heart. Your machine learning algorithm (and your written report about it) shows correlations, not causation. The only policy decision a written report on a machine learning algorithm can inform, is whether or not to deploy the machine learning algorithm <em>in situ</em>.</p><p>What you can do about it? There’s a few things. Firstly, let the predictive features discovered by your machine learning algorithm (you know the correlated ones), let them be the ‘shortlist’ for further analysis or testing. Second, use those predictive features you’ve discovered to revisit the evidence-base and research articles, <em>the existing body of evidence. </em>You may have only discovered correlated features but if those already exist in the evidence base within a causal relationship then the further analysis has been done for you. In short, make causation analysis, the ‘next steps’.</p><p><strong>Lesson 3: Completeness of data</strong></p><p>This lesson can go either way. At its core is the question <em>‘do you have sufficient data to predict what you are trying to predict’.</em> One school of thought is, for a machine learning algorithm that is to replace a <em>human </em>decision, then, <a href="https://www.cs.cornell.edu/home/kleinber/w23180.pdf">everything the human takes into account when making the decision should be available in the form of data for the machine to also parse</a>. The other school of thought is, ‘hey, we know we don’t have all the data that, strictly speaking, would sensibly account for the thing we are trying to predict. But we DO HAVE A F@*K TONNE OF DATA so let’s just see if we find a proxy for it…!!#*@!!’. I’m not saying this isn’t a valid approach. You could find an incredible breakthrough proxy predictor… But not without two clear risks. The first risk is that you won’t find that proxy. Second, even if you do discover the proxy, you are signing up to <a href="https://research.google.com/pubs/pub43146.html">technical debt</a> in the form of the fact that YOU ONLY EVER HAD A PROXY and you will constantly need to tune and check your algorithm for concept drift in lieu of the fact that it will only ever be a proxy and that its relationship as a proxy may change or fail over time.</p><p>This one is hard to stay on top of. Everything in a data scientists’s mind will be telling you, with pupil’s dilated to the size of your budget, <em>‘let’s just pass this pile of data in and see what we find!’</em>. After all, <a href="https://www.ft.com/content/d86ad460-8802-11e7-bf50-e1c239b45787">this is the new gold rush</a>. But if there is no logical reason for these data points to predict what you are trying to predict, then be prepared. Be prepared to state the risks aloud. Quantify them. Let’s be adults.</p><p><strong>Conclusion</strong></p><p>You may not be working in the domain of machine learning on government administrative datasets (luck you). You may have learnt nothing. Nothing more than to, in the least, <strong>share your failures.</strong> And perhaps, you will have taken away different lessons. I mean, is the second lesson the knowledge that you are working with correlation and not causation, or is it <em>really</em> about knowing from the outset what the threshold of success for your algorithm is, whom is the stakeholder that would ultimately implement and own it, and how repeatable (implementable) is your data processing chain (tell me all the data you used is actually available in real time, ahead of time, or in enough time to predict [and take action on] what the hell you are trying to predict).</p><p>I live in a world where there is a great temptation to start using existing government administrative datasets to power data-driven decision making. I warn that we do so with care. Get the project scope right, from the outset. <a href="https://dssg.uchicago.edu/2016/10/27/scoping-data-science-for-social-good-projects/">Learn from the University of Chicago’s Data Science for Social Good’s vast experience.</a> Comment here with your learnings. Let’s share our failures in order to make this crazy little thing we call data science a better place to be.</p><p>Cover art ID 92440785 © <a href="https://www.dreamstime.com/ilyakalinin_info">Ilya Kalinin</a> | Dreamstime</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2F8f5f8b%3Fas_embed%3Dtrue&amp;dntp=1&amp;url=https%3A%2F%2Fupscri.be%2F8f5f8b&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/c43026df6fee7cdb1aab8aaf916125ea/href">https://medium.com/media/c43026df6fee7cdb1aab8aaf916125ea/href</a></iframe><figure><a href="https://becominghuman.ai/artificial-intelligence-communities-c305f28e674c"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*2f7OqE2AJK1KSrhkmD9ZMw.png" /></a></figure><figure><a href="https://upscri.be/8f5f8b"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*v-PpfkSWHbvlWWamSVHHWg.png" /></a></figure><figure><a href="https://becominghuman.ai/write-for-us-48270209de63"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*Wt2auqISiEAOZxJ-I7brDQ.png" /></a></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4f53040b529e" width="1" height="1" alt=""><hr><p><a href="https://becominghuman.ai/we-need-to-spend-more-time-talking-about-data-science-failures-4f53040b529e">We need to spend more time talking about data science failures</a> was originally published in <a href="https://becominghuman.ai/">Becoming Human: Artificial Intelligence Magazine</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>