<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ahmet Akyurek on Medium]]></title>
        <description><![CDATA[Stories by Ahmet Akyurek on Medium]]></description>
        <link>https://medium.com/@Akyuerek?source=rss-23f948c3c8ad------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*6ZixIy_skPMGpWE8KuI5Cg@2x.jpeg</url>
            <title>Stories by Ahmet Akyurek on Medium</title>
            <link>https://medium.com/@Akyuerek?source=rss-23f948c3c8ad------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 09:48:47 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@Akyuerek/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Is It Still Your Document, or Does the AI Own It Now?]]></title>
            <link>https://medium.com/@Akyuerek/is-it-still-your-document-or-does-the-ai-own-it-now-2f8a2ef36021?source=rss-23f948c3c8ad------2</link>
            <guid isPermaLink="false">https://medium.com/p/2f8a2ef36021</guid>
            <category><![CDATA[ai-writing]]></category>
            <category><![CDATA[ai-safety]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[ai-governance]]></category>
            <dc:creator><![CDATA[Ahmet Akyurek]]></dc:creator>
            <pubDate>Thu, 16 Jul 2026 10:03:27 GMT</pubDate>
            <atom:updated>2026-07-16T10:03:27.841Z</atom:updated>
            <content:encoded><![CDATA[<p>You asked an AI to update an Excel file. New column, some renaming, nothing special. The file came back in seconds, looking exactly like the original file. Clean, complete, same formatting. You were grateful and moved on.</p><p>Weeks later you realised. One number in the middle of the sheet had changed. Not deleted, not marked, just different. Nobody had touched that cell. Nobody noticed either, because the file looked right.</p><p>I kept thinking about this story when I read a new paper from Microsoft Research. The title says everything: “LLMs Corrupt Your Documents When You Delegate.” The researchers built a benchmark called DELEGATE-52 and tested 19 AI models on real work documents from 52 professions: contracts, ledgers, recipes, code, subtitles, music notation. Then they simulated what we all do in practice, handing the same document to the AI again and again. Twenty rounds of “please also change this.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/896/1*k7_brj3EX-1XURbk0tYHVQ@2x.jpeg" /></figure><p>Three findings:</p><p>First: even the best models available today, Claude Opus 4.6, Chat GPT 5.4, Gemini 3.1 Pro … damaged about 25% of the document content by the end of 20 rounds. Weaker models lost half or more. Not one of the 19 models came out clean.</p><p>Second: the damage does not come as many small scratches. Models work almost perfectly for several rounds, then in one single round they make a big mistake and 10 to 30 percent of the content is gone or wrong. About 80% of all damage comes from these rare, large failures. You cannot predict which round it will be.</p><p>Third, and this is the part that worries me most: the better the model, the less it deletes and the more it corrupts. A missing paragraph catches your eye. A wrong but reasonable-looking number does not. The strongest models produce the most invisible errors.</p><p>Where does this meet your daily work? Three places I would watch.</p><p>A contract draft. You ask the AI to adjust the payment section, it rewrites the whole document, and a liability clause quietly changes its meaning. The document still reads like a contract. Your lawyer notices in the worst possible moment, or never.</p><p>An invoice or a report with numbers. You ask for a translation or a format change, and one amount shifts. Everything else is perfect, which is exactly why the one wrong number survives.</p><p>Any document you improve in rounds. “Add this too. Now shorten it. Now make it formal.” Each round feels like a small edit. The study says each round is also a small risk, and the risks add up. The tenth version of your report is not your report with ten improvements. It is a document that went through the machine ten times.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/799/1*IP6wtugDucEZKpW0c_nXZg@2x.jpeg" /></figure><p>Why does this happen? Because the AI does not edit your document the way you do. It does not open the file and change one cell. It retells the whole document from the beginning, every single time. Think of the telephone game we played as children. The message never becomes noise. It becomes a different message that still sounds sensible, and nobody can say at which ear it changed. After twenty rounds the sentence you hold is fluent. It is just not your sentence.</p><p>So what can a normal person do, without buying any tool?</p><p>**Never let the AI overwrite your original.**</p><p>Keep your last human-approved version in a separate file that the AI never touches. This costs you one copy-paste.</p><p>**Compare before you accept.**</p><p>Word has a Compare Documents function. Excel can show differences between two sheets. Use them before the AI version becomes the official version, not after.</p><p>**Check numbers and names by hand.**</p><p>They are the most fragile content and the least visible when wrong. If a document carries amounts, dates, or names, those get their own thirty seconds.</p><p>One honest note before you quote the 25% somewhere. That number is the average damage after a 20-round simulation, not the error rate of a single edit. The authors themselves ask readers to keep the scale in mind. I have already seen the paper travel on LinkedIn with more dramatic framing, and it does not need any. The real finding is uncomfortable enough: the damage is silent, it looks reasonable, and it grows with every round.</p><p>I use AI for documents daily and I will keep doing it. The paper did not change that. It changed where I put my attention. The model is not the problem. The missing habit of checking is.</p><p>In the next piece I will look at the same problem from the builder’s side: why giving the AI tools made the results worse, not better, and what actually works.</p><p>The model is not guilty. The process is missing.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2f8a2ef36021" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Your Router Can Recognise You. The AI Act Isn’t Sure It Counts.]]></title>
            <link>https://medium.com/@Akyuerek/your-router-can-recognise-you-the-ai-act-isnt-sure-it-counts-f950af6a762d?source=rss-23f948c3c8ad------2</link>
            <guid isPermaLink="false">https://medium.com/p/f950af6a762d</guid>
            <category><![CDATA[ai-governance]]></category>
            <category><![CDATA[ai-compliance]]></category>
            <category><![CDATA[responsible-ai-governance]]></category>
            <category><![CDATA[responsible-ai]]></category>
            <dc:creator><![CDATA[Ahmet Akyurek]]></dc:creator>
            <pubDate>Wed, 15 Jul 2026 19:21:49 GMT</pubDate>
            <atom:updated>2026-07-15T19:21:49.339Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*e2FezB9cS8R-_H2C0INlSA.png" /></figure><p><em>The EU banned real-time biometric identification in public spaces. Then researchers identified people through walls using ordinary Wi-Fi signals, with no camera anywhere. This is the story of a definition that leaks.</em></p><p>In June I submitted a policy paper to the European Commission’s AI Office. It makes one argument, and I want to make the public version of that argument here, because it is not really a story about Wi-Fi. It is a story about how regulation can fails: not loudly, at the level of principles, but quietly, at the level of definitions.</p><p>Here is the short version.</p><h3>The ban</h3><p>Article 5 of the EU AI Act lists the practices Europe decided it does not want at all. One of them, Article 5(1)(h), prohibits real-time remote biometric identification (RBI) in publicly accessible spaces for law enforcement, with narrow exceptions. This is the “no mass facial recognition on our streets” rule. It has been in force since 2 February 2025. Unlike the high-risk obligations, whose dates moved under the omnibus package, the prohibitions did not move. They are live law today.</p><p>The ban is anchored to the concept of “biometric data” in Article 3(34), and to the RBI definition in Article 3(41). Keep those two definitions in mind. They are where the leak is.</p><h3>The technology</h3><p>Wi-Fi signals bounce around a room. A human body disturbs them in measurable ways. Two data streams on ordinary Wi-Fi equipment record those disturbances: Channel State Information (CSI) and beamforming feedback information (BFI), which devices broadcast unencrypted so access points can aim transmissions at them.</p><p>Researchers have been pulling information out of those disturbances for over a decade, and the arc is steady:</p><p>In 2012, researchers in London detected moving people through walls using the signals of an ordinary Wi-Fi access point. In 2018, MIT CSAIL estimated body poses behind walls in real time. In 2022, Carnegie Mellon reconstructed dense 3D body surfaces of multiple people from standard Wi-Fi antennas. And in 2025, researchers at the Karlsruhe Institute of Technology presented BFId at CCS, one of the main security conferences: a system that re-identified individuals with up to 99.5% accuracy in a study of 197 recorded participants, using gait and body signatures extracted from beamforming feedback.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_jMVTVux1JqFwTGWqmSyBA.png" /></figure><p>Read that last one slowly. Commodity hardware. No access to the target network. The person carries no device at all. The signal is broadcast unencrypted by every Wi-Fi device around them.</p><p>Detection became pose estimation, pose estimation became body reconstruction, body reconstruction became identification. Thirteen years, one direction.</p><p>And one more thing: IEEE 802.11bf, published in 2025, makes sensing a native feature of the Wi-Fi standard itself. Every router implementing that amendment is, in principle, a passive human identification node. The KIT researchers themselves called for privacy safeguards. They saw where this goes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7Jj7DFDipG1ekxFz0-7hAQ.png" /></figure><h3>The gap</h3><p>So: a technology that can identify a person in a public space, at a distance, without their involvement. That sounds exactly like what Article 5(1)(h) wants to stop. Is a BFId-style system a prohibited RBI system?</p><p>The honest answer is: not unambiguously. Three problems stack up.</p><p><strong>First, the processing-stage problem.</strong> Raw CSI or BFI is radio-frequency data. It describes how signals propagate through a space. On its face it is not “personal data relating to physical, physiological or behavioural characteristics”. The biometric character appears only after a model has processed the signal. The Act does not say at which stage of the pipeline “biometric data” comes into existence. A camera image of a face is biometric data at the point of collection. A radio echo of a walking body might not be, until the moment the model has already done its work.</p><p><strong>Second, the architecture problem.</strong> Article 3(41) defines RBI through “the comparison of a person’s biometric data with the biometric data contained in a reference database”. That wording assumes a certain design: collect a template, compare against a database of templates. End-to-end neural systems do not have to work that way. The “reference” can live inside the model weights. No discrete template, no discrete database, no classic comparison step. Same outcome, different architecture, and the definition tracks the architecture.</p><p><strong>Third, the enforcement problem.</strong> Definitional ambiguity is not neutral. It creates litigable space, it invites different national interpretations, and it hands operators a compliance pathway: not “we don’t identify people”, but “what we process is not biometric data within the meaning of the Act”. The harm the ban targets, covert identification of people in public spaces, is exactly the harm this technology can produce. The words just don’t reach it cleanly.</p><p>Gait, by the way, is expressly listed in Recital 15 of the Act as a behavioural feature usable for biometric identification. The lawmakers thought about gait. What they did not anticipate is gait arriving through a radio channel instead of a camera lens.</p><p>That is the pattern I keep finding, and it became the central argument of the book I am writing: the regulation leaks at its definitions. The principles are sound. The prohibition is broad in intent, “modality-silent” as the paper puts it. But in practice it is modality-dependent, because its definitions quietly assume the camera-and-microphone paradigm the drafters had in front of them.</p><h3>Does GDPR catch it instead?</h3><p>A reasonable hope. But GDPR’s biometric data definition, Article 4(14), is tied to “unique identification” and is arguably narrower, and it carries the same processing-stage question. A backstop with the same crack in it is not a backstop.</p><p>There is also a second, quieter gap. Article 5(1)(h) covers law enforcement use only. Commercial use of the same capability, retail analytics, workplace monitoring, insurance profiling, would be governed as high-risk under Annex III only if the system counts as RBI at all. The same definitional dependency controls that door too.</p><h3>What I proposed</h3><p>The paper puts the options in order of legal weight. Strengthen Recital 17 and add outcome-based language to Article 3(41), so that a system is RBI when it identifies a natural person without their active involvement, “whether or not a discrete biometric template is generated”. Use an Article 7 delegated act to clarify the commercial side through Annex III. And fastest of all: the AI Office can issue interpretive guidance tomorrow. No legislative procedure, no waiting for the next omnibus.</p><p>The fix is narrow on purpose. The point is not to ban Wi-Fi sensing, which has real and benign uses: fall detection for elderly people, presence detection, gesture control. The point is to make the ban’s coverage depend on what a system does, not on which sensor it uses.</p><h3>Why I keep coming back to this</h3><p>Because the prohibitions are the part of the Act everyone agrees on. If Europe’s strongest “never do this” rule can be sidestepped by changing the sensor, the problem will not stay in Article 5. Every definition in every regulation was written with some technology in front of the drafters. Technologies change. Definitions wait.</p><p>The uncomfortable question for all of us in this field is not “is Wi-Fi sensing banned?” It is: how many other definitions are quietly leaking right now, and who is checking?</p><p><em>I work in IT quality and risk and hold the ISO/IEC 42001 Lead Auditor certification. The full policy paper, “The Modality Gap in Article 5”, was submitted to the European Commission AI Office in June 2026.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f950af6a762d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Article 5 Is Growing: What the Nudifier Ban Reveals About the Future of EU AI Prohibitions]]></title>
            <link>https://medium.com/@Akyuerek/article-5-is-growing-what-the-nudifier-ban-reveals-about-the-future-of-eu-ai-prohibitions-3fd77c3b0dec?source=rss-23f948c3c8ad------2</link>
            <guid isPermaLink="false">https://medium.com/p/3fd77c3b0dec</guid>
            <category><![CDATA[eu-ai-act]]></category>
            <category><![CDATA[ai-compliance]]></category>
            <category><![CDATA[ai-governance]]></category>
            <category><![CDATA[tech-policy]]></category>
            <category><![CDATA[digital-omnibus]]></category>
            <dc:creator><![CDATA[Ahmet Akyurek]]></dc:creator>
            <pubDate>Tue, 07 Jul 2026 23:59:08 GMT</pubDate>
            <atom:updated>2026-07-07T23:59:08.064Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F5REkxNap1Qows3zfFS0TA.png" /></figure><p>When the EU AI Act was finalised in 2024, Article 5 was presented as a closed list. Seven categories of AI practices so harmful, so fundamentally incompatible with EU values, that no risk assessment, no legitimate purpose, no proportionality argument could justify them. Subliminal manipulation below conscious perception. Exploitation of vulnerabilities based on age or disability. Social scoring by public authorities. Real-time remote biometric identification in public spaces with narrow exceptions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9LCpVcFpAGzw2-G9aWLrYA.png" /></figure><p>The list was exhaustive by design. That design just changed.</p><h3>What the Digital Omnibus Added</h3><p>The Digital Omnibus amendments, approved by the European Parliament on 16 June 2026 and given final sign-off by the Council on 29 June, added two new categories to Article 5’s prohibited practices: AI systems that generate child sexual abuse material, and AI systems that generate non-consensual intimate imagery. The second category captures what has become commonly known as “nudifier” applications, tools that use generative AI to produce sexualised images of real people without their consent.</p><p>Both categories carry a compliance deadline of 2 December 2026.</p><p>The addition is legally significant not because of what it prohibits, which was already covered in part by other EU legislation, but because of what it signals about the structure of Article 5 itself.</p><h3>From Exhaustive to Evolutionary</h3><p>A closed list has a logic: the legislature surveyed the landscape of harmful AI practices, identified the worst cases, and drew a hard boundary. Everything inside the boundary is banned. Everything outside is subject to risk classification and proportionality.</p><p>An expanding list has a different logic. It says: the legislature can identify new categories of prohibited harm as technology creates them. The boundary is not fixed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kvoLKbydh1hKNpKab9o0tQ.png" /></figure><p>This is the first time the list has grown, and it grew through an omnibus package whose headline purpose was simplification. That detail matters. The same legislative vehicle that pushed high-risk obligations back to December 2027 and August 2028 also expanded the scope of what is absolutely forbidden. Brussels loosened the middle of the pyramid and hardened the top in a single move.</p><p>The precedent is now set. When the next category of clearly unacceptable AI harm emerges, and it will, the path to adding it to Article 5 is proven and politically cheap. Nobody wants to vote against a nudifier ban. Nobody will want to vote against whatever comes next.</p><h3>How the New Prohibition Actually Works</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mRWNhLnRr2PgbVaFQjzRRQ.png" /></figure><p>The drafting is broader than most headlines suggest. The prohibition catches three distinct actors:</p><p>Providers who place an AI system on the EU market with the purpose of generating CSAM or non-consensual intimate imagery. This is the obvious case, the dedicated nudifier app.</p><p>Providers who place a general-purpose image system on the market without reasonable safety measures to prevent such generation. This is the case that should get every product team’s attention. You do not need to intend the harm. Missing safeguards are enough.</p><p>Deployers who use any AI system with the purpose of creating such content, regardless of what the system was designed for.</p><p>That middle category quietly converts a criminal-adjacent prohibition into an engineering requirement. “Reasonable safety measures” will need a definition, and until guidance arrives, every provider of an image generation capability serving the EU market has to decide what reasonable means and be ready to defend that decision.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*c1RjUlGMXtEE8l7yk_zbWw.png" /></figure><h3>What This Means If You Run Compliance</h3><p>Most organisations treated Article 5 screening as a one-time gate. Early 2025, when the prohibitions became applicable, legal ran the seven categories against the AI inventory, concluded “we do not do any of that,” filed a memo, and moved on. That approach was defensible when the list was closed. It is not defensible now.</p><p>Three practical adjustments follow.</p><p>First, prohibited-practices screening becomes a recurring control, not a project. If the list can grow, your screening has to re-run when it does. Put Article 5 review on the same cadence as your regulatory watch, and log each pass. The screening memo from 2025 does not cover a prohibition added in 2026.</p><p>Second, generative image capability is now a compliance trigger on its own. If any system in your stack can produce images of people, whether you built it, fine-tuned it, or licensed it, someone needs to document what safety measures prevent misuse for intimate imagery, and why those measures are reasonable. For most organisations this lands on the vendor questionnaire: ask your image-model supplier what changed in their safeguards before 2 December 2026, and keep the answer.</p><p>Third, the audit evidence expectation shifts. An auditor who understands the omnibus will no longer accept a static “not applicable” for Article 5. They will ask when you last reviewed the list, who owns the review, and how a new prohibition would reach your inventory. If the honest answer is “we would read about it on LinkedIn,” that is a finding.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SwsbFdsuQMElg1laRJuVQA.png" /></figure><h3>The Deadline Is Closer Than It Looks</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tP6AtwuhgkJmPmuT3OkKzw.png" /></figure><p>2 December 2026 is five months away. That is a short runway for anyone who needs to ship safeguard changes, and the enforcement backdrop makes it shorter: Article 5 violations sit in the highest penalty band of the Act, up to 35 million euros or 7 percent of global annual turnover under Article 99. The omnibus moved many deadlines. It did not move the cost of getting a prohibition wrong.</p><p>There is also an uncomfortable asymmetry worth naming. High-risk obligations, the ones with mature draft standards and years of preparatory guidance, got extra time. The new prohibition, which arrives with no harmonised standard, no guidance document, and an undefined “reasonable safety measures” test, got less than six months. The practices that are easiest to comply with received the longest runway.</p><h3>The Bigger Picture</h3><p>Article 5 was supposed to be the stable part of the AI Act, the moral floor that everything else was built on. It turned out to be the first structural element to change. For practitioners the lesson is not about nudifiers specifically. It is that the boundary between “banned” and “regulated” in EU AI law is now a moving line, and compliance programmes built on the assumption of a fixed list have a design flaw.</p><p>The list grew once. Plan for the version of your programme that handles the second time gracefully.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3fd77c3b0dec" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[When the Kill Switch of your AI tool Belongs to a Foreign Government]]></title>
            <link>https://medium.com/@Akyuerek/when-the-kill-switch-of-your-ai-tool-belongs-to-a-foreign-government-ba6d5bcd87e3?source=rss-23f948c3c8ad------2</link>
            <guid isPermaLink="false">https://medium.com/p/ba6d5bcd87e3</guid>
            <category><![CDATA[eu-ai-act]]></category>
            <category><![CDATA[ai-governance]]></category>
            <category><![CDATA[ai-compliance]]></category>
            <category><![CDATA[ai-security]]></category>
            <dc:creator><![CDATA[Ahmet Akyurek]]></dc:creator>
            <pubDate>Tue, 07 Jul 2026 23:16:09 GMT</pubDate>
            <atom:updated>2026-07-07T23:16:09.308Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nbhR3r04rtruzyZ3CsiflQ.png" /></figure><p><em>AI compliance in the age of geopolitical enforcement</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vrUdGNsjhxK8OCFO8ALNLQ.png" /></figure><p>At 5:21 in the evening on 12 June 2026, Anthropic received an export control directive from the United States government. The directive cited national security authorities. It required the immediate suspension of access to the company’s two most advanced models by any foreign national, whether located inside or outside the United States. It allowed no phase-in period and offered no appeal window.</p><p>Because access to a modern cloud-served AI model cannot be filtered by nationality in real time across a global platform, Anthropic did what the directive left as the only technically compliant option: it disabled both models for every customer worldwide. Services running in production in Berlin, London, Singapore, and São Paulo went dark. Enterprises that had built compliance, operations, and customer-facing workflows on those systems discovered simultaneously that their provider had gone offline and that they had no documented response plan for that event.</p><p>Nineteen days later, on 30 June, the US government lifted the controls. Access was restored globally from 1 July. If you stopped reading there, you might file the whole episode under “resolved.”</p><p>That would be the wrong lesson. The switch was flipped off, and then it was flipped back on. Both moves were made in Washington. Neither involved any European authority, any affected deployer, or any process the EU AI Act recognises. The restoration does not close the gap the suspension exposed. It confirms who controls the switch.</p><p>I have spent the past weeks reading the June directive and the regulatory frameworks it collided with. What follows is what I found.</p><h3>The instrument</h3><p>The first thing to understand about the June directive is what it was not. It was not an AI safety intervention. It was not a product liability action. It was not a privacy enforcement measure. It was an export control order, administered by the Bureau of Industry and Security under the Export Administration Regulations, using a legal concept called the deemed export.</p><p>The deemed export doctrine holds that granting a foreign national access to controlled technology constitutes an export to that person’s home country, regardless of where the technology is physically located. The doctrine was developed for hardware and physical components. Applied to cloud inference, it treats remote access to a model running on a server in Virginia as legally equivalent to moving a controlled component across a border.</p><p>That legal reach is the whole story. A doctrine from the 1980s, built for missile parts and semiconductors, removed one of the most widely used AI systems from worldwide circulation in a matter of hours. No AI statute authorised it. None was needed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ffs0bQ612kGP6a2j1AxiFA.png" /></figure><h3>The collision</h3><p>For EU AI Act compliance practitioners, the June events did three things simultaneously, and the regulation was not written to handle any of them.</p><p>First, deployer obligations under Articles 26 to 29 assume the system is available. On 13 June, it was not. A deployer’s duties around human oversight, input data control, and usage monitoring all presuppose a running system supplied by a cooperating provider. There is no regulation-compliant continuity playbook for provider-side termination ordered by a foreign government, because the drafters never imagined the provider could be forced to walk away overnight.</p><p>Second, risk management documentation under Article 9 has rarely, if ever, listed third-country enforcement action as a foreseeable risk in the product lifecycle. Article 9 asks providers of high-risk systems to identify reasonably foreseeable risks and address them. After June 2026, an export directive from the provider’s home government is no longer a hypothetical. It is a documented event with a date, a legal mechanism, and a measurable blast radius. Risk files that do not mention it are now describing a world that no longer exists.</p><p>Third, the GPAI concentration risk embedded in the Act’s governance chapters has no explicit provision, no evaluation criterion, and no minimum continuity standard. A significant share of EU digital infrastructure runs on models operated by a handful of US providers, all of them subject to US export jurisdiction. The June suspension made that systemic exposure visible. The July restoration did not reduce it by one percentage point.</p><h3>The gap</h3><p>The EU AI Act is an ex-ante, horizontal framework designed for a world where providers are stable, accessible, and answerable to EU authority. The June events introduced a category of AI risk that sits above the conformity assessment and below the force majeure clause: geopolitical enforcement by a foreign sovereign.</p><p>This is not a compliance failure. No deployer broke a rule when the models went dark, and no conformity assessment would have caught it. It is a framework gap, and it matters because the Act’s whole logic assumes the riskiest thing about an AI system is the system itself. June demonstrated that the riskiest thing can be also the jurisdiction the system answers to.</p><p>The European AI Office has the authority to examine this through a systemic risk evaluation under Article 88. Whether it moves in that direction is, in my view, one of the most significant open questions in EU AI Act implementation for the rest of 2026. The 19-day duration of the suspension will tempt regulators to treat it as a blip. Practitioners do not have that luxury.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*l07c66xbDACPFX41EuWh6w.png" /></figure><h3>What practitioners can do now</h3><p>The instruments for managing this risk are practitioner decisions right now, not regulatory ones. Four are worth naming.</p><p><strong>Jurisdictional diversification of model access.</strong> If every model in your stack answers to the same export regime, your continuity plan has a single point of failure that no amount of redundancy inside that jurisdiction can fix.</p><p><strong>Fallback routing documented in Article 9 risk plans.</strong> Not a vague commitment to “evaluate alternatives,” but tested routing to a named secondary model, with the quality and compliance deltas assessed in advance.</p><p><strong>Contract language covering government-mandated termination.</strong> Most enterprise AI agreements address provider insolvency and ordinary termination. Very few address a provider being ordered by its own government to cut access. Ask for notice obligations, data export guarantees, and transition assistance that survive a directive.</p><p><strong>Business continuity planning for AI-reliant services.</strong> If a workflow cannot tolerate the model disappearing for three weeks, that workflow needs a documented degraded mode. June provided the tabletop scenario for free. Use it.</p><p>None of this is exotic. It is ordinary supply chain risk management applied to a dependency that most organisations have not yet admitted is a dependency.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WczWKx5VS1irrsU-G8KLyA.png" /></figure><h3>The closing</h3><p>The kill switch exists. It is operated from Washington. In June it was used, and in July it was released, and at no point did anyone outside one government have a say in either decision.</p><p>EU AI Act compliance programmes that do not account for that are, after June 2026, incomplete. The models came back. The question is what your documentation says about the day they left.</p><p>If your primary GPAI provider went dark again tomorrow, this time for six months, what would your organisation actually do?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*btNQEQ5nj7Oi4SOGYFCaaA.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ba6d5bcd87e3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What Keeps an AI Agent Safe? Not What You Think]]></title>
            <link>https://medium.com/@Akyuerek/what-keeps-an-ai-agent-safe-not-what-you-think-4354c543c67f?source=rss-23f948c3c8ad------2</link>
            <guid isPermaLink="false">https://medium.com/p/4354c543c67f</guid>
            <category><![CDATA[ai-risk-mitigation]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[generative-ai-risks]]></category>
            <category><![CDATA[ai-governance]]></category>
            <category><![CDATA[ai-safety]]></category>
            <dc:creator><![CDATA[Ahmet Akyurek]]></dc:creator>
            <pubDate>Sat, 09 May 2026 22:03:05 GMT</pubDate>
            <atom:updated>2026-05-14T11:40:08.042Z</atom:updated>
            <content:encoded><![CDATA[<p><em>By Ahmet Akyürek | QA/Risk Analyst, AI management and governance.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bt0F_ip6uXa8Vn_7wN2NXQ.jpeg" /></figure><p>I’m in the middle of a Claude Code course when the news hits my feed. An AI agent “Cursor running Anthropic’s flagship Claude Opus 4.6” has just deleted a company’s entire production database. Not a test environment. Not a sandbox. The live database. Along with every automated backup. In nine seconds.</p><p>The company, PocketOS, powers car rental businesses. Months of bookings data, gone before a human could even register what was happening [1].</p><p>I pause the course module when I just completed hooks and think about that. That is it. The only deterministic way to set rules for an AI agent.</p><p>Noted to further research and found really nice work like <a href="https://aiagentindex.mit.edu/">https://aiagentindex.mit.edu/</a> from MIT.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/724/1*nJGg2W8yw27WU44vqYbJHw.png" /></figure><h3>The Failure That Should not Possible</h3><p>The story that can scare anyone who has deployed AI agents in production. The Cursor agent encountered a credential mismatch in a staging environment. Rather than stopping and asking, it went looking for a solution. It found a broadly scoped Railway API token in an unrelated file. It decided on its own, without being asked, that deleting the problematic volume was the right call. It executed. Done!</p><p>PocketOS founder Jer Crane had explicit written rules in the system prompt. “NEVER FUCKING GUESS!” was one of them. The agent guessed anyway. When pressed afterward, the agent confessed: <em>“I violated every principle I was given”</em> [2].</p><p>Crane’s conclusion, reported widely in the aftermath, was that the failure was <em>“systemic”</em> — not a rogue model, but an architecture that made the outcome <em>“not only possible but inevitable”</em> [1].</p><p>He is right. And this is the conversation that enterprise leaders urgently need to have.</p><h3>The Fundamental Flaw: Trusting Instructions to Behave Like Rules</h3><p>When most organizations deploy AI agents today, their safety strategy looks like this: write a good system prompt. Define what the agent should and shouldn’t do. Trust the model to follow it.</p><p>This is like writing road safety advice on a piece of paper and handing it to a new driver instead of installing seatbelts, lane markings, and traffic lights.</p><p>Instructions given to a large language model are probabilistic guidance. The model reads them, weights them against everything else it knows, reasons about the situation, and decides what to do. Sometimes it decides the rules don’t apply to this particular edge case. Sometimes it guesses. This isn’t a bug, it’s how these systems work. We all wanted from Agents to think hard and find solutions. The model is trying to be helpful, and helpfulness occasionally overrides caution.</p><p>The 2025 AI Agent Index, published by MIT researchers and covering deployed agentic systems across the industry, found exactly this pattern at scale [4]. Guardrails against harmful actions differ dramatically between agents, and developers frequently <em>delegate safety responsibilities to users</em> rather than building them into the architecture. The Index also found that only a handful of AI providers ‘btw Claude Code one among them’ publish agent-specific safety documentation at all [4].</p><p>Meanwhile, a McKinsey survey of 1,993 companies (June–July 2025) found that 62% were already experimenting with AI agents [9]. Yet according to the Cloud Security Alliance, only 26% of organizations have comprehensive AI security governance policies in place which means %74 are deploying agents without a governance framework to match [10]. You read correct, %74! Well I also think, to have start-up mindset is cool. Pick trying and failing instead of excessive planning for myself. But, if you manage an Enterprise you have responsibilities, serious ones... You should consider compliance and governance as priority.</p><p>We are deploying fast and governing slow or NOT. PocketOS was the warning shot.</p><h3>What Hooks Actually Are (And Why They’re Different)</h3><p>I want to explain hooks in a way not technical, because this is fundamentally a governance concept.</p><p>A hook is a piece of code (we use often in our Test Automation frameworks too) that runs automatically at a specific moment, <em>outside</em> the model’s own reasoning. Before the agent uses a tool: hook runs. After a file is written: hook runs. Before the session ends: hook runs. Critically, the hook can <em>block</em> the action entirely, regardless of what the model decided [6].</p><p>The distinction from prompt-based rules is categorical:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/688/1*__kU-tsjyrbz7cfE2NPUAg.png" /></figure><p>A PreToolUse hook watching for any destructive API call on a production environment “a database delete, a volume wipe, a bulk record removal” would have blocked what happened to PocketOS before a single byte was touched. Not because the model would have been better instructed. Because the instruction would have been enforced by infrastructure, not by goodwill.</p><p>As independent legal and technology analysts note, hooks “transform best-practice guidelines into enforced rules that run every time” a fundamentally different category of control [8].</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/724/1*VS8v1Du8BbIfNJviSvxMmQ.png" /></figure><h3>The Enterprise Governance Layer</h3><p>For organizations deploying agents at scale, hooks are the foundation of a governance architecture, not a developer convenience.</p><p>Anthropic’s enterprise configuration includes allowManagedHooksOnly, which lets administrators distribute vetted, organization-wide hooks through a managed marketplace and block any user or project from overriding them [6]. This means your security team can define what an agent is never allowed to do and that definition cannot be talked around, edited out of a prompt, or reasoned away by a model that thinks it found a better solution.</p><p>This maps directly onto established security frameworks. The 2025 Agentic AI Safety Playbook from Dextra Labs describes it as <em>defense-in-depth applied to AI</em> [9]: multiple independent layers, permission rules, pre-tool hooks, session classifiers, sandboxing each capable of blocking an action independently. No single point of failure. No single prompt that, if the model ignores it, leaves you exposed.</p><p>The arXiv analysis of Claude Code’s architecture describes the default safety posture as <em>“deny-first with human escalation”</em>: unrecognized actions are blocked and surfaced to a human rather than allowed silently [5]. This is the correct default posture for enterprise deployment, and it’s one that organizations can extend and customize through hooks to match their own risk profiles.</p><h3>What Leaders Should Be Asking Right Now</h3><p>If you have AI agents running in your organization and statistically I am sure you do! Here are the questions worth putting on the table this quarter:</p><p><strong>On architecture:</strong> Are our agents operating under deterministic guardrails, or probabilistic instructions? If a model decides the rules don’t apply today, what stops it?</p><p><strong>On access scope:</strong> Do our agents have access to more than they need? The Railway token that doomed PocketOS was found in <em>an unrelated file</em>. Broadly scoped credentials plus autonomous agents is a risk equation that ends badly.</p><p><strong>On auditability:</strong> When an agent takes an action, is there a log? Can we reconstruct what it decided, what it touched, and why? The MIT AI Agent Index found this transparency gap is nearly universal across the industry [4].</p><p><strong>On recovery:</strong> Irreversible actions like “deletes, sends, publishes, transactions” deserve a special category. Is there a hook that requires human confirmation before any of them execute?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GbDD39MkWKeGbWh3355eXw.png" /></figure><h3>The Moment We’re In</h3><p>PocketOS recovered their data, eventually. But the incident is documented across The Register, Fast Company, Tom’s Hardware, Euronews, and Gizmodo [1][2][3]. The story of an AI agent confessing “I violated every principle I was given” is now part of the public record of this technology.</p><p>More incidents are coming. The MIT Index documents that agents are being deployed faster than safety frameworks are being built around them [4]. The gap between “we’re experimenting with AI agents” and “we have governance infrastructure for AI agents” is where the next PocketOS is going to happen.</p><p><em>Updated: corrected source attribution for governance statistics — 14.05.2026</em></p><h3>References</h3><p>[1] Clark, L. (2026, April 27). <em>Cursor-Opus agent snuffs out startup’s production database</em>. The Register. <a href="https://www.theregister.com/2026/04/27/cursoropus_agent_snuffs_out_pocketos/">https://www.theregister.com/2026/04/27/cursoropus_agent_snuffs_out_pocketos/</a></p><p>[2] Diaz, A. (2026). <em>‘I violated every principle I was given’: An AI agent deleted a software company’s entire database. It may not be the AI’s fault</em>. Fast Company. <a href="https://www.fastcompany.com/91533544/cursor-claude-ai-agent-deleted-software-company-pocket-os-database-jer-crane">https://www.fastcompany.com/91533544/cursor-claude-ai-agent-deleted-software-company-pocket-os-database-jer-crane</a></p><p>[3] Zenity Research. (2026). <em>AI Agent Destroys Production Database in 9 Seconds</em>. Zenity.io. <a href="https://zenity.io/blog/current-events/ai-agent-database-deletion-pocketos">https://zenity.io/blog/current-events/ai-agent-database-deletion-pocketos</a></p><p>[4] Khlaaf, H., et al. (2025). <em>The 2025 AI Agent Index: Documenting Technical and Safety Features of Deployed Agentic AI Systems</em>. MIT AI Agent Index. <a href="https://aiagentindex.mit.edu/">https://aiagentindex.mit.edu</a> (Research paper: arXiv:2602.17753. <a href="https://arxiv.org/html/2602.17753">https://arxiv.org/html/2602.17753</a>)</p><p>[5] Anthropic Research. (2026). <em>Dive into Claude Code: The Design Space of Today’s and Future AI Agent Systems</em>. arXiv:2604.14228v1. <a href="https://arxiv.org/html/2604.14228v1">https://arxiv.org/html/2604.14228v1</a></p><p>[6] Anthropic. (2026). <em>Hooks Reference — Claude Code Documentation</em>. <a href="https://code.claude.com/docs/en/hooks">https://code.claude.com/docs/en/hooks</a></p><p>[7] Anthropic. (2026). <em>Intercept and control agent behavior with hooks — Claude Agent SDK</em>. <a href="https://platform.claude.com/docs/en/agent-sdk/hooks">https://platform.claude.com/docs/en/agent-sdk/hooks</a></p><p>[8] Dotzlaw, J. (2026). <em>Claude Code Hooks: The Deterministic Control Layer for AI Agents</em>. Dotzlaw Team. <a href="https://www.dotzlaw.com/insights/claude-hooks/">https://www.dotzlaw.com/insights/claude-hooks/</a></p><p>[9] McKinsey &amp; Company. (2025). <em>The State of AI in 2025: Agents, Innovation, and Transformation</em>. <a href="https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai">https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai</a></p><p>[10] Cloud Security Alliance. (2025). <em>AI Governance: A Maturity Multiplier</em>. <a href="https://cloudsecurityalliance.org/blog/2025/12/18/ai-security-governance-your-maturity-multiplier">https://cloudsecurityalliance.org/blog/2025/12/18/ai-security-governance-your-maturity-multiplier</a></p><p><em>Ahmet Akyürek is a QA/Risk Analyst at Verti Versicherung AG, specializing in AI risk management and governance.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4354c543c67f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Çoklu Kombinasyonlar ve Pairwise Testing: Yapay Zeka Sistemlerini Test Etmenin Akıllı Yolu]]></title>
            <link>https://medium.com/@Akyuerek/%C3%A7oklu-kombinasyonlar-ve-pairwise-testing-yapay-zeka-sistemlerini-test-etmenin-ak%C4%B1ll%C4%B1-yolu-3d9d7ca1a731?source=rss-23f948c3c8ad------2</link>
            <guid isPermaLink="false">https://medium.com/p/3d9d7ca1a731</guid>
            <category><![CDATA[kobinasyon-test]]></category>
            <category><![CDATA[ai-testing]]></category>
            <category><![CDATA[software-testing]]></category>
            <category><![CDATA[pairwise-testing]]></category>
            <dc:creator><![CDATA[Ahmet Akyurek]]></dc:creator>
            <pubDate>Mon, 20 Apr 2026 23:40:03 GMT</pubDate>
            <atom:updated>2026-04-21T07:18:45.589Z</atom:updated>
            <content:encoded><![CDATA[<p><em>QA olarak şu soruyla kaç kez karşılaştınız: “Bu konfigürasyonları nasıl test edeceğiz?” ve cevap her seferinde “hepsini” olamaz.</em></p><p>Son 4–5 senedir Sigortacılık sektöründe test alanında çalışıyorum. Açıkçası, fonksiyonel testlerde kombinasyon kapsamından söz etmek, hatta o hesaba girmek çoğu zaman mümkün olmuyor. Çoğunlukla sınır değerleri, iş kritikliğine dayalı risk analizi ve tecrübeye dayalı alan bilgisiyle olası hata kümelerini hedef alarak test planı oluşturuyorum.</p><p>Yaklaşık bir yıldır büyük bir sistem değişiminin Pazarlama Kanalları bölümünde Test Lead olarak görev alıyorum. Çok fazla paydaş, çok fazla beklenti ve bunların tam ortasında kronik bir kaynak sıkıntısı. Bir noktadan sonra şunu fark ettim: riskin gerçek boyutunu ve ihtiyaç duyulan kaynağı şeffaf, matematiksel bir şekilde ortaya koymazsam, argümanlar havada kalıyor.</p><p>Pairwise testi adını daha önce duymuştum ama içi boş bir kelimeydi benim için. Oldukça sağlam bir temeli olduğunu gördüm. Ilgimi cekince derinlemesine okumalar yaptim. Bir yazi kaleme alarak sizi de bu yöntemle tanıştırmak istedim.</p><p>Makaleyi yazarken bir de tarayıcıda çalışan, kurulum gerektirmeyen <strong>pairwise hesaplayıcı</strong> geliştirdim. Hem Türkçe hem ingilizce kullanabilirsiniz. Parametrelerinizi girip tek tıkla minimum test setini üretebilir, sonucu CSV olarak indirebilirsiniz. <a href="https://sewo-code.github.io/pairwise-calculator-/"><strong>→ Hesaplayıcıyı deneyin</strong></a><strong> .</strong></p><h3>Taguchi’nin Fabrikasından Yazılım Testine: Kısa Bir Tarih</h3><p>Pairwise testing’in kökü yazılım dünyasında değil, 1950&#39;lerin Japonya’sında yatıyor.</p><p>Japon mühendis <strong>Genichi Taguchi</strong>, o yıllarda üretim tesislerinde ciddi bir problemle boğuşuyordu: bir ürünün kalitesini etkileyen onlarca parametre vardı “sıcaklık, basınç, malzeme türü, işlem süresi” ve bunların tüm kombinasyonlarını denemek hem imkânsız hem de ekonomik açıdan saçmaydı. Taguchi’nin çözümü, istatistikçiler Bose ve Bush’un 1940&#39;larda geliştirdiği <strong>ortogonal dizileri</strong> (orthogonal arrays) üretime uyarlamaktı (Bose &amp; Bush, 1952). Fikir basitti: her parametre çiftini en az bir kez kapsayan dengeli bir deney tasarımı yap; tüm kombinasyonları tek tek deneme.</p><p>Bu matematiksel fikrin yazılım testine taşınması 1985 yılında gerçekleşti. Mandl, derleyici (compiler) testinde yaşanan kombinasyonel patlamayı çözmek için ortogonal dizileri yazılıma uyarladı ve “bu yaklaşımın test kapsamını korurken test senaryosu sayısını dramatik biçimde azalttığını” deneysel olarak gösterdi (Mandl, 1985). Bu çalışma, pairwise testing’in yazılım alanındaki ilk akademik kaydı olarak kabul edilmektedir.</p><p>Fikir yerini bulmuştu; ama gerçek ivmeyi kazanması için on yıl daha beklemesi gerekecekti.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/974/1*y7hEerHbo__8LSZKPeRsGA.png" /></figure><h3>Endüstriyel Ölçek: AETG ve 1997</h3><p>1990&#39;larda AT&amp;T’nin araştırma kolu Bellcore’da bir ekip, pairwise testing’i elle uygulanabilir olmaktan çıkarıp otomatikleştirdi. Cohen, Dalal, Fredman ve Patton (1997), <strong>AETG (Automatic Efficient Test Generator)</strong> sistemini tanıttıkları çalışmalarında “verilen parametre uzayı için minimum test seti üretmenin NP-zor bir problem olduğunu, ancak greedy algoritmaların pratikte yeterince küçük setler ürettiğini” ortaya koydu (Cohen et al., 1997, s. 440). AETG, yalnızca bir araç tanıtımı değil; aynı zamanda “neden ikili kombinasyonlar yeterli olabilir?” sorusunu sistematik biçimde yanıtlayan ilk kapsamlı çalışmaydı.</p><p>Ama asıl 2004 de bilimsel olarak hataların yaklaşık <strong>%85&#39;i, en fazla iki parametrenin etkileşiminden</strong> kaynaklandigi gösterildiginde bu yöntem kitlesel ölcekte kullanima girdi.</p><h3>“Yüzde Seksen Beş” Bulgusu: Kuhn (2004)</h3><p>Pairwise testing’in bugün bir standart olarak kabul görmesinin arkasında ağırlıklı olarak bu makale yatıyor:</p><blockquote><em>Kuhn, D.R., Wallace, D.R., &amp; Gallo, A.M. (2004). </em>Software Fault Interactions and Implications for Software Testing.<em> IEEE Transactions on Software Engineering, 30(12), 418–421.</em></blockquote><p>NASA ve NIH dahil çeşitli kurumların gerçek hata veri tabanlarını inceleyen bu çalışma, temel bir soruya ampirik cevap verdi: <strong>Bir yazılım hatası, kaç parametrenin yanlış kombinasyonundan kaynaklanır?</strong></p><p>Kuhn ve arkadaşları, inceledikleri sistemlerdeki hataları kombinasyon derinliğine göre sınıflandırdı ve şu tabloyu ortaya çıkardı (Kuhn et al., 2004, s. 419):</p><p>Kombinasyon Derinliği Hataların Oranı 1 parametre tek başına ~%35, 2 parametrenin kombinasyonu ~%50, 3 parametrenin kombinasyonu ~%10, 4 ve üzeri parametre ~%5</p><p>Yazarlara göre bu bulgular, “test kaynaklarının büyük çoğunluğunun iki faktörlü etkileşimleri kapsayan test setlerine yönlendirilmesinin rasyonel bir strateji olduğunu güçlü biçimde desteklemektedir” (Kuhn et al., 2004, s. 420). Yani gerçek sistemlerdeki hataların yaklaşık <strong>%85&#39;i, en fazla iki parametrenin etkileşiminden</strong> kaynaklanıyordu.</p><p>Bu bulgu pairwise testing’e hem teorik bir zemin hem de pratik bir meşruiyet kazandırdı. İkili kombinasyonları kapsayan bir test seti, tüm kombinasyonların küçük bir fraksiyonunu oluştursa da hataların büyük çoğunluğunu yakalamaya yeterliydi.</p><p>Kuhn ve ekibi bu araştırmayı sonraki çalışmalarında genişleterek <strong>t-way testing</strong> kavramını geliştirdi. Güvenlik kritik sistemlerde 3-way veya 4-way kapsamın daha uygun olabileceğini gösterdiler; ancak “iki faktörlü etkileşim testinin genel yazılım sistemleri için maliyet-etkin bir taban oluşturduğu” sonucunu korudular (Kuhn &amp; Reilly, 2002, s. 3).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/965/1*qAhItKyfM0gzX0taPGB_FQ.png" /></figure><p>Bu araştırmanın pratik uzantısı olarak Microsoft, Czerwonka liderliğinde <strong>PICT</strong> (Pairwise Independent Combinatorial Testing) adlı ücretsiz aracı geliştirdi. Czerwonka (2006), PICT’in endüstriyel kullanımını aktardığı çalışmasında “gerçek dünya projelerinde pairwise test setlerinin exhaustive testlere kıyasla %90&#39;a varan hacim azaltımı sağladığını, buna karşın hata tespit oranlarının yüksek kaldığını” bildirdi (Czerwonka, 2006, s. 5).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/961/1*Mq2_77GAWU5nzO1HOACQtQ.png" /></figure><h3>Yazılım Testinde Kullanım Alanları</h3><p>Pairwise testing, her test senaryosu için her parametre kombinasyonunu denemek yerine tüm ikili kombinasyonları en az bir kez kapsayan minimum test setini oluşturmayı hedefler. ISTQB müfredatı bu tekniği, “tüm girdi parametrelerinin tüm kombinasyonlarını test etmenin pratik olmadığı durumlarda sistematik kapsam sağlamak için kullanılan bir siyah kutu test tasarım tekniği” olarak tanımlar (ISTQB CTFL Syllabus, 2023, s. 47).</p><p>Klasik yazılım testinde en çok şu senaryolarda görürüz:</p><ul><li><strong>Tarayıcı/işletim sistemi/cihaz matrisleri:</strong> Chrome × Windows × Masaüstü, Safari × macOS × Dizüstü gibi kombinasyonlar pairwise ile yönetilir.</li><li><strong>API parametre testleri:</strong> Onlarca giriş parametresi olan REST API’larda tüm kombinasyonlar yerine ikili kapsamla başlanır.</li><li><strong>Konfigürasyon testleri:</strong> Farklı veritabanı, sunucu ve uygulama sürümlerinin bir arada çalışma senaryoları.</li><li><strong>Form validasyonu:</strong> Çok sayıda alan içeren formda geçerli/geçersiz değerlerin kombinasyonları.</li></ul><h3>Yapay Zeka Testinde Neden Farklı Bir Yeri Var?</h3><p>Geleneksel yazılım sistemleri deterministik davranır: aynı girdi, aynı çıktı. Ancak yapay zeka ve makine öğrenmesi sistemleri, model ağırlıkları, hiperparametreler, veri ön işleme adımları, dağıtım ortamı ve çalışma zamanı konfigürasyonlarının karmaşık etkileşimiyle davranış sergiler. Bu durum kombinasyonel test problemini çok daha kritik bir hale getirir.</p><p>ISTQB’nin AI Testing (CT-AI) sertifikasyon müfredatı, pairwise testing’i özellikle yapay zeka sistemlerinin konfigürasyon testinde “tercih edilen kombinasyonel teknik” olarak konumlandırır ve “AI sistemlerinin büyük parametre uzaylarının exhaustive kapsam yerine pairwise gibi kombinasyonel yaklaşımlar gerektirdiğini” açıkça ifade eder (ISTQB CT-AI Syllabus, 2021, s. 38).</p><p>Kuhn et al.’ın (2004) bulgularının AI bağlamındaki önemi burada belirginleşiyor. Bir güvenlik filtresi tek başına doğru çalışabilir; belirli bir dil modeliyle birlikte kullanıldığında beklenmedik bir boşluk ortaya çıkabilir. Hataların %85&#39;inin ikili kombinasyonlardan kaynaklandığı gerçeği, bu tür etkileşim hatalarının sistematik biçimde yakalanabileceğini göstermektedir.</p><h3>Somut Örnek: Bir AI Chatbot’u Test Etmek</h3><p>Bir AI chatbot geliştirdiğimizi düşünelim. Sistem şu parametrelerle konfigüre edilebiliyor:</p><p>Parametre Değerler Sayı Dil Modeli LM1, LM2, LM3, LM4, LM5 5 Yanıt Tonu Resmi, Yarı-resmi, Samimi, Teknik 4 Bağlam Penceresi 2K token, 4K token, 8K token 3 Güvenlik Filtresi Temel, Gelişmiş 2</p><p><strong>Exhaustive test:</strong> 5 × 4 × 3 × 2 = <strong>120 test senaryosu</strong></p><p><strong>Pairwise test:</strong> yaklaşık <strong>~20 test senaryosu</strong></p><p>PICT çıktısı şuna benzer bir tablo üretir:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/968/1*oLZImshwYQYsHymNSGdYZw.png" /></figure><p>Bu tabloda herhangi iki parametrenin değer çiftine bakın — “LM3 + Gelişmiş Güvenlik” veya “8K + Teknik Ton” — en az bir satırda karşılaşacaksınız. Czerwonka’nın (2006) aktardığı pratik deneyimle örtüşür biçimde, 120 kombinasyon yerine ~20 ile tüm ikili etkileşimler kapsanmış olmaktadır.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/965/1*ZtfUVBY8m6WuyKhHLjqqOQ.png" /></figure><h3>Sonuç: Neden Yeterli Olduğunu Bilmek</h3><p>Pairwise testing’i sıradan bir kısaltma yöntemi olmaktan çıkaran şey, Kuhn ve arkadaşlarının (2004) ortaya koyduğu ampirik temele dayanmasıdır. “Tüm hataların %85&#39;i en fazla iki parametrenin birleşiminden kaynaklanır” bulgusu, bu tekniğin neden işe yaradığını açıklamaktadır.</p><p>Yazarlara göre, “t-way testinin değeri yalnızca test hacmini azaltmaktan ibaret değildir; asıl değer, en yüksek hata riski taşıyan etkileşimlerin öncelikli biçimde kapsanmasını sağlamasındadır” (Kuhn et al., 2004, s. 421). AI sistemlerinin artan konfigürasyon karmaşıklığı göz önünde bulundurulduğunda, bu yaklaşım artık isteğe bağlı bir teknik olmaktan çıkmış; sistematik bir zorunluluk haline gelmiştir.</p><p>Makaleyi yazarken bir de tarayıcıda çalışan, kurulum gerektirmeyen <strong>pairwise hesaplayıcı</strong> geliştirdim. Parametrelerinizi girip tek tıkla minimum test setini üretebilir, sonucu CSV olarak indirebilirsiniz. <a href="https://sewo-code.github.io/pairwise-calculator-/"><strong>→ Hesaplayıcıyı deneyin</strong></a><strong>.</strong></p><h3>Kaynaklar</h3><p>Bose, R.C., &amp; Bush, K.A. (1952). Orthogonal arrays of strength two and three. <em>Annals of Mathematical Statistics, 23</em>(4), 508–524.</p><p>Cohen, D.M., Dalal, S.R., Fredman, M.L., &amp; Patton, G.C. (1997). The AETG system: An approach to testing based on combinatorial design. <em>IEEE Transactions on Software Engineering, 23</em>(7), 437–444. <a href="https://doi.org/10.1109/32.605761">https://doi.org/10.1109/32.605761</a></p><p>Czerwonka, J. (2006). Pairwise testing in the real world: Practical extensions to test-case scenarios. <em>Proceedings of the 24th Pacific Northwest Software Quality Conference</em>, 1–8.</p><p>Kuhn, D.R., &amp; Reilly, M.J. (2002). An investigation of the applicability of design of experiments to software testing. <em>Proceedings of the 27th Annual NASA Goddard Software Engineering Workshop</em>, 91–95.</p><p>Kuhn, D.R., Wallace, D.R., &amp; Gallo, A.M. (2004). Software fault interactions and implications for software testing. <em>IEEE Transactions on Software Engineering, 30</em>(12), 418–421. <a href="https://doi.org/10.1109/TSE.2004.24">https://doi.org/10.1109/TSE.2004.24</a></p><p>Mandl, R. (1985). Orthogonal Latin squares: An application of experiment design to compiler testing. <em>Communications of the ACM, 28</em>(10), 1054–1058. <a href="https://doi.org/10.1145/4372.4375">https://doi.org/10.1145/4372.4375</a></p><p>ISTQB. (2021). <em>Certified Tester AI Testing (CT-AI) syllabus v1.0.</em> International Software Testing Qualifications Board.</p><p>ISTQB. (2023). <em>Certified Tester Foundation Level (CTFL) syllabus v4.0.</em> International Software Testing Qualifications Board.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3d9d7ca1a731" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Data Leakage in AI: The Bug That Looks Like Success]]></title>
            <link>https://medium.com/@Akyuerek/data-leakage-in-ai-the-bug-that-looks-like-success-efaf019a36f4?source=rss-23f948c3c8ad------2</link>
            <guid isPermaLink="false">https://medium.com/p/efaf019a36f4</guid>
            <dc:creator><![CDATA[Ahmet Akyurek]]></dc:creator>
            <pubDate>Fri, 17 Apr 2026 23:43:00 GMT</pubDate>
            <atom:updated>2026-04-17T23:43:00.293Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Your model just hit 97% accuracy. Before you celebrate, ask yourself one question.</em></p><p>There’s a particular kind of problem in machine learning that reflects itself as a win.</p><p>The model scores 97% accuracy. Validation looks clean. Everyone is happy. You deploy it and within days it falls apart completely. Predictions that seemed sharp in testing are useless in the real world.</p><p>No code broke. No server failed. The model simply wasn’t doing what you thought it was doing.</p><p>This is data leakage.</p><h3>Wait: Isn’t “Data Leakage” a Security Term?</h3><p>Yes, and that’s worth clarifying upfront because the two meanings are easy to confuse.</p><p>In <strong>cybersecurity</strong>, data leakage means sensitive information escaping to the wrong place. A customer database exposed, private records going to an unauthorised party. It’s about confidentiality.</p><p>In <strong>machine learning and AI</strong>, data leakage means something completely different. Here, “leakage” refers to information sneaking into your model’s training process in a way that gives it an unfair (and ultimately useless) advantage. It’s not about data escaping. It’s about the wrong data getting in.</p><p>When an AI practitioner says “there’s leakage in this dataset,” they mean the model is learning from information it wouldn’t have access to in the real world. Nothing is being stolen or exposed. The problem is internal to the model itself.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lSntCr5vhtemSDUwDs5Dkg.png" /></figure><h3>The Core Idea</h3><p>Data leakage in AI happens when your model learns from information it would never actually have in real life.</p><p>During training, the model finds every signal it can. But if those signals only exist <em>because of</em> the outcome you’re trying to predict, or come from the future, the model is learning a shortcut. One that works perfectly on your dataset and fails completely the moment it meets reality.</p><p>The dangerous part: it feels like success. High accuracy is what you want. There’s no error message. Just numbers that look great and a model that is, quietly, broken.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_w1PiCd-ygkBnnDd-RBlNw.png" /></figure><h3>Two Types of Leakage</h3><p><strong>Target leakage</strong> is when a feature in your data is <em>caused by</em> the outcome you’re trying to predict, not the other way around. The logic is backwards. The model is effectively learning from the answer.</p><p><strong>Train-test contamination</strong> is when information from your test set sneaks into the training process, usually through data preparation steps done before the split. Your evaluation looks better than it really is because the “unseen” test data wasn’t actually unseen.</p><h3>Two Examples</h3><h3>The Loan Default Model</h3><p>A team builds a model to predict whether a customer will default on a loan. They include the feature has_debt_settlement_plan.</p><p>The problem: people enter debt settlement <em>after</em> they’ve already defaulted. It’s a consequence of the thing they’re trying to predict. In training, this feature looks like a near-perfect predictor. In real life, when you need to make a lending decision, that field is blank. The customer hasn’t defaulted yet.</p><p>The model wasn’t learning who <em>might</em> default. It was learning who <em>already had</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*E54bjHMRiteodGDIj-g-tg.png" /></figure><h3>Normalising Before Splitting</h3><p>A team builds an image classifier. Before splitting data into training and test sets, they normalise all pixel values using statistics from the <em>entire dataset</em>, including the test images.</p><p>The test data was supposed to be completely invisible. But it subtly shaped the training process. Evaluation results now look better than they really are.</p><p>The fix is simple: split first, then compute statistics only on training data, then apply those same statistics to the test set. The test set should never be involved in any step before final evaluation.</p><h3>How to Catch It</h3><p>When reviewing a feature, ask:</p><ul><li><strong>Would this value exist at decision time?</strong> If it only appears after the outcome, it’s leakage.</li><li><strong>Is this caused by the outcome?</strong> If yes: target leakage.</li><li><strong>Was any preprocessing done before the train/test split?</strong> It shouldn’t be.</li><li><strong>Is one feature suspiciously powerful?</strong> Real signals are messy. If something’s too clean, be suspicious.</li></ul><p>In the ISTQB CT-AI framework, catching leakage is a core <strong>data quality testing</strong> responsibility. The test itself has to be valid before any performance metric means anything.</p><p>The uncomfortable truth: leakage is common, easy to miss, and the symptom is a number everyone wants to see. Learning to recognise it is one of the most valuable things anyone building or testing AI systems can do.</p><p><em>I write about AI Compliance, testing and practical ML challenges. More soon.</em></p><p>You can also follow me on Linkedin for shorter informative practices.</p><p><a href="https://www.linkedin.com/in/ahmetakyurek/">https://www.linkedin.com/in/ahmetakyurek/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=efaf019a36f4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How Do You Test Something That Thinks? A QA Guide to AI Testing Methodologies]]></title>
            <link>https://medium.com/@Akyuerek/how-do-you-test-something-that-thinks-a-qa-guide-to-ai-testing-methodologies-dc1f5ca23f88?source=rss-23f948c3c8ad------2</link>
            <guid isPermaLink="false">https://medium.com/p/dc1f5ca23f88</guid>
            <category><![CDATA[ai-testing]]></category>
            <category><![CDATA[istqb-ai]]></category>
            <category><![CDATA[llm-testing]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[ai-testing-framework]]></category>
            <dc:creator><![CDATA[Ahmet Akyurek]]></dc:creator>
            <pubDate>Sun, 12 Apr 2026 21:40:44 GMT</pubDate>
            <atom:updated>2026-04-12T21:40:44.151Z</atom:updated>
            <content:encoded><![CDATA[<h3>How Do You Test Something That Thinks? A Practitioner’s Guide to AI Testing Methodologies</h3><p><em>AI systems are unlike any software we’ve built before and that means testing them requires a completely different playbook.</em></p><p>Testing traditional software is hard. Testing AI is harder. With a conventional program, you write a function, define its expected outputs, and run assertions. Either the tests pass or they don’t. There’s a right answer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F2uWWOVEb2iClpKU2dj9wA.png" /></figure><p>AI systems within machine learning models, especially large language models (LLMs) don’t work that way. Ask a model to summarize a document and there are dozens of valid outputs. Ask it the same question twice and you might get different answers. Ask it a slightly reworded question and watch it confidently say something completely different. Correctness is fuzzy, behavior is probabilistic, and failure modes are subtle and sometimes dangerous.</p><p>Research confirms this isn’t just an impressionistic problem. A tertiary study of AI applications in software testing published in <em>ACM Computing Surveys</em> found that while AI has been widely applied to <em>support</em> testing, the challenge of testing the AI systems themselves remains a distinct and underexplored domain [1]. Meanwhile, a 2025 systematic review in <em>MDPI Algorithms</em> catalogued 229 AI algorithms applied to software testing challenges over the past decade, underscoring just how fast the landscape is evolving and how fragmented current practice remains [3].</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eDlkV5XeJViRTdi2yyL4XA.png" /></figure><p>The good news? The profession has been catching up. The ISTQB Certified Tester AI Testing (CT-AI) syllabus provides a structured framework for thinking about AI-specific testing challenges, and the practitioner community is converging on a set of approaches that actually work. Here’s an overview.</p><h3>The ISTQB Framework: A Foundation for AI Testing</h3><p>Before diving into specific techniques, it’s worth grounding ourselves in the vocabulary the ISTQB CT-AI syllabus establishes. Testing an AI-based system spans several distinct concerns [4]:</p><p><strong>Data quality</strong> — AI models are only as good as the data they’re trained on. The syllabus treats data testing (training sets, validation sets, test splits) as a first-class concern, not an afterthought.</p><p><strong>ML functional performance metrics</strong> — Accuracy alone is insufficient. Practitioners need to understand confusion matrices, precision, recall, and F1 scores to meaningfully evaluate model behavior.</p><p><strong>AI-specific quality characteristics</strong> — Beyond traditional software qualities like reliability and performance, AI systems introduce new dimensions: <em>explainability</em>, <em>fairness</em>, <em>freedom from bias</em>, and <em>safety</em>. The CT-AI syllabus explicitly includes these as testable characteristics.</p><p><strong>Testing techniques specific to AI</strong> — Metamorphic testing, back-to-back testing, adversarial testing, and A/B testing are all named as relevant approaches for AI-based systems.</p><p>This framework is a useful map. The rest of this article covers the major territories in that map in practical terms.</p><h3>Start with Functional Evals: Does It Do the Job?</h3><p>The most straightforward testing question is also the most fundamental: does the model produce the right output for a given input?</p><p><strong>Functional evaluations</strong> (or “evals”) are task-specific tests that measure this directly. You define a dataset of inputs, specify what a good output looks like, and score the model’s responses against that standard.</p><p>For classification tasks sentiment analysis, intent detection, document routing; this maps cleanly to familiar metrics: accuracy, precision, recall, F1 score. The ISTQB CT-AI syllabus dedicates a full chapter to these ML functional performance metrics precisely because they’re the baseline vocabulary any AI tester needs [4]. For open-ended generation tasks “summarisation, Q&amp;A, code generation” you need more creative approaches: reference-based scoring with metrics like BERTScore, or using a capable model as an automated judge.</p><p>A practical discipline: build your eval set <em>before</em> you build your system. It forces clarity about what success actually looks like, and prevents inadvertently overfitting your prompts to your own intuitions.</p><h3>Data Testing: Garbage In, Garbage Out (Doubly So)</h3><p>Traditional software testing focuses on code. AI testing must also focus on <em>data</em> and this is one of the most distinctive shifts the ISTQB CT-AI syllabus emphasises [4].</p><p>Bad training data doesn’t just produce bad outputs. It encodes bad patterns into the model weights. This means data quality problems are often invisible at the point of failure and extremely difficult to diagnose after the fact.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EZFu-JxC8ry4yICklREXYg.png" /></figure><p>Effective data testing covers several areas: checking for class imbalance in training sets, verifying that training, validation, and test splits are properly separated (no leakage), auditing labeling consistency, and ensuring the data distribution reflects the real-world inputs your model will see in production. Many practitioners underinvest here —&gt; focusing on model evaluation while treating data as a given. The research literature and the ISTQB framework both push back against this tendency.</p><h3>Behavioural Testing: Probing for What Could Go Wrong</h3><p>Functional evals tell you whether the model performs well under normal conditions. Behavioural testing asks: <em>what happens at the edges?</em></p><p>The ISTQB CT-AI syllabus specifically calls out <strong>metamorphic testing</strong> as a key technique for AI-based systems [4]. Metamorphic testing works by defining relationships between inputs and outputs that <em>should</em> hold, then verifying them. You don’t need to know the exact correct answer BUT you just need to know how the output should <em>change</em> (or not change) in response to a modified input.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LK7ew9-e4fw73x6ZiAjXiA.png" /></figure><p>Classic examples: adding obviously negative language to a review should lower its sentiment score. Swapping a gendered pronoun in a job application should not change the hiring recommendation. If it does, you’ve found a bias, a failure on the <em>fairness</em> quality characteristic that the ISTQB framework treats as a first-class testing concern.</p><p>This approach is powerful precisely because it doesn’t require labeled ground truth for every test case. You can generate behavioral tests at scale by reasoning about the properties your model <em>should</em> have.</p><h3>Adversarial and Red-Team Testing</h3><p>The ISTQB CT-AI syllabus includes <strong>adversarial testing</strong> as an explicit technique, and for good reason [4]. AI systems can behave in unexpected and potentially harmful ways when given inputs they weren’t designed for including inputs carefully crafted to exploit their weaknesses.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sNjzgCbHZzcxn-nkkKnHzA.png" /></figure><p>Red-teaming involves having testers (human or automated) systematically try to elicit harmful outputs, bypass safety guardrails, extract private information, or produce confident wrong answers. The 2024 study reviewing AI-powered testing tools found that automated approaches tools that generate adversarial inputs at scale are increasingly viable, but noted significant limitations in current tooling: most tools are optimized for particular failure modes and don’t generalize well [2]. And we can see all see in production daily, sometimes personally sometimes from news.</p><p>A mature adversarial testing practice usually combines both: human red-teamers who surface creative failure modes that automated tools miss, and automated tools that provide breadth and scalability.</p><h3>Back-to-Back and A/B Testing: Comparing Versions</h3><p>When you update a model, a prompt, or a pipeline component, you need a reliable way to know whether things got better, got worse, or stayed the same. <strong>Back-to-back testing “</strong>comparing two versions of a system on the same inputs” is the ISTQB CT-AI syllabus’s answer to this [4].</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jEpBBvfbsD0PU7xOe2oryg.png" /></figure><p>This approach is closely related to what practitioners commonly call <strong>A/B testing</strong> or <strong>regression testing in AI</strong>. The key is maintaining a frozen, consistent eval set that both versions are measured against, with clear thresholds for what counts as a regression.</p><p>One critical practice: always evaluate by <em>slice</em>, not just in aggregate. A new model version may improve average performance while regressing significantly for a specific language, topic, demographic group, or input type. Aggregate metrics hide this. Slice analysis reveals it.</p><h3>Explainability and Fairness Testing</h3><p>Perhaps the most distinctly AI-specific dimension of testing is the need to assess qualities that have no direct analog in traditional software: <em>explainability</em>, <em>fairness</em>, and <em>freedom from bias</em>. The ISTQB CT-AI syllabus treats these as testable quality characteristics, not aspirational goals [4].</p><p><strong>Explainability </strong>asks: can the model’s outputs be traced back to understandable inputs or reasoning steps? For high-stakes applications — medical diagnosis, loan decisions, fraud detection actually this isn’t optional. Techniques like LIME, SHAP, and attention visualization help probe whether a model’s explanations are accurate and consistent.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7laECikVUPjO1zA4MHfd_Q.png" /></figure><p><strong>Fairness and bias testing</strong> measures whether the model’s performance or behavior differs across demographic groups in ways that are unjustified. This requires deliberate construction of test sets that vary protected attributes, and careful analysis of differential outcomes.</p><p>The broader research community has catalogued the difficulty here: as the <em>MDPI Algorithms</em> systematic review found, defect prediction and model evaluation metrics are still actively evolving, with no universal consensus on best practices [3]. The ISTQB framework provides structure, but specific implementations require judgment and domain knowledge.</p><h3>Putting It Together</h3><p>The most important shift for anyone moving from traditional software testing to AI testing is accepting that <em>certainty is off the table</em>. You won’t write a test suite that guarantees correct behavior. What you can do is systematically reduce the probability of unexpected failures, quantify the ways your system falls short, and build feedback loops that catch regressions before users do.</p><p>The ISTQB CT-AI syllabus provides the most structured practitioner framework currently available, covering data testing, functional performance metrics, behavioral techniques like metamorphic testing, adversarial approaches, and AI-specific quality characteristics like fairness and explainability [4]. The research literature confirms both the importance of this domain and how much remains to be standardized [1, 3].</p><p>A mature AI testing practice combines:</p><ul><li>A <strong>core eval suite</strong> covering key tasks with automated scoring</li><li><strong>Data quality checks</strong> throughout the ML pipeline</li><li><strong>Behavioral and metamorphic test cases</strong> targeting known edge cases and biases</li><li><strong>Red-team exercises</strong> before major releases</li><li><strong>Back-to-back regression testing</strong> on every change</li><li><strong>Slice-based fairness analysis</strong> across relevant subgroups</li><li><strong>Production observability</strong> — logging, sampling, and human review</li></ul><p>No single methodology is sufficient. Together, they give you the coverage and confidence to ship AI-powered systems responsibly.</p><h3>References</h3><p>[1] Amalfitano, D., Faralli, S., Hauck, J.C.R., Matalonga, S., &amp; Distante, D. (2023). <em>Artificial Intelligence Applied to Software Testing: A Tertiary Study</em>. ACM Computing Surveys, 56(3). <a href="https://dl.acm.org/doi/10.1145/3616372">https://dl.acm.org/doi/10.1145/3616372</a></p><p>[2] Rafi, S. et al. (2024). <em>AI-Powered Software Testing Tools: A Systematic Review and Empirical Assessment of Their Features and Limitations</em>. arXiv preprint arXiv:2409.00411. <a href="https://arxiv.org/abs/2409.00411">https://arxiv.org/abs/2409.00411</a></p><p>[3] (2025). <em>Artificial Intelligence in Software Testing: A Systematic Review of a Decade of Evolution and Taxonomy</em>. MDPI Algorithms, 18(11), 717. <a href="https://www.mdpi.com/1999-4893/18/11/717">https://www.mdpi.com/1999-4893/18/11/717</a></p><p>[4] ISTQB. (2021). <em>Certified Tester AI Testing (CT-AI) Syllabus, Version 1.0</em>. International Software Testing Qualifications Board. <a href="https://istqb.org/certifications/certified-tester-ai-testing-ct-ai/">https://istqb.org/certifications/certified-tester-ai-testing-ct-ai/</a></p><p>[5] Why does AI lie?” (Hallucination Testing) <a href="https://medium.com/@drcagriataseven?source=post_page---byline--4369abae8b13---------------------------------------">Dr. Çağrı ATASEVEN</a> <a href="https://medium.com/@drcagriataseven/why-does-ai-lie-hallucination-testing-4369abae8b13">https://medium.com/@drcagriataseven/why-does-ai-lie-hallucination-testing-4369abae8b13</a></p><p><em>If you found this useful, I write about practical AI engineering, Compliance, Implementations and the real challenges of AI Risk Management . Follow along.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dc1f5ca23f88" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>