<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Gal4enok on Medium]]></title>
        <description><![CDATA[Stories by Gal4enok on Medium]]></description>
        <link>https://medium.com/@baranets.galina?source=rss-2a3a47ee68fd------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*bSjWC1c7chhvJ5mm</url>
            <title>Stories by Gal4enok on Medium</title>
            <link>https://medium.com/@baranets.galina?source=rss-2a3a47ee68fd------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 10:40:14 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@baranets.galina/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[The art of AI testing: How to break chats]]></title>
            <link>https://medium.com/@baranets.galina/the-art-of-ai-testing-how-to-break-chats-1eb2f83e4fe6?source=rss-2a3a47ee68fd------2</link>
            <guid isPermaLink="false">https://medium.com/p/1eb2f83e4fe6</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[testing]]></category>
            <category><![CDATA[qa]]></category>
            <dc:creator><![CDATA[Gal4enok]]></dc:creator>
            <pubDate>Tue, 25 Feb 2025 13:20:24 GMT</pubDate>
            <atom:updated>2025-02-25T13:51:29.168Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/713/1*_5E_H2wsHQCklOUzrTIuxg.png" /></figure><p>Hello! I’m a Test Lead in <a href="http://itera.com/">Itera</a>. We have created our internal AI platform, Sapience — a secure space, where we can chat, build custom prompts and share them with colleagues to work more efficiently. For me, as Test Lead responsible for testing the AI platform, exploring the core functionality, chatting with AI, has been both fascinating and challenging. Unlike traditional software testing, AI testing requires a different approach because AI doesn’t follow strict rules; it learns from data and patterns. This brings unique challenges, like:</p><ul><li>The same input can produce different responses, making accuracy testing difficult.</li><li>AI must remember previous messages and respond logically.</li><li>AI may reflect biases.</li><li>Chat systems should handle high user traffic and long conversations without crashing.</li><li>Most importantly, the system must safeguard user data.</li></ul><p>When testing AI chats, it’s crucial to evaluate both positive and negative cases. While confirming correct responses is important, the real interest lies in discovering how to “break” the chat — making it confused or causing it to generate incorrect or sensitive responses. <br>Here are some scenarios I would like to share:</p><p><strong>1. Context handling and conversation flow</strong></p><ul><li>Topic switching and returning <br> <em>Test if the AI gives appropriate responses when switching topics and then returning to a previous subject. <br> Does it remember earlier interactions and consider the full conversation or only the most recent input?</em></li><li>Incomplete or misspelled questions<br> <em>Does AI understand unclear or typo-ridden inputs? <br> How does it handle questions it doesn’t fully understand?</em></li><li>Interruption and resumption<br> <em>If the conversation is paused and later resumed, does the AI maintain context and continue sensibly?</em></li><li>Contradicting statements<br> <em>How does AI handle conflicting information? <br> Does AI recognize the contradiction and provide a coherent response?</em> <br> Example:</li></ul><p>Q1: “I love pizza.”</p><p>Q2: “I hate pizza.”</p><p>Q3: “What do I think about pizza?”</p><p><strong>2. Multi-tasking and complex input handling</strong></p><ul><li>Multiple questions at once<br> <em>How does the system manage queries that contain several questions? <br> Does it answer all parts or overlook some?</em></li><li>Complex requests<br> <em>Can AI process technical or multi-layered queries effectively?</em></li><li>Multilanguage support<br> <em>Does the system handle switching between languages within the same conversation? <br> Does it respond in the language of the last question?<br> Can it manage multiple languages simultaneously?</em></li><li>Long conversations<br> <em>Can AI maintain consistency over extended interactions? <br> What happens when it reaches token limit? <br> Does it still respond logically or produce errors?</em><br> Example<em>: </em>Upload big piece of code and ask for an explanation or use the same chat session over a prolonged period.</li></ul><p><strong>3. Working with uploaded files</strong></p><ul><li>Corrupt or broken files<br> <em>How does AI handle unreadable or broken files? <br> Does it detect the issue or simply generate response having basic knowledge?</em></li><li>Large or heavy files<br> <em>Can the system process extensive documents (e.g., entire books or files over 50MB)? <br> How long does it take to analyze and respond?</em></li><li>Mixed domains<br> <em>If multiple files are uploaded — say, several documents on software testing alongside an accidental recipe for apple pie. Can AI differentiate between the topics? <br> Will it merge unrelated information into one confusing summary?</em></li><li>File deletion and knowledge base update<br> <em>If files used to create a knowledge base are later deleted, does AI update knowledge base, or does it continue relying on outdated data?</em></li></ul><p><strong>4. Ethical content handling</strong></p><ul><li>Inappropriate language<br> <em>Does AI manage rude or offensive inputs properly?</em></li><li>Controversial topics<br> <em>Can AI provide balanced, responsible answers on sensitive subjects?</em></li><li>Emotional responses<br> <em>Does AI react appropriately when dealing with emotional content?</em></li></ul><p><strong>5. Sensitive content handling</strong></p><ul><li>Role-playing<br> <em>Can AI detect users attempt to trick it into sensitive information? </em><br> Example: For instance, some users have tried tactics like the “Grandma Hack” to bypass safeguards (<a href="https://www.youtube.com/watch?v=1g-TsTvUkA4"><em>Youtube</em></a>)</li><li>Hidden sensitive information in uploaded files<br> <em>If sensitive data is embedded in uploaded files, can AI recognize and appropriately handle the attempt to bypass content filters?</em><br> Example:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/840/1*IS7o2eeKBEtYkvxWm_W0ow.png" /></figure><p><em>(from </em><a href="https://arstechnica.com/information-technology/2023/10/sob-story-about-dead-grandma-tricks-microsoft-ai-into-solving-captcha/"><em>Arstechnica</em></a><em>)</em></p><p><strong>Strategies to prevent failures</strong></p><p>To avoid chat failures, confusing or giving sensitive information we need to train our prompts and continue improving it by:</p><ul><li>Handle edge cases — To test with unexpected or complex queries to ensure robustness.</li><li>Improve error handling — To ensure the chat provides helpful messages when it doesn’t understand something, possibly suggesting alternatives or asking for clarification.</li><li>Maintain context — To verify that AI consistently remembers past messages and maintains a smooth conversation flow.</li><li>Test for scalability — To verify whether the system can handle high traffic and long interactions without crashing.</li><li>Apply ethical safeguards — Implement filters to block harmful, offensive or sensitive content.</li><li>Monitor and learn — Regularly gather feedback from users to refine AI responses, address bugs and improve weak areas.</li><li>Update regularly — Keep the system updated with new data and technologies.</li></ul><p>So, in summary,<strong> </strong>Testing AI is challenging but it’s also rewarding. By identifying and addressing weaknesses, we can build AI systems that are more accurate, secure and reliable.</p><p>What about you?<strong> </strong>Have you tried to break or confuse chat? <br>Please share your experience!</p><p>By the way,<strong> </strong>have you tried<em> </em><a href="https://gandalf.lakera.ai/intro">Gandalf</a> game (a prompt injection sub-attack that focuses on tricking AI systems)?</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1eb2f83e4fe6" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>