<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Florian Treml on Medium]]></title>
        <description><![CDATA[Stories by Florian Treml on Medium]]></description>
        <link>https://medium.com/@floriantreml?source=rss-ddc3d6734b89------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*E_Li-MehM9oo7MPGN_enZA.png</url>
            <title>Stories by Florian Treml on Medium</title>
            <link>https://medium.com/@floriantreml?source=rss-ddc3d6734b89------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 08:53:08 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@floriantreml/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Continuous Speech Recognition Testing]]></title>
            <link>https://blog.chatbotslife.com/continuous-speech-recognition-testing-7f012b7e9f11?source=rss-ddc3d6734b89------2</link>
            <guid isPermaLink="false">https://medium.com/p/7f012b7e9f11</guid>
            <category><![CDATA[chatbots]]></category>
            <category><![CDATA[conversational-ai]]></category>
            <category><![CDATA[voice-assistant]]></category>
            <category><![CDATA[chatbot-development]]></category>
            <category><![CDATA[speech-recognition]]></category>
            <dc:creator><![CDATA[Florian Treml]]></dc:creator>
            <pubDate>Wed, 23 Feb 2022 11:21:08 GMT</pubDate>
            <atom:updated>2022-03-03T14:19:25.032Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*cqsDlA_QU66MSKS1u4suZQ.png" /></figure><h3>Table of Contents:</h3><blockquote><a href="#0be8">Conversational AI and Voice</a><br><a href="#72d0">Continuous Speech Recognition Testing</a></blockquote><blockquote><a href="#b145">Configure Speech Services in Botium Box</a></blockquote><blockquote><a href="#43dd">Speech Recognition (Speech-to-Text)</a><br><a href="#d470">Speech Synthesis (Text-to-Speech)</a><br><a href="#d21e">Custom Speech Engines</a><br><a href="#961b">Prepare Chatbots for Speech Synthesis and Recognition</a><br><a href="#2d80">Prepare Test Set</a></blockquote><blockquote><a href="#c5c7">Gathering Test Data</a></blockquote><blockquote><a href="#04ee">Labelled Reference Audio Files</a><br><a href="#be00">Synthesizing Speech</a><br><a href="#0a65">Record Speech</a><br><a href="#6054">Run Speech Recognition Test Session</a></blockquote><blockquote><a href="#c50c">Humanification: Adding Noise</a></blockquote><blockquote><a href="#b898">Voice Effects Pipeline</a><br><a href="#5e9c">Apply Noise Effects</a></blockquote><blockquote><a href="#8d47">Bonus: Check Word Error Rate</a></blockquote><blockquote><a href="#efa1">Bonus: Speech Recognition Regression Tests</a></blockquote><h3>Conversational AI and Voice</h3><p>From traditional IVR systems, still very common in customer service, to voice assistants on Alexa or Google Assistant — voice technology is nowadays part of everyday life, and it will continue to gain importance in future. The component stack of a typical voice assistant includes speech recognition and speech synthesis, apart from the usual conversational components like NLU (natural language understanding), dialogue management and NLG (natural language generation).</p><p>Botium Box enables companies to implement a holistic test strategy for voice assistants on all levels of the component stack. This article focuses on the speech recognition part and shows how to build a regression test suite based on audio files, transcription files and word error rate verification.</p><h4>Continuous Speech Recognition Testing</h4><p>The big cloud service providers Google, Amazon, Microsoft and IBM all provide high quality speech services with the best recognition rates on the market. But even with those cloud providers, some make it possible to add your own optimizations by uploading additional training data — this is often used for improving recognition rates for domain-specific vocabulary. Apart from the big cloud service providers there are also a number of free software packages available like Kaldi (for speech recognition) or MaryTTS (for speech synthesis), which companies install, train and operate on their own infrastructure.</p><p>Continuous speech recognition testing has most benefits for voice assistants using an optimized cloud speech service or completely self-trained language models. As part of a voice assistant test strategy, the quality of the speech recognition should be verified continuously, as all the other stack components. This article will show how to set up a continuous speech recognition test suite.</p><h3>Configure Speech Services in Botium Box</h3><p>In Botium Box, switch to the <em>Voice Services</em> section in the <em>Settings</em> menu (from the user icon on the top right).</p><h4>Speech Recognition (Speech-To-Text)</h4><p>In the <em>Speech Recognition</em> list, we register the speech service (including credentials) and the language and to use. Depending on the speech service there are additional parameters required — for example, for cloud services like Google the credentials are required.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Gix9RTk3Tqeflw_fZgrJnA.png" /></figure><p>Botium Box allows very fine grained control over how the speech service is used with the <em>Custom Engine Configuration</em> field. For example, to select your own customized Azure speech model:</p><pre>{</pre><pre>  &quot;speechConfig&quot;: {</pre><pre>    &quot;endpointId&quot;: &quot;xxx-yyyy-zzzzzzzzzz&quot;</pre><pre>  }</pre><pre>}</pre><p>It is possible to immediately test the speech service configuration by using your own microphone.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GwcA-w9igeeG702OKSfVVw.png" /></figure><h4>Speech Synthesis (Text-To-Speech)</h4><p>In the <em>Speech Synthesis</em> list, we register the speech service (including credentials), the language and the voice to use for synthesizing audio from text. Depending on the speech service there are additional parameters required — for example, for cloud services like Google the credentials are required.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SEIn5UrAjquiztaMZ64xmg.png" /></figure><p>Again, you can test the configuration by listening to a first example.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uE7wYKj8OTg_pCPgttcXyw.png" /></figure><h4>Custom Speech Engines</h4><p>Botium Box is using the open source <em>Botium Speech Processing</em> software stack (see <a href="https://github.com/codeforequity-at/botium-speech-processing">Github project</a>). By using the Github source code as base it is possible to add your own custom speech recognition or speech synthesis as well as voice effects to your Botium tests. <em>Be prepared to do some Node.js coding!</em></p><h4>Prepare Chatbot for Speech Synthesis and Recognition</h4><p>Next we will create a chatbot in Botium Box with the Speech Synthesis &amp; Recognition connector. In the Chatbots section, register a new chatbot, select the technology from the list and the previously configured speech recognition and speech synthesis profiles.</p><p>The <em>Speech Synthesis &amp; Recognition</em> connector is a very slim Botium connector that is capable of doing simple 2-step conversations:</p><ul><li>use pre recorded audio file or synthesize audio file for a given text with a given voice</li><li>send this audio file to the speech recognition engine and show the text results and WER</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xIUNBDGAgNYDcv8QDec8UA.png" /></figure><p>The configuration can be tested immediately with the Say Hello or even with the Live Chat. You can either enter some text in the live chat for which voice will be synthesized, or you can record your own voice as well.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sR9b_lY21Tth8qlHGlTQGw.png" /></figure><h4>Prepare Test Set</h4><p>As the next step we have to prepare a <em>Test Set</em>. A Test Set in Botium Box is the container for the test data itself as well as the configuration of how to interpret the test data to inject it into the Botium testing pipeline.</p><p>Usually, in Botium Box test cases are defined in the scripting language <em>BotiumScript</em>, conversation flow tests as well as NLP tests and other test types supported by Botium. In the case of speech recognition testing, it is not required to use BotiumScript: for testing speech recognition labeled audio files are sufficient to build up the test cases.</p><ul><li>Audio files have to be uploaded as wav or mp3 files to Botium Box</li><li>Botium looks up the expected transcription …</li><li>in a file with the same name but with extension .txt instead</li><li>in a CSV file transcript.csv</li><li>from the file name itself</li></ul><p>In the <em>Test Sets</em> menu, create a new Test Set and navigate to the <em>Media Resources Settings</em>. In the <em>Audio File Usage</em> field select the option <strong>Use all audio files as Test Case input, and read transcription from file</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*l11uPzkvHVig3mTgx26XVA.png" /></figure><p>The <em>Default folder for attachment files</em> is the folder in Botium Box where we will place our test audio files in the next steps. You can navigate there by clicking on <em>Open in File Browser</em> to show a now still empty folder in the Botium Box File Browser.</p><h3>Gathering Test Data</h3><p>As always, a test strategy is only as good as the quality of the test data. Depending on your goals there are some ways to gather test data.</p><h4>Labelled Reference Audio Files</h4><p>If you are working on your own voice assistant you most likely already own reference data in the form of labeled audio files (audio file + expected transcription). Botium expects the expected transcription in a separate text file named after the audio file, or in a separate CSV file.</p><p>For example, consider there are a couple of labeled audio files:</p><ul><li><strong>F01-hello-how-are-you.wav</strong>, expected transcription “hello how are you”</li><li><strong>F02-hi-whats-up.wav</strong>, expected transcription “hi whats up”</li></ul><p>In Botium Box, open the <em>Default folder for attachment files</em> of the Test Set you created above in the File Browser.</p><ol><li>Upload the two audio files</li><li>do either of the following</li><li>Place two additional text files in this folder</li><li><strong>F01-hello-how-are-you.txt</strong> with text content “hello how are you”</li><li><strong>F02-hi-whats-up.txt</strong> with text content “hi whats up”</li><li>Or place a file transcript.csv in this folder</li><li>you can prepare this file with a text editor or with a spreadsheet like Excel</li></ol><pre>F01-hello-how-are-you.wav;hello how are you</pre><pre>F02-hi-whats-up.wav;hi whats up</pre><p>Now Botium knows what to expect when sending the audio file to the speech recognition engine.</p><h4>Synthesizing Speech</h4><p>Another way to start testing your speech recognition is to synthesize audio files from text. Botium Box has the <em>Voice Wizard</em> to support you in doing so. Paste the text for which to generate the audio files, select the voices you want to use and the output folder. The Voice wizard will place the audio files as well as the transcription files right there.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mWVpwUuXvAUeGTUQfpELdg.png" /></figure><p>When opening the folder now in the File Browser, you can see a bunch of audio files — one for each line of text and voice — as well as one transcription file per audio file, ending in <em>.txt</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JtwXCBx-V0EkUau-dPNdnA.png" /></figure><p>This is by far the quickest way to generate audio test data en masse.</p><h4>Record Speech</h4><p>For smaller scale projects it might also be feasible to record your own voice and use it for testing the speech recognition. The Voice Wizard also has the option to record an audio file and place it in the Test Set folder.</p><p>You will have to add the transcription files manually if choosing this option.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3U0ZQEPAXV5ibC7YANwKeg.png" /></figure><h4>Run Speech Recognition Test Session</h4><p>Now that we have everything in place for a first test run — the connection to our speech recognition engine and the labeled test data — we run a first test. For this we create a Test Project — in the <em>Test Projects</em> menu, register a new Test Project, select the Chatbot and the Test Set we created in the previous steps and click on Save.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uTtGN9hYM8NJO6sgu8OeEw.png" /></figure><p>Now click on <em>Start Test Session Now</em> to start the first test session. A few minutes later you can already start to inspect the results.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*e5B8LTt72MRVEU9x04trrQ.png" /></figure><p>You can see the list of audio files where the speech recognition matched the expected transcription, and the ones where it failed. You can dive into the results, listen to the audio files and view the result details on a JSON code level (use the <strong>&lt;&gt; button</strong>).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*q89rS8XrApz0DBo1jd4TnA.png" /></figure><p>As expected, most of the tests pass now, as we are using the best quality audio files we can get — labeled reference data, your own voice recordings or even synthesized voice, which is per definition extremely clean. We can now add some noise to the audio files to challenge the speech recognition with real-life scenarios.</p><h3>Humanification: Adding Noise</h3><p>In Botium Box, the term <em>Humanification</em> is used to describe the application of algorithms to introduce noise into the test data. For text-based testing, this means to consider typical human behaviour patterns and typical human flaws like typographical errors, case insensitivity, whitespaces (or lacking of whitespaces), using emojis and others. For voice based testing, <em>noise</em> can be indeed noise — like adding some environment-specific background noise.</p><h4>Voice Effects Pipeline</h4><p>Switch to the <em>Voice Services</em> section in the <em>Settings</em> menu (from the user icon on the top right). In the <em>Voice Effects</em> list you can configure your pipeline of additional noise layers to apply to an audio file. There are several common audio effects available to simulate real-life environments:</p><ul><li>Adding background noise</li><li>Make it sound like a low-bandwidth GSM phone call</li><li>Simulate a slightly interrupted phone line by adding breaks</li><li>… and more</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4tqkg_5FZVltR_igMml2qA.png" /></figure><p>Clone the Test Set from above and add something with “Noise” to the name. This Test Set is initially empty, but has the same settings as the above.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Sap3uDVAVf2MBhxsvXwO9w.png" /></figure><p>Verify that the <em>Audio File Usage</em> field is indeed set to <strong>Use all audio files as Test Case input, and read transcription from file</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qZdtbDf0uuNLPXP5DZFF6g.png" /></figure><h4>Apply Noise Effects</h4><p>In your clean Test Set, open the <em>Voice Effects</em> section in the <em>Voice Wizard</em>. Select the folder where you placed your audio files before and select the noise profile from above. In the <em>Output Folder</em>, select the media folder from your new noise Test Set you created in the previous step. When clicking on <strong>Apply Voice Effect</strong>, Botium Box will run all of the audio files in the selected folder (including subfolders!) through the voice effect pipeline and place the resulting audio files in the output folder.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*u30axp9U5XLGC8gnx1N4NA.png" /></figure><p>Now create a new Test Project for the new Test Set with noise, and start a first test session.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FAIuikuozj_tmSVPZfHzJA.png" /></figure><p>Again, you can inspect the test results after a few minutes to check if the applied noise makes the speech recognition struggle.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GQJKiMM3v1wqqe41I4bAHw.png" /></figure><p>Congratulations, you can now run the speech recognition tests with your reference data and with the noisy data over and over again — it makes most sense when you retrain your speech recognition of course.</p><p>Trending Bot Articles:<br>1. How Conversational AI can Automate Customer Service<br>2. Automated vs Live Chats: What will the Future of Customer Service Look Like?<br>3. Chatbots As Medical Assistants In COVID-19 Pandemic<br>4. Chatbot Vs. Intelligent Virtual Assistant — What’s the difference &amp; Why Care?</p><h3>Bonus: Check Word Error Rate</h3><p>In many cases, you might not actually be interested in the exact transcription of the audio file, but rather if certain quality criteria about the transcription are met — this is where the <em>word error rate</em> comes into play. It’s a measure of how many words in a single transcription have been recognized correctly — for a perfect transcription fully matching the label it is 0, and the value is between 0 and 1. Depending on your requirements you might consider a word error rate of 0.1 (one wrong transcription out of 10 words) ok. Botium Box can verify the word error rate on a single utterance level instead of the exact transcription.</p><p>First, switch the value of <em>Audio File Usage</em> in the Test Set settings to <strong>Use all audio files as Test Case input, and calculate word error rate from transcription from file </strong>— this will Botium instruct to calculate the word error rate, instead of asserting on the exact transcription.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0upKSU8zfXbUYJ29bV6oZw.png" /></figure><p>Currently there is no special Word Error Rate asserter in Botium Box, but we can tailor the <a href="https://botium-docs.readthedocs.io/en/latest/05_botiumscript/index.html#jsonpath-asserter">Generic JSONPath Asserter</a> for doing this. In Botium Box, in the <em>Settings</em> menu open the <em>Registered Components</em> section. Register a new component there, name it <strong>Check WER</strong> and select <strong>Test Case Asserter</strong> as <em>Component Type</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5xlTC6k20Tz9IcuXsJKWfQ.png" /></figure><p>As <em>Component Ref Code</em>, use <strong>WER</strong> and in the <em>Component Source</em> field enter <strong>JsonPathAsserter</strong>. This will instruct Botium to use the Generic JSONPath Asserter for this component. Enable the switch <em>Register as global scripting component</em> to make Botium use this asserter for all test cases. Finally, in the Component Configuration, we tell Botium the <a href="https://support.smartbear.com/alertsite/docs/monitors/api/endpoint/jsonpath.html">JSONPath</a> expression which we expect to match the speech recognition response.</p><p>If we expect a perfect word error rate of 0, we can enter this expression:</p><pre>{</pre><pre>  &quot;path&quot;: &quot;$.response..[?(@.wer == 0)]&quot;</pre><pre>}</pre><p>If we are fine with a word error rate of exactly or below 0.1, this is the expression to use:</p><pre>{</pre><pre>  &quot;path&quot;: &quot;$.response..[?(@.wer &lt;= 0.1)]&quot;</pre><pre>}</pre><p>We now have to tell Botium Box to use this asserter in our tests. Save the registered component, and navigate to your Test Project. Open the <em>Test Execution</em> section in <em>Settings</em>, and add the <strong>Check WER</strong> component to the <em>Involved Registered Component(s)</em> field.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vXNVUMwXXHMajwZBN4qbwg.png" /></figure><p>When now running a test session, Botium will not assert on the exact transcription anymore, but it will instead use the <strong>Check WER</strong> asserter to fail a test case if the word error rate is above 0.1 — you can inspect the detailed results in the test session again:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F_IwVMtjLmPBVCPJNJ5CAw.png" /></figure><h3>Bonus: Speech Recognition Regression Tests</h3><p>When regression testing a speech model, often the word error rate itself is not the main criteria — but to make sure that the transcription for some reference audio files is always the same, before training a new speech model and afterwards. With this it is possible to tell if the newly trained speech model has any obvious defects.</p><p>In these cases you most likely have a library of reference audio files, labeled or unlabeled. In this use case we can start with the unlabeled audio files to build up a regression test suite.</p><ol><li>Create a new Test Set, set the <em>Audio File Usage</em> to <strong>Use all audio files as Test Case input</strong></li><li>Upload your unlabeled audio files to the <em>media</em> folder of this Test Set</li><li>Start a first test session — Botium Box will send all the audio files to the speech recognition engine and accept any transcription</li><li>In the <strong>Downloads</strong> section of the test result, download the full transcript as CSV file and open it in Excel</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*owRc8exMvUwHTp4R_k3xeQ.png" /></figure><p>In this CSV file, we are only interested in the columns named <strong>testCaseName</strong> (which contains the file name) and <strong>convoStepActual</strong> (which contains the transcription). Filter the <strong>convoStepSender</strong> column on the value <strong>bot</strong> and hide all other columns except the two mentioned ones. Copy the remaining data as file transcript.csv to the folder with your audio files in Botium Box.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Vrc6KkIkKYRyMVdHMent3g.png" /></figure><p>When now switch the <em>Audio File Usage</em> field to Use all audio files as Test Case input, and read transcription from file, Botium will read the expected transcriptions from this file and any changes in the transcription for your reference files will popup up in the test results.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7f012b7e9f11" width="1" height="1" alt=""><hr><p><a href="https://blog.chatbotslife.com/continuous-speech-recognition-testing-7f012b7e9f11">Continuous Speech Recognition Testing</a> was originally published in <a href="https://blog.chatbotslife.com/">Chatbots Life</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Top 3 Chatbot Security Vulnerabilities in 2022]]></title>
            <link>https://blog.chatbotslife.com/top-3-chatbot-security-vulnerabilities-in-2022-e3b8bec8eac0?source=rss-ddc3d6734b89------2</link>
            <guid isPermaLink="false">https://medium.com/p/e3b8bec8eac0</guid>
            <category><![CDATA[chatbots]]></category>
            <category><![CDATA[automated-testing]]></category>
            <category><![CDATA[conversational-ai]]></category>
            <category><![CDATA[chatbot-security]]></category>
            <category><![CDATA[security-testing]]></category>
            <dc:creator><![CDATA[Florian Treml]]></dc:creator>
            <pubDate>Tue, 25 Jan 2022 09:22:28 GMT</pubDate>
            <atom:updated>2022-01-25T16:07:27.893Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*-cMnHE1MWFZeAjYf9nrCdA.png" /></figure><h3>Chatbots are here to stay</h3><p>Chatbots have been around for a long time and based on the global chatbot market size (and the expected growth), they will stick around for a long time and gain importance. In the past they’ve rarely met customer expectations or provided much positive experience. However, over the last few years, advances in conversational AI have transformed how they can be used. Since chatbots offer a wide range of applications, in certain cases they become responsible for collecting and protecting personal information as well. Consequently, they are a great attraction for hackers and malicious attacks too. The responsibility of ensuring chatbot security has become more pronounced after the introduction of GDPR in Europe. As statistics show that this technology will be a determining factor in our lives, security testing must also become part of our daily tasks, so that these chatbots can be used with confidence.</p><figure><a href="https://www.chatbotconference.com/"><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*lwlUzb2E7V5ZAsT4.png" /></a></figure><h3>Security Risks, Threats and Vulnerabilities</h3><p>The words <strong>risk</strong>, <strong>threat</strong> and <strong>vulnerability</strong> are often confused or used interchangeably when reading about computer security, so let’s first clarify the terminology:</p><ul><li><em>Vulnerability</em> refers to a weakness in your software (or hardware, or in your processes, or anything related). In other words, it’s a <strong>way</strong> hackers could find their way into and exploit your systems.</li><li>A <em>threat</em> exploits a vulnerability and can cause loss, damage or destruction of an asset — <strong>threats exploit vulnerabilities</strong></li><li><em>Risk</em> refers to the potential for lost, damaged, or destroyed assets — <strong>threats + vulnerability = risk</strong>!</li></ul><p>The well-known <a href="https://owasp.org/www-project-top-ten/">OWASP Top 10</a> is a list of top security risks for a web application. Most chatbots out there are available over a public web frontend, and as such all the OWASP security risks apply to those chatbots as well. Out of these risks there are two especially important to defend against, as in contrast to the other risks, those two are nearly always a serious threat — <strong>XSS (Cross Site Scripting) and SQL Injection</strong>.</p><p>In addition, for artificial intelligence enabled chatbots, there is an increased risk for <strong>Denial of Service</strong> attacks, due to the higher amount of computing resources involved.</p><h3>Trending Bot Articles:</h3><blockquote><a href="https://chatbotslife.com/how-conversational-ai-can-automate-customer-service-ce1e6e330902">1. How Conversational AI can Automate Customer Service</a></blockquote><blockquote><a href="https://chatbotslife.com/automated-vs-live-chats-what-will-the-future-of-customer-service-look-like-69c90432c8f4">2. Automated vs Live Chats: What will the Future of Customer Service Look Like?</a></blockquote><blockquote><a href="https://chatbotslife.com/chatbots-as-medical-assistants-in-covid-19-pandemic-3d2731b85128">3. Chatbots As Medical Assistants In COVID-19 Pandemic</a></blockquote><blockquote><a href="https://chatbotslife.com/chatbot-vs-intelligent-virtual-assistant-whats-the-difference-why-care-180aebde9358">4. Chatbot Vs. Intelligent Virtual Assistant — What’s the difference &amp; Why Care?</a></blockquote><h3>Vulnerability 1: XSS — Cross Site Scripting</h3><p>A typical implementation of a chatbot user interface:</p><ul><li>There is a chat window with an input box</li><li>Everything the user enters in the input box is mirrored in the chat window</li><li>Chatbot response is shown in the chat window</li></ul><p>The XSS vulnerability is in the second step — when entering text including malicious Javascript code, the XSS attack is fulfilled when the web browser is running the injected code:</p><pre>&lt;script&gt;alert(document.cookie)&lt;/script&gt;</pre><h4>Possible Attack Vector</h4><p>For exploiting an XSS vulnerability the attacker has to trick the victim to send malicious input text.</p><ol><li>Attacker tricks the victim to click a hyperlink pointing to the chatbot including some malicious code in the hyperlink</li><li>The malicious code is injected into the website, reads the victims cookies and sends it to the attacker without the victim even noticing</li><li>The attacker can use those cookies to get access to the victim’s account on the company website</li></ol><h4>Defense</h4><p>This vulnerability is actually easy to defend by validating and sanitizing user input, but still we are seeing this happening over and over again.</p><h3>Vulnerability 2: SQL Injection</h3><p>A typical implementation of a task-oriented chatbot backend:</p><ul><li>User tells the chatbot some information item</li><li>The chatbot backend queries a data source for this information item</li><li>Based on the result a natural language response is generated and presented to the user</li></ul><p>With SQL Injection, the attacker tricks the chatbot backend to consider malicious content as part of the information item:</p><pre>my order number is “1234; DELETE FROM ORDERS”</pre><h4>Possible Chatbot Attack Vector</h4><p>When the attacker has personal access to the chatbot, an SQL injection is exploitable directly by the attacker (see example above), doing all kinds of SQL (or no-SQL) queries.</p><h4>Defense</h4><p>Developers typically trust their tokenizers and entity extractors to defend against injection attack. Additionally, simple regular expression checks of user input will in most cases close this vulnerability.</p><h3>Vulnerability 3: Denial of Service</h3><p>Artificial intelligence requires high computing power, especially when deep learning as in state-of-the-art natural language understanding (NLU) algorithms is involved. The Denial of Service (DoS) attack is focused on <strong>making a resource unavailable</strong> for the purpose it was designed, and it is not hard to imagine that chatbots are more vulnerable to Denial of Service (DoS) attacks than the usual backends based on highly optimized database systems. If a chatbot receives a very large number of requests, it may cease to be available to legitimate users. These attacks introduce large response delays, excessive losses, and service interruptions, resulting in direct impact on availability.</p><h4>Possible Chatbot Attack Vector</h4><p>A typical DoS attack sends a large number of large requests to the chatbot to intentionally exhaust the available resources. It will happen that the <strong>computing resources</strong> are not available anymore to legitimate users.</p><p>But there is an additional risk to consider: it is quite common for chatbot developers to use <strong>cloud based services</strong> like IBM Watson or Google Dialogflow. Depending on the chosen plan there are usage limits and/or quotas in effect which can be <strong>exhausted</strong> pretty quickly — for example, the Google Dialogflow Essential <strong>free plan limits</strong> access to 180 requests per minute, all other requests will be denied. For a usage-based plan without any limits a DoS attack can easily cost a fortune due to the increased number of requests.</p><h4>Defense</h4><p>The established methods for defending against Denial of Service attacks apply for chatbots as well.</p><h3>Mitigation Strategies</h3><p>Apart from the defense strategies from above, there are generic rules for mitigating the risks of system breaches.</p><h3>Software Developer Education</h3><p>Naturally, the best way to defend against vulnerabilities is to <strong>not even let them occur</strong> in the first place. Software developers should get special training to consider system security as part of their day-to-day development routine. Establishing the mindset and processes in the development team is the first and most important step.</p><h3>Continuous Security Testing</h3><p>Security testing should be part of your continuous testing pipeline. The earlier in the release timeline a security vulnerability is identified and fixed the cheaper.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*OmR0UFmOzVCmsvx8piR_QA.png" /></figure><p>Basic tests based on the OWASP Top 10 should be done on API level as well as on End-2-End-level. Typically, defense against SQL Injections is tested best on API level (because of speed), while defense against XSS is tested best on End-2-End level (because of Javascript execution).</p><p>Specialised tools like <a href="https://www.botium.ai/">Botium</a> help in setting up your continuous security testing pipeline for chatbots.</p><h3>Conclusion</h3><p>Chatbots open the same kind of vulnerabilities, threats and risks as other customer facing web applications. Due to the nature of chatbots, some vulnerabilities are more probable than others, but the well established defense strategies will work for chatbots.</p><h3>Don’t forget to give us your 👏 !</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/780/0*ZxaGcKC3UQEydQIX" /></figure><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2F3e59ef%3Fas_embed%3Dtrue&amp;dntp=1&amp;url=https%3A%2F%2Fupscri.be%2F3e59ef&amp;image=http%3A%2F%2Fapi.screenshotlayer.com%2Fapi%2Fcapture%3Faccess_key%3Dfe59908dad3baab69ffab249a2224b03%26viewport%3D1024x612%26width%3D1000%26url%3Dhttps%253A%252F%252Fupscri.be%252F3e59ef%253Fscreenshot&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/7078d8ad19192c4c53d3bf199468e4ab/href">https://medium.com/media/7078d8ad19192c4c53d3bf199468e4ab/href</a></iframe><figure><a href="https://chatbotslife.com/bot-communities-mastermind-group-d2dae9876709#.53x0py6ou"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*6XUspT9JOSq0w0Fi35HIaA.png" /></a></figure><figure><a href="https://m.me/ChatbotsLife"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*c1LDMH5vbnIz9rmAka8Hwg.png" /></a></figure><figure><a href="https://chatbotslife.com/chatbot-development-hire-top-ai-chatbot-developers-c8b45ef7f207"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*D0Jf3dI6ZThtqcfwDYY7mg.png" /></a></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e3b8bec8eac0" width="1" height="1" alt=""><hr><p><a href="https://blog.chatbotslife.com/top-3-chatbot-security-vulnerabilities-in-2022-e3b8bec8eac0">Top 3 Chatbot Security Vulnerabilities in 2022</a> was originally published in <a href="https://blog.chatbotslife.com/">Chatbots Life</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Testing Chatbots for the Unexpected]]></title>
            <link>https://medium.com/chatbots-journal/testing-chatbots-for-the-unexpected-b5a995cc2861?source=rss-ddc3d6734b89------2</link>
            <guid isPermaLink="false">https://medium.com/p/b5a995cc2861</guid>
            <category><![CDATA[naturallanguageprocessing]]></category>
            <category><![CDATA[chatbots]]></category>
            <category><![CDATA[testing]]></category>
            <category><![CDATA[botium]]></category>
            <dc:creator><![CDATA[Florian Treml]]></dc:creator>
            <pubDate>Tue, 18 May 2021 08:35:01 GMT</pubDate>
            <atom:updated>2021-05-21T17:53:07.162Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Quite often we are consulted to design a robust test strategy for a mission-critical enterprise chatbot. How is it possible to test something for all possible unexpected user behaviour in the future ? How can someone confidently make assumptions on the quality if we have no clue what the users will ask the chatbot ?</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*0TW21tLCo6lAYyq-qllXfQ.png" /><figcaption>Testing Chatbots for the Unexpected</figcaption></figure><h3>Short-Tail vs Long-Tail Topics</h3><p>While we do not own a magic crystal ball to look into future usage scenarios, from our experience we gained the best results with a systematic approach in a continuous feedback setup. In almost every chatbot project the use cases can be categorized:</p><ul><li><strong>short-tail topics</strong> — the topics serving most of your users needs (typcially, the 90/10 rule applies — 90 % of your users are asking for only 10% of the topics)</li><li><strong>long-tail topics</strong> — all the other somehow exotic topics that the chatbot could also answer</li><li><strong>handover topics</strong> — topics where for whatever reason handover to a human agent is required</li></ul><p>Examples for <strong>short-tail topics</strong> in the telecom domain:</p><ul><li>Opening hours of the flagship stores</li><li>WLAN connectivity issues</li><li>Questions on individual invoice lines</li></ul><p>Examples for <strong>long-tail topics</strong> in the telecom domain:</p><ul><li>iPhone availability in the flagship stores</li><li>International roaming conditions</li><li>Lost SIM-card phone</li></ul><p>Examples for <strong>handover topics</strong> in the telecom domain:</p><ul><li>Contract cancellation</li><li>Business products and business customers</li></ul><h3>Getting Started</h3><p>Our recommendations for the first steps in a chatbot project are always the same:</p><ol><li><strong>Focus training and testing exclusivly on the short-tail topics</strong> — typically, with a customer support chatbot, which are the majority of the chatbots we’ve been working with, there are a handful of topics only for which you have to provide a good test coverage</li><li>Apart from that, <strong>leave the long-tail topics aside</strong> and design a <strong>clear human handover process</strong></li></ol><p>The challenge now is how to get a good test coverage for the short-tail topics — this is the real hard work in a chatbot project. I once wrote a blog post about how to <a href="https://chatbotsmagazine.com/chatbot-training-quest-for-the-holy-grail-6a028ca3a79b">gather training data for a chatbot</a>:</p><p><a href="https://chatbotsmagazine.com/chatbot-training-quest-for-the-holy-grail-6a028ca3a79b">Chatbot Training: Quest For the Holy Grail</a></p><h3>Continuous Feedback</h3><p>As soon as the chatbot is live, there is constant re-training required — this process involves manual work to evaluate real user conversations that for some reason went wrong and to deduct the required training steps. During this process, the test coverage will be increased — for the short-tail topics we can expect a near 100% test coverage within several weeks after launch: those are the topics asked over and over again, and as complex as human language may be, there is only a finite number of options how to express an intent in a reasonable short way.</p><p>As opposed to the title <em>“Testing Chatbots for the Unexpected”</em>, you can see that we suggest a very down-to-earth-approach — not testing for the unexpected or the unknown, but testing for “the most likely options” and try to get a high test coverage for those cases (this is possible as human language is complex but finite).</p><h3>Automated Testing</h3><p>When it comes to test automation, the critical question is if something is worth investing the initial automation effort.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dI01eo91Tb4HdHZvWE4UpQ.png" /><figcaption>Test Automation Ramp-Up</figcaption></figure><ul><li>Establishing automated testing for the short-tail topics is a must</li><li>Ensure a smooth handover process to a human agent</li><li>Long-tail topics can be handled with manual testing in the beginning</li></ul><h3>Getting the Tools</h3><p>Without proper tools you will be lost. Our flagship product <a href="https://www.botium.ai/">Botium Box</a> helps you in your path to successful chatbot testing.</p><p><strong>Disclaimer:</strong> Being a follower of ‘Chatbots Journal’ offers lots of perks :) A consultation session with experts from across the industries is a major one. <a href="https://goo.gl/forms/hycKQHlqCRAOoGUh2"><strong>Submit your query here</strong></a> and we will connect you with the right chatbot experts. He might be sitting next door, you never know.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b5a995cc2861" width="1" height="1" alt=""><hr><p><a href="https://medium.com/chatbots-journal/testing-chatbots-for-the-unexpected-b5a995cc2861">Testing Chatbots for the Unexpected</a> was originally published in <a href="https://medium.com/chatbots-journal">Chatbots Journal</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The 5 Step Journey to Becoming a Chatbot Tester]]></title>
            <link>https://blog.chatbotslife.com/the-5-step-journey-to-becoming-a-chatbot-tester-14a441e70e8b?source=rss-ddc3d6734b89------2</link>
            <guid isPermaLink="false">https://medium.com/p/14a441e70e8b</guid>
            <category><![CDATA[testing]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[botium]]></category>
            <category><![CDATA[education]]></category>
            <category><![CDATA[chatbots]]></category>
            <dc:creator><![CDATA[Florian Treml]]></dc:creator>
            <pubDate>Tue, 20 Apr 2021 10:03:41 GMT</pubDate>
            <atom:updated>2021-05-11T12:43:30.539Z</atom:updated>
            <content:encoded><![CDATA[<p>One question we hear a lot is <em>“What’s the best way to start testing a </em><a href="https://chatbotslife.com/why-chatbots-could-be-the-next-big-thing-for-smes-e167a972420"><em>chatbot </em></a><em>?” </em>The answer is surprisingly simple: Build up knowledge by educating yourself, and you will be able to answer this question yourself. This post will walk you through the major milestones that test automation engineers have to take for building this knowledge. We will suggest readings and other material that you might find helpful as well.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*AkrYhK55qOOr6COZ6ruHTQ.png" /><figcaption>Five Step Journey</figcaption></figure><h3>Step One: Build a First Sample Chatbot</h3><p>The best way to get going is to actually <strong>build yourself a sample </strong><a href="https://chatbotslife.com/how-chatbots-and-email-marketing-integration-can-help-your-business-a9b92ce5f31d"><strong>chatbot</strong></a>. The reason is that some things like natural language processing (NLP) might appear like <em>magic</em> if you are new to the topic. They aren’t. I suggest to take one of these tutorials — don’t worry, all of them have <strong>free plans</strong> available:</p><ul><li><a href="https://cloud.google.com/dialogflow/es/docs/quick/build-agent">Quickstart: Build an agent</a> (Google Dialogflow)</li><li><a href="https://cloud.ibm.com/docs/assistant?topic=assistant-getting-started">Getting started with Watson Assistant</a> (IBM Watson Assistant)</li></ul><p>Go the whole way from starting the project to publishing a <a href="https://chatbotslife.com/chatbots-vs-virtual-assistants-what-is-the-difference-f23287e32165">chatbot </a>on a preview website — everything you need is <strong>included in those cloud platforms</strong>.</p><blockquote>I do not recommend to start with a free, self-hosted stack like <a href="https://blog.rasa.com/the-five-step-journey-to-becoming-a-rasa-developer/">Rasa</a>, as it might distract from the journey to become a <a href="https://chatbotslife.com/case-study-building-appointment-booking-chatbot-99fb9047374b">chatbot </a>tester by tearing your focus more towards infrastructure topics and actual chatbot development.</blockquote><figure><a href="https://www.chatbotconference.com/"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*e2_KzP64Cta52NyCG5Uwgw.png" /></a></figure><h3>Step Two: Understand Machine Learning Concepts and Metrics</h3><p>When dealing with non-trivial chatbots, there is always a <strong>machine learning component</strong> for natural language processing (NLP) involved. You do not have to get an expert in machine learning to efficiently test them, but you definitely have to <strong>understand the taxonomy</strong> used in machine learning projects. The central concepts to learn are:</p><ul><li>Intents, Entities, Utterances</li><li>Accuracy, Sensitivity, Specificity, Precision, Recall, F1-Score</li><li>Confidence and Confidence Threshold</li><li>Confusion Matrix</li></ul><p>Here are some links to introductory articles:</p><ul><li><a href="https://www.linkedin.com/pulse/upskilling-test-automation-engineers-testing-ai-florian-treml/">Upskilling for test automation engineers — Testing Conversational AI</a></li><li><a href="https://medium.com/analytics-vidhya/quality-metrics-for-nlu-chatbot-training-data-part-1-confusion-matrix-91ac71b90270">Quality Metrics for NLU/Chatbot Training Data</a> (<a href="https://medium.com/analytics-vidhya/quality-metrics-for-nlu-chatbot-training-data-part-2-embeddings-57aa341d81fa">Part 2</a>)</li><li><a href="https://floriantreml.medium.com/upskilling-test-engineers-for-chatbot-projects-44c96e4f26a0">Upskilling Test Engineers for Chatbot Projects</a></li></ul><h3>Trending Bot Articles:</h3><blockquote><a href="https://chatbotslife.com/chatbot-trends-report-2021-4181eef67dcc">1. Chatbot Trends Report 2021</a></blockquote><blockquote><a href="https://chatbotslife.com/4-dos-and-3-don-ts-for-training-a-chatbot-nlp-model-536f949e21ae">2. DO’s and 3 DON’Ts for Training a Chatbot NLP Model</a></blockquote><blockquote><a href="https://chatbotslife.com/concierge-bot-handle-multiple-chatbots-from-one-chat-screen-3321f785b32a">3. Concierge Bot: Handle Multiple Chatbots from One Chat Screen</a></blockquote><blockquote><a href="https://chatbotslife.com/an-expert-system-conversational-ai-vs-chatbots-6b7c17c99258">4. An expert system: Conversational AI Vs Chatbots</a></blockquote><h3>Step Three: Get Comfortable with BotiumScript</h3><p>You learned how to test websites with Selenium. You learned how to test smartphone apps with Appium. You might be tempted to think <em>A chatbot is basically a website answering questions, why not using Selenium or Appium for testing a chatbot ? </em>Chatbots are a new kind of apps that require <strong>new kind of tools to perform effective and high-coverage testing</strong> — a goal that cannot be reached with slow, real-time, flaky end-2-end tests.</p><blockquote>Wrote about this conflict in <a href="https://floriantreml.medium.com/upskilling-test-engineers-for-chatbot-projects-44c96e4f26a0">one of my previous blog posts</a>.</blockquote><p>So invest some days to educate yourself with <strong>BotiumScript</strong>, an easy-to-learn scripting language for describing chatbot test cases. You can start at the official <a href="https://botium-docs.readthedocs.io/en/latest/03_testing/01_testing_conversational_ai.html">Botium Docs</a> or watch <a href="https://medium.com/u/53edf505a94b">Shama Ugale</a> in one of her awesome talks:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FqIetL10YAU0%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DqIetL10YAU0&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FqIetL10YAU0%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/5d225386334b0a702b7b7ba12a0f24b2/href">https://medium.com/media/5d225386334b0a702b7b7ba12a0f24b2/href</a></iframe><h3>Step Four: Get Your Toolstack In Place</h3><p>In kitchen one of the most important phrases is <a href="https://en.wikipedia.org/wiki/Mise_en_place">mise en place</a> — it means to put everything into place before getting started with cooking. Now that you have a good understanding of the basics, it is time to <strong>get your tool stack in place</strong> you can use for your chatbot testing projects.</p><ul><li>Subscribe to a <a href="https://www.botium.ai/pricing/"><strong>free-forever Botium Box Mini</strong></a> instance to <strong>optimize your learning curve</strong> — Botium Box includes <a href="https://wiki.botiumbox.com/get-started/get-started-with-botium-box/">everything to get you started</a>, all you need is a web browser</li><li>For <strong>command line aficionados</strong>, install <a href="https://botium-docs.readthedocs.io/en/latest/04_usage/index.html#botium-cli-documentation"><strong>Botium CLI</strong></a>, the swiss-army-knife for chatbot testing</li><li>If you are used to <strong>test runners</strong> like Mocha, Jest or Jasmine, you will love <a href="https://botium-docs.readthedocs.io/en/latest/04_usage/index.html#botium-bindings-documentation"><strong>Botium Bindings</strong></a></li></ul><blockquote>To get started in Botium Box, read through the <a href="https://wiki.botiumbox.com/">Botium Box Wiki</a> with tutorials, howtos, technical background and reference docs.</blockquote><h3>Step Five: Understand the Challenges of Chatbot Testing</h3><p>There are some important differences when testing a chatbot as compared to websites or smartphone apps. What will strike experienced test automation engineers most is that <strong>a user can ask a chatbot literally anything</strong> (and the users will do as soon as the chatbot is live, believe me). Dealing with non-determinism is a <a href="https://martinfowler.com/articles/nonDeterminism.html">critical topic in software testing</a>. Due to the nature of human language it is <strong>impossible for software tests to cover all possible situations</strong>.</p><blockquote>You can read more in <a href="https://chatbotsmagazine.com/how-to-test-a-chatbot-part-1-why-is-it-so-hard-10f1ee8ca37d">one of my previous blog posts</a>.</blockquote><p>Because of this it is even more important to <strong>understand the required test levels</strong> to meet at least the <a href="https://christoph-boerner.medium.com/unveiling-the-value-of-chatbot-testing-5c5278087d41"><strong>minimum quality standards</strong></a> in software projects.</p><ul><li><a href="https://botium-docs.readthedocs.io/en/latest/03_testing/01_testing_conversational_ai.html#testing-conversational-flow">Regression Testing on API level</a>— Identify flaws in the conversation flow before going to production</li><li><a href="https://wiki.botiumbox.com/how-to-guides/nlu-nlp-testing/">NLP Testing</a> — Improve your chatbot understanding</li><li><a href="https://wiki.botiumbox.com/how-to-guides/e2e-testing/">E2E Testing</a> — Verifying the end-user experience</li><li><a href="https://wiki.botiumbox.com/how-to-guides/voice-app-testing/">Voice Testing</a> — Understand your users on voice channels</li><li><a href="https://wiki.botiumbox.com/how-to-guides/performance-testing/">Performance Testing</a> — Ensure your chatbot is responsive under high load</li><li><a href="https://wiki.botiumbox.com/how-to-guides/security-testing/">Security Testing</a> — Making your chatbot secure</li><li>Monitoring — Get notified when problems arise</li></ul><blockquote>Read about some important pitfulls here: <a href="https://chatbotslife.com/4-dos-and-3-don-ts-for-chatbot-testing-strategies-763446aa91e9">4 DO’s and 3 DON’Ts for Chatbot Testing Strategies</a></blockquote><p>The good news is that you can <strong>shine and partially re-use</strong> your test automation skills gathered in your previous projects in chatbot testing as well.</p><h3>Up and Beyond</h3><p>Wherever you are in your journey, from building your first sample chatbot to running a full-blown continuous chatbot testing pipeline, <strong>we’re delighted to have you as part of our Botium community</strong>. Please feel free to <a href="https://forum.botium.ai/">ask questions on our <strong>Botium forum</strong></a>. We look forward to helping you in your journey to become a chatbot testing expert!</p><h3>Don’t forget to give us your 👏 !</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/780/0*ZxaGcKC3UQEydQIX" /></figure><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2F3e59ef%3Fas_embed%3Dtrue&amp;dntp=1&amp;url=https%3A%2F%2Fupscri.be%2F3e59ef&amp;image=http%3A%2F%2Fapi.screenshotlayer.com%2Fapi%2Fcapture%3Faccess_key%3Dfe59908dad3baab69ffab249a2224b03%26viewport%3D1024x612%26width%3D1000%26url%3Dhttps%253A%252F%252Fupscri.be%252F3e59ef%253Fscreenshot&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/7078d8ad19192c4c53d3bf199468e4ab/href">https://medium.com/media/7078d8ad19192c4c53d3bf199468e4ab/href</a></iframe><figure><a href="https://chatbotslife.com/bot-communities-mastermind-group-d2dae9876709#.53x0py6ou"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*6XUspT9JOSq0w0Fi35HIaA.png" /></a></figure><figure><a href="https://m.me/ChatbotsLife"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*c1LDMH5vbnIz9rmAka8Hwg.png" /></a></figure><figure><a href="https://chatbotslife.com/chatbot-development-hire-top-ai-chatbot-developers-c8b45ef7f207"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*D0Jf3dI6ZThtqcfwDYY7mg.png" /></a></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=14a441e70e8b" width="1" height="1" alt=""><hr><p><a href="https://blog.chatbotslife.com/the-5-step-journey-to-becoming-a-chatbot-tester-14a441e70e8b">The 5 Step Journey to Becoming a Chatbot Tester</a> was originally published in <a href="https://blog.chatbotslife.com/">Chatbots Life</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Upskilling Test Engineers for Chatbot Projects]]></title>
            <link>https://floriantreml.medium.com/upskilling-test-engineers-for-chatbot-projects-44c96e4f26a0?source=rss-ddc3d6734b89------2</link>
            <guid isPermaLink="false">https://medium.com/p/44c96e4f26a0</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[skills]]></category>
            <category><![CDATA[botium]]></category>
            <category><![CDATA[testing]]></category>
            <dc:creator><![CDATA[Florian Treml]]></dc:creator>
            <pubDate>Fri, 26 Mar 2021 08:07:37 GMT</pubDate>
            <atom:updated>2021-05-11T12:43:28.391Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>If you have a hammer, every problem looks like a nail.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*3u82AVwG0-5Rjz-LJBofJQ.png" /><figcaption>This is Mjolnir, Thor’s hammer</figcaption></figure><p>With <a href="https://www.botium.ai/">Botium</a>, we are currently defining the industry standard for testing chatbots. In our support and developer channels we are regularily receiving questions like:</p><ul><li><em>I have to test a Whatsapp chatbot, can you help me to set up Appium for it ?</em></li><li><em>For our client I have to test a chatbot embedded in their app, can I test it with Botium ?</em></li><li><em>I have troubles with testing the customer support chatbot on our website, Selenium says &lt;some random Selenium error code&gt;</em></li><li>… and so on</li></ul><p>The conclusion to draw from these questions is: the test engineers learned how to test websites with Selenium and smartphone apps with Appium in the past, and now they try to <strong>apply this valueable knowledge again </strong>— neglecting the fact that chatbots are a new kind of apps that require <strong>new kind of tools (like Botium).</strong></p><blockquote>You can read about the most important differences in <a href="https://chatbotsmagazine.com/how-to-test-a-chatbot-part-1-why-is-it-so-hard-10f1ee8ca37d">one of my previous blog posts</a>.</blockquote><p>With Selenium and Appium, we are talking about <strong>End-2-End testing</strong> (E2E) — simulating the full user experience on a graphical user interface. Those tests</p><ol><li>are extremely slow in execution, as they are basically running in <strong>realtime</strong>, and even for a medium-size chatbot project there typically is a <strong>5-figure number of test cases</strong> for having a satisfying test coverage — running those tests in and E2E scenario will take hours in best case</li><li>require a <strong>high amount of computing resources</strong> or access to expensive browser/device cloud services</li><li>are <strong>flaky</strong> as the required infrastructure is error-prone as well</li><li>cannot provide a holistic view of the test object quality, as some important assertions as the pure NLP performance are <strong>technically not possible</strong> at all with E2E testing.</li></ol><p>So here are my recommendations for test engineers how to get going when asked for testing a chatbot.</p><h3>API First</h3><p>The most important metric for a chatbot is: <strong>is it able to do a meaningful conversation with a client ?</strong> In every chatbot project team there are conversation designers which, well, design the conversations that will make up the final user experience. The chatbot engine is trained (or coded) to provide the logic for these conversations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZPxMA4MgJ3zMReOeL4HOfw.png" /><figcaption>Conversation flow as visualized by Botium</figcaption></figure><p>And this is the place to start testing: make sure that the <strong>conversations are working as designed</strong>, from a content perspective. You can read more about conversation flow testing <a href="https://botium-docs.readthedocs.io/en/latest/03_testing/01_testing_conversational_ai.html#testing-conversational-flow">in the Botium docs</a>.</p><p>One <strong>important skill</strong> to have is knowing <a href="https://botium-docs.readthedocs.io/en/latest/05_botiumscript/index.html">BotiumScript</a>, the scripting language to define conversation flow test cases.</p><h3>Testing the NLP engine</h3><p>Most chatbots have <strong>some kind of natural language processing</strong> (NLP) component as part of the processing pipeline — it enabled users to communicate with the chatbot in natural language, and that’s what actually makes up a chatbot. As a test engineer it is your job to <strong>explore the limits</strong> of the NLP engine, and this requires <strong>basic skills in machine learning concepts</strong>, such as</p><ul><li>intents, entities and prediction confidence</li><li>accuracy, sensitivity, specificity, precision, recall, F1-score</li><li>confusion matrix</li></ul><p>You can read about it in my blog series Quality Metrics for NLU/Chatbot Training Data:</p><p><a href="https://medium.com/analytics-vidhya/quality-metrics-for-nlu-chatbot-training-data-part-1-confusion-matrix-91ac71b90270">Quality Metrics for NLU/Chatbot Training Data, Part 1: Confusion Matrix</a></p><h3>E2E Smoketest</h3><p>Testing the end-user experience on user interface level is an <strong>important part of a testing strategy</strong>. When doing it right you now have the confidence the conversation flow and the NLP component are doing their work, so it is now time to add some user interface testing to the mix. The recommendation is to</p><ul><li>do a <strong>small number of test cases</strong>, which cover all of the possible user interaction elements</li><li>do those tests on a <strong>mix of representative browser versions</strong> / operating systems / smartphone devices, both virtual and physical</li></ul><blockquote>The good news is that here test engineers can shine with the existing knowledge on Selenium and Appium!</blockquote><p>Read on in the <a href="https://wiki.botiumbox.com/how-to-guides/e2e-testing/">Botium Wiki</a> how to setup this with Botium!</p><h3>Non-Functional Testing</h3><p>Finally, there are also non-functional tests like performance tests and security tests to add to the test mix. Opposed to the other test types those are typically done on certain milestones in the project.</p><p><a href="https://chatbotslife.com/security-threats-and-security-testing-for-chatbots-325d704da9af">Security Threats and Security Testing for Chatbots</a></p><h3>Summary</h3><p>A new generation of apps such as chatbots require a new generation of testing tools, like Botium. Test engineers have to develop additional skills for testing conversational interfaces like chatbots.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*WCN13a8CRM4CoO6v.png" /><figcaption>Botium Test Project Types</figcaption></figure><blockquote><a href="https://www.botium.ai/pricing/">Get your free Botium Box Mini instance here</a></blockquote><ul><li><a href="https://botium-docs.readthedocs.io/en/latest/03_testing/01_testing_conversational_ai.html#testing-conversational-flow">Conversation Flow Testing</a> — Identify flaws in the conversation flow before going to production</li><li><a href="https://wiki.botiumbox.com/how-to-guides/nlu-nlp-testing/">NLP Testing</a> — Improve your chatbot understanding</li><li><a href="https://wiki.botiumbox.com/how-to-guides/e2e-testing/">E2E Testing</a> — Verifying the end-user experience</li><li><a href="https://wiki.botiumbox.com/how-to-guides/voice-app-testing/">Voice Testing</a> — Understand your users on voice channels</li><li><a href="https://wiki.botiumbox.com/how-to-guides/performance-testing/">Performance Testing</a> — Ensure your chatbot is responsive under high load</li><li><a href="https://wiki.botiumbox.com/how-to-guides/security-testing/">Security Testing</a> — Making your chatbot secure</li><li>Monitoring — Get notified when problems arise</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=44c96e4f26a0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[4 DO’s and 3 DON’Ts for Training a Chatbot NLP Model]]></title>
            <link>https://blog.chatbotslife.com/4-dos-and-3-don-ts-for-training-a-chatbot-nlp-model-536f949e21ae?source=rss-ddc3d6734b89------2</link>
            <guid isPermaLink="false">https://medium.com/p/536f949e21ae</guid>
            <category><![CDATA[chatbots]]></category>
            <category><![CDATA[training]]></category>
            <category><![CDATA[botium]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[nlp]]></category>
            <dc:creator><![CDATA[Florian Treml]]></dc:creator>
            <pubDate>Mon, 08 Mar 2021 20:05:54 GMT</pubDate>
            <atom:updated>2021-05-11T12:45:07.081Z</atom:updated>
            <content:encoded><![CDATA[<p><em>A quick summary of 7 important DO’s and DON’Ts when training an NLP model for a </em><a href="https://chatbotslife.com/ultimate-guide-to-leveraging-nlp-machine-learning-for-you-chatbot-531ff2dd870c"><em>chatbot</em></a><em>. They are best applied before starting a project, but can also help to build a mindset for quality training data in all chatbot project phases.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/0*OBNTmkj5uYENuNcu.png" /></figure><h3>DO’s and DON’Ts</h3><h4>✅DO: think in problem space, not in solution space</h4><p>Users typically think <strong>in problem space, not in solution space</strong>, and so should you. As a quick example, consider the case of a user who ordered a shirt in an online shop and wants to know when it is expected to arrive. Consider this question:</p><ul><li><em>when will my shirt arrive</em></li></ul><p>This is a question from <strong>problem space</strong>, describing the problem the user wants to be solved, while these are from <strong>solution space:</strong></p><ul><li><em>what is the estimated shipping time</em></li><li><em>show me the order status</em></li></ul><p>They are describing how your business will react to the problem.</p><blockquote>Benefit: the <a href="https://chatbotslife.com/how-to-teach-your-chatbot-with-training-data-46c58b873c31">chatbot </a>and the users <strong>speak the same language</strong></blockquote><figure><a href="https://www.chatbotconference.com/"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*e2_KzP64Cta52NyCG5Uwgw.png" /></a></figure><h4>❌DON’T: overload your intents with too many problems</h4><p>As rule-of-thumb, your intents should handle <strong>at most 3–6 user problems</strong> as described above. For each problem you should provide <strong>at least 3 user examples</strong>. Put your focus on the essence of the intent — the solution your <a href="https://chatbotslife.com/smart-chatbots-and-virtual-assistants-are-coming-591638108933">chatbot </a>can provide for your users.</p><blockquote>Benefit 1: content stays <strong>maintainable</strong> and focused</blockquote><blockquote>Benefit 2: <a href="https://en.wikipedia.org/wiki/Separation_of_concerns">separation of concerns</a> makes <strong>dialog building</strong> straight forward</blockquote><h4>✅DO: clear separation of intents vs entities</h4><p>To our surprise it is still a very common pattern to intermix the concepts of intents and entities, and we strongly suggest to <strong>stop doing it</strong>. Consider a real-life example of a fashion store which has trained an NLP model with the <strong>3 intents</strong></p><ul><li><em>order_tshirt</em></li><li><em>order_pants</em></li><li><em>order_socks</em></li></ul><p>In this case there is room for exactly <strong>1 intent</strong> (<em>order</em>) and <strong>3 entities</strong> (<em>shirt</em>, <em>pants</em>, <em>socks</em>). Data scientists training the NLP model maybe won’t notice a real difference, but your developers will be grateful when coding <strong>dialog flow and fulfillment</strong> based on the NLP model output.</p><blockquote>Benefit: maintainable and <strong>clearly defined</strong> NLP model output</blockquote><h3>Trending Bot Articles:</h3><blockquote><a href="https://chatbotslife.com/how-chatbots-and-email-marketing-integration-can-help-your-business-a9b92ce5f31d">1. How Chatbots and Email Marketing Integration Can Help Your Business</a></blockquote><blockquote><a href="https://chatbotslife.com/why-chatbots-could-be-the-next-big-thing-for-smes-e167a972420">2. Why Chatbots could be the next big thing for SMEs</a></blockquote><blockquote><a href="https://chatbotslife.com/my-journey-into-conversation-design-a86dce6b0111">3. My Journey into Conversation Design</a></blockquote><blockquote><a href="https://chatbotslife.com/practical-nlp-for-language-learning-9b3a259fe7b5">4. Practical NLP for language learning</a></blockquote><h4>❌DON’T: repeat sentence patterns in training data</h4><p>When thinking about the question <em>how much training data is sufficient ?</em> you have the resist the general answer <em>the more the better</em>. Having training examples following the same patterns like</p><ul><li><em>order me a shirt</em></li><li><em>order me some shirts</em></li><li><em>order me shirts</em></li></ul><p>in <strong>best case don’t help your NLP model</strong> in classification and will in <strong>worst case even have negative effect</strong> by overfitting your NLP model (but to be honest, when using a state-of-the-art pre-trained NLP model this is usually prevented out-of-the-box).</p><blockquote>Benefit: keeps your training data <strong>small and focused</strong></blockquote><h4>✅ DO: vary sentence structure and key terms</h4><p>Instead of repeating same patterns you absolutely should <strong>vary the sentence structure</strong> for teaching the NLP model <strong>different ways</strong> of a user expressing the problem — here are some good training examples:</p><ul><li><em>order me a shirt</em></li><li><em>need a new shirt</em></li><li><em>dress me up with a fancy new shirt</em></li></ul><p>Depending on the domain it may even make sense to <strong>use a thesaurus</strong>, but — IMPORTANT — only on <strong>entity and key term level</strong>: everything else a state-of-the-art NLP model will learn itself. A special thing to consider here are <strong>country-specific variations</strong>.</p><blockquote>Benefit: makes classification <strong>robust for variations</strong></blockquote><h4>❌DON’T: train the model with misspelled data (but prepare for it)</h4><p>This is one is obvious — especially for entity resolution some kind of <strong>spellchecking</strong> not only in <strong>training but also on live inference</strong> is a must. But also for intent classification the NLP model will in worst case learn rubbish.</p><blockquote>Benefit: makes classification <strong>robust for real user input</strong></blockquote><h4>✅DO: edit and use real user input as training data</h4><p>While you shouldn’t blindly copy&amp;paste real user input to your training data, it is without any doubt <strong>the most valueable source of training data</strong> and future improvements of your <a href="https://chatbotslife.com/ultimate-guide-to-leveraging-nlp-machine-learning-for-you-chatbot-531ff2dd870c">chatbot’s </a>understanding. As long as unsupervised learning for NLP tasks is still in it’s infancy, having some kind of <strong>manual interception and editing process</strong> in place is a must to establish continuous learning.</p><blockquote>Benefit: <strong>improves the quality</strong> of your NLP model with each interaction</blockquote><h3>Action Plan</h3><p>You can find information about how Botium can help in our Wiki and in our Blog:</p><ul><li><a href="https://wiki.botiumbox.com/technical-reference/nlp-nlu-testing/reference-nlp-quality-metrics/">NLP Quality Metrics</a></li><li><a href="https://wiki.botiumbox.com/technical-reference/nlp-nlu-testing/reference-test-set-insights/">Test Set Insights</a></li><li><a href="https://medium.com/@floriantreml/quality-metrics-for-nlu-chatbot-training-data-part-1-confusion-matrix-91ac71b90270">Confusion Matrix</a> and <a href="https://medium.com/analytics-vidhya/quality-metrics-for-nlu-chatbot-training-data-part-2-embeddings-57aa341d81fa">Embeddings</a></li><li><a href="https://wiki.botiumbox.com/how-to-guides/nlu-nlp-testing/">Articles about NLU/NLP in our Wiki</a></li></ul><h3>Don’t forget to give us your 👏 !</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/780/0*ZxaGcKC3UQEydQIX" /></figure><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2F3e59ef%3Fas_embed%3Dtrue&amp;dntp=1&amp;url=https%3A%2F%2Fupscri.be%2F3e59ef&amp;image=http%3A%2F%2Fapi.screenshotlayer.com%2Fapi%2Fcapture%3Faccess_key%3Dfe59908dad3baab69ffab249a2224b03%26viewport%3D1024x612%26width%3D1000%26url%3Dhttps%253A%252F%252Fupscri.be%252F3e59ef%253Fscreenshot&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/7078d8ad19192c4c53d3bf199468e4ab/href">https://medium.com/media/7078d8ad19192c4c53d3bf199468e4ab/href</a></iframe><figure><a href="https://chatbotslife.com/bot-communities-mastermind-group-d2dae9876709#.53x0py6ou"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*6XUspT9JOSq0w0Fi35HIaA.png" /></a></figure><figure><a href="https://m.me/ChatbotsLife"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*c1LDMH5vbnIz9rmAka8Hwg.png" /></a></figure><figure><a href="https://chatbotslife.com/chatbot-development-hire-top-ai-chatbot-developers-c8b45ef7f207"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*D0Jf3dI6ZThtqcfwDYY7mg.png" /></a></figure><h3>Don’t forget to give us your 👏 !</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/780/0*ZxaGcKC3UQEydQIX" /></figure><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2F3e59ef%3Fas_embed%3Dtrue&amp;dntp=1&amp;url=https%3A%2F%2Fupscri.be%2F3e59ef&amp;image=http%3A%2F%2Fapi.screenshotlayer.com%2Fapi%2Fcapture%3Faccess_key%3Dfe59908dad3baab69ffab249a2224b03%26viewport%3D1024x612%26width%3D1000%26url%3Dhttps%253A%252F%252Fupscri.be%252F3e59ef%253Fscreenshot&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/7078d8ad19192c4c53d3bf199468e4ab/href">https://medium.com/media/7078d8ad19192c4c53d3bf199468e4ab/href</a></iframe><figure><a href="https://chatbotslife.com/bot-communities-mastermind-group-d2dae9876709#.53x0py6ou"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*6XUspT9JOSq0w0Fi35HIaA.png" /></a></figure><figure><a href="https://m.me/ChatbotsLife"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*c1LDMH5vbnIz9rmAka8Hwg.png" /></a></figure><figure><a href="https://chatbotslife.com/chatbot-development-hire-top-ai-chatbot-developers-c8b45ef7f207"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*D0Jf3dI6ZThtqcfwDYY7mg.png" /></a></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=536f949e21ae" width="1" height="1" alt=""><hr><p><a href="https://blog.chatbotslife.com/4-dos-and-3-don-ts-for-training-a-chatbot-nlp-model-536f949e21ae">4 DO’s and 3 DON’Ts for Training a Chatbot NLP Model</a> was originally published in <a href="https://blog.chatbotslife.com/">Chatbots Life</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Giving a Voice to SAP Conversational AI]]></title>
            <link>https://floriantreml.medium.com/giving-a-voice-to-sap-conversational-ai-9e4e42d4be1d?source=rss-ddc3d6734b89------2</link>
            <guid isPermaLink="false">https://medium.com/p/9e4e42d4be1d</guid>
            <category><![CDATA[botium]]></category>
            <category><![CDATA[chatbots]]></category>
            <category><![CDATA[testing]]></category>
            <category><![CDATA[test-automation]]></category>
            <category><![CDATA[voice-assistant]]></category>
            <dc:creator><![CDATA[Florian Treml]]></dc:creator>
            <pubDate>Mon, 01 Mar 2021 17:26:03 GMT</pubDate>
            <atom:updated>2021-03-01T17:26:03.817Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Voice platforms like Alexa and Google Assistant make it easy to provide a custom voice experience to your clients, even without going deeper in audio processing — everything is part of the platform. But what if you already invested quite some effort into building a chatbot on SAP Conversational AI ? You certainly don’t want to switch to a totally new platform now.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*O9s1V4mVO8qJ5jKYrxf8DA.png" /><figcaption>Voice Testing with Botium and SAP Conversational AI</figcaption></figure><p><a href="https://blogs.sap.com/2021/02/22/giving-a-voice-to-sap-conversational-ai-challenge-submission/">Head on to my blog article in the SAP Community!</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9e4e42d4be1d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[4 DO’s and 3 DON’Ts for Chatbot Testing Strategies]]></title>
            <link>https://blog.chatbotslife.com/4-dos-and-3-don-ts-for-chatbot-testing-strategies-763446aa91e9?source=rss-ddc3d6734b89------2</link>
            <guid isPermaLink="false">https://medium.com/p/763446aa91e9</guid>
            <category><![CDATA[botium]]></category>
            <category><![CDATA[automation]]></category>
            <category><![CDATA[chatbots]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[testing]]></category>
            <dc:creator><![CDATA[Florian Treml]]></dc:creator>
            <pubDate>Tue, 09 Feb 2021 08:12:00 GMT</pubDate>
            <atom:updated>2021-05-11T13:08:19.461Z</atom:updated>
            <content:encoded><![CDATA[<p><em>A quick summary of 7 important DO’s and DON’Ts when designing a chatbot testing strategy. We are continuously seeing teams ignoring those actually rather simple rules.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*nLAkMZhTvG_UuC_nleJSoA.png" /></figure><h3>DO’s and DON’Ts</h3><h4>✅<strong>DO plan for iterations</strong></h4><p>In german we say <em>rome was not built in a day — </em>same applies for your <a href="https://chatbotslife.com/ultimate-guide-to-leveraging-nlp-machine-learning-for-you-chatbot-531ff2dd870c">chatbot </a>training data. A robust chatbot is built by multiple iterations, training and testing cycles and by ongoing monitoring and performance tuning: <strong>CODE, TEST, DEPLOY, REPEAT</strong></p><h4>❌ <strong>DON’T underestimate the need for constant performance measurement</strong></h4><p>Without measuring performance with real user conversations, you will never know if your <a href="https://chatbotslife.com/how-to-teach-your-chatbot-with-training-data-46c58b873c31">chatbot </a>is <strong>really working</strong> for your users.</p><figure><a href="https://www.chatbotconference.com/"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*e2_KzP64Cta52NyCG5Uwgw.png" /></a></figure><h4>✅DO<strong> apply the 80/20 rule for testing utterances</strong></h4><p>Most teams are tempted to use 100% of the available data for training. Do not do this. You won’t know if your training data works if you use parts of the training data additionally for testing. Rule of thumb is to use <strong>80% of data for training and 20% for testing</strong>.</p><p>If <strong>and only if</strong> the amount of available data is very small you may try K-Fold validation to get some insights about the quality of your data.</p><h4>❌ DON’T rely on smoke tests or happy path tests</h4><p>Again the 80/20 rule — 20% is the work spent for the comfort zone, 80% of the work is testing and bugfixing. 20% of your users will follow the happy path, <strong>80% will break out</strong>. Prepare for this.</p><h4>✅DO: spend a reasonable amount of time with explorative testing</h4><p>Automated regression testing is superior for finding defects that you know can happen. <strong>It won’t help to find defects you don’t know about.</strong> Spend some time with explorative (=manual) testing: try to bring your <a href="https://chatbotslife.com/10-tips-on-creating-an-addictive-chatbot-experience-b796ea6d1178">chatbot </a>to its limits and beyond.</p><h3>Trending Bot Articles:</h3><blockquote><a href="https://chatbotslife.com/how-chatbots-and-email-marketing-integration-can-help-your-business-a9b92ce5f31d">1. How Chatbots and Email Marketing Integration Can Help Your Business</a></blockquote><blockquote><a href="https://chatbotslife.com/why-chatbots-could-be-the-next-big-thing-for-smes-e167a972420">2. Why Chatbots could be the next big thing for SMEs</a></blockquote><blockquote><a href="https://chatbotslife.com/my-journey-into-conversation-design-a86dce6b0111">3. My Journey into Conversation Design</a></blockquote><blockquote><a href="https://chatbotslife.com/practical-nlp-for-language-learning-9b3a259fe7b5">4. Practical NLP for language learning</a></blockquote><h4>❌ DON’T:<strong> ignore the need to re-test after training</strong></h4><p>You can never know what effect adding some training data on one end of your fine-tuned NLU model will have on the other end, until you try out. Do a full regression test of your NLU model <strong>every single time</strong> you make changes.</p><h4>✅DO: test processing of out of order messages</h4><p>One of the most human-like bevaviour is to scroll up the conversation history in the <a href="https://chatbotslife.com/10-tips-on-creating-an-addictive-chatbot-experience-b796ea6d1178">chatbot </a>window and <strong>resume from a previous step</strong>. Most chatbots out there will fail this challenge if not prepared accordingly.</p><h3>Action Plan</h3><p>Here are suggestions to address the DO’s and DON’Ts.</p><h4>Establish Continuous Testing Mindset</h4><p>Testing is a crucial part of the development process. There is no such thing as a <em>single testing phase</em> when bringing a <a href="https://chatbotslife.com/how-to-make-your-chatbot-sound-natural-606dda87e782?">chatbot </a>to life. Testing has to be part of the team’s daily business, just like coding, design and monitoring.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/0*nx6zt_gaGCJm0o_y.png" /><figcaption>Awesome picture showing continuous testing</figcaption></figure><h4>Holistic Testing</h4><p>For chatbots as for software products in general, there is more than unit tests coded by the programmers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DXWz_lmn9Ia1nwPLHgd-eg.png" /><figcaption>Botium Test Project Types</figcaption></figure><ul><li><a href="https://botium-docs.readthedocs.io/en/latest/03_testing/01_testing_conversational_ai.html#testing-conversational-flow">Regression Testing on API level</a> — Identify flaws in the conversation flow before going to production</li><li><a href="https://wiki.botiumbox.com/how-to-guides/nlu-nlp-testing/">NLP Testing</a> — Improve your chatbot understanding</li><li><a href="https://wiki.botiumbox.com/how-to-guides/e2e-testing/">E2E Testing</a> — Verifying the end-user experience</li><li><a href="https://wiki.botiumbox.com/how-to-guides/voice-app-testing/">Voice Testing</a> — Understand your users on voice channels</li><li><a href="https://wiki.botiumbox.com/how-to-guides/performance-testing/">Performance Testing</a> — Ensure your chatbot is responsive under high load</li><li><a href="https://wiki.botiumbox.com/how-to-guides/security-testing/">Security Testing</a> — Making your chatbot secure</li><li>Monitoring — Get notified when problems arise</li></ul><h4>Get the Right Tools In Place</h4><p>Without the right tools you will be lost. With <a href="https://www.botium.ai/">Botium Box</a> you are prepared for the challenges of getting them in place and integrate them into your chatbot development lifecycle.</p><blockquote><a href="https://www.botium.ai/pricing/">Get your free Botium Box Mini instance here</a></blockquote><p>See this article in spanish <a href="https://planetachatbot.com/consejos-probar-entrenamiento-chatbot-a881650aaa32">here</a>! 🇪🇸</p><h3>Don’t forget to give us your 👏 !</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/780/0*ZxaGcKC3UQEydQIX" /></figure><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2F3e59ef%3Fas_embed%3Dtrue&amp;dntp=1&amp;url=https%3A%2F%2Fupscri.be%2F3e59ef&amp;image=http%3A%2F%2Fapi.screenshotlayer.com%2Fapi%2Fcapture%3Faccess_key%3Dfe59908dad3baab69ffab249a2224b03%26viewport%3D1024x612%26width%3D1000%26url%3Dhttps%253A%252F%252Fupscri.be%252F3e59ef%253Fscreenshot&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/7078d8ad19192c4c53d3bf199468e4ab/href">https://medium.com/media/7078d8ad19192c4c53d3bf199468e4ab/href</a></iframe><figure><a href="https://chatbotslife.com/bot-communities-mastermind-group-d2dae9876709#.53x0py6ou"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*6XUspT9JOSq0w0Fi35HIaA.png" /></a></figure><figure><a href="https://m.me/ChatbotsLife"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*c1LDMH5vbnIz9rmAka8Hwg.png" /></a></figure><figure><a href="https://chatbotslife.com/chatbot-development-hire-top-ai-chatbot-developers-c8b45ef7f207"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*D0Jf3dI6ZThtqcfwDYY7mg.png" /></a></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=763446aa91e9" width="1" height="1" alt=""><hr><p><a href="https://blog.chatbotslife.com/4-dos-and-3-don-ts-for-chatbot-testing-strategies-763446aa91e9">4 DO’s and 3 DON’Ts for Chatbot Testing Strategies</a> was originally published in <a href="https://blog.chatbotslife.com/">Chatbots Life</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Beginners Guide to Automated Voice App Testing]]></title>
            <link>https://medium.com/swlh/beginners-guide-to-automated-voice-app-testing-4596dd9130fd?source=rss-ddc3d6734b89------2</link>
            <guid isPermaLink="false">https://medium.com/p/4596dd9130fd</guid>
            <category><![CDATA[chatbots]]></category>
            <category><![CDATA[voice-assistant]]></category>
            <category><![CDATA[botium]]></category>
            <category><![CDATA[testing]]></category>
            <dc:creator><![CDATA[Florian Treml]]></dc:creator>
            <pubDate>Wed, 03 Feb 2021 13:39:09 GMT</pubDate>
            <atom:updated>2021-05-11T13:09:54.156Z</atom:updated>
            <content:encoded><![CDATA[<p><em>This guide suggests </em><strong><em>best practices, infrastructure and tools</em></strong><em> to ensure your voice app continues to deliver outstanding user experience.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*eEwuXFv9T7mvZrd7IFQNxw.png" /></figure><h3>Questions When Testing Voice Apps</h3><p>Application of the suggested practices helps answering the questions:</p><ul><li>Is my voice app following the designed conversation flow ? Is the conversation flow working as expected ?</li><li>How does my voice app work under real-life conditions ? Does is handle low audio quality ? Does it handle slow network connections ?</li><li>Is my voice app available 24x7, or are there any interruptions in service ?</li></ul><h3>The Art of Challenging Chatbots</h3><p>The challenges when testing chatbots, escpecially voice-enabled ones, are different ones than when testing apps with a graphical user interface: while a graphical user interface restricts the possible user interactions by the controls it offers, with natural language, the <strong>number of possible user inputs are limitless</strong>. Additional when using voice as user input there are again more variables to take into account: the <strong>individual nuances</strong> in voices, the <strong>quality of the microphone</strong>, the <strong>background noises</strong> sourrounding the speaker, and more — when testing a graphical user interface, a button click is always perceived the same by the application, regardless of who actually clicked it.</p><p>The platforms behind powerful voice applications are still evolving and are subject to <strong>constant improvements</strong> — which means that developers have to <strong>rely on components that they do not own</strong> and the possible influence is limited.</p><h3>Testing the Voice Conversation Flow</h3><p>The open source product <a href="https://www.botium.ai/">Botium</a> provides you with all the tools required for implementing a <strong>comprehensive, holistic test strategy</strong> for your voice apps. You can read about Botium and the background on testing conversation flow in the <a href="https://botium-docs.readthedocs.io/en/latest/03_testing/01_testing_conversational_ai.html">official Botium documentation</a>.</p><p><a href="https://botium-docs.readthedocs.io/en/latest/03_testing/01_testing_conversational_ai.html">Botium Tutorial - Botium documentation</a></p><blockquote>We will use <a href="https://www.amazon.com/Bring-Labs-AG-Shopping-List/dp/B01N188I03">Bring! Shopping List</a> as an example of a voice app to test. It is published as Alexa Skill, and we can use the <a href="https://github.com/codeforequity-at/botium-connector-alexa-avs">Botium Connector for Amazon Alexa with AVS</a> for simulating voice input and output with Botium.</blockquote><p>For details about the presented steps and tools please take a look at the <a href="https://wiki.botiumbox.com/">Botium Wiki</a> and <a href="https://www.botium.ai/blog/">our Blog</a>!</p><h4>Record Test Cases</h4><p>The quickest way to get started is to use the <strong>Live Chat</strong> in Botium Box to record your own voice with your microphone. You can immediately see and listen to the response of your voice app.</p><blockquote>Depending on the <strong>technology</strong> of your voice app, both text and audio response are shown or either of them.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1010/1*xcXzgnUjyEdqfmzU-GQGKg.png" /><figcaption>Botium Box Live Chat — Recorder</figcaption></figure><p>You can save the conversation as test case and make some changes afterwards.</p><ul><li>Refining input and output text and audio</li><li>Using wildcard matching or <a href="https://botium-docs.readthedocs.io/en/latest/05_botiumscript/index.html">utterance lists</a> instead of full text</li><li>Add additional test steps or <a href="https://wiki.botiumbox.com/technical-reference/botium-asserters/">asserters</a></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5a2LAHmLIdCraSdy7heBng.png" /><figcaption>Botium Box Voice Test Case</figcaption></figure><h4>Synthesize Test Cases with Text-To-Speech</h4><p>Instead of recording your own voice for the test cases, you may decide to instead (or additionally) use <strong>synthesized voice sample</strong>s. Botium has it’s own Text-To-Speech and Speech-To-Text platform based on the best open source and cloud engines available — <a href="https://github.com/codeforequity-at/botium-speech-processing">Botium Speech Processing</a>.</p><p>Test cases are showing plain text now instead auf audio input:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1019/1*p1axweuZaW6oXzSeKyZ-Fw.png" /><figcaption>Botium Box Live Chat — Text Input</figcaption></figure><h4>Eliminating Flakiness — Homophone Mappings</h4><p>A typical problem when testing voice apps is that audio transcriptions, especially for low quality audio, can be rather unstable — in test automation we usually rely on hard facts (fixed text assertions), and this will lead to <strong>increased flakiness</strong> of the test results.</p><p>In this example, you can see that instead of <em>okay milch ist auf deiner liste</em> the transcription says <em>okay milch is auf </em><strong><em>s</em></strong><em>einer liste</em> — this one character difference will make a test case fail:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CrpJPLCUaru52SWsxRVVnA.png" /><figcaption>Transcription problem</figcaption></figure><p>Botium provides the option to specify homophone mappings to deal with audio snippets that are often misinterpreted by the Speech-To-Test engine.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Kjle-9Z5m9t6xJbSBexEZA.png" /><figcaption>Specifying Homophone Mappings</figcaption></figure><p>Test cases use these mappings to qualify transcription results as success or failed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LDPOeylTGPF58gyqfEKNDQ.png" /><figcaption>Transcription Problem — Homophone Mapping applied</figcaption></figure><h3>Testing Real-Life Scenarios</h3><p>Using your own microphone in front of your laptop might be a good starting point, but in real-life <strong>voice apps are used in another way</strong> — with smartphone, with a home automation or entertainment device like Alexa or Google Home, in a car. To come up with meaningful End-2-End test cases for these scenarios you will have to make your test data <strong>similar to those scenarios</strong>.</p><ul><li>Add background noise on various levels</li><li>Pitch volume up or down</li><li>Simulate various levels of distance</li><li>Simulate technical restrictions like GSM phone line or low bandwith</li><li>Simulate otherwise bad audio quality like interruptions or various levels of silence</li><li>… you name it …</li></ul><p>In Botium Box you can apply various effects for <strong>simulating real-life usage scenarios</strong> to your own clean recordings or synthesized audio samples.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sKxb__118eIbCJ-aEVaELw.png" /><figcaption>Botium Box Voice Effects</figcaption></figure><h3>Continuous Monitoring</h3><p>The recipe for ensuring availability of your voice app is actually rather simple — all you need is:</p><ul><li>a <strong>smoke test</strong> for checking basic behaviour (for instance, just sending a simple <em>hello</em> to the voice app and listing for a response)</li><li>a <strong>scheduler</strong> to run the smoke test every few minutes</li><li>a <strong>notification mechansim</strong> to inform you in case of failures</li></ul><p>With Botium Box, everything you need is coming out of the <em>box</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yFge4QHoCbtGU4UEENQ5CQ.png" /></figure><h3>Summary</h3><p>Now you know what is needed for automated testing of your voice app, you may give <a href="https://www.botium.ai/">Botium Box</a> a try, or you can stick to the free and open source plan with Botium Core.</p><ul><li>Record your own voice or use synthesized voice</li><li>Apply audio effects for real-life simulation</li><li>Conversation flow testing with Botium</li></ul><p>See this article in spanish <a href="https://planetachatbot.com/automatizar-testing-aplicaciones-voz-9f93e934299d">here</a>! 🇪🇸</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4596dd9130fd" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/beginners-guide-to-automated-voice-app-testing-4596dd9130fd">Beginners Guide to Automated Voice App Testing</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Giving a Voice to Rasa with Botium Speech]]></title>
            <link>https://floriantreml.medium.com/giving-a-voice-to-rasa-with-botium-speech-1c65a7513bdf?source=rss-ddc3d6734b89------2</link>
            <guid isPermaLink="false">https://medium.com/p/1c65a7513bdf</guid>
            <category><![CDATA[chatbots]]></category>
            <category><![CDATA[botium]]></category>
            <category><![CDATA[voice-assistant]]></category>
            <category><![CDATA[rasa]]></category>
            <dc:creator><![CDATA[Florian Treml]]></dc:creator>
            <pubDate>Tue, 02 Feb 2021 08:36:33 GMT</pubDate>
            <atom:updated>2021-06-27T00:25:38.296Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Voice platforms like Alexa and Google Assistant make it easy to build your own voice experience, even without going deeper in audio processing — everything is part of the platform. But what if you want to rather go for a solution hosted by yourself, running an assistant on your own website, in your own infrastructure ?</em></p><p>The goal of this article is to show a way how you can build your own voice platform using Open Source tools <a href="https://rasa.com/">Rasa</a> and <a href="https://github.com/codeforequity-at/botium-speech-processing">Botium Speech Processing</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sA-l3c-rxpwZj_wtRqU5tg.png" /><figcaption>An awesome isometric visualization</figcaption></figure><p>Rasa is a developer-friendly and extensible chatbot building tool for self-hosting. Botium Speech Processing is a unified, developer-friendly API to the best available free and Open-Source Speech-To-Text and Text-To-Speech services. Let’s combine this, but first let’s quickly have a look on the architecture.</p><h3>Architecture</h3><ol><li>User speaks into a microphone</li><li>A Speech-To-Text service translates into text <strong>(Botium Speech Processing)</strong></li><li>An NLU engine extracts information out of the text <strong>(Rasa)</strong></li><li>A dialogue engine builds text response <strong>(Rasa)</strong></li><li>A Text-To-Speech service translates into spoken text <strong>(Botium Speech Processing)</strong></li><li>User listens to the audio file</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*lNFooupZpiOaJPEM" /><figcaption>Picture is from <a href="https://blog.rasa.com/how-to-build-a-voice-assistant-with-open-source-rasa-and-mozilla-tools/">this Rasa blog post</a></figcaption></figure><h3>Installation Steps</h3><p>So let’s come to the fun part.</p><h4>Prerequisites</h4><p>Here is what you need to have available on your workstation:</p><ul><li>Git client</li><li>Docker and Docker-Compose</li></ul><h4>Launch Botium Speech Processing Service</h4><p>Botium Speech Processing comes with a reasonable default configuration.</p><ul><li><a href="http://mary.dfki.de/">MaryTTS</a> for Text-To-Speech</li><li><a href="https://kaldi-asr.org/">Kaldi</a> for Speech-To-Text</li></ul><blockquote>Both of them are free and Open Source and a good match to get started with voice technologies, on the other hand they are without a doubt among the best free voice tools available.</blockquote><p>Launching it can be done with a few command line calls.</p><pre>$ git clone <a href="https://github.com/codeforequity-at/botium-speech-processing.git">https://github.com/codeforequity-at/botium-speech-processing.git</a><br>$ cd botium-speech-processing<br>$ docker-compose up -d</pre><blockquote>Depending on network speed and hardware this step can take a while.</blockquote><p>Pointing your browser to <a href="http://localhost/">http://localhost</a> will show the API explorer for Botium Speech Processing.</p><h4>Setup Rasa</h4><p>We will use <a href="https://github.com/RasaHQ/rasa-demo">Sara, the Rasa Demo Bot</a>, as an example.</p><blockquote>You can find first-hand information from the <a href="https://github.com/RasaHQ/rasa-demo">Github repository</a></blockquote><p>I prefer to use Docker instead of installing everything locally. So you can use these command line calls to download the Rasa demo bot and run a first training:</p><pre>$ git clone <a href="https://github.com/RasaHQ/rasa-demo.git">https://github.com/RasaHQ/rasa-demo.git</a><br>$ cd rasa-demo<br>$ docker run --rm -v .:/app rasa/rasa:latest-full train --domain domain.yml --data data/core data/nlu --out models/dialogue --augmentation 0</pre><blockquote>Depending on network speed and hardware this step can take a while.</blockquote><p>Place this <em>docker-compose.yml</em> file into the Rasa folder:</p><pre>version: &#39;3.0&#39;<br>services:<br>  rasa:<br>    image: rasa/rasa:latest-full<br>    ports:<br>      - 5005:5005<br>    volumes:<br>      - ./:/app<br>    environment:<br>      RASA_DUCKLING_HTTP_URL: http://rasa-duckling:8000<br>    command: run --model models/dialogue --endpoints endpoints.yml<br>  rasa-actions:<br>    build:<br>      context: .<br>    ports:<br>      - 5055:5055<br>  rasa-duckling:<br>    image: rasa/duckling<br>    ports:<br>      - 8000:8000</pre><p>In the file <em>endpoints.yml</em> change the actions endpoint url from <em>http://localhost:5055/webhook</em> to <em>http://rasa-actions:5055/webhook</em>. Now launch the Rasa service:</p><pre>$ docker-compose up -d</pre><p>The Rasa service is now waiting for connections.</p><h4>Add Voice Capabilities to Rasa</h4><p><a href="https://github.com/codeforequity-at/botium-speech-processing/tree/master/connectors/rasa">This Github repository</a> includes a <a href="https://rasa.com/docs/rasa/connectors/custom-connectors/">custom connector</a> based on the Rasa builtin Socket.io-connector which adds Speech-To-Text and Text-To-Speech capabilities to Rasa.</p><p>First, clone the repository and copy the connectors folder to the Rasa folder:</p><pre>$ git clone <a href="https://github.com/codeforequity-at/botium-speech-processing.git">https://github.com/codeforequity-at/botium-speech-processing.git</a><br>$ cd botium-speech-processing<br>$ cp -R connectors &lt;rasa-dir&gt;</pre><p>In the file <em>connectors/rasa/credentials.yml</em>, there is a <strong>sample</strong> <strong>configuration</strong> for the Rasa custom connector.</p><blockquote>You can either use this file directly or copy the configuration of the <em>botium.SocketIOVoiceInput</em> connector to your existing Rasa <em>credentials.yml</em></blockquote><p>Change the file to point to your local workstation for speech processing (it also starts a REST connector for convenience and other tests):</p><pre>botium.SocketIOVoiceInput:<br>  socketio_path: /socket.io<br>  user_message_evt: user_uttered<br>  bot_message_evt: bot_uttered<br>  session_persistence: false<br>  botium_speech_url: <a href="https://speech.botiumbox.com/">http://l</a>ocalhost<br>  botium_speech_apikey:<br>  botium_speech_language: en<br>  botium_speech_voice: dfki-poppy-hsmm</pre><pre>rest:</pre><p>Then, change the <em>docker-compose.yml</em> file for Rasa to use this connector.</p><pre>version: &#39;3.0&#39;<br>services:<br>  rasa:<br>    image: rasa/rasa:latest-full<br>    ports:<br>      - 5005:5005<br>    volumes:<br>      - ./:/app<br>    environment:<br>      PYTHONPATH: &quot;/app/connectors/rasa:/app&quot;<br>      RASA_DUCKLING_HTTP_URL: http://rasa-duckling:8000<br>    command: run --cors &quot;*&quot; --credentials /app/connectors/rasa/credentials.yml --enable-api --model models/dialogue --endpoints endpoints.yml<br>  rasa-actions:<br>    build:<br>      context: .<br>    ports:<br>      - 5055:5055<br>  rasa-duckling:<br>    image: rasa/duckling<br>    ports:<br>      - 8000:8000</pre><p>Restart Rasa to make the changes to your Docker containers.</p><pre>$ docker-compose up -d</pre><h3>Testing</h3><p>There is a simple test client based on the <a href="https://github.com/RasaHQ/rasa-voice-interface">Rasa Voice Interface</a> available in the Botium Speech Processing project.</p><p>In the <em>connectors/rasa/client</em> directory, change the Rasa endpoint in the <em>docker-compose.yml</em> file:</p><pre>version: &#39;3&#39;<br>services:<br>  frontend:<br>    build:<br>      context: .<br>      args:<br>        RASA_ENDPOINT: <a href="https://demo.botiumbox.com/">http://l</a>ocalhost:5005<br>        RASA_PATH: /socket.io<br>        PUBLIC_PATH: /<br>    image: botium/botium-speech-rasa-voice<br>    restart: always<br>    ports:<br>      - 4700:8080</pre><p>Then launch the website with <em>“docker-compose up -d” </em>and access the web interface at <a href="http://localhost:4700/">http://localhost:4700</a> to give a chat to your Rasa chatbot.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6646Otl8L_OqEH_P5Oc81Q.png" /><figcaption>The voice interface in action</figcaption></figure><p>Now it is time to run on your microphone and speakers and have a chat with Rasa!</p><p>See this article in spanish <a href="https://planetachatbot.com/dando-voz-a-rasa-con-botium-speech-4b6b67319efd">here</a>! 🇪🇸</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1c65a7513bdf" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>