<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Lior Messinger on Medium]]></title>
        <description><![CDATA[Stories by Lior Messinger on Medium]]></description>
        <link>https://medium.com/@liormessinger?source=rss-54cfc3017bca------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*y3hVk7Nu04Q4LUeC.jpg</url>
            <title>Stories by Lior Messinger on Medium</title>
            <link>https://medium.com/@liormessinger?source=rss-54cfc3017bca------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 17:52:11 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@liormessinger/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How did scientists discover the AI species?]]></title>
            <link>https://medium.com/@liormessinger/how-did-scientists-discover-the-ai-species-44d1c868503b?source=rss-54cfc3017bca------2</link>
            <guid isPermaLink="false">https://medium.com/p/44d1c868503b</guid>
            <category><![CDATA[science]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[towards-data-science]]></category>
            <category><![CDATA[species]]></category>
            <dc:creator><![CDATA[Lior Messinger]]></dc:creator>
            <pubDate>Tue, 18 Nov 2025 18:03:31 GMT</pubDate>
            <atom:updated>2025-11-18T18:03:31.208Z</atom:updated>
            <content:encoded><![CDATA[<p>In this article, I’d like to tell you a story of a witness. When one starts a new company, one ventures into a new land. That was what I thought when I founded Korra.ai, a company devoted to understanding and applying large-scale AI models. But what I experienced was a new land that was moving under my feet, continuing to expand as I walked forward. In this article, I’d like to explain the string of discoveries that took place and brought us to where we are today: living and breathing and laughing alongside a new species that was created to serve us.</p><p>A few words of explanation are due. The first of them, about the title. One might ask why the term ‘discover’ is used. Isn’t ‘invent’ more appropriate? Or even ‘build’? In my view, the term ‘discover’ is better suited to the situation science is at right now. When people discovered 6,000 years ago that heating sand makes glass, they used a natural phenomenon without understanding it. Meaning, they didn’t have any underlying theory — phases of matter in this case. Thet could not predict, for example, that cooling air would make it liquid.</p><p>In much the same way, today scientists don’t understand LLMs and cannot “explain” them. As we will see, they know how to use them, how to manipulate them, but, alas, a scientific theory is not on the horizon.</p><p>A second justification is needed for the use of the word “species”. Are LLMs a new species? What is a species? One may say it’s a species because it can hear, see, talk, and it just can’t move (yet). Evolutionary processes created several species, each presents a subset of these traits. On the other hand, a species can reproduce. Here, it can’t (yet!). So, I will leave the species-or-not decision to you, the smart Homo Sapiens reader.</p><h3>How did it all begin?</h3><p>Restraint is needed here as we are not going to describe how it <em>all</em> began. We would just go back to the renewed rise of deep neural networks, to the first noticeable milestone that happened in computer vision, in 2012, when deep learning was used to win the ImageNet competition by a large margin. The <a href="https://proceedings.neurips.cc/paper_files/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf">AlexNet paper</a>, authored by now-famous Geoffrey Hinton and Ilya Sutskever, showed how to do that. Soon thereafter, these deep learning concepts were borrowed into Natural Language Processing (NLP, does anyone still use this term?). Two seminal papers, published in 2013, were <a href="https://en.wikipedia.org/wiki/Word2vec">Word2Vec</a> (led by a scientist called Mikolov, from Google) and a paper nicknamed “king — man + woman ≈ queen”, by the same Mikolov, now joined by — yes, you guessed it, Ilya Sutskever. These teams invented a way to convert words into vectors, a list of numbers. Numbers that represented, for the first time, <em>meaning.</em></p><h3>Meaning</h3><p>What is <em>meaning</em>? When we are asked for something’s meaning, we would describe it using other words that are semantically close to it. What is bread? Something you eat that is made of wheat. What’s a chair? Something you sit on.</p><p>So, Meaning is proximity. So that if you draw the numbers that represent ‘king’, on a graph, they will be located near similar words such as ‘queen’. And the distance would be the same as the distance between Man and Woman.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/606/0*l0tPXZlygwCMnlXf.png" /></figure><p>That was really the first foundation on which LLMs stand today. When you turn somethinginto numbers, life becomes so much easier. You can use numerical models to try to predict these numbers, and more importantly, you can measure how well the model did. So, scientists began to work on different ways to improve these predictions.</p><p>What is a ‘model’? Imagine that 39 is the number for a horse, 13 represents dog, and you need to solve the equation X — 13 = Y — 39. In other words, you want to find ‘hare’ and ‘bitch’, right? On the semantic side, our goal is really to create a model that can be used for the male-female analogies. We want to find such X and Y that would also work with other words like say, lion and lioness, or headmaster and headmistress.</p><p>But on the numeric level, how would you “solve” this equation? You’d just try some values and see if they fit it. For example, 15 and 41 would do. Or 113 and 139. Or an infinite number of others. So what’s the big deal?</p><p>That’s what “training models” mean — find the X and Y that would work <em>best</em> for <em>most</em> words. So that, if presented to it waiter — waitress = steward-Z? It could predict that Z would be a stewardess.</p><p>The training starts with random numbers put in X and Y, see how much the error was, and change those numbers in a certain efficient way until it hits as close as possible for all cases. The equations these models need to solve have many more Xs and Ys and results to reach, but the essence is the same.</p><p>By the way, the math is also not much more complicated than presented above. It is multiplication and additions — but ones that are made on vectors and matrices — simply groups of numbers. Math simply took the x multiplication sign and assigned another recipe, another definition, that will work on more than a single number. It looks something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/491/0*Oy6nj9nPDF8cE27X.png" /></figure><p>In this example, we can see that there’s a way to multiply a matrix by a vector, and it would produce another vector. “To train” means to somehow change the numbers inside the matrix, so it will fit the meaning of the sentence. The exact way to change the numbers is not complicated, either, but we can just say it’s out of scope for this article.</p><h3>Training</h3><p>The problem was that to measure the accuracy of a prediction, you need to know what’s the right words. And to achieve good models, you need to have a lot of examples. But how do you get those examples?</p><p>One way that people used was simply to hire low-salaried people to create them. Amazon even created an app for that — called ‘Mechanical Turk’ where you could hire students and off-shore freelancers to create and curate those examples. But that only went so far — it’s expensive, it takes time, and companies had to build and manage structured sub-organizations to produce quality examples. A breakthrough was badly needed.</p><p>NLP science chased its own tale for a few years, until in 2018, Ilya Sutskever <a href="https://www.wired.com/story/what-openai-really-wants/?utm_source=chatgpt.com">co-foudned OpenAI</a> with Musk, Brockman and Altman and took the first step towards what we have today. He used two new ideas. One, was a new architecture called Transformers which was introduced a couple of years earlier. Second, was a method to train models automatically. This method was unbelievably simple. All that is needed was to take an existing sentence from somewhere, give the model a few words, and train it to predict the next word. With no more humans in the loop, the main question left was about the textual material on which you can train models. Where do you find such vast amounts of text? Look no further than here. It’s called the World Wide Web. The whole internet is open, public and available for us to use.</p><p>Ilya hypothesized that using Transformers, and maybe more importantly, next-word prediction to train models at scale could give language models <em>understanding</em>. He was right.</p><h3>Understanding</h3><p>In our day-to-day world, what does ‘understanding’ mean? If we see a woman stumbling and falling to the ground, we understand that she might be aching. If we see someone laugh, we understand he’s happy. If we see a bird on the ground, we know it will soon take off. We have knowledge about the world, which was acquired during our lifetime of learning or by billions of years of evolution (some knowledge is innate — for example, interpreting face expressions).</p><p>This world knowledge is nicely demonstrated when we converse. For example, if one would say ‘I saw Ben in class, he was laughing. I’m sure he feels _______ — you could predict the next word to be ‘happy’ or ‘content’ or some other positive feeling. You know that because you have knowledge about the world stored in your head somehow.</p><p>The ingenuity of Sutskever and friends was to turn this concept on its head. We humans were trained in the world, so we can predict the next word. What if we train a model to predict the next word — would it have world knowledge?</p><p>And so they started to train GPT-1. In 2018, around the same time I started to work on Korra, making my first steps in deep learning and totally unaware of the dramatic events around me, they published the paper called ‘Improving Language Understanding by Generative Pre‑Training’. See the nice usage of the word ‘improving’. As it will turn out, that would be one of the biggest understatements in scientific history.</p><h3>Supersize this brain</h3><p>The goal was really to create an improved version of the then-top language models. A generic model that could be used as a foundation for various tasks, such as translation, classification, sentiment analysis, and others. Interestingly, they didn’t come up with new ingenious ways to architect models, no innovative algorithms or anything of that sort. Instead, they changed one parameter: the model size. Just like evolution showed itself that enlarging a creature’s brain <a href="https://askananthropologist.asu.edu/stories/brains-over-brawn">makes it smarter</a>, they simply made the model bigger. How simple was it? Think about it as if you want to build a bigger building. You can try to invent new architecture, add towers, and new structures. Or, you could just add more floors. That’s what they did, adding more floors. And more. And many more. That’s GPT-1.</p><p>What is each floor, you ask? Remember the matrix above? In essence, you add more matrices. And more. And…</p><p>You get the idea.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/733/0*Tgz0uVfNiHhoRFy8.png" /></figure><p>Indeed, performance improved. When measured, translation results came out closer to the correct values, sentiment was identified better, and word classifications were more accurate. The whole thing was not conversational. The tests were done as a completion of a sentence. If given a sentence such as “Good morning, it is still ______”, the model would assign probabilities to the words that could complete it (for example ‘Early’ — 40% ‘Cold’ — 30% ‘Late’ — 0.1% ‘Donkey’ — 0.001%). Or “I hated that film. The film is _____” (‘Bad’ — 45%? ‘Terrible’ — 30% ‘Good’ — 0.004%). Yes, it might be called <em>understanding, </em>but such a term still would be viewed as a marketing slogan, too. The model, so it was argued, simply read enough paragraphs to make statistical connections and no more.</p><p>But there was a measurable improvement, and since the only change to prior models was the size, Ilya and friends just continued the trend. What if we made it bigger, they asked? And so they did, and saw that performance improved even more. They enlarged it further, and interestingly, performance kept improving. And again.</p><p>That was a very interesting discovery. One would expect that the improvement rate would flatten. The bigger you make a car motor, the faster the car goes, yes. But only up to a certain point. At some point, we should see the returns diminishing — because of different factors, like friction and metal strength. In deep learning models, the performance kept increasing linearly with model size, no end in sight. That was strange. When would improvement slow down?</p><h3>Emergent Behaviors</h3><p>As it will turn out, the only limit to increasing performance and AI abilities is the energy we have on this planet, which can drive big enough computers that can host ever-increasing model sizes. But at that time, they didn’t know it yet. What they did notice, however, was another strange phenomenon: the model started to display behaviors it wasn’t trained for.</p><p>When designing a model, the main question is not how well it does with the examples it was given, but how well it generalizes to new ones. For example, when a model was trained on regular images of cats, how well would it identify cats if we show it a partial picture, but still a picture with a cat in it?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/0*F291yTpvQUVJEAr0.png" /></figure><p>The same generalization tests were done for LLMs. For example, one such test could be to complete the question “<em>If a remote is on the table, and the table is in the living room, and the living room is on the first floor, where is the remote? ________</em>“. The model should complete it with ‘first floor’.</p><p>When tested on smaller models, they would reply ‘table’. Which is still impressive, because there is some story here, a question, and a model that understands it needs to answer — and it doesn’t answer with a random word, like ‘computer’. On the other hand, one could argue that this is not world knowledge. In some sense, the prediction is just parroting what was told to the model. In addition, there’s no alignment with what a person would expect. So this is not the intelligence we seek.</p><p>But in <a href="https://arxiv.org/abs/2206.07682#:~:text=Scaling%20up%20language%20models%20has,of%20capabilities%20of%20language%20models.">2022</a>, as model sizes increased, models started to output the right answers. They just unpredictably “emerged” without any special training. That was complemented with answers to other challenges, too: solving multi-step math problems. Generalizing syntactic rules. The list goes on, and it all points in one direction: models that are intelligent. Models that show deeper conceptual understanding, which goes beyond the text they predict. Models that clearly know <em>concepts</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*9N437qSmGkBtPzzD" /><figcaption><em>Graphs of several metrics that improve suddenly and unpredictably as models increase in size (</em><a href="https://arxiv.org/abs/2202.07785"><em>source</em></a><em>)</em></figcaption></figure><p>These models answered questions they couldn’t have been trained on. For example, given the question “Where was Plato born?” a model would answer “Greece”, and one would argue that it could have seen it on some web page. But what if we ask, “What is the country code of the place where Plato was born”? Clearly, there’s no page that answers this silly question. And models started to answer such questions. That’s called <em>reasoning</em>. Or better, <em>intelligence</em>.</p><p>Not only that these behaviors emerged with increasing size, but there was also another surprising pattern. When measuring model size against the intelligence it demonstrates, scientists saw a jump: up to a certain size, bad grades. Suddenly, a jump. No linearity here — and no explanation, too.</p><p>Fast-forward to our days, and we can make a list of all those achievements that we see when interacting with ChatGPT, Claude, Llama, or any other LLM: LLMs explain jokes, write rhyming poems, build applications, and even cheer you up. They generate music, movies, and stories. They have human-like capabilities. And all that <em>emerged</em> just by increasing size! Just by adding more floors to the building.</p><p>Was it all planned? No.</p><p>So, is that a discovery? Yes.</p><h3>How is this species going to evolve?</h3><p>Truth to be told, it’s not a regular scientific discovery. Yes, it emerged spontaneously without prediction, and it wasn’t engineered with intent. But on the other hand, it’s man-made. So, it was both discovered and created — even if by chance.</p><p>Now that this species has been discovered and created, it begs the question: how will it evolve? To what extent will scientists be able to improve it, and will we see the performance curve flattens at some point? Furthermore, scientists use AI capabilities to accelerate scientific progress itself. So, AI improves itself, a clear evolutionary process. To what extent?</p><p>As you can see, many questions afloat and prediction is hard, especially about the future. To me, the main question is different. I’m really interested to know when all this will be explainable and predictable. When will we know how to build a new sense of humor, or add a new field of knowledge, just by changing some numbers inside the model? Will it take mankind 6000 years, like it took us to build a theory about melting sand? To be honest, I think it might.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=44d1c868503b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a knowledge portal platform]]></title>
            <link>https://medium.com/@liormessinger/building-a-knowledge-portal-platform-8cc68973557c?source=rss-54cfc3017bca------2</link>
            <guid isPermaLink="false">https://medium.com/p/8cc68973557c</guid>
            <category><![CDATA[software-architecture]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[knowledge]]></category>
            <category><![CDATA[search-engines]]></category>
            <dc:creator><![CDATA[Lior Messinger]]></dc:creator>
            <pubDate>Tue, 10 Jan 2023 10:33:54 GMT</pubDate>
            <atom:updated>2023-01-11T06:34:20.942Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/768/0*JNybnKCEqChoese_.png" /></figure><p>This is the 2nd post in my “knowledge portal architecture” series. See the first article and introduction <a href="https://korra.ai/the-bible-of-knowledge-management/">here</a></p><h3>Pride In Performance — Which Performance?</h3><p>Imagine you ask a young, nerdish developer what’s the most important goal of his efforts in coding. What would be his answer? I would bet that the majority would go for ‘Performance’. When you think about large-scale big data systems, the first major challenge that comes to mind is how would one handle such enormous amounts of data and still be able to provide a reasonable performance, measured in processing time. There are 3.7 million videos uploaded to YouTube daily, but when you upload a video there, you only care about the one you have uploaded, and you expect to see it online in a reasonable time. And when you search Google, you expect a result to be returned within So <em>processing performance </em>is of extreme importance.</p><p>But there’s another kind of performance that is often overlooked and has an equal, and sometimes even greater importance. How fast can the development team develop and deploy a new feature? When we say ‘develop and deploy’ we obviously imply that the team needs to make it in a way that would not introduce any new bugs and keep the system stable and running. In other words, how does one maintain decent <em>team performance</em>?</p><p>The role of any software architect, VP of R&amp;D, and in fact, any developer, lies in the tension between these two types of performance. We need to develop a system in a timely fashion, that would run reasonably fast, and be flexible enough to accept ongoing development. There’s no silver bullet for that, and every system is different, but still, the industry has treaded constant evolutionary paths all aim to improve these two goals. The <em>processing performance </em>usually is improved by Moore’s law and faster hardware — faster CPUs, faster memory units, more parallelism in GPUs. Making the team perform faster, on the other hand, needs practices like what I’m about to describe in the next part of this limited series.</p><h3>The micro-services bus</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Zy5O5JdnCNxzeJLR" /></figure><p>Indeed the major development that aims to improve what happened in recent years is the shift to micro-services. The need is to build a loosely-coupled system, which means that a break in one system, will stay in that system. A bug, a machine failure, a third-party error — nothing will affect the others. Much like your home fuse box: if one room goes black, the rest can still shine.</p><p>Isn’t that simple to solve? We can just split the huge code base that we probably have into smaller pieces, maybe along business lines. For example — the user manager module, the pricing module, the search module, and So on. The problem is, that a simple split of a monolithic app into microservices is not the whole story. That is easy. The bigger question is how will these modules work together. How will they communicate with each other?</p><p>For example, take the most basic event in Korra’s system — a new tutorial video was uploaded to the system by the admin, in our case, the knowledge manager. There’s a microservice that is responsible for the upload operation, called, in our case, the <em>FS-Uploader</em>. It runs on the backend, waits with an exposed API for upload calls from the browser, and saves the file to the disk.</p><p>Then, once the upload is done, several other operations should be performed. For example, the audio in a / video needs to be recognized and transcribed into text, so that it will be searchable. The video should be converted into various bitrates so that users with low bandwidth — for example, mobile users — can watch it without stutters. We also need to save information about its size so that we can charge users by the amount of data they use.</p><p>In a regular system, we might be tempted to call 3 kinds of services once the file has been uploaded:</p><ol><li>Call the Automatic Speech Recognition service on the file</li><li>Convert the video into multiple bitrates</li><li>Save its size into a database</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/0*Ma6-0MZ-ih8C-Qx4.png" /></figure><p>Looks neat and clean, right? But with a few additional features and a few more months of development, it probably will look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/0*Ug1g3uG77PVG_0tS.png" /></figure><p>They are well separated, but one can clearly see the mess. If one service has a bug, or its machine falters, then all the other services that call it, fail as well.</p><p>The best way of having loosely decoupled modules is by using an event-driven architecture. Going by the name of Publisher-Subscriber pattern, or <strong><em>Pub-Sub </em></strong>for short, it’s been the go-to approach for decades for any well-distributed system. In this architecture, a module publishes an event with data, without knowing who will use it and how.</p><p>There is no direct connection between any of the modules. Once the file was uploaded, the Uploader module publishes a message that a file was uploaded. This message has a type — say, FileWasUploaded, and also carries the information needed — for example, about what file it is, which knowledge base it belongs to, what’s the user id, and so on.</p><p>Now any module that might need to act upon this event, subscribes to FileWasUploaded</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/0*EjJsTeWEOvzZBlwB.png" /></figure><p>The advantages of this approach are huge. It’s more stable — a bug in one module does not crash the rest. Even more importantly, it’s easier to add new modules. What will happen if we have to add a module that, say, emails an administrator on any new file?</p><p>In the first approach, we’ll have to touch two modules: one that does the email, and one that calls it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/0*Y1I___OYbgV8GLpQ.png" /></figure><p>With a bus, we need half of the work! Just add the email module. have it subscribe to events of FileUploaded, the name of the message your FSUploader module sends to the bus when a file is loaded.</p><p>That summarizes my second part of ‘Building a knowledge portal platform’. Be sure to check the next one!</p><p><em>Originally published at </em><a href="https://korra.ai/building-a-knowledge-portal-platform/"><em>https://korra.ai</em></a><em> on January 10, 2023.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8cc68973557c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[ChatGPT, customer service and knowledge management: do we hear a roar of a revolution?]]></title>
            <link>https://medium.datadriveninvestor.com/chatgpt-customer-service-and-knowledge-management-do-we-hear-a-roar-of-a-revolution-2aec14806233?source=rss-54cfc3017bca------2</link>
            <guid isPermaLink="false">https://medium.com/p/2aec14806233</guid>
            <category><![CDATA[knowledge-management]]></category>
            <category><![CDATA[customer-service]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[chatgpt]]></category>
            <category><![CDATA[chatbots]]></category>
            <dc:creator><![CDATA[Lior Messinger]]></dc:creator>
            <pubDate>Sat, 10 Dec 2022 20:28:32 GMT</pubDate>
            <atom:updated>2023-01-12T06:01:32.560Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/512/1*Apgxwb1fypEHbvwGnrCtHw.png" /><figcaption>Photo: Midhourney (“photo of a stage with a non humanoid droid talking angrily to a big crowd of humans, human salesmen, cool lighting, neon, futuristic, dystopia”)</figcaption></figure><h3>TL;DR</h3><p>The internet is exploding about ChatGPT. Can it be industrially used for customer service and knowledge management? Or should we stick with companies like Zendesk, Freshdesk, Korra.ai, Document360 and the like?</p><h3>Advantages:</h3><ul><li>Appears super intelligent! And very well understood</li><li>No need to browse through many search results: can summarize many answers into one</li></ul><h3>Disadvantages</h3><ul><li>Not reliable for non-popular, or too recent subject matters</li><li>Can’t browse through search results and get to the original document</li><li>Limited and costly customization</li></ul><h3>A.m.a.z.i.n.g</h3><p>The internet is exploding about the new ChatGPT, the new AI-powered chat bot from OpenAi. It can yack with you, code for you, and best of all: it answers questions really well. Ask it anything known to man, and it will reply</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*eRkoGvEhgOnVSngc.png" /></figure><p>But what’s more important for businesses, is its ability to answer questions about product use:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*8mRmVwghUOTeGOeu.png" /></figure><p>It has been a long time coming: sales departments, customer success execs and knowledge managers have all been waiting for such a development for years now: a virtual customer service representative that can truly answer any question. Not only answer, but have a conversation with the customer about it. Meaning, answer follow up questions that rely on information provided in the previous steps:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*A3QBjC3q0dTYh5WY.png" /></figure><p>It’s even better than Google: users don’t need to scan lists of search results. The bot chooses the best answer and summarizes it for them.</p><p>A. m. a. z. i. n. g.</p><p>So now, the question is how to put it to use. Let’s think in practical terms: say we are working for an electric vehicle manufacturer. We’d like to have a ChatGPT-based representative that can answer questions prospects has — this seems like a no brainer:</p><p>So, this is a Tesla. But what if the EV has only come out this year, 2022? The bot has knowledge only for until 2021.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*fxL0mGMm6TP3oV8O.png" /></figure><p>Well, it <a href="https://www.saicmotor.com/english/latest_news/mg/57767.shtml">was announced </a>on Jun 22, 2022: “<em>The MG4 Electric accelerates from 0 to 100 km/h in less than 8 seconds, while the top speed is set at 160 km/h.”</em></p><p>Other problems might occur when it’s a more technical device, with a small user base?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*XgsDPuwrz6mmtSVo.png" /></figure><p>As ChatGPT said, it is a <a href="http://www.mennenmedical.com/bilicare">brand of medical devices</a>, so the answer makes perfect sense. But apart from that, it is of no use ,neither to users nor to knowledge managers.</p><h3>Customization</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/512/1*wvp5rkyMuYvuvA2tKhPBAQ.jpeg" /><figcaption>Photo: Dall-E (“photo of a stage with a non humanoid droid talking angrily to a big crowd of humans, human salesmen, cool lighting, neon, futuristic, dystopia”)</figcaption></figure><p>So, all in all, we see that we need to have a way to teach ChatGPT new information. How do we do this?</p><p>If you are using a search engine or a knowledge base like <a href="http://zendesk.com/">Zendesk</a>, Freshdesk, <a href="http://korra.ai/">Korra</a> or Google Drive — you know how easy it is to feed new documents and videos to the engine: just drag and drop it, or provide a Dropbox or a YouTube link. Can one do the same to GPT-type models?</p><p>OpenAI, the company behind ChatGPT, has created the Large Language Models GPT1,GPT2 and then GPT3. These language models are forming the foundation to ChatGPT. They even nick-named ChatGPT as “GPT 3.5”. While we still don’t have documentation for ChatGPT APIs, we can learn a lot by looking at GPT 3 and the ways it can be used.</p><p>These models are pretty similar in use: you ask it a question, and it will generate an answer based on the knowledge that is stored inside. And the question is: can we add knowledge to it?</p><p>The answer is yes! but unfortunately, there are many limitations.</p><p>There are two main ways that GPT3 can be used as a base for customer service, support, or knowledge base management.</p><p>One is called embeddings. Embeddings are the underlying mechanism of AI-based search. In the old Elastic-based knowledge world, sentences were broken into keywords, and stored in a dictionary called ‘inverse index’. A question would then be matched against this dictionary based on a smart formula called TF-IDF, which measures the frequency of a keyword inside a document or a web-page, and how special this keyword is.</p><p>This kind of keyword-based search technology is no longer good enough. Modern AI-based search engines take advantage of language models like BERT, GPT3 and others, by taking each sentence in the document, feeding it into the model and receiving a vector out. This vector — a large list of numbers called embedding — is then stored in a database. When a question comes in, the engine creates an embedding for it too, and the engine finds the closest embedding from the database. Miraculously, the geometric closest embedding is also the one that is the most meaningful to the question. For single words, it looks something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*cob2fkGOMVs1FnAD.png" /></figure><p>So GPT3 has an API for feeding documents into it. The limitation is that the documents has to be short — 2–3 pages in length and not more, and besides, API spells development. There’s no application and UI that can be used to upload documents, search, present them etc.</p><p>Secondly, GPT3 can be fine-tuned. This is useful for exactly those cases where the subject matter at hand is special: a medical device, a unique technological area, etc. But the way to do it is rather painful: knowledge managers should collect question-answer pairs and feed it to the beast. Again the problem is, this is a very long and involved process. One has to know all the</p><p>questions, select answers, and do all that among mountains of materials every product has — in Tesla we are talking about 250 hours of reading materials, 66 videos — oh, and what about videos? Videos are not included, too. And again, all that requires a pretty big effort of development.</p><h3>Advantages and disadvantages</h3><p>To summarize, when we talk about usage of ChatGPT — or GPT in general, for customer service tasks and knowledge bases we see the following:</p><h3>Advantages:</h3><ul><li>Appears super intelligent! And very well understood</li><li>No need to browse through many search results: can summarize many answers into one</li></ul><h3>Disadvantages</h3><ul><li>Not reliable for non-popular, or too recent subject matters</li><li>Inability to browse through search results — can lead to errors</li><li>Limited and costly customization</li></ul><p><em>Originally published at </em><a href="https://korra.ai/chatgpt-customer-service-and-knowledge-management-do-we-hear-a-roar-of-a-revolution/"><em>https://korra.ai</em></a><em> on December 10, 2022.</em></p><p>Subscribe to DDIntel <a href="https://ddintel.datadriveninvestor.com/">Here</a>.</p><p>Visit our website here: <a href="https://www.datadriveninvestor.com/">https://www.datadriveninvestor.com</a></p><p>Join our network here: <a href="https://datadriveninvestor.com/collaborate">https://datadriveninvestor.com/collaborate</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2aec14806233" width="1" height="1" alt=""><hr><p><a href="https://medium.datadriveninvestor.com/chatgpt-customer-service-and-knowledge-management-do-we-hear-a-roar-of-a-revolution-2aec14806233">ChatGPT, customer service and knowledge management: do we hear a roar of a revolution?</a> was originally published in <a href="https://medium.datadriveninvestor.com/">DataDrivenInvestor</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Conquering the Fear of Managing Knowledge]]></title>
            <link>https://medium.com/@liormessinger/conquering-the-fear-of-managing-knowledge-korra-ai-fb899e2d2249?source=rss-54cfc3017bca------2</link>
            <guid isPermaLink="false">https://medium.com/p/fb899e2d2249</guid>
            <category><![CDATA[search]]></category>
            <category><![CDATA[knowledge-economy]]></category>
            <category><![CDATA[search-engines]]></category>
            <category><![CDATA[knowledge-sharing]]></category>
            <category><![CDATA[knowledge-management]]></category>
            <dc:creator><![CDATA[Lior Messinger]]></dc:creator>
            <pubDate>Thu, 08 Sep 2022 14:37:11 GMT</pubDate>
            <atom:updated>2022-11-27T06:50:38.116Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/640/0*cAOkGfRDKGVqw__F.jpeg" /></figure><h3>Why knowledge management is really people leadership</h3><p><em>This is the first article in a series about knowledge management and software architecture. More coming soon.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/433/0*JiYCXYe8qVLj5oXG.png" /></figure><p>In the beginning there was a definition. Before we talk about the challenges associated with Knowledge Management, and how to solve some of them, let’s formalize the different parts of the “Knowledge Management” 2-words duo.</p><h3>What is knowledge?</h3><p>Often overlooked and taken for granted, <em>Knowledge is the basis of every productive human process</em>.</p><p>Now, “productive” often is thought about as producing some tangible product. But it is really not. <em>Productive</em> is not synonymous with <em>manufacturing</em>. Mind you, even a customer service representative, answering the usual “why my bill is so high” question, needs to produce answers in a manner that will keep her interlocutor happy.</p><p>Second, Knowledge definitely is essential for “<a href="https://en.wikipedia.org/wiki/Knowledge_worker">knowledge workers</a>” — those whose raw material and final product is knowledge, like scientists, programmers, or lawyers. But these form only one self-evident example. In reality, any person doing any job needs some amount of knowledge to do her job. It might be the amount that varies between professions.</p><p>Let’s add the “Management” part in. The word is great — it allows some people to finally become a manager. But luckily, managing knowledge does not resemble managing people. According to “Essentials of Knowledge Management” by Bryan Bergeron (2003, Wiley &amp; Sons):</p><p><em>Knowledge Management (KM) is a deliberate, systematic business optimization strategy that selects, distills, stores, organizes, packages, and communicates information essential to the business of a company in a manner that improves employee performance and corporate competitiveness.</em></p><p>(And just like that, we got ourselves an acronym, too: KM).</p><p>Bergeron goes on to describe the process of managing knowledge, which is comprised of several interrelated activities:</p><ul><li><strong>Creation</strong>: create or acquire the knowledge</li><li><strong>Modification</strong>: change the knowledge if needed</li><li><strong>Use</strong>: what good is knowledge if not used?</li><li><strong>Store</strong>: make sure knowledge is safely archived</li><li><strong>Distribute</strong>: allow knowledge to get to the ones who needs it</li><li><strong>Repurpose</strong>: if needed, change the format of the knowledge so that it could be used for other purposes</li></ul><p>As you read this, I’m sure you think of examples for the different activities. It’s easy to think about it in the context of, say, documenting a process in a manufacturing facility — acquiring the knowledge, modifying it, distributing it and so on. However, how about a <a href="https://korra.ai/marketing/">marketing</a> company? What does its knowledge management processes look like?</p><h3>Knowledge is not documentation</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/0*3APwGLTymbZ1mNZu.jpeg" /></figure><p>As a business, your main asset is knowledge. That might be a great catch phrase, kind of those inspirational quotes some managers like to hang on the walls of the office — like “if you can dream it, you can do it” (by the way: that’s simply not true).</p><p>But really, in many many (many) organizations, knowledge is simply the main thing they produced. Take a software company. The code lines developers write are knowledge. As a new employee, your best bet at understanding what is going on is to just read the code. At a law firm, the past contracts and data rooms are where every lawyer spends time finding the right clauses and ideas. Even a marketing agency has past designs, videos and write-ups its workers created and should reuse.</p><p>Now you can understand why, when said that probably we should have added one more “many” when we said that in many many organizations knowledge is the main thing that they produce.</p><p>The problem is that many knowledge managers confuse <em>knowledge </em>with <em>documentation</em>. They believe that the <em>activities </em>described above — creating, maintaining, storing and so on — are actually <em>steps </em>in a separate process that should be initiated, marketed and sold throughout the organization.</p><p>They think that in order to excel as a knowledge manager, they should either do it themselves or motivate employees to help them in that process. What would be the next step then, if you hold such a belief? Just go out and buy a nice authoring software, spend a lot of time on ruminating how to categorize the different subject matters and discuss the right approach to organizing the material in a beautiful hierarchy. The finished system is indeed something one can take pride in, but hey, is that Knowledge Management?</p><p>Well, no. This process could be great in order to create structured courses and e-learning sessions — for either customers or employees. The organization, the hierarchy, the on-going theme — all lend themselves as a great way to explain and teach newly arrived employees and newly acquired customers. That’s why many <a href="https://www.researchgate.net/profile/Vandy-Pacetti-Donelson/publication/334204733_A_Personal_Learning_Environment_Polk_County_School_District_Connects_With_Students_Through_the_Student_Portal/links/615b4d4649da3d5b1139f82e/A-Personal-Learning-Environment-Polk-County-School-District-Connects-With-Students-Through-the-Student-Portal.pdf#page=88">knowledge management</a> departments are overseeing e-learning activities.</p><h3>Knowledge is not writing</h3><p>Knowledge management is not about writing documentation, nor about harassing fellow employees to do it. Yes, good writing always goes a long way to explain concepts and ideas, but it should not be an external activity. It is a major part of the job.</p><p>Great knowledge managers know they should be doing two things.</p><p>One, they should <em>educate</em>. They talk with everyone in the organization — from executives to team leaders to employees. They carry the message of knowledge — and for each department, knowledge management is different.</p><p>For example, dev teams need to tidy up their code, use remarks for explanations (or not — just use super lengthy variable names) — and add supporting articles and diagrams.</p><p>By the way, diagrams and flowcharts are often confused with design documents. They are not the same. The documentation should be done on the finished product — the design phase was just one step towards that. When the product is finished, that’s usually when the engineers know how it works. As Yogi Berra said, prediction is really hard, especially about the future. For knowledge management, developers should explain what they did, not what they plan to do.</p><p>So knowledge managers should work with every department manager and team leader and design a knowledge management plan. For some teams, it could be a part of the review process. For others, a point person whose job is to make sure every piece of knowledge is correctly created, saved and organized.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/0*sgqhUtE_UdMPnGwa.jpeg" /></figure><p>That’s why knowledge management should be like parenting. Parents don’t need to live their kids’ life. They need to make their children comfortable, safe, loved — and let them do the growing-up. Similarly, the knowledge manager’s (in fact, any manager’s) task is to create a safe environment, educate, teach and remove any obstacles that can harm the process.</p><h3>Finding the gold</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/0*w7rbkwfKJB20i2VE.jpeg" /></figure><p>The second important task of great knowledge managers is creating a good way to find the gold. They should make sure knowledge is accessible. What good is a good design the company made two years ago, if no one knows about it? If no one can find it?</p><p>For example, one of the most important tools of any software developer is the<strong> Ctrl-Shift-F </strong>— searching in the project’s source files to understand the flow of data. But developers have an easy way: different variable names mean they have different purposes, even if in English they <em>mean </em>the same thing. lowestProfitForYear and yearSmallestProfit are not the same! They may contain different information and belong to different flows.</p><p>However, when it comes to searching documents and free text, there’s a big problem — how would you search for <em>a meaning</em>? The simple Ctrl-F, or even its older brother Ctrl-Shift-F, wouldn’t do anymore.</p><p>That’s actually one of the main reasons why organizational knowledge management has been somewhat of a black sheep. People spent enormous amounts of time looking for stuff — an old email, a past contract, an answer inside a folder structure of technical documents — sometimes to no avail. Most of them would not complain, trying not to hurt anyone’s feelings, but they would refrain from the same painful process.</p><p>That is only one example of obstacles users face. More and more knowledge is stored in videos, as people prefer to learn from another person, even if it’s just a recording of another person. So video has become the go-to media for creating tutorials and training courses.</p><p>And how do you find an answer in a video? Say the search engine did bring you a link to the right video. How many minutes would you need to watch until you find it?</p><p>Luckily, recent research in Artificial Intelligence has brought in a new breed of search engines. You can see it first in Google — if you search, for example, “controlling light in canon EOS” you’ll find that Google brings you an answer for “How to adjust Shutter, Aperture &amp; ISO on a Canon EOS DSLR”. None of the words, except for the model name are the same, but that’s exactly what you needed to get the right answer! In other cases, Google provides links to exactly the right position in the YouTube video with the answer. For example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/682/0*p8lQtoR009NiQT1K.png" /></figure><p>Due to the deep chasm between the pretty amazing performance of Google, and the broken promises of search engine vendors, IT managers have somewhat lost interest in trying to improve their internal search engine performance. That means users are left with a terrible search experience — which they also know wouldn’t work. What do they do? They leave the company website, search using Google and then go back through the links it gave them! How terrible is that?</p><p>Actually, there are a few vendors that are starting to deliver on that promise. Using AI and other technologies, their search engine yields Google-level results. <a href="https://korra.ai/">Korra.ai</a>, the company I’ve co-founded, is one. And there are others too — each one with its unique characteristics and strong points.</p><p>To summarize, we contend that great knowledge managers do two main things: one, they promote good knowledge sharing. Second, they make knowledge accessible.</p><p>Go out there and start with these two things. You’ll soon discover that good things will follow. And we will follow suit on the next steps, in more articles to come. Cheers!</p><p><em>Originally published at </em><a href="https://korra.ai/knowledge-management/"><em>https://korra.ai</em></a><em> on September 8, 2022.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fb899e2d2249" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Build an End-to-End Conversational AI System using Behavior Trees]]></title>
            <link>https://medium.com/free-code-camp/how-to-build-an-end-to-end-conversational-ai-system-using-behavior-trees-658a7122e794?source=rss-54cfc3017bca------2</link>
            <guid isPermaLink="false">https://medium.com/p/658a7122e794</guid>
            <category><![CDATA[bots]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[tech]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[nodejs]]></category>
            <dc:creator><![CDATA[Lior Messinger]]></dc:creator>
            <pubDate>Fri, 12 Apr 2019 15:48:13 GMT</pubDate>
            <atom:updated>2019-04-16T12:06:42.472Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gr-Ixi5QqOjf5bFR9ybNag.jpeg" /></figure><p>At their core, AI projects can be depicted as a simple pipeline of a few building blocks. The diagram above explains that pretty nicely: Unstructured content, usually in huge amounts of data, comes in from the left, and is fed into AI classifiers. These pre-trained machine- or deep- learning models separate the wheat from the chaff and reduce the input to a few numerical or string output values.</p><p>For example, megas of pixels and colors in an image are reduced to a label: this is a giraffe. Or a zebra. In audio, millions of wave frequencies produce a sentence through Speech To Text models. And in conversational AI, that sentence can be reduced further to a few strings representing the intent of the speaker and the entities in the sentences.</p><p>Once the input has been <em>recognized,</em><strong><em> </em></strong>we need to do things, and generate some meaningful output. For example, a car recognized too close should turn the wheel in an autonomous car. A request to book a flight should produce some RESTFul database queries and POST calls, and issue a confirmation, or a denial, to the user.</p><p>This last part, shown in the diagram as rule-based logic, is an inseparable part of any AI system, and there’s no change in sight to that. It has usually been done by coding, thousands and thousands of lines of code — if it’s a serious system — or a couple of scripts if it’s a toy chatbot.</p><h3>Behavior Trees</h3><p>A <a href="https://en.wikipedia.org/wiki/Behavior_tree_(artificial_intelligence,_robotics_and_control)">Behavior Tree</a> is a programming paradigm that emerged in video games to create human-like behaviors in non-player characters. They form an excellent visual language with which a software architect, a junior developer and even a non-coder, technical designer can all create complex scripts. In fact, since Behavior trees (BTs) allow logic operations like AND and OR, loops and conditions, any program that can be created by code, can be created with BTs.</p><p><a href="https://github.com/servo-ai/servo-platform">Servo </a>is an open-source AI conversational framework built on top of a JavaScript behavior tree framework called <a href="https://github.com/behavior3/">Behavior3</a>. It’s designed to do the needed orchestration of inputs and outputs for conversational AI systems. It’s what is called “low-code” framework: you only need to code a little, and most of the tasks can be done in the visual editor.</p><p>It’s not your usual newbie&#39;s toy: it was designed to be extended using real-life, debuggable JS source files and classes, and abide by any team project-management methodologies. Moreover, it’s suitable for teams that grow in size, allowing the introduction and reuse of new modules through abstracted and decoupled sub-processes.</p><p>I’m the main developer of Servo. After 30-something years of coding, feeling the pain in long-delayed projects and watching legacy systems break under their own weight, I wanted to achieve maximum flexibility with minimum coding. Here I’m going to explain the magic that can be done when one combines Behavior Trees with NLU/NLP engines, using Servo and <a href="https://www.wit.ai/">Wit.ai</a>.</p><p>Any developer can benefit from this tutorial, but it’s best if you are a developer with experience in building chat- or voice-bots and have knowledge working with NLU/NLP engines like LUIS, Wit.AI, Lex (the Alexa engine), or Dialogflow. If you do not, it’s ok, but I’ll be covering some subjects a bit briefly.</p><p>If you want to learn about NLU and NLP engines, there are excellent resources all over the Internet — just search for ‘Wit tutorial’. If you want to learn how to build a heavyweight assistant, then just continue reading.</p><h3>Getting Started with Servo</h3><p>I won’t get into details of installing Servo here, but just say that getting started with Servo is really easy. You can read about it <a href="https://medium.com/datadriveninvestor/building-context-aware-stateful-bots-using-servo-a2dc3f557469">here</a>. In essence, you clone the <a href="https://github.com/servo-ai/servo-platform">Github re</a><a href="https://github.com/servo-ai/servo-platform">po</a>, npm-install it according to the readme, and run it locally. Then, every New Project would start you off with a small ready-made bot to get things going:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wXJFdoNn3MzRDr3R7Vqd5Q.png" /><figcaption>Every New Project has a starting tree</figcaption></figure><p>You can notice here the green hexagons named ‘chit-chat’, ‘cancel’ and others. By the end of this article, you’ll have a clear idea of what they are and how they work. But first, let’s tackle the first challenges.</p><h3>Building An NLU Model</h3><p>Let’s talk about building a banking assistant, and specifically, one that works for the money transfer department. Had it been a web application, we would have a form with few fields, among which the amount and the transferee’s (also called beneficiary) account are the most important. Let’s use these here for this tutorial. Actually, when using NLU engines, we can still think of it as forms, with the fields now called <em>entities</em> (also <em>slots, </em>in Alexa lingo). The NLU engines also produce an intent, which can be viewed as the name of the form that will guide the assistant to the area of that functionality of the user’s intent.</p><p>We should train the NLU engine with a few sentences, such as:</p><ul><li><em>“I’d like to send some money”</em></li><li><em>“I’d like to send $100 “</em></li><li><em>“Please transfer $490 to account #01–10099988”</em></li></ul><p>And for these, we need to tell the engine to output the following:</p><ul><li>a <em>TransferIntent </em>intent for such sentences</li><li>A <em>wit/number </em>for the amount</li><li>An <em>accountNumberEntity </em>for the beneficiary account</li></ul><p>Let’s do that on Wit.ai. Again, I won’t get into a Wit tutorial — there are plenty of guides. Servo comes with a general Wit model which you can take from Servo’s Github <a href="https://github.com/servo-ai/servo-platform/tree/master/server/convocode/nlu-models/wit.ai">here</a>. Then, open your own Wit app and import it.</p><p>I created a <a href="https://wit.ai/docs/recipes#which-entity-should-i-use"><em>free-text</em></a> entity for the account numbers (as account number might include other symbols), and a <em>wit/number </em>entity for the amount. I found <a href="https://medium.com/wit-ai/introducing-composite-entities-ba2639a26e0">composite entities</a> work pretty well, too, although they need some training. For simplicity, for account numbers, I trained the model to be a # followed by 8 digits.</p><p>In general, it’s always better to experiment with different entity models. In our case, we might get two numbers in the same sentence (account number and amount) and we need a way to tell them apart, so it’s best if it’s two different entity names. But you can try other types: AI is still a very empirical science…</p><p>We then trained it with a few sentences and let Wit build the bank-transfer model. For convenience, I added it <a href="https://github.com/servo-ai/servo-platform/tree/master/server/convocode/nlu-models/wit.ai">here</a>, and also set the whole banking bot tutorial bot to come along with the pre-loaded examples.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/879/1*_0QkDep2FkdoiIyXuoaLgA.png" /><figcaption>Training Wit with sample bank transfer sentences</figcaption></figure><p>Last, we need to connect the NLU to the assistant. Go to <em>Settings </em>in Wit, and copy the <strong><em>access token</em></strong>. We need to paste it in our tree root’s properties. We do this by opening Servo’s editor, selecting the root, opening its properties, and pasting it under <strong><em>nlu</em></strong>. As you can see, Servo supports multi-language assistants and different NLU engines:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DpNd__fpR1oG-6QAKZxdJQ.gif" /></figure><h3>Start The Bank Assistant</h3><p>Now, we can turn to Servo. We should construct a small tree with a question for each entity and intent.</p><p>As a reminder, the basic rules of Servo behavior-trees is as follows</p><ol><li>The main loop of the tree is executing the root continuously</li><li>Each node executes its children</li><li>An AskAndMap node (the “Age” node in the diagram above) outputs a question to the user and waits for an answer</li><li>Once an answer comes in, the flow is routed to the appropriate child according to the intent and entities that the NLU engine gave it</li></ol><p>Let’s first change the main, topmost question from<strong><em> “Age?”</em></strong> into <strong><em>“What would you like to do?”</em></strong>. Also, let’s delete the first (that is, left-most) child and its nodes, as we are not going to use them anymore:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Xw4uVJVuuCFfbzfSM_GUvQ.gif" /><figcaption>Type in the initial assistant question</figcaption></figure><p>Why are we seeing the red dashes around the node? Hover over it and you’ll see the error:</p><p><em>Count of contexts number should be equal to the number of children</em></p><p>We will fix that in a minute.</p><p>Now let’s build the transfer flow. We’ll assume that once the user says “I’d like to wire money”, we want to descend into the first, leftmost child. For that, we’ll select the “How can I help you” node and go into its properties. There, change the first context to have an <strong>intentId </strong>of <em>“TransferIntent”</em>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/961/1*HaP2Nr_DG1I7BFeobsvZjA.png" /></figure><p>This will cause any sentence that the Wit determines to have a TransferIntent, to be routed there.</p><h3>Mapping an entity</h3><p>Now, once the NLU has recognized our intent to transfer money, we should get all the different “fields”, or entities. Let’s add a node for the <strong><em>amount</em></strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LPItCiljpS7-JtIgmLQlxg.gif" /></figure><p>We added an AskAndMap node, and set its prompt to a question about the amount. We also changed its title — it’s always a good practice. Last, don’t forget to save your work using the Save button or Ctrl-S.</p><p>You can also notice the red warning disappeared from the <em>How can I help you</em> node.</p><p>Last, let’s add a <strong><em>number </em></strong><em>entity</em> to one of the child contexts of the Amount node, and map the value into a field called <strong><em>amount</em></strong><em>.</em></p><pre>“contexts”: [<br> {<br>   “entities”: [<br>    {<br>      “contextFieldName”: “amount”,<br>      “entityName”: “number”,<br>      “expectedValue”: “”,<br>      “entityIndex”: 0<br>    } <br>   ]<br> }</pre><p>All this seems very simple, and it is: if a user says something like “I need to send some money”, they will be asked, “What is the amount?”. Once they enter the amount, the <strong><em>number</em> </strong>will be extracted by the NLU and mapped to the <strong><em>context.amount </em></strong>in Servo. Then, we can use it later in the game. Visually, the flow started from the root:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/596/1*OSan4OmHEnCdgcK9yH3OXQ.png" /></figure><p>And the assistant would ask:</p><p><strong><em>“How can I help you?”</em></strong></p><p>If the user answered:</p><p><strong><em>“I’d like to transfer some money”</em></strong></p><p>the NLU engine would output a <strong><em>TransferIntent </em></strong>and the flow would continue downstream to the context it identified — the leftmost child — and ask the next question, about the amount:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/816/1*qvxW7H__EpfNpV_cGDdxNQ.png" /></figure><p>But what if the user doesn’t enter an amount?</p><h3>Building Helpers</h3><p>AskAndMap nodes support another type of a context child, called a <strong><em>Helper</em></strong>. This context is selected when the user answered something that couldn’t be mapped to <em>any other context</em>. Let’s add one into our What’s the amount AskAndMap:</p><pre><strong>&quot;contexts&quot;</strong>:[  <br>   {  <br>      <strong>&quot;entities&quot;</strong>:[  <br>         {  <br>            <strong>&quot;contextFieldName&quot;</strong>:&quot;amount&quot;,<br>            <strong>&quot;entityName&quot;</strong>:&quot;number&quot;,<br>            <strong>&quot;expectedValue&quot;</strong>:&quot;&quot;,<br>            <strong>&quot;entityIndex&quot;</strong>:0<br>         }<br>      ]<br>   },<br>   {  <br>      <strong>&quot;helper&quot;</strong>:true<br>   }<br>]</pre><p>Let’s now add a right-most child with a message help. Something like:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OUajGIZitit0CB4kFK6ktg.gif" /></figure><p>Of course, there can be only one <em>helper </em>context child for the AskAndMap.</p><p>One could imagine an example of the flow:</p><p>User:<strong><em> “I’d like to transfer some money”</em></strong></p><p>Assistant: “<strong><em>What is the amount?</em></strong>”</p><p>User: “<strong><em>You think I’d know. But I’m not sure</em></strong>”</p><p>Assistant: “<strong><em>Please provide the amount to transfer</em></strong>”</p><p>That looks simple: obviously, the assistant didn’t understand the “<strong><em>You think I’d know. But I’m not sure” </em></strong>and went on with the helper message “<strong><em>Please provide the amount to transfer</em></strong>”.</p><p>But in fact, if you’ll run the bot, you will get a surprising sentence after that last line:</p><p>User: “<strong><em>You think I’d know. But I’m not sure</em></strong>”</p><p>Assistant: “<strong><em>Please provide the amount to transfer</em></strong>”</p><p>Assistant: <strong><em>“How can I help you?”</em></strong></p><p>What happened here? Where did the <strong><em>“How can I help you?” </em></strong>come from?</p><p>Here’s the flow. The helper node said its line and returned SUCCESS to its parent, the AskAndMap. This, in turn, returned SUCCESS too, and so on, until the root was reached. At which point, the whole tree was restarted, and we get the initial <strong><em>How can I help you?” </em></strong>question.</p><p>So, to avoid that, we need to put a loop before the AskAndMap, so that it won’t return until it <em>really </em>succeeded. That is done with something called a <em>decorator.</em></p><h3>Adding a repeat decorator</h3><p>Behavior Trees implement loops using <em>decorators</em>, which are nodes that have one parent and one child. Depicted as a rhombus ⧫, we will use here the RepeatUntillSuccess decorator to loop the AskAndMap until it is successfully completed. Receiving a help message would not complete it, so we need to return a FAILURE after the help message. We do that by sequencing a Failer node right after the message. All in all, that’s the decoration we add to the AskAndMap construct:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Pmw1VRmaKcmn3vSjeKhXVQ.gif" /></figure><p>Now it’s the time to add the next node that would map the beneficiary account number. Again, pretty straight-forward: as before, we add an an AskAndMap with the question for the account number and a map from accountNumberEntity to an accountNumber member on the context. We set it as a child of a RepeatUntilSuccess decorator, and a helper child that explains what’s needed for this entity.</p><p>Then, we should add the actual business logic to do the transfer. This would probably mean several API calls with the entities collected. We would simulate this with a message: <strong><em>we are going to transfer $X to account #Y</em></strong>. For that, you need to drag in a GeneralMessage as the first child of the accountNumberEntity, and make its properties as follows:</p><pre><strong>“debug-log”</strong>:””,<br><strong> “runningTimeoutSec”</strong>:600,<br><strong> “maxRetriesNumber”</strong>:5,<br><strong> “replayActionOnReturnFromContextSwitch”</strong>:true,<br><strong> “view”</strong>:false,<br><strong> “prompt”</strong>:[ <br> “About to transfer &lt;%=context.amount%&gt; to account &lt;%=context.accountNumber%&gt;”<br>],<br>…</pre><p>This is how the tree looks like now:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZDeBLCRs_tIatVEQbnrkzw.png" /></figure><p>The tree comes with Servo. It’s files are under <em>server/convocode/anonymous/drafts/bank-bot.</em></p><h3>Running and testing</h3><p>Let’s test the bot and see what happens with various inputs. Click on the Debugger tab, then the play button ▶️. On the right hand the simulator will pop:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SjRfL69E-5rEGZi1Q5AE5A.png" /></figure><p>You can enter a sentence like:</p><p><strong><em>I’d like to send money.</em></strong></p><p>That would be answered, as expected, with</p><p>“<strong><em>What is the amount?</em></strong>”</p><p>And you can put in the amount, and continue.</p><p>But what if we say</p><p><strong><em>I’d like to send $14141</em></strong>??</p><p>Test it, and you’ll see how the assistant nicely jumps over the amount question straight to the account number:</p><p>“<strong><em>What is the account number?</em></strong>”</p><p>Now, let’s make its life even harder:</p><p><strong><em>I’d like to send money to account #87654321</em></strong></p><p>Nicely enough, it asks only for the amount. Say you enter $3400, it would then skip the account number (since it knows it already) into the final confirmation sentence:</p><p><strong><em>About to transfer 3400 to account #87654321.</em></strong></p><p>How does it know to do all that magic?</p><h3>The Context Flow</h3><p>Servo comes equipped with a powerful context-recognition set of algorithms that helps it do all that. What happened here shows a bit of it. Let’s take the last example. After the assistant asked:</p><p><strong><em>“How can I help you?”</em></strong></p><p>And the user answered:</p><p><strong><em>I’d like to send money to account #87654321</em></strong></p><p>The NLU engine output a <strong><em>TransferIntent </em></strong>and the flow continued downstream to the next question, about the amount:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/816/1*qvxW7H__EpfNpV_cGDdxNQ.png" /></figure><p>But the NLU also returned an <strong>accountNumberEntity! </strong>So before descending, this entity is saved on the <em>‘How can I help you’ </em>context. And, every AskAndMap defines its own context.</p><p>That’s actually an important remark, so I’ll repeat it: <strong><em>every AskAndMap defines its own context.</em></strong></p><p>At any point in the flow, when an entity is mentioned, Servo searches back (read: upwards) in the conversation to find it. If it hasn’t, it would ask for it.</p><p>So after the amount is entered, once we continue to the account number node, Servo finds that the <strong><em>accountNumberEntity </em></strong>was already mentioned, and uses it.</p><p>By the way, a process of similar characteristics happens also when we get to the last confirming GeneralMessage node. Its prompt reads:</p><p><strong>About to transfer &lt;%=context.amount%&gt; to account &lt;%=context.accountNumber%&gt;</strong></p><p>To resolve that, Servo searches up the context tree to find the needed entities, or context members.</p><p>Does this remind you of something? Folks familiar with JavaScript prototypical inheritance would see that it basically uses the same design. In Servo, we implemented that since we need more control over the variables. But it’s always interesting to see how object-oriented concepts are actually applied to real life, natural conversations.</p><p>But what if the user asks something much more unrelated, like:</p><p><strong><em>“How much money do I have in my account?”</em></strong></p><p>Or more so</p><p><strong><em>“Who are you, for heaven’s sake??”</em></strong></p><p>To which to bot responds:</p><p><strong><em>“I’m an artificial intelligence assistant built by Servo Labs.”</em></strong></p><p>Whaaaaaaaaaat?? Where did this come from?</p><h3>Context and sub-trees</h3><p>Almost all of the structural designs architects use to build manageable large systems can be divided into one of two categories:</p><ul><li>Reuse</li><li>Modularize</li></ul><p>If Servo is to stand as the infrastructure of large AI systems, it must provide some mechanism for allowing developers to achieve these goals. And that’s where sub-trees come into play.</p><p>We mentioned before the green hexagons:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/251/1*dpWneTdbrfa1l7rOtFKkLA.png" /></figure><p>This is a sub-tree. <strong>Double-click it</strong>, and you’ll enter a new tree, with that name. To create a new sub-tree, hover over the <strong>Trees </strong>on the left pane and select <strong>New:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/250/1*QiGAMSC6mgWi24fo1T7YPw.png" /></figure><p>A tree with a unique GUID name would appear. Change its name to something meaningful, and build it using any node from the left pane. Once built, you can drag, drop and connect it at any point in any other tree (including itself, by the way, but be very careful about that). Since sub-trees can have many leaf nodes, you can connect them only as leaves, too.</p><p>What happened when the user asked the assistant “Who are you”?</p><p>First, the NLU, already trained for such questions, returned a <strong><em>WhoAreYouIntent</em></strong>. Then context search was activated. If the conversation was somewhere down in the middle of a transfer conversation, the search went upwards, trying to find a context with WhoAreYouIntent. This context is found: it sits on the 4th context, in the <em>How can I help you</em> node. The flow then was redirected there, meaning, that route was made the active route. The flow here continued downstream into the chit-chat subtree, answered the question, returned up with a SUCCESS and the routing was returned to its previous context, the transfer one.</p><p>Here we learned something actually very important. <strong><em>The conversation flows down, but context is searched up. </em></strong>Never forget that:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/779/1*rgPNfRNfCJIDAl4Kp30aNw.png" /><figcaption><strong><em>The conversation flows down, but context is searched up</em></strong></figcaption></figure><h3>Connecting to a messenger client</h3><p>Until now, we have used the internal simulator and debugger as our messaging client. Let’s connect our small assistant to a real Facebook messenger. There is one big important change on the root properties of our tree, and that is to change the channel name from the default channel <em>“chatsim”</em> to <em>“facebook”</em>:</p><pre>&quot;channels&quot;:&quot;facebook&quot;</pre><p>On the Facebook side, these are the main high-level steps that one needs to take:</p><ol><li>Open a Facebook page under your Facebook account</li><li>Create a new Facebook app in the <a href="https://developers.facebook.com/">Facebook developer center</a></li><li>Add a messenger functionality to your app</li><li>Subscribe the app to listen to events in the page</li><li>Set the assistant callback address as the webhook to post to. Servo always publish its bot with the format of</li></ol><p>&lt;URI&gt;/entry/&lt;channel id&gt;/&lt;assistant name&gt;</p><p>So for a bank-bot assistant, running on <a href="http://www.mydomain.com,/">www.mydomain.com,</a> the address would be:</p><p><a href="https://www.mydomain.com/entry/fb/">https://&lt;www.mydomain.com&gt;/entry/fb/</a>bank-bot</p><p>You should set it in the page subscription section of the Facebook app, at the developer’s portal. You need to select at least <em>messages</em>, <em>messaging_postbacks</em>, and to match the <em>verify token</em> with the validation token you set in the bot’s root properties:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/687/1*TMhPYx27ARuqxeYlMb-Pgw.png" /></figure><p>By the way, <a href="https://serveo.net/">https://serveo.net</a> is a great tunneling system (another alternative is ngrok), if you are developing your assistants, like me, on localhost.</p><p>On the assistant root properties, set the same verify token and re-publish it:</p><pre>&quot;facebook&quot;: {<br>    &quot;validationToken&quot;: &quot;mytoken123&quot;,<br>    &quot;accessToken&quot;: &quot;&lt;token here&gt;&quot;<br>  },</pre><p>The access token should be set too, taken from Facebook’s messenger area:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/942/1*HZOFkLwElRrE_LDTY3YAOQ.png" /></figure><p>Last, select a page to subscribe your webhook to the page events:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/452/1*BOmEgeEluDA5jj5W-4kStA.png" /></figure><p>and… once you connect all these ends, you should have, at last, a full, orchestrated, end-to-end conversational AI system!</p><h3>Connecting Backends</h3><p>Real life connections vary, but luckily, most of them these days are done by using RESTFul API. For these, check out the <a href="https://servo-ai.github.io/servo-platform/">documentation </a>on <strong><em>RetrieveJSONAction </em></strong>and<strong> <em>PostAction</em></strong>. Once data is retrieved, or a response is received, it is set into a<a href="https://github.com/servo-ai/servo-platform/wiki/Building-context-aware,-stateful-chatbots-using-Servo#hierarchical-memory"> memory field (context/global/volatile)</a>. You probably would want to query it. This is done using <strong><em>ArrayQueryAction, </em>which implements an in-memory Mongo-like query language<em>. </em></strong>For direct MongoDB queries, use the <strong><em>MongoQuery </em></strong>action.</p><h3>In Summary</h3><p>Servo is an open-source IDE and framework that uses a context-recognition search to place the user on the right conversation and output the right questions. We learned how to construct a simple conversation, and how to wrap such conversations in sub-trees for decoupling and re-use. Servo has many other features that are worth exploring, among which you could find</p><ul><li>Connectors to Facebook, Alexa, Twilio and Angular</li><li>Connectors to MongoDB, Couchbase and LokiJS databases</li><li>Harness for automated conversation testing</li><li>A Conversation debugger</li><li>More actions, conditions and decorators</li><li>Flow control mechanisms</li><li>Field assignment and compare</li><li>Context manipulation</li><li>Validation</li><li>In-memory mongo-like queries</li><li>And any customized action you come up with</li></ul><p>Feel free to check it out and ask questions on the Github forum or at my @lmessinger Github name. Enjoy!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=658a7122e794" width="1" height="1" alt=""><hr><p><a href="https://medium.com/free-code-camp/how-to-build-an-end-to-end-conversational-ai-system-using-behavior-trees-658a7122e794">How to Build an End-to-End Conversational AI System using Behavior Trees</a> was originally published in <a href="https://medium.com/free-code-camp">We’ve moved to freeCodeCamp.org/news</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A fix to one bug that Twilio doesn’t want you to know about]]></title>
            <link>https://medium.com/hackernoon/a-fix-to-one-bug-that-twilio-doesnt-want-you-to-know-about-bfcf7472c558?source=rss-54cfc3017bca------2</link>
            <guid isPermaLink="false">https://medium.com/p/bfcf7472c558</guid>
            <category><![CDATA[chatbot-platforms]]></category>
            <category><![CDATA[twilio]]></category>
            <category><![CDATA[chatbot-design]]></category>
            <category><![CDATA[chatbots]]></category>
            <category><![CDATA[messaging]]></category>
            <dc:creator><![CDATA[Lior Messinger]]></dc:creator>
            <pubDate>Sat, 09 Feb 2019 22:15:18 GMT</pubDate>
            <atom:updated>2019-02-09T22:17:20.757Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/638/1*NJ0-Oa-bu3oPp-DJ0SN03Q.jpeg" /></figure><p>Short Message Service is one of the best channels to get a message across. Everyone with a smartphone can send and receive messages, regardless of the instant messenger they use. It can contain images and links, and you can send it as a push notification without much hassle.</p><p>Several companies offer easy to handle API to allow programmatic SMS handling. Some are less known and localized to their own country, but clearly the international leader is Twilio, which offers developers many ways to integrate messaging, voice, and even emails — starting from RESTful API and ending in Autopilot visual flow editor. It seems that a developer can easily build a chat bot, for example, that communicates over text messages.</p><p>Or can she? Well, not many developers know, but <em>Twilio does not guarantee the order of its messages</em>! This means, that if you’d like to send your customers a reminder that is split across several sentences, you might be heading for trouble. For example, what would be more natural than sending the following 4 messages:</p><pre>Hi!<br>It seems you haven&#39;t activated your account. <br>The faster you activate it, the more you save!<br>Would you like me to help you activate it?</pre><p>But as it turns out, these could appear on the customer’s phone as:</p><pre>The faster you activate it, the more you save!<br>Would you like me to help you activate it?<br>It seems you haven&#39;t activated your account.<br>Hi!</pre><p>How embarrassing is that?</p><p>The problem is, that due to carriers ancient infrastructure, one cannot guarantee the order upon arrival, as <a href="https://support.twilio.com/hc/en-us/articles/223134247-Can-my-SMS-messages-arrive-in-order-"><em>Twilio itself admits</em></a>.</p><p>Fortunately, there are some ways around it. We came across the problem when developing a channel driver for Twilio in <a href="http://servolabs.org/">Servo’s conversational framework</a>. Unfortunately, this discovery was made just one hour before an important demo to a client, so we couldn’t fix it right away and had to <a href="https://image.slidesharecdn.com/meteorworkshopucll-150723074331-lva1-app6892/95/craftcamp-for-students-introduction-to-meteorjs-15-638.jpg?cb=1438860570">pray to the demo gods</a>. But right after the meeting we started hacking away on the keyboard, trying a few options.</p><p>Most natural method that came to mind was to allow some time between the messages. It has the drawback that it would slow the conversation, but the hope was that 500ms delay wouldn’t be too bad. However, this didn’t work well enough.</p><p>The next evolution step was simple enough, and that one worked. We simply waited until Twilio drivers acknowledged that the message was received — by sending back the response — and then wait 500ms, or what was left of it. As always, it’s better to use decoupled queues to implement it. I’ll give you the essence here, and if you need the rest of the details, check the code itself in <a href="https://github.com/servo-ai/servo-platform">Github</a> — under the server/chat folder.</p><p>When a message arrive the channel driver for sending, we put it into a <strong><em>messages</em></strong> queue.</p><pre>...<br>   _clients[process.fsm_id].messages.push(msg);</pre><pre>...</pre><p>And once for every conversation, we call a recurrent function that would extract this message from the queue:</p><pre>if (!_clients[process.fsm_id].messages.length) {</pre><pre>this.send1Message(msg, process);</pre><pre>} </pre><p>which would call</p><pre>function <strong>send1Message</strong>(msg, process) {</pre><pre> twilio.sendMessage(msg, (err, responseData) =&gt; { </pre><pre>   if (!err) { </pre><pre>         setTimeout(() =&gt; {</pre><pre>            msg = _clients[process.fsm_id].messages.pop();</pre><pre>            this.<strong>send1Message</strong>(msg, process);// &lt;== recursive loop</pre><pre>          }, this.timeoutMs);</pre><pre>   } else {</pre><pre>      console.error(&#39;twilio message err&#39;, err.message + &#39; &#39; +   err.moreInfo, msg);</pre><pre>   }</pre><pre> });</pre><pre>}</pre><p>that’s it! it worked.</p><p>And what about that demo? Well, after all, the demo did go well. Unexpectedly, and against all our usual bad luck, demo gods was on our side that day.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bfcf7472c558" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hackernoon/a-fix-to-one-bug-that-twilio-doesnt-want-you-to-know-about-bfcf7472c558">A fix to one bug that Twilio doesn’t want you to know about</a> was originally published in <a href="https://medium.com/hackernoon">HackerNoon.com</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building context aware, stateful bots using Servo]]></title>
            <link>https://medium.datadriveninvestor.com/building-context-aware-stateful-bots-using-servo-a2dc3f557469?source=rss-54cfc3017bca------2</link>
            <guid isPermaLink="false">https://medium.com/p/a2dc3f557469</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[nlu]]></category>
            <category><![CDATA[bots]]></category>
            <dc:creator><![CDATA[Lior Messinger]]></dc:creator>
            <pubDate>Tue, 22 Jan 2019 12:13:07 GMT</pubDate>
            <atom:updated>2019-01-28T13:50:46.530Z</atom:updated>
            <content:encoded><![CDATA[<figure><a href="http://www.track.datadriveninvestor.com/1B9E"><img alt="" src="https://cdn-images-1.medium.com/max/700/0*re5ObHSnsz3pzLkS" /></a></figure><h3>Building context-aware bots using Servo</h3><p>Us developers are super proficient at developing applications: mobile, desktop or web applications. These apps usually follow the same flow: user fills a form (or clicks a button), the application calculates new data to show, and an updated screen is shown. With the years, architectures and design patterns have evolved to support this type of application development.. To name a few, one can mention dependency injection, micro-services, aspect-based-programming, MVC (Model View Controller), and others.</p><p>However, these do not work well for chatbots or voice assistants, which don’t have one form to fill or specific buttons to click. The user can say anything, sometimes unrelated to the current question. A sales bot might ask the user for the size of the shoes she was looking for, and the user would respond by asking the bot if they have boots instead. A car voice assistant might be in the middle of a conversation about finding a restaurant nearby, while the user tells it to call his friend that lives in the area. The friend is not picking up, and the bot is expected to return to the same point it left off.</p><p>Of course, one could confine the conversation, but that would obviously hurt the user experience. Or, one could try to provide ad-hoc logical solutions — but that might result in complex, large and hard-to-maintain code base making it hard to maintain.</p><p>At its heart, this is a state management problem. Servo is one of the few frameworks that can solve that. It’s doing so by utilizing Behavior Trees, which is a programming paradigm developed in a software industry that has been developing bots for years now — the gaming industry.</p><p>In this tutorial, we will explain how to start a simple bot with Servo. I assume here that you are a developer familiar with Github, NodeJS, and importantly, you know how to use NLU and NLP engines (with intent, entities etc). If you don’t, there are excellent resources all over the Internet — just search for ‘Wit tutorial’ or ‘LUIS tutorial’.</p><h3>Getting Started</h3><p>Servo is an open source framework and as such, you can fork it at <a href="https://github.com/servo-ai/servo-platform.git">Github</a> and follow the readme to install and <em>npm start</em>. It should run the behavior tree editor and the Servo server on your local machine, each on its own process. Then, open localhost:8000 in your browser and you’ll get a sign-in screen. Passed that, select Projects, And open your very own <strong>New Project:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wXJFdoNn3MzRDr3R7Vqd5Q.png" /><figcaption>New Project initial template</figcaption></figure><p>This tree represents a bot that deals with a nice set of issues and can serve as a simple tutorial for the framework. Click on Debugger and then on the Run ▶️ button, and you will be asked for your age. Let’s enter a few numbers and see what happens:</p><ol><li>If an old age is put in (say, 55), the bot will respond by quoting your age, and sending you to vote</li><li>If a number smaller than 18 is entered, the response would be that you are too young</li><li>At 32, the bot will give you a geeky remark about your age</li></ol><p>But: what if we enter something completely different?</p><ol><li>If you go with something like “who are you?”, you’ll get an answer. Then, you’ll be directed back to the age question</li><li>For responses that are not understood, the bot will give you a help message before asking the question again</li></ol><h3>How does it work?</h3><p>Let’s start by looking on the central node “Age?”. Select it and click on the Properties tab. You’ll see that the Type of the node is AskAndMap, it has a unique GUID and a title. Click on the properties📄 icon and inside the JSON shown you’ll see a few interesting items:</p><p>First, the prompt member:</p><pre><strong>“prompt”: [<br>“What’s your age?”,<br>“How old are you?”<br>]</strong></pre><p>Holds the questions the bot asks the user. If <strong>cyclePrompts </strong>member is <em>true</em> it will cycle through them, otherwise it will reach the end and keep with the last one.</p><p>Second, the <strong>contexts </strong>array is used for selecting the right child. Once a user responds to the prompt, the response is sent to an NLU engine (Servo comes with a default model, configured in the root properties). The NLU engine extracts intents and entities from the sentence. These are then matched against the contexts, and the best match is selected.</p><p>So, if the user responded with a number (eg 22, fifteen etc.), the first context is going to be selected, because it is a number entity which is expected here:</p><pre><strong>“contexts”: [<br>{<br> “entities”: [<br> {<br> “contextFieldName”: “age”,<br> “entityName”: “number”,<br> “entityIndex”: 0<br> }<br>]<br>}</strong></pre><p>Then, the flow continues downwards for the rest of the conversation. We’ll delve into that in a minute.</p><p>If the user responded with something which the NLU didn’t understand, the third context is selected:</p><pre>“helper”: true,<br>“default”: true</pre><p>There are some minute differences between <strong>default</strong> and <strong>helper</strong>, but on that, some other time.</p><p>If, on the other hand, the user says something that the NLU recognizes, but it’s contextually different (eg “who are you?”), the bot then selects a context to continue on, based on the <strong>intentId</strong>. As one can see, some of the contexts are selected with one of several <strong>intentId</strong>’s.</p><h3>Behavior Trees</h3><p>Servo didn’t invent too much proprietary programming methodologies, but rather chose to rely on industry standards as much as possible. One of the most successful paradigms, especially in gaming AI, has been Behavior Trees (BTs). Most gaming engines, such as Unity or Unreal, come with a BT editor, and they are very useful to construct rule-based behaviors.Servo is built on top of the super-well-crafted Behavior3 editor, written in Javascript by Renato de Pontes Pereira.</p><p>A word about AI is in place here. While deep learning receives a lot of hype (and rightfully so) it seems that for many real world applications rules are still needed, at least as an orchestration framework. While AI does wonders at classifying big data streams, the outcome needs to evoke some action, and that is best dealt with by rule-based logic, that connect these classifiers to input/output channels. In that sense, Servo combines the best of both of these paradigms.</p><p>One could read more about <a href="https://en.wikipedia.org/wiki/Behavior_tree_(artificial_intelligence,_robotics_and_control)">Behavior Trees</a>, but here I’m going to teach you quickly just the important stuff. Let’s a look on the left hand side of the tree, which is reached through the leftmost child upon entry of an age:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/425/1*H8Q13tKOBZgF4QFOxRXGJw.png" /><figcaption>Priority, Sequence, conditions and action nodes</figcaption></figure><p>Behavior Trees has a main loop, executing the current node a few times a second. A node execution could return one of three results: <strong><em>Success</em></strong>, <strong><em>Failure </em></strong>or Currently <strong><em>Running</em></strong>. If a node has any children, it executes its children and then, based on the execution result, returns its own execution result.</p><p>A node that has children is called a Composite node, and these have only two main types. Let’s follow the execution path here and understand them.</p><p>The ? node is called a <strong>Priority</strong> node, and acts like an OR selector. It tries to execute its children from left to right. If one of the children returns <em>Success</em>, it stops trying and returns a success, too. That’s why it’s called “priority”, because this gives a priority to the left-most children.</p><p>If no child succeeded, then the <strong>Priority</strong> doesn’t succeed either, and returns a <em>Failure</em>.</p><p>Here, the execution then continues downwards to the → node, called a Sequence. The <strong>Sequence</strong> is like an AND: it executes its children from left to right, expecting all of them to succeed. If one of the children fails, the whole <strong>Sequence</strong> fails, and returns <em>Failure</em>.</p><p>So, the execution continues on down, to the <em>age</em> &gt;= 18 node. This is a <strong>Condition</strong>, that compares the <em>age</em> to <em>18</em> (we’ll talk in a minute on how this comparison is made).</p><p>If the <strong>Condition </strong>succeeds, the execution continues to the <em>‘time to vote’</em> node. This is an Action, and as the name implies, it’s where all the action happens. Select it, and you’ll see that it’s a <strong>GeneralMessage </strong>action, that outputs a “you can vote” sentence to the user. We then continue to the green <em>‘good-bye’</em> hexagon. This is a sub-tree! Double-click it and you’ll go into it.</p><p>What if the age is less than 18? You might want to take a look above before continuing reading and work it out by yourself.</p><p>If the <em>age</em> &lt; 18, the <strong>Condition </strong>fails, causing the → <strong>Sequence </strong>to fail, and the <strong>Priority </strong>then goes to the next child: <em>too young.</em></p><p>Easy enough, isn’t it?</p><h3>Hierarchical Memory</h3><p>Now let’s look into the details of <strong>Conditions</strong> and <strong>Actions</strong>, and talk about memories. Select the <em>age&gt;=18</em> condition and open its properties:</p><pre>“left”: “context.age”,<br>“operator”: “&gt;”,<br>“right”: “18”</pre><p>What happens here? Actually you can read a short help section in the Description field of the node. It reads <em>”Compare fields across global,context, volatile and message memories. left and right operands should have a dot notation with the object name. Eg: message.chat_message, context.amount etc. Operator could be any logical operator like ===, &lt;, &lt;==, !==, ==&gt; etc. “</em></p><p>Indeed, this is a simple <a href="https://docs.microsoft.com/en-us/cpp/cpp/relational-operators-equal-and-equal?view=vs-2017">relational operator</a>, returning <em>true</em> or <em>false</em> for <em>Success</em> or <em>Failure</em>. So far so good. But where did the <em>context.age</em> expression come from?</p><p>Well, remember the <strong>contexts </strong>array in the “<em>age?”</em> node? That’s where it is coming from. Turns out that once a context is selected, all the entities and intents are “mapped” into the fields the context defined. We had, for that context:</p><pre>“contextFieldName”: “age”,<br>“entityName”: “number”,<br>“entityIndex”: 0</pre><p>This defines what happened: the NLU recognized a ‘number’ entity. The system then created an <em>age </em>field in the <em>context.</em> Once there, <strong><em>that field is available to all of the context descendants</em></strong>. This is really important, not in and by itself, but because of a question it brings in: what if there’s another context. In other words, what if another question is to follow the age question?</p><p>Luckily the way it works is known to anyone who knows anything about object-oriented programming, and especially JavaScript inheritance. If another question follows a parent question, then a new, child context will be created. If nodes then refer to some <em>context.fiel</em>d it is searched upwards, until it finds a field that matches the name or until it reaches the root of the tree.</p><p>This is pretty powerful, because once the bot understood the age of your user, and even if it continued to talk about new topics, you could still refer to <em>context.age</em> and the framework will fetch the most recent talked-about age. By the way, why the fancy name “Hierarchical Memory”? Well, this is probably <a href="https://en.wikipedia.org/wiki/Hierarchical_temporal_memory">how our brain identifies entities</a>.</p><p>Other types of memory include:</p><ul><li><strong>Global</strong>: whole conversation global memory</li><li><strong>Message</strong>: the latest message arrived from the user</li><li><strong>Volatile</strong>: memory that is never serialized into the database. This is good for in-memory complex objects.</li><li><strong>Local</strong>: per-node memory</li><li>And also, an undocumented Fsm memory, where one can access properties of the conversation process, as defined at the root of the main behavior tree.</li></ul><p>With these comes also an important <strong>Action </strong>type, called <strong>SetFieldAction</strong>. If you need to set a field at one of those memory areas, that’s the place.</p><h3>Delivering the Message</h3><p>The last piece still missing in the flow is the message that goes back to the user. How does one goes to construct it? For that you could take a look at the <em>“time to vote!”</em> <strong>GeneralMessage </strong>properties:</p><pre>“prompt”: [<br>“Congrats! at &lt;%=context.age%&gt; you can vote”,<br>“At &lt;%=context.age%&gt; you are old and wise, you can vote!”<br>],</pre><p>I’m sure you’ve noticed the <strong>&lt;%= %&gt;</strong> notation. This is a well known technique for web development called <em>“templating”</em>. The &lt;%= tells the framework to evaluate the expression against an object containing all memory areas, so we could also use &lt;%=global.fieldName%&gt; and others. Interestingly, we are not limited to expressions, but could use code, too. For example, adding <em>&lt;% if (context.age&gt;=100) { %&gt; you are one of our eldest voters &lt;%}%&gt; or &lt;% if (context.age&gt;=100) print ‘you are one of our eldest voters’ %&gt;</em> would print that sentence for those with age&gt;=100.</p><p>Templating is available for many actions and node types. Look at the description to see if the node asks specifically for a ‘dot notation’ or “memory field”. If it doesn’t, then you can use templating instead.</p><h3>Debugging</h3><p>As you probably saw, Servo comes equipped with a built-in debugger. Assuming you are a developer, it’s pretty straight-forward. You can set breakpoints (leaf only at the time of writing), run, step, and view the different memory areas discussed above.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/519/1*W66hLVlG6DSL5bHwH2UiSQ.png" /><figcaption>Debugger stops at a breakpoint</figcaption></figure><p>Two important remarks:</p><ul><li>The breakpoints are reached “post-tick”, which means, <em>after</em> the execution of the node</li><li>If you change things in the tree, just remember to publish! Although Servo gives you a warning, it’s easy to forget</li></ul><h3>Into the Deep</h3><p>Servo is a large framework that brings many more features that can accelerate development for bots, automation and flow-oriented systems. Among its other features:</p><ul><li>Alexa, Facebook, Web and other custom clients</li><li>Database interface</li><li>RESTful services</li><li>Context switching</li><li>Backtracking: change of mind</li><li>Automated dialog testing</li><li>Data-driven image rendering</li><li>Custom nodes and drivers</li><li>Sub-process and sub trees</li></ul><p>Some of these are covered in the documentation that comes with Servo and some, as they say, are there for the brave explorer. Enjoy!</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2Fb2a0d6%3Fas_embed%3Dtrue&amp;dntp=1&amp;display_name=Upscribe&amp;url=https%3A%2F%2Fupscri.be%2Fb2a0d6%2F&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/0707f5c806284d01a4a13c7b13a91ce3/href">https://medium.com/media/0707f5c806284d01a4a13c7b13a91ce3/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a2dc3f557469" width="1" height="1" alt=""><hr><p><a href="https://medium.datadriveninvestor.com/building-context-aware-stateful-bots-using-servo-a2dc3f557469">Building context aware, stateful bots using Servo</a> was originally published in <a href="https://medium.datadriveninvestor.com/">DataDrivenInvestor</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>