<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Mohammed Terry-Jack on Medium]]></title>
        <description><![CDATA[Stories by Mohammed Terry-Jack on Medium]]></description>
        <link>https://medium.com/@b.terryjack?source=rss-4e6913b61e2b------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*SPc3T4ysJenzhj9zVkXSnw.png</url>
            <title>Stories by Mohammed Terry-Jack on Medium</title>
            <link>https://medium.com/@b.terryjack?source=rss-4e6913b61e2b------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 14:39:56 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@b.terryjack/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Information Theory and Artificial Intelligence]]></title>
            <link>https://medium.com/wluper/information-theory-and-artificial-intelligence-aae553715fc4?source=rss-4e6913b61e2b------2</link>
            <guid isPermaLink="false">https://medium.com/p/aae553715fc4</guid>
            <category><![CDATA[kolmogorov]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[information-theory]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Mohammed Terry-Jack]]></dc:creator>
            <pubDate>Thu, 27 Jul 2023 13:25:49 GMT</pubDate>
            <atom:updated>2023-07-27T14:55:33.132Z</atom:updated>
            <content:encoded><![CDATA[<h4>Will they find their way back together?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*97rQItk7pFzAnijU1oI1sg.png" /></figure><p>Artificial Intelligence (AI) and Machine Learning (ML) differ from other fields as they tend to lack a coherent theoretical basis. In fact, they have often been compared with alchemy than science, with ML practitioners experimenting on combinations of mathematical and computational techniques by trial and error, tweaking and fine-tuning various hyperparameters (and/or prompts) until, by some touch of luck, they stumble upon that perfect recipe that magically works and should no more be touched!</p><p>Information Theory (IT), by contrast, has a strong theoretical and mathematically rigorous basis and it has become well-respected and oft-utilised in many scientific fields (such as physics, communication theory, dynamical systems, etc)</p><p>In brief, it is a theory which formalises the notion of “information” and defines what it could be, how it can be quantified, in which ways it may underpin and influence a system, etc. The field actually shares a common origin story with AI (both stemming from early work in Cybernetics). However, they both quickly branched off and parted ways, flourishing into their own respective fields which continue to grow to this day</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/650/1*HfILcRNsnQLHYSj3l8uaWg.jpeg" /></figure><p>Recently there has been a push in academia for research which combines Information Theory with Machine Learning.</p><p>For instance, the popular Elsevier journal <a href="https://www.sciencedirect.com/journal/physica-d-nonlinear-phenomena/vol/453/suppl/C">Physica D: Nonlinear Phenomena</a> released some special issues dedicated to work at the interface of Machine Learning and Algorithmic Information Theory (<a href="https://www.sciencedirect.com/journal/physica-d-nonlinear-phenomena/about/call-for-%20papers">ML and AIT</a>), of which we contributed a couple of papers (e.g. <a href="https://www.sciencedirect.com/science/article/pii/S0167278923001409">0–1 Test for Chaos…</a> and <a href="https://www.sciencedirect.com/science/article/pii/S0167278923001781">FT-bounded Kolmogorov Complexity</a>)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JL__HsO0yL_MS-qOrwdTbg.png" /></figure><p>In a recent paper entitled <a href="https://aclanthology.org/2023.findings-acl.426/">“Low-Resource” Text Classification: A Parameter-Free Classification Method with Compressors</a>, the authors combine an information- theoretic distance metric with a classical ML algorithm — the k-nearest-neighbours (KNN) classifier — to demonstrate that even this very simple classifier, when combined with the power of Information Theory, can outperform SOTA Deep Learning (DL) models! In the authors’ own words, their approach is:</p><blockquote>“a non-parametric alternative to DNNs that’s easy, lightweight, and universal in text classification” with “results that are competitive with non-pretrained deep learning methods on six in-distribution datasets. It even outperforms BERT” and their “method also excels in the few- shot setting, where labeled data are too scarce to train DNNs effectively”.</blockquote><p>To summarise their approach, they combine a KNN classifier with a custom metric that measures the information distance between strings of text. To measure the information distance of texts they rely on a concept from <a href="https://en.wikipedia.org/wiki/Algorithmic_information_theory">AIT</a> called<a href="https://en.wikipedia.org/wiki/Kolmogorov_complexity"> Kolmogorov Complexity</a>.</p><p>In essence, the length of a string of text may be long and yet contain very little information (e.g. “aaaaaaaaaaaaaaaaaaaa”) but it will also contain a lot of redundancy (i.e. repetitions, patterns, etc) which can utilised to rewrite the string into a more concise and efficient form and so, the length of its summarised representation would end up being very short (e.g. “a”*20). However, if the original string does not contain any redundancy, then its summary form would remain long and it can be safely assumed that the string contains more information.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/749/1*BHnVXmmOzo4zaHP_7iJgSg.jpeg" /></figure><p>In short (pun-intended), the length of a string’s optimally summarised representation is an accurate indication to the amount of information it contains. <em>(For more detailed examples and definitions of Kolmogorov Complexity, please see </em><a href="https://www.scirp.org/journal/paperinformation.aspx?paperid=120701"><em>here</em></a><em> and </em><a href="https://www.sciencedirect.com/science/article/pii/S0167278923001781"><em>here</em></a><em>)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*I3mlhSfUdbY4nhdG2E7dZA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OtqSm1EGF0qibXp17_FGiQ.jpeg" /><figcaption>A visualisation to show the relative space taken by strings of text both before (left) and after (right) a compressor like gzip has been applied.</figcaption></figure><p>Now the Kolmogorov Complexity (K) is technically incomputable, however, it can be approximated using any lossless compressor (the authors use gzip as their compressor of choice)</p><pre>from gzip import compress<br><br>K = lambda text: len(compress(text.encode(&#39;utf-8&#39;)))</pre><pre> K(&quot;aaa&quot;)</pre><blockquote>&gt;&gt; 23</blockquote><pre>K(&quot;aaaaaaaa&quot;)</pre><blockquote>&gt;&gt; 23</blockquote><pre> K(&quot;aaaaaaaaaaaaaaaaaaaaaaaaaaaaa&quot;)</pre><blockquote>&gt;&gt; 23</blockquote><p>The <strong>information distance</strong> presented in the paper is defined as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/408/1*e3zHxjKdl9SnSBrtl61Bcg.jpeg" /></figure><p>which can be “<em>interpreted as how many bytes do we still need to encode y based on the information of x</em>”. In other words, if two texts (x and y) are identical (informationally speaking) then no additional bytes (of information) would be required to encode text y than the bytes already used to encode text x.</p><pre>information_distance = lambda x,y: K(x+y) - min(K(x), K(y))</pre><p>So if there is no difference (informationally) between two strings of text, their information distance from one another is 0</p><pre>information_distance(&quot;aaa&quot;,&quot;aaa&quot;)</pre><blockquote>&gt;&gt; 0</blockquote><pre>information_distance(&quot;aaa&quot;,&quot;aaaaaaaaaaaaaaaaaaaaa&quot;)</pre><blockquote>&gt;&gt; 0</blockquote><p>Similarly, the more different they are (informationally), the further their information distance grows.</p><pre> information_distance(&quot;aaa&quot;, &quot;abcdefg&quot;)</pre><blockquote>&gt;&gt; 7</blockquote><p>The information distance is not upper bounded, so the final information-theoretic metric presented in the paper was normalised (to be within 0 and 1) to give the Normalised Compression Distance (NCD)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/412/1*Aut-PfLDrSU-KgZm0WgZ2Q.png" /><figcaption>(where the notation C here refers to the compressor-based approximation of K — since the ideal K is an incomputable function)</figcaption></figure><pre> NCD = lambda x,y: information_distance(x,y)/max(K(x), K(y))</pre><pre> NCD(&quot;aaa&quot;, &quot;abcdefg&quot;)</pre><blockquote>&gt;&gt; 0.25925925925925924</blockquote><p>However, for most cases, using the information distance (without normalisation) works just as fine. Here is an example where we can use it to rank texts:</p><pre>rank = lambda target,candidates,top_n: sorted( <br>  candidates,<br>  key=lambda candidate:information_distance(<br>    x=target,<br>    y=candidate<br>  ) <br>)[:top_n]</pre><p>Lets use a list of food items for the text</p><pre>menu = [&quot;Arepa&quot;,&quot;Biscuits&quot;, &quot;Cookie&quot;, &quot;Cracker&quot;, &quot;Bread&quot;, &quot;Croissant&quot;, &quot;Burrito&quot;,<br>&quot;Cabbage roll&quot;, &quot;Cake&quot;, &quot;Pancake&quot;, &quot;Cheese&quot;, &quot;Donuts&quot;, &quot;Dumplings&quot;, &quot;Fun guo&quot;, &quot;Har gow&quot;,<br>&quot;Pierogi&quot;, &quot;Wonton&quot;, &quot;French fries&quot;, &quot;Poutine&quot;, &quot;Grains&quot;, &quot;Cereal&quot;, &quot;Popcorn&quot;, &quot;Rice&quot;,<br>&quot;Ice cream&quot;, &quot;Mashed potatoes&quot;, &quot;Meats&quot;, &quot;Beef&quot;, &quot;Steak&quot;, &quot;Eggs&quot;, &quot;Pork&quot;, &quot;Bacon&quot;, &quot;Ham&quot;,<br>&quot;Poultry&quot;, &quot;Buffalo wing&quot;, &quot;Chicken balls&quot;, &quot;Chicken nuggets&quot;,&quot;Chicken steak&quot;,&quot;Chicken strips&quot;,<br>&quot;Ribs&quot;,&quot;Seafood&quot;,&quot;Shrimp and prawn as food&quot;,&quot;Onion rings&quot;, &quot;Pasta&quot;,&quot;Lasagna&quot;,&quot;Linguini&quot;,&quot;Ravioli&quot;,<br>&quot;Spaghetti&quot;,&quot;Spaghetti and meatballs&quot;,&quot;Pupusa&quot;,&quot;Pizza&quot;,&quot;Rolls&quot;,&quot;Croquette&quot;,&quot;Egg roll&quot;,&quot;Spring roll&quot;,<br>&quot;Sandwiches&quot;,&quot;Cheeseburgers&quot;,&quot;Hamburgers&quot;,&quot;Hot dogs&quot;,&quot;Peanut butter and jam sandwich&quot;,&quot;Submarine sandwich&quot;,<br>&quot;Soup&quot;,&quot;Stew&quot;,&quot;Taco&quot;,&quot;Tamale&quot;,&quot;Turnover&quot;,&quot;Jamaican patty&quot;,&quot;Waffle&quot;]</pre><pre>rank(<br>  target=&quot;i would like to eat sushi&quot;, <br>  candidates=menu,<br>  top_n=1<br>)</pre><blockquote>&gt;&gt; [‘Shrimp and prawn as food’]</blockquote><pre>rank(<br>  target=&quot;i would like to eat a burger&quot;, <br>  candidates=menu,<br>  top_n=2<br>)</pre><blockquote>&gt;&gt; [‘Cheeseburgers’, ‘Hamburgers’]</blockquote><pre>rank(<br>  target=&quot;i would like to eat chicken&quot;, <br>  candidates=menu,<br>  top_n=4<br>)</pre><blockquote>&gt;&gt; [‘Chicken balls’, ‘Chicken nuggets’, ‘Chicken steak’, ‘Chicken strip<br>s’]</blockquote><p>Now lets use the information distance in combination with a KNN classifier.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xrtrZ-UFednnyh7xw-1W7Q.png" /></figure><p>The KNN classifier from Scikit learn is convenient to use, however, it requires vectors as inputs (not strings or bytes) and since our metric takes in strings (not vectors) we need to momentarily convert the strings into vectors and back to strings again (just to bypass the sklearn input format validation checks — or alternatively we must code up our own KNN classifier as the authors of the paper did)</p><pre>string2digits = lambda text,max_length:list(map( <br>  ord,<br>  text + &quot; &quot;*(max(0,max_length-len(text))) <br>))[:max_length]<br><br>digits2string = lambda digits:&#39;&#39;.join(map( <br>  chr,<br>  map(int,digits)<br>)).strip()</pre><pre>N = 1000<br>digits2string(string2digits(&quot;testing&quot;,N))</pre><blockquote>&gt;&gt; “testing”</blockquote><pre>from sklearn.neighbors import KNeighborsClassifier<br><br>classifier = KNeighborsClassifier( <br>  metric=lambda x,y:information_distance(<br>    x=digits2string(x),<br>    y=digits2string(y) <br>  )<br>)</pre><pre><br>train = lambda X,y: classifier.fit( <br>  X=list(map(<br>    lambda text:string2digits( <br>      text=text,<br>      max_length=N <br>    ),<br>    X<br>  )),<br>  y=y <br>)</pre><p>Now lets get the training data. We use the huggingface api to download <em>nlu_evaluation_data</em></p><pre>from requests import get<br><br>download_dataset = lambda : get( <br>    &quot;https://datasets-server.huggingface.co/rows?dataset=nlu_evaluation_data&amp;config=default&amp;split=train&amp;offset=0&amp;limit=100&quot;<br>).json()</pre><pre>data = download_dataset()<br>data[&#39;rows&#39;][:3]</pre><pre>[{&#39;row_idx&#39;: 0,<br>  &#39;row&#39;: {&#39;text&#39;: &#39;wake me up at five am this week&#39;,<br>   &#39;scenario&#39;: &#39;alarm&#39;,<br>   &#39;label&#39;: 2},<br>  &#39;truncated_cells&#39;: []},<br> {&#39;row_idx&#39;: 1,<br>  &#39;row&#39;: {&#39;text&#39;: &#39;wake me up at nine am on friday&#39;,<br>   &#39;scenario&#39;: &#39;alarm&#39;,<br>   &#39;label&#39;: 2},<br>  &#39;truncated_cells&#39;: []},<br> {&#39;row_idx&#39;: 2,<br>  &#39;row&#39;: {&#39;text&#39;: &#39;set an alarm for two hours from now&#39;,<br>   &#39;scenario&#39;: &#39;alarm&#39;,<br>   &#39;label&#39;: 2},<br>  &#39;truncated_cells&#39;: []}]</pre><pre>get_training_inputs = lambda dataset:list(map(<br>    lambda row:row[&#39;row&#39;][&#39;text&#39;],<br>    dataset[&#39;rows&#39;]<br>))<br><br>get_training_outputs = lambda dataset:list(map(<br>    lambda row:row[&#39;row&#39;][&#39;label&#39;],<br>    dataset[&#39;rows&#39;]<br>))</pre><pre>X=get_training_inputs(data)<br>y=get_training_outputs(data)</pre><p>Now fit the KNN classifier (super quick)</p><pre>train(X,y)</pre><blockquote>&gt;&gt; KNeighborsClassifier(metric=&lt;function &lt;lambda&gt; at 0x13c78e940&gt;)</blockquote><p>Et voila! Our classifier is now ready to be tested</p><pre>intents = [&quot;alarm_query&quot;, &quot;alarm_remove&quot;, &quot;alarm_set&quot;, &quot;audio_volume_down&quot;, &quot;audio_volume_mute&quot;, &quot;audio_volume_other&quot;, &quot;audio_volume_up&quot;, &quot;calendar_query&quot;, &quot;calendar_remove&quot;, &quot;calendar_set&quot;, &quot;cooking_query&quot;, &quot;cooking_recipe&quot;, &quot;datetime_convert&quot;, &quot;datetime_query&quot;, &quot;email_addcontact&quot;, &quot;email_query&quot;, &quot;email_querycontact&quot;, &quot;email_sendemail&quot;, &quot;general_affirm&quot;, &quot;general_commandstop&quot;, &quot;general_confirm&quot;, &quot;general_dontcare&quot;, &quot;general_explain&quot;, &quot;general_greet&quot;, &quot;general_joke&quot;, &quot;general_negate&quot;, &quot;general_praise&quot;, &quot;general_quirky&quot;, &quot;general_repeat&quot;, &quot;iot_cleaning&quot;, &quot;iot_coffee&quot;, &quot;iot_hue_lightchange&quot;, &quot;iot_hue_lightdim&quot;, &quot;iot_hue_lightoff&quot;, &quot;iot_hue_lighton&quot;, &quot;iot_hue_lightup&quot;, &quot;iot_wemo_off&quot;, &quot;iot_wemo_on&quot;, &quot;lists_createoradd&quot;, &quot;lists_query&quot;, &quot;lists_remove&quot;, &quot;music_dislikeness&quot;, &quot;music_likeness&quot;, &quot;music_query&quot;, &quot;music_settings&quot;, &quot;news_query&quot;, &quot;play_audiobook&quot;, &quot;play_game&quot;, &quot;play_music&quot;, &quot;play_podcasts&quot;, &quot;play_radio&quot;, &quot;qa_currency&quot;, &quot;qa_definition&quot;, &quot;qa_factoid&quot;, &quot;qa_maths&quot;, &quot;qa_stock&quot;, &quot;recommendation_events&quot;, &quot;recommendation_locations&quot;, &quot;recommendation_movies&quot;, &quot;social_post&quot;, &quot;social_query&quot;, &quot;takeaway_order&quot;, &quot;takeaway_query&quot;, &quot;transport_query&quot;, &quot;transport_taxi&quot;, &quot;transport_ticket&quot;, &quot;transport_traffic&quot;, &quot;weather_query&quot;]</pre><pre>classify = lambda text: intents[ <br>  classifier.predict(<br>    X=[ <br>      string2digits(<br>        text=text,<br>        max_length=N <br>      )<br>    ] <br>  )[0]<br>]</pre><pre>classify(&quot;can you dim the lights please&quot;)</pre><blockquote>&gt;&gt; ’iot_hue_lightdim’</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*2v89dqmebfzo3kh3.png" /></figure><pre>from random import randint<br><br>i = randint(0,len(X))<br>text = X[i]<br>expected = intents[y[i]]<br>predicted = classify(text)<br><br>print(f&#39;sample: {i}\n text: {text}\n expected: {expected}\n predicted: {predicted}&#39;)</pre><blockquote>&gt;&gt; sample: 77<br> text: could you order sushi for tonight dinner<br> expected: takeaway_order<br> predicted: takeaway_order</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/512/0*nOCtCPHVJyo2XlQs.jpeg" /></figure><p>In conclusion, the intersection of Information Theory and Machine Learning has shown tremendous potential in enhancing classical machine learning algorithms. While AI and ML have historically relied on experimental techniques, the infusion of information-theoretic principles brings a new level of mathematical rigour and efficiency to the field. Such a potent interdisciplinary mix will undoubtedly open up new avenues for research and innovation and contribute to the maturity and evolution of AI.</p><p>It’s an exciting time for the world of AI, where the magic of algorithms and the depth of information theory converge to create cutting-edge solutions that push the boundaries of what’s possible.</p><p>You liked this post? Please leave some claps and feel free to share.</p><p>Any comments or questions? Let us know what you think.</p><p>Follow us also on <a href="https://twitter.com/wluper_">Twitter </a>and <a href="https://www.linkedin.com/company/wluper_">LinkedIn</a> for more.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=aae553715fc4" width="1" height="1" alt=""><hr><p><a href="https://medium.com/wluper/information-theory-and-artificial-intelligence-aae553715fc4">Information Theory and Artificial Intelligence</a> was originally published in <a href="https://medium.com/wluper">Wluper Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[NLP With Biologically-Inspired Neural Networks]]></title>
            <link>https://medium.com/wluper/nlp-with-biologically-inspired-neural-networks-ac36b3170b90?source=rss-4e6913b61e2b------2</link>
            <guid isPermaLink="false">https://medium.com/p/ac36b3170b90</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[neuroscience]]></category>
            <dc:creator><![CDATA[Mohammed Terry-Jack]]></dc:creator>
            <pubDate>Tue, 21 Dec 2021 09:25:37 GMT</pubDate>
            <atom:updated>2021-12-22T10:17:04.685Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jqPZSKEEj5m3phJ5gWrczw.png" /></figure><p>Is it possible to reverse engineer the human brain, the <strong>Neocortex</strong>? <a href="https://medium.com/u/2f729572f85a">Numenta</a> has proposed a novel machine learning algorithm which is a bold attempt in doing exactly that.</p><p>This biologically-inspired neural network called <strong>Hierarchical Temporal Memory (HTM)</strong> has some major differences to the Artificial Neural Networks we have all become familiar with from Deep Learning.</p><p>For instance, HTM has:</p><ul><li>Mini-columns (like a capsule in a capsule network)</li><li>Binary-valued neurons!</li><li>Two types of weights (analogous to Feedforward weights and Self-attention weights)</li></ul><p>But aside from these differences in detail, HTM stands apart as an overall ML model because it performs:</p><ul><li>Simple Hebbian Learning (no Complex Backpropagation Calculations)</li><li>A novel style of Unsupervised Learning (not Supervised Learning)</li><li>Online Learning (i.e. Continuously Learning Live)</li><li>Few shot learning</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/267/1*DuC-_DFP1rXaGNVzFFC9IA.png" /></figure><p>In this post we shall attempt to demystify this powerful machine learning algorithm by relating its underlying components to analogous concepts in Deep learning. In our next post we will be using this biologically-inspired neural network to learn and solve a few NLP tasks.</p><h3>Neurons</h3><p>For an Artificial Neural Network (ANN), a <strong>layer</strong> simply contains <strong>neurons<em>.</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/519/1*Q9cYQijuoGaXDKhM42wE0w.png" /><figcaption>A layer of neurons in an ANN.</figcaption></figure><p>However, for Hierarchical Temporal Memory (HTM), a <strong>layer</strong> contains HTM <strong>capsules</strong> (<em>mini-columns</em>) — each of which contains HTM <strong>neurons</strong> (<em>pyramidal cells</em>).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/77/1*PLoHOdZyapvh-Hnzm8n6UQ.png" /><figcaption>Let there be <strong><em>n</em></strong> ‘pyramidal cells’ (HTM neurons) per ‘mini-column’ (HTM capsule) and let there be <strong><em>m</em></strong> mini-columns per ‘region’ (HTM layer).</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/0*fDJ511BP5FkEBI94" /><figcaption>Image 1: (left) a pyramidal cell [HTM neuron], (centre) a mini-column [HTM capsule], (right) a region [HTM layer].</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/334/1*sq3Hf1QLKO6vkJwIqYe1YQ.png" /><figcaption>Image 2: Cells [HTM neurons] and (mini-)columns [HTM capsules] in a network with three regions [HTM Layers].</figcaption></figure><p>Furthermore, while ANN neurons output continuous values, HTM neurons are simply binary (i.e. ‘firing’ or ‘non-firing’).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/99/1*CwXnnGC0ohetveHTiR3ggQ.png" /></figure><p>And since HTM neurons are binary-valued, inputs and outputs are encoded as a ‘sparse distributed representation’ SDR <strong>x</strong> (i.e. a sparse binary vector).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/163/1*7LKWHHuiW8mmmYJKN4q8Ng.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/504/1*_RkGLhN4Q7ANAxdgYKdiqQ.png" /><figcaption>SDRs encoding the words ‘cat’, ‘dog’ and ‘fish’.</figcaption></figure><h3>Weights</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/134/1*X-TaXUSjHGyjfDoHd5sBiA.png" /><figcaption>Two sets of connections (weights).</figcaption></figure><p>ANN neurons have <strong>weights</strong> (<em>weighted synapses</em>) that connect to other neurons, but HTM neurons have two different types of weighted connections:</p><ol><li>‘<em>Proximal Connections</em>’ between neurons within different layers <strong><em>Wp </em></strong>(analogous to interlayer, <strong>feedforward weights</strong>) and</li><li>‘<em>Distal Connections’</em> between neurons within the same layer <strong><em>Wd </em></strong>(i.e. intralayer / intra-column, lateral connections analogous to <strong>self-attention weights</strong>)</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/247/0*JL1m_rCwU9GGyO35" /><figcaption>Image 3: A region / layer depicting the two types of weights (lines) between cells / neurons (circles) arranged in mini-columns (rows). The proximal connections / feedforward weights (bottom) and distal connections / self-attention weights (top).</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/828/0*EynVKORsKYehvHsl" /><figcaption>Image 4: Two Layers shown with both types of weights labelled: Proximal Connections (~ Feedforward Weights) and Distal Connections (~ Self-Attention Weights).</figcaption></figure><h3>Layers</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ZtB1YWg7n6fvMha7" /><figcaption>Image 5: A biologically-inspired neural network with a depth of 2 regions (layers), each containing columns of cells (neurons) and both feedforward weights as well as intra-column lateral (distal) connections (self-attention weights).</figcaption></figure><p>A HTM layer <strong><em>L</em></strong> predicts two things (using each set of weights):</p><p>1) A feedforward prediction <strong><em>H </em></strong>for the next layer</p><p>2) A self-attention prediction <strong><em>A </em></strong>determined by the input’s context within a sequence (an initially context-free input is represented by all mini-columns firing at once — all 1s)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/263/1*W8vCDW7e4tkLaswGQR7YWA.png" /></figure><h3>Feedforward Predictions (Spatial Pooler)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/553/1*BidBVQe8joJPu7f4ZMLMdg.png" /><figcaption>Image 5: A network with two regions (layers) with the cell (neuron) and mini-column labelled. The arrows show the direction of the proximal connections (feedforward weights) similar to other feedforward neural networks.</figcaption></figure><p>The spatial pooler <strong><em>σ </em></strong>projects a layer’s firing neurons <strong><em>H</em></strong> into a representational space using feedforward weights <strong><em>Wp</em></strong> (similar to mainstream feedforward neural networks).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/224/1*IjsLNw9MQSXVp_PD7sgYvA.png" /></figure><p>Where a high-pass filter <strong><em>f</em></strong> binarises the weights, thus determining if they are ‘connected’ (i.e. above some predefined threshold <strong><em>θ </em></strong>) or not. (The high-pass binarisation filter <strong><em>f</em></strong> also acts as the non-linear activation function since binary-values are inherently non-linear).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/269/1*aUWjpYvPh_9sHd22322I0w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/230/0*f72qSBdsnSNW17Ld" /></figure><p>Weights are iteratively optimised using hebbian learning (unlike mainstream neural networks which use a more compute intensive optimisation known as backpropagation). Weights for all ‘firing’ neurons <strong><em>H</em></strong> get incremented by the learning rate <strong><em>δ </em></strong>while weights for ‘non-firing’ neurons <strong>H_bar</strong> get decremented.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/356/1*77Rx8YfK17xTdJ0Axe79Ew.png" /></figure><h3><strong>Attention (Temporal Memory)</strong></h3><p>The anticipated state <strong><em>At+1</em></strong> is predicted by a ‘temporal memory’ <strong><em>τ </em></strong>(similar to an attention mechanism) which uses the self-attention weights <strong><em>Wd</em></strong> to predict the columns which will most likely fire on the next time step in this particular context (i.e. the columns that have active distal connections to the currently firing neurons <strong><em>Ht</em></strong> ).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/510/1*GfKrC7Q3vG0R64Cm7eQt0w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/690/0*ZtBwae8T0FjcvnF6" /><figcaption>Image 7: The ‘connected’ (red lines) distal connections (self-attention weights) between firing neurons (red circles) in a region (layer).</figcaption></figure><p>Where the conditional gate <strong><em>g</em></strong> ‘bursts’ a mini-column that hasn’t got any connected neurons, so that all the neurons in that column act as if they are connected (if none, then all)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/336/1*ONeGqyNQ3rNrMv55_oYQfg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/689/0*CRnEbxBP-hS_t1RL" /><figcaption>Image 8: Mini-columns of neurons ‘bursting’ because they do not have any ‘connected’ weights (if none, then all).</figcaption></figure><p>Only the weights <strong><em>Wd</em></strong> which connect to the next active neurons <strong><em>S</em></strong> are optimised (using hebbian learning) and weights for all other neurons are left unchanged.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/408/1*IuV6Z02iZL2unVI98tVrEA.png" /></figure><p>Since a mini-column ‘fires’ if any of its neurons are connected, only one neuron per mini-column needs to learn a distal connection (self-attentive weight) because more than one would be redundant information. Therefore, only the mini-column’s most connected neuron (the one with the strongest weight already) is incremented by some learning rate <strong><em>δ </em></strong>and all other neurons in that mini-column are decremented. <em>(Furthermore, numenta mentions decrementing neurons which are connected to non-firing mini-columns </em><strong><em>H_bar</em></strong><em> , however this is one of the few details which have been omitted).</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/539/1*wplvgTOHcikChj6kp7-LOQ.png" /></figure><h3>Output Predictions</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/309/1*knHmmTuXhdglp8OcBcaimA.png" /><figcaption>Image 9: A region (layer) depicting cells (neurons) which are red for ‘firing’ (connected via feedforward weights / spatial pooler predictions S) and yellow for ‘predictive’ (connected via self-attention weights / temporal memory predictions A). The output H are those neurons which are both red and yellow (jointly predicted).</figcaption></figure><p>The spatial pooler <strong><em>σ </em></strong>predictions (i.e. feed-forward predictions) <strong><em>S</em></strong> are compared with the temporal memory <strong><em>τ </em></strong>predictions (i.e. anticipated predictions based on self-attention weights) <strong><em>A</em></strong> and all joint predictions make up the final contextualised predictions (i.e. hidden layer) <strong><em>H</em></strong> which serves as the input to the next layer. This means that predictions which align with what was anticipated for the given context are given preference by the algorithm</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/249/1*1-yNMte8uckHYi_KTavKRA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/777/0*cT4pE2DxlfnbEnHN" /><figcaption>Image 10: A region/layer updated over three time steps, with the second time step uncovered to reveal how the temporal memory <strong><em>τ </em></strong>updates interact with the spatial pooler <strong><em>σ </em></strong>updates.</figcaption></figure><p>A prediction<strong><em> H</em></strong> can be mapped to an output SDR <strong><em>x</em></strong> by returning the layer’s ‘firing’ mini-columns (i.e. mini-columns containing one or more firing neurons = 1).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/191/1*ABYk7Mc7vnHWWXunE4TvJQ.png" /></figure><p>Any comments or questions? Feel free to reach out and leave a clap.</p><p>Don’t forget to follow us on <a href="https://twitter.com/wluper_">Twitter </a>and <a href="https://www.linkedin.com/company/wluper_">LinkedIn</a> for updates.</p><p>Check out our latest <a href="https://wluper.com/careers">job openings</a>.</p><h3>Further Reading</h3><ul><li><a href="https://numenta.com/">https://numenta.com/</a></li><li><a href="https://numenta.com/neuroscience-research/research-publications/papers/">https://numenta.com/neuroscience-research/research-publications/papers/</a></li><li><a href="https://numenta.com/assets/pdf/spatial-pooling-algorithm/Spatial-Pooling-Algorithm-Details.pdf">https://numenta.com/assets/pdf/spatial-pooling-algorithm/Spatial-Pooling-Algorithm-Details.pdf</a></li><li><a href="https://numenta.com/assets/pdf/temporal-memory-algorithm/Temporal-Memory-Algorithm-Details.pdf">https://numenta.com/assets/pdf/temporal-memory-algorithm/Temporal-Memory-Algorithm-Details.pdf</a></li><li><a href="https://github.com/numenta">https://github.com/numenta</a></li><li><a href="https://arxiv.org/pdf/1601.06116.pdf">https://arxiv.org/pdf/1601.06116.pdf</a></li><li><a href="https://github.com/jmnatzaganian/mHTM">https://github.com/jmnatzaganian/mHTM</a></li></ul><h3>Further Watching</h3><ul><li><a href="https://www.youtube.com/watch?v=rHvjykCIrZM">https://www.youtube.com/watch?v=rHvjykCIrZM</a></li><li><a href="https://www.youtube.com/watch?v=UBzemKcUoOk">https://www.youtube.com/watch?v=UBzemKcUoOk</a></li><li><a href="https://www.youtube.com/watch?v=1OhY_u3NjdM">https://www.youtube.com/watch?v=1OhY_u3NjdM</a></li><li><a href="https://www.youtube.com/watch?v=u-BIsnN-xxM">https://www.youtube.com/watch?v=u-BIsnN-xxM</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ac36b3170b90" width="1" height="1" alt=""><hr><p><a href="https://medium.com/wluper/nlp-with-biologically-inspired-neural-networks-ac36b3170b90">NLP With Biologically-Inspired Neural Networks</a> was originally published in <a href="https://medium.com/wluper">Wluper Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deep-Learning 2.0? A Quicker, Cheaper Replacement to Backpropagation]]></title>
            <link>https://medium.com/wluper/deep-learning-2-0-a-quicker-cheaper-replacement-to-backpropagation-c2cbd152f9f1?source=rss-4e6913b61e2b------2</link>
            <guid isPermaLink="false">https://medium.com/p/c2cbd152f9f1</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[naturallanguageprocessing]]></category>
            <category><![CDATA[backpropagation]]></category>
            <category><![CDATA[nlp]]></category>
            <dc:creator><![CDATA[Mohammed Terry-Jack]]></dc:creator>
            <pubDate>Tue, 02 Mar 2021 14:45:21 GMT</pubDate>
            <atom:updated>2021-12-22T10:18:07.729Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RHb2cc2XOo5uGkHUKVy_wQ.png" /></figure><p>Neural Networks, especially the earlier, shallower ones (like those with a single hidden layer), have a range of learning algorithms, i.e. ways to train their weights. To name but a few:</p><ul><li><strong>Gradient Descent </strong>(adjusting the weights incrementally, learning step by learning step, <strong>down</strong> the <strong>gradient</strong>/slope of error to minimise/reduce that error);</li><li><strong>Evolutionary </strong>techniques (to search for the optimal weights);</li><li>or perhaps one-shot learning methods (e.g. pseudo-inverse matrix methods used in <strong>Extreme Learning </strong>Machines);</li><li>or even simple, biologically-inspired algorithms like <strong>Hebbian learning</strong> (“<em>neurons that fire together, wire together</em>”) still used by some modern variants of neural networks (e.g. Hierarchical Temporal Memory, which, unlike most artificial neural networks, aims to model the human brain).</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0KcR8uldcVm2GToJgsCb-g.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Pegi_c8FRm8hP2o9y3dLNA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/270/1*L9evQPLOwKYPa-XjDkxGug.jpeg" /></figure><p>However, why aren’t many of these methods used for <strong>Deep Learning</strong>? Mainly because it is not easy to extend them to neural networks with deeper layers since many of the supervised learning signals which these methods depend upon are given only in respect to the final layer, and not for any of the deeper layers. For example, if a network has learnt to distinguish cars from horses, the final layer (which outputs the prediction of “car” or “horse”) can be checked against the supervised learning signal (the labels of each training example). However, the deeper layers which may learn to predict certain car features (e.g. wheels, windows and metallic textures), or features of horses (e.g. tail and legs) cannot be checked against the supervised signal since the training examples are not explicitly labelled with such features.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LJKWS3p967-zmV05BvrAiw.png" /></figure><p><strong>Back-propagation </strong>was revolutionary as it provided a way for learning algorithms, like Gradient Descent, to be extended to neural networks with multiple layers. A new age of Deep Learning was born.</p><p>By comparing network’s prediction to the supervision signal (i.e. the label “car”), we get a difference (the network’s error) which is also a gradient (slope) and thus the direction which the final weights can be adjusted to reduce that error (i.e. by descending the gradient). However, to apply gradient descent to the weights of deeper layers, we require the error gradients for each of those deeper layers. But, since we do not have any supervision signals corresponding to these layers (i.e. hierarchical feature labels of a car, like “wheels”, “windows”, etc) then we cannot use direct comparisons to obtain the error as we did with final layer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/1*jeMxJLZz-o5xniDMKqcTAg.jpeg" /></figure><p>Thanks to the back-propagation of errors, however, we are able to propagate a final layer’s error backward through each preceding layer, to approximate their own errors. Et voila! Problem solved.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/883/1*eV88CnsfmEeDZ-igssNF8g.png" /></figure><p>Unfortunately, backprop is no cheap process. It heavily relies on costly calculations to compute things like the inverse of an activation function which also restricts the neural activation functions to those which are easier to differentiate, etc. Aren’t there any easier, cheaper ways to estimate the errors of prior layers? Well, at Wluper, we discovered one such simpler, cheaper alternative during this year’s NeurIPS conference. It is known as Feedback Alignment, or more precisely, Direct Feedback Alignment and Indirect Feedback Alignment (as well as other variants gaining popularity).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/394/1*VUAT37pjse_mepzFP5pkzA.png" /></figure><p><strong>Feedback Alignment </strong>(FA) propagates the final layer’s error backward through each layer just like backprop. However, the weights in between each layer (through which the error propagates) are left completely random, static and untrained (as opposed to backprop which uses the Transverse of the incrementally fine-tuned weights also used in the forward pass during inference time). It seems that the network is still able to learn perfectly well when random weights are used to project the errors backward instead (this is because the forward weights also learn to align themselves in a way that make the random backward weights meaningful).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*PRxR0_looXckl-_WzJsqOg.gif" /><figcaption>Whereas Backprop (BP) propagates the error back through the incrementally-tuned weights, Feedback Alignment (FA) propagates it back through a separate set of static, untrained weights which remain completely random throughout training — and the network still learns!</figcaption></figure><p><strong>Direct Feedback Alignment </strong>(DFA) is a newer variant which propagates the final layer’s error directly to the relevant prior layer bypassing any intermediate layers between the final layer and that particular layer (i.e. it passes the error to all layers in parallel)…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/536/1*gp_RFB3icY5g0yILBNG_tw.png" /></figure><p>…instead of propagating the error back through each layer in-turn (i.e. sequentially) as with backprop and feedback alignment. This means that propagating the error backwards is not dependent on the errors of other prior layers — greatly simplifying the process of calculating prior errors!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/316/1*NHa8Nh9ATCHwceuZbifwXg.png" /><figcaption>Feedback Alignment (FA) propagates the error back through layers sequentially, while Direct Feedback Alignment (DFA) directly passes errors to each layer in parallel.</figcaption></figure><p>Lets implement a simple Neural Network class to use with DFA:</p><pre>class NeuralNetwork:<br>...  <br>def infer(self, input):<br>    layers,_= self.forward_pass(inputs)<br>    return layers[-1]</pre><pre>def forward_pass(self,inputs):<br>    hidden_layers_activated,hidden_layers_logits = self._hidden_layers(inputs)<br>    predicted_outputs = self._output_layer(hidden=hidden_layers_activated[-1])<br>    hidden_layers_activated.append(predicted_outputs)<br>    return hidden_layers_activated,hidden_layers_logits</pre><pre>def _hidden_layers(self,layer):<br>  activated_layers = []<br>  unactivated_layers = []<br>  for depth in range(self.N_HIDDEN_LAYERS):<br>    logits = layer @ self.WEIGHTS[depth]<br>    layer = self.activation_function(logits)<br>    unactivated_layers.append(logits)<br>    activated_layers.append(layer)<br>  return activated_layers,unactivated_layers</pre><pre>def _output_layer(self, hidden):<br>  return hidden @ self.WEIGHTS[-1]</pre><p>Now for the DFA related functions:</p><pre>def direct_feedback_alignment(self, inputs, outputs, epochs,learning_rate):<br>  desired_inputs = inputs<br>  desired_outputs = outputs<br>  for epoch in range(epochs):<br>    error,loss = self.direct_feedback_alignment_step(desired_inputs,desired_outputs,learning_rate)</pre><pre>def direct_feedback_alignment_step(self, desired_inputs, desired_outputs, learning_rate):<br>    hidden_layers, unactivated_layers = self.forward_pass(desired_inputs)<br>    predicted_outputs = hidden_layers.pop()<br>    self.update_weights(<br>      delta_weights=self.get_delta_weights_DFA(<br>        error=predicted_outputs - desired_outputs,<br>        hidden_layers= [desired_inputs] + hidden_layers,<br>        random_weights=self.RANDOM_WEIGHTS,<br>        unactivated_layers=unactivated_layers,<br>        desired_inputs=desired_inputs,<br>        n_layers=self.N_HIDDEN_LAYERS<br>      ),<br>    learning_rate=learning_rate<br>  )</pre><pre>def update_weights(self, delta_weights,learning_rate):<br>  for depth in range(self.N_HIDDEN_LAYERS):<br>    self.WEIGHTS[depth+1] += learning_rate*delta_weights[depth]</pre><pre>@staticmethod<br>def get_delta_weights_DFA(error,hidden_layers,random_weights,unactivated_layers,desired_inputs,n_layers):<br>  delta_weights = []<br>  e = da = error.T<br>  for depth in range(n_layers,1,-1):<br>    h = hidden_layers[depth-1]<br>    B = random_weights[depth]<br>    a = unactivated_layers[depth-1]<br>    dW = da @ h<br>    delta_weights.append(-dW.T)<br>    da =  B @ e * self.differential_of_sigmoid(a).T<br>  dW = da @ h<br>  delta_weights.append(-dW.T)<br>  return delta_weights[::-1]</pre><p><strong>Indirect Feedback Alignment</strong> (IFA) is another variant which uses just one set of random weights (reducing the number of weights stored in memory) to project the final layer’s error back to the farthest hidden layer (bypassing any intermediate layers, like direct feedback alignment). Then, the error is propagated forward to each subsequent layer (via the weights being tuned like with backprop, except it is back-to-front — so…forwardprop). This also means there is no need to take the transverse of the weights as done when going in the reverse direction (as with backprop).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/442/1*_bmogUFIQfEQ9apCtHL1zw.png" /><figcaption>Direct Feedback Alignment (DFA) vs Indirect Feedback Alignment (IFA)</figcaption></figure><p>Wluper has built upon the concepts introduced by IFA to develop a simple, one-shot deep-learning variant. So stay tuned for more on that soon. In the meantime, check out our latest <a href="https://wluper.com/careers">job openings</a>.</p><p>Any comments or questions? Feel free to reach out and leave a clap.</p><p>Don’t forget to follow us on <a href="https://twitter.com/wluper_">Twitter </a>and <a href="https://www.linkedin.com/company/wluper_">LinkedIn</a> for updates.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c2cbd152f9f1" width="1" height="1" alt=""><hr><p><a href="https://medium.com/wluper/deep-learning-2-0-a-quicker-cheaper-replacement-to-backpropagation-c2cbd152f9f1">Deep-Learning 2.0? A Quicker, Cheaper Replacement to Backpropagation</a> was originally published in <a href="https://medium.com/wluper">Wluper Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Reasoning is essential to humans and should be equally important to AI]]></title>
            <link>https://medium.com/wluper/reasoning-is-essential-to-humans-and-should-be-equally-important-to-ai-214dbba8d25e?source=rss-4e6913b61e2b------2</link>
            <guid isPermaLink="false">https://medium.com/p/214dbba8d25e</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[reasoning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[dialogue]]></category>
            <dc:creator><![CDATA[Mohammed Terry-Jack]]></dc:creator>
            <pubDate>Tue, 22 Sep 2020 01:40:30 GMT</pubDate>
            <atom:updated>2021-12-22T10:18:27.312Z</atom:updated>
            <content:encoded><![CDATA[<h4>An overview of existing reasoning approaches</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*chxS_YutKxwkKER9Z9508g.png" /><figcaption>“Hey assistant, complete the sequence: Sixty-six, Sixty-seven, …”</figcaption></figure><p>As advanced as Artificial Intelligence has come, with all the funding and research being thrown at it, it still struggles at some of the most basic tasks due to a lack of reasoning skills. This can make products and services equipped with AI seem very unintelligent, despite the leaps that have been achieved over the past years. In this piece, we will touch on how AI can reason and suggest it is a necessary component and ability of any advanced AI of the future — in particular Conversational AI — to truly become intelligent.</p><h3>1. Knowledge Reasoning vs Knowledge Retrieval</h3><p>Imagine we want to store the information given in the following sentence:</p><blockquote>“Charles has a 5 year-old daughter called Mary and a 15 year-old son called Tom”</blockquote><p>inside a knowledge base so that we can query that information later. We could memorise it exactly as it is (i.e. store it as a raw string) but this would make things like compressing the knowledge and querying it efficiently quite difficult. It is often preferable to store the information in a structured form, (like a table or graph, etc).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/744/1*JiJ2LNFan_szZIAXd3iylA.png" /><figcaption>Structuring raw data</figcaption></figure><h4>Knowledge Retrieval</h4><p>Different structures have limitations and advantages over others. Let’s say we choose to structure the sentence’s information into a table. It may look something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/354/1*zbELeZtj_-tbP_Du9xQ5lA.png" /><figcaption>An example tabular knowledge-base</figcaption></figure><p>This type of structure serves well for efficiently querying it for the knowledge it contains (knowledge retrieval).</p><p>For example:</p><ul><li>&gt;&gt;Query: “Which people are mentioned in the text?”</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/293/1*Zma3nyChU8oO_eJpcTfPMA.png" /></figure><ul><li>&gt;&gt;Query: “How old is Tom?”</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/574/1*qXNFm4Z0dnGWwpyKtajSmA.png" /></figure><ul><li>&gt;&gt;Query: “is Tom a girl?”</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/703/1*TRU0Rb6kZTU2SX53kxzqug.png" /></figure><ul><li>&gt;&gt;Query: “is Mary one of Charle’s children?”</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/708/1*uroQrf9EAnXIJvsLVOt-qw.png" /></figure><h4>Knowledge Reasoning</h4><p>However, we cannot query the knowledge base for information like:</p><ul><li>&gt;&gt;Query: “who is younger, Tom or Mary?”</li><li>&gt;&gt;Query: “is Charles younger than Mary?”</li><li>&gt;&gt;Query: “who are Tom’s parents?”</li><li>&gt;&gt;Query: “who are Mary’s siblings?”</li><li>&gt;&gt;Query: “how old is Charles?”</li></ul><p>Even though we can quite easily answer the above questions given the same sentence. We do this by reasoning. In fact, we reason so often, we are not always aware when we are doing it. If we were to analyse our thoughts, the reasoning we use to answer the above questions may sound something like this:</p><pre>Q.1) who is younger, Tom or Mary?</pre><p>Mary is younger because:</p><ul><li>Mary is 5 (“5 year-old …called Mary”)</li><li>Tom is 15 (“15 year-old …called Tom”)</li><li>5 is less than 15</li></ul><pre>Q.2) is Charles younger than Mary?</pre><p>No. Charles cannot be younger than Mary because:</p><ul><li>Mary is the child of Charles (“Charles has a…daughter called Mary”)</li><li>parents are born before their children</li></ul><pre>Q.3) who are Tom’s parents?</pre><p>Charles is a parent of Tom because:</p><ul><li>Tom is the child of Charles (“Charles has a …son called Tom”)</li><li>if you have a child A then you a parent of A</li></ul><pre>Q.4) who are Mary’s siblings?</pre><p>Tom is Mary’s sibling because:</p><ul><li>Charles is Mary’s father (“Charles has a… daughter called Mary”)</li><li>Charles is Tom’s father (“Charles has a… son called Tom”)</li><li>Someone with the same father is your sibling</li></ul><pre>Q.5) how old is Charles?</pre><p>Charles is definitely older than 15 because:</p><ul><li>Tom is the son of Charles (“Charles has a…son called Tom”)</li><li>Tom is 15 (“…15 year-old …called Tom”)</li><li>a parent is older (born before) their child</li></ul><h4>Types of Reasoning: Deductive vs Abductive Logic</h4><p>As you can see from the examples above, the actual information we used to conclude our answers was only partly extracted from the information in the text itself. We also use principals to expand that knowledge to lead us to answers that were not explicitly stated in the text. This type of reasoning is known as <strong>deductive </strong>logic and the deduced information is necessarily true as long as their underlying premises (i.e. the information from the sentence) are true.</p><p><strong>Abductive </strong>logic is another type of reasoning which is more often used by us. The last question — “<em>how old is Charles?</em>” — is a prime example of where we may use such reasoning to further conclude that Charles is quite likely older than 15. The reasoning may look something like this:</p><ul><li>Charles was likely past the age of puberty when he became a parent (because this is usually the case — a reasonable assumption);</li><li>the exact age of puberty varies from person to person, but is usually around the teenage years (another fair assumption);</li><li>Thus we can safely assume that Charles was older than 12 when he became a father;</li><li>and conclude, with some level of confidence, that Charles is most probably older than 27 years (at least 12 years for puberty + 15 years to be older than his eldest son).</li></ul><p>Such reasoning is useful for real world problems, however, even if the underlying premises are true, the conclusions may not be due to the assumed theories used during the construction of the conclusions — e.g.Charles may have been abnormally young when he became pubescent, etc.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/587/1*_0fv8EGgK4-Mgemlbc2bMw.png" /></figure><h4>Relational Knowledge Bases (e.g. Knowledge Webs, etc.)</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/0*CBnqIzMcY4uHOOxq.jpg" /></figure><p>One way to perform reasoning over our knowledge base is to use relational structures such that all knowledge is stored as a relation to another piece of knowledge (forming a big knowledge web or relational graph). The sentence stored in our relational knowledge base may look something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/221/1*MoZThPR3ORkXd432ZedbYQ.png" /><figcaption>Horn clauses are used to structure the sentence’s info into an example relational KB (syntax = SWI Prolog).</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/618/1*emNcPNfV-yNyHarzHq6duA.png" /><figcaption>The relational KB displayed as a graph</figcaption></figure><h4>Knowledge Retrieval</h4><p>Now we can perform standard knowledge retrieval queries as we did with the tabular knowledge base, e.g.</p><ul><li>&gt;&gt;Query: “Which people are mentioned in the text?”</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/152/1*zozr11KxVCaGj38xrX_Iiw.png" /></figure><ul><li>&gt;&gt;Query: “How old is Tom?”</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/157/1*9gkAdHIIGPcDGgkT5rxYkg.png" /></figure><ul><li>&gt;&gt;Query: “is Tom a girl?”</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/201/1*IQzW-VSspTklYXVxCzNxmQ.png" /></figure><ul><li>&gt;&gt;Query: “is Mary one of Charle’s children?”</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/254/1*tE0EcdtryF2XPb0A-DcNvA.png" /></figure><h4>Knowledge Reasoning</h4><p>We can also encode reasoning rules to perform over our relational database too. For example, the reasoning we used before for deducing someone is your sibling was “<em>Someone with the same father is your sibling</em>” which can be formally encoded like so:</p><pre>is_sibling_of(PERSON_A, PERSON_B) :- <br>   is_child_of(PERSON_A, PERSON_C),<br>   is_child_of(PERSON_B, PERSON_C), <br>   PERSON_A \= PERSON_B.</pre><p>Which essentially means a new relation-type (called “is_sibling_of/2”) will be automatically created in our database between any two objects (here referred to by the generic placeholder names PERSON_A and PERSON_B) if the first object (PERSON_A) is a child of a third person (PERSON_C) and the second person (PERSON_B) is also a child of that same third person (PERSON_C). Whats-more, the two objects (PERSON_A and PERSON_B) cannot be the same (PERSON_A \= PERSON_B).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/285/1*oY1r_9BZmdMrY6EbPD066A.png" /><figcaption>New relation (is_sibling_of/2) is automatically deduced when the conditions of the reasoning rule are met.</figcaption></figure><p>If we encode the rest of the reasoning rules used when solving the answers above, we may end up with something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/289/1*eBqaG78jY9Hr1QkC4jLqKA.png" /><figcaption>Example reasoning rules stored in our knowledge base (syntax = SWI Prolog)</figcaption></figure><p>Now we can query our knowledge base for things that we previously could not in our tabular knowledge base, like:</p><ul><li>&gt;&gt;Query: “who is younger, Tom or Mary?”</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/314/1*IJzpAuD1abCKaCp0BS-tWQ.png" /><figcaption>the first results returned are the answer to the question. tom (older), mary (younger). The other answers are bonus comparisons.</figcaption></figure><ul><li>&gt;&gt;Query: “is Charles younger than Mary?”</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/284/1*S-xxC2G4WeSazXR_I0Br2Q.png" /></figure><ul><li>&gt;&gt;Query: “who are Tom’s parents?”</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/225/1*f5qn-ADUtGMl1Eod7_WrjQ.png" /></figure><ul><li>&gt;&gt;Query: “who are Mary’s siblings?”</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/235/1*aKnR1w6W2Z8bOKrHNEf9wA.png" /></figure><p>and so on…</p><h3>2. Reasoning to Answer Complex Queries</h3><p>Now that you can do reasoning over your knowledge base, handling complex queries becomes possible. To illustrate this, let’s imagine we have a relational knowledge base of restaurants to query.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/673/1*dRzNtvsFKiF6lBK42QPuFg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/273/1*JRR3bM3siq01otzWKxZ90Q.png" /><figcaption>A small restaurant KB</figcaption></figure><p>Now suppose someone expressed a fairly complex query, like:</p><blockquote>Hey, find me a restaurant that sells good Sushi near Charing Cross please</blockquote><p>This query can be expressed as multiple relations coupled together via a shared variable to enforce a constrained search. For example:</p><pre>restaurant(PLACE),<br>sells(PLACE, sushi),<br>near(PLACE, `Charing Cross`).</pre><p>This is a single query, composed of three relations, but the shared PLACE variable ties the relations together such that the returned value for PLACE needs to satisfy each and every relation it is in. I.e. the value for PLACE needs to be a restaurant, sell sushi and be near to Charing Cross.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/632/1*WUZ4cFEgpdHqzXwkMd4Ceg.png" /><figcaption>The query displayed as a relational graph.</figcaption></figure><p>The only value in our small database which satisfies all three conditions specified in the query is a restaurant called “Sakagura”.</p><h4>Reasoning to handle Queries with Negations</h4><p>Reasoning can even be useful for queries which contain negations, like:</p><blockquote>Show me a restaurant near Charing Cross that does <strong>not</strong> sell Sushi</blockquote><pre>restaurant(PLACE), <br>near(PLACE, Charing Cross), <br>sells(PLACE, FOOD_TYPE), <br>FOOD_TYPE\=sushi.</pre><p>Here we specify that the value for the variable FOOD_TYPE must not be sushi (FOOD_TYPE\=sushi).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/664/1*-jnHLJchJuVX9w1myvHIEw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/283/1*VR7Bdc8ShG-EPhIG08C3DQ.png" /><figcaption>Query displayed as a relational graph and the returned values which satisfy the conditions of the query.</figcaption></figure><h4>Reasoning to handle Co-reference Resolution</h4><p>Another powerful example where reasoning becomes crucial. It can help resolve ambiguous co-references, such as:</p><blockquote>I have a key and a lock, so I turned <strong>it </strong>and <strong>it</strong> opened!</blockquote><p>The question here is, what does each “it” refer to? The lock? The key?</p><p>We know that the first “it” refers to the key and the second “it” to the lock, but how? Perhaps we reasoned that a key is usually <em>turned</em> but not locks. Likewise, that only locks can be <em>opened</em>, not keys.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/94/1*OVaV1qYR1viG-Yh93HG_5Q.png" /><figcaption>Our example KB containing information on which objects can be opened, etc.</figcaption></figure><p>&gt;&gt;Query: it opened?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/133/1*4TrGe3a04OV2ROOiJBvX-A.png" /><figcaption>Co-reference resolution, “IT” refers to the lock</figcaption></figure><h3>3. Conclusion</h3><p>We have seen great things that reasoning over our knowledge base enables. The examples range from deducing new knowledge from the seen knowledge, answering complex questions, creating new facts in our knowledge base and resolving ambiguities. Reasoning clearly goes beyond the data seen in training examples.</p><p>So what does the future hold for us? <br>Hint: Neural-Symbolic Machine Learning</p><p>Large language models such as BERT, T5, or GPT-3 consume large amounts of knowledge from huge amounts of text (e.g. all of Wikipedia, the whole web, etc.). This knowledge is stored in a distributed way in the underlying neural network alongside information on how language works (i.e. word order, grammar, vocabulary). That again is a very powerful paradigm and extends the usual paradigm of data-storage, as this knowledge is injected into the neural network without human supervision. (Obviously at the same time the knowledge is stored implicitly and therefore more obscurely.)</p><p><a href="https://medium.com/u/d7e74ac84d28">Gary Marcus</a> and <a href="https://medium.com/u/e72b6b0b165e">Nikolai Rozanov</a> have in the past criticised pure Neural Network approaches that simply learn from incoming data and create a distributed version of the seen knowledge. Even though this is a powerful ideal of knowledge storage, there are limitations as to what knowledge storage and retrieval can achieve. <strong>That limit is reasoning.</strong></p><p>Reasoning truly is a fascinating, defining feature of human intelligence, but how can it be utilised to make AI more intelligent? How do we reason over the distributed knowledge? How do we extract the relational queries automatically from natural language?</p><p>These are all questions that we at <a href="https://wluper.com/">Wluper </a>are addressing. So stay tuned for more and check out our upcoming <a href="https://wluper.com/careers">job openings</a>.</p><p>Any comments or questions? Feel free to reach out to us and leave a clap.</p><p>Don’t forget to follow us on <a href="https://twitter.com/wluper_">Twitter </a>and <a href="https://www.linkedin.com/company/wluper_">LinkedIn</a>.</p><h3>References</h3><p>To run Python in browser, try Colab.<br>To run Prolog in browser, try (<a href="https://swish.swi-prolog.org/">https://swish.swi-prolog.org/</a>)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=214dbba8d25e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/wluper/reasoning-is-essential-to-humans-and-should-be-equally-important-to-ai-214dbba8d25e">Reasoning is essential to humans and should be equally important to AI</a> was originally published in <a href="https://medium.com/wluper">Wluper Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Zero-Shot Classification!]]></title>
            <link>https://medium.com/@b.terryjack/zero-shot-classification-584bb39da2e2?source=rss-4e6913b61e2b------2</link>
            <guid isPermaLink="false">https://medium.com/p/584bb39da2e2</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[pre-trained-model]]></category>
            <category><![CDATA[language-model]]></category>
            <category><![CDATA[classification]]></category>
            <category><![CDATA[zero-shot-learning]]></category>
            <dc:creator><![CDATA[Mohammed Terry-Jack]]></dc:creator>
            <pubDate>Thu, 13 Aug 2020 11:48:21 GMT</pubDate>
            <atom:updated>2020-08-13T11:49:58.755Z</atom:updated>
            <content:encoded><![CDATA[<p>Hugging Face is amazing — they’ve released a Zero-shot-classification pipeline using pre-trained language models in their transformers library</p><pre>pip install git+https://github.com/huggingface/transformers.git</pre><pre>from transformers import pipeline<br>from numpy import argmax</pre><p>Thats all thats needed to download the classifier. Then test it on any sentence and set of classes you like?</p><pre>zero_shot_classifier = pipeline(&quot;zero-shot-classification&quot;)</pre><pre>results = zero_shot_classifier(<br>  &quot;computers are so slow&quot;, <br>  [&quot;IT issue&quot;, &quot;Price request&quot;, &quot;Datetime query&quot;]<br>)<br>SCORES = results[&quot;scores&quot;]<br>CLASSES = results[&quot;labels&quot;]<br>BEST_INDEX = argmax(SCORES)<br>predicted_class = CLASSES[BEST_INDEX]</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/102/1*-QLpZ3heu8J3o8I9UX9fKg.png" /><figcaption>predicted class for “computers are so slow”</figcaption></figure><p>Keep in mind that the class labels provided are dynamic (can be changed at any time)</p><p>Another example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/133/1*oF2D2q2mQVIgYg2LJYZGuw.jpeg" /><figcaption>Predicted class for “What time will i arrive?”</figcaption></figure><p>So you can even use this as a pre-trained NLU classifier — or most other NLP classification tasks for that matter (since it doesnt require training)</p><p><a href="https://colab.research.google.com/github/mohammedterryjack/Medium/blob/master/ZeroShotClassification.ipynb">Google Colaboratory</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=584bb39da2e2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Amazing Alternatives to Classifiers]]></title>
            <link>https://medium.com/swlh/amazing-alternatives-to-classifiers-b7bd7e85b60d?source=rss-4e6913b61e2b------2</link>
            <guid isPermaLink="false">https://medium.com/p/b7bd7e85b60d</guid>
            <category><![CDATA[multilabel-classifier]]></category>
            <category><![CDATA[siamese-networks]]></category>
            <category><![CDATA[classification]]></category>
            <category><![CDATA[algorithms]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Mohammed Terry-Jack]]></dc:creator>
            <pubDate>Tue, 04 Aug 2020 18:15:24 GMT</pubDate>
            <atom:updated>2020-08-10T16:38:17.993Z</atom:updated>
            <content:encoded><![CDATA[<p>Many tasks in Machine Learning are setup as classification tasks. The name would imply you have to learn a classifier to solve such a task. However, there are many popular alternatives to solving classification tasks which do not involve training a classifier at all!</p><h4>An Example Classification Task</h4><p>For the purpose of illustrating these alternative methods, let’s use a very simple, yet visual classification problem. Our dataset shall be a random image from the internet. But only a single image… of Ironman!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/315/1*TAL2j6YrZTcyTIBGJ4hAuw.png" /><figcaption>(Left) Original Image from internet. (Right) RGBA Colours simplified to 5 Classes</figcaption></figure><p>Each image pixel of this image will count as a unique data sample. Our set of inputs (X) shall be the pixel coordinates and our set of outputs (Y) shall be the pixel colours.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/594/0*vIVOqPfmdLs8-AhE.png" /></figure><p>The classification task is to predict a pixel colour (y) given a coordinate (x).</p><ul><li>Inputs (X) = pixel coordinates</li><li>Outputs (Y) = pixel colour class</li></ul><p>We can convert the pixel’s continuous RGBA colour vector into a discrete, categorical class index, so that this toy classification task is simplified.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/381/1*27CcCRy4B3UxZ4jdX1Ks7Q.png" /><figcaption>The 5 colour classes for our toy classification task</figcaption></figure><h4>The Train-Test Split</h4><p>As a small aside, when splitting the dataset into a training set and a test set, they should have a similar distribution. This means we cant simply split the data down the middle, because the distribution of data that we would test our results on would be very different to the data we used in training (thus leading to bad and/or misleading results)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/253/1*CC-VHh0DtAcv9K7scoCVmw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/217/1*nJU4HZnLlGlMEuSPLqRgmA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/258/1*4SOy6sDVjVxFMx2S9bmsrw.png" /><figcaption>(Left) How NOT to split your data. (Right) Bad results. Test Predictions from a classifier trained on the top Train data which has a very different distribution to the test data</figcaption></figure><p>That is why you should take care when preparing your data (this is all taken care of in the background when using statistical libraries like sklearn)</p><pre>from sklearn.model_selection import train_test_split</pre><pre>X_train, X_test, Y_train, Y_test = train_test_split(X, Y)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/249/1*CtgsCYsktS7Cd875-x2YsA.png" /><figcaption>Much better. The training set and test set have been split in such a way to keep similar distributions.</figcaption></figure><h4><strong>The Confusion Matrix</strong></h4><p>We shall use a confusion matrix to plot how well our method classified the colour for each pixel coordinate in the test set. A perfect classification would look like the confusion matrix below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/446/1*cJMIS-1hCMSQUgyBASc7cQ.png" /><figcaption>Confusion Matrix plotting an Ideal Classification</figcaption></figure><h4>1. The Direct Approach: Learn a Classifier</h4><p>Now most people solve classification tasks by training a classifier — i.e. Learning a nonlinear function (f) (the classifier) which maps a set of inputs (X) to a set of outputs (Y).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/710/0*z2ERyPYLAes1pUVW.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/0*86Ly38uxpYYamnT1" /></figure><p>Let us do that too so we have a baseline solution to this classification task to compare against. There are a range of ML models out there to approximate a nonlinear function (the classifier) — but we shall simply use a feed-forward neural network with 3 hidden layers (100 neurons per layer)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/589/1*0sE_7fWqvUajIKQa0xrtvQ.png" /><figcaption>training our classifier</figcaption></figure><p>The results were not bad. They weren’t perfect — but it does a good enough job on our toy classification task (despite occasionally missing off the right eye of ironman)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/256/1*Uu07wm8ufH4EuiKNIyJv7Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/234/1*_r8TsiEqm8KHSNMTZi7mWQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/238/1*SNO2fY8vD2K94WYoi-zkXw.png" /><figcaption>Three trained instances of our classifier predicting the pixel colours on the ironman test set.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/435/1*XETT0IuW3CO2Qg7qs5GL4g.png" /></figure><h3>2. Alternative Approaches: Ranking</h3><p>Ranking (aka Retrieval) is a widely used alternative to training a function (classifier) which directly predicts an output for a given input. By measuring the distance of an input (x) to an output (y), a given input can be ranked against a range of outputs and the best can then be selected (retrieved). Ranking like this is an indirect way to map a given input (x) to an output (y) without requiring a trained classifier (f). Or another way to think about it is — you have decomposed your classifier (f) into an evaluation/distance function (Q) and ranking algorithm (arg max).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/311/1*QyTlM4QnoBjUetYzJ_nefQ.png" /><figcaption>The infamous Q-function equation in RL is actually an example of ranking to classify if we interpret the notation as: input (s), output (a), distance function (Q)</figcaption></figure><p>However, doesn’t this mean you just substituted one type of problem for another ? Haven’t you just shifted the problem of learning one function (the classifier) for another function (the distance function)? Yes — but… the latter function may be much easier to learn or you may not even need to learn a distance function if you can use a predefined distance measure (e.g. euclidean distance, cosine similarity, etc).</p><h4>2.a K Nearest Neighbours</h4><p>K nearest neighbours (KNN) is actually a very well known ranking algorithm used to solve classification tasks. It essentially ranks the new input (x) against a set of K example inputs using a linear distance measure (e.g. euclidean, cosine, etc) and then uses the output (y) of the most similar example input as its prediction.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*9dIHpqBHgWocQXaQSQCFuw.gif" /></figure><p>The KNN algorithm is one of the simpler forms of machine learning known as example-based learning (aka retrieval-based learning, instance-based learning, lazy learning, etc), as it relies on memorising examples to avoid learning / approximating a function. It simply uses an off-the-shelf distance function to compare inputs against memorised, exemplar inputs (embedded in the same space).</p><pre>from sklearn.neighbors import KNeighborsClassifier</pre><pre>nearest_neighbour_algorithm = KNeighborsClassifier(n_neighbors=3)<br>nearest_neighbour_algorithm.fit(X_train, Y_train)<br>Y_predicted = nearest_neighbour_algorithm.predict(X_test)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/228/1*mfPs6lQ-kawBDuWakxBLug.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/434/1*S4Ldws7qpP5dz_gH90f5rw.png" /><figcaption>Almost perfect Test Predictions for the KNN method</figcaption></figure><p>Siamese Networks actually work on a very similar principal to KNNs (i.e. they compare new inputs to exemplar inputs to retrieve their corresponding outputs), however it learns its own, custom distance function.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*oxRQfFiHCyHlXMax.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/735/0*BvuNkztfDo-m__yO.jpeg" /><figcaption>Siamese Neural Network</figcaption></figure><h4>2.b Joint Embeddings</h4><p>We can get away with using custom distance functions by comparing inputs (x) with other inputs (x) which are embedded in the same space. However, this is more tricky when measuring the distance between inputs (x) with outputs (y) directly due to them being embedded in two different spaces. We can overcome this by using joint embeddings (i.e. embedding the inputs and outputs in a shared space).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/390/1*x2XEHjpBnAgpMRIVhHQDOQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*j3sofoN4q85fKDM9.png" /><figcaption>examples of joint embeddings</figcaption></figure><p>Therefore this method involves learning joint embeddings such that the inputs (X) and outputs (Y) can be embedded into a shared space. Then we can rank a new input against the outputs using linear distance metrics (e.g. euclidean distance, cosine distance, etc). There are a few different ways to embed the outputs and inputs onto a shared manifold — such as taking the learnt feature representations of a neural model (e.g. Recurrent Embedding Dialogue Policy, etc).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*-olDJS1-4ZUKHdsA.png" /></figure><p>We shall be using Matrix Factorisation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*ARZdddf_5euiv_yu.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/349/1*dibGRKa5n5K7dwFOCzfCzQ.png" /><figcaption>(Right) The Outputs embedded into the input’s XY coordinate space using Matrix Factorisation</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/307/1*PAgQQRD0peGAuCEXj7IS4w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/427/1*z8NqtasJwIuz51TViaIbHw.png" /><figcaption>Test Predictions using the learnt joint embeddings</figcaption></figure><h4>2.c Learning a Distance Function</h4><p>You can also approximate a custom, nonlinear distance measure which would have learnt to measure the relative distances of input-output pairs (despite being embedded differently). We can do this by feeding in a set of concatenated input-output vectors as the model input, and training the model to predict a distance score (a value between 0 and 1) for the concatenated input-output pair.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/702/1*JK-nXaEXD0fWdHGnWPmYIQ.png" /></figure><p>We therefore modify the training data by concatenating inputs and outputs randomly (to become our new training inputs) and setting their expected distance to 0 if the input-output pair is a valid mapping,and 1 if its an invalid input-output mapping. The model used to approximate the distance function is simply a feedforward neural network (with 3 hidden layers, each with 100 neurons) almost identical to the one used for the classifier method (except this neural network is set up for learning a regression task as opposed to a classification task).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/325/1*LlTHeELLv_Ok8dsEtwvO1A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/614/1*IJR7SmfX3VvqQzr0yzV_8w.png" /><figcaption>Training a feedforward neural network to be a nonlinear distance function (aka cost function). Input = Input-Output pair. Ouput=distance.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/203/1*ZrOnXIrDijUxA0tLIWiBZw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/244/1*w5U0wYEfIBp3MQApqV0BPg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/431/1*zfjI_RcfBSMDvkrLr-5DTA.png" /><figcaption>Test Predictions using the Learnt Distance Function</figcaption></figure><h4>Conclusion</h4><p>Performance wise, the results of the alternative methods were similar to the classifier, however, retrieval based methods do present additional benefits over a classifier in other areas, like adding a stochastic element (if the top ranking result is always chosen it is identical to using a classifier or another deterministic mapping, however, adding some uncertainty can be done by ensuring the top ranking result is not always chosen), or like updating classes more easily, solving much larger multiclass classification problems (i.e. those with hundreds of classes) and being able to solve multilabel classification problems (i.e. predicting the top N applicable outputs for a given input), etc.</p><h4>Thanks</h4><p>Thanks for reading. All the code is included in the attached Colab notebook. Have fun experimenting</p><p><a href="https://colab.research.google.com/github/mohammedterryjack/Medium/blob/master/Alternatives_to_Classifiers.ipynb">Google Colaboratory</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b7bd7e85b60d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/amazing-alternatives-to-classifiers-b7bd7e85b60d">Amazing Alternatives to Classifiers</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What makes a good conversation]]></title>
            <link>https://medium.com/@b.terryjack/what-makes-a-good-conversation-8d22a9b96c34?source=rss-4e6913b61e2b------2</link>
            <guid isPermaLink="false">https://medium.com/p/8d22a9b96c34</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[nlu]]></category>
            <category><![CDATA[conversational-ai]]></category>
            <category><![CDATA[nlg]]></category>
            <dc:creator><![CDATA[Mohammed Terry-Jack]]></dc:creator>
            <pubDate>Mon, 01 Jun 2020 19:36:10 GMT</pubDate>
            <atom:updated>2020-06-01T19:36:10.900Z</atom:updated>
            <content:encoded><![CDATA[<h4><a href="https://arxiv.org/pdf/1902.08654.pdf">Paper Summary</a></h4><h3>Open Problems</h3><h4>…with system metrics and evaluations:</h4><blockquote>● other NLP metrics do not necessarily map to successful conversational AI (“word-overlap metrics are effective for question-answering and machine translation, for dialogue they have little to no correlation with human judgments”)<br>● “automatic metrics and their relation to human ratings are less well-understood”.<br>● “factors that determine human judgments …are almost entirely unexplored”<br>● “multi-turn evaluation is necessary” (“Most …works only consider single turn evaluations”)</blockquote><h4>…with system responses:</h4><p>● Contradicting previous statements<br>● Fail to balance asking questions with other dialogue acts (“Considerate chitchat requires a reciprocal asking and answering of questions — asking too few or too many can appear self-centred or nosy”)<br>● Genericness (short, dull responses — e.g. “I don’t know”)<br>● Repetitiveness (3 types):</p><blockquote>● Partner repetition (repeating the conversational partner)<br>● internal repetition (self-repetition within utterances)<br>● external repetition (self repetition across utterances)<br>○ self-repetition across utterances (external repetition) is by far the most severe<br>form of repetition in our beam search baseline model</blockquote><h3>Aim</h3><blockquote>“Identify …factors governing human judgments (of full conversations)”</blockquote><h3>Findings</h3><blockquote>● “existing work has ignored the importance of conversational flow”<br>● “the importance of measuring more than one quality metric when evaluating dialogue agents”<br>● “we obtain significantly higher engagingness scores …by optimising control of repetition, specificity and question-asking over multiple turns”<br>● “controlling for repetition, specificity and question-asking all lead to large engagingness improvements over the greedy and beam-search baseline models”<br>● “found no improvement by controlling response-relatedness”</blockquote><blockquote>Using these findings, our best model matches the performance of the winning entry in the recent NeurIPS ConvAI2 competition</blockquote><h3>Research Approach</h3><blockquote>1. Study 8 aspects of conversation (measured by human judgments)<br>2. Vary 4 types of low-level attributes (algorithmically controlled)</blockquote><blockquote>“We manipulate four low-level attributes and measure their effect on human judgments of individual conversational aspects, as well as overall quality.”</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jVYsLcXJRHbJIfS4ZO-NwQ.png" /></figure><h4>Single-Turn Attributes</h4><p>“Researchers propose several approaches to control… natural language generation, such as sentiment, length, speaker style, tense, etc”</p><h4>Multi-Turn Attributes</h4><blockquote>● “systems have systemic problems when applied to open-ended dialogue, some of which (e.g. repetition) are only observable in the multi-turn setting”<br>● “we focus on …multi-turn interactive dialogue …to control attributes at the dialogue level rather than the utterance level… attribute is naturally continuous”<br>● “Four important attributes for chitchat dialogue:”</blockquote><blockquote>1. repetition<br>2. specificity</blockquote><blockquote>3. response-relatedness<br>4. question-asking</blockquote><h4>Other Attributes</h4><p>Conversational aspects such as interestingness, listening, and inquisitiveness are all important — though optimising these can introduce a trade-off against certain types of errors (such as repetitive, disfluent, or nonsensical output)</p><h3>Control Methods</h3><p>“We study two control methods:”<br>● <strong>conditional training</strong> (CT)<br>● <strong>weighted decoding</strong> (WD)</p><blockquote>conditional training requires retraining; weighted decoding doesn’t, but is slower at test time.</blockquote><h3>Model</h3><p>2-layer LSTM sequence-to-sequence model with attention</p><h4>Encoder Input (x)</h4><p>● entire dialogue history (separated using unique speaker identifying tokens)<br>● with the model’s own persona prepended<br>● word embedding matrix (300-dimensional GloVe)</p><h4>Data</h4><p>● pre-trained: 2.5 million Twitter message-response pairs (approximately 79 million words)<br>● fine-tuned: PersonaChat</p><h4>Conditional Training (CT)</h4><p>● requires training examples of the controllable attribute<br>● automatically annotate every (x, y) pair in the training set with the attribute we wish to control (e.g. whether y contains a question mark)</p><p>several possible ways to condition the sequence-to-sequence model on z<br>● append z to the end of the input sequence<br>● use z as the START symbol for the decoder<br>● concatenate z to the decoder’s input on every step</p><p>optimise the cross-entropy loss</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/548/1*hg4HxMdYojDaxkT6Bd93Pg.png" /></figure><h4>Weighted Decoding (WD)</h4><p>● decode using beam search with beam size 20<br>● control any computable feature without requiring examples<br>● increases or decreases the probability of words with certain features<br>● controllable attribute must be defined at the word-level</p><p>● <strong>Weighting Repetition</strong>: identify repeating bigrams for the three repetition types…(and) repeating content words. …applying a negative weight to these features… do not use the feature …bigram for weighted decoding during beam search, but we do use it to rerank the candidates after beam search.</p><p><strong>Weighting Specificity</strong>: Normalised Inverse Document Frequency (NIDF) as a measure of word rareness… mean NIDF of the words</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/546/1*nd8sfGvrofWL8_ZiLcNsnA.png" /></figure><p>● <strong>Weighting Response-relatedness</strong>: cos_sim(word_emb(w), sent_emb(l)) … word_emb = GloVe embedding for the word w…sent_emb is the sentence embedding for the partner’s last utterance (l)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/546/1*lkeNqgsUilapDiUdMSxhFQ.png" /></figure><p>● <strong>Weighting Question-asking</strong>: is_qn word(w) … 1 if … word w is in … list of interrogative words (how, what, when, where, which, who, whom, whose, why, ?). negative weight can discourage valid non-question utterances that happen to contain interrogative words (“such as I’m learning how to knit”)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8d22a9b96c34" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A gentle introduction to Embedding Trees and Graphs (with code)]]></title>
            <link>https://medium.com/wluper/a-gentle-introduction-to-embedding-trees-and-graphs-with-code-92dc3b6b6ec7?source=rss-4e6913b61e2b------2</link>
            <guid isPermaLink="false">https://medium.com/p/92dc3b6b6ec7</guid>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[data-structures]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Mohammed Terry-Jack]]></dc:creator>
            <pubDate>Tue, 07 Apr 2020 03:00:44 GMT</pubDate>
            <atom:updated>2021-12-21T02:54:45.156Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gFrj27Ogw8u5c3E-Ke42lA.png" /></figure><p>Deep Neural Networks (Deep Learning) are dominating advances in Machine Learning and leading to incredible leaps in various fields. However, as prominent figures in the likes of <a href="https://medium.com/u/d7e74ac84d28">Gary Marcus</a> and <a href="https://medium.com/u/412dfd9a98f8">David Cox</a> suggest, there are serious limitations with these approaches. We, too, believe that <a href="https://medium.com/wluper/conversational-ai-but-where-is-the-i-599e2430f42a">new approaches</a> are needed, which may even originate in <strong>symbolic AI.</strong></p><p>This blog post is a primer on how to leverage structured knowledge, i.e. trees and graphs, with deep learning for NLP.</p><h3>Contents</h3><ol><li><strong>Advantages of Tree/Graph Embeddings</strong></li><li><strong>Example Embedding Methods</strong></li><li><strong>Embedding WordNet</strong></li><li><strong>Using the Embedded WordNet in a Downstream ML task</strong></li><li><strong>Conclusion and code</strong></li></ol><h3>1. Advantages of Tree / Graph Embeddings</h3><p>Symbolic representations like trees and graphs are favourable due to their interpretable nature, however, they can become computationally burdensome when scaled up and it becomes important to perform searches and other tree/graph-based computations over a approximated vector representations of the data structure.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/660/0*vFho-ASMre1b6lzs" /></figure><p>Furthermore, if we wish to utilise structured information from trees and graphs in downstream machine learning tasks (i.e. to recommend new friendships in social networks or predict a new drug designs, etc)…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/405/1*Itl9SIhNP31fEPpIMhfeKA.png" /></figure><p>…we must be able to convert the tree or graph into an appropriate representation required by the machine learning algorithm (usually a factored input representation). Therefore we also need embedding methods to encode trees and graphs into compatible vectors.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/279/1*aRfOfTQg0lOqv8EIEC-Lzw.png" /></figure><h3>2. Example Embedding Methods</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/695/1*MkAJRQVYDAZFd-Wh0gziHQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/584/1*2Z4YWWzUyV5t5njYs3en7g.png" /></figure><p><strong>Encoding Nodes using Edges</strong></p><p>Starting with a very simple method to encode a graph into a vector, each node (or vertex) in the node can be encoded according to the edges which are connected to it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*9Qe442Wx5UVEG9_L" /></figure><pre>def create_vectors_from_edges(graph:Graph) -&gt; List[List[int]]:<br>   return [<br>      [<br>         int(node in edge) for edge in graph.edges<br>      ]  for node in graph.nodes<br>   ]</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/257/1*GiyUkxYzIB7kqDP_tIwrpQ.png" /><figcaption>Each node is encoded into a vector like so (whereby each feature corresponds to an edge in the graph)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/445/1*muTGP_5WenWsv3qiLtkH2g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/384/1*tS1hriovy5PlNh0m7p4QNg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/458/1*K6A7Mzd5Ph_pGFD5VPhGtQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/381/1*pvFoBldtEjECt_y7CSSPbw.png" /></figure><p>Unfortunately, the number of edges in a graph can become very large (depending on the complexity of the graph) and so this approach does not scale well.</p><p><strong>Encoding Nodes using Adjacent Nodes</strong></p><p>A similar approach is to make each feature of the vector correspond to the nodes themselves (as opposed to the edges). Each node can then be vectorised according to its adjacent / neighbouring nodes.</p><pre>def create_vectors_from_adjacent_nodes(graph:Graph) -&gt; List[List[int]]:<br>   adjacent_nodes = [<br>      set(graph[node_id].keys()) | {node_id} for node_id in graph.nodes<br>   ]<br>   return list(<br>      map(<br>         lambda neighbours: one_hot_encoding(<br>            indexes=neighbours,<br>            vector_length=len(graph.nodes)<br>         ),<br>      adjacent_nodes<br>      )<br>   )</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/445/1*muTGP_5WenWsv3qiLtkH2g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/386/1*fRHkidUmgR3550eMlT6gBA.png" /></figure><p>Notice how nodes which share similar neighbours are given similar embeddings and thus are plotted close together (indicating they are very similar). The intuition behind why this works becomes clearer if we imagine the nodes represent academic papers (connected in a network of references). Two papers that cite very similar references are assumed to have similar content. Another analogy would be that people with a similar social network would be more similar in personality (or as they say — your friends are a reflection of yourself)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/458/1*K6A7Mzd5Ph_pGFD5VPhGtQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/396/1*YuXbp7gkzcRcFyA4-BpUyw.png" /></figure><p>Unfortunately this method only captures local structures formed by immediate neighbours and ignores more global, indirect connections. Embedding methods face an unusual challenge since they must successful encode local neighbourhood relations of each node (i.e. sub-graph structures) while simultaneously maintaining global network information (i.e. topology).</p><p><strong>Encoding Nodes using every Node</strong></p><p>This next technique assigns vector features to nodes (again), but instead of only encoding the neighbouring / adjacent nodes, the distance to all other node are used as feature values to ensure both local and global structures are encoded in the vector.</p><pre>def create_vectors_from_all_nodes(graph:Graph) -&gt; List[List[int]]:    <br>   return [    <br>      [     <br>         sigmoid(shortest_path_length(graph, node_start, node_end)) for node_end in graph.nodes    <br>      ] for node_start in graph.nodes  <br>   ]</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/445/1*muTGP_5WenWsv3qiLtkH2g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/448/1*NZP_43D1-U29mmr7if0KDQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/458/1*K6A7Mzd5Ph_pGFD5VPhGtQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/395/1*KYGFQSiUda41bD2ANRME2Q.png" /></figure><p><strong>Encoding Nodes using Parent Nodes</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*erb2770mtgM8DooA.jpg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/593/0*uh7EzuM0keM8DptS.jpg" /></figure><p>With trees or hierarchical graphs (e.g. WordNet), an special embedding method can make use of the node’s parent nodes to create an embedding (i.e. all the nodes between it and the root node). Unlike using adjacent / neighbouring nodes, this method captures both local and global relational structures. The intuition behind this approach is that nodes with similar ancestral chains will be highly related and thus very similar. For example, if we assume that node 0 is the root node in the graph below, then the vector of node 3 [1,1,1,0,0,0,0] and node 7 [1,1,1,0,0,0,0] are identical since they both share an identical chain of parent nodes (node0, node1, node2).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/458/1*K6A7Mzd5Ph_pGFD5VPhGtQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/381/1*H8L0P8RhJrf7Wx2dpSn_Pg.png" /></figure><p>If you wish, you can also include the current node in the embedding itself to make each embedding slightly unique (e.g. the embedding for node 3 [<strong>1,1,1</strong>,1,0,0,0] would slightly differ now from its sister node 7 [<strong>1,1,1</strong>,0,0,0,1])</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/458/1*K6A7Mzd5Ph_pGFD5VPhGtQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/387/1*SwRGLLZTGGdmQJkRziWBtg.png" /></figure><pre>def get_all_parent_nodes(graph:Graph,node_start:int, node_end:int, include_node_end:bool) -&gt; Set[int]:<br>  parent_nodes = reduce(<br>    lambda path_a,path_b: set(path_a)|set(path_b),<br>    all_shortest_paths(graph,node_start,node_end)<br>  )<br>  return parent_nodes if include_node_end else set(parent_nodes) ^ {node_end}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/821/1*hBfbWPB68LqEf5xwlUZcZQ.png" /><figcaption>A section of the WordNet graph</figcaption></figure><p>We can modify this methods slightly to apply it to a real-world graph known as WordNet:</p><pre>def get_all_parent_nodes_wordnet(start_synset, include_node_end:bool) -&gt; list:<br>  parent_nodes = list(<br>    start_synset.closure(lambda synset: synset.hypernyms())<br>  )<br>  return [start_synset] + parent_nodes if include_node_end else parent_nodes</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/457/1*paqQCJVvOkccljSRQ8u6kQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/279/1*uKVMl8VtkAeRZjCbVF6LSw.png" /><figcaption>Example chain of parent nodes for the node/synset “dog.n.01”</figcaption></figure><p>However, what you will find when trying to embed the whole of WordNet (or any other large graph) will lead to massive vectors — revealing the problem with some of the techniques described above (i.e. they are limited by the size of the graph since the vector length / number of features is equivalent to the number of nodes in the graph and thus will grow as the network grows — which is not too useful for large or dynamically changing graphs)</p><p><strong>Encoding Nodes using Graph Features</strong></p><p>This method equates the vector features to graph features, such as circuits (sub-graphs). The below graph has three sub-graphs and so the resulting vector length can be as small as three values</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Q8HqcA540eJtTcgL" /></figure><p>There are a number of ways to detect these graph features automatically to make this approach scalable.</p><p><strong>Encoding using Artificial Neural Networks</strong></p><p>A common approach is to use artificial neural networks to automatically identify sub-graph features within their hidden layers (and hence they will not be interpretable). One of the simplest examples is DeepWalk, which is essentially Word2Vec for graphs (since sentences are like graphs of words).</p><pre>from gensim.models import Word2Vec</pre><pre>def create_vectors_from_random_walks(graph:Graph) -&gt; List[List[float]]:<br>  skipgram_model = Word2Vec()<br>  training_set_for_graph = generate_training_set(graph,30,40)<br>  skipgram_model.build_vocab(training_set_for_graph, progress_per=2)<br>  skipgram_model.train(<br>    training_set_for_graph,<br>    total_examples = skipgram_model.corpus_count,<br>    epochs=20,<br>    report_delay=1<br>  )<br>  return skipgram_model[list(map(str,graph.nodes))]</pre><p>To train a skipgram neural network model which underlies word2vec (and thus deepwalk), we need to generate some training “sentences”. This is done by creating paths (which are like sentences) from each node (like a seed word). Now we could take every possible path from every node! However, random sampling works just as well</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*C8c5LkJdYFhHEdPPeQiUdg.gif" /><figcaption>For each node 30 random paths are generated (to ensure a good range of variant paths are captured) using a random walk with a depth of 40 steps (The deeper the walk, the more chance of capturing less localised features)</figcaption></figure><pre>from random import choice</pre><pre>def random_walk(node:int, steps:int, graph:Graph) -&gt; List[str]:<br>  path = []<br>  for _ in range(steps):<br>    path.append(str(node))<br>    adjacent_nodes = list(graph[node])<br>    node = choice(adjacent_nodes)<br>  return path</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/451/1*PeC7PyTS4luCmXOltU6_3A.png" /><figcaption>generated paths from random-walks</figcaption></figure><pre>def generate_training_set(graph:Graph, random_walk_repeats:int, random_walk_path_length:int) -&gt; List[List[str]]:<br>  return [<br>    random_walk(<br>      node = node,<br>      steps = random_walk_path_length,<br>      graph = graph<br>    ) for node in graph.nodes for _ in range(random_walk_repeats)<br>  ]</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*I_x65IUlUiKSH8bA" /><figcaption>Once the training sentences are generated, the skipgram model is trained and the embeddings obtained from its hidden layer</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/445/1*muTGP_5WenWsv3qiLtkH2g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/410/1*lKErGIz7Efr33pCAQERHRw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/458/1*K6A7Mzd5Ph_pGFD5VPhGtQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/385/1*iIJHB22KPiFeMqb-tSH9oQ.png" /></figure><p>You may have noticed from the random walks that the same node is often visited multiple times and that the random exploratory nature of the search is not too effective at building an effective picture of the graph. Node2Vec slightly modifies the random-walk using weights to better control the depth-first / breadth-first aspect of the search / walk.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*GT9oSo0agIeIj6nTg3jFEA.gif" /><figcaption>Depth-first-search (DFS) vs Breadth-first search (BFS)</figcaption></figure><pre>from node2vec import Node2Vec</pre><pre>def create_vectors_node2vec(graph:Graph) -&gt; List[List[float]]:<br>  node2vec = Node2Vec(graph, dimensions=64, walk_length=30, num_walks=200, workers=4)<br>  model = node2vec.fit(window=10, min_count=1, batch_words=4)<br>  return model[list(map(str,graph.nodes))]</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/445/1*muTGP_5WenWsv3qiLtkH2g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/387/1*89M_dO5-1oDPnbgATkYDKA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/458/1*K6A7Mzd5Ph_pGFD5VPhGtQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/384/1*nY8AlPe_r1pxyLfbaIDJpw.png" /></figure><p>You may have noticed that the approximated feature vectors produced by hidden layers in approaches like deepwalk , node2vec , etc, do not produce as accurate embeddings as the simpler, more exhaustive methods shown earlier. This is true, however, the slight trade-off in accuracy allows for fixed-size vectors that are invariant to the size or complexity of the graph being embedded. Thus the approach can scale to almost any graph.</p><h3><strong>Embedding WordNet</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/613/1*q74Imdadyvi7-05c2eCoBA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/939/1*-bfq5eymkaN9y5emUKw3rA.png" /><figcaption>Sections of the WordNet Graph</figcaption></figure><p>Let’s demonstrate how to embed a real-world graph — WordNet — (using the deepwalk method shown above):</p><pre>import nltk<br>nltk.download(&#39;wordnet&#39;)<br>from nltk.corpus import wordnet</pre><pre>def get_all_related_synsets(synset) -&gt; list:<br>  return synset.hypernyms() + synset.hyponyms() + synset.part_meronyms() + synset.substance_meronyms() + synset.member_meronyms() + synset.part_holonyms() + synset.substance_holonyms() + synset.member_holonyms() + synset.topic_domains() + synset.region_domains() + synset.usage_domains() + synset.entailments() + synset.causes() + synset.also_sees() + synset.verb_groups() + synset.similar_tos()</pre><p>We slightly modify our previous random walk function to make it compatible with the wordnet graph.</p><pre>def random_walk_wordnet(synset,steps:int) -&gt; List[str]:<br>  path = []<br>  for _ in range(steps):<br>    path.append(synset.name())<br>    adjacent_nodes = get_all_related_synsets(synset)<br>    if adjacent_nodes:<br>      synset = choice(adjacent_nodes)<br>    else:<br>      break<br>  return path</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/282/1*ArUsULsU2HhTkPOgWEYm4g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/234/1*mD_A2A944tZTcGMcSOFziQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/184/1*Q6D8Jeu1fWRWAnmawg7LNA.png" /><figcaption>3 x random walks of path length 10 starting at the synset/node “dog”</figcaption></figure><p>Now we are ready to generate our training data from wordnet to train a skipgram model as before.</p><pre>def generate_training_set_wordnet(random_walk_repeats:int, random_walk_path_length:int) -&gt; List[List[str]]:<br>  return [<br>    random_walk_wordnet(<br>      synset = synset,<br>      steps = random_walk_path_length,<br>    ) for synset in wordnet.all_synsets() for _ in range(random_walk_repeats)<br>  ]</pre><p>We then train the skipgram model as before and extract the hidden layers:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/682/1*o2h4Dt9bZXDJ6vsfvQfzpQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/636/1*CFDrIM-Mmcj1t4TkR4T-UQ.png" /><figcaption>example vector for the word “cat”</figcaption></figure><p>And et Voila — we now have WordNet vectors.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/417/1*WA1XCgAQIU-VyKgTNRm60w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/410/1*uUGEyuX7AAnvuASEVia0gA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/417/1*2OxOaXXgmlJkAIo4W2Z4Nw.png" /><figcaption>Plotting the first 300, 30 and 12 embedded wordnet synsets respectively</figcaption></figure><h3>Using the Embedded WordNet in a Downstream Machine Learning task</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/439/1*vOKca9psFR8Q7UAbrdhOHA.png" /><figcaption>Comparing our WordNet vectors for the following example words [“cat”, “dog”, “puppy”, “robot”, “ai”, “machine”,”nnaskans”]</figcaption></figure><p>We can now use these vectors as inputs to a wide range of probabilistic machine learning models (which require factorised representations as inputs). To demonstrate this we train a binary classifier (Support Vector Machine) to identify a word as an “animal” or “not_animal”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/237/1*cgeYvtkhvvv66Y4EzS2KmQ.png" /><figcaption>we use as few as 8 training examples for our “animal word classifier” (just to give you the idea)</figcaption></figure><pre>from sklearn.svm import SVC<br>svm_classifier = SVC()<br>svm_classifier.fit(training_inputs, training_output_labels)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/599/0*YZ5Fyk3-dyUQOOgG.jpg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/363/1*psFF6l7ij-6Jxgw_yt1xIw.png" /><figcaption>Our Animal Classifier trained on our very own WordNet Embeddings</figcaption></figure><h3>Conclusion</h3><p>In this blog post, our aim was to give an overview to some embedding principals for structured knowledge representations for use in combination with deep learning algorithms. The need for such neural-symbolic AI hybrids stem from deeper limitations of neural networks alone, e.g. its relatively easy to do common-sense reasoning on a graph, but less-so for a neural network.</p><p>Structured (and interpretable) knowledge representations (as well as reasoning) are the frontiers of current AI systems. We believe that Neuralsymbolic AI is a way forward that will enable and empower the next generation of AI, where we agree with <a href="https://medium.com/u/412dfd9a98f8">David Cox</a> from IBM Research on his Lecture on <a href="https://www.youtube.com/watch?v=4PuuziOgSU4">Neuralsymbolic AI</a> at MIT (co-organised by <a href="https://medium.com/u/8ab242de94e0">Ava Soleimany</a> ) about the potential prospects…</p><h3>Try it for yourself</h3><p>You can find code snippets for the above in the “graph-embeddings” repo on our <a href="https://github.com/Wluper"><strong>Github page</strong></a>.</p><h3>What’s Next</h3><p>Join us next time as we look at some of the more sophisticated approaches for embedding graphs, such as <strong>Graph Neural Networks</strong> and <strong>Graph Convolutional Networks</strong>.</p><p>Any comments or questions? Feel free to reach out to us.</p><p>Don’t forget to follow us on <a href="https://twitter.com/wluper_">Twitter </a>and <a href="https://www.linkedin.com/company/wluper_">LinkedIn</a>.</p><h3>References</h3><ul><li><a href="https://towardsdatascience.com/graph-embeddings-the-summary-cc6075aba007">https://towardsdatascience.com/graph-embeddings-the-summary-cc6075aba007</a></li><li><a href="https://towardsdatascience.com/overview-of-deep-learning-on-graph-embeddings-4305c10ad4a4">https://towardsdatascience.com/overview-of-deep-learning-on-graph-embeddings-4305c10ad4a4</a></li><li><a href="https://towardsdatascience.com/a-gentle-introduction-to-graph-neural-network-basics-deepwalk-and-graphsage-db5d540d50b3">https://towardsdatascience.com/a-gentle-introduction-to-graph-neural-network-basics-deepwalk-and-graphsage-db5d540d50b3</a></li><li><a href="https://github.com/palash1992/GEM">https://github.com/palash1992/GEM</a></li><li><a href="https://arxiv.org/abs/1705.02801">https://arxiv.org/abs/1705.02801</a></li><li><a href="https://www-cs.stanford.edu/people/jure/pubs/graphrepresentation-ieee17.pdf">https://www-cs.stanford.edu/people/jure/pubs/graphrepresentation-ieee17.pdf</a></li><li><a href="https://arxiv.org/pdf/1905.13192.pdf">https://arxiv.org/pdf/1905.13192.pdf</a></li><li><a href="https://tkipf.github.io/graph-convolutional-networks/">https://tkipf.github.io/graph-convolutional-networks/</a></li><li><a href="http://jmlr.csail.mit.edu/papers/volume16/dhillon15a/dhillon15a.pdf">http://jmlr.csail.mit.edu/papers/volume16/dhillon15a/dhillon15a.pdf</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=92dc3b6b6ec7" width="1" height="1" alt=""><hr><p><a href="https://medium.com/wluper/a-gentle-introduction-to-embedding-trees-and-graphs-with-code-92dc3b6b6ec7">A gentle introduction to Embedding Trees and Graphs (with code)</a> was originally published in <a href="https://medium.com/wluper">Wluper Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Methods to grow your own data sets for Conversational AI]]></title>
            <link>https://medium.com/wluper/methods-to-grow-your-own-data-sets-for-conversational-ai-1bfeac9508ef?source=rss-4e6913b61e2b------2</link>
            <guid isPermaLink="false">https://medium.com/p/1bfeac9508ef</guid>
            <category><![CDATA[dialogue]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[naturallanguageprocessing]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Mohammed Terry-Jack]]></dc:creator>
            <pubDate>Tue, 14 Jan 2020 19:49:18 GMT</pubDate>
            <atom:updated>2021-12-21T02:54:54.783Z</atom:updated>
            <content:encoded><![CDATA[<h4>Scaling from tiny to large amounts of NLP and Dialogue data</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aA6mA1p4Fq1Df0uHxuhnhg.png" /></figure><p><strong>Supervised Learning</strong> algorithms require a significant amount of <strong>labelled</strong> training examples to properly approximate a function that is robust to the richness and variation inherent in <strong>natural languages</strong>. Providing too few examples may produce a model which fails to <strong>generalise </strong>to underlying patterns, making it brittle and easily broken when exposed to unseen examples encountered in the wild.</p><p>However, collecting and annotating training data within a domain demands considerable time and resources. Fortunately, there are a range of high-quality <strong>augmentation </strong>techniques to artificially inflate textual datasets, including methods using state-of-the-art <strong>language models</strong> like <strong>BERT</strong> and <strong>GPT-2</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/896/1*IdLJIaaandrB_aR_2ZCnlg.jpeg" /><figcaption>Recent Language Models within NLP</figcaption></figure><h3>Contents</h3><p><strong>1. Generating Longer Conversations (using GPT-2)</strong></p><p><strong>2. Inserting Words (using BERT)</strong></p><p><strong>3. Back-Translation (aka Spinning)</strong></p><p><strong>4. Substituting Synonyms (with POS filtering)</strong></p><p><strong>5. Shifting</strong></p><h3>1. Generating Longer Conversations (using GPT-2)</h3><p>Given a tiny dataset of just three conversations:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*l1f76C4ISyqASBynGmUpEQ.png" /></figure><p>Open-AI’s massive GPT-2 language model was trained on so much data that it is able to generate very realistic sentences. We can use this fact to produce new variant examples by extending each conversation’s final sentence (e.g. <em>“i’m just in a bad mood” → “…because I lost in the qualifiers”</em>):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/973/1*-TSTqYsE1uVREbUW29p59A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/510/1*HVsYigxA3bAZzIL7LNschQ.png" /></figure><p>Similarly, we can use it to additional sentences to the end of the conversation too. In fact, each time you run this language model, you get slightly different results so you could re-run this augmentation method multiple times to introduce even more variant conversations into your dataset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*o61BsxP0LfZvnSsl86khJQ.png" /></figure><p>First we will need to install GPT-2-simple, an open-source library designed to make accessing this powerful language model very easy. We download the 774M parameter version of the model and load it up.</p><pre>!pip3 install gpt-2-simple</pre><pre>from nltk.tokenize import sent_tokenize<br>import gpt_2_simple as gpt2<br>model_name = &quot;774M&quot;<br>gpt2.download_gpt2(model_name=model_name)<br>gpt2.load_gpt2(<br>  gpt2.start_tf_sess(),<br>  model_name=model_name<br>)</pre><p>We create a small function which takes an example conversation as an input and calls GPT-2 to generate the next 100 words which it thinks could follow on from this conversation (we have chosen to extend the conversation by a single sentence [:n+1], but feel-free to modify this to extend the conversation more).</p><pre>def _extend_conversation(conversation_as_string):<br>  generated_samples = gpt2.generate(<br>    sess,<br>    model_name=model_name,<br>    prefix=conversation_as_string,<br>    length=100,<br>    return_as_list = True<br>  )<br>  n = len(<br>    sent_tokenize(<br>      conversation_as_string<br>    )<br>  )<br>  return sent_tokenize(<br>    generated_samples[0]<br>  )[:n+1]</pre><h3>2. Inserting Words (using BERT)</h3><p>Google’s BERT is another powerful language model which has revolutionised NLP but it is trained slightly differently to other language models, like GPT-2. This difference makes it well suited for predicting <em>masked </em>words; wherein a word is masked (hidden) and BERT uses the words surrounding that masked word to predict what the masked word could be. E.g. <em>“One day she [MASK] down the hall” → “One day she </em><strong><em>ran</em></strong><em> down the hall”</em></p><p>We inserted masks between words in a complete sentence and <strong>could trick BERT</strong> into predicting new words and extending the sentence from the middle (as opposed to the end). E.g. <em>“the fox” → “the [MASK] fox” → “the </em><strong><em>brown</em></strong><em> fox” → “the [MASK] brown fox” → “the </em><strong><em>striped</em></strong><em> brown fox” …</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/612/1*JYIST0WiAO1tPlLDPfn5fg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/666/1*7erDES6JoENod4tt_ieY0A.png" /></figure><p>Therefore, we use BERT by iteratively placing a mask between every word in a given conversation to produce multiple variant conversations. We could even iterate this process to produce even more variations by inserting masks into the newly created variant conversations too.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3ShLXipFfv49F1WL-QOIPg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8M77yb3_qdv3fC3ZF408Jw.png" /><figcaption>variant conversations produced by BERT</figcaption></figure><p>To do this method, we must first install the open-source library, pytorch-pretrained-bert, and download the language model along with its accompanying tokeniser.</p><pre>!pip3 install -U pytorch-pretrained-bert<br>from pytorch_pretrained_bert import BertTokenizer, BertModel, BertForMaskedLM<br>import torch</pre><pre>model_name = &#39;bert-base-uncased&#39;<br>bert_tokeniser = BertTokenizer.from_pretrained(model_name)<br>bert_model = BertForMaskedLM.from_pretrained(model_name)</pre><p>We also need to create a function which appropriately formats the input string with special tokens ([CLS], [SEP]), splits the string into tokens (using the model’s accompanying tokeniser) and then inserts a special mask token ([MASK]) to indicate the word we wish the model to predict.</p><pre>def _format_model_input(text, tokeniser, insert_mask_at_idx):<br>  tokens = tokeniser.tokenize(<br>    f&quot;[CLS] {text} [SEP]&quot;<br>  )<br>  tokens_with_mask = tokens[:insert_mask_at_idx] + [<br>    &quot;[MASK]&quot;<br>  ] + tokens[insert_mask_at_idx:]<br>  return torch.tensor(<br>    [<br>      tokeniser.convert_tokens_to_ids(tokens_with_mask)<br>    ]<br>  )</pre><p>For the output of the model, we want to return the sentence with an additional word inserted between two other words. Therefore, we create a function which converts the token indexes back into words (using the same tokeniser), and then fetch the token index predicted by the model corresponding to the location of the mask token (and convert it to a word in the same way). We then join the words together into a single string, clean it up a bit by removing any special tokenisation symbols (e.g. ## , [CLS] at the beginning, [SEP] at the end, etc).</p><pre>def _format_model_output(model_output, token_idxs, tokeniser, masked_idx):<br>  tokens = tokeniser.convert_ids_to_tokens(<br>    token_idxs.tolist()[0]<br>  )<br>  tokens[masked_idx] = tokeniser.convert_ids_to_tokens(<br>    [<br>      torch.argmax(<br>        model_output[0, masked_idx]<br>      ).item()<br>    ]<br>  )[0]<br>  return &#39; &#39;.join(tokens[1:-1]).replace(&quot;##&quot;,&quot;&quot;)</pre><p>We now define a function to connect everything together; the input formatting, the BERT model, the output formatting.</p><pre>def _insert_mask_and_predict(sentence, model, tokeniser, masked_idx):<br>  tokens_with_mask_inserted = _format_model_input(<br>    text = sentence,<br>    tokeniser = tokeniser,<br>    insert_mask_at_idx = masked_idx,<br>  )<br>  segment_ids = torch.tensor(<br>    [[0]*len(tokens_with_mask_inserted)]<br>  )<br>  with torch.no_grad():<br>    return _format_model_output(<br>      model_output = model(<br>        tokens_with_mask_inserted,<br>        segment_ids<br>      ),<br>      tokeniser = tokeniser,<br>      token_idxs = tokens_with_mask_inserted,<br>      masked_idx = masked_idx,<br>  )</pre><p>You may have noticed that the function requires you to specify which position you want to insert the mask. Well, why not each and every position in turn? This is what this next function does; iteratively creating variants by placing a mask at every index in the sentence until it reaches the end and fails, at which point it returns all the newly created variants.</p><pre>def _insert_words(example):<br>  new_examples = [example]<br>  idx = 1<br>  try:<br>    while True:<br>      new_examples.append(<br>        _insert_mask_and_predict(<br>          sentence = example,<br>          model = bert_model,<br>          tokeniser = bert_tokeniser,<br>          masked_idx = idx<br>        )<br>      )<br>      idx += 1<br>  except:<br>    new_examples.pop()<br>    return new_examples</pre><h3>3. Back-Translation (aka Spinning)</h3><p>Back translation (a.k.a. spinning) uses a machine translation model to translate a sentence into a foreign language and then to back again into the original language. We found this method very good at producing natural sounding sentences which are grammatically consistent yet slightly different from the original.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/870/1*3Nb1MdXiLh7AM7OBgTalKg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/538/1*xJ802zwbVnc1Evg_eYAf8A.png" /><figcaption>Spinning the text using Spanish (left) or Arabic (right) as the foreign language</figcaption></figure><p>First we need to import textblob, or any other open-source library with access to free translation.</p><pre>from textblob import TextBlob</pre><p>Next, we define a function which translates the sentence into some specified foreign language and back again into English (the assumed original language). This could also be extended to include translations into more than one foreign language before being translated back into English.</p><pre>def _spin_text(text, foreign_language):<br>  try:<br>    spun_text = _clean_word(<br>      TextBlob(<br>        TextBlob(text).translate(<br>          from_lang=&quot;en&quot;,<br>          to=foreign_language<br>        ).raw<br>      ).translate(<br>        from_lang=foreign_language,<br>        to=&quot;en&quot;<br>      ).raw<br>    )<br>    return spun_text if spun_text != _clean_word(text) else None<br>  except:<br>    return None</pre><p>If the translation failed, or the spun sentence turns out identical to the original (disregarding formatting or punctuation changes), then the function returns None.</p><pre>from string import punctuation</pre><pre>def _clean_word(word):<br>  return word.lower().strip(punctuation)</pre><h3>4. Substituting Synonyms (with POS filtering)</h3><p>A more classical technique is to pick a word in the sentence and substitute it for one of its synonyms. This method can produce a huge number of variations by substituting one word in the conversation at at a time (e.g. “I’m just in a bad mood” → “I’m simply in a bad mood”).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/585/1*XaMOOKgZcRPgT01O_ltb2A.png" /><figcaption>“hi …” -&gt; “how do you do …”</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/522/1*2J9eC0FzqYlroZbzfU55_Q.png" /><figcaption>“im just in a bad mood” → “im simply in a bad mood”</figcaption></figure><p>We can scan through a sentence and substitute each word with its synonyms to produce a variant sentence.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/722/1*ZjE8OLWMjXXojw84hIXLlQ.png" /></figure><p>In fact, substituting combinations of words will lead to an exponentially large pool of variants for any given example. However, this method needs careful filtering (we show you one filtering technique using POS tags below) since many of these variants will not sound natural if synonyms are substituted blindly. For instance, <em>“a bad mood” → “a unsound mood”</em> or <em>“nice to meet you Carla” → “nice to conform to you Carla”</em>, etc.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/574/1*be794U_LPxHG7lyxnlqsrg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QAeMxTMjWyiTijYqubxqkA.png" /></figure><p>We first download the relevant files from NLTK (an older yet giant NLP library).</p><pre>import nltk<br>nltk.download(&#39;wordnet&#39;)<br>nltk.download(&#39;punkt&#39;)<br>nltk.download(&#39;averaged_perceptron_tagger&#39;)<br>from nltk.corpus import wordnet as wn</pre><p>We define a function which fetches synonyms for a word from WordNet (a massive, hand-curated web of words and their various relations with one another)</p><pre>def synonyms(word, pos_tag):<br>  return list(<br>    {<br>      lemma.replace(&quot;_&quot;,&quot; &quot;).replace(&quot;-&quot;,&quot; &quot;) for synset in wn.synsets(<br>        _clean_word(word),<br>        pos_tag,<br>      ) for lemma in synset.lemma_names()<br>    }<br>  )</pre><p>We have added a filter here which takes into account the word’s part-of-speech before fetching any synonyms (i.e. is it a noun, verb, adjective, adverb, etc). E.g. The word “<em>test</em>” can be used as a noun or verb, etc:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/202/1*5Yhkx4B-EwdN2Ktu1fqFLg.png" /><figcaption>synonyms for the Noun “test”</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/633/1*8XaEyOBnNqhKd1sQo7jCog.png" /><figcaption>synonyms for the Verb “test”</figcaption></figure><p>We then define another function to automatically infer the part-of-speech tag:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/497/1*TsaxhFFDJn1KiILev9Cafw.png" /></figure><p>Fortunately, NLTK has some pre-trained POS Taggers which considerably simplifies our lives:</p><pre>def _infer_pos_tags(tokens):<br>  return [<br>    (<br>      token,<br>      _convert_nltk_to_wordnet_tag(nltk_tag)<br>    ) for token,nltk_tag in nltk.pos_tag(tokens)<br>  ]</pre><p>This function takes in tokens, as opposed to a string, so be sure to split the string into words or use one of NLTK’s provided tokenisers (<em>nltk.word_tokenize(some_string_to_be_tokenised)</em>). The function outputs the resulting POS tags returned by the tagger but first converts them into the POS notation required for compatibility with WordNet:</p><pre>def _convert_nltk_to_wordnet_tag(pos_tag):<br>  if pos_tag.startswith(&quot;N&quot;):<br>    return wn.NOUN<br>  if pos_tag.startswith(&quot;V&quot;):<br>    return wn.VERB<br>  if pos_tag.startswith(&quot;R&quot;):<br>    return wn.ADV<br>  if pos_tag.startswith(&quot;J&quot;):<br>    return wn.ADJ</pre><h3>5. Shifting</h3><p>Our final technique is a very basic one which can be applied to any time-series data (like conversations). The position of each sentence (or data point) in the conversation (training example) is simply shifted (offset) by one place to produce a valid variant. E.g.<em> “hi”, “how are you”, “fine thanks”… → “how are you”, “fine thanks”…</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/697/1*68-V2dbqe6f6XZYum11vug.png" /></figure><p>You can also combine time-series samples (conversations) by appending them to each other, producing a “new”, longer example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/285/1*ik3ClM4n8EW1N1QBb8mulA.png" /></figure><p>Finally, if you wish to make your model robust to textual errors which can occur in real-world scenarios, you can create variants by inserting textual noise (e.g. random spelling mistakes, additions, deletions and word order changes, etc).</p><h3>Conclusion</h3><p>We at <a href="https://wluper.com/">Wluper </a>combined the above five techniques, added a little bit of magic sauce, and were able to achieve a 10,000x fold improvement in training data for actual conversations, without even exhausting the number of possible variants each of these five technique alone are able to offer. Powerful, yet efficient.</p><p>As well as state-of-the-art language models, some of our unrevealed NLP augmentation methods involve multi-task learning, semi-supervised, and even unsupervised learning<strong>.</strong> One specific example would be Clustering, a great way to find natural variations for simple intents like greetings, requesting alternatives, etc. <br>We use clustering algorithms to analyse public data sets and discover groups of phrases which share some underlying semantic relationships. Although one will not be told explicitly which similarities these clusters share, they can be inferred using other techniques. Further, a seed phrase with a known class label can be used to solve such ambiguities.</p><p>We are constantly striving to find efficient and effective methods to grow our quality data sets and allow our Dialogue system to become better, more accurate, and more robust.</p><p>You can find code snippets for the above NLP and Dialogue data augmentation in the “dsag” repo on our <a href="https://github.com/Wluper"><strong>Github page</strong></a>.</p><p>You liked this post? Please leave some claps and feel free to share.</p><p>Any comments or questions? Let us know what you think.</p><p>Follow us also on <a href="https://twitter.com/wluper_">Twitter </a>and <a href="https://www.linkedin.com/company/wluper_">LinkedIn</a> for more.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1bfeac9508ef" width="1" height="1" alt=""><hr><p><a href="https://medium.com/wluper/methods-to-grow-your-own-data-sets-for-conversational-ai-1bfeac9508ef">Methods to grow your own data sets for Conversational AI</a> was originally published in <a href="https://medium.com/wluper">Wluper Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How do Dialogue Systems decide what to say or which actions to take?]]></title>
            <link>https://medium.com/wluper/how-do-dialogue-systems-decide-what-to-say-or-which-actions-to-take-b32ca223aff1?source=rss-4e6913b61e2b------2</link>
            <guid isPermaLink="false">https://medium.com/p/b32ca223aff1</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[conversational-ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Mohammed Terry-Jack]]></dc:creator>
            <pubDate>Thu, 17 Oct 2019 15:21:19 GMT</pubDate>
            <atom:updated>2021-12-21T02:55:07.213Z</atom:updated>
            <content:encoded><![CDATA[<h4>Continuation of our blog series on emerging Dialogue Management — Part 2: Dialogue Policies</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*g-DkZA1kVa57rIR_hUqULA.png" /></figure><p>Welcome back to our series on state-of-the-art research in Dialogue Management. <a href="https://medium.com/wluper/not-another-conversational-ai-report-4a094337e6f1"><strong>In the previous post we focused on various aspects related to State Tracking</strong></a><strong> </strong>(including how to take the personality of the user and system into account). In this post we look at the strengths and weaknesses of various types of <strong>Dialogue Policies</strong>.</p><h4>Part 1: State Tracking ✔️<br>Part 2: Dialogue Policies️ ◄ ︎️️you are here<br>Part 3: Common Sense<br>Part 4: Dynamic Memory<br>Part 5: Learning</h4><p>A dialogue policy decides how the system should respond given the current state. <a href="https://arxiv.org/pdf/1812.08989.pdf">XiaoIce</a>, <a href="https://rasa.com/docs/">Rasa</a> and various <a href="https://developer.amazon.com/alexaprize/challenges/past-challenges/2018">Alexa Prize</a> teams use a hybrid approach (i.e. a hierarchical ensemble of policies trained for different kinds of conversations — e.g. form filling, question-answering, story-telling, profanity, jokes, comforting, the weather, the news, recommending restaurants, films, music, sports, etc) for a diversity of conversation modes that could better maintain a user’s interest.</p><p>To reduce the computational load, a decision-making mechanism or two control which policy/policies are used at a given time (e.g. <a href="https://m.media-amazon.com/images/G/01/mobile-apps/dex/alexa/alexaprize/assets/pdf/2017/Alquist.pdf">Alqist</a> decides based on the detected topic). The use cases of policies usually do not overlap, however, if multiple policies are triggered simultaneously there is usually a method for selecting the winning policy (e.g. <a href="https://m.media-amazon.com/images/G/01/mobile-apps/dex/alexa/alexaprize/assets/pdf/2017/Alana.pdf">Alana</a> assigns static priorities to all policies while <a href="https://arxiv.org/pdf/1812.08989.pdf">XiaoIce</a> bases the decision on the confidence scores of each trigger).</p><p>There are several types of policies, each with its own strengths and weaknesses:</p><ol><li>Rule-based policies</li><li>Retrieval-based policies</li><li>Generative policies</li><li>Graph-based policies</li></ol><h4>1. Rule-based Policies</h4><p><strong>Weaknesses. </strong>Rule-based policies are one of the earliest and easiest approaches to conversational AI (e.g. ELIZA) involving a mountain of carefully hand-crafted responses and heuristics. This approach struggles with many-turn dialogues (the rules are brittle and easily broken by unexpected dialogues) and extremely time consuming to create and maintain (the rules do not scale well).</p><p><strong>Strengths. </strong>Apart from one or two chatbots (<a href="https://en.wikipedia.org/wiki/Mitsuku">Mitsuku</a>, <a href="https://www.rong-chang.com/tutor_mike.htm">Tutor</a> and other <a href="https://medium.com/@b.terryjack/chatbot-conversational-design-insights-132d52ef2b1d">Loebner Prize</a> winners), conversational AI systems of today only use rule-based policies to support other policies because they are computationally efficient and can save the system the expense of processing frequent queries or commonly cached utterances which require little thought. Heriot-Watt University’s <a href="https://m.media-amazon.com/images/G/01/mobile-apps/dex/alexa/alexaprize/assets/pdf/2017/Alana.pdf">Alana</a> [6] used a rule-based approach to script custom replies for common out-of-scope utterances (“<em>I love you”</em> or<em> “you are stupid”</em>) that were not easily handled by their other dialogue policies. <a href="https://m.media-amazon.com/images/G/01/mobile-apps/dex/alexa/alexaprize/assets/pdf/2018/Gunrock.pdf">GunRock</a> script multiple responses and selects one at random to make the predefined responses less repetitive. They further create dynamic template responses which can be customised by swapping out the value of its slots.</p><p><strong>Methods. </strong><a href="https://en.wikipedia.org/wiki/Mitsuku">Mitsuku</a>, <a href="https://www.rong-chang.com/tutor_mike.htm">Tutor</a> and Alexa Prize teams used AIML to create their rule-based policies. <a href="https://rasa.com/docs/rasa/core/policies/#keras-policy">Rasa</a> also offers heuristic Memoization and Mapping Policies [26] and <a href="https://deeppavlov.ai/">DeepPavlov</a> offers a similar Pattern Matching Skill.</p><h4>2. Retrieval-based Policies</h4><p><strong>Weaknesses. </strong>Retrieval-based policies take a search-oriented approach to conversations which assumes dialogues are a sequence of search queries (initiated by the user) and search results (provided by the system). Naturally, this type of approach favours task-oriented dialogues like FAQ answering, Open Domain Question Answering (which is the task of finding an exact answer in Wikipedia articles to any question asked), and eCommerce (i.e. user’s utterance are treated as search queries and task actions become the search results) which are each implemented in <a href="https://deeppavlov.ai/">DeepPavlov</a>. Seeing dialogues in this way, however, leads to conversations that are very direct, dull and user driven (i.e. the system merely reacts to what the user says as opposed to proactively initiating ideas and topics for a more natural and engaging dialogue).</p><p><strong>Strengths. </strong>Retrieval-based policies guarantee high-quality responses that are coherent and well formed (since they are retrieved from human-generated conversations or texts) <a href="https://m.media-amazon.com/images/G/01/mobile-apps/dex/alexa/alexaprize/assets/pdf/2018/Alana.pdf">Alana</a> scrapes text snippets from humorous subreddits like “ShowerThoughts” and “Today I Learned”. Other AlexaPrize competitors makes use of various datasets, including News API, EVI, Wikidata, IMDB, ESPN Washington Post, DuckDuckGo, Rotten Tomatoes, Spotify, Bing, Common Alexa Prize Chats (CAPC), online forums, social media (e.g. Twitter), movie subtitles (e.g. Cornell Movie Dialogs, OpenSubtitles), Jabberwacky chatbot chat logs, CNN chat show transcripts, etc. <a href="https://arxiv.org/pdf/1902.00098.pdf">ConvAI</a> makes mention of PersonaChat, DailyDialog and Reddit (<a href="http://files.pushshift.io/reddit/comments"><em>files.pushshift.io/reddit/comments</em></a>). [29] provide a very comprehensive list of datasets.</p><p>Furthermore, the quality of the responses can easily be controlled by filtering and curating the data. Alexa Prize teams filtered out inappropriate content (e.g. sexual language, profanity and other toxic phrases) and <a href="https://m.media-amazon.com/images/G/01/mobile-apps/dex/alexa/alexaprize/assets/pdf/2017/Alana.pdf">Alana</a> [11] noticed users give low ratings if their system reports news with strong political opinions, so they curate their data to ensure their content is politically neutral. <a href="https://arxiv.org/pdf/1812.08989.pdf">XiaoIce</a> filter personally identifiable information and spelling mistakes and curate their content to retain only empathetic utterances that fit XiaoIce’s personality.</p><p><strong>Method. </strong>Using a specified distance measure (e.g. cosine similarity), the distance from the query (e.g. the system’s current state) to each candidate response (e.g. possible system actions, answers, replies, etc) is computed and the closest is retrieved as the result.</p><p><strong>Pre-Trained Embeddings. </strong>There are various ways to embed the search queries and responses into a shared vector space so that they become mathematically comparable. <a href="https://deeppavlov.ai/">DeepPavlov</a> offer Tf-Idf and BLEU ranking retrieval methods while most Alexa Prize teams prefer pre-trained word or sentence embeddings (such as those detailed in section 1.2).</p><p><strong>Training Custom Embeddings. </strong>More sophisticated approaches involve jointly learning similar vector representations for dialogue states (the query) and system actions (the replies) in a supervised setting. <a href="https://deeppavlov.ai/">DeepPavlov</a> offer Siamese Neural Networks for this task while <a href="https://rasa.com/docs/rasa/core/policies/#keras-policy">Rasa</a> offers the Keras Policy [26] (a bi-LSTM with attention). Rasa also has the Embedding Policy which uses a newer embedding algorithm (<a href="https://arxiv.org/pdf/1811.11707.pdf">Recurrent Embedding Dialogue Policy (REDP)</a> [27]) which is specifically designed to handle adversarial interjections (i.e. users talking off topic during the completion of a task). It does this by attending over the dialogue history to learn which user utterances and system actions are important for deciding the next action. System actions are initially represented as bags of features (e.g. the class’ hierarchy, the tokens derived from the action’s name / label — e.g. <em>“utter_explain_details_hotel → {utter, explain, details, hotel}”</em>, the functions executed by this action, etc) such that similar actions have more features in common than dissimilar actions. However, they are then transformed by the network’s attended embedding layer to produce the final representation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/770/0*0-auhqIaWtfmwlzZ" /></figure><p><strong>End-to-End Models. </strong>There are also retrieval-based models which are end-to-end differentiable (i.e. the model can learn to output text responses or action functions without requiring an explicit representation for the dialogue state and other internal components). <a href="https://deeppavlov.ai/">DeepPavlov</a> offer two; the Key-Value Retrieval Network and the Hybrid Code Network (HCN) which combines an RNN with domain-specific knowledge and system action templates. <a href="https://m.media-amazon.com/images/G/01/mobile-apps/dex/alexa/alexaprize/assets/pdf/2017/Alquist.pdf">Alqist</a> also experiment with HCNs in the Alexa Prize while <a href="https://m.media-amazon.com/images/G/01/mobile-apps/dex/alexa/alexaprize/assets/pdf/2017/Rubystar.pdf">Ruby Star</a> uses Dynamic Memory Networks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/911/0*_E6wJToam-rTSat5" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/948/0*7sG1xRl5SRW0__Hg" /></figure><p><a href="https://arxiv.org/pdf/1902.00098.pdf">LittleBaby</a> uses the Profile-Encoded Multi-Turn Response Selection via Multi-Grained Deep Match Network while <a href="https://arxiv.org/pdf/1902.00098.pdf">ConvAI</a> organisers suggest the “Retrieve and Refine” model [31].</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ZOHsPgDcqdwoWt3B" /></figure><h4>3. Generative Policies</h4><p><strong>Weaknesses. </strong>Whereas retrieval-based policies guarantee coherent responses, they are but bounded imitations of the natural variety found in human-human dialogues. Generative policies, on the other hand, compose natural sentences on the fly for a broad range of topics (e.g. books, games, holidays, movies, music, news, sports, general knowledge, etc) without needing an external knowledge base (ungrounded). Despite the rapid improvements of end-to-end generative models (e.g. GPT-2), they are still not favoured over more reliable, retrieval-based methods because they are too often utter nonsensical and informationally inconsistent sentences.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/899/0*iIUol0eOFtwKmlzR" /></figure><p><strong>Strengths. </strong>Although generative policies cannot always guarantee high-quality responses, their robustness and high coverage nicely complement retrieval-based policies. For instance, if conversations drift into new or previously unseen topics and the retrieved candidate responses become too sparse or irrelevant, a generative policy would kick and compose additional candidates for inclusion into the rank and retrieval pool. This is how XiaoIce’s retrieval-based and neural-model-based generators work together.</p><p><strong>Methods. </strong>In the Alexa Prize, <a href="https://arxiv.org/abs/1709.02349">MilaBot</a> experimented with a combination of neural architectures including Hierarchical Latent Variable Encoder-Decoder (VHRED), a Dual Encoder and Skip Thought Network. <a href="https://arxiv.org/pdf/1812.08989.pdf">XiaoIce</a> follows a popular sequence-to-sequence GRU architecture for its generative policy while ConvAI recommend a similar baseline model (LSTM-based attentive sequence-to-sequence model). Teams that performed highly in the <a href="https://arxiv.org/pdf/1902.00098.pdf">ConvAI</a> competition implement variations of the Transformer for their generative policies (<a href="https://github.com/atselousov/transformer_chatbot">Lost In Conversation</a> modified the OpenAI GPT transformer architecture while <a href="https://github.com/huggingface/transfer-learning-conv-ai">Hugging Face</a> fine-tuned the BERT transformer architecture).</p><h4>4. Knowledge Graph based Policies</h4><p>Retrieval-based and generative policies both share a common problem: they produce dull dialogues. The Alexa Prize notes that retrieval-based policies produce responses which are on topic but not necessarily interesting. <a href="https://m.media-amazon.com/images/G/01/mobile-apps/dex/alexa/alexaprize/assets/pdf/2018/Slugbot.pdf">SlugBot</a> astutely observes that they are too reactive to form engaging conversations: <em>“it will not be possible to carry on a 20 minute conversation if SlugBot is simply responding to user initiatives”. </em>Generative policies are better at generalising to the unseen but they are still not engaging. <a href="https://arxiv.org/pdf/1812.08989.pdf">XiaoIce</a> mentions its generative policy <em>“often generates well-formed but short responses” </em>while even generative models like <a href="https://github.com/huggingface/transfer-learning-conv-ai">Hugging Face </a>(ConvAI runner-up) are a<em> “bit boring”</em> and <a href="https://github.com/atselousov/transformer_chatbot">Lost In Conversation</a> (ConvAI winner) are <em>“not that bad, just really uninteresting” </em>and <em>“not super interesting, but it’s able to respond well”. </em>Knowledge Graph-based policies are a novel approach incorporated into <a href="https://arxiv.org/pdf/1812.08989.pdf">XiaoIce</a> and at least two Alexa Prize teams to complement their generative and retrieval-based policies for a more stimulating and engaging discourse. KG-based policies give a system the ability to take the initiative during conversations and proactively add information and steer a conversational into new yet related topics.</p><p>Many teams activate their KG-based policy when the measured “dullness” of their response passes a specified threshold. Alana v1 [11] used an word2vec embedding of the phrase <em>“I don’t know”</em> as a representation of a dull reply and measured the similarity of sentences to it to calculate their dullness score (i.e. more similar sentences are more dull). Another Alexa Prize team trained a classifier (a bi-LSTM with fully connected layer to merge the encoder states) to rate retrieved responses as “appropriate”, “inappropriate” or “potentially appropriate”. XiaoIce’s topic switching mechanism is triggered if it has insufficient knowledge to talk about a topic in depth (i.e. its reply simply repeats the user inputs or contains no new information) or the user exhibits signs of boredom (bland user utterances like <em>“OK”, “I see”, “go on”,</em> etc).</p><p><strong>Methods. </strong>KG-based policies try to link extracted entities mentioned by the user (e.g. “Paris”) to a knowledge graph so that related entities (e.g. “Eiffel Tower”, “Louvre”, “Mona Lisa”, etc) guide toward a more informative reply. <a href="https://m.media-amazon.com/images/G/01/mobile-apps/dex/alexa/alexaprize/assets/pdf/2017/Alana.pdf">Alana’s</a> Ontology bot (Contextualised Linked Concept Generator [6] ) links entities to a Wikipedia graph, while other teams used Evi, FreeBase, Microsoft Concept Graph, etc. <a href="https://m.media-amazon.com/images/G/01/mobile-apps/dex/alexa/alexaprize/assets/pdf/2018/Slugbot.pdf">SlugBot’s</a> discourse relation dialogue model links extracted entities to its own domain ontology (UniSlug) to find additional entities related by time, contingency (one entity causally influences the other), comparison (including contrast and concession) or expansion (to continue to talk about the same topic but with a more specific attribute of the entity — e.g. “Paris” &gt; “Eiffel Tower”). <a href="https://arxiv.org/pdf/1812.08989.pdf">XiaoIce</a> also uses its own knowledge graph (a collection of head-relation-tail triples) however it then uses a custom model (boosted tree ranker) to rank the related entities according to its features (e.g. how related the entity is, has it been discussed yet, is it related to the user’s interests or persona information, is it related to the news and thus current, is it popular on the internet, is it often received highly by other users, etc).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/756/0*Fr_UXbkNoDZc1IXH" /></figure><p>In summary, there are many ways in which we can combine different types of Dialogue Policies to achieve a more balanced, fluent and well-rounded conversation.</p><h3>What’s Next</h3><p>Now that we learnt more about the various approaches in Dialogue Policies, we will next look at one of the most fascinating topics in Conversational AI: Common Sense — every intelligent assistant&#39;s ultimate “life” goal.</p><p>Please reply below if you have any comments or questions.</p><p>Don’t forget to follow us on <a href="https://twitter.com/wluper_">Twitter </a>and <a href="https://www.linkedin.com/company/wluper_">LinkedIn</a>.</p><h3>Appendix</h3><p>See the full list of references and links <a href="https://wluper.com/appendix">here</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b32ca223aff1" width="1" height="1" alt=""><hr><p><a href="https://medium.com/wluper/how-do-dialogue-systems-decide-what-to-say-or-which-actions-to-take-b32ca223aff1">How do Dialogue Systems decide what to say or which actions to take?</a> was originally published in <a href="https://medium.com/wluper">Wluper Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>