<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Anna Krogager on Medium]]></title>
        <description><![CDATA[Stories by Anna Krogager on Medium]]></description>
        <link>https://medium.com/@anna.krogager?source=rss-75b2f923f859------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*FUGHK0aYyQOIRObCY-7VpQ.jpeg</url>
            <title>Stories by Anna Krogager on Medium</title>
            <link>https://medium.com/@anna.krogager?source=rss-75b2f923f859------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 20:32:51 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@anna.krogager/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Neural text generation from 1 million Belgian real estate deeds]]></title>
            <link>https://medium.com/ml6team/neural-text-generation-from-1-million-belgian-real-estate-deeds-9230c940432c?source=rss-75b2f923f859------2</link>
            <guid isPermaLink="false">https://medium.com/p/9230c940432c</guid>
            <category><![CDATA[text-generation]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[nlp]]></category>
            <dc:creator><![CDATA[Anna Krogager]]></dc:creator>
            <pubDate>Mon, 07 Sep 2020 07:46:51 GMT</pubDate>
            <atom:updated>2020-09-07T07:46:51.475Z</atom:updated>
            <content:encoded><![CDATA[<h4>How we trained a large-scale keyword-to-text model for composing real estate deeds in Dutch</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*bzMH5beA1Hs66ICg" /></figure><h3>The basic idea</h3><p>Generating text with artificial neural networks, or neural text generation, has become very popular the last couple of years. Large-scale transformer language models such as <a href="https://openai.com/blog/openai-api/">OpenAI’s GPT-2/3</a> have made it possible to generate very convincing text that looks like it could have been written by a human. (If you haven’t tried it out yourself already, I highly recommend checking out the <a href="https://transformer.huggingface.co/doc/gpt2-large">Write With Transformer page</a>.)</p><p>While this causes a lot of concern about potentially misusing the technology, it also brings with it a lot of potential. Many creative applications have already been built using GPT-2, such as for example the text-based adventure game <a href="https://aidungeon.io/">AI Dungeon 2</a>. The key idea behind such applications is to fine-tune the language model on your own dataset, which teaches the model to generate text in line with your own specific domain. Make sure to check out <a href="https://blog.ml6.eu/gpt-2-artificial-intelligence-song-generator-lets-get-groovy-3e7c1f55030f">this blog post</a> about how my colleagues at ML6 used this approach for generating song lyrics.</p><p>However, it is often difficult to generate text of equal quality in languages different from English since no large pre-trained models are available in other languages. Training these large models from scratch is prohibitive for most people due to the extreme amounts of compute power necessary (as well as the need for a large enough dataset).</p><h3>Applying this idea</h3><p>At ML6 we wanted to experiment with large-scale Dutch text generation on a unique dataset. From our collaboration with <a href="https://www.fednot.be/nl_BE/home">Fednot</a> (the Royal Federation of Belgian Notaries), we created a large dataset consisting of 1 million anonymized Belgian real estate deeds, which Fednot kindly agreed to let us work on for this experiment.</p><p>The idea: train an autocomplete model that can ultimately be used as a writing tool for the notaries to assist in writing real estate deeds. In order to make the tool even more useful to the notaries we decided to spice up the model a bit by adding keywords as extra side input. This allows for steering the context of the text to be generated, as in the following example generated by our model:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/812/0*mthmRjZl_iO1FIh7" /><figcaption>Example output from our model. The special tokens &lt;s&gt; and &lt;/s&gt; denote the start and end of a paragraph. English translations are obtained with Google Translate.</figcaption></figure><p>In this blog post we will discuss how we trained our model. We will cover both the choice of model architecture and the data preprocessing, including how we extracted keywords for our training data. At the end we will show some results and discuss possible improvements and future directions.</p><h3>The model architecture</h3><p>Text generation can be phrased as a language modeling task: predicting the next word given the previous words. Recurrent neural networks (RNNs) used to be the architecture of choice for this task because of its sequential nature and success in practice.</p><h4>RNNs</h4><p>In an RNN language model, an input sequence of tokens (e.g. words or characters) is processed from left to right, token by token, and at each time step the model will try to predict the next token by outputting a probability distribution over the whole vocabulary. The token with the highest probability is the predicted next token. (For an in depth explanation of how RNNs work, check out Andrej Karpathy’s legendary blog post <a href="http://karpathy.github.io/2015/05/21/rnn-effectiveness/">“The Unreasonable Effectiveness of Recurrent Neural Networks”</a>.)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/902/0*ApWOm8EI6O2gnUME.jpeg" /><figcaption>An example RNN with character-level input tokens. Source: <a href="http://karpathy.github.io/2015/05/21/rnn-effectiveness/">Andej Karpathy</a>.</figcaption></figure><h4>Leveling up: transformers</h4><p>More recently, the transformer architecture from the paper <a href="https://arxiv.org/abs/1706.03762">Attention Is All You Need</a> (Vaswani et al) has taken over the NLP landscape due to its computational efficiency. In a transformer model there is no sequential computation but it instead relies on a self-attention mechanism that can be completely parallelized, thereby taking full advantage of modern accelerators such as GPUs. You can find a great explanation of the transformer architecture in Jay Alammar’s <a href="http://jalammar.github.io/illustrated-transformer/">“The Illustrated Transformer”.</a></p><p>Transformers are particularly successful in large scale settings, where you have several gigabytes of training data. An example of this is GPT-2, which was trained on 40GB of raw text from the internet at an estimated training cost of $256 per hour! However, RNNs (or its variant LSTM) remain competitive for language modeling and might still be better suited for smaller datasets.</p><h4>Our model</h4><p>Returning to our use case, we actually have enough data (around 17GB) to train a GPT-2 like model but chose to go with an LSTM-based architecture as the data is quite repetitive since many phrases are reused in several deeds so we figured that it would probably not be necessary to go for a full-fledged transformer architecture in order to obtain good results (also check out the recent movement towards simpler and more sustainable NLP, eg. the <a href="https://sites.google.com/view/sustainlp2020/home">SustaiNLP2020</a> workshop at EMNLP).</p><p>In the end we went for a 4 layer LSTM model with embedding size 400 and hidden size 1150, in line with the architecture in <a href="https://arxiv.org/abs/1708.02182">Merity et al</a>. We tie the weights of the input embedding layer to the output softmax layer, which has been shown to improve results for language modeling. Moreover, we add dropout regularization to the non-recurrent connections of the LSTM. (Adding dropout to the recurrent connections is not compatible with the CuDNN optimized version of the LSTM architecture, which is needed for training efficiently on a GPU.)</p><p>We use subword tokenization since it provides a good compromise between character-level and word-level input (see <a href="https://blog.floydhub.com/tokenization-nlp/">this blog post</a> for more explanation). More specifically, we train our own BPE tokenizer with a vocab size of 32k using the <a href="https://github.com/huggingface/tokenizers">Hugging Face tokenizers</a> library.</p><h4>Introducing keywords</h4><p>Our basic language model architecture is now in place but the next question is how to incorporate keywords as side input. For this we take inspiration from the machine translation literature, where typically an encoder-decoder model with attention is being used (originally introduced in <a href="https://arxiv.org/abs/1409.0473">Bahadanu et al</a>). Given a sentence in the source language the encoder first encodes each token into a vector representation and the decoder then learns to output the translated sentence token by token by “paying attention” to the encoder representations. In this way the decoder learns which parts of the input sentence are most important for the output at each time step. (Once again, Jay Alammar has a nice visual <a href="http://jalammar.github.io/visualizing-neural-machine-translation-mechanics-of-seq2seq-models-with-attention/">blog post</a> about this.)</p><p>In our case, we will “encode” our input keywords with a simple embedding lookup layer. Our “decoder” is the aforementioned LSTM model, with an extra attention layer added before the final softmax layer. This attention layer allows the language model to “pay attention” to the keywords before predicting the next token.</p><p>Our final model architecture looks as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Y2JNQHSDnwE3OKsbl4AXKg.png" /><figcaption>High level picture of our model architecture at time step <em>i.</em></figcaption></figure><h3>The data</h3><p>In order to train our model we need a dataset consisting of pieces of text paired with their corresponding keywords. Our raw dataset from Fednot actually comes from scanned PDF files so the first step is to convert these PDF files into (pieces of) text. Our full data pipeline looks as follows:</p><ol><li><strong>Clause detection</strong>: A deed is built up of clauses. For example there could be a clause describing the real estate property, another clause about the price, and so on. We start by training a clause detection model, which splits up a full deed PDF file into its building block clauses using computer vision.</li><li><strong>OCR</strong>: We convert each clause into text by using out-of-the-box optical character recognition. This results in somewhat noisy text, which we clean up a bit by filtering out clauses with the worst OCR mistakes.</li><li><strong>Pseudonymization</strong>: The clauses are pseudonymized by replacing all sensitive named entities, such as names, addresses and account numbers, with randomized entities. The advantage of this approach is that even if our model would ever miss an entity, an adversary would never be able to recognize which entities are real and which are randomized. We trained our own custom named entity recognition model using s<a href="https://spacy.io/">paCy</a>.</li><li><strong>Splitting into paragraphs</strong>: Full clauses can be very long and diverse, so we split up each clause into paragraphs that are easier to capture with keywords. Moreover, we filter out paragraphs that are either very short or very long because those paragraphs are more difficult to learn from. We only keep paragraphs that are between 8 and 256 tokens long.</li><li><strong>Keyword extraction</strong>: For each paragraph we extract all nouns, verbs and adjectives using spaCy. These are the keyword candidates. In some cases the paragraph starts with a short title, which we can detect heuristically. In these cases we include the title as a keyword since it contains very concise information on what the paragraph is about. We now have a (rather large and diverse) list of keywords per paragraph. At training time we sample a number of these keywords based on their TF-IDF weights so that rare words get sampled more frequently. The number of keywords to sample is also changed dynamically during training: it can be anywhere from 0 to 6 keywords per paragraph. This allows for a very diverse way of using the model since the model learns to handle both inputs without any keywords and inputs with up to 6 keywords.</li></ol><h3>Training time</h3><p>The model was trained for 10 epochs using the Adam optimizer with gradient clipping and a learning rate of 3e-4. We did not do a lot of hyperparameter tuning so these training settings could probably be improved further. Our loss curve looks reasonable:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/469/0*USZgUFeUb7d-4tPG" /><figcaption>Loss per epoch during training, visualized using Tensorboard. The green curve is the training loss and the grey curve the validation loss. Both are smoothed with smoothing parameter 0.6.</figcaption></figure><h3>Inference time</h3><p>It is finally time to take our newly trained model out for a spin but before doing so, let us first have a quick discussion on how to actually generate text from a language model.</p><p>Recall that a language model outputs a probability distribution over the whole vocabulary, capturing how likely each token is of being the next token.</p><h4>Greedy decoding</h4><p>The easiest way to generate text is to simply take the most likely token at each time step, also known as greedy decoding. However, greedy decoding (or its less greedy variant, beam search) is known to produce quite boring and repetitive text that often gets stuck in a loop, even for sophisticated models such as GPT-2.</p><h4>Pure sampling</h4><p>Another option is to sample the next token from the output probability distribution at each time step, which allows the model to generate more surprising and interesting text. Pure sampling, however, tends to produce text that is too surprising and incoherent, and while it is certainly more interesting than the text from greedy decoding, it often doesn’t make much sense.</p><h4>Best of both worlds</h4><p>Luckily there are also alternative sampling methods available that provide a good compromise between greedy decoding and pure sampling. Commonly used methods include temperature sampling and top-k sampling.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HSNbdFPgQRYcChEB7rNEkQ.png" /><figcaption>Example text generated from GPT-2 using beam search and pure sampling, respectively. Degenerate repetition is highlighted in blue while incoherent gibberish is highlighted in red. Source: Figure 1 from the paper <a href="https://arxiv.org/abs/1904.09751">“The Curious Case of Neural Text Degeneration”.</a></figcaption></figure><p>We chose to go with the more recently introduced sampling technique called nucleus sampling or top-p sampling, since it has been shown to produce the most natural and human-like text. It was introduced in the paper <a href="https://arxiv.org/abs/1904.09751">“The Curious Case of Neural Text Degeneration”</a> (Holtzman et al) from last year, which is a very nice and interesting read that includes a comparison of different sampling strategies.</p><p>In nucleus sampling the output probability distribution is truncated so that we only sample from the most likely tokens — the “nucleus” of the distribution. The nucleus is defined by a parameter p (usually between 0.95 and 0.99) which serves as a threshold: we only sample from the most likely tokens whose cumulative probability mass just exceeds p. This allows for diversity in the generated text while removing the unreliable tail of the distribution.</p><p>Let’s have a look at some examples from our model (we use nucleus sampling with p=0.95 in all examples):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*5E0y3TBqmgzHonNQ" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*n-_ni-qVgSS5i0Nx" /></figure><p>We see that the text is pretty coherent and the model has actually learned to take the keywords into account. Next, let’s try with some more rare keywords from our corpus such as “brandweer” (occurs in only 51 training examples) and “doorgang” (occurs in 114 training examples).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/810/0*H5AncSGehePQ5u__" /></figure><p>The text is now quite nonsensical and includes some made-up words (<em>gehuurgen</em>?) but still the model managed to get the context more or less right.</p><p>Let’s finish off with a couple of more examples using more keywords as input.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/809/0*yK-hRIU7ASBuvq4Z" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/815/0*jzV7kmBZ4lqHj8P9" /></figure><h3>Conclusion</h3><p>Our first results look promising, at least when using common keywords as input. For keywords that don’t occur very often in our training corpus, the model seems to get confused and generates poor quality text. In general, we also noticed artifacts of OCR mistakes and pseudonymization mistakes from our training data, which limits the quality of the generated text. There is still lots of potential for improving our results, for example by further cleaning up the training data and by tweaking or scaling up the model architecture.</p><p>One of the lessons learned from this experiment is that <strong>you don’t always need a huge transformer model in order to generate good quality text</strong> and that LSTMs can still be a viable alternative. The model architecture should be chosen according to your data. That being said it would be interesting to further scale up our approach, using more training data and a bigger model to analyse this trade-off.</p><p>Text generation is a fun topic within natural language processing but applying it to real-life use cases can be hard due to the lack of control of the output text. Our keyword-enriched autocomplete tool provides a way of controlling the output through the use of keywords. We hope that this will provide notaries with a useful writing tool for assisting in writing Dutch real estate deeds.</p><h3>About ML6</h3><p>We are a team of AI experts and the fastest growing AI company in Belgium. With offices in Ghent, Amsterdam, Berlin and London, we build and implement self learning systems across different sectors to help our clients operate more efficiently. We do this by staying on top of research, innovation and applying our expertise in practice. To find out more, please visit <a href="http://www.ml6.eu/">www.ml6.eu</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9230c940432c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/ml6team/neural-text-generation-from-1-million-belgian-real-estate-deeds-9230c940432c">Neural text generation from 1 million Belgian real estate deeds</a> was originally published in <a href="https://medium.com/ml6team">ML6team</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How ML6 built a multilingual COVID-19 assistant powered by NLP]]></title>
            <link>https://medium.com/ml6team/how-ml6-built-a-multilingual-covid-19-assistant-powered-by-nlp-9c75bc4ea0c1?source=rss-75b2f923f859------2</link>
            <guid isPermaLink="false">https://medium.com/p/9c75bc4ea0c1</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[coronavirus]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[software-architecture]]></category>
            <category><![CDATA[semantic-search]]></category>
            <dc:creator><![CDATA[Anna Krogager]]></dc:creator>
            <pubDate>Fri, 10 Apr 2020 11:44:59 GMT</pubDate>
            <atom:updated>2020-04-10T12:06:55.352Z</atom:updated>
            <content:encoded><![CDATA[<p>The COVID-19 pandemic is changing the world rapidly. Every day new guidelines or containment measures are announced and it is not easy to keep up with the latest updates. There is a wealth of information available on the internet but the information is scattered on many different websites, some more trustworthy than others, and finding the right answers can take a lot of time. This is especially the case for expats or immigrants who may not speak the native language of their country of residence.</p><p>Keeping people well-informed is an important step in flattening the curve. This is why we at ML6 decided to build a multilingual COVID-19 assistant powered by AI, more specifically Natural Language Processing (NLP). Our COVID-19 assistant makes it possible to ask any question in any of 16 languages and search through regulated (Belgian) FAQs for similar questions and answers. You can try it out for yourself at <a href="http://corona.ml6.eu/">corona.ml6.eu</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8kTqOQWhogiW9s3yvO8pkA.png" /></figure><p>In this blog post I will explain how I built the COVID-19 assistant in just 3 days. First we will have a quick look at how the FAQ database was scraped from multiple source URLs using the python library scrapy. Then we will dive into the part where the magic is happening: making the FAQ database searchable in a smart and multilingual way using state-of-the-art NLP techniques. Finally I will explain how I built an API around this and deployed it to Google Cloud Platform using Cloud Run. Since my own front end skills are very limited, my colleague Robin De Neef helped me out with the front end development and did an excellent job at making a nice user interface for querying the API. This will however not be covered in this blog post.</p><h3>Scraping the data</h3><p>In order to get started we first need a solid database of question-answer pairs to serve as the “knowledge base” of our COVID assistant. There are lots of FAQs available on the internet that we can simply scrape and combine into one database. To this end, we selected a small set of trustworthy URLs that are being kept up to date by official organizations (<a href="http://info-coronavirus.be/">info-coronavirus.be</a>, <a href="http://www.voka.be/">voka</a>, <a href="http://www.agoria.be/">agoria</a>, <a href="http://vrt.be/vrtnws">vrtnws</a>). Since I am working at the Ghent office of ML6, we focus on Belgian data sources that provide Belgium-specific information.</p><p>Web scraping can easily be done using the python library <a href="https://scrapy.org/">scrapy</a>. All you need to do is define a “spider” containing the URLs you want to scrape as well as the logic for parsing the underlying HTML. The following code snippet shows how this is done for one of our data sources:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/12ff62e46ea601fcb7eec0b7c6d43bb8/href">https://medium.com/media/12ff62e46ea601fcb7eec0b7c6d43bb8/href</a></iframe><p>The start_urls variable contains the URLs you want to scrape and the parse method contains the logic for extracting the question-answer pairs from the resulting HTML, and for saving the result to a file in our local file system.</p><p>The code for running the spider looks like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5ac13667163e729968145e01f1276c1e/href">https://medium.com/media/5ac13667163e729968145e01f1276c1e/href</a></iframe><p>After running the spider for all of our data sources we end up with around 500 question-answer pairs saved to local files in json format. Each of the json files contains a list with question-answer pairs along with their source URL.</p><h3>Semantic Search using NLP</h3><p>Having our FAQ database ready, the next thing we need is a way to make it searchable. This is where NLP comes into play. Traditionally, search engines are based on keyword search where you only get results matching the exact keyword that you typed in (a popular solution for this is <a href="https://www.elastic.co/">elasticsearch</a>). However, you are probably also interested in results containing synonyms for your input keyword or just any related word. For example, in our COVID-19 use case, if you search with the keyword “travel” you probably also want to see results for “abroad”, “flight”, “airplane”, “train”, etc. Searching in this way based on the meaning of a word instead of the exact word is known as semantic search and it is a common problem in NLP.</p><p>In modern NLP, text is usually represented as (high-dimensional) mathematical vectors, also called embeddings, that capture the semantics of the text. One of the most popular techniques for creating such vector representations is Word2Vec, developed by <a href="https://arxiv.org/abs/1310.4546">Thomas Mikolov in 2013</a>, where word embeddings are learned based on a huge corpus of raw text, for example all of Wikipedia. The idea is that words occurring in similar contexts should have similar meaning (i.e. similar embeddings). The resulting word embeddings from Word2Vec look something like this, when projected down to 2 dimensions:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/492/1*coyujClNdNN2_SvdUROXug.png" /></figure><p>Notice that words with similar meaning are located close to each other in the word embedding space. For this reason, word embeddings are very useful for semantic search: if we want to find words similar to “travel” we simply look for which word embeddings are lying closest to the word embedding of “travel”.¹</p><p>Word embeddings are great but we typically want to capture the meaning of a whole sentence, not just separate words. Luckily, there are lots of NLP models out there that do just that: compute sentence embeddings. The easiest way of obtaining a sentence embedding is to simply take the average of all word embeddings of the words in the sentence. This simple baseline works surprisingly well but we can do better. A recent trend in NLP is to learn contextualized embeddings, where the whole sentence (“context”) is taken into account before computing its embedding. This trend is motivated by the fact that the same word can have different meanings depending on the context. For example, the word “bank” has very different meanings in the following two sentences:</p><ul><li>Yesterday I went to the bank to withdraw some money.</li><li>Yesterday I went for a walk along the river bank.</li></ul><p><a href="https://research.google/pubs/pub46808/">The Universal Sentence Encoder</a> developed by Google is a state-of-the-art model for computing contextualized sentence embeddings. The model even has a multilingual version, where you can input text in any of 16 different languages and get a sentence embedding that captures the meaning, independent of the language. This is perfect for our semantic search use case: even though our scraped FAQ database is in Dutch, a user can search with a question in, say, English and still get good results.</p><p>To make matters even better the multilingual Universal Sentence Encoder is readily available at <a href="https://tfhub.dev/">TF Hub</a> and can be loaded in just one line of (python) code:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/26588adfb40357f53eb9d71da3b470e6/href">https://medium.com/media/26588adfb40357f53eb9d71da3b470e6/href</a></iframe><p>Using the model is as easy as passing a list of text to the model object that we just loaded:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d85b06b00908c43297c874b7efa9138a/href">https://medium.com/media/d85b06b00908c43297c874b7efa9138a/href</a></iframe><p>Voila, we now have a very easy way of computing state-of-the-art sentence embeddings that can be used for semantic search.</p><p>[1] Since word embeddings are high-dimensional (typically around 300 dimensions) the distance actually doesn’t behave very well. This is known as <a href="https://en.wikipedia.org/wiki/Curse_of_dimensionality">the curse of dimensionality</a>. For this reason, a different similarity measure is used instead of the distance, namely the so-called cosine similarity. Roughly speaking, the cosine similarity measures the angle between two vectors instead of their distance.</p><h3>Building the API</h3><p>Having our FAQ database and NLP model in place it is time to put the pieces together in an efficient API that can serve user requests. The API is implemented with the <a href="https://github.com/zalando/connexion">Connexion</a> framework of Zalando, which is a Swagger/OpenAPI first framework for Python on top of Flask.</p><p>The NLP model and our scraped FAQ database (located on Google Cloud Storage) are loaded into the API during startup, and we immediately compute sentence embeddings for all of the questions in our database, resulting in an “embedding space” that is used for semantic search. Note that we only embed the questions, not the answers, since we will only be searching for similar questions given a user input. Having an embedding space pre-computed during start up reduces latency when serving user requests. The start up process takes around 20 seconds in total (most of this time is spent loading the NLP model from TF Hub).</p><p>The API has a single route for serving user requests (GET /search), with query parameters to specify the user query, the amount of results to return, and the language of the resulting question-answer pairs. When a request comes in, the user query is embedded on the fly with our NLP model, the similarity score is calculated for all embeddings in our embedding space (FAQ database), and the top K question-answer pairs are returned. In case a language parameter was specified (with a non-Dutch value) we also use the Google Translate API to translate the resulting question-answer pairs before returning them. Note that since our embedding space is quite small (only around 500 embeddings) we can do the similarity search in a brute-force way, simply computing all similarity scores and taking the argmax. In case you have a bigger search space you might want to look into more efficient similarity search libraries such as <a href="https://github.com/facebookresearch/faiss">faiss</a>, <a href="https://github.com/spotify/annoy">annoy</a> or even <a href="https://www.elastic.co/">elasticsearch</a>. However, in our case a full API call (excluding Google Translate) only takes around 20ms so no need to look into more efficient solutions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/551/1*oWGfeK8iB8vAb5GrSBhq2Q.png" /></figure><h3>Deploying to GCP</h3><p>Now that we have a working API the final step of our project is to deploy it so that it is ready to serve client requests via a publicly accessible https URL. At ML6 we are a partner of Google Cloud so naturally we deploy all of our workloads to Google Cloud Platform (GCP).</p><p>GCP has many options for deploying APIs but given that we only have a single stateless container, I chose to go with <a href="https://cloud.google.com/run">Cloud Run</a>, which is a fully managed compute platform for deploying containerized applications. The nice thing about Cloud Run is that it handles all infrastructure management for you and automatically scales up and down depending on your traffic. It even scales down to 0 when your API is not receiving any traffic so that you don’t pay anything when your API is not being used.</p><p>To deploy our API with Cloud Run we first need to containerize it using <a href="https://www.docker.com/">docker</a>. This amounts to adding a dockerfile (make sure to include an entrypoint in the dockerfile so that Cloud Run knows how to run your code) and building the docker image with the docker build command.</p><p>Once we have built the docker image we push it to the google cloud container registry (eu.gcr.io/{project}/covid-assistant) where it can be picked up by Cloud Run. Now, deploying our API is as simple as running one command using the <a href="https://cloud.google.com/sdk/gcloud">gcloud command line tool</a>:</p><pre>gcloud run deploy covid-assistant \<br>--image eu.gcr.io/{project}/covid-assistant \<br>--platform managed</pre><p>After a couple of minutes the API is up and running. Cloud Run automatically generates a service URL which you can use to send requests to your API.</p><p>Now that the API is up and running only one question still remains: how do we keep the FAQ database up to date over time? We solve this by running an automatic job every morning which re-scrapes all of our FAQ sources, uploads the result to Google Cloud Storage and reloads these new files into the API. In order to achieve this, we deploy our web scraping code as a function with <a href="https://cloud.google.com/functions">Cloud Functions</a>, which can then be called periodically every morning using <a href="https://cloud.google.com/scheduler">Cloud Scheduler</a>. Cloud Scheduler is also used to call a /refresh endpoint of our API which reloads the FAQ database from cloud storage and re-computes the embedding space. Voila, our API is now automatically being kept up to date with a fixed schedule every morning.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/986/1*sHs2CV8GePl2bKCVXy-TZw.png" /></figure><h3>Conclusion</h3><p>We have now seen how publicly available tools can easily be leveraged to develop and deploy a production ready QA assistant. We described how to use scrapy to scrape your favorite FAQs from the internet and how to use the multilingual universal sentence encoder from TF Hub to perform embedding based semantic search within our FAQ database. We then explained how to wrap this functionality in an API and how to deploy it to GCP with Cloud Run, giving us efficient autoscaling for free. Finally, we saw how to automate the web scraping on a periodic basis using Cloud Functions and Cloud Scheduler.</p><p>The COVID-19 pandemic is having a big impact on all of our lives. We all need to stay home to help flatten the curve, in which communication is key to all members of our society. With our multilingual COVID-19 assistant we hope to help make information more accessible to a wide group of people, regardless of their native language. Go ahead and find the answer to all of your COVID-related questions at <a href="http://corona.ml6.eu/">corona.ml6.eu</a>.</p><h3>About ML6</h3><p>We are a team of ML experts composing the fastest growing AI company in Belgium. With offices in Ghent, Amsterdam, Berlin and London, we implement self learning systems across different sectors to improve and optimise our clients operations. We do this by staying on top of research, innovation and practical applications. Find out more at <a href="http://www.ml6.eu/">www.ml6.eu</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9c75bc4ea0c1" width="1" height="1" alt=""><hr><p><a href="https://medium.com/ml6team/how-ml6-built-a-multilingual-covid-19-assistant-powered-by-nlp-9c75bc4ea0c1">How ML6 built a multilingual COVID-19 assistant powered by NLP</a> was originally published in <a href="https://medium.com/ml6team">ML6team</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>