<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Kumar Shridhar on Medium]]></title>
        <description><![CDATA[Stories by Kumar Shridhar on Medium]]></description>
        <link>https://medium.com/@shridhar743?source=rss-c7ad13abfbb7------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*m5fhlOhPpSDnLc9jPMtxOg.jpeg</url>
            <title>Stories by Kumar Shridhar on Medium</title>
            <link>https://medium.com/@shridhar743?source=rss-c7ad13abfbb7------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 14:02:04 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@shridhar743/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Natural Language Understanding for Chatbots]]></title>
            <link>https://medium.com/neuralspace/natural-language-understanding-for-chatbots-2eb7a81b9390?source=rss-c7ad13abfbb7------2</link>
            <guid isPermaLink="false">https://medium.com/p/2eb7a81b9390</guid>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[naturallanguageprocessing]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[chatbots]]></category>
            <dc:creator><![CDATA[Kumar Shridhar]]></dc:creator>
            <pubDate>Sat, 26 Dec 2020 07:36:32 GMT</pubDate>
            <atom:updated>2020-12-26T07:39:06.493Z</atom:updated>
            <content:encoded><![CDATA[<h4>Natural language understanding (NLU) is a branch of artificial intelligence (AI) that deals with understanding human language.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*65eHnk03RWv1uVmp" /></figure><p><strong>The NLU is used to accomplish two main tasks: to identify the intent behind what a person is saying, and to generate a response based on the identified intent.</strong></p><p>In simple terms, the intent is an action or task that the user wants to accomplish. Based on the users’ needs, the NLU software provides a text or a voice response. In any case, it should be tailored to the user’s needs.</p><blockquote>If the user wants to “check” a movie’s rating, its response should be the movie’s rating (e.g. “The movie was rated as PG-13”).</blockquote><h3>Who is using NLU?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*tJu-OCSRfT4bZ6lj.jpg" /></figure><p>NLU software can be used in a variety of applications including customer service, sales, chatbots and virtual assistants, etc. Some NLU software solutions are designed to only identify the intent of the input while other software solutions are designed to also generate a response. Many companies are using NLU, e.g., <strong>Amazon</strong> uses it to help you complete purchases, <strong>Google</strong> uses it in its search engine and <strong>Netflix</strong> uses it to recommend movies or TV shows. NLU helps these companies provide a better user experience to their customers.</p><h3>How does NLU work?</h3><p>A naive NLU system takes a person’s speech or text as input, and tries to find the correct intent in its database. The database includes possible intents and corresponding responses that are prepared by the developer. The NLU system then compares the input with the sentences in the database and finds the best match and returns it.</p><p>NLU researchers and developers are trying to create a software that is capable of understanding language in the same way that humans understand it. <strong>While we have made major advancements in making machines understand context in natural language, we still have a long way to go.</strong></p><p><a href="https://chatbotsmagazine.com/how-close-are-chatbots-to-pass-turing-test-33f27b18305e">How close are chatbots to pass Turing test?</a></p><p>Some of the most prominent use of NLU is in chatbots and virtual assistants where NLU has gained recent success. These systems are designed to understand the intent of the users through text or speech input.</p><h3>What are chatbots and why do we need them?</h3><p>Chatbots are gaining popularity day by day. Not just the ones you see popping up on various websites, but there are many bots working to improve business use cases like helping improve supply chain processes. One example is Amazon’s Seller Messaging Assistant.</p><p><strong>However, it seems ridiculous to believe that any day bots will replace websites and apps. But, there are more than one reasons to believe so, be it ease of bots usage, no apps needed, no memory space needed, the same platform works for a lot of bots and so on.</strong></p><blockquote>The future of Web is definitely going to be bots and according to several tech reports, the bot internet traffic will be doubled by 2025.</blockquote><h4>Reasons why chatbots will replace websites and apps in near future</h4><ol><li><strong>Ease of Use</strong>: If we look at the basic points of chatbots, it’s hard to believe that it’s going to replace any website or app, but still there are good reasons to believe so. When we chat with anyone, the first thing we do is to understand if the person is a bot or a human, if it’s a bot, we don’t expect too much from it and we try to get the work done with the least possible effort, we try to keep the conversation as short as possible and we try to do it as fast as possible, so that we can move on to the next task. On the contrary, when we talk to human, we expect a lot from them, we try to keep the conversation long and we try to keep it interesting. This is the main difference between human and bots. Now, what happens when we visit a website, first impression matters a lot, it’s not like we’ll talk to the website or the app, but at least we expect a lot from it, like, we expect to get all the information we need at one place, as fast as possible, and we expect the website to be responsive. If the website fails to meet these expectations, we simply close it and try to find another website, it’s not that we talk to the website, but we talk to the service provider via website or app. If a website or app fails to keep us engaged and if it fails to provide a great user experience, we simply move on to another website or app.</li><li><strong>No app installation</strong>: When we talk about bots, they are not like normal applications, they don’t require any app installation, they don’t take any memory space and a lot of chatbots don’t need any Internet connection to work.</li><li><strong>Always available</strong>: Another important thing is that they are always available, and they never close, in short they are always there for you.</li></ol><h3>Components of a good chatbot</h3><p><a href="https://medium.com/@shridhar743/step-by-step-guide-to-make-a-dumb-bot-smart-32343334cd6c">Step by Step guide to make a dumb bot smart</a></p><p>Let’s take an example of a chatbot for a music streaming application. When you ask your music assist chatbot to “Play Coldplay”, you are implicitly asking it to do two things:</p><p><strong>1.Identify what is the intent behind the command (which in this case is to play something on a device).</strong></p><p><strong>2.Identify what to play (which is Coldplay in this case).</strong></p><p>Here the bot has to identify the intent of the user. This is called Intent Classification. Then, it has to figure out what is Coldplay. Is it an Artist, Song, Playlist, etc. This is called Entity Extraction, where Coldplay is an Entity.</p><h3>Intent Classification</h3><blockquote><strong>Intent classification is the process of classifying the customer’s intent by analysing the language they use.</strong></blockquote><p>For instance, a customer who types “where is the nearest pharmacy?” is probably looking for a pharmacy, while a customer who types “where is the nearest restaurant?” is probably looking for a restaurant.</p><h4>How to classify intents correctly?</h4><p>There are two main methodologies for intent classification: <strong>rule-based</strong> and <strong>statistical</strong>.</p><p><a href="https://medium.com/botsupply/rule-based-bots-vs-ai-bots-b60cdb786ffa">Rule based bots vs AI bots</a></p><p><strong>1.Rule-based intent classification</strong> is based on expert-defined rules. This technique is cheaper and faster to build, and is flexible enough to be customised, but requires a large amount of human effort to maintain.</p><p><strong>2. Statistical intent classification</strong> is based on Machine Learning algorithms. Statistical classification methods are faster to train, require less human effort to maintain, and are more accurate. However, they are more expensive and less flexible than rule-based classification.</p><h4><strong>How to do intent classification in a chatbot?</strong></h4><p><a href="https://chatbotslife.com/know-your-intent-sota-results-in-intent-classification-8e1ca47f364c">Know your Intent: State of the Art results in Intent Classification for Text</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/784/1*4HQBa-ePr3OyggJmH5TV9w.png" /></figure><p>The first step in building a chatbot is to define the intents it will handle. Intents can be modelled as a hierarchical tree, where the topmost nodes are the broadest or highest-level intents. The lowest level intents are self-explanatory<strong> </strong>and are more catered to the specific task that we want to achieve.</p><blockquote>For example a top level intent could be ‘ordering food’. The next level could be ‘ordering food of a specific cuisine’ At the last level, we will have specific dish names like ‘Chicken Biryani’.</blockquote><h4>Let’s see an example of intent classification in action:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/363/1*25rkpAu7xEV2G1dWS1W48A.gif" /></figure><h3>Entity Extraction</h3><blockquote>The process of extracting targeted information from a piece of text is called NER. E.g., person names, organizations, locations, medical codes, time expressions, quantities, monetary values, percentages, etc.</blockquote><p>Let’s revisit our previous example where we asked our music assist bot to “<em>play Coldplay</em>”. An intuitive understanding from the given command is that the intent is to play somethings and entity is what to play. When we say “<em>play Coldplay</em>”, a chatbot would classify the intent as “<em>play music</em>”, and classify Coldplay as an entity, which is an <em>Artist</em>.</p><h4>An entity is anything that exists in the real world, and can be a person, place, product, organization, or a concept.</h4><blockquote>Depending upon the application, there can be a large variety of entity types. For example, in news articles, entities could be people, places, companies, and organizations. In healthcare, entities could be diseases, drugs, and procedures. In the military, entities could be weapons, ships, and people.</blockquote><h3>How to extract entities from a given text?</h3><p>There are many ways in which we can extract the important information from text.</p><p>It can range from a simple solution like<strong> rule based string</strong> matching to an extremely complex solution like <strong>understanding the implicit context behind the sentence </strong>and then extracting the entity based on the context. E.g., “<em>Play Cricket</em>”, and “<em>Play Coldplay</em>”.</p><p>A <strong>simple string / pattern matching example</strong> is identifying the number plates of the cars in a particular country. Since the pattern is fixed, we can write a regular expression to extract the pattern correctly from the sentence.</p><p>A<strong> complex sentence</strong> can be understood by identifying the implicit context behind the sentence. The entities can be extracted using the information stored in the context. The entity can be any from the list of : <em>person, organization, place, event, product, service, time, quantity, etc</em> .</p><blockquote><strong>Example</strong>: <em>Apple announced the launch of their new device last Thursday that will compete with the likes of Samsung and Google. The device will be launched in India on the 11th of September .</em></blockquote><p>Here, last Thursday refers to the last Thursday from the current date. To extract this information, we can use the information available in the context. That is, the current date, the day before yesterday, the day before that, etc.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MvyTgA1Bl5VnRKMoZkRwEA.png" /><figcaption>Source: Displacy</figcaption></figure><h3>But why do we need Entity extraction?</h3><p>Entity Extraction is a very important task in the following situations:</p><p><strong>1.Information Retrieval</strong>: to fetch relevant information from a large collection of documents. Example: When we say ‘Play Coldplay’ we need to ensure that Coldplay exists in our database and then extract the information present.</p><p>2. <strong>Knowledge Discovery</strong>: to find information about entities and their relations between each other. Example: Identifying Google as ORG and Alphabet as a parent company. Based on this relation, we can infer Alphabet to be an ORG.</p><p>3. <strong>Information Filtering</strong>: to recommend relevant documents based on the identified entity. Example: We can recommend Maroon 5 based on the interest in Coldplay.</p><p>4. <strong>Question Answering</strong>: to answer questions about entities (who, what, when, where, why). Example: When was Google started, who are the founders, their business model, mode of operation and so on.</p><p>5. <strong>Security</strong>: to identify and track suspicious activities. Example: To identify entities not present in DB and to categorize them. Identifying non-members of an organization to prevent them from accessing confidential information.</p><h3>NLU will be required in every application that uses or contains text in natural language. From search engines or recommender systems to chatbots, NLU is being used everywhere. But, to use an NLU system efficiently, you need to understand what it can do and does it fit your use-case or not.</h3><p>In this blog we have discussed basics about NLU and main components of a simple chatbot. In the next blog, we will discuss the entire development life cycle of a chatbot.</p><h4>If you are interested in knowing how we are tackling the problem of low resource languages like Indian Languages, check out this blog:</h4><p><a href="https://medium.com/neuralspace/indic-transformers-an-analysis-of-transformer-language-models-for-indian-languages-c6b4db0643b">Indic Transformers: An Analysis of Transformer Language Models for Indian Languages</a></p><h3>Stay tuned to understand more about end-to-end NLU systems and how to choose the right one for your use-case.</h3><p>If you have any questions, feel free to comment or find me on <a href="https://twitter.com/JupyterAI">Twitter</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2eb7a81b9390" width="1" height="1" alt=""><hr><p><a href="https://medium.com/neuralspace/natural-language-understanding-for-chatbots-2eb7a81b9390">Natural Language Understanding for Chatbots</a> was originally published in <a href="https://medium.com/neuralspace">NeuralSpace</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Indic Transformers: An Analysis of Transformer Language Models for Indian Languages]]></title>
            <link>https://medium.com/neuralspace/indic-transformers-an-analysis-of-transformer-language-models-for-indian-languages-c6b4db0643b?source=rss-c7ad13abfbb7------2</link>
            <guid isPermaLink="false">https://medium.com/p/c6b4db0643b</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[india]]></category>
            <dc:creator><![CDATA[Kumar Shridhar]]></dc:creator>
            <pubDate>Fri, 06 Nov 2020 09:58:24 GMT</pubDate>
            <atom:updated>2020-11-06T09:58:24.980Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/691/0*DB5jkfjfR6HgD1RI.png" /></figure><p>This post is about <strong>our recent work </strong>focusing on application of various <strong>transformer-based architectures on Indian languages</strong>.</p><blockquote>Our <a href="https://arxiv.org/abs/2011.02323">paper</a> has been accepted at <a href="https://ml-retrospectives.github.io/neurips2020/"><strong>ML-RSA @ NeurIPS 2020</strong></a><strong>. </strong>🥳</blockquote><p>This blog begins with an overview of modern Natural Language Processing (NLP) and how NLP has evolved and progressed in the deep-learning era. We then move onto the premise of our work, the linguistic disparity in NLP research and highlight some relevant work in NLP for Indian languages. Finally, we delve into the details of our contributions, experimental setups and share some key insights from our research.</p><p>This work was done jointly by <a href="https://medium.com/u/de8a2bd94732">KUSHAL JAIN</a>, <a href="https://medium.com/u/57f955c90c95">Felix Laumann</a> and <a href="https://medium.com/u/cd445afd79a2">Ayushman Dash</a> and in collaboration with <a href="https://reverieinc.com/">Reverie Language Technologies.</a></p><h3>An Overview of Modern NLP</h3><p>Natural Language Processing (NLP) has witnessed a lot of developments in recent years. <strong>Modern NLP</strong> is now synonymous with <strong>transformers</strong> and architectures like <a href="https://en.wikipedia.org/wiki/BERT_(language_model)">BERT</a>, <a href="https://ai.facebook.com/blog/roberta-an-optimized-method-for-pretraining-self-supervised-nlp-systems/">RoBERTa</a>, ELECTRA and so on. Architectures based on transformers have improved the SOTA(state-of-the-art) on almost every downstream task. Training task-specific architectures is being quickly replaced by this new paradigm of pre-training and fine-tuning large language models. The underlying principles of these models, however, are quite similar to previous techniques in NLP.</p><p>Let’s take a short tour of NLP research in the deep-learning era.</p><ul><li><a href="https://en.wikipedia.org/wiki/Word_embedding"><strong>Word Embeddings</strong></a>: These are nothing but fixed-size low dimensional vectors that encode the semantics or the meaning of a word. These vectors are also trained using a large corpus in an unsupervised way. The most common approach, called the <strong>skip-gram</strong> method involves training a proper end-to-end neural network. Here the input is a given word in the corpus and the outputs are the words surrounding the chosen word. In short, we make the model predict the context based on the current word. Eventually after training the neural network, we remove the prediction layer and use the weights of the hidden layer as word-vectors. E.g. <a href="https://en.wikipedia.org/wiki/Word2vec">Word2vec</a>, <a href="https://nlp.stanford.edu/projects/glove/">GloVe</a>, <a href="https://fasttext.cc/">fastText</a>.</li></ul><blockquote>Refer to this <a href="http://jalammar.github.io/illustrated-word2vec/"><strong>blog</strong></a><strong> </strong>if you need to understand it in more details.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*g6QSEBrT-WZ1J7wt.png" /><figcaption>Vector spaces <a href="https://towardsdatascience.com/creating-word-embeddings-coding-the-word2vec-algorithm-in-python-using-deep-learning-b337d0ba17a8">showing </a>the algebraic relationship between related words.</figcaption></figure><ul><li><strong>Language Modeling</strong>: One of the main drawbacks of word embeddings discussed above is that they are static or context-free. A word is associated with only one meaning in general. <a href="https://allennlp.org/elmo"><strong>ELMo</strong> </a>overcomes this by training multilayer bi-LSTM language models which capture both semantics and syntax of the language. While contextualized word-embeddings from ELMo improved SOTA on many tasks, the idea proposed by <a href="https://arxiv.org/abs/1801.06146"><strong>ULMFiT</strong> </a>was perhaps the turning point in NLP research. It introduced the idea of pre-training language models on large amounts of text corpora and then fine-tuning these models on downstream tasks like text-classification.</li></ul><p>In their own words,</p><blockquote><em>Our method significantly outperforms the state-of-the-art on six text classification tasks, reducing the error by 18- 24% on the majority of datasets.</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/523/0*LovJLCJyzHnwkU16.png" /><figcaption>Simple example depicting the task of language modeling.</figcaption></figure><ul><li><strong>Masked Language Modeling &amp; BERT</strong>: BERT uses the <strong>transformer encoder </strong>and is <strong>pre-trained on a huge text corpus</strong>. The pre-training technique is also different from previous language models and uses <a href="https://demo.allennlp.org/masked-lm"><strong>masked language modeling</strong></a><strong> </strong>objective. BERT reported SOTA results on almost every downstream task including text-classification, <a href="https://en.wikipedia.org/wiki/Named-entity_recognition">NER</a>, <a href="https://en.wikipedia.org/wiki/Question_answering">QA</a>, <a href="http://nlpprogress.com/english/natural_language_inference.html">NLI </a>and so on. It was considered to be a major breakthrough in NLP research. Since then, there have been multiple models built around BERT or transformer architecture.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*YjG7AxN7a_jjtnLe" /><figcaption>Number of mentions of key trends in NLP in ACL papers taken from <a href="https://towardsdatascience.com/major-trends-in-nlp-a-review-of-20-years-of-acl-research-56f5520d473">here</a>.</figcaption></figure><h3>The State of Indic-NLP</h3><p>All the advancements and cool developments in NLP that we boasted about in the previous section are surprisingly <strong>limited to English </strong>and a handful of other high-resource languages. <strong>Indian languages are highly under-represented in the current NLP research</strong> despite having a population close to 1.3 billion. Even in India, there are a lot of languages spoken in different parts of the country.</p><blockquote>Only <a href="https://en.wikipedia.org/wiki/List_of_languages_by_number_of_native_speakers_in_India">10% of the Indian population speak English</a>.</blockquote><p>This essentially means that a large number of people that speak or communicate in low-resource languages have not benefited from the recent developments in NLP.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*cYNTsLN4IXsMWNP4" /><figcaption>A map showing the diversity of languages in India.</figcaption></figure><p>However, there have been some concerted efforts to improve NLP research in Indian languages.</p><ul><li><a href="https://github.com/goru001/inltk"><strong>inltk</strong></a>: Along with a toolkit to pre-process and normalize Indian languages, inltk also provides language models trained on Indian languages with their performance on classification tasks. These language models are based on ULMFiT.</li><li><a href="https://indicnlp.ai4bharat.org/home/"><strong>Indic-NLP</strong></a>: Classification datasets for 11 languages, fast-text word-embeddings and normalizers for Indian languages. Recently, they also released an IndicBERT model which is a multilingual ALBERT pre-trained on 11 Indian languages.</li></ul><h3>Our Contribution</h3><p>We at <a href="https://neuralspace.ai/"><strong>NeuralSpace</strong> </a>&amp; <a href="https://reverieinc.com/">Reverie</a> believe in pursuing research that matters and benefits the community. When we set out to see the performance of these SOTA models on Indian languages, we were surprised to find the lack of research and benchmarks of these models. So we decided to start things from scratch. Some of our contributions are as follows:</p><ul><li>We <strong>trained 4 variants of monolingual contextual language models from scratch.</strong> These variants include <a href="https://arxiv.org/abs/1910.01108"><strong>DistilBERT</strong></a>, <strong>BERT</strong>, <strong>RoBERTa</strong> and <a href="https://arxiv.org/pdf/1911.02116"><strong>XLMRoBERTa</strong></a>. We do so for 3 different languages viz. <strong>Hindi</strong>, <strong>Telugu</strong> and <strong>Bengali</strong> which cover more than <a href="https://en.wikipedia.org/wiki/List_of_languages_by_number_of_native_speakers_in_India">60% of native speakers</a> in the country.</li><li>We <strong>evaluate</strong> the performance of these models on <strong>3 downstream tasks</strong>: <strong>text</strong>-<strong>classification</strong>, <strong>POS</strong>-<strong>tagging</strong> and <strong>Question-Answering</strong> (QA).</li><li>We present an <strong>exhaustive analysis of transformer-based for Indian languages </strong>by comparing them directly with their multilingual counterparts.</li></ul><h4>Some challenges that we faced during this work:</h4><p>While we explain the issues and how we overcame those in the paper in a greater detail, I would like to outline them in brief over here.</p><ul><li><strong>Lack of monolingual data to train language models</strong>: BERT reported SOTA results on various downstream tasks in <strong>English</strong> and it was trained on <strong>3.3 billion</strong> tokens. If we compare that to the number of tokens available for Indian languages, we have ~<strong>700M for Hindi</strong>, <strong>300M for Bengali</strong> and <strong>79M for Telugu</strong> respectively. If we are to train powerful and efficient language models, we need to have more data especially for languages like Telugu.</li><li><strong>Lack of labeled datasets for downstream tasks</strong>: In order to evaluate our language models or any other supervised learning approach, we must have some annotated datasets for all major languages spoken in the country. More importantly, these datasets should be easily accessible to the research community. <strong>Universal Dependencies</strong> is a great initiative in order to realize this aim by curating tree-banks for a lot of low-resource languages. However, we faced challenges for finding such tree-banks for Bengali. Even for QA, there are no major datasets for Indian languages.</li></ul><h3>Our Experiments and Findings</h3><p>We run exhaustive experiments on the selected downstream tasks under 3 different setups.</p><ul><li><strong>Setup A:</strong> In this setup we evaluate multilingual models that are available <strong>out-of-the box</strong> on<a href="https://github.com/huggingface/transformers"> Hugging Face</a> model hub. Multilingual models are generally trained by combining vocabularies and datasets of 100+ languages and training the language model in the hope of cross-lingual transfer of features. Multilingual variants are available for DistilBERT, BERT and XLM-R.</li><li><strong>Setup B</strong>: Next we <strong>fine-tune each multilingual languag</strong>e model from setup A with data from a single language. Since multilingual models are trained with data from multiple languages, we believe that fine-tuning them with monolingual data would be beneficial, In our case we fine-tune mBERT with 300MB data of Hindi, Bengali &amp; Telugu separately.</li><li><strong>Setup C</strong>: Finally, in this setup <strong>we train monolingual contextual models from scratch</strong>. This allows us to directly compare our models with their multilingual counterparts.</li></ul><p>For each of these setups we run experiments under 2 different training regimes/settings</p><ul><li><strong>Frozen</strong>: In this setting, <strong>we freeze the language model </strong>and use its output as embeddings to train BiLSTM, linear and transformer layers on top.</li><li><strong>Fine-tuned</strong>: Here, <strong>we fine-tune the entire language model</strong> with a simple linear layer on top.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*GGFA-rL29QKrWha-.png" /></figure><p>Direct comparison between the best results from setup A and setup C and related work. The empty cells denote models were either not evaluated on that particular task/language (e.g., IndicBERT on POS-Tagging) or that the model was only evaluated on one specific task (e.g., TyDiQA).</p><p>The table above summarizes our key results and also compares it with other recent and relevant work. <strong>We report state-of-the-art results for Hindi and Bengali text classification</strong>. Our results for POS tagging are also competitive with the latest approaches.</p><h4>Some key insights/trends from our work:</h4><ul><li>We find tha<strong>t training a multi-layer LSTM/biLSTM using language model output as embeddings gives comparable or sometimes even better results than fine-tuning the entire mode</strong>l. This is particularly helpful in cases where you do not have access to high-power GPUs. In our work, LSTM layers usually outperformed other layers that we experimented with under the “frozen” setting.</li><li>The above point holds true for text-classification and POS tagging for all 3 languages. However for <strong>question-answering</strong>, we find that <strong>fine-tuning outperforms freezing </strong>by a considerable margin.</li><li>Our models in <strong>setup C outperform their multilingual counterparts</strong> in case of BERT and DistilBERT for all languages and tasks.</li><li><strong>RoBERTa models in setup C perform poorly for all tasks</strong> and languages.We believe this is due to the fact that RoBERTa uses a Byte-level BPE tokenizer which affects the typology of morphologically rich languages. This has been pointed out <a href="https://arxiv.org/pdf/2004.03720v2.pdf">earlier</a> for other languages. We found this to be true for Indian languages as well. Hence the tokenizer choice becomes a key factor when dealing Indian languages.</li></ul><h4>You can read in detail about our experiments, results and analysis in the paper <a href="https://arxiv.org/abs/2011.02323">here</a>.</h4><p>You can find our pre-trained models over <a href="https://huggingface.co/neuralspace-reverie">here</a>.</p><p><a href="https://huggingface.co/neuralspace-reverie">neuralspace-reverie (NeuralSpace-Reverie)</a></p><p>At <strong>NeuralSpace</strong>, we are working towards <strong>bringing the state-of-the-art NLP research to low-resource Indian languages</strong>. Feel free to reach out to us to discuss new ideas!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c6b4db0643b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/neuralspace/indic-transformers-an-analysis-of-transformer-language-models-for-indian-languages-c6b4db0643b">Indic Transformers: An Analysis of Transformer Language Models for Indian Languages</a> was originally published in <a href="https://medium.com/neuralspace">NeuralSpace</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Are you indulging in a toxic conversation?]]></title>
            <link>https://medium.com/neuralspace/are-you-indulging-in-a-toxic-conversation-c67708b8895?source=rss-c7ad13abfbb7------2</link>
            <guid isPermaLink="false">https://medium.com/p/c67708b8895</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[nlp]]></category>
            <dc:creator><![CDATA[Kumar Shridhar]]></dc:creator>
            <pubDate>Sat, 06 Jun 2020 12:12:41 GMT</pubDate>
            <atom:updated>2020-06-06T16:32:52.414Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*n0BmWpVcZfVW3cOk.jpg" /><figcaption><a href="https://www.google.com/url?sa=i&amp;url=https%3A%2F%2Fwww.womenshealthmag.com%2Frelationships%2Fa19739065%2Fsigns-of-toxic-relationship%2F&amp;psig=AOvVaw2ehujdQh0TlXW_wkpJW7yy&amp;ust=1591294761535000&amp;source=images&amp;cd=vfe&amp;ved=0CAIQjRxqFwoTCLCfz4qh5ukCFQAAAAAdAAAAABAF">Source</a></figcaption></figure><p>As with every other story nowadays, ours also begins with the <strong>COVID-19</strong> quarantine. This project is a result of bored, but motivated AI enthusiasts. This may not be as impressive as the roof garden that Andrew built, or the new customization on <a href="https://www.indianmotorcycle.com/en-us/scout-bobber-sixty/">Indian scout Bobber</a> by Kate, but its a start. Before we waste any more of your time, let us start with what exactly this project is. And why, if any, you should give a frick about it.</p><p>This project is aimed at<strong> developing an application that detects toxicity in an online conversation</strong>. This project is inspired by the ongoing <a href="https://www.kaggle.com/c/jigsaw-unintended-bias-in-toxicity-classification">Kaggle toxicity detection competition</a>, which focuses on using machine learning algorithms to identify toxicity in online conversations.</p><blockquote>A toxic comment is defined as a <em>rude, disrespectful, or profane comment that makes other people uncomfortable, leading to leaving the discussion.</em></blockquote><h3>WHY DO WE CARE ABOUT THIS PROJECT?</h3><p><strong>Social platforms</strong> like YouTube offer an option for comments so that people can have an open discussion about the content. However, these days what we find in the comment section is graded so bad that US-officials have passed <a href="https://www.ftc.gov/tips-advice/business-center/guidance/complying-coppa-frequently-asked-questions">an act that includes disabling comments</a> from YouTube videos made for kids. The <strong>comment</strong> section nowadays consists of <strong>trash talks</strong>, <strong>vulgar comments</strong>, and <strong>abusive offensive conversations</strong>; leading to hate spread, emotional distress, and aggressive behavior especially in the younger generation.</p><p><strong>TLDR</strong>: it is a project that <strong>trains a Deep Learning model</strong> to detect whether a person’s comment is toxic or not. Since, none of us are descendants of Socrates (maybe we are, but we don’t care for now) or Social Science major, we are not going to go in the discussion of what is toxic and what is not. We will use the definition as used by many online gaming platforms to ban justly (or unjustly) gamers.</p><blockquote><em>Toxic person is simply causing spread of hate.</em></blockquote><blockquote>If you have any issues with this definition, or if you want a more spirited debate about this, please reach out to one of our project members.</blockquote><h3>WHAT TO EXPECT FROM THIS PROJECT?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PtIOlYQEbm7KEfeb8re2SA.png" /><figcaption>Project APOLLO</figcaption></figure><p>By the end of this article, you will be able to</p><ol><li><strong>Run</strong> a toxicity detection web application.</li><li><strong>Learn</strong> how to build this application from scratch.</li></ol><p>The implementation of this work is made public on our <a href="https://github.com/kumar-shridhar/Online-Toxicity-Detection">GitHub repo.</a> We advise you to clone it and work on it simultaneously as you read this article.</p><p>To start with, we assume that the reader is familiar with the <strong>basics </strong>of <strong>Deep Learning </strong>and <strong>Python</strong>. If not, here are some basics of <a href="https://medium.com/botsupply/a-beginners-guide-to-deep-learning-5ee814cf7706">Deep Learning</a> and <a href="https://www.learnpython.org/">Python</a> to start with.</p><h3>DATASET USED</h3><p>We tackled this problem in a supervised learning setup. The <strong>dataset</strong> used in this project is from <a href="https://www.kaggle.com/c/jigsaw-unintended-bias-in-toxicity-classification">Jigsaw/Conversation AI</a>. The dataset consists of ~2m public comments from the <a href="https://medium.com/@aja_15265/saying-goodbye-to-civil-comments-41859d3a2b1d">Civic Comments</a> platform. This dataset served as our <strong>training</strong> and <strong>validation</strong> set.</p><p>The data consists of the following toxicity attributes:</p><ul><li>severe_toxicity</li><li>obscene</li><li>threat</li><li>insult</li><li>identity_attack</li><li>sexual_explicit</li></ul><p>In this work, we do not classify the attributes of the toxic comments. Rather we tackle the problem in a <strong>binary</strong> fashion: <strong>Toxic</strong> or <strong>Non-Toxic</strong>.</p><h3>NETWORK USED</h3><p>For our task, we used<strong> </strong><a href="https://arxiv.org/abs/1911.02116"><strong>XLM-ROBERTa</strong></a><strong> (XLM-R),</strong> which is a powerful <strong>transformer</strong>-based unsupervised model. It is a cross-lingual sentence encoder network that obtains state-of-the-art results on many cross-lingual understanding benchmarks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/570/1*ZR1N7vn9bdzMGLuwfL6myw.png" /><figcaption><a href="https://arxiv.org/pdf/1706.03762.pdf">Source</a></figcaption></figure><p><strong>XLM-R</strong> is trained on 2.5T filtered <a href="https://commoncrawl.org/">CommonCrawl </a>data in 100 languages. More information can be found <a href="https://huggingface.co/transformers/model_doc/xlmroberta.html">here</a> and the official implementation is available <a href="https://github.com/pytorch/fairseq/tree/master/examples/xlmr">here</a>.</p><h3>TRAINING</h3><p>We trained our network on a <strong>single 12 GB Nvidia GPU</strong>. We also used <a href="https://colab.research.google.com/">Google Colab</a>, to experiment with the TPUs settings. We trained our network for 2 epochs and achieved a validation accuracy of around <strong>85%.</strong></p><p>As the end goal of the project was to create a web application, the trained network weight was quite large. So we <a href="https://www.tensorflow.org/tutorials/keras/save_and_load">compressed</a> the saved model and <strong>reduced</strong> it’s size from<strong> 3GB </strong>to <strong>1GB</strong>. The final weight is provided <a href="https://drive.google.com/file/d/1RNd4L_zGVrFF_Cl-6KfoHIInMO-5A0e3/view">here</a> to download.</p><h3>TESTING</h3><p>As we have our trained model ready, we will now move on towards building our final application. In order to <strong>test</strong> our application, we <strong>scraped</strong> comments from various social media platforms, but we realized it might be unethical in some sense to release it publically.</p><p>So, we limited the scope of this project to <strong>YouTube</strong> comments <strong>only</strong>.</p><p>We scraped the user comments from a given YouTube video URL and pass it to our trained model. We output the predicted toxic/non-toxic metrics on our application homepage.</p><h3>DEPLOYMENT</h3><p>As our final goal is to host this project as a web application, we used <a href="https://flask.palletsprojects.com/en/1.1.x/">Flask </a>for hosting the app and HTML with CSS for web styling.</p><blockquote>Flask is a web framework that provides tools, interfaces, libraries and technologies to build a web application.</blockquote><h3>HOW TO RUN THE PROJECT?</h3><ol><li>Clone the <a href="https://github.com/kumar-shridhar/Online-Toxicity-Detection">GitHub repository,</a> and download the trained weights from <a href="https://drive.google.com/file/d/1RNd4L_zGVrFF_Cl-6KfoHIInMO-5A0e3/view">here</a>.</li></ol><p>2. Install pre-requisites as mentioned in the <strong>GitHub</strong> <a href="https://github.com/kumar-shridhar/Online-Toxicity-Detection">README</a>.</p><p>3. Run app.pyand enter a YouTube URL to get the results.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*9UbGXRht4cJo7oUIYTr7fA.gif" /></figure><blockquote>Please note that <strong>to maintain the anonymity of the users</strong>, the <strong>users names</strong> and <strong>profile pictures</strong> has been <strong>randomized</strong> and do not correspond to the name and the profile picture displayed.</blockquote><p>You can download our code and try it yourselves. So use this time to learn something new and good. Our code is available on <a href="https://github.com/kumar-shridhar/APOLLO-1">GitHub</a> with all the required information.</p><p><a href="https://github.com/kumar-shridhar/Online-Toxicity-Detection">kumar-shridhar/Online-Toxicity-Detection</a></p><h4>Feel free to reach out to us in case of any queries!</h4><p><strong>Contributing Authors</strong>: <a href="https://medium.com/u/96ad4bb31990">Ashutosh Mishra</a>, <a href="https://medium.com/u/b08776846651">Naveed Akram</a>, <a href="https://medium.com/u/9872782f8662">Ritu Yadav</a>, <a href="https://medium.com/u/f4b2310be279">Sadique Adnan Siddiqui</a>, <a href="https://medium.com/u/2e147b50be40">Venkatesh Iyer</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c67708b8895" width="1" height="1" alt=""><hr><p><a href="https://medium.com/neuralspace/are-you-indulging-in-a-toxic-conversation-c67708b8895">Are you indulging in a toxic conversation?</a> was originally published in <a href="https://medium.com/neuralspace">NeuralSpace</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Bayesian Neural Network Series Post 2.5: Why the world needs a Bayesian perspective?]]></title>
            <link>https://medium.com/neuralspace/bayesian-neural-network-series-post-2-5-why-the-world-needs-a-bayesian-perspective-bee8060195b?source=rss-c7ad13abfbb7------2</link>
            <guid isPermaLink="false">https://medium.com/p/bee8060195b</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Kumar Shridhar]]></dc:creator>
            <pubDate>Sat, 15 Feb 2020 22:17:04 GMT</pubDate>
            <atom:updated>2020-02-17T11:15:51.576Z</atom:updated>
            <content:encoded><![CDATA[<h3>Why the world needs a Bayesian perspective?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*Pq5EnUt2-rkILXun.png" /><figcaption>Source: Xkcd</figcaption></figure><p>This post is a part of the series <strong>Bayesian Neural Networks (</strong>Check <a href="https://medium.com/neuralspace/bayesian-neural-network-series-post-1-need-for-bayesian-networks-e209e66b70b2"><strong>Post1</strong></a> and <a href="https://medium.com/neuralspace/bayesian-neural-network-series-post-2-background-knowledge-fdec6ac62d43"><strong>Post 2</strong></a><strong>) </strong>that covers some <strong>history</strong> of Bayesian-ism and why we need to see the world through a Bayesian perspective.</p><blockquote><em>We take for granted that the cases are equally possible, that is to say, that each case can occur as easily as any other. — Jakob Bernoulli, Ars Conjectandi 1713</em></blockquote><h3>Do certainties exist?</h3><p>On a fine Friday evening, a group of four friends were enjoying a fine game of poker. In no time, the game gets serious. Some fine amount of money was on the table. A fifth friend who was not part of the playing four members but was a connoisseur of the game was making predictions about the winning hand.</p><p>Soon another friend of the group joined the game. Since he had no idea of the game, he decided to watch the game. He hoped the game would be over by the time he will join and so all of them can have a good time together. But the intensity of the game was so high, that everyone decided to continue the game for some more rounds. The new friend sat with the expert of the game (fifth friend) and he decided to make predictions about the winning hand for every round.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/675/0*oO-zcMvsY24UyFEI" /><figcaption><a href="https://www.yahoo.com/news/professional-poker-player-pleads-guilty-092505131.html?guccounter=1&amp;guce_referrer=aHR0cHM6Ly93d3cuZ29vZ2xlLmNvbS8&amp;guce_referrer_sig=AQAAAJcF7v62Vv4N-BCVXosvlbQbM9qcz7bf3WY4vI9Qo2oE21IjvRlVzL3DdZlQ860fAAzNRfaM5rXky2Rm8wkRXNigj-IyNVYz9bRCiZ-JqJqx2DiDj4CcFXEPh5QGEQRvjVtIu_27ZcNClrTAEvRLluEcKkwf_NU5F_G9SOCzhAEF">Source</a></figcaption></figure><p>Like any new member in the room would have done, the new friend counted the number of players in the room. He observed the equal distribution of cards and the way the game is played. Based on the observations, he predicted the winning chance of every person to be 1 in 4 i.e. 25%. A fair observation indeed. However, the expert in the game was moderating the game for some time. He had some prior knowledge of the game. He decided to use his prior knowledge before making a prediction on the winner. Based on his prior knowledge, his bets were close to the winner for that round. However, poker is a highly uncertain and complex game and it is very difficult to make a correct prediction on the winner based on just some observations in the past.</p><p><strong>What was the real difference between the two predictions made?</strong> The new friend was observing the state of the game and based on that he made his decision. There are 4 players in the game and every one of the four players has an equal chance of winning the game. This is similar to tossing a coin and predicting the chance of getting a head or a tail. The chance is equal for both sides (Head or Tail) with a probability of 1 out of 2 or 50 per cent. But what if the coin is tossed 10 times and every time head appears as the results. <strong>Will the next prediction still 50–50 or will it change?</strong></p><h3>Is everything a probability?</h3><p>The event of tossing two coins one after the other are independent of each other, meaning the results of two coin tosses do not depend on each other. In that case, the probability distribution should be 1 out of 2 again or a 50% chance of getting a head or a tail. But isn’t the previous pattern has to play some role here.</p><p>1: Let’s start with the belief that everything in the world can be represented in the form of probabilities. <em>This belief is not certain and can be argued upon</em>. However, let us analyze this belief: There are things that are certain in this world and there are lots of uncertainty attached to many things. Things like if it is sunny outside and it actually is sunny is almost certain at the moment. It can be assumed that it is happening with a very high probability (close to 1). Again this can be argued upon but let’s take this into account.</p><blockquote><em>Statistics is the ‘Science of Uncertainty’ -</em>Noel Cressie and Christopher K. Wikle, <em>Statistics for Spatio-Temporal Data</em>, Wiley, 2011, p. 4</blockquote><p>Back to the coin toss case. We know the probability of an event happening (Head or Tail in this case) is equally likely (based on the assumption of a fair coin). It can be argued that the next outcome will be equally likely for both the outcomes. An equal probability of 50 per cent can be associated with each case and in doing so, we are seeing the world with<strong> CLASSICAL FRAMEWORK</strong>.</p><blockquote>Under a classical framework, outcomes that are equally likely to happen have equal probabilities.</blockquote><p>So, in our case, getting a Head is equally likely as getting a Tail and hence, the probability will be one-half for each case.</p><h4><strong>NOTE</strong>: A high probability expresses a high degree of certainty about the occurrence of an event. However, certainty is still a term under discussion and 100% probability does not mean the certainty of an event unless observed. It is possible to put a 100 per cent belief in an event whose occurrence cannot be assumed to be certain unless observed by the believer.</h4><p>2: Let’s start by thinking about how do we know that the probability of an outcome is one-half for a coin toss. We know that there are two possible outcomes and both are equally likely. But we have seen in the previous 10 tosses that every time the outcome is a Head. <strong>How do we know that a coin is fair and not biased towards Head outcome</strong>? We can do it by tossing it again for 10 more times. Are 20 tosses enough to determine the dice is fair? Maybe not. Let’s toss it for 10 more times and 10 more times and 10 more times and so on. How do we know when we should stop? We can define a stopping point ourselves and see if the probability of both outcomes are equally likely or not. However, if we toss it for an infinite number of times, there is a high chance that the outcome will become equally likely for both the outcomes if the coin is unbiased. This is how a <strong>FREQUENTIST</strong> will see the world.</p><blockquote>A frequentist framework assumes to have a hypothetical infinite sequence of events and then based on the relevant frequency, in that hypothetical infinite sequence, makes a decision.</blockquote><p>If assuming the coin is biased which can be checked based on tossing the coin for an infinite number of times and recording the outcome, the outcome for the 11th coin toss will be one-half again. But it relies on the belief that the coin is fair and unbiased. <strong>But how can we check if the coin is fair or biased?</strong></p><p><strong>Well, the answer is simple</strong>. Toss the coin again and again for an infinite number of times to determine whether the coin is fair or not. Well tossing a coin for <em>n</em> number of times cannot tell us if the coin is completely unbiased or not. There might be many factors that may be responsible for the unequal outcome: like tossing the coin in a certain way. But assume that there are no external factors determining the outcome and it completely dependent on the coin toss, then we can say that the coin is biased or fair based on the outcome of the toss. But all we would then say is coin is fair or biased or simply 0 or 1. So under the frequentist paradigm, this probability is either a 0 or 1, which does not completely make sense. Let’s dig in a bit deeper using the sunny day example that we used before to determine whether an event occurrence is certain or not. How about we want to know if tomorrow will be sunny or not? Now from a classical point of view, we can say that tomorrow will be sunny with a chance of one-third (<em>assuming the days can be sunny, rainy or cloudy</em>). But from a <strong>frequentist</strong> point of view, we have to think for all the possible hypothetical sequences of tomorrow with the probability of being sunny which seems a bit tough. So what we should do?</p><p><strong>3: </strong>Let’s assume that there is a trick that is being used to get the desired outcome in our coin toss experiment. Once the person who was tossing the coin earlier was replaced, the results for the next 10 tosses were 6 heads and 4 tails which looks more believable for the coin toss. But we are the only ones who know about the trick here. When we brought the same person back into the picture to toss the coin, we have prior information that might not be known to many people. But we can use the prior information that we have to determine the possible outcome for the next coin toss. We know that if the same trick is going to be used again, there is a high chance of getting a Head. So we put our bet more in favour of Head compared to Tails. This way of seeing the world is through <strong>BAYESIAN</strong> perspective.</p><p><em>We have prior knowledge about the event which we used alongside with the information to make a prediction of the outcome.</em> It looks more like a real-life way of doing things. However, we are the ones who knew about the trick and another person who might be watching the coin toss might not know about it. <strong>So his prior information will be different than ours and hence his prediction will be different than ours.</strong></p><h3>The ‘Dutch Book’ Argument</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/185/0*cBZQzZ6D9tvaPy9j" /><figcaption>A different dutch book (a movie) but with similar motive</figcaption></figure><p><strong>So how can a person use the Bayesian perspective if it subjective to one’s belief?</strong> There is a way to deal with it. Let’s get back to the coin toss example again. This time before tossing a coin, let’s put a moderator there who will have two jobs:</p><ol><li>He will make sure that the coin toss results are what it is being claimed for. So we reduce the chances of interference with the outcome.</li></ol><p>2. He will allow only those people to make a prediction who are willing to bet in favour or against of getting a Head.</p><p>But there is one <strong>condition</strong>: For every bet, the winning amount would be 1 Euro lesser than the loss amount. Meaning, if you put a 5 Euros bet in favour, if you lost the bet, you have to pay 6 Euros. This is tricky and risky. Now person A knows the secret of the person tossing the coin as he knows if the coin tosser will use a trick, the chance of getting a Head would be high. He will use his prior knowledge to put bets on when he sees the trick is being used. On the other hand, person B is unaware of the tricks. He puts bets based on his knowledge about the coin toss i.e: the outcome of a coin toss is either Head or Tail with equal probability of one-half. This shows that a Bayesian perspective is very subjective as the prior information can be subjective.</p><p>Suppose player B plays the game for 10 times and we assume that the no trick was involved. What do you think the outcome will be at the end? <strong>Will person B takes home some winning money or did he lost some? </strong>No matter what outcome Player B puts his bet on, he is going to lose an amount at the end of 10 games. The reason is his bets are not coherent.</p><blockquote><em>Coherence refers to the phenomenon where standard rules of probability are followed.</em></blockquote><p>In our case, the bets are not coherent. Let’s calculate it: Winning here is getting the desired outcome (the one on which we put our bet on). Suppose we put our money on Head as the desired outcome. Probability of winning here is ½. If we win, we will win 5 Euros and if we lose we will lose 6 Euros. Overall win/loss for our case is defined by the overall probability P(<em>E</em>) score as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*R-lde8vigk1AnwKg" /></figure><p>This means on average player B will lose 50 cents on every game. This is not a desirable bet to place the money on. This phenomenon where someone constructs a series of bets where losing money is inevitable is known as ‘<strong>Dutch book</strong>’. <em>A popular term used in gambling as well as in statistics</em>.</p><h4>The phenomenon of Dutch Book can be avoided however by following the coherent rules of probability and Bayesian statistics.</h4><p>Now let’s get back to the poker game. We have four players playing a fair game of poker. Poker is, however, a highly unpredictable game. Two friends, let’s name them Adam and Bob are watching the game closely and are trying to put their bets on who will win the given round. <strong>Adam is an expert of the game</strong> and has been watching the game closely for some rounds. <strong>Bob </strong>was outside and just joined the game and<strong> has no prior knowledge about the winning person </strong>of previous rounds.</p><h3><strong>Subjective Probability</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*TgE48KtaLcNQXTGO.jpg" /><figcaption>Subjectivity (<a href="https://www.google.com/url?sa=i&amp;url=https%3A%2F%2Ftempleofpoi.wordpress.com%2Ftag%2Fobjective-vs-subjective%2F&amp;psig=AOvVaw0PlABP4-Mm_ZKUGI0EOcLw&amp;ust=1581810394767000&amp;source=images&amp;cd=vfe&amp;ved=0CAIQjRxqFwoTCIiAsrub0ucCFQAAAAAdAAAAABAD">Source</a>)</figcaption></figure><p>Now let’s assist Bob and Adam in their prediction based on the three frameworks as <strong>Classical</strong>, <strong>Frequentist</strong> and <strong>Bayesian</strong> respectively that we saw just now:</p><ol><li>If Bob follows the <strong>classical</strong> approach, this means his prediction is based on the concept that the outcomes that are equally likely to happen will have equal probabilities. Since probabilities sum up to 1, this means every person winning chance is equally and it is hard to put a bet on any person.</li><li>If Bob follows the <strong>frequentist</strong> framework, it is hard to run infinite simulations of who is going to win in normal circumstances and the game of poker involves a lot of luck and mental state of the players. This is an example of a single-case probability sample that we have and we have to make a prediction based on that. <strong>This is where the frequentist framework has a hard time to analyze the states and to make a decision.</strong></li><li>The more ideal way for the analysis would be to use the <strong>Bayesian</strong> framework. Now, a Bayesian Bob would say that he has a certain degree of belief that player A is going to win the game. Now, from where does this belief comes from and what does this belief indicate? Let’s start by saying a statement that “<em>I believe that there is a 10 per cent chance that it is going to rain tomorrow</em>”. Now what I am saying here: I am implying that I put a 10 per cent confidence in my belief that the event (raining tomorrow) is going to happen. Now, this degree of belief is very subjective and we will see soon that it is also dependent on the subjective prior knowledge of the event. So, let’s take a second to define probability in terms of Bayesian perspective or we can call it subjective probability.</li></ol><blockquote><em>Subjective probability is the confidence or the degree of belief that someone puts into the happening of an event or in simple words a degree of belief that something occurs.</em></blockquote><p>Now, what about Adam. Let’s take all the cases into account for Adam:</p><ol><li>From a <strong>classical</strong> framework perspective, there is<strong> no change</strong> in Adam’s prediction. It will be the same as Bob. All four players are equally likely to win here and hence he will either choose randomly one of those or if he doesn’t want to take a risk, will not put a bet.</li><li>From a <strong>frequentist</strong> point of view, there is a difference now. He has seen some finite number of games being played and he has outcomes for a finite number of events. Now one way to predict is to count the total number of games that he witnesses and see who won more games and put a bet on that person. However, a frequentist person would not be sure here if the number of observations is considerate enough to put a bet. Any event cannot be performed an infinite number of times. There is a finite number where the experiments need to stop. It can be decided pre-hand. But certainly, the number of observations here are not enough to favour one over the other. So it is hard to say if a bet should be placed by Adam or not.</li><li>Let’s take the <strong>Bayesian</strong> perspective here. Now based on the game observations and the results, there is a prior belief that one person (considering he is using some techniques like bluffing) is winning most of the hands. Now the prior is very subjective as discussed above. Adam is free to choose his own prior knowledge before he makes a prediction. Now there are two scenarios: <strong>First</strong>, all players are equally likely to win. There is no guarantee that a certain trick that made a player win a majority of the hands is being applied in this round. So either player can win with a 25 per cent chance. <strong>Secondly</strong>, we know that a player is playing well and we should put a bet in favour of that player. Adam believes that there is a 50 per cent chance that the favourite player will win this round.</li></ol><p>Now there is a subjective probability involved here. Now, it is up to someone to decide what bets he is ready to put in favour of the occurrence of certain events. Example: If Adam can predict the winner of a certain round and if he is right, he will get 100 Euros and if he is wrong he will not lose any money. In this case, he will go with the player with the most wins in the past. On contrary to that, if for wrong prediction he will lose 100 Euros, then he might not want to put his bet on any player.</p><h3><strong>Probabilities do not exist!</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/568/0*fr5v5Mk6WeJCF1Rd.jpg" /><figcaption>Famous lines by Bruno de Finetti (<a href="https://www.google.com/url?sa=i&amp;url=https%3A%2F%2Ftwitter.com%2Fgeorgeshiber%2Fstatus%2F762427696441798656&amp;psig=AOvVaw1BSMXEz8ezUCWbAQZAATdt&amp;ust=1581890778036000&amp;source=images&amp;cd=vfe&amp;ved=0CAIQjRxqFwoTCJiZg_rI1OcCFQAAAAAdAAAAABAK">Source</a>)</figcaption></figure><p>The famous line by a subjective Bayesian believer Bruno de Finetti: ‘<strong>Probabilities do not exist</strong>’. With this statement, de Finetti believes that objective probabilities do not exist. Meaning, <em>the probability is very subjective and it rather relies on the bet that a person is willing to take on the occurrence of an event</em>. De Finetti introduces the concept of <strong><em>Prevision</em></strong>, or probability via expectation.</p><p><em>The prevision of a random variable </em><strong><em>X</em></strong><em>, “according to your opinion, is the value </em><strong><em>x’ </em></strong><em>which You would choose” if “You are committed to accepting any bet whatsoever with gain </em><strong><em>C (X-x’)</em></strong><em> where </em><strong><em>C</em></strong><em> is arbitrary (positive or negative) at the choice of an opponent”. </em>-Bruno de Finetti, Theory of probability (p 87)</p><h3><strong>The dilemma</strong></h3><p><strong>If Bayesian probability is so subjective and if it is dependent on subjective beliefs, are we good at reasoning our beliefs?</strong> Is everyone who thinks in Bayesian way is equally good? <strong>How do we know that one belief is better than the other?</strong> How do we make sure that we follow the rules of probability here?</p><p><strong>How to reason someone’s belief?</strong> Remember we made a statement that I believe there is a 10 per cent chance that it will rain tomorrow. The statement 10 per cent chance can be written in the form of a probability of an event happening. In other words, it is the same as writing ‘<em>there is a 10 per cent chance that it will rain tomorrow’</em>.</p><h4><strong>Basically we have associated a number to the degree of belief. This work was done by Frank Ramsey and Bruno de Finetti in early 1900s.</strong></h4><p>We have assigned numerical values to the beliefs. But how do we make sure that the numbers make sense and it follows the rules of probability? Again we answered this before. The concept of <strong><em>Dutch book</em> </strong>is applied here. No one will put his bets on something where he knows he will lose the bet for sure. This will ensure that the bets are coherent to each other meaning we already follow the rules of probability.</p><blockquote>The term ‘<strong><em>Dutch book</em></strong>’ was coined by <strong>Frank Ramsey</strong>.</blockquote><p>Finally, as in the game of poker as discussed above, there was prior information with Adam. He was making predictions based on his prior knowledge. Now, this prior knowledge is getting updated as the game progresses. The prior plays an important role in his predictions.</p><h4>I will write about priors in a different post.</h4><h3>Summary</h3><ol><li>If a die is rolled once, a <strong>classicist</strong> will see all six outcomes to be equally likely and will predict each outcome with equal probability of one-sixth. A <strong>frequentist</strong> will analyze all the empirical evidence of the past which might have landed equally on either side and will base his prediction on that. A <strong>Bayesian</strong> subjectivity might agree to his beliefs which could state the equal likelihood of all sides or might base his belief on some different priors or on other information and will try to find more concrete information on which he should place his bets on.</li><li><strong>De Finetti</strong> introduced the idea of <strong>previsions</strong> and these <strong>previsions should be coherent in order to avoid an inevitable loss</strong>. This phenomenon is known as a <strong>Dutch book</strong> that ensures a sure loss at the hands of a clever opponent.</li><li><strong>Bayesian inference uses Bayes theorem to update the probability of a hypothesis as more evidence is available</strong>. Posterior is defined as prior times likelihood over evidence. The prior is subjective and may differ for two people observing the same event.</li></ol><p>If you are interested in creating your own <strong>Bayesian Neural Networks</strong> in PyTorch, check out the <a href="https://github.com/kumar-shridhar/PyTorch-BayesianCNN"><strong>code</strong></a> here.</p><p>If you have any comments/suggestion, find me <a href="https://twitter.com/JupyterAI"><strong>here</strong></a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bee8060195b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/neuralspace/bayesian-neural-network-series-post-2-5-why-the-world-needs-a-bayesian-perspective-bee8060195b">Bayesian Neural Network Series Post 2.5: Why the world needs a Bayesian perspective?</a> was originally published in <a href="https://medium.com/neuralspace">NeuralSpace</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Bayesian Neural Network Series Post 2: Background Knowledge]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/neuralspace/bayesian-neural-network-series-post-2-background-knowledge-fdec6ac62d43?source=rss-c7ad13abfbb7------2"><img src="https://cdn-images-1.medium.com/max/761/1*ozBVCzy6acVfLuSESiyeBw.png" width="761"></a></p><p class="medium-feed-snippet">This post is the second post in an eight-post series of Bayesian Convolutional Networks. The posts will be structured as follows:</p><p class="medium-feed-link"><a href="https://medium.com/neuralspace/bayesian-neural-network-series-post-2-background-knowledge-fdec6ac62d43?source=rss-c7ad13abfbb7------2">Continue reading on NeuralSpace »</a></p></div>]]></description>
            <link>https://medium.com/neuralspace/bayesian-neural-network-series-post-2-background-knowledge-fdec6ac62d43?source=rss-c7ad13abfbb7------2</link>
            <guid isPermaLink="false">https://medium.com/p/fdec6ac62d43</guid>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[bayesian-statistics]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Kumar Shridhar]]></dc:creator>
            <pubDate>Fri, 18 Jan 2019 16:49:02 GMT</pubDate>
            <atom:updated>2020-02-17T11:12:28.358Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Bayesian Neural Network Series Post 1: Need for Bayesian Networks]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/neuralspace/bayesian-neural-network-series-post-1-need-for-bayesian-networks-e209e66b70b2?source=rss-c7ad13abfbb7------2"><img src="https://cdn-images-1.medium.com/max/2600/1*n6Td0BSmvCGaTYaIJEqF-g.png" width="3264"></a></p><p class="medium-feed-snippet">This post is the first post in an eight-post series of Bayesian Convolutional Networks. The posts will be structured as follows:</p><p class="medium-feed-link"><a href="https://medium.com/neuralspace/bayesian-neural-network-series-post-1-need-for-bayesian-networks-e209e66b70b2?source=rss-c7ad13abfbb7------2">Continue reading on NeuralSpace »</a></p></div>]]></description>
            <link>https://medium.com/neuralspace/bayesian-neural-network-series-post-1-need-for-bayesian-networks-e209e66b70b2?source=rss-c7ad13abfbb7------2</link>
            <guid isPermaLink="false">https://medium.com/p/e209e66b70b2</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[neural-networks]]></category>
            <dc:creator><![CDATA[Kumar Shridhar]]></dc:creator>
            <pubDate>Wed, 02 Jan 2019 14:44:25 GMT</pubDate>
            <atom:updated>2020-02-17T11:14:54.833Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Know your Intent: State of the Art results in Intent Classification for Text]]></title>
            <link>https://blog.chatbotslife.com/know-your-intent-sota-results-in-intent-classification-8e1ca47f364c?source=rss-c7ad13abfbb7------2</link>
            <guid isPermaLink="false">https://medium.com/p/8e1ca47f364c</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[chatbots]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Kumar Shridhar]]></dc:creator>
            <pubDate>Sun, 30 Sep 2018 12:48:20 GMT</pubDate>
            <atom:updated>2019-10-23T17:58:07.005Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SojvC4Sh4qDTjHrz2Uk47Q.png" /><figcaption>F1 Score comparison of various platforms with our method on three datasets</figcaption></figure><p>This blog post shows the <strong>state-of-the-art results</strong> in Intent Classification obtained on the three corpora:</p><ol><li><a href="https://askubuntu.com/">Ask Ubuntu Corpus</a></li><li><a href="https://webapps.stackexchange.com/">Web Application Corpus</a></li><li>Chatbot Corpus</li></ol><p>The notebook with the code and results is available <a href="https://github.com/kumar-shridhar/Know-Your-Intent">here</a>:</p><p><a href="https://github.com/kumar-shridhar/Know-Your-Intent">kumar-shridhar/Know-Your-Intent</a></p><p>If you already have a basic understanding of the Intent classification for text, check out the original paper:</p><p><a href="https://arxiv.org/abs/1810.07150">[1810.07150] Subword Semantic Hashing for Intent Classification on Small Datasets</a></p><p>This post is divided into two parts:</p><p>1 we used a <strong>count based vectorized hashing technique</strong> which is enough to beat the previous state-of-the-art results in Intent Classification Task.</p><p>2 we will look into the<strong> training of hash embeddings based language models </strong>to further improve the results.</p><p>Let’s start with the <strong>Part 1.</strong></p><h3>THE CHALLENGE</h3><p>Three datasets for Intent classification task. Two of the corpora were extracted from <a href="https://data.stackexchange.com/">StackExchange</a> and the third one from a Telegram chatbot. The datasets looks like the following:</p><ol><li><strong>AskUbuntu Corpus</strong>: 5 Intents, 162 samples</li><li><strong>Web Application Corpus</strong> : 8 Intents, 100 samples</li><li><strong>Chatbot Corpus</strong>: 2 Intents, 206 samples</li></ol><p>More information about the datasets are available <a href="https://github.com/sebischair/NLU-Evaluation-Corpora">here</a>:</p><p><a href="https://github.com/sebischair/NLU-Evaluation-Corpora">sebischair/NLU-Evaluation-Corpora</a></p><h3>THE BENCHMARKS</h3><p>We checked the previous benchmarks on these datasets. The initial results were reported in the paper ‘<em>Evaluating Natural Language Understanding Services for Conversational Question Answering Systems</em>’. The results looked pretty good.</p><iframe src="https://drive.google.com/viewerng/viewer?url=http%3A//workshop.colips.org/wochat/%40sigdial2017/documents/SIGDIAL22.pdf&amp;embedded=true" width="600" height="780" frameborder="0" scrolling="no"><a href="https://medium.com/media/d42786cebf7ae7c0c3b2da842ee456f3/href">https://medium.com/media/d42786cebf7ae7c0c3b2da842ee456f3/href</a></iframe><p>Soon after, another benchmark was released by Botfuel, that surpassed the previous results and became the new benchmark results for all three datasets.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*kdERCRUnrDNRX-4Q9Yhxuw.png" /><figcaption>Benchmarking results for all three datasets as released by Botfuel</figcaption></figure><p><a href="https://github.com/Botfuel/benchmark-nlp-2018">Botfuel/benchmark-nlp-2018</a></p><p>Also, in January 2018, <a href="https://snips.ai/">Snips </a>ran an additional series of experiments to evaluate their Natural Language Understanding (NLU) solution against <a href="https://rasa.com/">RASA </a>NLU. Below are the results obtained.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/539/1*c0XlrBA42-_cl5Omyoa3rQ.png" /><figcaption><a href="https://github.com/snipsco/nlu-benchmark/tree/master/2018-01-Braum-et-al-extension">Source</a></figcaption></figure><p>In short, the benchmark before our experiments is well plotted in the following graph.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kmsnWVTxjyF-h7iusRWFqg.png" /><figcaption><a href="https://github.com/Botfuel/benchmark-nlp-2018">Source</a></figcaption></figure><p>Since no tech description is available for any of these bot platforms, it is good to believe that they use some featurizer and a classifier for classifying the Intents. But nothing can be said for sure what exactly is used by which platform.</p><p>For featurizer, <a href="https://en.wikipedia.org/wiki/Word2vec">Word2vec </a>seems to be a default good choice for embeddings. Even <a href="https://fasttext.cc/">fastText</a> seemed to be a good alternative. Since the dataset is too small, we decided not to use deep learning based embeddings. Also, the data is in form of free text <em>(commonly typed text sentences by people on the internet),</em> which makes a higher chance of spelling mistakes and out of vocab words. Also, we wanted to keep the training and inference time low. So we decided to use a semantic hashing for our case.</p><blockquote>If you want to replicate the results as show above, here is the Evaluation Script for the same : <a href="https://github.com/sebischair/NLU-Evaluation-Scripts">https://github.com/sebischair/NLU-Evaluation-Scripts</a></blockquote><h3>THE DATASET SPLIT</h3><p>The first task before evaluation was to <strong>split</strong> the dataset into <strong>train</strong>, <strong>test</strong> and <strong>validate</strong>. We used <strong>KStratified Splits</strong> to perform a <strong>k-fold cross valdiations </strong>on the data splits.</p><p>However, in the benchmark provided by Botfuel, they used a different test-train split. So to compare the consistency in the results, we used the same split as used by Botfuel. Our final results and the plots uses the same split as used by Botfuel benchmarks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/574/1*KECLKQjtowd1rmcN4ax34g.png" /><figcaption>Train Test Split as used in other benchmarks</figcaption></figure><blockquote>We did not do any kind of modification on test datasets and neither did we see the datasets in advance to fit our data more towards the test set.</blockquote><p>The dataset used by Botfuel and Recast can be found <a href="https://github.com/Botfuel/benchmark-nlp-2018/tree/master/results">here </a>which was used for the benchmarking.</p><p><a href="https://github.com/Botfuel/benchmark-nlp-2018/tree/master/results">Botfuel/benchmark-nlp-2018</a></p><h3>THE PRE-PROCESSING STEP</h3><p>We did use the standard preprocessing pipeline : <strong>tokenization, lemmatization</strong> and <strong>stemming</strong>. Although no better results were achieved when using lemmatization, and hence we removed it from the pipeline.</p><p>Upon looking into the training data distribution, we found that it was highly imbalanced. Techniques like <strong>SMOTE</strong> and <strong>ADASYN</strong> are good for data balancing, but in our case the dataset was only imbalanced for one of the Intents. <strong>Undersampling</strong> the dataset was not a good idea as the training dataset was very small. Also, <strong>oversampling</strong> of the dataset did not work as the number of samples for one specific class was very less (only 2 samples). So used some augmentation techniques to increase the data size.</p><h3>THE AUGMENTATION</h3><p>We started with a simple data augmentation technique. We took the samples from that class that had less samples in the training split. On these samples, we performed a <strong>dictionary based synonym replacement</strong> of nouns and verbs to generate new sentences.</p><pre><strong>Original sentence : </strong>What can I do to improve my running skills?</pre><pre><strong>Augmented data generated :<br></strong>* What can I do to <strong>advance</strong> my running skills?<br>* What can I do to <strong>better</strong> my running skills?<br>* What can I do to correct my running <strong>know-how</strong>?<br>* What can I do to correct my running <strong>proficiency</strong>?</pre><p>It did help in our case. There was a <strong>gain in accuracy of 1.4 – 2 percent</strong>.</p><p>Since this is a <a href="https://chatbotslife.com/">chatbot</a> dataset and there are higher chances of making spelling mistakes, we need to take that into account too. One simple way to correct the spelling mistakes is to find the <strong>Levensthein distance</strong> and map the word to it’s nearest neighbour when a spelling mistake is encountered.</p><blockquote><em>A dictionary with vocabulary specific to the dataset can be used instead of a general English word dictionary for domain specific spelling correction. However, our dataset is very small and there is a high chance of a word which might be out of the dataset specific dictionary and hence we did not create a domain specific dictionary.</em></blockquote><p>However, we tried a different approach. We took a key present in the keyboard and we mapped the nearest keys around it. <em>For example</em>: We took the key ‘<strong>s</strong>’ and we mapped nearby keys ‘<strong>a</strong>’, ‘<strong>w</strong>’, ‘<strong>d</strong>’, and ‘<strong>z</strong>’. <em>It is very likely that the typo errors are due to pressing the nearby key in the keyboard to a specific character instead of the real one</em>. We created a list of all the keys and their nearby distances. We also kept the nearby keys fixed to one surrounding key only.</p><p>We then set a Mistake Probability <strong>P. </strong>For any character <strong>C</strong>, swap it with another character <strong>C’ </strong>that is within <strong>d </strong>distance from <strong>C</strong>, and has an error probability greater than <strong>P</strong>. We predefined <strong>d </strong>and <strong>P</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1021/1*W4uj7yNF_Wtg6Jc7c3Vlbw.png" /><figcaption>Cartesian points for keys from others</figcaption></figure><p>The above mentioned augmentation technique <strong>boosted our accuracy by</strong> <strong>1.5–2 percent</strong>. (Enough to achieve the state-of-the-art results).</p><h3>THE FEATURIZER</h3><p>For the featurizer, we decided to use Semantic Hashing. To do this, we came up with two ideas:</p><p>1.Using a hash function, make a vocabulary and keep a count associated with it. The advantage is that it is easy to train, takes less time, and is good for smaller datasets. One big disadvantage is that the method fails when there is a very high collison rates.</p><p>2.Training a language model using hash vectors in an unsupervised fashion and then using it for our use case.</p><p>In this post, we will only talk about the first case and second case will be discussed in details in<strong> Part II of this blog</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/596/1*p_l1WYIVWe_PnQUHvlR5vA.png" /><figcaption>Original idea of semantic Hashing</figcaption></figure><p>Given an input text <strong>T,</strong></p><pre>“I have a flying disk”</pre><p>tokenize it to create a list of tokens <strong>t_i</strong>. The output of the tokenization should look like:</p><pre>[“I”, “have”, “a”, “flying”, “disk”]</pre><p>Pass each token into a pre-hashing function <strong>H(t_i) </strong>to generate sub-tokens <strong>t^j_i</strong>, where<strong> j</strong> is the index of the sub-tokens. E.g.:</p><pre><strong>H</strong>(<strong>have</strong>)=[\#ha, hav, ave, ve\#]</pre><p><strong>H(t_i) </strong>first appends a <strong>#</strong> at the beginning and at the end of a token, and then extracts trigrams from it. These trigrams are the sub-tokens <strong>t^j_i</strong>.</p><p><strong>H(t_i) </strong>can then be applied to the entire corpus to generate sub-tokens. These sub-tokens are then used to create a <a href="https://en.wikipedia.org/wiki/Vector_space_model"><strong>Vector Space Model</strong></a> (VSM). This <strong>VSM</strong> should be used to extract features for a given input text. In other words, this <strong>VSM acts as a hashing function </strong>for an input text sequence <strong>T</strong>.</p><p>Some other work has been done in past to overcome the shortcomings of the embedding based approaches by training hash embedding. The following diagram shows a couple of ways to train hash embedding.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EUUHdvrN9n_sKC8JyFWq9Q.png" /><figcaption>Source</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vEoK1oYEiN2B04JY0Bi9bg.png" /><figcaption>Source</figcaption></figure><p><a href="https://www.microsoft.com/en-us/research/publication/learning-deep-structured-semantic-models-for-web-search-using-clickthrough-data/">Learning Deep Structured Semantic Models for Web Search using Clickthrough Data - Microsoft Research</a></p><h3>THE CLASSIFIER</h3><p>We benchmarked the results across multiple classifiers to evaluate the results of using semantic hashing as featurizer. For that, we used all the classifiers provided by <a href="http://scikit-learn.org/stable/">Scikit-learn</a>.</p><p>For most classifiers, we used the default parameters but we did make some changes in the parameters of some classifiers like defining a prior value based on the class distribution for <strong>Naive Bayes</strong>, a grid search for the number of hidden layers in <strong>MLP</strong> and so on. All the information is available in the <a href="https://github.com/kumar-shridhar/Know-Your-Intent/blob/master/semhash_pipeline.ipynb">Notebook</a>.</p><p>We used the <strong>same training and test datasets for all the classifiers </strong>and plotted the results of the same.</p><blockquote>We used a Grid Search on KNN, MLP and SGD Classifiers to find the best hyper-parameters.</blockquote><h3>THE PLOT</h3><p>While training various classifiers, we plotted the <strong>accuracies</strong> of the classifiers along with their <strong>Train-Test time</strong> on a single graph to see how various classifiers performs with SemHash. Following are some of the plots achieved:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JP2VUAP3KEGjAWNyXWzsmw.jpeg" /><figcaption>Accuracy plots on all three datasets</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5N-hlVtM8E9MEh2kFoBP1w.jpeg" /><figcaption>Accuracy and Train-Test time plots for AskUbuntu Corpus</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5N-hlVtM8E9MEh2kFoBP1w.jpeg" /><figcaption>Accuracy and Train-Test time plots for Chatbot Corpus</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jo2t_gO0wlARrdUvXZ-KAA.jpeg" /><figcaption>Accuracy and Train-Test time plots for WebApplication Corpus</figcaption></figure><h3>THE RESULTS</h3><p>Results were evaluated based on the <strong>Precision, Recall, F1 score </strong>and the <strong>Accuracy</strong>. For benchmarking and comparison, we sticked with the F1 score as it was used in the previous benchmarks.</p><blockquote>Classifiers like Random Forest and SGD Classifier were consistent with good results across all the datasets while the F1 score went up and down a bit for the others.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/844/1*nawbqSGRrnWdaNYvMfwlgA.png" /><figcaption>Result Comparison of all platforms</figcaption></figure><p><strong>NOTE :<em> </em></strong><em>One thing to note here is that the results in the graph is taken from the benchmarks of the Recast and the average of Watson over the 3 coprus (0.97+0.92+0.83)/3 = 0.9066 which does not matches with the result of 92 mentioned in average section. However, a weighted average (0.97*100+0.92*53+0.83*30)/183 gives 0.9325 which is again not equal to 92 as mentioned.</em></p><blockquote>So, we are totally uncertain about the averaging method used above. <strong>In our case</strong>, we simply did an <strong>average over the 3 datasets</strong>.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SojvC4Sh4qDTjHrz2Uk47Q.png" /><figcaption>F1 Score comparison of different platforms with our method</figcaption></figure><h3>THE PRINCESS CAKE (Klassisk Prinsesstårta)</h3><p>Time to take a bite.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VSOF5GyDMO6uYJEdesXB-A.png" /><figcaption><a href="https://www.passionforbaking.com/blog/2017/06/27/a-classic-swedish-princess-cake-klassisk-prinsesstarta/">Source</a></figcaption></figure><blockquote>The results were achieved during a HackNight in <a href="https://www.ltu.se/?l=en">Lulea Technical University</a>, in Prof. Marcus Liwicki lab along with Fotini, Pedro, Ayushman, Amit, Vinay and Gustav. A big thanks to all of the them for their contributions individually and as a team.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QgWeQpLAieLx7Pfi9DMfMA.jpeg" /><figcaption>The A-Team</figcaption></figure><p>Check out this<a href="https://medium.com/neuralspace/kaggle-1-winning-approach-for-image-classification-challenge-9c1188157a86"> blog </a>for good hacks to achieve state of the art results in Image Recognition task:</p><p><a href="https://medium.com/neuralspace/kaggle-1-winning-approach-for-image-classification-challenge-9c1188157a86">Kaggle #1 Winning Approach for Image Classification Challenge</a></p><p>If you have any questions/suggestions, feel free to comment.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2F3e59ef%3Fas_embed%3Dtrue&amp;dntp=1&amp;url=https%3A%2F%2Fupscri.be%2F3e59ef&amp;image=http%3A%2F%2Fapi.screenshotlayer.com%2Fapi%2Fcapture%3Faccess_key%3Dfe59908dad3baab69ffab249a2224b03%26viewport%3D1024x612%26width%3D1000%26url%3Dhttps%253A%252F%252Fupscri.be%252F3e59ef%253Fscreenshot&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/7078d8ad19192c4c53d3bf199468e4ab/href">https://medium.com/media/7078d8ad19192c4c53d3bf199468e4ab/href</a></iframe><figure><a href="https://chatbotslife.com/bot-communities-mastermind-group-d2dae9876709#.53x0py6ou"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*6XUspT9JOSq0w0Fi35HIaA.png" /></a></figure><figure><a href="https://m.me/ChatbotsLife"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*c1LDMH5vbnIz9rmAka8Hwg.png" /></a></figure><figure><a href="https://chatbotslife.com/chatbot-development-hire-top-ai-chatbot-developers-c8b45ef7f207"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*D0Jf3dI6ZThtqcfwDYY7mg.png" /></a></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8e1ca47f364c" width="1" height="1" alt=""><hr><p><a href="https://blog.chatbotslife.com/know-your-intent-sota-results-in-intent-classification-8e1ca47f364c">Know your Intent: State of the Art results in Intent Classification for Text</a> was originally published in <a href="https://blog.chatbotslife.com/">Chatbots Life</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kaggle #1 Winning Approach for Image Classification Challenge]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/neuralspace/kaggle-1-winning-approach-for-image-classification-challenge-9c1188157a86?source=rss-c7ad13abfbb7------2"><img src="https://cdn-images-1.medium.com/max/1254/0*ae9HGgbSVjkeuXFh" width="1254"></a></p><p class="medium-feed-snippet">This post is about the approach I used for the Kaggle competition: Plant Seedlings Classification.</p><p class="medium-feed-link"><a href="https://medium.com/neuralspace/kaggle-1-winning-approach-for-image-classification-challenge-9c1188157a86?source=rss-c7ad13abfbb7------2">Continue reading on NeuralSpace »</a></p></div>]]></description>
            <link>https://medium.com/neuralspace/kaggle-1-winning-approach-for-image-classification-challenge-9c1188157a86?source=rss-c7ad13abfbb7------2</link>
            <guid isPermaLink="false">https://medium.com/p/9c1188157a86</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[kaggle]]></category>
            <category><![CDATA[convolutional-network]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Kumar Shridhar]]></dc:creator>
            <pubDate>Wed, 20 Jun 2018 17:59:19 GMT</pubDate>
            <atom:updated>2018-09-30T12:57:25.480Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Step by Step guide to Version Control your Machine Learning and Deep Learning tasks — II]]></title>
            <link>https://becominghuman.ai/how-to-version-control-your-machine-learning-task-ii-d37da60ef570?source=rss-c7ad13abfbb7------2</link>
            <guid isPermaLink="false">https://medium.com/p/d37da60ef570</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[version-control]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[git]]></category>
            <dc:creator><![CDATA[Kumar Shridhar]]></dc:creator>
            <pubDate>Fri, 04 Aug 2017 13:49:03 GMT</pubDate>
            <atom:updated>2018-10-06T07:32:16.063Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ne-Pf2CmqoJ9CeRb5XfZhw.png" /></figure><p>This post is in continuation of the previous post : <a href="https://medium.com/towards-data-science/how-to-version-control-your-machine-learning-task-cad74dce44c4"><strong>How to Version Control your Machine Learning - I</strong></a>. If you have not read the previous post, I would recommend to have a look at the previous post to understand the terminologies better.</p><p><a href="https://medium.com/towards-data-science/how-to-version-control-your-machine-learning-task-cad74dce44c4">How to Version Control your Machine Learning task — I</a></p><p>For the <strong>Ninjas</strong> out there in Version Control and Machine Learning, you can go ahead.</p><p>By now, we already know the importance of <strong>Version Control</strong>, let’s go ahead and implement it to see the real use.</p><p>Before going ahead, make sure that we have <a href="https://dataversioncontrol.com/"><strong>DVC</strong></a> installed in the system. We can check that based on the operating system we are using. Since, DVC can only be installed using pip command, it is necessary to make sure the <strong>Pip</strong> is installed in the system. It can be checked using the command:</p><pre>$ pip -V</pre><p>Once we are sure that the pip is installed, we can go ahead and <strong>install DVC </strong>by using the following command:</p><pre>$ pip install dvc</pre><p>Once we have DVC installed in the system, let’s go ahead and take a real life case and see how it works.</p><p><a href="https://www.eventbrite.com/e/the-chatbot-conference-tickets-34868758395?aff=CBL">The Chatbot Conference</a></p><h3>Example with Steps</h3><p>For this I am taking a code from <a href="https://numer.ai/"><strong>Numerai</strong></a>, which allows any data scientist to build machine learning models on their data, and submit predictions to control the capital in their Hedge Fund.</p><blockquote>Numerai abstracts its financial data, data scientists do not know what the data represents and human biases and overfitting are overcome.</blockquote><p>They also have an unique way of providing benefits with their own crypto currency which they call <strong>Numeraire</strong>.</p><blockquote><em>In February, </em><a href="https://numer.ai/"><em>Numerai</em></a><em> </em><a href="https://medium.com/numerai/a-new-cryptocurrency-for-coordinating-artificial-intelligence-on-numerai-9251a131419a"><em>announced Numeraire</em></a><em>, a cryptographic token to incentivize data scientists around the world to contribute artificial intelligence to our hedge fund (see </em><a href="https://www.forbes.com/sites/laurashin/2017/02/21/this-is-the-worlds-first-cryptocurrency-issued-by-a-hedge-fund/#6b41ee7d60b6"><em>Forbes</em></a><em>, </em><a href="https://www.wired.com/2017/02/ai-hedge-fund-created-new-currency-make-wall-street-work-like-open-source/"><em>Wired</em></a><em>, </em><a href="https://www.smithandcrown.com/numerai-profile-tokenization/"><em>Smith+Crown</em></a><em>). Earlier today, the Numeraire smart contract was deployed to Ethereum, and over 1.2 million tokens were sent to 19,000 data scientists around the world. — </em><a href="https://medium.com/numerai/an-ai-hedge-fund-goes-live-on-ethereum-a80470c6b681"><em>Source</em></a></blockquote><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fplayer.vimeo.com%2Fvideo%2F205032211%3Fh%3D10efc39660%26app_id%3D122963&amp;dntp=1&amp;display_name=Vimeo&amp;url=https%3A%2F%2Fvimeo.com%2F205032211&amp;image=https%3A%2F%2Fi.vimeocdn.com%2Fvideo%2F619688796-afeb255116379e73f36180631cca8ba78d4ba5102370b4d0d50be8640b2090de-d_1280&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=vimeo" width="1920" height="1080" frameborder="0" scrolling="no"><a href="https://medium.com/media/5dd971b8be006d8a0899bdd23c95062b/href">https://medium.com/media/5dd971b8be006d8a0899bdd23c95062b/href</a></iframe><blockquote>I will not talk more about Numerai here, but I will definitely mention it again in my next post in details about Cryptocurrency and Blockchain technology.</blockquote><p>Once signed in, Numerai data can be downloaded from the website of <a href="https://numer.ai/"><strong>Numerai</strong></a>, where they update the data after every 4 days. So, it might be possible that by the time you are reading this post, new data set is available by now.</p><p>Irrespective of what dataset you download, the following steps will almost be same with a few modifications:</p><h4>Steps</h4><ol><li>First initialize a git repo and put the downloaded code there.<br>$ mkdir numerai_code #create a repo<br>$ cd numerai_code <br>$ git init # initialize git<br>$ git add numerai_code_downloaded # Add downloaded data to git repo<br>$ git commit -m &#39;Numerai code added&#39;<br>$ git push</li><li>Install and Initialize DVC repository<br>$ pip install dvc<br>$ dvc init</li><li>‘<strong><em>numerai_training_data.csv</em></strong>’ and ‘<strong><em>numerai_tournament_data.csv</em></strong>’ will be present in the dataset downloaded which can be used to train and predict the results respectively.</li><li>Now, the time is to create a prediction model that predicts the data based on the dataset available and then put that file in the same repository (numerai_code) as above. Let’s call it ‘<strong><em>prediction.py</em></strong>’</li></ol><p>For the prediction model, I used an <strong>LSTM</strong> (Long Short Term Memory) <strong>Recurrent Neural Networks</strong> in <strong>Python</strong> with <strong>Keras. </strong><br>To know more about the RNN and LSTM architecture and other deep learning terms, please have a look at <a href="https://medium.com/@shridhar743/a-beginners-guide-to-deep-learning-5ee814cf7706">this post</a>:</p><p><a href="https://medium.com/@shridhar743/a-beginners-guide-to-deep-learning-5ee814cf7706">A Beginners Guide to Deep Learning</a></p><p>5. Run the python code within dvc with the following command:<br> $ dvc run python prediction.py</p><p>6. The model saves the checkpoints in a CSV file with the name assigned and the saved CSV can be submitted on the Numerai submission page.</p><blockquote><strong>NOTE</strong>: DVC automatically derives the dependencies between the steps and builds <a href="https://en.wikipedia.org/wiki/Directed_acyclic_graph">the dependency graph (DAG)</a> transparently to the user. Also, if there is a change in the files that changes everything, all the files will be reproduced.</blockquote><p>Not only can<a href="https://dataversioncontrol.com/"><strong> DVC</strong></a> streamline your work into a single, reproducible environment, it also makes it easy to share this environment by Git including the dependencies (DAG) — an exciting collaboration feature which gives the ability to reproduce the research results in different computers. Moreover, you can share your data files through cloud storage services like AWS S3 or GCP Storage since DVC does not push data files to Git repositories.</p><p>Check this blog post to know more about Data Version Control:</p><p><a href="https://blog.dataversioncontrol.com/data-version-control-beta-release-iterative-machine-learning-a7faf7c8be67">Data Version Control beta release: iterative machine learning</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/614/0*Z2Tf9_gOqaRf1lIy." /></figure><blockquote>In the <strong>next post</strong>, I will write more about how the environment and the dependencies can be shared between different scientists working on the same project and the sharing of files over cloud. <strong>Stay tuned!</strong></blockquote><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2F8f5f8b%3Fas_embed%3Dtrue&amp;dntp=1&amp;url=https%3A%2F%2Fupscri.be%2F8f5f8b&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/c43026df6fee7cdb1aab8aaf916125ea/href">https://medium.com/media/c43026df6fee7cdb1aab8aaf916125ea/href</a></iframe><figure><a href="https://becominghuman.ai/artificial-intelligence-communities-c305f28e674c"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*2f7OqE2AJK1KSrhkmD9ZMw.png" /></a></figure><figure><a href="https://upscri.be/8f5f8b"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*v-PpfkSWHbvlWWamSVHHWg.png" /></a></figure><figure><a href="https://becominghuman.ai/write-for-us-48270209de63"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*Wt2auqISiEAOZxJ-I7brDQ.png" /></a></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d37da60ef570" width="1" height="1" alt=""><hr><p><a href="https://becominghuman.ai/how-to-version-control-your-machine-learning-task-ii-d37da60ef570">Step by Step guide to Version Control your Machine Learning and Deep Learning tasks — II</a> was originally published in <a href="https://becominghuman.ai/">Becoming Human: Artificial Intelligence Magazine</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Version Control your Machine Learning task — I]]></title>
            <link>https://medium.com/data-science/how-to-version-control-your-machine-learning-task-cad74dce44c4?source=rss-c7ad13abfbb7------2</link>
            <guid isPermaLink="false">https://medium.com/p/cad74dce44c4</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[version-control]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Kumar Shridhar]]></dc:creator>
            <pubDate>Thu, 20 Jul 2017 10:01:28 GMT</pubDate>
            <atom:updated>2018-10-06T07:31:58.150Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ne-Pf2CmqoJ9CeRb5XfZhw.png" /></figure><h3>What is Version Control ?</h3><p>A component of <a href="https://en.wikipedia.org/wiki/Software_configuration_management">software configuration management</a>, <strong>version control</strong>, also known as <strong>revision control</strong> or <strong>source control</strong>,<a href="https://en.wikipedia.org/wiki/Version_control#cite_note-Mercurial-1">[1]</a> is the management of changes to documents, <a href="https://en.wikipedia.org/wiki/Computer_program">computer programs</a>, large web sites, and other collections of information. Changes are usually identified by a number or letter code, termed the “revision number”, “revision level”, or simply “revision”. For example, an initial set of files is “revision 1”. When the first change is made, the resulting set is “revision 2”, and so on. Each revision is associated with a <a href="https://en.wikipedia.org/wiki/Timestamp">timestamp</a> and the person making the change. Revisions can be compared, restored, and with some types of files, merged.</p><h3>Why Version Control ?</h3><p>An important question: <strong>Why do we need Version Control ?</strong> I am doing task on my local computer/cloud and I am deploying it at my server once the model is ready and only if I am done testing it. So why do I need version control ?</p><p>Now let’s look at a scenario: I am working for a company like <a href="http://botsupply.co/"><strong>Botsupply</strong></a> and I have clients. I am the AI guy. I made a question answering search using TF-IDF based model. I deployed it on my server. In the next phase, I made some changes to it and on my dummy data my accuracy increases. I deployed it on the server. Now due to the complexity in the test data, the performance decreases. Now I want to go back to the previous version.</p><p>One way is to <strong>deploy the previous version again</strong>. Second, or <strong>the better solution is version control</strong> and revert to the previous version.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/857/1*A4l0MO6Qi98hRHhTQA-WRg.png" /><figcaption><a href="https://dataversioncontrol.com/">Source</a></figcaption></figure><h3>How to do version control ?</h3><ol><li>One of the most popular ways for Version Control is <a href="https://git-scm.com/"><strong>Git</strong></a>. Very popular and basically everyone knows how to use it today. (Every programmer and data scientist at least). <br>Now, Git is really cool but for a data scientist, keeping all the folders synced in Git is a hard task. The models checkpoints and data size takes all the unnecessary space. So, one way is to store all the data set in the cloud server like Amazon S3 and all the reproducible codes in the Git and generate the models on the fly. Seems a good choice but multiple data sets, if used in the same code will create confusion and might lead to mixing of data sets in the long run, if not documented properly. <br>Also, if the data changes/upgrades and all the commits are not documented properly, the model might lose the context.</li></ol><blockquote>Results without contexts are more deadly than poison — <a href="https://medium.com/u/5a927ef41003">Giovanni Toschi</a> , <a href="http://botsupply.co/">Botsupply</a></blockquote><p>If the files cannot be reproduced in the fly, <a href="http://git-annex.branchable.com/"><strong>git-annex</strong></a> might be an option.</p><p><a href="https://git-annex.branchable.com/">git-annex</a></p><p>2. The second option is to do everything in a <strong>sandbox environment</strong> , see the results and if not good, don’t commit the changes to production. <a href="https://jupyter.org/">Ipython notebook (Jupyter Notebook)</a> is a good way to do so. The code can be broken into smaller segments in different cells and then results can be seen at every step which makes Ipython one of the best editors for Machine Learning.</p><p><a href="https://jupyter.org/">Project Jupyter</a></p><p>3. The best option (in my opinion) is to <a href="https://dataversioncontrol.com/"><strong>Data Version Control or DVC</strong></a>. DVC is similar to Git in many ways (like the command structure) but it also provides tracking of the steps, dependencies between the steps, dependencies between the code and data files and all code running arguments, so it combines version control for the code and for the database.</p><p><a href="https://dataversioncontrol.com/">Data Version Control - Make your data science projects reproducible and shareable.</a></p><blockquote>It is hardly possible in real life to develop a good machine learning model in a single pass. ML modeling is an iterative process and it is extremely important to keep track of your steps, dependencies between the steps, dependencies between your code and data files and all code running arguments. — <a href="https://medium.com/u/2b1c8f1b5d10">Dmitry Petrov</a> , DVC</blockquote><h3>What is DVC and 5 good reasons to use it?</h3><p>DVC makes data science projects reproducible by automatically building data dependency graph (DAG). Your code and the dependencies could be easily shared by Git, and data — through cloud storage (AWS S3, GCP) in a single DVC environment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ekGRk4XNNEMqr2PHPQqhIg.png" /><figcaption><a href="https://dataversioncontrol.com/">Source</a></figcaption></figure><ol><li>It is completely <strong>open source </strong>and can be install in simple commands with pip :</li></ol><pre><strong>pip install dvc</strong> #pip3 for python3</pre><p>2. The <strong>command is similar to git</strong>:</p><pre><strong>dvc run python train_model.py data/train_matrix.p data/model.p</strong></pre><p>3. It is <strong>language independent</strong> and machine learning processes can be easily transformed into reproducible DVC pipelines for any language.</p><p>4. Not only can DVC streamline your work into a single, reproducible environment, it also makes it easy to <strong>share this environment by Git</strong> including the dependencies (DAG) — an exciting collaboration feature which gives the ability to <strong>reproduce the research results in different computers</strong>.</p><p>5. Data files can be <strong>shared by any cloud files sharing services </strong>like AWS S3 or GCP Storage since DVC does not push data files to Git repositories.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/928/1*v6vO1fMPEg6sPWA6k_d13w.png" /><figcaption><a href="https://dataversioncontrol.com/">source</a></figcaption></figure><p>To learn more about the installation and usage of DVC in detail, please check the following blog post:</p><p><a href="https://blog.dataversioncontrol.com/data-version-control-beta-release-iterative-machine-learning-a7faf7c8be67">Data Version Control beta release: iterative machine learning</a></p><blockquote>Read the <a href="https://medium.com/@shridhar743/how-to-version-control-your-machine-learning-task-ii-d37da60ef570"><strong>part II</strong></a> to know how <strong>DVC </strong>helped me in version control while working with <strong>Numerai</strong>, where data scientists use machine learning to make predictions that power Numerai’s hedge fund.</blockquote><p><a href="https://medium.com/@shridhar743/how-to-version-control-your-machine-learning-task-ii-d37da60ef570">How to Version Control your Machine Learning task — II</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cad74dce44c4" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/how-to-version-control-your-machine-learning-task-cad74dce44c4">How to Version Control your Machine Learning task — I</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>