<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Branden Chan on Medium]]></title>
        <description><![CDATA[Stories by Branden Chan on Medium]]></description>
        <link>https://medium.com/@branden-chan?source=rss-51b74d7c71f4------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*1IPyTBj5mjaRRLqWzrBpdA.png</url>
            <title>Stories by Branden Chan on Medium</title>
            <link>https://medium.com/@branden-chan?source=rss-51b74d7c71f4------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 00:32:40 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@branden-chan/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Haystack V1.0 Official Release]]></title>
            <link>https://medium.com/deepset-ai/haystack-v1-0-official-release-18ff1b24224d?source=rss-51b74d7c71f4------2</link>
            <guid isPermaLink="false">https://medium.com/p/18ff1b24224d</guid>
            <category><![CDATA[transformers]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[haystack]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Branden Chan]]></dc:creator>
            <pubDate>Wed, 08 Dec 2021 10:49:12 GMT</pubDate>
            <atom:updated>2023-01-16T08:15:18.985Z</atom:updated>
            <content:encoded><![CDATA[<h4>Start creating NLP services for Question Answering, Semantic Document Search and much more!</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yEHv-XXIDIkkjqjKF0ykMQ.jpeg" /></figure><p>Since the inception of <a href="https://www.deepset.ai/">deepset</a>, we have held a strong belief that the developments coming out of the world of research would bring about significant improvements to the way that text is processed and searched through in enterprise settings. To pursue this, we started writing code to make the latest transformer models easier to use, more scalable and more suited to production environments.</p><p>Over time, the code, the tools and the resources that we created have been incrementally added to <a href="https://www.deepset.ai/haystack">Haystack</a>, our open source Python framework. And as our ambitions have grown, so has the response from our community who find new and exciting ways to use and improve Haystack. Looking over what we created, we now see that it is starting to resemble the vision that we had in our minds when we first embarked on this journey. And so it is with our greatest pleasure today that we are announcing the release of Haystack V1.0!</p><p>Indeed there is a raft of very significant new features in this release including support for <a href="https://haystack.deepset.ai/guides/table-qa">Table Question Answering</a>, reworked <a href="https://haystack.deepset.ai/tutorials/evaluation">Evaluation</a> and simpler <a href="https://haystack.deepset.ai/components/pipelines#returning-debugging-information">debugging</a> that we recommend you to read about in our blog article <a href="https://www.deepset.ai/blog/new-features-in-haystack-v1.0">here</a>. But to really understand the significance of this release, we want to talk a bit about the state of search technologies today, the needs of our users and how Haystack fits into enterprise technology ecosystem.</p><h3>The State of Search</h3><p>Search is ubiquitous. It’s expected to be packaged with just about any program that interacts with text. With breakthroughs in Machine Learning and Natural Language Processing (NLP), search technology has taken a big leap forward and <a href="https://github.com/deepset-ai/haystack">Haystack</a> dares to take the leap with it. The latest search systems that you can build in Haystack can now handle full sentence questions with sensitivity to their syntax and semantics. They no longer rely solely on keyword matching algorithms like TF-IDF and BM25 as the last generation of systems did. Thanks to these developments, search now comes in many different flavors and has become one of the most popular applications of NLP.</p><p>Also, research from elite institutions has pushed previously aspirational tasks, like question answering and summarization, firmly into the realm of both the possible and the practical. <a href="https://en.wikipedia.org/wiki/Transformer_(machine_learning_model)">Transformer models</a> have been a revelation in NLP. Performance on many tasks increased greatly in a short period of time and the transfer learning paradigm has allowed for new models to be trained with only a modest amount of data. Transformers now are the core technology powering the large majority of the best systems out there.</p><p>With these developments, we have seen growing interest in developers to try out the latest methods and build their own NLP services. Haystack has been a toolkit providing everything they need to create scalable custom NLP systems that perform semantic search, document ranking, question answering and summarization, just to name a few. In fact some of these creations are already in use at companies such as Etalab, Airbus and Alcatel Lucent.</p><p>But to understand why Haystack has been so valuable to these teams, it’s important to zoom out and see the full scope of the repository.</p><h3>From Storage to Models to Deployment</h3><p>We can’t stress enough that Haystack is not just about models. It’s an end-to-end framework. Our aim is to provide tooling and support for any step in the process of deploying an NLP service, like a Search system or Question Answering engine. For example, we understand the impact that choosing the right storage option can make to the success of your project. As such, we provide support for popular databases like Elasticsearch, SQL and OpenSearch, as well as vector optimized options like Milvus, FAISS and Weaviate in our <a href="https://haystack.deepset.ai/components/document-store">Document Store</a> classes.</p><p>Also, a core design feature of Haystack is the Pipeline. This feature enables different components to be chained together and handles the routing of data dynamically. For example, you can create a standard Question Answering system with a Document Store, a <a href="https://haystack.deepset.ai/components/retriever">Retriever</a> and a <a href="https://haystack.deepset.ai/components/reader">Reader</a> using a Pipeline. However, you might also want to have different Retrievers to handle different types of queries or create replicas of the Reader in a distributed setting to speed up inference or even slot in a custom Node that you defined into an existing system. All of this is possible through the flexibility and power that the Pipeline offers. We know that it is crucial for many that Pipelines are easily defined, and so we have provided the ability to <a href="https://haystack.deepset.ai/components/pipelines#yaml-file-definitions">configure, save and load</a> full Pipelines from a single YAML file.</p><p>Of course, many users need to be able to fine-tune models to their specific, often jargon filled domains. Haystack’s <a href="https://haystack.deepset.ai/guides/domain-adaptation">Domain Adaptation</a> and <a href="https://haystack.deepset.ai/guides/evaluation">Evaluation</a> features are designed to ease this process. We also know from personal experience that Annotation is a crucial and often very laborious step in a data driven project. The <a href="https://haystack.deepset.ai/guides/annotation">Annotation Tool</a> is there not only to provide a graphical interface for labeling, but also provides features that help teams organize their manpower.</p><p>Finally, a lot engineering effort is required to take a demo script and turn it into a production system. A truly useful NLP service needs to be reliably stopped and started to fit different hardware environments via Docker containers which is why we have included both <a href="https://github.com/deepset-ai/haystack/tree/main/docker">Docker</a> and <a href="https://github.com/deepset-ai/haystack/blob/main/docker-compose.yml">Docker Compose</a> files. It also needs to handle an incoming stream of requests and return results in a format that is readily consumable by other programs. For this we have provided a simple process for Haystack users to deploy their Haystack <a href="https://haystack.deepset.ai/overview/intro#the-3-levels-of-haystack">Nodes and Pipelines</a> as a fully fledged service complete with <a href="https://haystack.deepset.ai/guides/rest-api">REST API</a>. These features are not afterthoughts to Haystack but rather core to the purpose of the framework and we believe that this is what has so far drawn many developers to our repository.</p><h3>This is Just the Beginning</h3><p>As more developers start building more search programs with features like question answering, semantic document search or summarization, we are going to see a paradigm shift take place. Users will start to expect specific answers to questions presented along with their supporting documents. We can’t expect those working in information seeking positions to wade through large swathes of text any more without clear and effective summaries. And search will no longer be synonymous with keyword search. Haystack is our contribution to this shift and with the support and feedback of our outstanding open source community, we will continue striving towards making this the best tool available to anyone looking to work with text in a smarter way.</p><p>If you’re ready to try out <a href="https://github.com/deepset-ai/haystack">Haystack</a> for yourself, please check out our <a href="https://github.com/deepset-ai/haystack#beginner-quick-demo">Quick Demo</a> or learn to install it with our <a href="https://haystack.deepset.ai/overview/get-started">Get Started</a> page. If you still have any questions or would just like to chat with us, feel free to join our <a href="https://haystack.deepset.ai/community">Discord channel</a>. We’re excited for what is to come in the near future and we hope that you’ll join us on this journey!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=18ff1b24224d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/deepset-ai/haystack-v1-0-official-release-18ff1b24224d">Haystack V1.0 Official Release</a> was originally published in <a href="https://medium.com/deepset-ai">deepset-ai</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[New Features in Haystack V1.0]]></title>
            <link>https://medium.com/deepset-ai/new-features-in-haystack-v1-0-f507505e13a1?source=rss-51b74d7c71f4------2</link>
            <guid isPermaLink="false">https://medium.com/p/f507505e13a1</guid>
            <category><![CDATA[haystack]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[transformers]]></category>
            <dc:creator><![CDATA[Branden Chan]]></dc:creator>
            <pubDate>Wed, 08 Dec 2021 10:39:22 GMT</pubDate>
            <atom:updated>2023-01-13T07:54:40.085Z</atom:updated>
            <content:encoded><![CDATA[<h4>Try out Table QA as well as our improved evaluation and debugging today!</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*q3pc0hn5-PG9kXO48YAvFQ.jpeg" /></figure><p>We have been investing our time and effort into building <a href="https://haystack.deepset.ai/">Haystack</a> because we believe in a future where state-of-the-art search is easily implementable in any software. Search technologies have been quietly evolving in the academic space and Transformer models have ushered in a paradigm shift. Previously aspirational tasks like question answering, semantic document search and summarization are now firmly in the realm of both the possible and the practical.</p><p>Today we have taken a big step fulfilling the vision by releasing Haystack v1.0! Key features include improved evaluation, table question answering, improved debugging and more. Along with this, our our public demo is now live! Thank you so much to our community contributors whose effort and enthusiasm has propelled us to where we are now. We hope that you enjoy the ease of use that these features bring and we hope to hear of your successes in the near future!</p><p>Note that with this release, there are a few breaking changes with regards to import statements, input document format and primitive classes. Have a look at our <a href="https://haystack.deepset.ai/overview/migration">migration guide</a> for more details.</p><h3>Demo Website</h3><p>The quickest way to interact with a running Haystack system is through our new <a href="https://haystack-demo.deepset.ai/">demo website</a>! While traveling around the world is tricky these days, we have you covered. Explore facts about cities and countries with your keyboard and your favorite NLP models instead. We have indexed a slice of Wikipedia articles about countries and capital cities and in the back end, there is an Elasticsearch backed Document Store, a Dense Passage Retriever and a RoBERTa based Reader model.</p><h3>Improved Evaluation</h3><p>Haystack’s evaluation features are there to help you find out how well your system is doing on your data. It provides Pipeline level evaluation to ensure that the system’s output is really what you’re after, but also Node level evaluation so that you can figure out whether its your Reader or Retriever that is holding back the performance.</p><p>In this release, evaluation is much simpler and cleaner to perform. All the functionality is now baked into the <em>Pipeline</em> class and you can kick off the process by providing <em>Label</em> or <em>MultiLabel</em> objects to the <em>Pipeline.eval()</em> method.</p><pre>eval_result = pipeline.eval(<br>  labels = labels,<br>  params = { &quot;Retriever&quot;:<br>    {&quot;top_k&quot;: 5} },<br>)</pre><p>The output is an <em>EvaluationResult</em> object which stores each Node’s prediction for each sample in a Pandas <em>DataFrame</em>. There is a <em>EvaluationResult.calculate_metrics()</em> method which will return the relevant metrics for your evaluation.</p><pre>metrics = eval_result.calculate_metrics()</pre><p>If you’d like to start evaluating your own systems on your own data, check out our <a href="https://haystack.deepset.ai/tutorials/evaluation">Evaluation Tutorial</a>!</p><h3>Table QA</h3><p>A lot of valuable information is stored in tables — we’ve heard this again and again from our community. While they are an efficient structured data format, it hasn’t been possible to search for table contents using traditional NLP techniques. But now, with the new <em>TableTextRetriever</em> and <em>TableReader</em> our users have all the tools they need to query for relevant tables and perform Question Answering.</p><p>The <em>TableTextRetriever</em> is the result of our team’s research into table retrieval methods which you can read about in <a href="https://arxiv.org/abs/2108.04049">this paper</a> that was presented at EMNLP 2021. Behind the scenes, it uses three transformer based encoders — one for text passages, one for tables and one for the query. However, in Haystack, you can swap it in for any other dense retrieval model and start working with tables. The <em>TableReader</em> is built upon the <a href="https://github.com/google-research/tapas">TAPAS model</a> and when handed table containing Documents, it can return a single cell as an answer or perform an aggregation operation on a set of cells to form a final answer.</p><pre>retriever = TableTextRetriever(<br>  document_store = document_store,<br>  query_embedding_model = &quot;deepset/bert-small-mm_retrieval-question_encoder&quot;,<br>  passage_embedding_model = &quot;deepset/bert-small-mm_retrieval-passage_encoder&quot;,<br>  table_embedding_model = &quot;deepset/bert-small-mm_retrieval-table_encoder&quot;,<br>  embed_meta_fields = [ &quot;title&quot;, &quot;section_title&quot; ] <br>)</pre><pre>reader = TableReader(<br>  model_name_or_path = &quot;google/tapas-base-finetuned-wtq&quot;,<br>  max_seq_len = 512<br>)</pre><p>Have a look at the <a href="https://haystack.deepset.ai/guides/table-qa">Table QA documentation</a> if you’d like to learn more or dive into the <a href="https://haystack.deepset.ai/tutorials/table-qa">Table QA tutorial</a> to start unlocking the information in your table data.</p><h3>Improved Debugging</h3><p>We’ve made debugging in Haystack much simpler and also more informative! As long as your node receives a boolean <em>debug</em> argument, it can propagate its input, output or even some custom information to the output of the pipeline. It is now a built in feature of all existing nodes and can also easily be inherited by your custom nodes.</p><pre>result = pipeline.run(<br>  query = &quot;Who is the father of Arya Stark?&quot;,<br>  params = {<br>    &quot;debug&quot;: True <br>  } <br>)</pre><pre># Result</pre><pre>{ &#39;ESRetriever&#39;: {<br>  &#39;input&#39;: { &#39;debug&#39;: True,<br>    &#39;query&#39;: &#39;Who is the father of Arya Stark?&#39;,<br>    &#39;root_node&#39;: &#39;Query&#39;,<br>    &#39;top_k&#39;: 1 },<br>  &#39;output&#39;: {<br>    &#39;documents&#39;: [ &lt;Document: {&#39;content&#39;: &quot;\n===In the Riverlands===\nThe Stark army reaches the Twins, a bridge strong&quot;, ...}&gt; ] <br>    ...}</pre><h3>FARM Code Migration</h3><p>Those of you following Haystack from its first days will know that Haystack first evolved out of the <a href="https://github.com/deepset-ai/FARM">FARM framework</a>. While FARM is designed to handle different NLP models and tasks, Haystack gives full support to search and question answering use cases with a focus on coordinating all components that take a proof-of-concept into production.</p><p>Haystack has always relied on FARM for much lower level processing. For example, the actual handling of transformer models is done in FARM. It also deals with the text on a tokenized level, splitting text to fit a model’s maximum sequence length and picking the valid and optimal QA answer span amongst multiple chunked passages.</p><p>While the dependency of Haystack on FARM worked, it made maintenance much trickier as some features would necessitate a change in both repos and an update to the version dependency. To reduce this overhead, we have migrated the parts of FARM that are used in Haystack and put them in the <em>haystack/modeling</em> folder. We’ve also found that this simplifies lower level debugging as all the code is now in the one repo!</p><h3>What are you waiting for?</h3><p>Haystack v1.0 is out now so start using it today! See our <a href="https://github.com/deepset-ai/haystack">GitHub repo</a> if you want to install from the source code or see our <a href="https://haystack.deepset.ai/overview/get-started">Get Started</a> page to learn about our other installation methods.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f507505e13a1" width="1" height="1" alt=""><hr><p><a href="https://medium.com/deepset-ai/new-features-in-haystack-v1-0-f507505e13a1">New Features in Haystack V1.0</a> was originally published in <a href="https://medium.com/deepset-ai">deepset-ai</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Build Smart Conversational Agents with the Latest Chatbot and Question Answering Technology]]></title>
            <link>https://medium.com/deepset-ai/build-smart-conversational-agents-with-the-latest-chatbot-and-question-answering-technology-eb0223d725a0?source=rss-51b74d7c71f4------2</link>
            <guid isPermaLink="false">https://medium.com/p/eb0223d725a0</guid>
            <category><![CDATA[question-answering]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[haystack]]></category>
            <category><![CDATA[chatbots]]></category>
            <category><![CDATA[rasa]]></category>
            <dc:creator><![CDATA[Branden Chan]]></dc:creator>
            <pubDate>Tue, 19 Oct 2021 08:57:21 GMT</pubDate>
            <atom:updated>2023-01-13T08:53:08.467Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1021/1*2Y1Ic1PEssxQ78b-OEQSMQ.png" /></figure><p>Right now, there are more chatbots out there than ever — you’ve probably seen them embedded in many of the websites that you visit. This great adoption is driven by different tools but especially by Rasa, the leading conversational AI platform. If you’ve ever built a chatbot yourself, you probably encountered at some point a fundamental challenge: It’s hard to anticipate all possible “intents” a future user might have. However with the rise of question answering technology, there’s a new option for tackling the long-tail of those informational intents, that are not specific to a user (e.g. “How to change my password?”).</p><p>Today, we’re going to show you how to combine the best that both worlds have to offer. By following this guide, you’re going to learn how to build a chatbot that can identify either an information seeking intent from a user or a fallback intent, perform question answering on a large scale database of documents and then compose a well informed answer. Of course, we are going to keep it open source. That’s why we’ll be using <a href="https://haystack.deepset.ai/">Haystack</a> and <a href="https://rasa.com/">Rasa</a>.</p><h3>What is a Chatbot Powered by Question Answering?</h3><p>Modern chatbot frameworks provide the tooling to turn free-form conversational text into structured data allowing for context dependent responses. Rasa chatbots achieve this by using fine-tunable machine learning models that assign an intent to each user utterance. For each intent, the bot has the option to trigger a range of actions depending on the situation or conversational flow in order to compose a response. To give a simple example, a user utterance like “Where are you from?” might be labelled with an intent like location_question which triggers the response utter_location by which the user would see something like “I’m based in the cloud” as the answer.</p><p>Conversations are often predictable and fall into certain common patterns but some conversations can take many twists and turns. Well designed AI systems can cover many of the popular topics that come up but it is near impossible to define all the directions in which a conversation can go. Question answering is a powerful and flexible tool for handling those intents where relevant information needs to be smartly extracted from an existing set of documents.</p><p>Chatbots can already label a question like “What is the capital of Australia?” as an information seeking request. However, a chatbot that is augmented with question answering capabilities can look through a Wikipedia corpus, make sense of relevant paragraphs and extract “Canberra” as an answer. There doesn’t need to be a well structured knowledge graph or database running the back. Rather, using the power of NLP via a framework like <a href="https://haystack.deepset.ai/">Haystack</a>, this chatbot is able to pull information directly out of unstructured text. The result is a more robust chatbot that is capable of handling the long tail of informational user requests.</p><h3>Core Components</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HEW2Knxa2-iL7vhk8Ii3gg.png" /></figure><p>Rasa Core will handle everything to do with dialogue flow, classifying utterances by intent and triggering actions. It is a great platform for defining new intents and we are going to teach a chatbot to recognise information seeking utterances. When these utterances are detected, we will want the chatbot to trigger the appropriate action in Haystack. This is achieved via the Rasa Action Server which is a separate service from Rasa Core that allows for the chatbot to interact with external services.</p><p>The external service that handles the information intents of this system will be a Haystack question answering pipeline. This is composed of a database filled with relevant text documents, a retriever that quickly identifies the sections of your corpus that might be useful, and a reader which closely reads the text and highlights the specific words, phrases or sentences which answer the user query. Thanks to the REST API support in Haystack, it is easy to get Rasa to communicate with Haystack. Apart from this, we just need to define a pair of functions that will translate a user utterance into a Haystack query and the Haystack output into an appropriate chatbot response.</p><p>Haystack can also be useful for fallback situations. In cases where the chatbot cannot easily classify the user’s utterance into any of its predefined intents, Haystack can be called to help respond to the utterance which the chatbot would otherwise not know how to deal with.</p><p>The next section will show you the lines of code that are needed to build an augmented chatbot but If you’d like to see one that we prepared earlier, then please check out our reference implementation <a href="https://github.com/deepset-ai/rasa-haystack">here</a>.</p><h3>Implementation</h3><p>The first step is to set up a Haystack service that is accessible via REST API. If you’re new to the concepts in Haystack, we have many resources to get you up to speed. A good starting point would be our <a href="https://haystack.deepset.ai/tutorials/01_basic_qa_pipeline">tutorial</a> or our <a href="https://www.deepset.ai/blog/supercharging-elasticsearch-with-haystacks-neural-qa">introductory article</a> — both will give you an understanding of what lies under the hood. Ultimately the configuration of this question answering system is up to you. It could be a light and efficient system that uses an Elasticsearch database with a faster Reader model. Or it could be a Milvus database that uses a Dense Passage Retriever and a large transformer model as the Reader. Both are possibilities in Haystack. Just make sure to deploy your pipeline with the REST API so that it can be called by Rasa. You can learn to do just this in this <a href="https://www.deepset.ai/blog/building-a-rest-api-for-question-answering-with-haystack">blog article</a>.</p><p>On the Rasa side, you’ll want to start by <a href="https://rasa.com/docs/rasa/installation">installing</a> the package and <a href="https://rasa.com/docs/rasa/command-line-interface/#rasa-init">initializing</a> a new chatbot project. We will then need to make some changes to various configuration files to get our chatbot ready for Haystack. The result of these changes, once again, can be seen in our <a href="https://github.com/deepset-ai/rasa-haystack">reference implementation</a>.</p><p>To start, we will want to define a new intent by providing some example utterances. Here, we will define the knowledge_question intent in the data/nlu.yaml file.</p><pre>nlu: <br>- intent: knowledge_question <br>  examples: | <br>    - Can you look the capital of Norway? <br>    - What is the tallest building in the world? <br>    - Who won the 2018 Champions League? <br>    - Where is Faker from? <br>    - Can you tell me who the Prime Minister of Australia is? <br>    - Why do leaves turn yellow? <br>    - Do you know who stole Christmas? <br>    - When is Easter?</pre><p>Next, you will want to define what action is taken when that intent is identified in the data/rules.yml file.</p><pre>- rule: Query Haystack anytime the user has a direct question for your document base<br>  steps: <br>  - intent: knowledge_question <br>  - action: call_haystack</pre><p>Note that instead of defining the knowledge_question intent, you can also use the fallback intent to trigger Haystack. This way you only query Haystack in situations where none of your defined intents is a match. You will need to add the FallbackClassifier to the pipeline in config.yml .</p><pre>pipeline: <br>   - name: FallbackClassifier <br>     threshold: 0.8 <br>     ambiguity_threshold: 0.1</pre><p>You will also need to add the following to data/rules.yml .</p><pre>- rule: Query Haystack whenever they send a message with low NLU confidence <br>  steps: <br>    - intent: nlu_fallback <br>    - action: call_haystack</pre><p>By default, the Rasa custom action server API is blocked so you will need to uncomment this line from <em>endpoints.yml </em>to get it to work.</p><pre>action_endpoint: <br>  url: &quot;http://localhost:5055/webhook&quot;</pre><p>Finally, we can define an action function that calls the Haystack API and handles the response in actions/actions.py. By default, the Haystack REST API is located on port 8000, hence the url in the code below.</p><pre>class ActionHaystack(Action):</pre><pre>    def name(self) -&gt; Text:<br>        return &quot;call_haystack&quot; </pre><pre>    def run(self, dispatcher: CollectingDispatcher,<br>            tracker: Tracker,<br>            domain: Dict[Text, Any]) -&gt; List[Dict[Text, Any]]:</pre><pre>        url = &quot;http://localhost:8000/query&quot;<br>        payload = {&quot;query&quot;: str(tracker.latest_message[&quot;text&quot;])} <br>        headers = { <br>            &#39;Content-Type&#39;: &#39;application/json&#39; <br>        } <br>        response = requests.request(&quot;POST&quot;, url, headers=headers, json=payload).json() </pre><pre>        if response[&quot;answers&quot;]: <br>            answer = response[&quot;answers&quot;][0][&quot;answer&quot;] <br>        else: <br>            answer = &quot;No Answer Found!&quot; <br>  <br>        dispatcher.utter_message(text=answer) </pre><pre>        return []</pre><p>You will also want to declare this new action in the domain.yml file.</p><pre>actions: <br>  - call_haystack</pre><h3>Running the Chatbot</h3><p>Whenever changes are made to your Rasa project, you will want to retrain your chatbot.</p><pre>rasa train</pre><p>You will also need to start the Rasa action server.</p><pre>rasa run actions</pre><p>To start interacting with the bot you have created use the following command.</p><pre>rasa shell</pre><p>Now you can start talking to the Haystack enabled chatbot!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/572/1*wl3Y3NBKeZlfs5LZ4F7BCA.png" /></figure><p>Alternatively, you can use <a href="https://rasa.com/docs/rasa-x/">Rasa X</a> which allows the chatbot to be run in a GUI. While interacting with you chatbot in this interface, you will also see the intent being assigned to each message as well as the action being taken.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/774/1*sX3UKpBrFOGeS1cIkmB5SA.png" /></figure><h3>Further Improvements</h3><p>Congratulations if you made it this far! You now have a Rasa chatbot with the question answering power of Haystack! That said, there are still many directions you can go in to make this chatbot even better.</p><p>For example, you could create different intents for questions about different domains. You might want to route your finance questions to a Haystack pipeline that is geared towards finance while questions about legal issues are passed on to a system that is loaded with and trained on legal data. This starts with giving more examples of new intents and retraining the Rasa model to get good at differentiating between the topics. Each topic should then have its own action, each of which should call a different Haystack service.</p><p>Also, when we were defining the function that parses Haystack’s output and composes a chatbot response, we were only making use of the answer string. In fact, Haystack returns a lot more useful information such as the id of the document from which the answer came, the context around the answer string and also the confidence with which it made this prediction. Presenting these extra fields to the user enables them to navigate to the answering document, understand why that specific answer is being returned and find more relevant information.</p><h3>Conclusion</h3><p>Chatbot integration is one of the most requested features in the Haystack <a href="https://github.com/deepset-ai/haystack">repository</a>. It’s not at all surprising considering how much both technologies enable humans to interact with data through a natural language interface. Now that it is here, we hope to see many more chatbots in the powerful combination of <a href="https://rasa.com/">Rasa</a> and <a href="https://haystack.deepset.ai/">Haystack</a>. We’re looking forward to seeing what you’ll build with it!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=eb0223d725a0" width="1" height="1" alt=""><hr><p><a href="https://medium.com/deepset-ai/build-smart-conversational-agents-with-the-latest-chatbot-and-question-answering-technology-eb0223d725a0">Build Smart Conversational Agents with the Latest Chatbot and Question Answering Technology</a> was originally published in <a href="https://medium.com/deepset-ai">deepset-ai</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Parameter-Tweaking: Get Faster Answers from Your Haystack Pipeline]]></title>
            <link>https://medium.com/deepset-ai/parameter-tweaking-get-faster-answers-from-your-haystack-pipeline-b94f0f528111?source=rss-51b74d7c71f4------2</link>
            <guid isPermaLink="false">https://medium.com/p/b94f0f528111</guid>
            <category><![CDATA[question-answering]]></category>
            <category><![CDATA[haystack]]></category>
            <category><![CDATA[semantic-search]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Branden Chan]]></dc:creator>
            <pubDate>Mon, 28 Jun 2021 10:57:40 GMT</pubDate>
            <atom:updated>2023-01-16T09:44:58.307Z</atom:updated>
            <content:encoded><![CDATA[<h4>HAYSTACK PERFORMANCE</h4><p><em>This article is the first in our series on optimizing your Haystack question answering system. We’ll link to the other articles here as they go online:</em></p><ul><li><em>Parameter-Tweaking</em></li><li><a href="https://www.deepset.ai/blog/accelerate-your-qa-system-with-gpus"><em>Accelerate Your QA System with GPUs</em></a></li><li><a href="https://www.deepset.ai/blog/picking-the-right-haystack-reader-model"><em>Picking The Right Reader Model</em></a></li><li><a href="https://www.deepset.ai/blog/metadata-filtering-in-haystack"><em>Metadata Filtering</em></a></li></ul><p>There are many ways you can optimize a parameter-rich system like a Haystack question answering pipeline. Specifically, the length of your documents and your setting for the <em>top_k_retriever</em> parameter can have an enormous impact on system performance. In this post, we’ll show you the most important tweaks to speed up your Haystack question answering system without sacrificing the quality of its results.</p><h3>The Haystack Pipeline</h3><p>To understand the impact of these parameters, let’s quickly look at the four stages of a Haystack question answering (QA) pipeline:</p><ol><li><a href="https://docs.haystack.deepset.ai/docs/preprocessor">Preprocessing</a>: Here, we read in our document collection and convert it to a list of dictionaries. Additionally, we may use the PreProcessor class to modify our data.</li><li><a href="https://docs.haystack.deepset.ai/docs/document_store">Document Store</a>: We pick the database that best fits our use case, and feed our preprocessed documents to the database via a document store object.</li><li><a href="https://docs.haystack.deepset.ai/docs/retriever">Retriever</a>: Choosing between “sparse vs. dense” retrieval methods comprises a crucial architectural decision. In this step, we index our database by means of the retriever, thereby making our documents searchable. Sparse methods are faster at indexing, while dense ones are powered by transformer-based language models.</li><li><a href="https://docs.haystack.deepset.ai/docs/reader">Reader</a>: In this step, we pick a model to perform question answering on the documents preselected by the retriever.</li></ol><p>The reader and retriever are commonly chained together by use of a <a href="https://docs.haystack.deepset.ai/docs/pipelines">pipeline</a>. This allows the user to send only one command to ask a question, which then triggers actions from both the reader and retriever.</p><p>Both the preprocessing and retrieval steps offer opportunities for optimization, the most important of which we’ll show you in this article. Let’s start off with document length.</p><h3>Increasing Pipeline Speed via Document Length Optimization</h3><p>A document in the question answering context is simply a passage of text of arbitrary length. Adjusting that length to the optimal value can go a long way to improving your system’s speed. The trick is to split your knowledge base into documents that are both a) neither too short nor too long, and b) as informative as possible.</p><p>The <a href="https://haystack.deepset.ai/tutorials/08_preprocessing">PreProcessor</a> class will help us segment our documents into passages of the optimal length. Let’s have a look at the parameters defined in its <em>__init__</em> function:</p><pre><strong>class</strong> <strong>PreProcessor</strong>(BasePreProcessor):<br>    <strong>def</strong> <strong>__init__</strong>(<br>        self,<br>        clean_whitespace: bool = True,<br>        clean_header_footer: bool = False,<br>        clean_empty_lines: bool = True,<br>        split_by: str = “word”,<br>        split_length: int = 1000,<br>        split_overlap: int = 0,<br>        split_respect_sentence_boundary: bool = True)</pre><p>We combine <em>split_by</em> and <em>split_length</em> to define our document length. <em>split_by</em> sets the unit for defining split length — e.g., “word,” “sentence,” or “passage” (defined as a paragraph). Since sentence and paragraph length may fluctuate widely, splitting by word is the best choice for most use cases. <em>split_length</em> then determines the number of words contained in a document.</p><p>However, splitting at the word level does not come without issue. A sentence may lose its context when split somewhere in the middle. To avoid any such unwanted behavior, you may turn on the parameter <em>split_respect_sentence_boundary</em>. When set to “True,” this parameter ensures that splitting only occurs at sentence boundaries. At the same time, it prevents longer sentences from being split up, even if they exceed the value defined for <em>split_length</em>.</p><p>Another method for minimizing the risk of losing the syntactic context of a sentence after splitting is to define an overlap of a few words between documents (the “sliding window” method). By setting the <em>split_overlap</em> parameter to a positive integer, the words at the seam of a split are included in both resulting chunks. Keep in mind though that a high overlap will inflate your corpus as overlapping sections are featured twice.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_rYBbX_sLu-HrixXZpDn5g.png" /><figcaption>How the split_overlap parameter works</figcaption></figure><h4>Document-splitting with dense methods</h4><p>Dense retrieval methods like <a href="https://haystack.deepset.ai/tutorials/06_better_retrieval_via_embedding_retrieval">DPR</a> (dense passage retrieval) use encoders both during indexing and querying. A transformer-based model processes documents and encodes them as dense vectors of a fixed length (usually 768).</p><p>The dense passage retriever has a limit for the number of tokens it can process at once, which is set to 256 by default. The encoder will simply shorten any passages exceeding the limit. Thus, to ensure that all of our passage contents get encoded, we’ll need to avoid using overly long passages. We recommend 100 words per passage, which is the document length that the DPR model is trained on:</p><pre>processor = PreProcessor(<br>    split_by=’word’,<br>    split_length=100,<br>    split_respect_sentence_boundary=<strong>True</strong>)</pre><h4>Document-splitting with sparse methods</h4><p>Sparse retrieval methods do not use encoders and are therefore much faster than dense retrievers. These methods represent documents as sparse vectors (made up of mostly zeros) that consist of the weighted counts of their words.</p><p>Since sparse vectors are not enriched with the linguistic knowledge of deep language models, they have no understanding of semantic similarity between words. Therefore, when working with a sparse method, we rely entirely on the actual query keywords appearing in the texts. This calls for documents that are longer than the dense representations described above. But documents should not be too long — otherwise the rest of a document may wash out our passage of interest. We recommend a <em>split_length</em> of 500 words when working with a retriever-reader pipeline.</p><p>Shorter documents also ease the reader’s computational load, especially when using a sparse retrieval method. While results may vary between runs, our experiments show that doubling the document length from 500 to 1000 words slows down the query pipeline by about fifty percent. For instance, a query might take one second to run on the shorter documents and 1.5 seconds on the longer ones. This can make a big difference when you want to scale the number of queries to your system.</p><p>A variable with an even bigger impact on your system’s runtime is the number of documents that the reader receives from the retriever. We tweak it using the <em>top_k_retriever</em> parameter.</p><h3>Get Faster Answers by Adjusting top_k_retriever</h3><p><em>top_k_retriever</em> gets its name from the variable “k” which is often used for integer hyperparameters in machine learning contexts. To understand why this parameter has such a high impact on our system’s performance, let’s briefly recap what we know about both the retriever and reader, as well how the two interact.</p><h4>Retriever vs. reader</h4><p>The retriever and reader are both indispensable elements of a question answering pipeline. The retriever’s role is to sift through a large amount of documents by performing quick vector computations to find the best candidate documents for the reader. The reader then looks at only these candidates, passing them through a pre-trained transformer model and determining which answers best fit a query. If the retriever were a large-scale excavator, the reader would be a pickaxe.</p><p>Speed is a major point of comparison for the two components. Because the reader is so much slower than the retriever, the number of documents that the retriever ultimately selects as answer candidates can make a big difference. We adjust that number by way of the <em>top_k_retriever</em> parameter, either when running the pipeline or upon the retriever’s initialization (where it’s simply called <em>top_k</em>).</p><h4>Working with <em>top_k_retriever</em></h4><p>If your system is too slow, try lowering <em>top_k_retriever</em>. In the below example, we’ll use a subset of the Harry Potter Wiki (from <a href="https://harrypotter.fandom.com/">https://harrypotter.fandom.com</a> under CC-BY-SA license). To get the best answer possible, we’ll set <em>top_k_retriever</em> to 100 documents. Let’s put everything together and ask our question:</p><pre>query = “Who is Dobby’s owner?”<br>timed_run = timer(pipeline.run)<br>answer = timed_run(query=query, top_k_retriever=100, top_k_reader=1)<br>&gt;&gt;&gt; Time elapsed: 5.86 seconds</pre><pre>short_answer(answer)<br>&gt;&gt;&gt; Answer: The Malfoy family, with 84 percent probability.</pre><p>Our system gave a reasonable answer, and it only took seconds. But what if I told you that we could have obtained an equally good result in a fraction of the time? Let’s see what happens when we set <em>top_k_retriever </em>— the number of documents seen by the reader — from 100 to 10.</p><pre>answer = timed_run(query=query, top_k_retriever=10, top_k_reader=1)<br>&gt;&gt;&gt; Time elapsed: 0.75 seconds</pre><pre>short_answer(answer)<br>&gt;&gt;&gt; Answer: The Malfoy family, with 84 percent probability.</pre><p>We got the same answer as before — but with only 10 documents to look at, our reader required much less processing time. This amounts to a dramatic increase in performance when scaling the number of queries. Have a look at <a href="https://haystack.deepset.ai/tutorials/05_evaluation">this tutorial</a> to learn more about evaluating your system.</p><p>Conversely, if you’re unhappy with your system’s answers and time is not an issue, you can try increasing <em>top_k_retriever</em> to feed a larger variety of answer candidates to the reader.</p><p>Because <em>top_k_retriever</em> so heavily impacts the reader’s processing time, some may be thrown off by its name, especially seeing as there’s a <em>top_k_reader</em> parameter, too (the number of answers returned by the reader). But it only takes some time to get used to — we promise.</p><h3>Start Optimizing Your Haystack Pipeline</h3><p>In this article, you learned how to increase your system’s speed by tweaking the <em>top_k_retriever</em> parameter and hitting the right document length. Keep an eye out for our next post, where we’ll show you how to work with metadata to get even faster answers.</p><p>To get started with <a href="https://www.deepset.ai/haystack">Haystack</a> now, check out our <a href="https://github.com/deepset-ai/haystack">GitHub repository</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b94f0f528111" width="1" height="1" alt=""><hr><p><a href="https://medium.com/deepset-ai/parameter-tweaking-get-faster-answers-from-your-haystack-pipeline-b94f0f528111">Parameter-Tweaking: Get Faster Answers from Your Haystack Pipeline</a> was originally published in <a href="https://medium.com/deepset-ai">deepset-ai</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Supercharging Elasticsearch with Haystack’s Question Answering]]></title>
            <link>https://medium.com/deepset-ai/supercharging-elasticsearch-with-haystacks-neural-qa-101c88184f2b?source=rss-51b74d7c71f4------2</link>
            <guid isPermaLink="false">https://medium.com/p/101c88184f2b</guid>
            <category><![CDATA[haystack]]></category>
            <category><![CDATA[neural-search]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Branden Chan]]></dc:creator>
            <pubDate>Thu, 03 Jun 2021 10:45:15 GMT</pubDate>
            <atom:updated>2023-01-16T09:50:47.295Z</atom:updated>
            <content:encoded><![CDATA[<h4>Start using the latest in NLP driven search technology today!</h4><p>By now we’re all aware of the power of data. Yet having access to data is only half the battle — what’s equally important is our ability to make sense of it. That’s why we created <a href="https://www.deepset.ai/haystack">Haystack</a>, a framework for building open-domain question answering systems that will allow your customers to ask questions in natural language, and receive answers right away.</p><p>Our technology lets you harness the power of BERT and other Transformer-based language models with the speed of Elasticsearch distributed storage. If you’re unfamiliar with Elasticsearch, you can check out the <a href="https://www.deepset.ai/blog/supercharging-elasticsearch-with-haystacks-neural-qa">FAQ</a> at the bottom of the page. Read on to learn more about using Haystack with Elasticsearch.</p><h3>Question Answering with Haystack</h3><p>Natural language is unstructured data, meaning that it’s tricky to handle computationally. Luckily, the recent advances in natural language processing (NLP) have brought about a number of ingenious solutions for representing language data.</p><p>Deep neural language models like BERT and RoBERTa are trained to represent words as complex vectors known as word embeddings. These vectors are used to perform various NLP tasks, often by stacking additional layers on top of the word embedding network. That, in short, is how language models may be used in the context of question answering (QA).</p><p>Haystack’s Retriever-Reader pipelines perform what’s known as “open-domain extractive QA.” For any query, they identify a number of answer candidates from a text collection, which are ranked according to their matching probability. The pipeline’s output, then, is a list of passages most likely to answer the given question — which might include no answer at all, if none passes the matching threshold. Check out <a href="https://www.deepset.ai/blog/modern-question-answering-systems-explained">our blog post</a> for more detail on how QA systems work under the hood.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NdGzmuuxC_M4GEIXeDjOWg.png" /><figcaption>A Retriever-Reader pipeline</figcaption></figure><p>Creating your own end-to-end QA pipeline with Haystack is easy, with many different configurations at your disposal. Our QA systems consist of three main customizable modules that are invoked in the following order:</p><ol><li><a href="https://docs.haystack.deepset.ai/docs/document_store">DocumentStore</a>: This is where you keep your documents to be searched by the model. Haystack has different storage solutions depending on the use case, and your choice will also depend on whether you opt for sparse or dense document representations. In this tutorial, we’ll be using <a href="https://docs.haystack.deepset.ai/docs/document_store#elasticsearch">Elasticsearch</a> as our document store. During initialization, you may create a new ES cluster, or connect to an existing one by specifying the host and port in the object’s parameters. If you’re using standard authentication with HTTP, you may pass on your username and password to the store using the respective parameters.</li><li><a href="https://docs.haystack.deepset.ai/docs/retriever">Retriever</a>: It’s not feasible to apply the computationally expensive QA model to all documents, each time we ask a question. The Retriever module acts as a quick filter for finding the best candidate documents by calculating the similarity between query and document vectors. Those vectors can be sparse (working with simple word count functions like tf-idf or BM25) or dense (using transformer-based embedding models). While sparse representations work out-of-the-box, dense ones rely on pretrained language models that may be adapted to your data. If you connected to an existing ES cluster in the previous step, and choose to work with a dense retriever, you’ll have to update your database with the dense vector representations.</li><li><a href="https://docs.haystack.deepset.ai/docs/reader">Reader</a>: After applying the retriever sieve, we’re ready to run the selected documents through our fine-meshed QA model. In this module, you may choose between FARMReader and TransformersReader (learn about their differences <a href="https://docs.haystack.deepset.ai/docs/reader">here</a>). You’ll also need to decide on a language model, which will depend on your computational resources, your use case — and of course, the language that you work with. You’ll also need to consider any time constraints. For example, the ALBERT XXL model is best in class in terms of performance, but it also requires much more computational power than RoBERTa. On the other hand, the MiniLM model manages to be twice as fast as the base model by sacrificing a bit of accuracy.</li></ol><p>In the next sections, we’ll show you how to adapt the components of a Haystack question-answering pipeline to your use case.</p><h3>Extending Elasticsearch’s Capabilities with Haystack</h3><p>Elasticsearch (ES) is a NoSQL database and search engine that stores its documents in a decentralized manner, distributing them over several nodes. In addition to its distributed and schemaless nature, ES offers solutions for querying natural language documents. Unstructured text documents are reverse-indexed, allowing for quick matching to search queries. Organizations of all sizes and purposes use ES as a search and analytics engine and to manage their documents.</p><p>However, despite using some preprocessing techniques that make it possible to, say, match a singular search term to its plural form in a document, ES queries are still keyword-based. In other words, they do not understand syntactic or semantic subtleties unlike semantic search. For instance, as any Game of Thrones fan would attest, there’s a big difference between the two questions, “Who does Daenerys kill?” and “Who kills Daenerys?”</p><blockquote><strong>Spoiler alert: </strong>the examples in this article contain answers that may spoil your Game of Thrones experience.</blockquote><p>This is where BERT and similar models come in. Combining complex transformer-based architectures with huge amounts of training data, these models understand language at a much deeper level than simple keyword combinations. By endowing the models with Elasticsearch’s quick lookup and storage capabilities, Haystack makes it possible to use natural language queries to search an ES database.</p><h4>Running Haystack with Elasticsearch: a demonstration</h4><p>To follow along with this example, make sure that you have access to a GPU. While some Haystack models can run on a CPU, it will slow them down considerably. <a href="https://colab.research.google.com/">Google’s Colaboratory</a> (or Colab) provides free GPU access, and you may want to use it to try out the models. If you want to see how we handle the setup and installation in Colab, check out <a href="https://colab.research.google.com/github/deepset-ai/haystack/blob/master/tutorials/Tutorial1_Basic_QA_Pipeline.ipynb">this tutorial</a>. For our present example, we’re using a GPU-enabled cloud compute instance and will run all commands through SSH.</p><h4>Setting up Haystack and Elasticsearch</h4><p>We’ll also be using Docker to create an Elasticsearch instance. To install Docker, follow the <a href="https://docs.docker.com/engine/install/">instructions</a> for your operating system.</p><p>Once Docker is set up, we use it to pull the ES image and start a single-node cluster:</p><pre>docker pull docker.elastic.co/elasticsearch/elasticsearch:7.9.2<br>docker run -d -p 9200:9200 -e &quot;discovery.type=single-node&quot; elasticsearch:7.9.2</pre><p>We start out by creating a virtual environment with conda and then activating it:</p><pre>conda create -n haystack python=3.7<br>conda activate haystack</pre><p>Within our virtual env, we install the Haystack framework from <a href="https://pypi.org/project/farm-haystack/">PyPI</a>:</p><pre>pip install farm-haystack</pre><h4>Initializing the question answering pipeline</h4><p>Let’s fire up Python and import all relevant functions and classes to build our QA pipeline:</p><pre>from haystack.document_store import ElasticsearchDocumentStore<br>document_store = ElasticsearchDocumentStore()</pre><pre>document_store.delete_all_documents()</pre><p>We included the last line to get rid of all leftover documents that might still occupy our storage from the last run. This step is important to avoid receiving duplicate answers.</p><blockquote>Note that the <em>delete_all_documents()</em> command is scheduled to be replaced by the more versatile <em>delete_documents()</em>.</blockquote><p>It’s now time to fill our document store with documents! ES expects a list of dictionaries as input. For this example, we’ll use our trusted Wikipedia subcorpus consisting of 183 articles about the Game of Thrones universe. The following functions are needed to shape our data into a format that ES likes:</p><pre>from haystack.preprocessor.utils import convert_files_to_dicts, fetch_archive_from_http<br>from haystack.preprocessor.cleaning import clean_wiki_text</pre><p>If you’d like to view the articles to get a feel for the documents we’re working with, simply download and unzip the file following the link below:</p><pre>doc_dir = “got_texts”<br>got_wiki_url = “https://s3.eu-central-1.amazonaws.com/deepset.ai-farm-qa/datasets/documents/wiki_gameofthrones_txt.zip&quot;</pre><pre>fetch_archive_from_http(url=got_wiki_url, output_dir=doc_dir)</pre><pre>got_dicts = convert_files_to_dicts(dir_path=doc_dir, clean_func=clean_wiki_text, split_paragraphs=True)</pre><p>How many documents have we got in our list of dictionaries?</p><pre>len(got_dicts)</pre><pre>&gt;&gt;&gt; 2497</pre><p>We get this result because we split the documents into paragraphs. Note that a “document” in ES is not necessarily equal to a document in the real world. Breaking longer documents up into shorter passages eases search and reduces computation times.</p><pre>document_store.write_documents(got_dicts)</pre><p>Now that we’ve read our text collection into the document storage, we’re ready to build a pipeline. It’s at this point that we actually have to make some decisions about the nature of the models that make up our system.</p><p>For our use case, we’ll stick to the sparse <a href="https://docs.haystack.deepset.ai/docs/retriever#bm25-recommended">ElasticsearchRetriever</a>:</p><pre>from haystack.retriever.sparse import ElasticsearchRetriever, TfidfRetriever</pre><pre>retriever = ElasticsearchRetriever(document_store=document_store)</pre><p>In the next step, we’ll have to decide on a Reader class and which language model to use. We’ll be using our very own <a href="https://docs.haystack.deepset.ai/docs/reader">FARMReader</a> and will load it with RoBERTa’s language model which has been pretrained on the <a href="https://rajpurkar.github.io/SQuAD-explorer/">SQuAD v2</a> question answering dataset. RoBERTa is a solid choice when working on a GPU. But if you only have access to a CPU, you might want to switch to the light-weight MiniLM model instead. Let’s import the model:</p><pre>from haystack.reader import FARMReader</pre><pre>my_model = “deepset/roberta-base-squad2”<br>reader = FARMReader(model_name_or_path=my_model, use_gpu=True)</pre><p>For every query, we want to run the Retriever first, followed by the Reader. With the help of Haystack’s pipeline module, we can simply chain the two steps together. Let’s also import the handy <em>print_answers()</em> function for pretty-printing the results of our query:</p><pre>from haystack.pipeline import ExtractiveQAPipeline<br>from haystack.utils import print_answers</pre><pre>pipe = ExtractiveQAPipeline(reader, retriever)</pre><h4>Running queries</h4><p>Will we finally be able to get an answer to our question? Let’s ask:</p><pre>question = “Who kills Daenerys?”<br>answer = pipe.run(query=question, top_k_retriever=10, top_k_reader=2)<br>print_answers(answer, details=”minimal”)</pre><p>This returns:</p><pre>[<br>    {‘answer’: ‘Jon’,<br>     ‘context’: ‘rion denounces Daenerys and is imprisoned for treason to await execution. Jon, unable to stop her, kills Daenerys. Bran Stark is proclaimed king, allo’<br>    },<br>    { ‘answer’: ‘Daario’,<br>      ‘context’: ‘aenerys that she must send a champion to fight the Champion of Meereen. Daario is selected and he kills the Champion. Daenerys tells slaves from Meere’<br>    }<br>]</pre><p>What if we wanted to know more about these answers — such as getting the model’s degree of certainty as to its predictions? Setting the <em>details</em> parameter of <em>print_answers()</em> to <em>all</em> gives us all the information we need. Because this setting produces quite a long output, let’s just look at the first result:</p><pre>{ <br>    ‘answer’: ‘Jon’,<br>    ‘context’: ‘rion denounces Daenerys and is imprisoned for treason to await execution. Jon, unable to stop her, kills Daenerys. Bran Stark is proclaimed king, allo’,<br>    ‘document_id’: ‘eb7f0b2f-688a-4134-beb0-ee6353309158’,<br>    ‘meta’: { <br>        ‘name’: ‘360_List_of_Game_of_Thrones_episodes.txt’<br>    },<br>    ‘offset_end’: 77,<br>    ‘offset_end_in_doc’: 751,<br>    ‘offset_start’: 74,<br>    ‘offset_start_in_doc’: 748,<br>    ‘probability’: 0.9234894514083862,<br>    ‘score’: 9.971123695373535<br>},</pre><p>The model is 92 % confident that Jon is Daenerys’ killer. Indeed, that’s the value of greatest interest at this point. Let’s write a quick function that will only return the answers and their degrees of probability:</p><pre>def short_answer(answers):<br>    for i, answer in enumerate(answers[‘answers’]):<br>        print(‘Answer no. {}: {}, with {} percent probability’.format(i, answer[‘answer’], round(answer[‘probability’] * 100)))</pre><p>In the previous example, our model seemed pretty certain. Let’s do a quick sanity check and see if our model indeed understands that the following question is different from the previous one:</p><pre>question = “Who did Daenerys kill?”<br>short_answer(answer)</pre><pre>&gt;&gt;&gt; Answer no. 0: Kraznys, with 43 percent probability<br>&gt;&gt;&gt; Answer no. 1: Thirteen, with 22 percent probability</pre><p>It definitely understood the difference between Daenerys the victim and Daenerys the agent. But the model doesn’t have much confidence in these answers. To increase the quality of the model’s output, we may extend the search space: by hiking up the <em>top_k_retriever</em> parameter, we increase the number of documents that our Reader sees.</p><p>Be aware that this decision will have an impact on our search time. Because the Reader is much more computationally expensive than the Retriever, passing more documents to the Reader means that our program will take significantly longer to run. To learn more about how your choice of parameters might impact the system’s speed, check out <a href="https://docs.haystack.deepset.ai/docs/optimization">our article</a> on optimization.</p><p>Let’s see how a higher number of potential answer passages impacts our output:</p><pre>question = “Who did Daenerys kill?”<br>answer = pipe.run(query=question, top_k_retriever=100, top_k_reader=5)<br>short_answer(answer)</pre><pre>&gt;&gt;&gt; Answer no. 0: slave masters, with 98 percent probability<br>&gt;&gt;&gt; Answer no. 1: thousands of innocents, with 73 percent probability<br>&gt;&gt;&gt; Answer no. 2: Samwell Tarly’s father and brother, with 90 percent probability<br>&gt;&gt;&gt; Answer no. 3: all slavers, with 40 percent probability<br>&gt;&gt;&gt; Answer no. 4: Khal Moro’s bloodriders, with 50 percent probability</pre><p>This looks great! Not only do these answers better address our question, the system has greater confidence in them. Sometimes it makes sense to sacrifice speed for accuracy.</p><p>As we’ve established, being able to return <em>None</em> in the absence of any solid answer candidates is part of a trustworthy QA system. This makes sense — after all, we’d rather have someone admit that they don’t know the answer to a question, rather than giving us incorrect information. Let’s see what happens when we ask about entities that are foreign to the GoT universe:</p><pre>question = “Where is Warsaw?”<br>answer = pipe.run(query=question, top_k_retriever=10, top_k_reader=1)<br>short_answer(answer)</pre><pre><strong>&gt;</strong>&gt;&gt; Answer no. 0: Iron Islands, with 1 percent probability</pre><p>It’s pretty unlikely that our GoT corpus contains any information about the Polish capital. However, our system still returns low-probability answers. To change that, we need to enable the <em>return_no_answer</em> parameter when we initialize the reader:</p><pre>reader = FARMReader(model_name_or_path=my_model, use_gpu=True, return_no_answer=True)</pre><p>When we run the same query through the reader, this is the output:</p><pre>&gt;&gt;&gt; Answer no. 0: None, with 67 percent probability</pre><p>In addition to enabling <em>return_no_answer</em>, we may set the <em>no_ans_boost</em> parameter to some positive integer value to nudge our models towards returning more <em>None</em> answers. This is useful if you want your system to only return an answer when it’s reasonably certain.</p><h3>Haystack: A Modern Solution for Semantic Search and Question Answering</h3><p>Looking to integrate question answering into your applications? At <a href="https://www.deepset.ai/">deepset</a>, we’re happy to guide you through the process!</p><p>To learn more about how to customize your question answering systems with the latest neural language models, check out the <a href="https://haystack.deepset.ai/">Haystack documentation</a>.</p><p>Find <a href="https://github.com/deepset-ai/haystack">Haystack</a> on GitHub and give us a star if you find it useful!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=101c88184f2b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/deepset-ai/supercharging-elasticsearch-with-haystacks-neural-qa-101c88184f2b">Supercharging Elasticsearch with Haystack’s Question Answering</a> was originally published in <a href="https://medium.com/deepset-ai">deepset-ai</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Enabling German Neural Search: Announcing GermanQuAD and GermanDPR]]></title>
            <link>https://medium.com/deepset-ai/enabling-german-neural-search-announcing-germanquad-and-germandpr-3470bd52a33b?source=rss-51b74d7c71f4------2</link>
            <guid isPermaLink="false">https://medium.com/p/3470bd52a33b</guid>
            <category><![CDATA[question-answering]]></category>
            <category><![CDATA[neural-search]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[transformers]]></category>
            <dc:creator><![CDATA[Branden Chan]]></dc:creator>
            <pubDate>Mon, 03 May 2021 09:48:53 GMT</pubDate>
            <atom:updated>2021-11-08T15:33:46.146Z</atom:updated>
            <content:encoded><![CDATA[<h4>Two new datasets, three models and a paper to push forward German NLP</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2eZdtElTUrzMeq82jguZhw.png" /></figure><p>A model is only as good as the data it is fed. Anyone in the field of machine learning knows this. Great datasets like SQuAD and Natural Questions are the direct catalysts of the breakthroughs which have made neural search as powerful and flexible as it is today. Inspired by their successes, we set to work, building our own human annotated QA and Passage Retrieval datasets in the German language and are very happy to announce the release of <a href="https://www.deepset.ai/germanquad">GermanQuAD and GermanDPR</a>! We can confidently say that the models that we trained on this data are the first to perform well on these tasks in German and we’d like to encourage you to try them out!</p><p>We learned a lot in the process of creating these resources and are committed to sharing our learnings so that other teams can more easily build semantic search technologies for their language and domain. Everything we used to create them is open source: from the <a href="https://www.deepset.ai/german-bert">German pre-trained language models</a>, to our frameworks <a href="https://github.com/deepset-ai/FARM">FARM</a> and <a href="https://github.com/deepset-ai/haystack">Haystack</a> where we trained and evaluated the models and even the <a href="https://github.com/deepset-ai/haystack/tree/master/annotation_tool">annotation tool</a> which our labelers used. By sharing our experience, we hope that other teams might find it less intimidating to create their own datasets and train models!</p><h3><strong>GermanQuAD: Dataset</strong></h3><p>The GermanQuAD dataset adopts the format and style of SQuAD in that it tackles extractive QA, the passages all come from Wikipedia and all answers are single text spans.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/962/0*A_u_YcUn7t3IYPvf" /><figcaption>A simplified question, passage, answer triple from the GermanQuAD dataset</figcaption></figure><p>However, in various areas, we made some crucial design decisions that distinguish GermanQuAD from SQuAD. For example, we ensured that all questions are <strong>self sufficient</strong>. That is, that they contain enough information that they could be unambiguously answered even in an open-domain setting. We also encouraged annotators to write more questions which required answers that are <strong>longer than just a single phrase</strong>, so that models are better tuned to make predictions of different lengths.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/926/1*KasEy7_AlNA48iyaQCKwdw.png" /><figcaption>The number of passages, questions and answers in the train and test sets of GermanQuAD</figcaption></figure><p>In the creation of the dataset, we also made sure to remove any duplicate question answer pairs and <strong>avoid cross-contamination</strong> between train and test set, a hidden problem that has been identified in other popular QA datasets. And we also asked our annotators to <strong>reformulate questions</strong> which share a lot of lexical overlap with the answer and tested the impact of this style of annotation generation.</p><p>These improvements were facilitated by the fact that we gathered a team of <strong>in-house annotators</strong> who we had regular close contact with rather than relying on crowd-sourced workers. Through regular review sessions, we got annotators to discuss and resolve discrepancies in their annotations.</p><h3><strong>GermanQuAD: Models</strong></h3><p>Using the dataset, we trained and evaluated two QA models, which are already available for you to download and use through Hugging Face’s Model Hub. Just by providing the model names <em>gelectra-base-germanquad</em> and <em>gelectra-large-germanquad </em>you can already start using them in Haystack to perform document search or to empower your German chatbot with QA abilities.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/858/0*tWIxE-5fjSPa_Ebv" /><figcaption>Comparison of our new German QA models against XLM-R and the human baseline when evaluated on the GermanQuAD dataset. Top1 Acc is 1 for a given sample if there is any overlap between the prediction and the answer.</figcaption></figure><p>Both of these models use our pre-trained GELECTRA models as their starting point. They are trained on SQuADtranslate, a machine translated version of SQuAD, and then on GermanQuAD. This second stage of training brings a 5–9% absolute increase in Top-1 accuracy, thus highlighting the quality of GermanQuAD’s human annotated data. You can find more details around the training setup in their model cards of the <a href="https://huggingface.co/deepset/gelectra-base-germanquad">base</a> and <a href="https://huggingface.co/deepset/gelectra-large-germanquad">large</a> models.</p><h3><strong>GermanDPR: Dataset</strong></h3><p>Strong retrieval is crucial to an open-domain search system and the <a href="https://github.com/facebookresearch/DPR">Dense Passage Retrieval</a> method designed by the team at Facebook research once again highlighted the transformer’s potential to revolutionize all parts of the search pipeline.</p><p>Inspired by its successes, we saw the chance for us to create a German DPR dataset based off of GermanQuAD. Since the questions in the QA dataset are <strong>self sufficient</strong>, we could treat them as queries in a retrieval setting, and use the passages that contain the answer as gold label documents to be retrieved. After a few steps of preprocessing, we were left with a dataset of about 10k samples spread over the train and test splits.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*RoZaSkvWgN3ZPrZw" /><figcaption>The same sample as the GermanQuAD example above, but converted to the DPR format of query, positive passage (blue) and negative passages (grey)</figcaption></figure><p>Following the original DPR paper, we also provided hard negative passages to each query by using BM25 to find the most relevant passages to that query, which don’t contain the answer string, and come from a wikipedia page that was not used in GermanQuAD.</p><h3><strong>GermanDPR: Model</strong></h3><p>Using this new retrieval dataset, we trained one of the first non-English DPR model, which is also already available for you to try on the Hugging Face model hub under the names <em>deepset/gbert-base-germandpr-question_encoder</em> and <em>deepset/gbert-base-germandpr-ctx_encoder</em>! It is based on the GBERT-base pretrained language model and it is trained using two hard negatives and also in-batch negatives. Our experiments show that this model can perform significantly better than BM25. If you’d like more details about its parameters, its training regime, and the correlation between number of samples and model performance, have a look at the model cards <a href="https://huggingface.co/deepset/gbert-base-germandpr-ctx_encoder">here</a> and <a href="https://huggingface.co/deepset/gbert-base-germandpr-question_encoder">here</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/920/1*cWRe955JseszVOOceXgAvg.png" /><figcaption>Recall and Mean Average Precision percentage metrics of our GermanDPR model vs BM25 when evaluated on the test set of our new GermanDPR dataset</figcaption></figure><p><strong>Conclusion</strong></p><p>deepset is committed to showing that NLP does not have to imply English NLP. Our German QA and DPR <a href="https://www.deepset.ai/models">resources</a> are available for you to use already and we hope they can form the basis of your neural search systems. We learned a lot on this journey and we hope that by sharing our experience, data and models, we might be able to empower other teams to build datasets and train models for their language and domain!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3470bd52a33b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/deepset-ai/enabling-german-neural-search-announcing-germanquad-and-germandpr-3470bd52a33b">Enabling German Neural Search: Announcing GermanQuAD and GermanDPR</a> was originally published in <a href="https://medium.com/deepset-ai">deepset-ai</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Semantic Search with Milvus, Knowledge Graph QA, Web Crawlers and more!]]></title>
            <link>https://medium.com/deepset-ai/semantic-search-with-milvus-knowledge-graph-qa-web-crawlers-and-more-837451eae9fa?source=rss-51b74d7c71f4------2</link>
            <guid isPermaLink="false">https://medium.com/p/837451eae9fa</guid>
            <category><![CDATA[knowledge-graph]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[semantic-search]]></category>
            <category><![CDATA[nlproc]]></category>
            <category><![CDATA[question-answering]]></category>
            <dc:creator><![CDATA[Branden Chan]]></dc:creator>
            <pubDate>Tue, 13 Apr 2021 14:43:20 GMT</pubDate>
            <atom:updated>2023-01-13T12:23:30.982Z</atom:updated>
            <content:encoded><![CDATA[<h4>Build a fully featured pipeline using the latest features from Haystack v0.8.0</h4><p>Those of you who follow our GitHub repository will know that there has been a lot of activity on <a href="https://github.com/deepset-ai/haystack">Haystack</a> recently! Not only have we been working hard on a set of new features, but we’ve received a lot of great code contributions and had a lot of interesting and productive conversations with our community. The sum of all this interaction is <a href="https://github.com/deepset-ai/haystack/releases/tag/v0.8.0">our latest release v0.8.0</a>, which features more changes than we could list here. Nonetheless, we wanted to walk you through the most exciting new features that will help you build your own semantic search pipeline.</p><h3>Milvus</h3><p>Everything as a vector. That’s the future that we see for neural search. It’s already working remarkably well in image, natural language and sound, and research is constantly driving us closer to building a single embedding space for all modalities.</p><p>With this vision comes the need for vector optimized databases that can perform indexing and similarity calculations fast and so we are very excited to be introducing the <a href="https://github.com/deepset-ai/haystack/blob/master/haystack/document_store/milvus.py">MilvusDocumentStore</a> in Haystack. Milvus encapsulates multiple Approximate Nearest Neighbours (ANN) libraries such as FAISS and ANNOY but is also built with the robustness and reliability required in production systems. It allows for dynamic data management, it can also run as a separate service via Docker and we found their library very smooth to work with. And of course, they are open source!</p><p>A MilvusDocumentStore is the perfect complement to a DPRRetriever or an EmbeddingRetriever and in our experimental runs, we have already seen great performance from it. You can expect to see a blog from us soon with benchmarks of all our retrievers and document stores.</p><p>Thanks go to our Github contributor lalitpagaria who was the driving force behind <a href="https://github.com/deepset-ai/haystack/pull/771">this new feature</a>!</p><h3>Knowledge Graph QA</h3><p>Our ambition with <a href="https://www.deepset.ai/haystack">Haystack</a> is to build the most comprehensive and practical framework for anything in the field of search or information retrieval and as such, we could not ignore Knowledge Graphs. Many companies build large, well maintained knowledge graphs to accurately capture the interrelations between different concepts and entities in their domain. Following strong interest from our community, we challenged ourselves to build a system that could interpret natural language questions and fetch structured data from such a graph.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*9yXxJcbus6nV8Xu1" /><figcaption>From <a href="https://yashuseth.blog/2019/10/08/introduction-question-answering-knowledge-graphs-kgqa/">Yashu Seth</a></figcaption></figure><p>In the latest release, you will see the first small steps we’ve made in this direction. Haystack now has a <a href="https://github.com/deepset-ai/haystack/blob/master/haystack/knowledge_graph/graphdb.py">GraphDBKnowledgeGraph</a> class which stores your triples and executes SPARQL queries. There is also a <a href="https://github.com/deepset-ai/haystack/blob/master/haystack/graph_retriever/text_to_sparql.py">Text2SparqlRetriever</a> class which connects to a GraphDBKnowledgeGraph and functions as a semantic parser. It turns natural language questions into SPARQL queries using a BART sequence-to-sequence model. Thanks must go to the <a href="https://github.com/huggingface/transformers">Transformers</a> library maintained by Huggingface which has made it incredibly easy to integrate new transformers models into Haystack!</p><p>Knowledge graph QA systems have a key advantage over extractive QA systems in that they can handle questions that require counting or operations like taking the maximum or minimum. However, the extraction of relational triples and the training of the semantic parser is by no means straightforward. But when done right, we see a lot of potential in the ensembling of the two methods.</p><p>If you want to see how to work with Knowledge Graphs in Haystack, please check out <a href="https://haystack.deepset.ai/tutorials/10_knowledge_graph">our end-to-end tutorial</a>! This will give you a sense of the first small successes we’ve had with the technique and we would love to hear from you if you want to drive Knowledge Graph QA forward with us!</p><h3><strong>Pipeline Configs</strong></h3><p>The flexibility of Pipelines in Haystack has proven to be very popular with our community and we ourselves have enjoyed experimenting with different configurations. For example, we created Pipelines where a BM25 retriever is ensembled with a DPR retriever to get the best of both worlds and it is now possible in Haystack to build retrieval systems that return not only the full document text, but also their summaries and translations.</p><p>Now, with the introduction of Pipeline config files, it is easier than ever to define, save and load your custom Pipeline. Rather than writing lines of code to tweak Pipelines, everything that configures a Pipeline can be written in a YAML file allowing for easier and quicker experimentation cycles and also simpler maintenance of production systems.</p><p>This is an example of what a YAML config might look like.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*3JVJdCz-rRZXwp8w" /></figure><p>These few lines are all you need to load that config and run a Pipeline.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*rmSWpVh18Mi3naUF" /></figure><p>These configs are here to lower the barriers to entry, and allow you to get creative with your Pipeline designs.</p><h3>Confidence Scores</h3><p>Apart from seeing your model’s top predictions, it is often very helpful to know how confident the model is in its predictions. For example, you might be building a system where you expect 90% accuracy and would rather route a request elsewhere if the model is unsure. The Softmaxed scores coming from a Question Answering head do not necessarily capture the model’s confidence of an answer being correct and can create many problems in practice.</p><p>The latest release takes us a step in the right direction as each prediction is now accompanied by a `probability` score in the range of 0 to 1, with values near 0 meaning very low confidence in the prediction and values near 1 meaning very high confidence. Using our transfer learning framework FARM, you can even go one step further and tune this `probability` score to be aligned with your metrics so that you can take confidence=0.9 to mean that the model will be right 90% of the time. If this is something that you’re interested in, have a look at <a href="https://github.com/deepset-ai/FARM/blob/master/examples/question_answering_confidence.py">this tutorial</a>.</p><p>There are many different approaches and complexities to calculating model confidence but you can expect further support and new tools to help you in tuning your system.</p><h3>Web Crawler</h3><p>Finally, it is central to Haystack’s design that it connects easily to other components and data sources. For this reason, we want to highlight a very nice community contribution from DIVYA-19 that has added a <a href="https://github.com/deepset-ai/haystack/pull/775">web crawler</a> to Haystack! Simply provide URLs of pages you’d like to search over and have the Crawler convert them into Haystack documents.</p><p>We anticipate that more use cases will pop up where there is a need to connect data from different sources, formats and streams. If you are working with a certain application, data source or platform that is missing a connection to Haystack, please feel free to bring this to our attention in our GitHub repository or <a href="https://haystack.deepset.ai/community">through Discord</a>! Haystack is designed to be an end-to-end search system but it is also our goal to make sure it integrates seamlessly into your tech stack.</p><h3>Conclusion</h3><p>This latest release is packed full of new features and improvements, so much so that we didn’t even find space in this blog to talk about Evaluation Nodes or SQuAD to DPR data scripts. Haystack v0.8.0 is available now and you can see a full list of changes in <a href="https://github.com/deepset-ai/haystack/releases/tag/v0.8.0">our release notes</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=837451eae9fa" width="1" height="1" alt=""><hr><p><a href="https://medium.com/deepset-ai/semantic-search-with-milvus-knowledge-graph-qa-web-crawlers-and-more-837451eae9fa">Semantic Search with Milvus, Knowledge Graph QA, Web Crawlers and more!</a> was originally published in <a href="https://medium.com/deepset-ai">deepset-ai</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Haystack: The State of Search in 2021]]></title>
            <link>https://medium.com/deepset-ai/haystack-the-state-of-search-in-2021-7388ecb15dfb?source=rss-51b74d7c71f4------2</link>
            <guid isPermaLink="false">https://medium.com/p/7388ecb15dfb</guid>
            <category><![CDATA[haystack]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[neural-search]]></category>
            <category><![CDATA[question-answering]]></category>
            <category><![CDATA[nlp]]></category>
            <dc:creator><![CDATA[Branden Chan]]></dc:creator>
            <pubDate>Tue, 09 Mar 2021 16:06:38 GMT</pubDate>
            <atom:updated>2023-01-13T16:43:22.710Z</atom:updated>
            <content:encoded><![CDATA[<h4>How to build a semantic engine for a better search experience</h4><p><a href="https://www.deepset.ai/haystack">Haystack</a> was born out of the belief that with the latest research from the field of Natural Language Processing, we can build search systems that can adapt to both your informational needs and way of speaking. Keyword search and directory filing systems have been used extensively but as the amount of digital text grows, there comes a need for more exact tools that can pinpoint what you’re looking for.</p><p>The latest NLP methods have proven to be more than capable at picking out relevant documents, rephrasing key points and creating summaries of long texts. Our goal with Haystack is to collect together these state-of-the-art technologies in the one framework so that you can build your own natural language interface into text data. Ask complex full sentence questions to Haystack and let it return to you a curated dossier of only the most useful text saving you time and effort.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*oxUqA6RDv1iEpy70" /></figure><h3><strong>Our First Retriever-Reader Pipeline</strong></h3><p>Those of you who have already tried Haystack will be familiar with the <a href="https://haystack.deepset.ai/docs/intromd#Technology">Retriever-Reader pipeline</a> that we started with. You start by saving your data in a Document Store which can be a database in Elasticsearch, SQL, Milvus or FAISS. You use a Retriever to quickly scan through all your documents and return a handful of the most relevant ones. Then you apply the Reader and harness the power of modern Question Answering models, to closely read each sentence and highlight the parts which answer your question. Already with this design, we move beyond seeing text as a bag of words, and acknowledge that sentence structure and word meaning are integral to building a flow of ideas. We are now able to ask fully grammatical questions in our searches and tackle complex and detailed queries which were beyond the scope of CTRL + F.</p><p>Our first design definitely struck a chord with many in our community and we were very pleased to see it used to <a href="https://medium.com/omdena/using-an-nlp-q-a-system-to-study-climate-hazards-and-nature-based-solutions-c191920766d7">study climate hazards</a>, <a href="https://datamuni.com/@shivanandroy/building-a-faster-accurate-search-engine-with-transformers">search through research papers</a> and even create a <a href="https://www.linkedin.com/posts/mesumrazahemani_ai-nlp-ml-activity-6765000842715156480-oKPT">tax assistant</a>. But over time, we were receiving ever more requests for new components and flexible ways to combine them. The tools that we are about to talk about are the fruits of the exchange of ideas, and often also the exchange of code, between us and our open source community to whom we are greatly thankful.</p><h3><strong>Generator</strong></h3><p>The way that that vital piece of information is phrased in your text might not necessarily be how you want it to be returned. Maybe the answer sentence contains more information than you need or the answer can only be inferred after reading multiple sections of text. That’s what motivated us to include the Generator in Haystack. Rather than simply highlighting a span of text in your documents, it will compose an answering response, based on documents that the retriever finds relevant and also the wording of the query. Our first iteration of this style of Question Answering is based on the <a href="https://arxiv.org/abs/2005.11401">Retrieval Augmented Generator paper</a> released in May 2020 which has shown some very impressive capabilities. The Generator serves as a very stylish alternative to the more direct Reader and as the field of generative research grows, we believe that these models will only get more powerful.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*jzvo8OxTFWNfn-qF" /></figure><p>An example where the answer can only be inferred after incorporating information from multiple sources. Generative models are likely to perform better on these cases than the extractive models found in Reader models.</p><h3><strong>Summarizer</strong></h3><p>Summaries exist because they help us navigate large amounts of text. News articles have titles and subtitles to give a two sentence understanding about what’s to come. Academic papers start with abstracts to give you a quick glance at the questions they tackle and the success of their experiments. The value of this has not been lost on the Machine Learning experts who have developed better and better models that give you the quick and to-the-point roundup and we have incorporated them into Haystack so that you can dedicate your time to reading only the documents that matter to you. Our implementation uses the <a href="https://ai.googleblog.com/2020/06/pegasus-state-of-art-model-for.html">Pegasus model</a> from Google by default but models from the Transformers model hub can be used by simply providing the model name.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lnktCWr1nDySTor2RteDMQ.png" /></figure><h3><strong>Pipelines — The Connector</strong></h3><p>All of the building blocks we have mentioned so far can have their role in helping you navigate large text stacks and we don’t believe that you should have to choose between them. In fact, they are often complementary and Haystack users should have the freedom to define their own bespoke combination. Our Pipelines allow for exactly that. Using the flexibility of Directed Acyclic Graphs, you can easily stick together the building blocks to create powerful pipelines. You can route the inputs and outputs of each component, save these configurations and visualize your designs. Ultimately we’re not the ones to tell you what your search should return, we want to give you the ability to find out for yourself!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*siQUnCe-D-ajz9Q6" /></figure><p>One case where we saw the need for this flexibility was in customers who didn’t want to compromise on performance when picking the right Retriever. Complex machine learning methods (e.g. <a href="https://docs.haystack.deepset.ai/docs/retriever#dense-passage-retrieval">Dense Passage Retrieval</a>) and keyword based methods (e.g. <a href="https://docs.haystack.deepset.ai/docs/retriever#bm25-recommended">the BM25 algorithm</a> built into Elasticsearch) each have <a href="https://docs.haystack.deepset.ai/docs/retriever#deeper-dive-dense-vs-sparse">their respective advantages</a>. With Pipelines, it is now possible to pass a query into multiple Retrievers and combine their results before routing it on to the next component. This is just one of many possible setups and we’re excited to see the creativity this will unleash when our users can mix and match modules.</p><h3><strong>What’s to Come</strong></h3><p>The list of Haystack building blocks is only going to grow and there are already many emerging methods on our radar that will help you build meaningful interfaces into your text data.</p><p>In the past few months, we have been experimenting with the idea of a <a href="https://haystack.deepset.ai/components/query-classifier"><strong>query classifier</strong></a> in order to optimize your retrieval performance. It is designed to predict for a given query whether the lightweight approach will be enough or whether the heavier neural network powered machinery should be engaged, ultimately making searches generally faster and more accurate.</p><p>This philosophy of engaging the heavy machinery only when needed is also behind our interest in <a href="https://haystack.deepset.ai/components/ranker"><strong>Reranking</strong></a> methods. After the Retriever has scanned for a selection of relevant documents, you might want closer analysis of just how useful they are. Using powerful transformer architectures, the latest Rerankers will perform a close reading of just the retrieved documents and give its ranking of the documents from most to least relevant, getting you to the right document quicker.</p><p>Finally, we believe that NLP is not synonymous with English NLP. We endeavor to incorporate any <strong>multilingual</strong>, <strong>cross lingual or domain specific language</strong> technologies that can extend Haystack’s search capabilities to previously <a href="https://haystack.deepset.ai/guides/languages">unsupported languages</a>. To that end, we are keeping an eye out on the latest translation technologies, constantly testing multilingual models and collaborating to train multilingual variants of previously monolingual models.</p><h3><strong>Conclusion</strong></h3><p>Over the course of developing <a href="https://github.com/deepset-ai/haystack">Haystack</a>, we have discovered just how different are the search needs of each individual. By contrast the limited set of search tools that are currently available do not always match their needs and we could see how Haystack could update and improve the end user’s search experience. Core to its design is the ability it gives to developers to craft their own search to understand their own dialect. Its power stems from the wealth of research coming from the world’s best research teams and it allows users to express their informational needs through the most commonly used form of human communication: natural language.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7388ecb15dfb" width="1" height="1" alt=""><hr><p><a href="https://medium.com/deepset-ai/haystack-the-state-of-search-in-2021-7388ecb15dfb">Haystack: The State of Search in 2021</a> was originally published in <a href="https://medium.com/deepset-ai">deepset-ai</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What We Learned from COLING2020]]></title>
            <link>https://medium.com/deepset-ai/what-we-learned-from-coling2020-35648d383a30?source=rss-51b74d7c71f4------2</link>
            <guid isPermaLink="false">https://medium.com/p/35648d383a30</guid>
            <category><![CDATA[nlproc]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[computational-linguistics]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Branden Chan]]></dc:creator>
            <pubDate>Tue, 12 Jan 2021 10:08:46 GMT</pubDate>
            <atom:updated>2023-01-16T10:01:43.672Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>A selection of QA focused papers that caught our attention</em></h4><p>The fields of linguistics and NLP have much to offer one another and COLING 2020 showcased the best of the two fields when they cross-pollinate. The inclusion of an industry track is a sign in itself of the real world value of linguistics-driven NLP. deepset took part in the conference in order to present <a href="https://www.aclweb.org/anthology/2020.coling-main.598/">our latest German language models</a> but also to keep a pulse on the latest research and trends in the field of NLP. Here’s our short list of the most notable papers:</p><h3><strong>Towards Building a Robust Industry-scale Question Answering System</strong></h3><blockquote>TLDR: Maximizing the performance of a BERT based extractive QA system</blockquote><blockquote>Paper: <a href="https://www.aclweb.org/anthology/2020.coling-industry.9/">here</a></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*KNzzxwtI_gdM6scj" /></figure><p>The QA team at IBM presented their GAAMA model which aims to squeeze the best performance out of BERT models on the Natural Questions dataset (if you’d like to learn more about how this differs from SQuAD, have a look at our <a href="https://medium.com/deepset-ai/going-beyond-squad-part-2-d085fb736191">other medium article</a>). One innovation is that they add an attention-over-attention layer which operates on the contextualized word embeddings and calculates query-to-doc attention and doc-to-query attention. They also enforce diversity in the attention heads by adding their cosine distances to the loss. They also augment the training dataset by incorporating other QA datasets (e.g. SQuAD, NewsQA and TriviaQA), creating perturbations of human annotated data via handcrafted rules and generating synthetic samples on unseen passages. At the time of publishing, their models outperformed previous work in the industry on both the short and long answer tasks. Since then, other works have regained the top of the leaderboard but the latest version of their GAAMA model still ranks 2nd in short answer. One notable insight from their presentation was that adding SQuAD data improves the short answer performance of the model while synthetic data had more of a positive impact on the long answer task. Their various tweaks to attention give about +1% on long answer performance.</p><h3><strong>Improving Conversational Question Answering Systems after Deployment using Feedback-Weighted Learning</strong></h3><blockquote>TLDR: How to best use user feedback to improve a live QA system</blockquote><blockquote>Paper: <a href="https://www.aclweb.org/anthology/2020.coling-main.230.pdf">here</a></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ywZ1fbzosSC0un_Z" /></figure><p>This next paper was of particular interest to us at deepset given that we recently created a <a href="https://docs.haystack.deepset.ai/docs/domain_adaptation#user-feedback">user feedback mechanism</a> within Haystack that allows for users to state whether a model’s predictions are correct or not. In many use cases, this process allows a cheaper and faster generation of domain-specific labels that can be used to continuously train your model. The authors of this paper simulate settings in Conversational Question Answering (CQA) where there may not be much gold label annotation data. To remedy this, they make use of user feedback, which is not a full annotation, but rather a binary correct or incorrect label. They run experiments to retrain the model and also formally define how to weight this user feedback showing that their method gives strong improvements when performing CQA in another domain. With their theoretical rigour and pragmatic approach to improving Conversational Question Answering, it is no wonder that they were nominated for best paper at COLING!</p><h3><strong>A Study on Efficiency, Accuracy and Document Structure for Answer Sentence Selection</strong></h3><blockquote>TLDR: A novel and very lightweight generative QA pipeline that relies on an Answer Sentence Selection model</blockquote><blockquote>Paper: <a href="https://arxiv.org/abs/2003.02349">here</a></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*SiaOHGa94SfT93Hk" /></figure><p>While extractive and generative have become the dominant styles of Question Answering in recent years, this paper shows how methods from Answer Sentence Selection still have much to offer the fast moving field of Question Answering. Relying on static word embeddings and CNN, they create Cosinet which computes the relevance of a given sentence to the query. They are also able to model the context around a given candidate sentence using a BiRNN component. When these are combined, they manage to reach 75.62 MAP on WikiQA, beating all other similarly cost-efficient models. Though BERT outperforms this model, reaching 81.32 MAP, Cosinet can be trained in just 7.5s on a GTX1080 Ti compared to 17mins 50s for BERT. Once the answering sentence is chosen, the query and sentence are also put through a generator in order to produce an answer phrase that more directly addresses the query. In the end, the full model can be seen as a very novel approach to creating a generative QA system that rivals systems such as <a href="https://arxiv.org/abs/2005.11401">RAG</a>. Since the system is so fast and requires very little indexing time, it would be very interesting to see whether it could scale to the open domain setting.</p><h3><strong>Deep Learning Framework for Measuring the Digital Strategy of Companies from Earnings Calls</strong></h3><blockquote>TLDR: Using NLP to build vectors that represent a company’s digital strategy</blockquote><blockquote>Paper: <a href="https://arxiv.org/pdf/2010.12418.pdf">here</a></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*QqV3HzdidAMG0yv9" /></figure><p>I’ve included this work into the list as a bonus because it shows a very simple yet effective approach to company clustering. The authors start by picking a set of aspects relevant to digital strategy such as “robotics,” or “operations” which are something like topics that could appear in a document. They train a token classifying BERT to identify spans of text that exemplify such aspects. The architecture of such a model is very similar to an NER model just that the model is used to label aspects instead of named entity classes. Using publicly available earning call documents, they extract these aspects and aggregate their occurrences for each company. In effect, they build a vector that characterizes the company where each dimension corresponds to a different aspect. For visualisation, they reduce the dimensionality of this vector using TSNE and cluster the companies in 2D space resulting in a clustered map of companies where similar companies come up near each other.</p><h3><strong>Conclusion</strong></h3><p>COLING is not only one of the most popular conferences for computational linguistics, it also offers really great topics for the more industry oriented NLP developers. With the explosion of research being done in the field, COLING is a vital venue for letting NLP researchers present their findings, whether they are theoretically or pragmatically driven. This year was no exception and we at deepset are very thankful to all the presenters and organizers who made this year’s edition happen in the most challenging of circumstances.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=35648d383a30" width="1" height="1" alt=""><hr><p><a href="https://medium.com/deepset-ai/what-we-learned-from-coling2020-35648d383a30">What We Learned from COLING2020</a> was originally published in <a href="https://medium.com/deepset-ai">deepset-ai</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[EMNLP 2020: The Latest Advancements in Question Answering and Information Retrieval]]></title>
            <link>https://medium.com/deepset-ai/emnlp-2020-the-latest-advancements-in-question-answering-and-information-retrieval-9e5c718b46ab?source=rss-51b74d7c71f4------2</link>
            <guid isPermaLink="false">https://medium.com/p/9e5c718b46ab</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[nlproc]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[question-answering]]></category>
            <dc:creator><![CDATA[Branden Chan]]></dc:creator>
            <pubDate>Thu, 03 Dec 2020 12:31:52 GMT</pubDate>
            <atom:updated>2020-12-03T14:28:22.128Z</atom:updated>
            <content:encoded><![CDATA[<p>EMNLP was, for reasons beyond the organisers control, moved entirely online this year. I never would have guessed that my first contact with some of the most eminent researchers in NLP would be mediated through the pixelated 2D avatar of <a href="https://gather.town/">Gather’s</a> online conferencing platform. But they did a great job replicating the thrill of being among the conference goers and proved to be a great venue for one of the most important NLP conferences of the year.</p><p>This year, the program was divided into different streams covering topics ranging from NLP ethics and sociolinguistics to information extraction and machine translation. Of course my focus was on the Question Answering and Information Retrieval side of things and I was not disappointed by the range of new ideas being presented.</p><p>A few different trends became evident to me over the course of the conference. There was certainly a lot of work addressing more efficient and effective annotation, generative approaches to tasks that were seemingly only treated with discriminative models, and improvements on models to make them run more efficiently. Here is a short recap of a few of the papers that caught my attention:</p><h3>Don’t Read Too Much Into It: Adaptive Computation for Open-Domain Question Answering</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/872/0*jUyn1mGn0DTy-aG6" /><figcaption>An illustration of how their ODQA system can choose not to calculate every Reader transformer layer for every passage fetched by the Retriever</figcaption></figure><p>Paper: <a href="https://arxiv.org/abs/2011.05435">here</a></p><p>In the standard Retriever-Reader pipeline in Open Domain QA, the Reader (a.k.a. The Question Answering model) will pass all documents fetched by the Retriever through all layers of the Transformer model. This paper proposes a new method whereby transformer layers are applied to individual documents one at a time. After each stage of processing, confidence scores are calculated and different layers and documents are reprioritised using a Reinforcement Learning model. This allows for the ODQA system to make a final prediction without pushing all documents through all layers of its model. They report that they could reduce the amount of Reader layer computations by 4.7x while maintaining 95% of the original performance. When using a GPU, this translated to a 2.6x Reader speedup.</p><h3>Beat the AI: Investigating Adversarial Human Annotation for Reading Comprehension</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*lpGcfROXMdctRdMB" /><figcaption>The work flow for collecting adversarial QA annotations</figcaption></figure><p>Paper: <a href="https://arxiv.org/abs/2002.00293">here</a></p><p>Manual labelling of question answering samples is time consuming and often very costly. This paper from the team at UCL proposes a form of adversarial annotation that challenges humans to generate not just more labels but also harder labels which can out smart existing models. Each question and answer pair that is generated by the human is passed on to an already trained QA model. If this model is able to predict the associated answer, the labeller is told to create another example. If the model is unable to predict that answer, this question and answer pair forms part of a new adversarial dataset. They report that training a BERT model on one of their datasets with only 10k examples can give an F1 of 69.8% when evaluated on Natural Questions as compared to F1 68.2% when the same model is trained on a 10k slice of SQuAD.</p><h3>Modularized Transfomer-based Ranking Framework</h3><p>Paper: <a href="https://www.aclweb.org/anthology/2020.emnlp-main.342/">here</a></p><p>Dense transformer based methods are increasingly being used earlier in the pipeline of Information Retrieval systems as they become more efficient and more powerful. The proposal in this work is to encode both query and document, not as a single vector each, but rather as matrices composed of contextualised word vectors output by transformer models. At query time, document relevance score is calculated using Interaction Blocks which allow for query and document tokens to attend to each other. By doing so, they get performance that is very close to a full BERT ranker with a speed up of up to 170x at query time. This method can be seen as a kind of intermediate approach between full BERT ranking where computation is done entirely at query time and Dense Passage Retrieval which shifts majority of the computation to indexing time.</p><h3>A Simple and Effective Model for Answering Multi-span Questions</h3><p>Paper: <a href="https://www.aclweb.org/anthology/2020.emnlp-main.248/">here</a></p><p>Standard QA models operate by assigning scores to each token in a passage for how likely they are to be the start or end of the answer sequence. While this is an effective approach in SQuAD where the full answer is a single contiguous text span, it is unable to deal with tasks which require multiple discontinuous text spans as the answer. The authors of this paper tackle this problem by porting Named Entity Recognition’s BIO annotation system to QA. Each token can now be labelled as being a part of an answer span or not. In doing so, their system reaches 53.6% EM on the multispan answer examples in DROP and 59.7% on the same in Quoref without sacrificing performance on single span answer examples.</p><h3>Augmented Natural Language for Generative Sequence Labeling</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*icDqPYfPWHSkM6NK" /><figcaption>Comparing different ways of encoding annotations</figcaption></figure><p>Paper: <a href="https://www.aclweb.org/anthology/2020.emnlp-main.27/">here</a></p><p>While this final paper is not strictly about QA, its elegance and generality make it worthy of a mention. The idea behind this work is not to attach a label to each token in an input sequence as is generally done in NER and slot labelling, but rather to insert labels, in natural language, into the input sequence itself. These token classification tasks are then treated as a sequence to sequence problem where the expected output is a copy of the input text but with natural language labels. The authors report big improvements in few shot slot labelling and low resource slot labelling but more importantly provide a simple yet effective approach to multitask learning.</p><h3>Conclusion</h3><p>In the end, EMNLP 2020 was a great experience and an excellent mixing pot of new trends and ideas. We certainly came away energised and inspired by the research coming from the discipline. We are privileged to be a beneficiary of such a vibrant community but also hope that <a href="https://github.com/deepset-ai/haystack">Haystack</a> can be our contribution back to the researchers and practitioners who built up the field!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9e5c718b46ab" width="1" height="1" alt=""><hr><p><a href="https://medium.com/deepset-ai/emnlp-2020-the-latest-advancements-in-question-answering-and-information-retrieval-9e5c718b46ab">EMNLP 2020: The Latest Advancements in Question Answering and Information Retrieval</a> was originally published in <a href="https://medium.com/deepset-ai">deepset-ai</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>