<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Mohit Soni on Medium]]></title>
        <description><![CDATA[Stories by Mohit Soni on Medium]]></description>
        <link>https://medium.com/@mohitdsoni?source=rss-587aa3bba9cb------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*2ta_c_ElbFQM0RjwxpC8Wg.jpeg</url>
            <title>Stories by Mohit Soni on Medium</title>
            <link>https://medium.com/@mohitdsoni?source=rss-587aa3bba9cb------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 15:45:04 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@mohitdsoni/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Training Large Language Model (LLM) on your data]]></title>
            <link>https://medium.com/walmartglobaltech/training-large-language-model-llm-on-your-data-2139eaad5f4f?source=rss-587aa3bba9cb------2</link>
            <guid isPermaLink="false">https://medium.com/p/2139eaad5f4f</guid>
            <dc:creator><![CDATA[Mohit Soni]]></dc:creator>
            <pubDate>Wed, 23 Aug 2023 08:27:37 GMT</pubDate>
            <atom:updated>2023-08-23T08:27:37.557Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/512/1*UyyPim7BABLKyihPP_fVYA.png" /><figcaption>Image generated using DALL-E</figcaption></figure><p>Large Language Models (LLM) have taken the internet by storm in the last few months. The recent launch of <a href="https://techcrunch.com/2023/05/10/google-launches-palm-2-its-next-gen-large-language-model/">PaLM 2</a> by Google and <a href="https://openai.com/gpt-4">GPT 4</a> by OpenAI has captured the imagination of enterprises. There are many potential use cases being ideated across the domains. Multilingual customer support, code generation, content creation and advanced chatbots are some examples. These use cases require LLMs to respond based on the custom data of the business.</p><p><em>How businesses can solve these use cases with the help of LLM on their custom data? </em>After some research and dabbling with LLMs, I found three ways of doing so.</p><ol><li>Prompt Engineering</li><li>Embeddings</li><li>Fine Tuning</li></ol><h3>Prompt Engineering</h3><h4>What is a prompt?</h4><p>Prompt is an input to the LLM. For example, in the below image the prompt is-</p><blockquote><em>“Recommend me 5 fiction novels similar to the Bourne Series</em>. <em>Write only the title and author name.”</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jiZPCMTJtMsbbEVHnl-o-w.png" /><figcaption>Prompt asking Chat GPT to recommend novels</figcaption></figure><h4>Custom data in prompt</h4><p>Injecting your data or context in a prompt is the easiest way of using custom data with LLM. For example, in the below image, the prompt has sales data for Q1 2023 and Q2 2023 of a company. The question to the LLM is to return the percentage change in the sales.</p><figure><img alt="Chat GPT responds to the query using custom data (revenue numbers) provided in the prompt." src="https://cdn-images-1.medium.com/max/1024/1*aeek418WCndtt1491JHK7w.png" /><figcaption>Adding data in Chat GPT prompt</figcaption></figure><h4>When to use prompt engineering?</h4><p>There are many applications of prompt engineering. In fact, prompt engineering is considered a niche skill that will be coveted in the future.</p><p>Passing the data in prompt works well in the scenarios where you want to provide instructions to the LLM, perform search operation or get answer to the queries from a smaller data set. But, it is not the best way to work with a large set of documents or web pages as input to LLM due to limitation in the size of prompt and cost associated with passing large text to the LLM.</p><h3>Embeddings</h3><h4>What are embeddings?</h4><p>Embeddings are a way of representing information, whether it is text, image, or audio, into a numerical form. Imagine that you want to group apples, bananas and oranges based on similarity. This can be done using “embeddings”.</p><p>An embedding will turn each type of fruit into a numerical form (vector). Consider the below example,<br>Apples -&gt; (3, 8, 7)<br>Bananas -&gt; (8, 1, 3)<br>Oranges -&gt; (4, 7, 6)</p><p>These embeddings for apples and oranges are closer to each other. We can say that apples and oranges are more alike.</p><h4>Using embeddings</h4><p>Below diagram explains conceptually how these embeddings are used to retrieve information from your documents using LLM. First, the documents are passed through a model that creates small chunks of it and then creates embedding of those chunks. These embeddings are then stored in a vector database. When a user wants to query the LLM, the embeddings are retrieved from the vector store and passed to the LLM. LLM generates the response from the custom data using the embeddings.</p><p><a href="https://mohitdsoni.medium.com/how-to-use-chat-gpt-and-langchain-to-get-responses-from-your-documents-768a9ca0e0d0">This post</a> is a working example for generating embeddings for documents and passing these embeddings to Chat GPT.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bYy116KZAanbxXta4PCkjQ.png" /><figcaption>Document embedding with LLM</figcaption></figure><h4>When to use embeddings?</h4><p>Embeddings work well when a large corpus of documents or web pages needs to be passed to an LLM. For example, this method would work well when a chatbot is built to provide a response from a set of policy documents to the user.</p><h3>Fine Tuning</h3><h4>What is fine tuning?</h4><p>Fine tuning is a way of teaching the model how to process the input query and how to represent the response. For example, LLM can be fine-tuned by providing data about the customer reviews and corresponding sentiment.</p><blockquote><strong>Prompt</strong>: I really liked this product. I highly recommend it. <br><strong>Completion</strong>: Positive<br><strong>Prompt</strong>: I found the delivery experience below average.<br><strong>Completion</strong>: Negative</blockquote><p>Here “prompt” is an example of an input and “completion” is an example of the corresponding output.</p><p>Now, if you provide another review as a prompt to the LLM, then it will respond with the corresponding sentiment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/978/1*vOFO9HQiHY5aeGmGLfGxhg.png" /><figcaption>Sentiment analysis using fine tuned Chat GPT</figcaption></figure><p>Another example is</p><blockquote><strong>Prompt</strong>: Q1 2023, $10 million, +5%, good, positive<br><strong>Completion</strong>: The revenue for Q1 2023 was $10 million, which is 5% higher than revenue in the previous quarter. The sentiment on the streets is good for this company and there is a positive outlook for the business.<br><strong>Prompt</strong>: Q2 2023, $20 million, -2%, neutral, negative<br><strong>Completion</strong>: The revenue for Q2 2023 was $20 million, which is 2% lower than revenue in the previous quarter. The sentiment on the streets is neutral for this company and there is a negative outlook for the business.</blockquote><p>You can provide many examples to fine tune the model and then ask questions to the model.</p><p>Fine tuning is available for limited base models — davinci, curie, babbage, and ada.</p><h4>When to use fine tuning?</h4><p>Fine tuning is typically used to tune the LLM for a specific task and get a response within that scope. The task can be email categorisation, sentiment analysis, entity extraction, generating product description based on the specs, etc.</p><p>This post barely scratched the surface of these techniques. There are many more nuances to each of these.</p><p>I am learning with you and will keep sharing my learnings about this exciting space. Do follow!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2139eaad5f4f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/walmartglobaltech/training-large-language-model-llm-on-your-data-2139eaad5f4f">Training Large Language Model (LLM) on your data</a> was originally published in <a href="https://medium.com/walmartglobaltech">Walmart Global Tech Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to use Chat GPT and LangChain to get responses from your documents]]></title>
            <link>https://mohitdsoni.medium.com/how-to-use-chat-gpt-and-langchain-to-get-responses-from-your-documents-768a9ca0e0d0?source=rss-587aa3bba9cb------2</link>
            <guid isPermaLink="false">https://medium.com/p/768a9ca0e0d0</guid>
            <category><![CDATA[embedding]]></category>
            <category><![CDATA[chatgpt]]></category>
            <category><![CDATA[chromadb]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Mohit Soni]]></dc:creator>
            <pubDate>Sat, 17 Jun 2023 13:54:14 GMT</pubDate>
            <atom:updated>2023-06-17T13:54:14.652Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/512/1*QOP_dzl8UQeaQ1mtE1DTBg.jpeg" /></figure><p>In the <a href="https://medium.com/@mohitdsoni/training-chat-gpt-on-your-data-efaa7b7f521b">previous post</a>, I explained methods that you can use to train Chat GPT on your data. In this post, we will use one of those methods “embeddings” to get Chat GPT response from our documents. Let’s dive in!</p><p>I am using an <a href="https://github.com/mohitdsoni/genAI/blob/main/docs/biography.txt">this</a> text file that is a biography of a fictitious character, to create embeddings. We will use this file to generate responses from Chat GPT. Below is the snippet of the text file.</p><pre>Early Life and Education:<br>Born on March 15th, 1985, in a small town named Greenridge, <br>John Anderson displayed an inquisitive nature from an early age. <br>Growing up in a supportive and nurturing family environment, <br>John was encouraged to pursue his interests and cultivate his talents. </pre><h4>Import Required Packages</h4><p>Let’s begin by importing the required packages. Please note that if you want to use Azure OpenAI endpoint, then use AzureOpenAI package. Otherwise, use openai. Read more about it <a href="https://learn.microsoft.com/en-us/azure/cognitive-services/openai/how-to/switching-endpoints">here</a>. I have used ChromaDB as a vector db to store the embeddings.</p><pre>#Import required packages<br>#If you have your endpoing then use AzureOpenAI<br>#else use<br>#import openai<br>from langchain.llms import AzureOpenAI<br>#This will help us create embeddings<br>from langchain.embeddings.openai import OpenAIEmbeddings<br>#Using ChromaDB as a vector store for the embeddigns<br>from langchain.vectorstores import Chroma<br>from langchain.text_splitter import RecursiveCharacterTextSplitter<br>from langchain.chains import RetrievalQA<br>from langchain.document_loaders import DirectoryLoader<br>import os</pre><h4>Set API Key and Endpoint</h4><p>Please replace “OPENAI_API_KEY” with your secret API Key. Read how to get your secret key <a href="https://help.openai.com/en/articles/4936850-where-do-i-find-my-secret-api-key">here</a>. Replace “OPENAI_API_BASE” with your endpoint name. Read how to get these details <a href="https://learn.microsoft.com/en-us/azure/cognitive-services/openai/quickstart?tabs=command-line&amp;pivots=programming-language-python#retrieve-key-and-endpoint">here</a>.</p><pre>os.environ[&quot;OPENAI_API_TYPE&quot;] = &quot;azure&quot;<br>os.environ[&quot;OPENAI_API_VERSION&quot;] = &quot;2022-12-01&quot;<br>#Set your API endpoint (API BASE) here if you are using Azure OpenAI<br>#If you are using openai common endpoing then you do not need to set this.<br>os.environ[&quot;OPENAI_API_BASE&quot;] = &quot;OPENAI_API_BASE&quot;<br>os.environ[&quot;OPENAI_API_KEY&quot;] = &quot;OPENAI_API_KEY&quot;</pre><h4>Load Documents</h4><p>Now we load our documents from a directory. The code below reads all the txt files in “docs” directory. If you want to read other file types, read the documentation <a href="https://python.langchain.com/docs/modules/data_connection/document_loaders.html">here</a>.</p><p>After loading all the text from the documents, we break them into small chunks to create embeddings.</p><pre>#Load all the .txt files from docs directory<br>loader = DirectoryLoader(&#39;./docs/&#39;,glob = &quot;**/*.txt&quot;)<br>docs = loader.load()<br>#Split text into tokens<br>text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=0)<br>texts = text_splitter.split_documents(docs)</pre><h4>Create Embeddings</h4><p>After ingesting the documents, let’s create embeddings using OpenAI Embeddings. You can provide the name of your deployment, if you are using Azure OpenAI. If not, then do not use this parameter. Read more about deployment name <a href="https://learn.microsoft.com/en-us/azure/cognitive-services/openai/quickstart?tabs=command-line&amp;pivots=programming-language-python#retrieve-key-and-endpoint">here</a>.</p><p>Once embeddings are created, we can store them in Chroma vector db. The embeddings will be stored in “chromadb” directory, which will be created in your working directory.</p><pre>#Turn the text into embeddings<br>embeddings = OpenAIEmbeddings(deployment=&quot;NAME_OF_YOUR_MODEL_DEPLOYMENT&quot;, chunk_size=1) #This model should be able to generate embeddings. For example, text-embedding-ada-002<br>#Store the embeddings into chromadb directory<br>docsearch = Chroma.from_documents(documents=texts, embedding=embeddings, persist_directory=&quot;./chromadb&quot;)</pre><h4>Ask Questions!</h4><p>We have reached the coolest part of the code. We can now ask Chat GPT any query and pass these embeddings to get response from our data.</p><pre>#Use AzureOpenAI, if you&#39;re using a endpoint from Azure Open AI<br>#This can be any QnA model. For example, davinci. <br>#Remember to provide deployment name here and not the model name<br>llm = AzureOpenAI(deployment_name=&quot;NAME_OF_YOUR_MODEL_DEPLOYMENT&quot;) <br>#Use OpenAI if you&#39;re using a Azure OpenAI endpoint <br>#llm = ChatOpenAI(temperature = 0.7, model_name=&#39;MODEL_NAME&#39;)<br>qa = RetrievalQA.from_chain_type(llm=llm, <br>                                 chain_type=&quot;stuff&quot;,<br>                                 retriever=docsearch.as_retriever(),<br>                                 return_source_documents=False<br>                                 )<br>query = &quot;Where was John born?&quot;<br>qa.run(query)</pre><h4>All Together</h4><p>Let’s put it all together. You can find the complete notebook <a href="https://github.com/mohitdsoni/genAI/blob/main/LLMEmbeddingsExample.ipynb">here</a>.</p><pre><br>#Import required packages<br>#If you have your endpoing then use AzureOpenAI<br>#else use<br>import openai<br>from langchain.llms import AzureOpenAI<br>#This will help us create embeddings<br>from langchain.embeddings.openai import OpenAIEmbeddings<br>#Using ChromaDB as a vector store for the embeddigns<br>from langchain.vectorstores import Chroma<br>from langchain.text_splitter import RecursiveCharacterTextSplitter<br>from langchain.chains import RetrievalQA<br>from langchain.document_loaders import DirectoryLoader<br>import os<br><br><br>os.environ[&quot;OPENAI_API_TYPE&quot;] = &quot;azure&quot;<br>os.environ[&quot;OPENAI_API_VERSION&quot;] = &quot;2022-12-01&quot;<br>#Set your API endpoint (API BASE) here if you are using Azure OpenAI<br>#If you are using openai common endpoing then you do not need to set this.<br>os.environ[&quot;OPENAI_API_BASE&quot;] = &quot;OPENAI_API_BASE&quot;<br>#Set your OPENAI API KEY here<br>os.environ[&quot;OPENAI_API_KEY&quot;] = &quot;OPENAI_API_KEY&quot;<br><br><br>#Load all the .txt files from docs directory<br>loader = DirectoryLoader(&#39;./docs/&#39;,glob = &quot;**/*.txt&quot;)<br>docs = loader.load()<br><br><br>#Split text into tokens<br>text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=0)<br>texts = text_splitter.split_documents(docs)<br><br><br>#Turn the text into embeddings<br>embeddings = OpenAIEmbeddings(deployment=&quot;NAME_OF_YOUR_MODEL_DEPLOYMENT&quot;, chunk_size=1) #This model should be able to generate embeddings. For example, text-embedding-ada-002<br>#Store the embeddings into chromadb directory<br>docsearch = Chroma.from_documents(documents=texts, embedding=embeddings, persist_directory=&quot;./chromadb&quot;)<br><br><br>#Use AzureOpenAI, if you&#39;re using a endpoint from Azure Open AI<br>llm = AzureOpenAI(deployment_name=&quot;NAME_OF_YOUR_MODEL_DEPLOYMENT&quot;) #This can be any QnA model. For example, davinci.<br>#Use OpenAI if you&#39;re using a Azure OpenAI endpoint <br>#llm = ChatOpenAI(temperature = 0.7, model_name=&#39;MODEL_NAME&#39;)<br><br><br>qa = RetrievalQA.from_chain_type(llm=llm, <br>                                 chain_type=&quot;stuff&quot;,<br>                                 retriever=docsearch.as_retriever(),<br>                                 return_source_documents=False<br>                                 )<br>query = &quot;Where was John born?&quot;<br>qa.run(query)<br></pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=768a9ca0e0d0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Training Large Language Model (LLM) on your data]]></title>
            <link>https://mohitdsoni.medium.com/training-chat-gpt-on-your-data-efaa7b7f521b?source=rss-587aa3bba9cb------2</link>
            <guid isPermaLink="false">https://medium.com/p/efaa7b7f521b</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[fine-tuning]]></category>
            <category><![CDATA[chatgpt]]></category>
            <category><![CDATA[embedding]]></category>
            <category><![CDATA[prompt]]></category>
            <dc:creator><![CDATA[Mohit Soni]]></dc:creator>
            <pubDate>Fri, 16 Jun 2023 14:05:43 GMT</pubDate>
            <atom:updated>2023-06-30T08:23:42.894Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/512/1*UyyPim7BABLKyihPP_fVYA.png" /><figcaption>Image generated using DALL-E</figcaption></figure><p>Large Language Models (LLM) have taken the internet by storm in the last few months. The recent launch of <a href="https://techcrunch.com/2023/05/10/google-launches-palm-2-its-next-gen-large-language-model/">PaLM 2</a> by Google and <a href="https://openai.com/gpt-4">GPT 4</a> by OpenAI has captured the imagination of enterprises. There are many potential use cases being ideated across the domains. Multilingual customer support, code generation, content creation and advanced chatbots are some examples. These use cases require LLMs to respond based on the custom data of the business.</p><p><em>How businesses can solve these use cases with the help of LLM on their custom data? </em>After some research and dabbling with LLMs, I found three ways of doing so.</p><ol><li>Prompt Engineering</li><li>Embeddings</li><li>Fine Tuning</li></ol><h3>Prompt Engineering</h3><h4><em>What is a prompt?</em></h4><p>Prompt is an input to the LLM. For example, in the below image the prompt is-</p><blockquote><em>“Recommend me 5 fiction novels similar to the Bourne Series</em>. <em>Write only the title and author name.”</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jiZPCMTJtMsbbEVHnl-o-w.png" /><figcaption>Prompt asking Chat GPT to recommend novels</figcaption></figure><h4><em>Custom data in prompt</em></h4><p>Injecting your data or context in a prompt is the easiest way of using custom data with LLM. For example, in the below image, the prompt has sales data for Q1 2023 and Q2 2023 of a company. The question to the LLM is to return the percentage change in the sales.</p><figure><img alt="Chat GPT responds to the query using custom data (revenue numbers) provided in the prompt." src="https://cdn-images-1.medium.com/max/1024/1*aeek418WCndtt1491JHK7w.png" /><figcaption>Adding data in Chat GPT prompt</figcaption></figure><h4>When to use prompt engineering?</h4><p>There are many applications of prompt engineering. In fact, prompt engineering is considered a niche skill that will be coveted in the future.</p><p>Passing the data in prompt works well in the scenarios where you want to provide instructions to the LLM, perform search operation or get answer to the queries from a smaller data set. But, it is not the best way to work with a large set of documents or web pages as input to LLM due to limitation in the size of prompt and cost associated with passing large text to the LLM.</p><h3>Embeddings</h3><h4>What are embeddings?</h4><p>Embeddings are a way of representing information, whether it is text, image, or audio, into a numerical form. Imagine that you want to group apples, bananas and oranges based on similarity. This can be done using “embeddings”.</p><p>An embedding will turn each type of fruit into a numerical form (vector). Consider the below example,<br>Apples -&gt; (3, 8, 7)<br>Bananas -&gt; (8, 1, 3)<br>Oranges -&gt; (4, 7, 6)</p><p>These embeddings for apples and oranges are closer to each other. We can say that apples and oranges are more alike.</p><h4>Using embeddings</h4><p>Below diagram explains conceptually how these embeddings are used to retrieve information from your documents using LLM. First, the documents are passed through a model that creates small chunks of it and then creates embedding of those chunks. These embeddings are then stored in a vector database. When a user wants to query the LLM, the embeddings are retrieved from the vector store and passed to the LLM. LLM generates the response from the custom data using the embeddings.</p><p><a href="https://mohitdsoni.medium.com/how-to-use-chat-gpt-and-langchain-to-get-responses-from-your-documents-768a9ca0e0d0">This post</a> is a working example for generating embeddings for documents and passing these embeddings to Chat GPT.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bYy116KZAanbxXta4PCkjQ.png" /><figcaption>Document embedding with LLM</figcaption></figure><h4>When to use embeddings?</h4><p>Embeddings work well when a large corpus of documents or web pages needs to be passed to an LLM. For example, this method would work well when a chatbot is built to provide a response from a set of policy documents to the user.</p><h3>Fine Tuning</h3><h4>What is fine tuning?</h4><p>Fine tuning is a way of teaching the model how to process the input query and how to represent the response. For example, LLM can be fine-tuned by providing data about the customer reviews and corresponding sentiment.</p><blockquote><strong>Prompt</strong>: I really liked this product. I highly recommend it. <br><strong>Completion</strong>: Positive<br><strong>Prompt</strong>: I found the delivery experience below average.<br><strong>Completion</strong>: Negative</blockquote><p>Here “prompt” is an example of an input and “completion” is an example of the corresponding output.</p><p>Now, if you provide another review as a prompt to the LLM, then it will respond with the corresponding sentiment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/978/1*vOFO9HQiHY5aeGmGLfGxhg.png" /><figcaption>Sentiment analysis using fine tuned Chat GPT</figcaption></figure><p>Another example is</p><blockquote><strong>Prompt</strong>: Q1 2023, $10 million, +5%, good, positive<br><strong>Completion</strong>: The revenue for Q1 2023 was $10 million, which is 5% higher than revenue in the previous quarter. The sentiment on the streets is good for this company and there is a positive outlook for the business.<br><strong>Prompt</strong>: Q2 2023, $20 million, -2%, neutral, negative<br><strong>Completion</strong>: The revenue for Q2 2023 was $20 million, which is 2% lower than revenue in the previous quarter. The sentiment on the streets is neutral for this company and there is a negative outlook for the business.</blockquote><p>You can provide many examples to fine tune the model and then ask questions to the model.</p><p>Fine tuning is available for limited base models — davinci, curie, babbage, and ada.</p><h4>When to use fine tuning?</h4><p>Fine tuning is typically used to tune the LLM for a specific task and get a response within that scope. The task can be email categorisation, sentiment analysis, entity extraction, generating product description based on the specs, etc.</p><p>This post barely scratched the surface of these techniques. There are many more nuances to each of these.</p><p>I am learning with you and will keep sharing my learnings about this exciting space. Do follow!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=efaa7b7f521b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Evasive Designs]]></title>
            <link>https://mohitdsoni.medium.com/evasive-designs-e5aaf58d8c96?source=rss-587aa3bba9cb------2</link>
            <guid isPermaLink="false">https://medium.com/p/e5aaf58d8c96</guid>
            <category><![CDATA[product-development]]></category>
            <category><![CDATA[product-design]]></category>
            <category><![CDATA[product-management]]></category>
            <dc:creator><![CDATA[Mohit Soni]]></dc:creator>
            <pubDate>Wed, 20 Jan 2021 14:10:51 GMT</pubDate>
            <atom:updated>2021-01-20T14:10:51.269Z</atom:updated>
            <content:encoded><![CDATA[<p>Product managers chase multiple user and business metrics such as an increase in the number of users, increase in the number of users subscribing to promotional offers, decrease in churn rate, high conversion rate, increase in ARPU etc. Every product or platform aims for high user retention rates, low churn rates, and a high lifetime value (LTV) of its users.</p><p>Sometimes PMs make design choices that serve only the interest of the business. These designs, though perform better on the metrics, are misleading for the users. I am sharing a few of my observations of such design techniques.</p><h3>1. Cookies usage consent</h3><p>After the introduction of GDPR laws in Europe, most websites take cookie usage consent from the users. You must have encountered a cookie consent message such as the one shown below while browsing the web. Notice that option to dissent to the use of non-essential cookies is hidden behind the “Cookies Management Tool” link.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9DZM761OtcuifenoUFq11A.png" /><figcaption>A sample cookie consent message presented to the user</figcaption></figure><p>In the image above</p><ul><li>Only the “Accept” button is visible.</li><li>The font size of the text is small.</li><li>The links are not visible.</li><li>The options to configure the policy are hidden behind the “Cookies Management Tool” link.</li><li>Even if you click on “Cookies Management Tool” link, the configuration options are not user friendly <em>(Image below</em>).</li><li>It is not clear whether enabling the button will block cookies or enable cookies. User has to make similar choices for all the different third-party vendors. Why not show a simple and promptly visible option to opt-out of all the cookies?</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*edCPBDEnu7zNqx7MkfkPpw.png" /><figcaption>Cookie management presented to the users</figcaption></figure><h3>2. Opting out of service or a feature</h3><p>The off-boarding experience for most of the platforms is not optimal. It is designed to create barriers for users from opting-out</p><h4>2.1 Credit Card</h4><p>You would have received numerous calls to convince you to get a credit card. However, have you faced difficulties while closing a credit card account? Most card providers don’t even have an option to close the credit card account on their web site. Many make it very difficult to reach out to any service executive for this issue. Card closure requires multiple emails and follow-ups. It becomes frustrating for the customers to the point that they vent their frustration on social media and warn people against using that product.</p><h4>2.2 Unsubscribe email</h4><p>Some websites make it difficult to unsubscribe emails from them. After a user clicks on the unsubscribe link in the email, they ask users to log in to the website to stop receiving emails. It is likely that users will not remember the password and instead of unsubscribing drop off from the workflow. The user will become frustrated and block all the emails coming from that domain. It is much better to provide an easy opt-out option to the users.</p><p>Once a customer has decided to leave, the focus should be on finding out the improvement opportunities and making the transition easy. An unpleasant off-boarding experience leaves a bad taste in customers’ mouth. It diminishes the possibility of customer’s returning in the future. Companies and PMs should think about making the off-boarding experience of their customers smooth. A suboptimal off-boarding experience might be due to non-streamlined business processes which translate to a bad or non-existent product feature. E.g. a business providing rental housing to its customer takes a deposit in advance. The off-boarding experience will be smooth only when there is a streamlined process of assessing the house quality and refunding the deposit.</p><h3>3. Default opt-in</h3><p>Many platforms pre-select the opt-in option for their feature or service. They design the UI such that users have to explicitly opt-out of a feature or a service. If users miss opting out, then they are signed up for a paid or promotional offering. This “assumed consent unless explicitly mentioned otherwise” can be extremely frustrating for the users.</p><p>You must have observed many websites pre-select donation option when you are about to pay for the purchased items or the flight tickets (<em>image below</em>). Many of the users pay the money because they miss the checkbox. Observe the checkbox in the image below. You would agree that it is easy to overlook. Why was it designed this way?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0VtkVI37BNTIVE-NNFqIdA.png" /><figcaption>Assumed consent</figcaption></figure><p>Similar tactics are employed while taking the user consent for receiving marketing or promotional emails. While users sign-up for a service, they accept terms and conditions. Generally, there are two checkboxes, one to accept the terms of use and another to subscribe to promotional emails. Both the boxes are selected by default. Users have to deselect one to opt-out of receiving the promotional emails.</p><p>According to a study mentioned in the book “Evil by Design”, pre-selecting the consent resulted in a 48% increase in the users opting for receiving promotional emails.</p><p>In another scenario, when the option was not selected and the description was changed to “Do <em>not</em> notify me about the future surveys”. The number of users providing the consent (<em>keep checkbox unselected</em>) increased by 98%.</p><p>User studies show that the users tend to take the path of least resistance. Users tend to go with the default choices made for them. Many A/B tests also prove that users are more likely to stay with the selected option than to read the details and change it.</p><h3>4. Below are a few more UI elements that mislead users into making a choice that they might not want</h3><p>4.1 In the below image, it seems that the “No thanks” button is disabled. It is because of the colour of the button and the text.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/660/1*8gsEITx4jZlFft0_HNbRVA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/698/1*qcqWpc0QyH48ZOfuxetDCA.png" /></figure><p>4.2 Advertisement banner in many of the websites has a close button that is extremely difficult to locate. Users inadvertently click such ads.</p><p>4.3 Many times the options of Interactive Voice Response (IVR) are designed in a manner that it makes it tedious to reach to the customer care executive.</p><h4>As a PM, you must decide how you want to achieve all the user and business metrics. Will you employ evasive designs to achieve the objectives but lose out on users’ trust or will you make the right design choices?</h4><p><em>Disclaimer: I have used images from the existing websites and tried to remove all the identifiers. I don’t wish to point out to a specific website or a business. Views expressed here are mine and not of my employer.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e5aaf58d8c96" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Product Dilemma]]></title>
            <link>https://mohitdsoni.medium.com/the-product-dilemma-475daf919916?source=rss-587aa3bba9cb------2</link>
            <guid isPermaLink="false">https://medium.com/p/475daf919916</guid>
            <category><![CDATA[product-management]]></category>
            <category><![CDATA[the-social-dilemma]]></category>
            <category><![CDATA[education]]></category>
            <category><![CDATA[addiction]]></category>
            <category><![CDATA[technology]]></category>
            <dc:creator><![CDATA[Mohit Soni]]></dc:creator>
            <pubDate>Tue, 22 Dec 2020 10:22:36 GMT</pubDate>
            <atom:updated>2020-12-22T10:22:36.775Z</atom:updated>
            <content:encoded><![CDATA[<p>The Social Dilemma — a recently released documentary on Netflix — catapulted concerning impact of social media to the fore of everyone’s mind. It made me think about the impact that a group of engineers, designers, and product managers are making on our society.</p><p>As a product manager, I want to make my product more engaging, meet revenue goals, and expand my user base but to what end.</p><p>Any product is built to solve some user problems and to achieve some business objectives. More often than not, the most significant business objective is to make money to maximize shareholders’ value. Hence, a product manager would always want to build an engaging product that has a large user community and the one that makes money.</p><p>The point seems trivial enough and is applicable to all the businesses. However, to understand the consequences of the social media platforms, we need to first dig deeper into how these platforms make money and what are they selling.</p><p>All major social media platforms are free to use for its users. They make money through advertisements shown to the users. If a user spends more time on the platform, then the platform can show more advertisements to the user. It will drive revenue growth. The goal of increasing user engagement stems from this business requirement.</p><p>Another metric that companies track is called click-through rate (CTR). It measures the ratio of users, who clicked on an advertisement to the total users who saw the advertisement. Companies need to show relevant advertisements to improve the CTR and to show relevant advertisements, companies need users’ data. This data helps in building machine learning models that can predict a user’s behaviour or action or need. So, while browsing whenever you get a message that asks your permission to collect your data to show more relevant content, rest assured that this data will be used to show you personalised advertisements.</p><p>This all looks pretty straight forward. So, what’s the problem?</p><blockquote>“The gradual, slight, imperceptible change in your own behaviour and perception is the product.” <br>-Jaron Lanier (The Social Dilemma)</blockquote><h4>Social Media Addiction</h4><p>The social media companies are employing scientific methods to integrate their product into the daily lives of its users. These methods are powerful enough to cause behaviour changes among its users. The Cue, Trigger, Response, Reward framework is used to make products addictive. It’s the same mechanism that makes people addicted to a slot machine.</p><p>There are hundreds of people and powerful algorithms working to grab and keep your attention. Have you ever wondered how the time flew by while scrolling to your social media feed? The feed never ends, thanks to the infinite scroll user design. Each refresh of the feed might give you fresh updates and to make it more engaging now you have a series of short videos that are shown by the platform without you selecting which video to watch. All these and many more such features are carefully crafted to keep you hooked to the platform.</p><p>Thousands of combinations of colours are tested to select the most engaging ones. Every tiny detail that goes into the product is tested on multiple user groups using A/B testing. Seemingly harmless notification, the icon to indicate that other person is typing over chat etc. all are cues to grab your attention.</p><p>These products are designed to provide you with a quick fix when you feel bored, sad, anxious, nervous etc. How many times in a day do you open one of the social media apps? Is phone first thing you check after waking up and last before you go to sleep?</p><h4>Information Bubble</h4><p>We are more connected with others but more polarized today than ever before. We all live in an information bubble that is created around us via recommendation engines of the products that we use.</p><p>How many times do you search a specific content to watch vs how many times do you view a recommended content? How much control do you exercise on your social media feed? Do you follow people or groups whose ideas are not in line with yours?</p><p>Have you ever wondered why someone close to you is blind to the logic when it comes to any political or religious opinion? It is because all of us are living in our information bubble. The social media platform and the recommendation engines cement the boundary of this bubble. This problem is accentuated by the fact that people follow information sources, influencers, groups, video channels etc., that reinforce their beliefs or their version of the truth.</p><p>These bubbles do not represent physical proximity. You might be getting completely different recommendations and information from a person sitting right next to you. The recommendation engines are finely tuned to show content based on your inclination and keep you engaged on the platform.</p><p>You must have heard about the flat-earth society. It’s a group of people who believe that earth is flat despite the scientific evidence proving otherwise. Need I say more?</p><h4>Misinformation</h4><p>Another major problem is an unchecked spread of misinformation through social media.</p><p>According to one <a href="https://news.mit.edu/2018/study-twitter-false-news-travels-faster-true-stories-0308">study</a>, “Fake news travels 6 times faster than real news on social media”. It is alarming especially when we are battling a global pandemic. <a href="https://www.kpbs.org/news/2020/may/20/researchers-nearly-half-of-accounts-tweeting/">Researchers at CMU</a> found over 100 false narratives involving corona virus by accounts controlled by Bots.</p><p>An example shown in The Social Dilemma was the pizzagate incident, where a piece of fake news about human trafficking from the basement of a specific pizza place resulted in a man showing up to that place with a gun to rescue people. It was later found that the pizza place did not even have a basement.</p><p>A few products have come up with steps to curb the spread of fake news or flag false information. WhatsApp, for example, limits message forwarding to only 5 chats. Twitter has fact-check to warn users about potentially false information. However, these measures are not proving enough.</p><p>Social media addiction, information bubble, and misinformation have far-reaching consequences. These impact us individually and as a society. It is not enough for you to distance yourself from these platforms and think that will solve the problem.</p><p>Today social media has become a tool in the hands of rich and powerful to alter the opinion of the masses. Propagandists find it far easier to manipulate the opinion of the target population on the critical issues for their benefits. It has a far-reaching impact on election results, policy decisions, global health etc.</p><p>The world we live in today is more divided than ever before about the issues like immigration, vaccination, unemployment, etc. The recommendation engines further act as a catalyst in polarizing society on these issues.</p><p>According to a Wall Street Journal <a href="https://www.wsj.com/articles/facebook-knows-it-encourages-division-top-executives-nixed-solutions-11590507499?mod=hp_lead_pos5">article</a>, 64% of all the extremist group joins were due to recommendation system employed by social media platforms.</p><p>As social platforms increasingly integrate with our lives, research shows a wide range of adverse effects on our mental health, self-image, and happiness. A <a href="https://honestdata.com/facebook-addiction">study</a> published by Honest Data shows that 30% of the surveyed 2000 Americans feel anxious if they don’t check social media platform for a couple of hours (<a href="https://ledger.humanetech.com/"><em>source</em></a>). This <a href="https://www.aeaweb.org/articles?id=10.1257/aer.20190658">paper</a> by Allcott et al. shows that a month away from social media platforms can lead to a significant improvement in emotional well-being and reduction in political polarization.</p><p>Arguably the most impacted ones are the kids who are born in the world of social media. Their idea of beauty and happiness revolves around the false image portrayed by social media. These kids compare their real-life with the fabricated virtual life of others on social media. Their self-esteem is tied to the number of followers, likes, and comments that they get. Many employees and executives, who are creators of social media platforms, don’t allow their kids to use social media. They are acutely aware of the adverse impact it has on the kids.</p><h4>Is it all bad?</h4><p>There are many positive stories of social media. Hundreds of thousands of people have searched employment opportunities using social media, house hunting using FB groups is a norm now, many people have found their significant other through social media, abundant educational resources are available to anyone with a smartphone and an internet connection. People from different walks of life have come together on social media for many good causes. It has even aided in starting people’s movement in many countries. The positive stories and the positive impact cannot be undermined or ignored. However, we must ensure that negatives do not overshadow positives.</p><blockquote>“It’s simultaneous utopia and dystopia.”<br>-Tristan Harris (The Social Dilemma)</blockquote><p>It is unfair to pin the blame solely on creators or solely on the users for the consequences of new technology. Both groups need to do their part.</p><h4>Action for Product Creators</h4><p>Before building any product or feature, start the discussion about its impact on society. You need to be the voice that brings up the consequences of the change. Anticipate and exaggerate the impact of the change on all the groups of the society/users.</p><p>Learn both the success stories and the consequences from the existing platforms. Can you design an A/B test to study not only the success metrics but the broader impact of the feature as well?</p><p>Can you determine the threshold time for the product usage and signal users to stop? YouTube has one such feature where it indicates users when they have been continually watching videos for a while. Similarly, there should be more checks to break the addiction of the users.</p><p>Hyper-personalization can make product or information more relevant to the users, but it can also create an information bubble. Can recommendation engines be tweaked to show an alternate view of the topic?</p><h4>Action for Product Users</h4><p>Turn off notifications or only keep the required ones. You don’t want to be distracted by every little event happening out there. Notifications are a gentle nudge to the users to use the product. It’s the first step in the cycle to keep you engaged for a longer period. Stop it at the very beginning and turn it off.</p><p>There are many alternate products to the existing ones that don’t track you over the internet and do not sell your data to third parties. One such free to use search engine is <a href="http://duckduckgo.com/">DuckDuckGo</a>. You can also install browser plugins that stop third party websites from collecting your data.</p><p>A way to fight misinformation is to follow trusted publications. There are many websites that do a fact check and debunk many false stories. You can follow those sources to get the right information. Do not believe a sensational story unless it comes from a trusted source.</p><p>Following people with opposing views can help you get out of the information bubble and understand both the sides of an argument. Finally, talk about these issues, make others around you aware of the problem.</p><h4>Conclusion</h4><p>Social media platforms are under more scrutiny than others because they impact billions of people. However, product managers, designers, and engineers across all the industries should consciously work to ensure that their platform does not become evil. The gaming industry needs to ensure that games do not adversely impact the mental health of the players. The financial industry needs to ensure that their algorithms don’t discriminate against any group in the society based on race, sexual orientation, gender etc.</p><p>The role of policymakers is also significant in fighting the negative consequences of social media. We need concrete regulation and laws around the highlighted issues.</p><p>The product dilemma is the choice of making a product more engaging and profitable vs curbing the consequences in the process of doing so.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=475daf919916" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How did I create a product plan from scratch?]]></title>
            <link>https://medium.com/design-bootcamp/how-did-i-create-a-product-plan-from-scratch-d8a1d770f4e4?source=rss-587aa3bba9cb------2</link>
            <guid isPermaLink="false">https://medium.com/p/d8a1d770f4e4</guid>
            <category><![CDATA[product-design]]></category>
            <category><![CDATA[mvp]]></category>
            <category><![CDATA[user-personas]]></category>
            <category><![CDATA[product-management]]></category>
            <category><![CDATA[new-product-development]]></category>
            <dc:creator><![CDATA[Mohit Soni]]></dc:creator>
            <pubDate>Sat, 12 Dec 2020 10:50:48 GMT</pubDate>
            <atom:updated>2020-12-15T05:49:24.259Z</atom:updated>
            <content:encoded><![CDATA[<p>A few weeks back, I had a discussion with one of my friends over Travel and Expense (T&amp;E) management process in different companies. I, being part of a large corporation, had a very different experience from what my friend, who was part of a start-up, described.</p><p>It got me thinking that if I were to create a T&amp;E product for a small company, how would I approach it as a product manager. This thought led to the creation of a product plan that I am sharing with you in this post.</p><h3><em>Problem Statement</em></h3><p>Build a product to cater to T&amp;E management for Indian small and medium businesses.</p><h3>Background research on T&amp;E</h3><p>Some research on T&amp;E industry helped me understand the existing T&amp;E management process/systems. I could also understand the challenges faced by companies and employees. It also helped me uncover challenges that I didn’t think initially. E.g. just by reading the problem statement, I never thought about tax (GST) implications of the reimbursement and did not consider the finance team as a user group.</p><p>According to a <a href="https://www.businesstravelnews.com/More-News/Survey-T-E-Second-Most-Difficult-Operating-Expense-To-Control">study</a> conducted by Forrester Research, T&amp;E spend is the second most difficult operating expense to manage. A majority, 59%, of the surveyed companies still rely on a manual process for tracking and managing T&amp;E spend. The executives of these companies do not get visibility into T&amp;E spend due to lack of easy reporting and dashboard capabilities.</p><p>Another <a href="https://economictimes.indiatimes.com/news/company/corporate-trends/about-half-of-indian-firms-lack-system-to-detect-travel-expense-frauds-survey/articleshow/66226549.cms?utm_source=contentofinterest&amp;utm_medium=text&amp;utm_campaign=cppst">study</a> on Indian companies reports that 33% of the companies do not have a GST compliant T&amp;E solution. Even the ones that have still need employees to submit the hardcopy of bills to claim input credit.</p><p>After reading multiple such studies, I narrowed down below-mentioned areas that the product would solve for:</p><ul><li>The tedious expense filing process</li><li>Enforcing policy compliance</li><li>Audit and Expense reporting</li><li>Expense Fraud</li></ul><h3>Assumptions and Scope</h3><p>Based on my research and understanding, I made a few assumptions about the target market and customers.</p><ul><li>Target market segment uses reimbursement process vs advance payment process</li><li>The first set of target customers either have a completely manual process or are using one of the popular account management systems and HRMS system</li></ul><h3>Target problems</h3><p>To understand the problem areas identified in the research better, I listed the existing process/solution, need, consequences etc. for each of these problem areas.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*U4KI0kLnr4EJNi9Bfc8ZFQ.png" /><figcaption>Problems to be addressed</figcaption></figure><h3>Market analysis and opportunity areas</h3><p>To understand what existing products are solving for and to identify the opportunity areas, I did a competitive analysis of a few of the existing players in T&amp;E management space around the target problem areas. Reading user grievances and positive feedback also helped me in understanding the gaps and strengths of existing solutions.</p><figure><img alt="Image represents comparison of features and opportunity areas among existing players on the basis of target problem areas" src="https://cdn-images-1.medium.com/max/1024/1*QIs90hoKW9kawBEwaNYN8A.png" /><figcaption>Competitive analysis (<a href="http://www.g2.com/">source</a>)</figcaption></figure><h3>User Personas</h3><p>Next step was to identify the user personas that will use this product. Along with identifying the personas, I also listed what is the role of each persona, what functions would they perform, and which features they would require to perform those functions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Cr2T-vVUmjZQoicuSRpxyQ.png" /><figcaption>User personas, role, function, and value proposition</figcaption></figure><h3>Minimal Viable Product (MVP)</h3><p>The idea of creating an MVP is to identify a minimal set of product features that will meet the requirements. It involves not only prioritizing product features but also first prioritizing the requirements/problem areas to be solved.</p><p>For this product, I prioritized Expense Management and Audit &amp; Reporting as high priority items as these are foundational components of T&amp;E. Policy Compliance was assigned medium priority and Fraud Detection is marked low priority.</p><p>In MVP, I have listed where (web or app) the features will be released and also added features like user onboarding, user roles, and access management. Below is the list of MVP features and their priority.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BK7TTrNtYkmspkemSTVZUQ.png" /><figcaption>MVP and feature prioritization</figcaption></figure><h3>Feature Roadmap</h3><p>Feature roadmap defines how the product would look like in next 12 to 18 months. I listed the features that will be developed after MVP is built, keeping the problem areas and platform in mind.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Cr1a71OEKcm-JXMMCfv_Lw.png" /><figcaption>Feature roadmap</figcaption></figure><h3>How would it look?</h3><p>Below are wireframes of a few of the screens for mobile and desktop versions of the product (Claim).</p><h4>Home page of mobile app</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XpkxPCcnpvp2hVrbR2LbeQ.png" /><figcaption>App home page — Mobile</figcaption></figure><h4>Submit expense flow</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KH6EgHLIWs4ifrJ1ZPW99Q.png" /><figcaption>Submit expense flow (1/4) — Add expense</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9lhIzHNpNsnbd92KUvD8rQ.png" /><figcaption>Submit expense flow (2/4) — Create new expense</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m_jhoIoroPSWxgJB4dsdzA.png" /><figcaption>Expense claim flow (3/4) — Add items</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ersIVgs7zLMzS3DpUbYJZw.png" /><figcaption>Expense claim flow (4/4) — Submit claim</figcaption></figure><h4>Expense approval screen — Desktop</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PhpKXM09_WpUyNqkiWvV_A.png" /><figcaption>Expense Approval (Desktop)</figcaption></figure><h3>Conclusion</h3><p>A small discussion led me to create this product plan for a T&amp;E portal. Please let me know what would you have done differently and how would you approach this problem?</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d8a1d770f4e4" width="1" height="1" alt=""><hr><p><a href="https://medium.com/design-bootcamp/how-did-i-create-a-product-plan-from-scratch-d8a1d770f4e4">How did I create a product plan from scratch?</a> was originally published in <a href="https://medium.com/design-bootcamp">Bootcamp</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>