<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Shyam Shinde on Medium]]></title>
        <description><![CDATA[Stories by Shyam Shinde on Medium]]></description>
        <link>https://medium.com/@shyamshinde?source=rss-bfa5fabce3e2------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*jqV6f9R_sp_hJA-DW8qxHw.jpeg</url>
            <title>Stories by Shyam Shinde on Medium</title>
            <link>https://medium.com/@shyamshinde?source=rss-bfa5fabce3e2------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 18:58:30 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@shyamshinde/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Finding top performing MP in 16th lok sabha]]></title>
            <link>https://medium.com/@shyamshinde/finding-top-performing-mp-in-16th-lok-sabha-34897e478a71?source=rss-bfa5fabce3e2------2</link>
            <guid isPermaLink="false">https://medium.com/p/34897e478a71</guid>
            <category><![CDATA[elections]]></category>
            <category><![CDATA[lok-sabha]]></category>
            <category><![CDATA[indian-politics]]></category>
            <category><![CDATA[politics]]></category>
            <category><![CDATA[india]]></category>
            <dc:creator><![CDATA[Shyam Shinde]]></dc:creator>
            <pubDate>Sat, 16 Mar 2019 18:36:06 GMT</pubDate>
            <atom:updated>2019-03-17T05:08:25.300Z</atom:updated>
            <content:encoded><![CDATA[<h3>Finding top performing MPs in 16th Lok Sabha- Part 1</h3><p>17th <em>Lok Sabha</em> elections will be happening in months of April-May 2019. <br>I was curious to know which current MPs really performed well in parliamentary sessions. I got to know that from <a href="https://www.prsindia.org/"><strong>prsindia</strong></a><strong> </strong>I can get data on the performance of MPs in parliamentary sessions.</p><p>After downloading the data, first question that came to my mind was —</p><blockquote><strong>How can I define a MP is performing well in parliamentary sessions?</strong></blockquote><p>I don’t have an accurate measure by which I can evaluate each MP. But in general A MP can be said to performing well if MP is<strong><em> </em></strong><em>attending session regularly, raising questions and participating in debates</em><strong><em>.</em></strong></p><blockquote><strong>What does data consist of ?</strong></blockquote><p>Data have basic personal information like <em>name, party name, his constituency/state name, details about the education</em>. Luckily data have other required features as well on which I wanted to evaluate the MPs performance. Following are the other features that I am using</p><pre>1. Attendance of MP <br>2. State&#39;s Attendance average <br>3. National Attendance average<br>4. Number of debates in which MP participated <br>5. National Debates average<br>6. State&#39;s Debates average<br>7. Number of questions raised <br>8. National Questions average <br>9. State Questions<br>10. Number of introduced &quot;Private Member Bills&quot;<br>11. National Private Member Bills average<br>12. State&#39;s Private Member Bills average</pre><blockquote><strong>How do I find performance score ?</strong></blockquote><p>After exploratory analysis, I get to know none of the above features are following normal distribution. So average/mode of above features are not much helpful.</p><p>I decided to find a <strong>performance_score metric </strong>by combing above 12 features so that I can rank MPs on it. I am using below-mentioned logic to find the performance_score.</p><pre>1. I converted each above features number to their percentile score. </pre><pre>2. Now each feature score was between 0 to 100,  I can add them to get their performance_score. </pre><pre>3. Final performance_score was again scaled for range 0 to 100 </pre><pre><strong>performance_score = sum(percentile_score_of_each_feature) * 100/1200</strong></pre><blockquote><strong>Toppers list from non-NDA parties</strong></blockquote><p>Here, I have excluded MPs from the NDA alliance. I want to call these MPs as leaders from the opposition who performed well. Proud to say that lady <a href="https://www.prsindia.org/mptrack/supriyasule"><strong>Supriya Sule</strong></a> topped this list.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1XmpiP08k4Ihwx_0a5TBqg.png" /><figcaption>Top leaders from opposition parties</figcaption></figure><p>My observations on the above list —</p><pre>1. 9 out of 14 from top opposition leader are coming from Kerala.   May be other opposition MP should learn from them.</pre><pre>2. All MP who topped the list have education Graduate / Post Graduate. </pre><blockquote><strong>Toppers list from NDA parties</strong></blockquote><p>Here I have included MPs from NDA alliance only</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zwsIQDWCvCFxn31PYcntvw.png" /></figure><p>My observation on the above list —</p><p>There is no MP from <strong>Uttar Pradesh </strong>in the above list. Please note NDA got a large number of seats from Uttar Pradesh. I have attached a list of only top 15 MPs, but I took the state-wise distribution of top 100 MPs from all parties and surprisingly found that only 5 MPs are from Uttar Pradesh. <br>Point here I want to make is<strong>, Current situation in UP is due to non-performing MPs.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/474/1*RW0l-A_794lAPFpNLPmLKA.png" /><figcaption>State wise distribution of top 100 MPs</figcaption></figure><blockquote><strong>Does education impacts the performance of MP ?</strong></blockquote><p>I divided all MPs by their education level in four categories <em>School, Diploma, Graduate and Post Graduate</em><strong><em>. </em><br></strong>* MPs having School or Diploma education are <strong>25%</strong> of all MP<br>* In top 100 MPs, only <strong>15% </strong>MP (expecting ~25%)<strong> </strong>are from School or Diploma education. <br>* I did not find much difference in average performance_score of different education category.</p><p>Though the percentage of MPs having less education was low in top MPs list, I would say education is not impacting the performance of MP given that average/mode was almost same across different education category. Also, we have to consider the competition from educated MPs as well. :)</p><blockquote><strong>Validity of analysis</strong></blockquote><p>When I checked top 15 MPs from NON-NDA parties(Check above list). I found that most of the MP have got <a href="https://en.wikipedia.org/wiki/Sansad_Ratna_Award"><strong>Sansad Ratna Award </strong></a>in the year 2014-2019. It confirms that <strong><em>Sansad</em></strong> also agrees with my performance score logic and findings.</p><blockquote><strong>How do I find performance index of MPs by their actual ground work ?</strong></blockquote><p>We can not confidently say MP is performing well just by his score in the parliamentary session. We need more data and analysis over it, so stay tuned for part 2 of this blog.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=34897e478a71" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[SensAI Predict — Building ticket classification service using NLP and ML]]></title>
            <link>https://medium.com/helpshift-engineering/sensai-predict-building-ticket-classification-service-using-nlp-and-ml-1180ac5e2cb6?source=rss-bfa5fabce3e2------2</link>
            <guid isPermaLink="false">https://medium.com/p/1180ac5e2cb6</guid>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[classification]]></category>
            <dc:creator><![CDATA[Shyam Shinde]]></dc:creator>
            <pubDate>Tue, 24 Jul 2018 10:24:22 GMT</pubDate>
            <atom:updated>2019-10-21T16:27:30.343Z</atom:updated>
            <content:encoded><![CDATA[<h4>By <a href="https://twitter.com/shindesh11">Shyam Shinde</a></h4><p>Helpshift’s customer service platform is used by companies such as Epic Games, Microsoft and Zynga to receive support tickets from their users via mobile apps, web chat and email. Thousands of new tickets are reported daily in the platform across our 2000+ customers and 2+ billion end-users worldwide. Every good company wants its customer service tickets to be resolved as soon as possible. To achieve that quick turnaround, it’s imperative that a ticket reaches the right agent who is best skilled at handling it. Hence, ticket classification and routing are two important functions to achieve the <a href="https://www.helpshift.com/blog/three-es-customer-service/">efficiency strategy</a> for a customer service operation.</p><h3>How is ticket classification done today?</h3><p>Today, companies have dedicated people focused on the mundane-yet-important work of ticket classification. As it is a manual step before tickets finally reach the agent, customer service centers face problems of scalability. As more tickets start coming in every day as the company grows, they need to hire more agents for classification, making the model slow, complicated to coordinate and unscalable. Because of the repetitive nature of this work, customer service centers also face problems of attrition and the company needs to hire and train their personnel frequently. This involves significant cost for the company.</p><p>Due to the nature of the customer service industry, there is a steep learning curve and even when skilled agents are available, there is an inadvertent delay in ticket handling and processing due to the classification of tickets. Due to this, the time for the first reply to the customer can increase even though the support center has sufficient and skilled agents to work on the tickets.</p><p>After speaking with our customers, it got us thinking — why not build a ticket classification service into our system?</p><h3>What are the challenges in building automated ticket classification?</h3><p>We wanted to build a system that could intelligently classify the tickets even before the customer support agent gets to see it, and depending on applied labels, an organization can set actions like routing to a particular agent.</p><p>The challenges in building such a classification system are:</p><ul><li>To not add significant latency in ticket creation because we are introducing<br> a significant step in ticket classification.</li><li>It must work for all of our customers.</li><li>Tickets are in different languages, so we needed a classification system supporting tens of different languages, ranging from Chinese to Portuguese.</li><li>If we were going to leverage automatic routing of tickets using applied labels, the accuracy of the classification system had to be as good as or better than existing company-specific manual routing.</li></ul><p>Given the above challenges, we needed a completely automated workflow, where our customers can build their own classification model through our platform in a few steps. This classification model will be used to apply an unique label for each ticket. Using this unique label, admin can route a particular label’s ticket to a <a href="https://support.helpshift.com/kb/article/how-do-i-add-predict-labels-into-automations/">group of agents</a>.</p><p>In this article, we will discuss the backend architecture of label prediction and present how the label prediction service has enabled automatic routing of incoming tickets.</p><h3>What does the ticket classification product look like?</h3><p>In this example, an end-user has filed a ticket to know the price of a particular bag. Our prediction service has labeled the ticket “billing”:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/724/1*nSwF82Jllh6M5g_E_IgyMw.png" /></figure><h3>How to use applied label on a ticket?</h3><p>Applying a correct label is not the end goal of ticket classification. We want to enable our customers to trigger particular actions like routing to a particular agent or group of agents (we call this a queue), send an auto-reply to the ticket, etc.</p><p>Here is one example — where action is set to <strong>Assign ticket to product queue if predicted label is billing</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nhRbwY-o03dYQ7jP7UE1nA.png" /></figure><h3>How did we build the ticket classification product?</h3><p>We, the data intelligence team, started exploring different approaches to solve ticket classification. One method was to find a set of keywords for each label. When new ticket comes, we will search keywords of each label in ticket’s text and predict a label on search result. But it was not possible to find unique set of keywords for each label. Example, label like order_tracking and order_nondelivery have common keywords like order placed, return, wait and delayed.</p><p>Next, we explored machine learning algorithms which can find patterns for each label from a dataset of the tickets filed. The machine learning experiments were promising and the early results were good. We decided to settle on a machine learning approach. The advantage of this model was that it can be updated by agent feedback on the correctness of the predicted label, which was not possible in the keyword search solution.</p><p>The engineers started working on a system which will be used to build the machine learning models and predict a unique label using pre-built model in real time. Helpshift’s label prediction service uses natural language processing and machine learning classification algorithms to accurately predict the label. The set of labels from which label is chosen is decided by each customer.</p><h3>General flow in building the model</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/790/1*x4Vy-FnSaaQm6GPPL28ytQ.png" /></figure><ol><li>The CSV dataset of tickets and their corresponding labels are required to train the classification model. The admin can use the SensAI tab in the dashboard to upload the dataset and build the classification model.</li><li>Our in-house ML platform prepares and validates the submitted tickets dataset.</li><li>Our in-house ML platform builds the classification model from the validated dataset.</li><li>Once model is built, it is stored in model store. This model is used to predict the label whenever a new ticket comes in.</li></ol><p><strong>Machine Learning steps while building model</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/784/1*nztS5XC0uX1NhRUOT-uP8w.png" /></figure><p><strong>Preprocessing</strong></p><p>The goal of the preprocessing step is to collect required words from corpus of tickets text and find association of these words in the corpus. Preprocessing starts with cleaning the ticket data by removing HTML tags, numbers, urls, etc. which are not useful to find interesting patterns inside the dataset.</p><p>After that we tokenize data by finding sentence boundaries and finding words in it. We apply stemming on words to create base words. To find words associations, we create bigrams and trigrams.</p><p><strong>Model building</strong></p><p>We generated NLP features like vocabulary, word frequency and ticket frequency matrix across labels. We started experimenting classification models using naive bayes classifier and logistic regression. But these common classification algorithms were not good enough to predict accurate label from large number of possible labels (solution space). These algorithms were predicting most common labels when the user writes short text in their ticket. This happened often when user submits ticket from their app or from<br> webchat window in conversational UI.</p><p>The data science team then started experimenting ML models which will work on short text tickets. We benchmarked accuracy of different algorithms against test data. We found that no single independent model was performing better than other models when compared on different parameters. In order to improve the accuracies, we decided to ensemble different algorithms and apply ADABoost technique. It was observed that, the overall accuracy of classification was better than individual independent models.</p><p>The last step in model building is running validation on test data set and determining <a href="https://en.wikipedia.org/wiki/Precision_and_recall">precision and recall</a> for each label and model accuracy. From validation results, we suggest optimal confidence threshold that should be applied on model to predict the label confidently.</p><h3>General flow in predicting label on new ticket</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/443/1*T3DYHBC4hCFVHgFfHYKc4g.png" /></figure><ol><li>Once a new support ticket comes in, it is collected by Helpshift servers. The Helpshift backend server knows company name for which end user has submitted ticket.</li><li>The backend service collects information such as whether label prediction service is enabled by our customer. If label prediction is enabled, then the backend service detects language of ticket text. The collected features are sent to label prediction service along with ticket text.</li><li>The prediction service fetches required models from the model store and predicts score for each available label in the model. The label with the highest score is attached to the ticket.</li><li>Once a label is attached to the ticket, our customer can decide what to do with that label — route to a particular team of agents (queue), auto-reply with a specific message, etc. This way, our customer can combine predicted label with Helpshift’s strong workflow features to enable automation of sophisticated workflows. <br>For example, if the ticket is about lost account, an automatic reply can be sent pointing to a FAQ explaining the required information and steps. Another example is if the ticket is about app crash, it might be routed to a high-priority queue where it is auto-assigned to the currently available agents.</li><li>When an agent sees applied label on ticket, they have an option to mark the predicted label as a wrong. The agent can also correct the predicted label for a ticket. This corrected label’s data is sent to ML platform as a feedback to classification model. Feedback collected from agent is used to update the model so that model is tuned for better prediction. This feedback loop can seem magical because with more feedback from agents, the model becomes more accurate automatically.</li></ol><p>This enables two things:</p><ul><li>Accuracy of model can improve to 90+%</li><li>New types of issues can be identified if agent marks predicted label as a wrong for many tickets.</li></ul><h3>Lessons learned so far</h3><p>While we are continuously working on tuning our system for better results, here are some lessons learned so far:</p><ol><li><strong>Update the model with negative feedback</strong></li></ol><p>Collect agent’s feedback on predicted label. This feedback is used to update model so that classification learns over time for accurate prediction. If agent marks predicted label as a wrong, we penalize the model parameters for predicting wrong label. In our case, we have observed that model always performs better when we update the model on feedback data.</p><p>Updating the existing model by feedback helps in two ways -</p><ul><li>Model starts performing better in accuracy. The model learns from its wrong prediction!</li><li>We do not need to build the model from scratch periodically when we have new set of data.</li></ul><p><strong>2. Try to create a label for a unique type of ticket category</strong></p><ul><li>When a model is built for a set of labels where each label represents a unique type of ticket category, such model always performs better in accuracy when compared with a model which has label representing more than one type of ticket category.</li><li>Example, labels order_nondelivery and order_delayed represents same ticket category, so avoid having such labels for predictions.</li></ul><p><strong>3. Build separate model for each languages</strong></p><p>We have found that language dependent model gives better accuracy when compared with a model built on all languages ticket text.</p><p>If you’re interested in working on such problems, <a href="https://www.helpshift.com/company/careers/">we are hiring, join us!</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1180ac5e2cb6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/helpshift-engineering/sensai-predict-building-ticket-classification-service-using-nlp-and-ml-1180ac5e2cb6">SensAI Predict — Building ticket classification service using NLP and ML</a> was originally published in <a href="https://medium.com/helpshift-engineering">helpshift-engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>