<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Shikha Verma on Medium]]></title>
        <description><![CDATA[Stories by Shikha Verma on Medium]]></description>
        <link>https://medium.com/@fpm16shikhav?source=rss-34a4c3da3609------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*s_S58SefuYAsjrKL8c2PkQ@2x.jpeg</url>
            <title>Stories by Shikha Verma on Medium</title>
            <link>https://medium.com/@fpm16shikhav?source=rss-34a4c3da3609------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 14:38:55 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@fpm16shikhav/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Opening the Black-Box: LIME v/s Shapley values for Explainable AI (XAI)]]></title>
            <link>https://medium.com/@fpm16shikhav/opening-the-black-box-lime-v-s-shapley-values-for-explainable-ai-xai-d6b3961e0d7?source=rss-34a4c3da3609------2</link>
            <guid isPermaLink="false">https://medium.com/p/d6b3961e0d7</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machinelerning]]></category>
            <category><![CDATA[business-intelligence]]></category>
            <category><![CDATA[explainable-ai]]></category>
            <category><![CDATA[analytics]]></category>
            <dc:creator><![CDATA[Shikha Verma]]></dc:creator>
            <pubDate>Tue, 22 Mar 2022 17:16:10 GMT</pubDate>
            <atom:updated>2022-03-28T04:06:16.356Z</atom:updated>
            <content:encoded><![CDATA[<p>Comparing LIME and Shapley values for explaining results of Machine Learning models</p><h3><strong>What is Explainable AI (XAI)? Why do we need it?</strong></h3><p>As Machine Learning and Artificial Intelligence models have become more pervasive in data-driven decision-making tasks, there have been concerns around these models being black-boxes- the process through which they arrive at a prediction being complex/unknown to decision-makers. A Machine Learning model is a garbage-in-garbage-out (GIGO) system whose predictions are only as good as the data fed into it. These models are in no way ‘objective’ as they can perpetuate the bias and stereotypes captured in the data. This is dangerous as<a href="https://journals.sagepub.com/doi/full/10.1177/0256090919853933"> ML models are increasingly forming the basis of recidivism predictions, teacher performance evaluation systems, predictive policing, and recruitment systems.</a> In short:</p><blockquote>What these models say, impacts REAL people and their everyday lives AT SCALE.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/988/1*dZjEhydJ_nUb95cdpB4eBA.jpeg" /><figcaption>Source: altexsoft.com</figcaption></figure><p>XAI is an upcoming area of research that aims to develop methods and techniques that ‘demystify’ the predictions of these models and attempt to explain them in a manner accessible to the decision-maker. I will use explainability and interpretability interchangeably in this article.</p><p>Methods for model interpretability can be broadly divided into two categories</p><p><strong><em>a) Global Methods</em></strong><em><br></em>provide an ‘average’ explanation for the model which holds for all instances. Consider a model trained to predict housing price based on its characteristics like #rooms, #floors, pets allowed/not, proximity to school etc. In this case, a global explanation method would provide an explanation of the nature: On average, for the instances in the dataset, #floors contribute most to the overall prediction. <br>However, if I am a homeowner and I want to know that for my specific instance, which feature explains the predicted price the most, global methods won’t be suitable</p><p><strong><em>b) Local Methods</em></strong><em><br></em>provide an explanation for a specific instance.<br>So, a local explanation method might be used to give the features that contribute the most to the predicted price of MY house, not for all instances on average.</p><p>Shapley values and LIME are among the most popular local explanation methods. Please refer to the table below for a complete list of global and local explanation methods.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JEsWz4KflnpggVcRaF-yeA.png" /><figcaption>Global and Local Methods for Model Interpretability (<a href="http://christophm.github.io/interpretable-ml-book/">Molnar,2022</a>)</figcaption></figure><p>In the following section, I discuss how to decide between LIME and Shapley values.</p><h3>Shapley Values v/s LIME: What should you choose?</h3><p>TLDR- It depends on what you are looking for in the explanations and how do you plan to use them for business decisions.<br>Long version- read on….</p><p><strong><em>Shapley Values</em></strong><br>Aims to explain how much each feature has to contribute in deviating the current prediction from the ‘average’ prediction for the dataset. It does so by measuring the impact of each feature across different feature combinations (coalitions in game-theoretic terminology) <br>Shapley values are driven by a theoretical underpinning of game theory where features are players in a game and prediction is the payout. It has the property of efficiency as the feature contributions will add up to the difference of current sample prediction and ‘average’ prediction. <br>However, it is computationally expensive as the method involves exploring 2^(#features) coalition/feature combinations for multiple iterations to arrive at the Shapley values. Additionally, it can’t provide what-if explanations. For example- If annual_income comes up as the top feature that explains why a loan was denied to a particular applicant, the decision can’t ascertain if the model decision would flip if the annual_income would increase by $10k.</p><p><strong><em>LIME</em></strong><br>LIME aims to obtain explanations for individual predictions by fitting surrogate models in the section of feature space which is proximal to the chosen data instance, instead of trying to explain a black box model entirely. It is model agnostic as the local surrogate models could be built to explain the predictions given by any black-box method- neural network, a random forest, or XGboost. The local surrogate models are trained on new data instances created by adding disturbances to the data instance,thereby creating a ‘neighbourhood’, for which explanations are needed.</p><p>LIME values can provide what-if explanation by plugging in different values of features into the local model and measuring how the prediction changes. However, they can be manipulated as defining the ‘neighbourhood’ of a sample is a non-trivial task. LIME also does not provide complete attribution of a prediction to the constituent features, unlike Shapley values</p><p>The table below summarises how LIME and SHAP differ on certain characteristics that define the application context of the two methods:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*S2PQpfKlW-SXCmm-JJMWlg.png" /><figcaption>Comparing LIME &amp; SHAP (Self-sourced)</figcaption></figure><p>Hope you enjoyed reading the article. Stay tuned for more data science focussed posts in the future!</p><p>References</p><pre>1. Molnar, C. (2022). Interpretable Machine Learning:<br>A Guide for Making Black Box Models Explainable (2nd ed.).<br>christophm.github.io/interpretable-ml-book/</pre><pre>2. <a href="https://github.com/ShikhaIIMA/Interpretable-machine-Learning-Using-LIME">https://github.com/ShikhaIIMA/Interpretable-machine-Learning-Using-LIME</a></pre><pre>3. Verma S. Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. <em>Vikalpa</em>. 2019;44(2):97-98. doi:<a href="https://doi.org/10.1177/0256090919853933">10.1177/0256090919853933</a></pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d6b3961e0d7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Finding an industry job after a Ph.D. in Data Science]]></title>
            <link>https://medium.com/@fpm16shikhav/finding-an-industry-job-after-a-ph-d-in-data-science-5a11bcb99922?source=rss-34a4c3da3609------2</link>
            <guid isPermaLink="false">https://medium.com/p/5a11bcb99922</guid>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[job-hunting]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[resume]]></category>
            <category><![CDATA[phd]]></category>
            <dc:creator><![CDATA[Shikha Verma]]></dc:creator>
            <pubDate>Fri, 25 Feb 2022 14:02:47 GMT</pubDate>
            <atom:updated>2022-02-25T14:02:47.181Z</atom:updated>
            <content:encoded><![CDATA[<p>My experiences of finding an industry job in Data Science in India as a Ph.D. graduate</p><p>Brief bio: I work as a data scientist at PayPal, with a team that develops fraud detection products using machine learning for medium and large enterprises. I have a Ph.D. in Data Science from the Indian Institute of Management, Ahmedabad, and have presented my research on streaming data and supply chain analytics at various international and domestic conferences. I have served as a visiting faculty for Artificial Intelligence and Business analytics at various business schools across the country</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7WIEIPuzPfOvudlzbVqh-w.jpeg" /><figcaption>Source- unsplash.com</figcaption></figure><p>Ph.D. programs in India are usually positioned to generate academicians and think-tank researchers. However, that should not be the case. In fact, I argue that is it is a very ’boxed’ way of thinking. A widely held belief in the doctoral candidate community is the only way to stay close to ‘research’ is to either be part of the professoriate or take up a postdoctoral position (which is a full-time job, not just a Ph.D++). However, there are plenty of positions in the industry that provide opportunities to work on exciting problems that involve out-of-the-box solution thinking. Some of them might prioritize (even incentivize) producing research output in the form of conference presentations and journal articles, while others might not. But it&#39;s unlikely an industry employer would actively discourage participation in such forums.</p><p>In this article, I present a starters kit,based on my experience, to begin your journey towards finding an industry job you would love! <br>Lets’ dive into it:</p><ol><li><strong>Don’t live with a ‘halo’</strong></li></ol><p>Following this involved a gradual attitude shift and is slightly trickier than just ticking off a task in your to-do list of getting an industry job. Doing a Ph.D. involves assimilating scholarly work (often written/presented in inaccessible ways), arriving at research questions that are worth spending 5 years on, and devising methods/frameworks that are novel. A Ph.D. degree is a ‘label’ for all of the painstaking work that goes before you become a ‘Dr.’- the highest academic <br>However, while looking for an industry job, focus more on skills that you acquired in the process rather than the degree itself.</p><p><strong>2. Build your ‘industry’ CV</strong><br>This involves rebranding your academic skills as industry-relevant skills. Ph.D. s are long-term programs (4–6 years) and it might have been a while since you stepped out in the industry. However, you did pick up skills that are useful for industry jobs, not just academic ones. You could follow these steps to build an ‘industry’ CV:</p><p>a) Rebrand skills<br>Experience in Ph.D.: <em>Skills learnt/exhibited</em><br>Volunteered for a summer school: <em>Teamwork</em><br>Assisted a course: <em>organization and stakeholder management</em><br>Taught a course: <em>Communication &amp; leadership</em><br>Conference presentation: <em>gaining visibility for your work</em><br>Co-authored a paper: <em>clear &amp; concise writing</em></p><p>b) Add numbers to experiences/achievements<br>Assisted a course?<br>add details on the number of students, number of tasks managed (evaluation, course design, crib handling, coordinating with other TAs)<br>Had a paper been accepted at a conference? <br>Add the acceptance rate, size of the audience/attendees</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cO_EdqEqiMlExmeYRCqWpw.jpeg" /><figcaption>Source-unsplash.com</figcaption></figure><p><strong>3. Read tech blogs by leading companies<br></strong>A majority of leading technology companies maintain a technology blog that documents their most exciting projects. Not only would it inform you about how mature the DS practice is at the company, but it might also give you ways to brand your research for contexts where the company has already identified problems.</p><p>Here is a list of technology blogs I enjoy reading:<br>Netflix- <a href="https://netflixtechblog.medium.com/">https://netflixtechblog.medium.com/</a><br>PayPal- <a href="https://medium.com/paypal-tech">https://medium.com/paypal-tech</a><br>Swiggy- <a href="https://bytes.swiggy.com/">https://bytes.swiggy.com/</a><br>Airbnb- <a href="https://medium.com/airbnb-engineering">https://medium.com/airbnb-engineering</a></p><p><strong>4. Use LinkedIn to your advantage</strong><br> One unusual way I used LinkedIn is to find professionals who had stepped out in the industry after a Ph.D. I took cues from their public profile and sent them requests for connections, initiating a conversation. Put together a well-framed introduction, be specific about what you want from them (referral/information about open positions/general mentorship) and add a line or two about what interested you in their journey. Following this had given a pretty decent response rate and I have been able to forge connections that stayed long after my job search. Most people are happy to have ‘informationals’ and talk about their journey and current work.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9A0WykNKtWCjhaMFNTtMlA.jpeg" /><figcaption>Source-unsplash.com</figcaption></figure><p><strong>5. Analyse Job Descriptions</strong><br>The Data Science interview process is not a really ideal one with a standard set of skills that once picked/revised, would work for interviews. The ‘asks’ in interviews include SQL, live coding, take-home assignments, case studies, product sense, statistics, machine learning,… (I am sure I let a couple out) across multiple and sometimes even in the same interview.</p><p>While applying for a specific position, ensure you do the following<br>a) Scan the job description and make a list of skills you would be tested on<br>b) Talk to the recruiter/HR partner and gain information on what you would be evaluated on in each round<br>c) Talk to folks in your network who are working in those roles/appeared for the interviews to gain ‘tribal knowledge’ on how the hiring process really works.</p><p>Following this would minimize any surprises in the process where you could be caught off-guard/unprepared.</p><p>For understanding data science job families and their requirements, I found this video by <a href="https://medium.com/u/d6c6f2b834a">Sundas Khalid</a> very informative.<br>For content on machine learning and preparing for DS interviews, one may follow <a href="https://medium.com/u/1b25d5393c4f">Emma Ding</a> and the Data Interview Pro channel (<a href="https://www.youtube.com/c/DataInterviewPro">https://www.youtube.com/c/DataInterviewPro</a>).</p><p>Finally, I’d like to say</p><blockquote>Stay at it and don’t lose hope</blockquote><p>The job search can be daunting and intimidating. Don’t let the initial rejections fool you into thinking that you don’t belong or are an imposter.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5a11bcb99922" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>