<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Anshul Goel on Medium]]></title>
        <description><![CDATA[Stories by Anshul Goel on Medium]]></description>
        <link>https://medium.com/@anshulgoel991?source=rss-367efb9a0a42------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*ArfKGWMXXPCZYIOu</url>
            <title>Stories by Anshul Goel on Medium</title>
            <link>https://medium.com/@anshulgoel991?source=rss-367efb9a0a42------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 16:59:22 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@anshulgoel991/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Model Explainability using SHAP (SHapley Additive exPlanations) and LIME (Local Interpretable…]]></title>
            <link>https://medium.com/@anshulgoel991/model-exploitability-using-shap-shapley-additive-explanations-and-lime-local-interpretable-cb4f5594fc1a?source=rss-367efb9a0a42------2</link>
            <guid isPermaLink="false">https://medium.com/p/cb4f5594fc1a</guid>
            <category><![CDATA[shap]]></category>
            <category><![CDATA[lime]]></category>
            <category><![CDATA[explainable-ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[model-explainability]]></category>
            <dc:creator><![CDATA[Anshul Goel]]></dc:creator>
            <pubDate>Wed, 13 Dec 2023 11:01:54 GMT</pubDate>
            <atom:updated>2023-12-14T04:42:29.463Z</atom:updated>
            <content:encoded><![CDATA[<h3>Model Explainability using SHAP (SHapley Additive exPlanations) and LIME (Local Interpretable Model-agnostic Explanations)</h3><p>Model explainability is a critical topic shaping the landscape of modern artificial intelligence. In an era where complex algorithms increasingly influence our decisions, understanding how these models arrive at their conclusions is paramount. Model explainability isn’t just a technical aspect; it’s the bridge between inscrutable black-box algorithms and human comprehension. In this article on Medium, I’ll delve into the profound significance of model explainability, exploring its implications across various sectors and highlighting its pivotal role in fostering trust, accountability, and ethical deployment of AI technologies along with the technical aspects around it (code and interpretation).</p><p>Model explainability plays a crucial role in several critical areas:</p><ol><li><strong>Healthcare:</strong></li></ol><ul><li><strong>Diagnostic Systems:</strong> Explainable AI helps doctors understand why a particular diagnosis or prediction was made, improving trust and aiding in decision-making.</li><li><strong>Drug Discovery:</strong> In pharmaceuticals, understanding how AI models arrive at conclusions about potential drug compounds is vital for validation and optimizing research efforts.</li></ul><p><strong>2. Finance:</strong></p><ul><li><strong>Credit Scoring:</strong> Explainability ensures fairness and transparency in credit decisions, allowing individuals to understand why they were granted or denied credit.</li><li><strong>Risk Assessment:</strong> Banks and financial institutions use AI for risk assessment; explainability helps in comprehending the factors influencing these assessments.</li></ul><p><strong>3. Autonomous Vehicles:</strong></p><ul><li><strong>Safety and Decision-Making:</strong> Understanding the reasoning behind an autonomous vehicle’s decision-making process is crucial for ensuring safety and building public trust in these technologies.</li></ul><p><strong>4. Criminal Justice:</strong></p><ul><li><strong>Recidivism Prediction:</strong> Explainability is crucial in predictive policing and recidivism prediction models to ensure fairness and prevent biases in decisions related to law enforcement and sentencing.</li></ul><p><strong>5. Ethical AI Deployment:</strong></p><ul><li><strong>Bias Mitigation:</strong> Transparent models enable the identification and mitigation of biases, promoting fairness and equity in various AI applications, such as hiring processes and resource allocation.</li></ul><p>In all these critical areas, model explainability isn’t just about understanding how AI works; it’s about ensuring accountability, fairness, and reliability in decision-making processes that significantly impact individuals and society as a whole.</p><p>Before starting of with the technical aspects, it is important to understand that there are <strong>2 ways of model explanation: Global and local.</strong></p><ol><li><strong>Local Explanation:</strong></li></ol><ul><li><strong>Focus:</strong> Local explanations aim to explain the prediction of a model for a specific instance or a small subset of instances.</li><li><strong>Methodology:</strong> Techniques like LIME (Local Interpretable Model-agnostic Explanations) or SHAP (SHapley Additive exPlanations) fall under this category. They work by perturbing or altering the input features slightly and observing how these changes affect the model’s predictions for that particular instance.</li><li><strong>Purpose:</strong> Local explanations are useful for understanding why a model made a specific prediction for an individual data point. For instance, in a medical context, it can help a doctor understand why a certain patient was diagnosed with a particular condition based on their unique set of symptoms.</li></ul><p><strong>2. Global Explanation:</strong></p><ul><li><strong>Focus:</strong> Global explanations aim to provide an overview or an understanding of the model’s behavior across the entire dataset.</li><li><strong>Methodology: </strong>Techniques like feature importance, partial dependence plots, or model-specific visualization methods fall into this category. They analyze the overall behavior of the model by assessing the contribution or impact of different features on the model’s predictions across the entire dataset.</li><li><strong>Purpose:</strong> Global explanations are beneficial for understanding the general trends and insights the model has learned from the entire dataset. For instance, in a financial application, global explanations can reveal which features (income, credit history, etc.) are most influential in determining credit risk across all applicants.</li></ul><p>In this article, I will be discussing about <strong>SHAPley values</strong> which can be used for <strong>both local and global explanations</strong> &amp; <strong>LIME</strong>, which is used <strong>exclusively for local explanations.</strong></p><p><strong>SHAP (SHapley Additive exPlanations)</strong> values are a method used in machine learning for explaining the output of a model by attributing each feature’s contribution to the final prediction. Derived from cooperative game theory, SHAP values provide a structured approach to understand the importance of individual features in a predictive model.</p><p>At its core, SHAP values aim to assign a numerical value to each feature, showcasing how much that particular feature influenced the model’s prediction for a specific data point. By considering all possible combinations of features and their contributions, SHAP values offer a comprehensive understanding of feature importance within the model.</p><p>This method not only aids in deciphering the decision-making process of complex models but also enables a deeper grasp of which features have the most impact on predictions, thus enhancing interpretability and transparency in machine learning algorithms.</p><p>SHAP values can be used for both local and global explanations, it is provided as a library which can be installed through following command.</p><pre>pip install shap<br>import shap</pre><p><strong>SHAP has forceplot, waterfall plot and beeswarm plot</strong><br><strong>Forceplot and waterfall plot</strong> are used to get explanation about particular rows in the data <strong>(local explanation)</strong>, <strong>beeswarm plot</strong> is for entire dataset <strong>(global explanation)</strong></p><ol><li><strong>Forceplot: </strong>For demonstration purposes, we will be using breast_cancer dataset available in sci-kit learn. Below code helps to produce forceplot for this dataset which can be used for local explanations:</li></ol><pre>import pandas as pd<br>from sklearn.datasets import load_breast_cancer<br>from sklearn.model_selection import train_test_split<br>from sklearn.ensemble import RandomForestClassifier<br>import shap<br><br># Load the Breast Cancer dataset<br>cancer = load_breast_cancer()<br>X = pd.DataFrame(cancer.data, columns=cancer.feature_names)<br>y = pd.DataFrame(cancer.target, columns=[&#39;target&#39;])<br><br># Split the dataset into training and testing sets<br>X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)<br><br># Initialize and train RandomForestClassifier<br>rf_classifier = RandomForestClassifier(random_state=42)<br>rf_classifier.fit(X_train, y_train.values.ravel())<br><br># Explain the model&#39;s predictions using SHAP<br>explainer = shap.TreeExplainer(rf_classifier)<br>shap_values = explainer.shap_values(X_test)<br><br># Visualize a single prediction<br>shap.initjs()<br>shap.force_plot(explainer.expected_value[1], shap_values[1][0, :], X_test.iloc[0, :])</pre><p>Above code helps to produce following plot:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/994/1*jAnKxT4kug6s3lUcZU75Iw.png" /><figcaption>Forceplot for a particular observation in breast cancer dataset</figcaption></figure><p><strong>Interpreting above plot:</strong></p><p><strong>1. Feature Contributions:</strong> It illustrates the contributions of each feature to the final prediction. Positive contributions (to the right side of the plot) increase the output value, while negative contributions (to the left side) decrease it.</p><p><strong>2. Feature Impact:</strong> The length of each feature’s arrow indicates the magnitude of its impact on the prediction. Longer arrows suggest higher influence, showcasing which features significantly affect the model’s decision for that instance.</p><p><strong>3. Direction of Influence:</strong> Arrows to the right indicate features that push the model’s output higher, while arrows to the left signify features that lower the prediction. Understanding the direction of influence helps grasp which features drive the model’s decision toward a particular outcome.</p><p><strong>4. Overall Prediction Shift: </strong>The distance from the base value (vertical dashed line) shows how much the model’s output for this instance differs from the average prediction (base value: 0.6321) for the dataset. This demonstrates how the combined effects of features shift the prediction for this specific data point.</p><p><strong>5. Interactions between Features:</strong> If multiple arrows interact or cross each other, it suggests interactions between those features. This indicates that these features might not act independently but could have combined effects on the prediction.</p><p><strong>6. Feature Importance Comparison:</strong> Comparing arrow lengths and directions across different instances or predictions can reveal which features consistently contribute more or less to the model’s decisions.</p><p>By interpreting the force plot, you can gain insights into which features are influential for a specific prediction, the direction and magnitude of their impact, and potential interactions between features. This helps in understanding the reasoning behind the model’s decision for that particular instance.</p><p>In this case, the prediction is 0.97, which is closer to 1 (meaning maligned case), and the various features contributing towards the result are also indicated.</p><p><strong>2. Beeswarm plot:</strong> Now, using the same dataset, we will look at global explanation using Shap’s beeswarm plot.</p><pre>import shap<br>import matplotlib.pyplot as plt<br><br># Assuming &#39;rf_classifier&#39;, &#39;X_test&#39;, and &#39;explainer&#39; are already defined<br><br># Calculate SHAP values for the entire test set<br>shap_values = explainer.shap_values(X_test)<br><br># Create a summary plot<br>shap.summary_plot(shap_values[1], X_test, show=False)<br><br># Show the plot<br>plt.show()<br><br>## Here, 0 means malignanat (diseased) and 1 means benign (healthy)</pre><p>Above code helps to produce following plot:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/616/1*_2U36QhqmeuhTqV5ZbBeyQ.png" /><figcaption>Beeswarm plot for breast cancer dataset</figcaption></figure><p>The beeswarm plot, often used in the context of SHAP values, provides a visual summary of feature importance across multiple instances or predictions. Observing a beeswarm plot can yield several insights:</p><p>1. <strong>Feature Importance Spread:</strong> The spread of points along the x-axis for each feature indicates the range and distribution of SHAP values across different instances. A wider spread signifies varying importance levels of that feature across the dataset.</p><p>2. <strong>Relative Impact:</strong> The horizontal position of the points for a feature showcases the magnitude of its impact on predictions. Features with points shifted to the right (higher SHAP values) indicate more substantial positive contributions, while those shifted to the left (lower SHAP values) represent negative contributions.</p><p>3. <strong>Consistency of Feature Influence:</strong> Consistent positions (clusters or consistent distributions) of points for a feature across different instances indicate its stable impact on predictions. Conversely, scattered or irregular distributions might imply fluctuating importance or varying effects.</p><p>4. <strong>Comparative Importance:</strong> Comparing the distribution and spread of points between different features allows you to assess their relative importance. Features with more spread-out or consistently shifted points might hold higher significance in the model’s predictions.</p><p>5. <strong>Outliers and Anomalies:</strong> Outlying points or clusters diverging from the main distribution for a feature might highlight specific instances where that feature has an unexpected or unique impact on predictions.</p><p>6. <strong>Interactions between Features:</strong> Beeswarm plots can reveal potential interactions or dependencies between features. If the distributions or positions of points change concerning specific combinations of features, it suggests potential interactions that influence predictions.</p><p>7. <strong>Model Consistency:</strong> Consistency in the distributions and positions of points across multiple models or model versions indicates robustness in feature importance assessments.</p><p>By examining the beeswarm plot, you can grasp the distribution, range, and relative importance of features, understand their consistency or variability in influencing predictions across instances, and identify potential interactions or anomalies within the dataset. This aids in gaining a deeper understanding of how features collectively drive the model’s decisions.</p><p>In this case, features are arranged in decreasing order of importance from top to bottom, with worst area being most significant feature.</p><p>The low values of worst area (indicated in blue color) are moving the predictions closer to 1 (Malign) as they have positive contribution and the values to the left are high and contributing negatively towards 0 (diseased).</p><p>Now, we will have a look at <strong>LIME</strong>, which stands for <strong>Local Interpretable Model-agnostic Explanations</strong>, it is a widely used technique for explaining individual predictions of machine learning models, particularly for tabular (structured) data and text data. It provides local interpretability by approximating complex models’ behavior with simpler, interpretable models for specific instances.</p><p><strong>Key Features:</strong></p><p>1. <strong>Local Explanations:</strong> LIME focuses on explaining individual predictions rather than providing global insights into the entire model.</p><p>2. <strong>Model-Agnostic: </strong>It’s not tied to any specific machine learning algorithm. LIME can explain black-box models such as random forests, support vector machines, neural networks, etc., without requiring internal knowledge of the model.</p><p>3. <strong>Simplification with Surrogate Models:</strong> LIME approximates the behavior of the black-box model around a particular instance by creating a local surrogate model (such as a linear model) trained on perturbed samples generated from the instance of interest.</p><p>4. <strong>Feature Importance:</strong> It identifies and highlights the features that contribute the most to a model’s prediction for a specific instance.</p><p>5. <strong>Human-Interpretable Explanations:</strong> LIME provides explanations in a human-readable format, often showing feature contributions or feature ranges that influence the prediction.</p><p><strong>How It Works:</strong></p><p><strong>1. Perturbation:</strong> LIME generates perturbed samples by slightly modifying the original instance (perturbing features) while keeping the prediction target constant.</p><p><strong>2. Model Explanation:</strong> It trains a surrogate model (e.g., linear regression) on the perturbed samples, considering the original model’s predictions as labels.</p><p><strong>3. Local Approximation:</strong> The surrogate model approximates the behavior of the complex model locally around the instance of interest.</p><p><strong>4. Feature Importance:</strong> LIME identifies the most influential features by analyzing the surrogate model’s coefficients or the magnitude of changes in predictions when perturbing individual features.</p><p>Use Cases:</p><ul><li><strong>Model Debugging:</strong> Understanding why a specific instance receives a particular prediction.<br>- <strong>Building Trust:</strong> Providing explanations to end-users or stakeholders to understand model decisions.<br>- <strong>Validating Predictions:</strong> Verifying whether the model relies on sensible features for certain predictions.</li></ul><pre>import matplotlib.pyplot as plt<br>import sklearn.inspection<br># Extract explanation data<br>exp_data = exp.as_list(label=1)  # label=1 for class &#39;1&#39;<br><br># Prepare data for plotting<br>features, values = zip(*exp_data)<br><br># Plot<br>plt.figure(figsize=(8, 6))  # Adjust figure size as needed<br>plt.barh(features, values, color=&#39;skyblue&#39;)<br>plt.xlabel(&#39;Impact on Prediction&#39;)<br>plt.title(&#39;Feature Contributions for Prediction&#39;)<br>plt.tight_layout()  # Adjusts spacing to prevent overlapping<br>plt.show()</pre><p>Above code is used to get LIME plot shown as below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/802/1*jQptnV-fIIBbl1kgnDCePA.png" /></figure><h4>The values separated by &lt;&gt; in the LIME explanation represent the feature ranges or intervals where the feature’s value has an impact on the model’s prediction.</h4><p>For instance, in the example 11.71&gt;mean_radius&lt;=13.30, it signifies that for the feature mean_radius, when its value is greater than 11.71 and less than or equal to 13.30, it has an impact on the model’s prediction for the particular class being explained.</p><p>This notation indicates a range of values for the feature that influences the prediction. In LIME explanations, these intervals or ranges are used to represent the feature’s values that contribute to the model’s output for a specific instance or prediction.</p><p>When visualizing LIME explanations, these intervals or ranges provide insights into how different feature values affect the model’s decision for individual instances or classes.</p><p><strong>Simply Explained:</strong> Values outside these ranges might not have a strong impact on the prediction for that class in that specific instance.</p><p><strong>Conclusion:</strong></p><p>In conclusion, model explainability emerges as the cornerstone of trustworthy AI. Techniques like SHAP values decode complex models, fostering transparency and ethical deployment. This quest for understanding isn’t just about algorithms; it’s about ensuring AI aligns with our values, forging a future of accountable technology.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cb4f5594fc1a" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>