<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Vishesh Gupta on Medium]]></title>
        <description><![CDATA[Stories by Vishesh Gupta on Medium]]></description>
        <link>https://medium.com/@guptavishesh264?source=rss-9837e50de4e8------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*sZXm8hEac7z_pN9I96jNmA.png</url>
            <title>Stories by Vishesh Gupta on Medium</title>
            <link>https://medium.com/@guptavishesh264?source=rss-9837e50de4e8------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 16:57:05 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@guptavishesh264/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Navigating Drift Detection in Machine Learning: A Comprehensive Guide]]></title>
            <link>https://medium.com/@guptavishesh264/navigating-drift-detection-in-machine-learning-a-comprehensive-guide-4a29ef9c630b?source=rss-9837e50de4e8------2</link>
            <guid isPermaLink="false">https://medium.com/p/4a29ef9c630b</guid>
            <category><![CDATA[drift-detection]]></category>
            <dc:creator><![CDATA[Vishesh Gupta]]></dc:creator>
            <pubDate>Thu, 07 Sep 2023 05:58:34 GMT</pubDate>
            <atom:updated>2023-09-07T05:58:34.088Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/981/1*_PQnSyqbDJCFAoBAAsZetw.png" /></figure><blockquote><strong>Introduction</strong>:</blockquote><p>In the fast-evolving landscape of machine learning (ML), ensuring model performance and accuracy in production is a challenging task. ML models are susceptible to drift, which can lead to a decline in their predictive power. In this article, I will delve into the world of drift detection, exploring its types, causes, and tools to help you keep your models on track.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*pK4GlEQmlU9iJjMcNJ5wNg.png" /></figure><blockquote><strong>Understanding Drift:</strong></blockquote><p><strong>What is Drift?</strong></p><p>Drift refers to the phenomenon where the performance of ML models degrades over time due to changes in data or underlying concepts. There are two primary types of drift:</p><ol><li><strong><em>Concept Drift:</em></strong> This occurs when the underlying concept the model is trained on changes. For example, customer preferences might shift, affecting sales predictions.</li><li><strong>Data Drift:</strong> Data Drift happens when the distribution of the input data changes, but the underlying concept remains the same. This could be due to changes in data sources, market dynamics, or human behavior.</li><li><strong>Feature Drift:</strong> When the statistical properties of input features change over time.</li><li><strong>Label Drift:</strong> Shifts in the distribution of target labels.</li><li><strong>Prediction Drift:</strong> Model predictions deviate from their expected patterns.</li></ol><blockquote><strong>Examples of Drift</strong></blockquote><ol><li><strong>Concept Drift: </strong>Imagine you have a spam email classifier trained on a dataset from 2010. Initially, it works well, but as email communication evolves, the language and characteristics of spam emails change dramatically. The model struggles to identify these new patterns, leading to concept drift.</li><li><strong>Data Drift:</strong> Consider a weather forecasting model that relies on historical weather data. If the data source shifts from one weather station to another, the data’s distribution might change slightly due to differences in sensor accuracy or placement. This shift in data distribution without a change in the underlying concept represents data drift.</li><li><strong>Feature Drift:</strong> Think of a predictive maintenance model for machinery. The model is trained on sensor data from a specific machine configuration. Over time, the machinery undergoes maintenance, and new sensors are added, while old sensors are removed. The model’s features change, affecting its performance due to feature drift.</li><li><strong>Label Drift:</strong> Suppose you have a fraud detection model in the financial industry. Initially, the definition of fraud is clear-cut, but regulations and policies change. What was once considered fraud may no longer be, or new forms of fraud emerge. The model’s labels or ground truth data no longer accurately represent the current concept of fraud, leading to label drift.</li><li><strong>Prediction Drift:</strong> In a stock price prediction model, you may observe that it consistently predicts price increases, but suddenly starts predicting price decreases despite no significant change in the underlying market conditions. This shift in predictions without a corresponding shift in the actual concept reflects prediction drift.</li></ol><blockquote><strong>Concept Drift — Four Facets:</strong></blockquote><ul><li><strong>Sudden:</strong> An abrupt and unexpected shift in the concept.</li><li><strong>Gradual:</strong> Concept changes occur slowly over time.</li><li><strong>Incremental: </strong>The concept evolves in incremental steps.</li><li><strong>Recurring:</strong> Concepts change periodically.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/879/1*k9atUhkcBVj3UcIzle3Ozw.png" /></figure><blockquote><strong>Why ML Projects Fail in Production:</strong></blockquote><p>Many ML projects fail to live up to expectations in production due to the following reasons:</p><ul><li>Neglecting ongoing monitoring of model performance.</li><li>Failing to adapt to changing data distributions and feature types.</li><li>Insufficient attention to detecting drift.</li><li>Model deployment is seen as the endpoint rather than the beginning of ongoing measurement and monitoring.</li></ul><blockquote><strong>What to Monitor?</strong></blockquote><p>When it comes to drift detection, it’s essential to monitor the following:</p><ol><li>Basic Summary Statistics: Keep an eye on median, mean, minimum, maximum, and the percentage of missing values in features and targets.</li><li>Feature and Target Distributions: Ensure that the distributions remain consistent over time.</li><li>Model Performance Metrics: Continuously evaluate metrics like MSE, ROC, and F1-score.</li><li>Business Metrics: Assess how model performance affects your business objectives.</li></ol><blockquote><strong>Monitoring Tests on Data:</strong></blockquote><p><strong>For numeric features</strong> — apply statistical tests like Kolmogorov-Smirnov, Mann Whitney</p><p><strong>For categorical features</strong> — apply Chi-Squared tests to identify drift</p><blockquote><strong>Detecting Drift with Tools:</strong></blockquote><h4>Detecting Drift with Deepchecks</h4><p>Deepchecks stands out as a valuable addition to your drift detection toolkit. It is a collection of suites designed explicitly for data drift detection, providing rich insights into your models’ performance over time.</p><p>Deepchecks offers a range of capabilities, including the analysis of statistical properties of your data and models, comparison of feature and target distributions, and evaluation of model performance metrics. Leveraging Deepchecks can help you stay ahead of drift-related issues and make timely adjustments to your machine learning pipelines.</p><p>In addition to Deepchecks, you can consider other popular tools like:</p><ol><li><strong>ML-Flow</strong></li><li><strong>WandB (Weights and Biases)</strong></li><li><strong>Scikit-multiflow (ADWIN)</strong></li><li><strong>River (online ML)</strong></li><li><strong>Evidently.ai</strong></li><li><strong>Custom Functions</strong></li></ol><pre>dir(deepchecks)</pre><pre>[&#39;BaseCheck&#39;,<br> &#39;BaseSuite&#39;,<br> &#39;CheckFailure&#39;,<br> &#39;CheckResult&#39;,<br> &#39;CheckRunContext&#39;,<br> &#39;Condition&#39;,<br> &#39;ConditionCategory&#39;,<br> &#39;ConditionResult&#39;,<br> &#39;Dataset&#39;,<br> &#39;DistributionNotFound&#39;,<br> &#39;ModelComparisonBaseCheck&#39;,<br> &#39;ModelComparisonSuite&#39;,<br> &#39;ModelOnlyBaseCheck&#39;,<br> &#39;SingleDatasetBaseCheck&#39;,<br> &#39;Suite&#39;,<br> &#39;SuiteResult&#39;,<br> &#39;TrainTestBaseCheck&#39;,<br> &#39;__all__&#39;,<br> &#39;__builtins__&#39;,<br> &#39;__cached__&#39;,<br> &#39;__doc__&#39;,<br> &#39;__file__&#39;,<br> &#39;__loader__&#39;,<br> &#39;__name__&#39;,<br> &#39;__package__&#39;,<br> &#39;__path__&#39;,<br> &#39;__spec__&#39;,<br> &#39;__version__&#39;,<br> &#39;base&#39;,<br> &#39;errors&#39;,<br> &#39;get_distribution&#39;,<br> &#39;is_notebook&#39;,<br> &#39;matplotlib&#39;,<br> &#39;pio&#39;,<br> &#39;pio_backends&#39;,<br> &#39;utils&#39;]</pre><h4>Using Suite in DeepChecks</h4><ul><li>Data Drift Detection</li><li>Model Performance /Confidence</li><li>Data Integrity Check</li><li>Label Ambuiguity</li><li>Other checks</li></ul><pre># Installing deepchecks Module<br>pip install deepchecks</pre><pre>import deepchecks<br>from deepchecks.suites import full_suite</pre><pre># Create the Dataset Objects<br>ds_train = deepchecks.Dataset(df=x_train,label=y_train,cat_features=[])<br>ds_test = deepchecks.Dataset(df=x_test,label=y_test,cat_features=[])</pre><pre># Create the suite<br>fsuite = full_suite()</pre><pre>results = fsuite.run(train_dataset=ds_train,test_dataset=ds_test,model=pipe_lr)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*n-eKYTNBWUQmxYPkgFGbZQ.png" /></figure><h4>Feature/Data Drift</h4><pre>from deepchecks.checks import TrainTestFeatureDrift<br>check = TrainTestFeatureDrift()<br>result = check.run(train_dataset=ds_train, test_dataset=ds_test, model=pipe_lr)<br>result</pre><blockquote><strong>Conclusion</strong></blockquote><p>In conclusion, understanding and detecting drift in ML models is vital to maintain their relevance and accuracy in production. Employing the right tools and strategies for monitoring and responding to drift will help ensure your models continue to deliver value over time. Remember, model deployment is just the beginning of a journey that demands ongoing attention and care.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4a29ef9c630b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Unveiling the Magic of MICE: (Multiple Imputation with Chained Equations)]]></title>
            <link>https://medium.com/@guptavishesh264/unveiling-the-magic-of-mice-multiple-imputation-with-chained-equations-37f4727da250?source=rss-9837e50de4e8------2</link>
            <guid isPermaLink="false">https://medium.com/p/37f4727da250</guid>
            <dc:creator><![CDATA[Vishesh Gupta]]></dc:creator>
            <pubDate>Mon, 28 Aug 2023 17:47:49 GMT</pubDate>
            <atom:updated>2023-08-28T17:47:49.921Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-s7A9_dzkGJJ9lTB2SbiVg.jpeg" /></figure><blockquote><strong>Introduction</strong></blockquote><p>In the realm of data science and analysis, missing data can be a thorny issue that taints the accuracy of insights. Multiple Imputation by Chained Equations (MICE) is a robust technique that addresses one specific category of missing data: Missing At Random (MAR). Before delving into the intricacies of MICE, it’s essential to understand the different categories of missing data and how MICE becomes a beacon of hope for handling MAR.</p><blockquote><strong>Categories of Missing Data</strong></blockquote><p>Missing data can be categorized into three main types:</p><ol><li><strong>Missing Completely At Random (MCAR): </strong>In this scenario, the missingness is entirely random, and it occurs independently of both observed and unobserved variables. There’s no underlying pattern or reason for the missingness. MCAR missing data doesn’t introduce bias into your analysis.</li><li><strong>Missing At Random (MAR):</strong> Missingness is categorized as MAR when the probability of a variable being missing depends on other observed variables but not on the unobserved data. In other words, although the data is missing, the reason for its absence is explained by other variables in the dataset. MAR is a more common scenario and can introduce bias if not handled properly.</li><li><strong>Missing Not At Random (MNAR):</strong> This type of missingness occurs when the probability of missingness depends on the unobserved data itself. In MNAR situations, the missing data can introduce significant bias into your analysis, as the missingness is related to the data that’s missing.</li></ol><blockquote><strong>MICE: A Savior for Missing At Random (MAR) Data</strong></blockquote><p>Multiple Imputation by Chained Equations (MICE) is particularly helpful in addressing the MAR category of missing data. Why? Because it capitalizes on relationships between variables to create imputations that reflect the missingness patterns. Let’s explore how MICE tackles the MAR challenge:</p><p>Understanding MICE’s Relevance to MAR</p><p>MICE operates under the assumption that the missing data mechanism is MAR. This means that the reason for the missingness is explained by observed variables, making it predictable. By leveraging these observed variables, MICE creates imputed values that align with the data’s inherent patterns.</p><blockquote><strong>How MICE Handles MAR</strong></blockquote><ol><li><strong>Predictive Models:</strong> MICE uses predictive models (For eg. Linear Regression, Random Forest, Decision Tree, etc..) to estimate missing values. These models are constructed based on observed variables, allowing MICE to impute missing values while considering relationships within the dataset.</li><li><strong>Chained Equations:</strong> The “chained equations” part of MICE refers to the iterative process of imputing values for each variable. As each variable’s missing values are imputed, they’re used as predictors for imputing missing values in other variables. This interwoven approach ensures imputed values are influenced by observed relationships.</li><li><strong>Multiple Imputations:</strong> MICE generates multiple imputed datasets, each reflecting a plausible variation of the missingness pattern. By pooling the results from these imputed datasets, the variability introduced by the imputation process is accounted for in subsequent analyses.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tRd-57fj6c_GgK5R-c6GSA.png" /><figcaption><a href="https://www.scirp.org/journal/paperinformation.aspx?paperid=112455">https://www.scirp.org/journal/paperinformation.aspx?paperid=112455</a></figcaption></figure><p>Stop Imputation when convergence is reached after your acceptable imputation times and the difference between the last two imputed values is zero or very small.</p><blockquote><strong>Conclusion</strong></blockquote><p>Multiple Imputation by Chained Equations (MICE) shines brightest in the realm of Missing At Random (MAR) data. While MCAR and MNAR situations present their own challenges, MICE’s power lies in its ability to harness observed relationships to impute missing values, ensuring that the imputations are consistent with the dataset’s underlying patterns. By understanding the categories of missing data and embracing techniques like MICE, data analysts and researchers can confidently handle missing data scenarios, unlocking accurate insights that might have otherwise remained obscured.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=37f4727da250" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>