<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Parth Sharma on Medium]]></title>
        <description><![CDATA[Stories by Parth Sharma on Medium]]></description>
        <link>https://medium.com/@sharmaparth17?source=rss-d456b2a4a420------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*5HiE2vb-MCWD8aFb5ny83w.png</url>
            <title>Stories by Parth Sharma on Medium</title>
            <link>https://medium.com/@sharmaparth17?source=rss-d456b2a4a420------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 07:52:21 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@sharmaparth17/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Understanding the Bias-Variance Tradeoff in Machine Learning]]></title>
            <link>https://medium.com/@sharmaparth17/understanding-the-bias-variance-tradeoff-in-machine-learning-7c58d3bcfdb5?source=rss-d456b2a4a420------2</link>
            <guid isPermaLink="false">https://medium.com/p/7c58d3bcfdb5</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Parth Sharma]]></dc:creator>
            <pubDate>Wed, 11 Oct 2023 07:34:18 GMT</pubDate>
            <atom:updated>2023-10-11T07:34:18.612Z</atom:updated>
            <content:encoded><![CDATA[<p>In the realm of machine learning, the Bias-Variance Tradeoff is a cornerstone concept. It provides crucial insights into the challenges associated with building predictive models, especially concerning overfitting and underfitting. But what exactly are bias and variance? And why is there a “tradeoff”?</p><h3>1. Bias and Variance: An Introduction</h3><p>a. Bias:</p><ul><li>Definition: Bias measures how far off in general these models’ predictions are from the correct value.</li><li>High Bias: If a model has high bias, this means it doesn’t do a good job of bending to the data. It’s likely missing relevant relations between features and target outputs, thereby oversimplifying the model. This leads to underfitting the data.</li></ul><p>b. Variance:</p><ul><li>Definition: Variance measures how much the predictions for a given point vary between different realizations of the model.</li><li>High Variance: A model with high variance pays a lot of attention to the training data and does not generalize well to new data that it hasn’t seen before. This means it’s modeling the random noise in the training data, leading to overfitting.</li></ul><h3>2. The Tradeoff Visualized</h3><p>Imagine shooting arrows at a target.</p><ul><li>High Bias: All arrows land far from the bullseye, but closely together.</li><li>High Variance: Arrows scatter all over the place.</li><li>Balanced Bias and Variance: Arrows land around the bullseye, closely packed.</li></ul><p>In machine learning terms, if we were to draw a similar analogy:</p><ul><li>High Bias: Predictions consistently wrong but close to each other.</li><li>High Variance: Predictions scattered and inconsistent.</li><li>Balanced: Predictions are consistently close to the truth.</li></ul><h3>3. Striking a Balance</h3><p>As we adjust our model to reduce bias, variance tends to go up, and vice-versa. Here’s why:</p><ul><li>Simplifying the Model: If we reduce the complexity of our model, we’re increasing its bias while reducing its variance.</li><li>Complexifying the Model: If we make our model more intricate, perhaps by adding more parameters, we decrease its bias at the cost of increasing its variance.</li></ul><p>Thus, the tradeoff. Striking the right balance is crucial for creating models that generalize well to unseen data. It’s a balancing act between the error introduced by the bias and the error introduced by the variance.</p><h3>4. How to Address the Tradeoff</h3><ul><li>Collect More Data: Helps algorithms detect the signal better.</li><li>Feature Engineering: Create more meaningful variables or remove irrelevant ones.</li><li>Model Complexity: Choose a simpler or more complex model depending on the problem.</li><li>Regularization: Techniques like L1 (Lasso) and L2 (Ridge) regularization can help in controlling model complexity.</li><li>Ensemble Methods: Bagging and boosting can reduce variance and bias, respectively.</li></ul><h3>5. Concluding Thoughts</h3><p>Understanding the Bias-Variance tradeoff is crucial when developing machine learning models. Remember, a model’s complexity can influence how well it performs, and the key is finding the sweet spot — not too simple, not too complex, just right.</p><p>By being aware of the pitfalls of high bias (underfitting) and high variance (overfitting), and using the strategies mentioned above, you can steer your model toward that balanced middle ground.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7c58d3bcfdb5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding Model Evaluation Metrics: A Comprehensive Guide]]></title>
            <link>https://medium.com/@sharmaparth17/understanding-model-evaluation-metrics-a-comprehensive-guide-a4ac6746463f?source=rss-d456b2a4a420------2</link>
            <guid isPermaLink="false">https://medium.com/p/a4ac6746463f</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[mathematics]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Parth Sharma]]></dc:creator>
            <pubDate>Wed, 11 Oct 2023 07:23:25 GMT</pubDate>
            <atom:updated>2023-10-11T07:23:25.121Z</atom:updated>
            <content:encoded><![CDATA[<p>Machine learning models are as good as their evaluation. Properly evaluating a model not only helps in improving its accuracy and reliability but also in understanding its behavior and characteristics. In this blog, we’ll dive deep into various metrics used for model evaluation.</p><h3>1. Classification Metrics:</h3><h4>a. Accuracy:</h4><ul><li><strong>Definition</strong>: It is the ratio of correctly predicted observation to the total observations. One of the most intuitive metrics used in classification.</li><li><strong>Mathematical Formula:</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/319/1*p5llZ9c8BbeRdOmb5P_x4Q.png" /></figure><ul><li><strong>Insight:</strong> While accuracy gives a general idea of model performance, it can be misleading in imbalanced datasets where the majority class dominates.</li><li><strong>Importance:</strong> It’s the most straightforward metric giving a quick snapshot of overall performance. However, in imbalanced datasets, a high accuracy can be misleading since the model might be just predicting the majority class. Thus, while accuracy is useful, it should be considered along with other metrics for a holistic view.</li></ul><pre>from sklearn.metrics import accuracy_score<br><br>y_true = [1, 0, 1, 1, 0, 1]<br>y_pred = [0, 0, 1, 1, 0, 1]<br><br>accuracy = accuracy_score(y_true, y_pred)<br>print(f&quot;Accuracy: {accuracy}&quot;)</pre><h4>b. Confusion Matrix:</h4><p>A confusion matrix is a table that is used to evaluate the performance of a classification model. The matrix provides insights into four categories:</p><ul><li>True Positive (TP): Actual and predicted both are positive.</li><li>True Negative (TN): Actual and predicted both are negative.</li><li>False Positive (FP): Actual is negative but predicted as positive. Type I error.</li><li>False Negative (FN): Actual is positive but predicted as negative. Type II error.</li></ul><pre>from sklearn.metrics import confusion_matrix<br><br>cm = confusion_matrix(y_true, y_pred)<br>print(cm)</pre><h4>c. Precision, Recall, and F1-Score:</h4><ul><li>Precision: It’s the ratio of correctly predicted positive observations to the total predicted positives. High precision relates to the low false-positive rate.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/319/1*Ey_BLRvAD7KbnoGBIH4qNw.png" /></figure><ul><li>Recall (Sensitivity): It’s the ratio of correctly predicted positive observations to all the actual positives. It quantifies the model’s ability to find all relevant instances in the dataset. ​</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/319/1*TBs52mmvIaIXl6vszjWr1Q.png" /></figure><ul><li>F1-Score: It’s the harmonic mean of precision and recall and gives a balanced measure. It’s particularly useful when class distribution is imbalanced. ​</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/319/1*_pNyjmI0i7tibWhPfkuaqQ.png" /></figure><pre>from sklearn.metrics import precision_score, recall_score, f1_score<br><br>precision = precision_score(y_true, y_pred)<br>recall = recall_score(y_true, y_pred)<br>f1 = f1_score(y_true, y_pred)<br><br>print(f&quot;Precision: {precision}\nRecall: {recall}\nF1 Score: {f1}&quot;)</pre><h4>d. ROC and AUC:</h4><ul><li>ROC (Receiver Operating Characteristic): It’s a graphical plot that illustrates the diagnostic ability of a binary classifier system as its discrimination threshold is varied. The curve is a plot of the True Positive Rate (Recall) against the False Positive Rate.</li><li>AUC (Area Under the Curve): It represents the model’s ability to distinguish between the positive and negative classes. An AUC of 1 indicates perfect classification, while an AUC of 0.5 suggests the model is no better than random guessing.</li></ul><pre>from sklearn.metrics import roc_curve, auc<br><br>fpr, tpr, thresholds = roc_curve(y_true, y_pred)<br>roc_auc = auc(fpr, tpr)<br><br>print(f&quot;AUC: {roc_auc}&quot;)</pre><h3>2. Regression Metrics:</h3><h4>a. Mean Absolute Error (MAE):</h4><ul><li>Definition: It is the average of the absolute differences between predicted and actual values. It gives a linear penalty to each error.</li><li>Mathematical Formula:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/230/1*C7LVKPsvqBE1uc8-VzYCeQ.png" /></figure><ul><li>Insight: It provides an idea of the magnitude of the error but doesn’t provide the direction (over or under-predicting).</li><li>Importance: Gives a linear penalty for each error, making it interpretable. It provides a clear idea of how far off the predictions are on average.</li></ul><pre>from sklearn.metrics import mean_absolute_error<br><br>y_true_reg = [3.0, -0.5, 2.0, 7.0]<br>y_pred_reg = [2.5, 0.0, 2.0, 8.0]<br><br>mae = mean_absolute_error(y_true_reg, y_pred_reg)<br>print(f&quot;MAE: {mae}&quot;)</pre><h4>b. Mean Squared Error (MSE):</h4><ul><li>Definition: It’s the average of the squared differences between predicted and actual values. Unlike MAE, MSE gives a quadratic penalty to errors.</li><li>Mathematical Formula:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/230/1*zzHpUGyaV_Gr1Dg1160T9g.png" /></figure><ul><li>Insight: Larger errors are noted more than with MAE, making MSE sensitive to outliers.</li><li>Importance: Squaring the errors amplifies the impact of large errors. Hence, MSE is more sensitive to outliers compared to MAE.</li></ul><pre>from sklearn.metrics import mean_squared_error<br><br>mse = mean_squared_error(y_true_reg, y_pred_reg)<br>print(f&quot;MSE: {mse}&quot;)</pre><h4>c. Root Mean Squared Error (RMSE):</h4><ul><li>Definition: It is the square root of the mean of the squared differences between predicted and actual values. Provides magnitude of error in the same units as the target.</li><li>Mathematical Formula:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/230/1*kHlpsLbGdFaGV5bwBSL5BQ.png" /></figure><ul><li>Insight: Like MSE, RMSE gives a higher penalty to larger errors.</li><li>Importance: RMSE gives a higher weight to larger errors. It’s more sensitive than MAE and provides a consolidated measure of the magnitude of error.</li></ul><pre>import numpy as np<br><br>rmse = np.sqrt(mse)<br>print(f&quot;RMSE: {rmse}&quot;)</pre><h4>d. R² (Coefficient of Determination):</h4><ul><li>Definition: It quantifies the amount of variance in the target variable that is predicted from the feature variables.</li><li>Mathematical Formula:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/230/1*-ZmeUGCyT7YGqMLsV4Qnjw.png" /></figure><ul><li>Insight: R² score close to 1 means the model explains a large portion of the variance in the target variable, while a score close to 0 means the model explains little of the variance.</li><li>Importance: R² is a measure of how well the model’s predictions match the actual data. An R² score close to 1 indicates a good fit, while a score close to 0 indicates the opposite. It provides insights into the goodness of fit of the model.</li></ul><pre>from sklearn.metrics import r2_score<br><br>r2 = r2_score(y_true_reg, y_pred_reg)<br>print(f&quot;R^2 Score: {r2}&quot;)</pre><p>In conclusion, understanding these metrics and their significance is essential for model evaluation. The right metric often aligns with specific project goals and the nature of the data.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a4ac6746463f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Essence of Data Science]]></title>
            <link>https://medium.com/@sharmaparth17/essence-of-data-science-e117d7920aa6?source=rss-d456b2a4a420------2</link>
            <guid isPermaLink="false">https://medium.com/p/e117d7920aa6</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Parth Sharma]]></dc:creator>
            <pubDate>Tue, 23 Jan 2018 00:14:09 GMT</pubDate>
            <atom:updated>2018-01-23T00:16:02.223Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/569/1*SkESnxcwWLmPaF4Fp87oyw.jpeg" /></figure><p>Data Science is the hottest field of 21st century and most of the students and techies want to be part of it. Most people think of it as applying cool machine learning algorithms and just fetch out the results (predictions). But the ground truth is not that!!</p><p>Data Science is all about data and business objective. Thumb rule says spend 80 percent of time to data and 20 percent of it to algorithms. But newbie to this field thinks algorithms are everything.</p><p>Let’s take an example, You are the owner of a restaurant chain and want to boost sales and lower down it’s operating cost. Which thing will you consider first, pattern in the data on which the sales and operating costs are dependent or be like “I have the data, just run some hit and trial of algorithms, algorithm will do everything for me”. It’s not like that you have to find patterns from data and then decide from the pattern which type of algorithm will be best fit for this business problem. Even lowering down 5 percent of the operating cost will make you profit. Think it as a business problem or what’s the objective behind giving me the data not running towards getting best result!!</p><p>Data Science is art of relating business problem with data and getting the best but optimal result with the aid of algorithms which will indeed make business profitable.</p><p>First understand the data available using statistical programming languages(commonly R and Python) to get patterns or on which features of data, objective is dependent. Then implement the machine learning algorithms over it. Even with machine learning algorithms one should able to answer “Why you particularly using this algorithm?”, “Is this the optimal solution using this algorithm?”(Think it in terms of complexity).</p><p>Hope this helps newbies to understand the domain space. Best of luck with your data journey!!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e117d7920aa6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Color detection in an intelligent way(Machine Learning)]]></title>
            <link>https://medium.com/@sharmaparth17/color-detection-in-an-intelligent-way-machine-learning-58b17428eef1?source=rss-d456b2a4a420------2</link>
            <guid isPermaLink="false">https://medium.com/p/58b17428eef1</guid>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[image]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[computer-science]]></category>
            <dc:creator><![CDATA[Parth Sharma]]></dc:creator>
            <pubDate>Sat, 23 Sep 2017 16:14:54 GMT</pubDate>
            <atom:updated>2017-09-23T16:14:54.409Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/644/1*BWVyHW0OsqEzNqSh_QptXA.png" /></figure><p>I first came across this non trivial problem while working on a project of predicting the colors from fashion catalogue images. Firstly, I searched for existing solution but they were having performance issues such as time to run the whole process was in minutes which can’t be used for commercial purpose. After facing these issues I started working to build my own color detection technique which was succesful and process time was reduced to 2–4 seconds.</p><p>I used the following techniques in accomplish this task:</p><blockquote>Adaptive skin detection</blockquote><blockquote>Image masking and background removal</blockquote><blockquote>Region of interest selection</blockquote><blockquote>Machine Learning</blockquote><p>Firstly I resized the image to 100x100.</p><p>Using this resized image as input the areas having skin were detected and were replaced with black color(i.e masked). So, skin color won’t affects predictions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/963/1*YkO3kun5oZpqex2yPiH2CQ.jpeg" /><figcaption><strong>Original Image vs removed skin color image</strong></figcaption></figure><p>Further image background was masked by separating image into background and foreground. The remaining region is marked as region of interest.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/962/1*91c_DUO9G5bWWuejj8e3ng.jpeg" /><figcaption><strong>After background removal</strong></figcaption></figure><p>The preprocessed image is then parsed into unsupervised machine learning algorithm, here I used <a href="https://en.wikipedia.org/wiki/K-means_clustering">KMeans clustering algorithm</a> to achieve this goal. Below is the image of the clusters being created with their centers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8ggjrK-46Rd9oTe43KCbEg.png" /><figcaption>Clusters formed with their centers</figcaption></figure><h3><strong>Results and performance:</strong></h3><blockquote><strong>Result :</strong> Base color: red, Shade color: sangria</blockquote><blockquote><strong>Performance : </strong>The overall process take approximately 2–4 secs.</blockquote><p>You can find the python codes from the same at <a href="https://github.com/parth1993/pycolor_detection">https://github.com/parth1993/pycolor_detection</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=58b17428eef1" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>