<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Saiyam Bhatnagar on Medium]]></title>
        <description><![CDATA[Stories by Saiyam Bhatnagar on Medium]]></description>
        <link>https://medium.com/@saiyambhtngr?source=rss-2791e41d0113------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*74KCZKqEgLnsCpDs49E3yg.jpeg</url>
            <title>Stories by Saiyam Bhatnagar on Medium</title>
            <link>https://medium.com/@saiyambhtngr?source=rss-2791e41d0113------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 02:47:39 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@saiyambhtngr/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[One2OneHub: The vision]]></title>
            <link>https://medium.com/@saiyambhtngr/one2onehub-the-vision-be3c8804f01f?source=rss-2791e41d0113------2</link>
            <guid isPermaLink="false">https://medium.com/p/be3c8804f01f</guid>
            <dc:creator><![CDATA[Saiyam Bhatnagar]]></dc:creator>
            <pubDate>Mon, 03 Feb 2025 14:28:48 GMT</pubDate>
            <atom:updated>2025-02-03T14:28:48.195Z</atom:updated>
            <content:encoded><![CDATA[<p>Hey fellas, hope you are all doing great. It has been a lot of time since I have penned down something and this time it won’t be a concept or a learning which popped out of a project or a side hustle. Since the past few years when I gave up on writing as a hobby, I picked up powerlifting, athletics and a few sports which I really enjoyed as a kid. <br>I haven’t started inking the idea of One2OneHub yet. I am just beating about the bush. Wonder why? Don’t be apprehensive about me wasting your time, I’ll get to One2OneHub in a while.<br>However, lets first hear about my experience as a powerlifter. The analogy of a muscle is of utmost mystery, precision and consistence. You need to train it; not overtrain or undertrain it. Well that is not all, you need to push it gradationally — progressive overloading (you know if you know). So, that all! That is how a powerlifter stays on the acme of his game. This brings me to my vision — One2OneHub.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/981/1*Mbof3YGoS4y9BKuScmcxbw.png" /></figure><p>One2OneHub is a personal coach that brings out the best in you in the shortest time and in the best way possible. It understands you objectively, psychologically and coaches you systematically. So briefly, the personal coach brings out the steepest learning curve possible. We, as in my cofounder and me have embarked on this journey to make learning as efficient as possible. We plan to support each student with a buddy who understands him/her in and out.</p><p>Our buddy would be able to —</p><ol><li>Understand the learner’s goal and carve a learning trajectory out of it.</li><li>Curate a short assignment in the beginning to assess learner’s skill.</li><li>Modularize the course structure accordingly</li><li>Tweak the course and learning trajectory as per the learner’s progress and ability.</li></ol><p>In entirety, out buddy ensure that each learner irrespective of the skill, shines out bright in the end. The end? Well although its contingent on the learner; however, our buddy makes sure the end arrives in the shortest time possible. Why? — Remember the muscle’s analogy — the right throttle, at the right time and there is not going back!<br>Thank you,<br><a href="https://www.linkedin.com/in/saiyam-bhatnagar-b6a84015a/">Saiyam Bhatnagar<br></a><a href="https://www.linkedin.com/in/deven-bansal/">Deven Bansal</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=be3c8804f01f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[FIFA Stars — a dive into 3 biggies.]]></title>
            <link>https://medium.com/analytics-vidhya/fifa-stars-a-dive-into-3-biggies-979637072602?source=rss-2791e41d0113------2</link>
            <guid isPermaLink="false">https://medium.com/p/979637072602</guid>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Saiyam Bhatnagar]]></dc:creator>
            <pubDate>Thu, 30 Jul 2020 09:51:17 GMT</pubDate>
            <atom:updated>2020-07-30T13:20:21.399Z</atom:updated>
            <content:encoded><![CDATA[<h3>FIFA Stars — a dive into 3 biggies.</h3><p>Hello readers! It was a few months ago when I picked up something which gave me a tough time to prepare for my mid-semester exams. As sophomores at an Engineering College — BITS Pilani, we were accustomed to begin preparing only a week before the examinations. The story begins with how in my hostel dorm, my <strong>Wingies</strong> got me addicted to FIFA-19, a mere 10 days before we ought to take our Mid-semester exams. However, despite diving into the apprehensive reflexes of the past lets discuss what we are meant to. Today, while browsing through the Kaggle data-sets I got through something interesting. So first lets talk about the data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PzWdm-yYmDPKrdED6lf59w.jpeg" /></figure><p>The datasets consists of 18207 players and 651 clubs. These players are represented by 40 features, some of which are —’Name’, ‘Age’,’Club’,’Crossing’, ‘Finishing’, ‘Heading Accuracy’, ‘Short Passing’, ‘Volleys’, ‘Dribbling’,‘Curve’, ‘F K- Accuracy’, ‘Long Passing’ etc. A couple of minutes after playing with data(though not with the same interest I play FIFA :-p), I found that such a large data-set made the features approximately Normal Distributed. Though there was some <strong>skewness</strong> present. Therefore, I decided to check how many outliers my 3 favorite teams had. (Barca, Juventus and Man City). These were a couple of steps I took to arrive at a few results.</p><ol><li>Feature Engineering</li></ol><p>I decided to calculate 2 new values for each of 18207 players- Attacking and Defense. The features were engineered using appropriate class weights. Attacking — {Finishing,Crossing, Heading Accuracy…etc}, Defense — {Long Passing, Marking, Tackle…etc}.</p><p>2. Plotting and Visualizing</p><p>I could speculate that such a large data-set could easily form a slightly Skewed <strong>Normal Distribution</strong>. I plotted the histogram using various values for the <strong><em>no. of bins</em></strong><em>. </em>I took the seemingly best ones and got two Histograms for Attacking and Defense.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/456/1*-diaPNw2FhZCTb61imLkuA.png" /><figcaption>Attacking score Histogram</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/447/1*5PF6CYjF9eIjF7DrMX8_gg.png" /><figcaption>Slightly skewed Defense Score Histogram</figcaption></figure><p>3. Inferring from the Distributions</p><p>Setting the <strong>C.I</strong> at 90% and 95%. Apology for those who are not statisticians yet great football fans and want to see the name of their stars in the list. For your reference, what I mean by the CI at 90% and 95% is that I’ll be bringing out the names of top 5% and 2.5% players. These were the mean scores and the respective scores of top attackers and defenders.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/604/1*1F0sQedczkE-wZQqjWHshw.png" /><figcaption>The scores out of 100</figcaption></figure><p>4. Results</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*D2f1EcmlMGJ6Ea9qWsFfOA.png" /></figure><p>The results then inspired me to check the<strong> Release Clauses</strong> of the Clubs as a whole. No wonder why so many outliers lie in these clubs. Check the per player Release Clause of the these biggies.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/687/1*B6Wi5c4G225R1XfJq7qCtw.png" /><figcaption>The values is in Millions of Euros</figcaption></figure><p>Although the inferences required a bit of data cleaning and pre-processing. Readers interested to go through the code can find it in my <a href="https://github.com/Saiyam1999/Statistical_Insights/blob/master/FIFA_data_analysis/FIFA.ipynb"><strong><em>github </em></strong></a><strong><em>code file</em></strong>. Have a Good day. :-)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=979637072602" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/fifa-stars-a-dive-into-3-biggies-979637072602">FIFA Stars — a dive into 3 biggies.</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Q-Q Plots: Guiding you to better results]]></title>
            <link>https://medium.com/analytics-vidhya/q-q-plots-guiding-you-to-better-results-b0db247d5872?source=rss-2791e41d0113------2</link>
            <guid isPermaLink="false">https://medium.com/p/b0db247d5872</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[data-visualization]]></category>
            <dc:creator><![CDATA[Saiyam Bhatnagar]]></dc:creator>
            <pubDate>Thu, 23 Jul 2020 06:39:14 GMT</pubDate>
            <atom:updated>2020-07-23T11:59:15.441Z</atom:updated>
            <content:encoded><![CDATA[<p>Hello readers. in one of my previous blogs I endeavored to demonstrate how to effectively model a Regression Problem. That post was large in large, concerned about how many features should actually be a part of you model. However, this post is concerned with <strong>improving a feature </strong>using <strong>Feature Engineering</strong> to enable it fit the <strong>Regression Assumptions</strong> well.</p><p>Among several assumptions on which a Linear or Logistic Regression is based — No Multicollinearity, No Heteroscedasticity etc , one is the <strong>Normality</strong> of Independent variables. The more Normal the distribution is, better are the results of the Regression model. The readers who aren’t well aware of the reason behind these assumptions are required to go through it once. :-) To practically illustrate my concept of visualizing through Q-Q plots, I downloaded the Kaggle- DataAnalyst dataset and applied few transformations to it to make as normal as possible. However, before diving into this lets see what a Q-Q plot actually demonstrates.</p><p>What is a Q-Q plot?</p><p><em>The Q-Q plot, or quantile-quantile plot, is a </em><strong><em>graphical tool </em></strong><em>to help us assess if a set of data plausibly came from some theoretical distribution such as a Normal or exponential. For example, if we run a statistical analysis that assumes our dependent variable is Normally distributed, we can use a Normal Q-Q plot to check that assumption. It’s just a </em><strong><em>visual check</em></strong><em>, not an </em><strong><em>air-tight</em></strong><em> proof, so it is somewhat subjective. But it allows us to see at-a-glance if our assumption is plausible, and if not, how the assumption is violated and what data points contribute to the violation. A Q-Q plot is a </em><strong><em>scatterplot</em></strong><em> created by plotting two sets of </em><strong><em>quantiles</em></strong><em> against one another. If both sets of </em><strong><em>quantiles</em></strong><em> came from the same distribution, we should see the points forming a line that’s roughly straight.</em></p><p>This largely means- greater the number of points on a line, greater is the extent to which the data sample follow the mentioned distribution — example Gaussian, Normal etc. Now, I’ll apply few transformations and plot the adjoining <strong>Histogram </strong>and a <strong>Q-Q</strong> plot for the readers to decide which transformation would best suit for the <strong>Feature Engineering</strong>. Please keep in mind this is not a conceptual analysis. This is just a methodology demonstration. The feature chosen is the rating of Data Analysts on a scale of 1–5.</p><ol><li>No Feature Engineering</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/517/1*sPuy0qVQpwqT5L5dGhylEQ.png" /><figcaption>Histogram and the corresponding Q-Q plot respective to Normal Distribution.</figcaption></figure><p>2. The feature is transformed using Box-Cox</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/511/1*R3S0FUG-1SVweHW7k6PMVA.png" /><figcaption>Histogram and the corresponding Q-Q plot respective to Normal Distribution.</figcaption></figure><p>3. The feature is transformed using Logarithmic transformation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/529/1*lLtFNbNf7NWnFpqio2yDpA.png" /><figcaption>Histogram and the corresponding Q-Q plot respective to Normal Distribution.</figcaption></figure><p>There is not much difference in the plots as I was unable to choose the appropriate feature which demonstrates this decently. Guessing the deviation of a Real World data-set from a distribution is too hard to arrive at. However, I do recommend the readers to try this illustration before using a Regression Model. Have a good day. Thank you. :-)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b0db247d5872" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/q-q-plots-guiding-you-to-better-results-b0db247d5872">Q-Q Plots: Guiding you to better results</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding the K-Means Algorithm better]]></title>
            <link>https://medium.com/analytics-vidhya/understanding-the-k-means-algorithm-better-da51d45bbadf?source=rss-2791e41d0113------2</link>
            <guid isPermaLink="false">https://medium.com/p/da51d45bbadf</guid>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Saiyam Bhatnagar]]></dc:creator>
            <pubDate>Tue, 21 Jul 2020 05:32:53 GMT</pubDate>
            <atom:updated>2020-07-21T13:43:18.294Z</atom:updated>
            <content:encoded><![CDATA[<p>Hello readers. K-means Clustering is undoubtedly one of the most popular unsupervised learning algorithm. The reason behind it being used so frequently is the strong yet simple statistical backbone. This story would first explain the logical approach behind K-Means clustering, then it would bring forth a practical drawback and a few suggestions to avoid it.</p><p>K-Means is a non-deterministic algorithm. This means that a compiler cannot solve the problem in polynomial time and doesn’t clearly know the next step. This is because some problems have a great degree of randomness to them. These algorithms usually have 2 steps — 1)Guessing step 2)Assignment step. On similar lines is the K-means algorithm. The K-Means algorithm divides the data space into K clusters such that the total variance of all data points with respect to the cluster mean is minimized.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/601/1*8YFeSeUBvA_yhbhB9oGw_w.png" /><figcaption>Fig (1) Function to be minimized. S(i) are clusters.</figcaption></figure><p>However, the approach that compiler takes does not involve Multivariate Calculus as it seems. Rather, the approach taken is iterative. Now, like any deterministic algorithm it has 2 phases. Guessing phase: Randomly initializing k means in the data space(Mu(k)s). Now, all the data points X(i)s (1,m) are assigned to clusters in accordance to which cluster mean they are closer to. Mathematically, this step tries to minimize the within cluster variance. Hence, every point is now assigned a cluster. Next is the assignment step. All the cluster means (Mu(k)s)are now assigned to the mean of the data points in the cluster. This step is repeated a couple of times. Refer to the image below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/439/1*7DR5ja2uWMrvpSB74eVh-A.png" /><figcaption>Fig(2) The K Means Algorithm</figcaption></figure><p>Now similar to the most of non-deterministic algorithms, K-Means has a bad habit. Which is that every time you run a K-Means clustering it would give you different results. The situation gets even worsened when you are unsure if the any modification to the K-Means would improve the results. Refer to the image below to see how bad sometimes can a K-Means Algorithm’s result get.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/693/1*4TZds393yR8Ovq5XSDr6-Q.png" /><figcaption>Fig(3) two different results of K-Means when K=3</figcaption></figure><p>We should understand that we cannot do much about this issue as it is almost impossible to analytically imagine a data-space so huge. Although, we do have a couple of suggestions to follow.</p><ol><li>How to choose the value of K?</li></ol><p>One should rely on the problem statement for this. For example in a tree specie classification problem, if one know the number of possible specie and given that all of them appear in significant numbers in the data-set, one can assign the number of species to K.</p><p>2. How to be sure if the solution obtained is appropriate?</p><p>There is no versatile approach for this issue. Rather one should focus on initializing the cluster means with the best possible estimate. There could be any statistical approach for this. For example — One could assign the calculated mean of a few data points which one is sure would fall in the same cluster. Secondly, one can iterate 10–100 times a K-Means Algorithm and decide the one which best minimizes the Mathematical function given above (Fig 1).</p><p>These were a few insights and suggestions into the K-Means clustering.</p><p>Thank You. :-)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=da51d45bbadf" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/understanding-the-k-means-algorithm-better-da51d45bbadf">Understanding the K-Means Algorithm better</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Grabcut algorithm —An  OpenCV feature that might improve image handling model’s performance greatly.]]></title>
            <link>https://medium.com/analytics-vidhya/grabcut-algorithm-an-opencv-feature-that-might-improve-image-handling-models-performance-greatly-cfc39de90d58?source=rss-2791e41d0113------2</link>
            <guid isPermaLink="false">https://medium.com/p/cfc39de90d58</guid>
            <category><![CDATA[image-classification]]></category>
            <category><![CDATA[image-processing]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Saiyam Bhatnagar]]></dc:creator>
            <pubDate>Tue, 30 Jun 2020 04:53:50 GMT</pubDate>
            <atom:updated>2020-06-30T13:35:40.522Z</atom:updated>
            <content:encoded><![CDATA[<h3>Grabcut algorithm —An OpenCV feature that might improve image handling model’s performance greatly.</h3><p>The objective behind this story is not to Mathematically illustrate the algorithm behind the Grabcut but is to demonstrate how greatly the feature can improve a model’s performance. Please keep in mind that the story does not, in any sense, infer that the feature works well on all Image handling models. Recently, while participating in a Computer Vision Hackathon that consisted of 5 different problem statements, I decided to read the famous OpenCV’s documentation. In my attempt to see how I could use different tools for data pre-processing, I initially landed upon the Grabcut. I’ll discuss my model performance with and without the Grabcut.</p><p>The problem statement was to classify images of leaves into healthy and non healthy category. Although I intended to use a larger and a better model initially but the five hour time constraint backed up my inability to do so. The data-set consisted of pictures of leaves with a few background details. However, the ROI (region of interest) was primarily the surface of the leaf.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/316/1*QJoHAdG8uv81rWlTST13Gg.png" /><figcaption>Sample image from data-set</figcaption></figure><p>The reason that the background was of no need to infer results, I decided to get rid of it without explicitly training a model to do so. Therefore, I decided to use Grabcut. As mentioned earlier the story won’t get into the intricacies of the algorithm, I won’t be discussing the functionality except the key points present. The <strong><em>rect</em></strong> variable in the code stores the dimensions of the RIO in the image. So, this should be chosen wisely.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/792/1*ZF21LyUq3jVhKOG7l_s3GA.png" /><figcaption>the function for implementing Grabcut</figcaption></figure><p>After using the algorithm, I could easily obtain images with no background. Look below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/321/1*IRy02tUGsCA6n2NV8bi0gw.png" /><figcaption>An image post Grabcut</figcaption></figure><p>For images in which the algorithm removed the complete leaf, I selected those using <em>image thresholding</em> and created a simple black border around them. See the figure below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/324/1*_CGCZJb9-uFVJcBM7-k3DQ.png" /><figcaption>image on which Grabcut didn’t work correctly</figcaption></figure><p>As a result I got highly improved model accuracy. The former model saturated at 95% accuracy while the latter achieved more than 99% in lesser epochs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/512/1*SxSMmFNaOzuHrTgkMMv6MQ.png" /><figcaption>the former model</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/507/1*wkohu6fU-Zyfz5h0-6Ejcw.png" /><figcaption>the latter model</figcaption></figure><p>This was how I built an image classification model within time constraints. However, please do not forget to augment the test data in similar way :-) . Special thanks to Google’s free TPU to help me rank in the top ten in the Hackathon.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cfc39de90d58" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/grabcut-algorithm-an-opencv-feature-that-might-improve-image-handling-models-performance-greatly-cfc39de90d58">Grabcut algorithm —An  OpenCV feature that might improve image handling model’s performance greatly.</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How Facebook AI used this Network to generate more data to train a Pose-Estimation Model.]]></title>
            <link>https://medium.com/analytics-vidhya/how-facebook-ai-used-this-network-to-generate-more-data-to-train-a-pose-estimation-model-339a45d09415?source=rss-2791e41d0113------2</link>
            <guid isPermaLink="false">https://medium.com/p/339a45d09415</guid>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Saiyam Bhatnagar]]></dc:creator>
            <pubDate>Fri, 17 Apr 2020 04:39:10 GMT</pubDate>
            <atom:updated>2020-04-17T17:53:31.107Z</atom:updated>
            <content:encoded><![CDATA[<p>Hello readers. This blog is written in the appreciation of a Research paper I am presently implementing in Tensorflow 2.0. I personally found the aim and the findings of the paper worth sharing with the audience who has not read it by now. Therefore, before diving into the beauty of this paper that is capable of generating more data to train itself and improve its accuracy, I would like to credit the authors —<strong> Gedas Bertasius , Christoph Feichtenhofer , Du Tran , Jianbo Shi , Lorenzo Torresani — Facebook AI, University of Pennsylvania.</strong></p><p>Modern approaches in deep neural networks for pose estimation require large amount of dense annotations. This means that if you want to train a neural network for Pose-Estimation you would require huge amount of manually labeled frames in a video with the coordinates of the points you would like your model to estimate. This process is very expensive as it requires manual labor. Now, this is where the network proposed in the paper steps it. Named a <strong><em>PoseWarper Network,</em></strong> this paper is capable of labeling frames in video with the aid of only a few labeled frames in a video.</p><p>For training, PoseWarper network inputs two frames of the same video- a labeled frame <strong>A</strong> and an unlabeled frame B which are separated only by a few steps of time<strong>. </strong>Now both of the frames are fed in to a network(<strong><em>SimpleHRNet</em></strong>) that generates the heatmaps <strong>f(A)</strong> and <strong>f(B). </strong>Now for those who are not familiar with what a HRNet is: It is a network that inputs images with a human and outputs the heatmaps (labeled image that specifies the joints of the person). For details of SimpleHRNet click <a href="https://github.com/stefanopini/simple-HRNet.git"><strong>here</strong></a>. For greater understanding refer to the figure given below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/610/1*dI14UcKJfPZfkUQ_oVHRbg.png" /><figcaption>the input and corresponding output of the SimpleHRNet</figcaption></figure><p>Now, after having got the two heatmaps (f(A), f(B)) from the <strong>pretrained</strong> HRNet, begins the trainable region of the <strong><em>PoseWarper</em></strong> network. In brief, this network uses the heatmaps of the unlabeled frame <strong>B</strong> and labeled frame <strong>A</strong> and computes the difference in the two. (f(A)-f(B)= <strong><em>phi</em></strong>). Now this <strong>phi </strong>is sent to the trainable region of the network which further warps <strong>phi </strong>with <strong>f(B)</strong> and tries it to match it with the ground truth reality in labeled <strong>frame A.</strong> This forms the training part of the network. This is the very aim of the PoseWarper network: training the weights so that the pose estimated in the unlabeled frame <strong>B</strong> matches the ground truth pose in the labeled frame <strong>A. </strong>Please refer to this pictorial representation for greater clarity.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/1*SxxxDXKjFZp5IWGn3YzrSw.png" /><figcaption>the training part of PoseWarper</figcaption></figure><p>A high level overview of the approach for using sparsely labeled videos for pose detection. In each training video, pose annotations are available only every k frames. During training, the system considers a pair of frames–a labeled Frame A, and an unlabeled Frame B, and aims to detect pose in Frame A, using the features from Frame B.</p><p>After training the model we are interested in passing some unlabeled frames and as an output getting those labeled. The very aim is to increase our training data set by continuously generating new data using our PoseWarper to label unlabeled frames in the videos. The picture below shows how model labels unlabeled frames.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/895/1*bGvgfw41Sp-RHw9H2B7caA.png" /><figcaption>labeling the unlabeled frame and generating more data simultaneously</figcaption></figure><p>Since the very purpose of the article is to bring out the idea mentioned in the paper and not the implementation details. Yet, we can ponder over the model architecture for greater clarity. Below is the image of the model architecture.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/1*LXELsc9-iHB9i8wZxPwFfA.png" /><figcaption>The model architecture</figcaption></figure><p>The readers can always read the research work <a href="https://arxiv.org/abs/1906.04016"><strong>here</strong></a> if they need to know more about the architecture, implementation details, experiments and the model’s capabilities.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=339a45d09415" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/how-facebook-ai-used-this-network-to-generate-more-data-to-train-a-pose-estimation-model-339a45d09415">How Facebook AI used this Network to generate more data to train a Pose-Estimation Model.</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Beginner’s guide to optimize Linear Regression models.]]></title>
            <link>https://medium.com/analytics-vidhya/beginners-guide-to-optimize-linear-regression-models-41272e6b5d91?source=rss-2791e41d0113------2</link>
            <guid isPermaLink="false">https://medium.com/p/41272e6b5d91</guid>
            <category><![CDATA[statistical-analysis]]></category>
            <category><![CDATA[linear-regression]]></category>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Saiyam Bhatnagar]]></dc:creator>
            <pubDate>Fri, 20 Mar 2020 18:45:04 GMT</pubDate>
            <atom:updated>2020-04-17T18:16:57.952Z</atom:updated>
            <content:encoded><![CDATA[<p>Linear Regression is one of the most widely used statistical tool for Machine Learning problems. For those who are not familiar with what a Linear regression model is; Linear Regression is an approach to model relationships between a <strong>dependent variable</strong> and several different <strong>independent variables</strong>. Briefly, it is sough to predict an unknown variable with the help of one or more known variables. However, this article focuses on ways to create a linear regression model which is devoid of over-fitting, generalizes the training data quite well and is computationally efficient at the same time.</p><p>The readers might be thinking why this is worth pondering over. The reliability of a large machine learning model whose inputs rely on the outputs of the Linear regression can be reasonably flawed if the Linear regression over-generalizes or over-fits. To add more to the problems, a Linear regression model’s computation expense <strong>increases</strong> with the addition for <strong>explanatory variables</strong>(the Variables used for predictions). For a quick look into what Linear regression equation looks like, see the figure below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/308/1*8IF8XumCJY5vfcl5IS0y9w.jpeg" /></figure><p>Suppose the data-set available contains hundreds of different features and the corresponding target values to train our regression model and get estimates of coefficients(b0, b1,b2….). However, with limited computation power the conundrum of not only choosing the <strong>appropriate features</strong> but also estimating the <strong>optimal number</strong> of features to be used, prevails. Thanks to the built in python libraries like scikit learn and numpy which readily bring forth estimates for regression coefficients. Now we would dive into the methodology of choosing the appropriate features and the number of features we want to be present in our regression equation. Let our variable to be predicted be called — ‘Y’ . Our primary objective is to get the <strong>correlation coefficients</strong> of Y and <strong>each of our independent variables</strong> separately. For this we can use the corrcoef() function in the numpy library. The sample implementation is given below. Here train_set is the training data(a data frame).</p><p>iterate x in (number_of_explanatory_variables):</p><p>print(np.corrcoef(Y,train_set[‘COLUMN NAME’]))</p><p>The output will be the correlation coefficients between the dependent and each of the corresponding independent variables. Those who are unfamiliar with correlation coefficient are advised to go through the <a href="https://youtu.be/bCpfd2PxBVA">link</a>. Now list down the variables in <strong>increasing</strong> order of the magnitude of the correlation coefficient(remember that correlation coefficient can also be negative). Prepare and pre-process these features to be a part of the regression equation. Too many independent variables will <strong>over-fit </strong>the training data and result in a not so good regression model. You can also get insights into over-fitting and under-fitting by searching about the <a href="https://towardsdatascience.com/understanding-the-bias-variance-tradeoff-165e6942">Bias-Variance trade-off.</a> There is also a simple figure illustrating the Bias-variance trade-off for those who are familiar yet wish to recall.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/830/1*ePMqfevdkn4GBgvsZKrCcw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/492/1*ZDWeg1VaHH84rxgwmcZ35Q.png" /></figure><p>Having prepared enough features to be a part of the actual training data-set, now get ready to check whether the <strong>addition</strong> of more independent variables improves the model’s accuracy and does not over-fit it. For this we use the concept of <strong>explained variation</strong> and <strong>unexplained</strong> <strong>variation</strong>. Generally speaking, better the explained variation better is the predictive power of our model. However, we should understand that in the quest of greater predictive power we tend to over-fit our model. Now let’s learn how to estimate the predictive power of our regression model. <strong>R-squared</strong> is the ratio of the <strong>explained variation</strong> to the <strong>total variation</strong> and is always less than 1 in magnitude. Intuitively, R-square signifies the <strong>predictive capacity</strong> of our model. Hence, it is always expected to increase when we add an independent variable to be a part of the regression model. Here comes adjusted-R-square to the rescue. Adjusted-R-square is <strong>not</strong> <strong>always</strong> <strong>increasing</strong>; it is like a modified version of R-square in such a way that its value <strong>decreases</strong> if the addition of another independent variable <strong>did not improve</strong> the predictive power of the model as <strong>expected</strong>. Therefore, if the value of Adjusted-R-Square decreases on addition of an extra independent variable, our model is not improving on its accuracy of predictions. Instead it has just started over-fitting the training data-set. For more insight into Adjusted-R_square <a href="https://analyticsindiamag.com/r-squared-vs-adjusted-r-squared/">click</a>. The computation of Adjusted-R-square can be done by using the formula below. Whereas R-square can be easily calculated by dividing the <strong>summation</strong> of squared deviation of predicted(Y) values from the actual mean value(mean(Y)) by the <strong>summation</strong> of squared deviation of actual(Y) values from the actual mean value(mean(Y)).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/474/1*rzqE8JVeqxoLobr6pg-34g.jpeg" /><figcaption>p is the number of parameters to be estimated</figcaption></figure><p>Keep on <strong>adding</strong> the independent variables to the model until you find that the value of Adjusted-r-square has eventually started <strong>decreasing</strong>. This is the point post which the addition of independent variables will invite <strong>over-fitting</strong>. Stop here and pass the <strong>selected variables</strong> in a linear regression model using scikit-learn, numpy etc. This is the proposed methodology behind the creation of a regression model that does not over-fit, generalizes well and is computationally efficient as the presence of number of features(independent variable) has also been checked.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=41272e6b5d91" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/beginners-guide-to-optimize-linear-regression-models-41272e6b5d91">Beginner’s guide to optimize Linear Regression models.</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>