<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Sathya Prakash V R on Medium]]></title>
        <description><![CDATA[Stories by Sathya Prakash V R on Medium]]></description>
        <link>https://medium.com/@sathyaprakash_2656?source=rss-27783573842a------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*NSdYmkbCWs7u_IqfnWt3_A.jpeg</url>
            <title>Stories by Sathya Prakash V R on Medium</title>
            <link>https://medium.com/@sathyaprakash_2656?source=rss-27783573842a------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 07:25:47 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@sathyaprakash_2656/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Logistic Regression]]></title>
            <link>https://medium.com/ml-and-deep-learning-blogs/logistic-regression-2f03826d6a31?source=rss-27783573842a------2</link>
            <guid isPermaLink="false">https://medium.com/p/2f03826d6a31</guid>
            <dc:creator><![CDATA[Sathya Prakash V R]]></dc:creator>
            <pubDate>Wed, 13 May 2020 17:08:30 GMT</pubDate>
            <atom:updated>2020-05-18T22:55:23.825Z</atom:updated>
            <content:encoded><![CDATA[<p>In this blog, I will try to explain logistic regression using Geometric intuition.</p><blockquote>Even though it has a regression in its name, it is a supervised classification algorithm.</blockquote><p><strong>Assumption: </strong>Logistic Regression assumes that data is linearly separable or almost linearly separable data, which means that it is applicable only when data is separated by using linear surfaces<strong><em>.</em></strong></p><p><strong>Note:</strong> linear surfaces mean a line in 2D, the plane in 3D, a hyperplane in nD.</p><p>All the ML algorithms we use will do the following things: model, estimate, predict, classify.</p><p>Logistic regression seeks to:</p><p>To <strong>model </strong>the probability of an outcome of the dependent variable(y) given the input data(xi)or independent variables. (on train data)</p><p>To<strong> estimate </strong>the probability of an outcome(y) of a randomly selected observation or data point(x)(on cross-validation data)</p><p>To <strong>predict </strong>the effect of independent variables(x) on the outcome or dependent variable.(on cross-validation data)</p><p>To<strong> classify</strong> the dependent variable(y) by estimating the probability that an outcome belongs to a particular class. (on test data)</p><blockquote><strong><em>GEOMETRICAL INTUITION</em></strong></blockquote><figure><img alt="Fig 1: +ve and -ve data points seperated by a series of lines." src="https://cdn-images-1.medium.com/max/800/1*Bgq3qq3S_zbRP5DzYlQRDA.png" /><figcaption>FIG 1</figcaption></figure><p>In the above figure, we can see that positive and negative data points are linearly separated by a series of lines(2D), whereas in the case of 3D, it is a plane, and in the higher dimension, it is a hyperplane. However, many lines separate our data points linearly. Which one to choose?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ULwP9AtH_CRbOCDrXjeHFA.jpeg" /><figcaption>Fig 2</figcaption></figure><p>Fig3: gives proof for the distance of a point p from plane/line.</p><h4>points to be noted which we use in future</h4><p>1. Plane or line equation passing through origin is y=wx.</p><p>Note: W, X are weight and feature vectors, respectively.</p><p>2. By assuming w is a unit vector:||w||=1. Hence, the distance of the point x from plane/line is d=(w.T).x</p><p>3.if w and x are on the same side of the plane then d=w.x&gt;0, and if w,x are opposite sides of the plane then d=w.x&lt;0</p><p>4.for positive points y=1, for negative points y=-1</p><p><strong>Goal: <em>Logistic regression aims to choose the best plane/line that separates the data.</em></strong></p><h4>How to decide which one is the best line?</h4><p>To decide which is the best line, let us first choose a classifier model that classifies the data points based on certain conditions.</p><p><strong>Classifier:</strong></p><p>if (w.T).x(i)&gt;0 then y(i)=+1 ,else if (w.T).x(i)&lt;0 then y(i)=-1<br>Decision surface for logistic regression: line/plane</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*qeYXZxPdQ7-k7tm6uLCxIg.png" /><figcaption>Fig 3</figcaption></figure><p>Now that we have created a model clasiffier let us use it to classify our data points:let us take 4 points which are marked in fig3</p><p>Now that we have created a model clasiffier let us use it to classify our data points:let us take 4 points which are marked in fig3<br>case1:point1 y(i)=+1,(w.T)*(x)&gt;0 then y*((w.T).x)&gt;0 then point is correctly classified.<br>case2:point2 y(i)=-1,(w.T)*(x)&lt;0 then y*((w.T).x)&gt;0 then point is correctly classified.<br>case3:point3 y(i)=-1,(w.T)*(x)&gt;0 then y*((w.T).x)&lt;0 then point is incorrectly classified.<br>case4:point4 y(i)=+1,(w.T)*(x)&lt;0 then y*((w.T).x)&lt;0 then point is incorrectly classified.</p><p>hence we can say that when y(i)*((w.T).x(i))&gt;0 then our classifier is correctly classified and y(i)*((w.T).x(i))&lt;0 then our classifier is incorrectly classified the points.<br>For our classifier to be best, we need minimum misclassified points or maximum correctly classified points, i.e., We need to find a plane(model) such that it classifies most data points correctly.</p><p>In other words, we can say that given training data &lt;x(i),y(i)&gt;<br>we need to find a w which maximizes (y(i)*((w(i).T).x(i)).hence our optimization problem we need to solve becomes</p><p>To find</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/402/1*arJW0-nsDnF7M9zOCXj1_g.jpeg" /></figure><p>(y(i)*((w(i).T).x(i)) is the signed distance .</p><p>But there is a problem with the above optimization problem. It doesn’t account for outliers or noise data points.</p><blockquote><strong><em>The need for squashing or sigmoid function</em></strong></blockquote><p>Letus see how the above optimization problem does not account for data points.<br>for positive points(black) y=1, for negative points(red) y=-1,d=(w^T)*x<br>we will follow the coordinate system to assign signs to d.Here d is weight vector*data point.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*rl90P8zfifCH90EbBZqqsA.png" /><figcaption>Fig:4</figcaption></figure><p>if we find the sum of signed distance using plane1 for the given data points we get:<br>signed distance of L1=(y.d)=5(1)+(-5)(-1)+50(-1)=-40.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*eiVb5nNN--_flg1xJ2t4dA.png" /><figcaption>Fig:5</figcaption></figure><p>similarly signed distance of L2=(y.d)=1(1)+1(2)+1(3)+1(4)+1(5)+1(1)+1(2)+1(3)+1(4)+1(5)+1(-1)+1(-2)+1(-3)+1(-4)+1(-5)+(-1)(-1)=1</p><p>By using signed distance, we get -40 for plane1 and 1 for plane2. Asper our optimization problem, the best plane is the one with the max signed distance.so plane 2 is the best, But if we see visually, plane 1 is the best as it classifies 10 of 11 points correctly, whereas plane 2 classifies 6 out of 11 data points correctly. From the example, we can see that how one noisy data point can affect our algorithm.</p><blockquote><strong>How to account for this outlier or noisy data point problem?</strong></blockquote><p>we need a function that squashes the distance between the data point and plane if it is too large.<br>f(signed distance)=signed distance if d is small ,else<br>f(signed distance)=squashed(sign distance) if d is large.<br>By squashing the signed distance, we are trying to reduce the weight of the noisy data point so that it will not drastically affect our classifier.<br>The sigmoid function is one such function that squashes the distance went it is significant. Now our optimization problem becomes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/527/1*k5tpHpjgpoJ9LBWp8JxcXA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/422/1*WNryFK9d_E23uvHO9fDmRw.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/787/1*D3BHCuXBEa31x1203AHCYQ.jpeg" /><figcaption>source:<a href="https://tex.stackexchange.com/questions/497949/drawing-a-sigmoid-function-and-its-derivative-in-tikz">https://tex.stackexchange.com/questions/497949/drawing-a-sigmoid-function-and-its-derivative-in-tikz</a></figcaption></figure><p>In the above figure, if we consider x-ais is the signed distance of data point from plane and y-axis as sigmoid(signed distance), which is linear when distance small and as distance increases it, tapers off the result.<br>As we can see from the above figure when the signed distance is small, sigmoid(signed distance) is equal to the signed distance, and when the distance is significant sigmoid(signed distance) is squashed between 0 and 1.</p><blockquote><strong>Why only sigmoid to squash the distance?</strong></blockquote><p>1.since sigmoid function squashes range of distance from (-inf,inf) to (0,1) .Hence it not only classifies the data points but also gives us the probabilistic interpretation of the model.<br>2. To solve our optimization problem, we need a function that can be easily differentiable and easy to compute.</p><p>differentiation of sigmoid function is given by:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/245/1*WXG4Ssd_twgZcSNum5MxyQ.jpeg" /></figure><p>credits:<a href="https://www.appliedaicourse.com/course/11/Applied-Machine-learning-course">https://www.appliedaicourse.com/course/11/Applied-Machine-learning-course</a>,<a href="https://drawisland.com/">https://drawisland.com/</a>(for drawing), <a href="https://medium.com/u/89355345dd13">Samiksha Nagrare</a> for helping in structuring this blog.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2f03826d6a31" width="1" height="1" alt=""><hr><p><a href="https://medium.com/ml-and-deep-learning-blogs/logistic-regression-2f03826d6a31">Logistic Regression</a> was originally published in <a href="https://medium.com/ml-and-deep-learning-blogs">ML and Deep Learning Blogs</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>