<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ming Ying on Medium]]></title>
        <description><![CDATA[Stories by Ming Ying on Medium]]></description>
        <link>https://medium.com/@mingying_57930?source=rss-332a8a41840d------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*Of2mFYLsyW7e8DJH.</url>
            <title>Stories by Ming Ying on Medium</title>
            <link>https://medium.com/@mingying_57930?source=rss-332a8a41840d------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 00:28:49 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@mingying_57930/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Building a (Special) Python Flask Alarm Clock: Day I]]></title>
            <link>https://medium.com/ming-learns-thing/building-a-special-python-flask-alarm-clock-day-i-6f35b53db691?source=rss-332a8a41840d------2</link>
            <guid isPermaLink="false">https://medium.com/p/6f35b53db691</guid>
            <category><![CDATA[alarm-clock]]></category>
            <category><![CDATA[python-flask]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[web-design]]></category>
            <dc:creator><![CDATA[Ming Ying]]></dc:creator>
            <pubDate>Fri, 05 Jan 2018 03:29:02 GMT</pubDate>
            <atom:updated>2018-01-05T03:30:11.885Z</atom:updated>
            <content:encoded><![CDATA[<p>If you’re anything like me, you <em>want</em> to get up early. You set your alarm clock — but when the alarm rings, you press the snooze button. Feeling guilty later on, you install a few alarm clock apps. Those don’t work— and why should they? No matter how loud the alarm is, or how soothing the music sounds, or how vigorously you must shake your phone to convince the alarm to stop ringing (yes, this app exists), you always have the option to force quit the app and return to another hour of blissful sleep.</p><p>But what if you just <em>had </em>to get up? What if the consequence of you not getting out of bed was losing ten dollars? What if, in order to prevent those ten dollars from leaving your bank account, you had to get out of bed, walk over to your computer, and visit a certain website? Would that convince you to get out of bed every morning?</p><p>It would definitely motivate me.</p><p>I’m going to build a web app using Python Flask that accomplishes exactly that. Here’s a general outline of what my website hopes to accomplish: the night before, users visit my website and set a wake-up time (say 7 AM). They enter their credit card information, which will be processed securely using Stripe’s API. Then, half an hour before their intended wake-up time, they will receive a text message that contains a unique, six-digit code. If they enter that code into my website, which forces them out of bed and over to their computer, likely located across the room or in another room, nothing will happen. If, however, they fail to enter the code by 7 AM, a small monetary amount will be extracted from their credit card.</p><p>Notice how this web app bypasses the need to create an actual alarm clock app. Since most people own an alarm clock — whether it be a physical clock or an iPhone alarm clock — I can trust them to set their own alarms using whatever method they see fit. My website simply provides the motivation to actually get <em>out </em>of bed.</p><p>This is about as far as I’ve thought this through. I’ll do some outlining and brainstorming below.</p><h4>Features</h4><ol><li><strong>Login/Register</strong>: Users will need their own personal account in order for the web app to recognize which six-digit code belongs to whom.</li><li><strong>Selecting a wake-up time</strong>: Users will have the option to set the wake-up time as a one-time wake-up call or as a recurring wake-up call.</li><li><strong>Text/email capability</strong>: Users will have the option to choose whether to receive the six-digit code through email or text message. This will also allow me to send notifications that remind users at night to set their alarms for the next day.</li><li><strong>Mobile incompatibility</strong>: In order to force users out of their beds to their computers, the code-entering portion of the website will be inaccessible on devices with small screens.</li><li><strong>Selecting a monetary punishment</strong>: Users will have the option to choose how much money they will lose if they fail to wake up on time. I believe that $5-$50 is a healthy range.</li><li><strong>Delayed credit card charging</strong>: This will likely be the biggest challenge, as users will not be charged immediately when they enter their credit card info, but later on, if they fail to wake up. I’ll have to look into whether Stripe supports this feature; if not, I may have to research other payment processors like Braintree or PayPal.</li><li><strong>Social media sharing</strong>: In order to gain virality, I’ll need to create Facebook and Twitter share buttons.</li></ol><p>OK, I think that’s a fairly comprehensive outline of what my web app will accomplish. I’ll start coding tomorrow.</p><p>Until then,</p><p>Signing off!</p><p>-January 4, 2018 at 10:28 PM</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6f35b53db691" width="1" height="1" alt=""><hr><p><a href="https://medium.com/ming-learns-thing/building-a-special-python-flask-alarm-clock-day-i-6f35b53db691">Building a (Special) Python Flask Alarm Clock: Day I</a> was originally published in <a href="https://medium.com/ming-learns-thing">Ming Learns Thing</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Machine Learning: Bayesian Linear Regression]]></title>
            <link>https://medium.com/ming-learns-thing/machine-learning-bayesian-linear-regression-f160c4eaef99?source=rss-332a8a41840d------2</link>
            <guid isPermaLink="false">https://medium.com/p/f160c4eaef99</guid>
            <category><![CDATA[linear-regression]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[bayesian-statistics]]></category>
            <dc:creator><![CDATA[Ming Ying]]></dc:creator>
            <pubDate>Sun, 24 Dec 2017 03:13:10 GMT</pubDate>
            <atom:updated>2017-12-24T04:10:31.457Z</atom:updated>
            <content:encoded><![CDATA[<p>Today, I’m starting Unit Two of CS 181. The reading, Chapter 3.3 of Bishop’s textbook, is quite obscure, as Bishop glosses over many important steps in the derivations of the formulas he presents. After two hours of watching YouTube tutorials, reading StackExchange posts, and skimming Wikipedia articles, I’ve developed a better understanding of Bayesian linear regression. Below, I hope to clarify the material covered in Chapter 3.3.</p><h4>Bayesian Linear Regression</h4><p>To begin: what is the difference between a Bayesian linear regression and an ordinary least squares (OLS) regression, and why do we prefer one over another? Put simply, Bayesian and OLS regressions are two different methods of finding the best predictive model for an unknown data set. An OLS regression employs a frequentist approach; that is, it treats parameters in a model as unknown constants, whose values must be derived. A Bayesian regression uses a Bayesian approach (hence the name) — it treats parameters as <em>random variables </em>with underlying distributions. Therefore, Bayesian linear regressions are better than OLS regressions at representing the uncertainty of a model’s predictions.</p><p>The goal of a Bayesian regression is to find a weight vector <strong>w </strong>that is the mode of its underlying distribution (i.e. the vector that appears most often in the distribution). To do this, we must solve for the probability density function (PDF) of <strong>w</strong>, which we can model as <em>p(w|D)</em>, where <em>D</em> is our set of training data. We call <em>p(w|D)</em> the posterior probability, since it is the probability distribution of <em>w</em> conditioned on information from the training data. Using Bayes’ theorem, we can divide our posterior probability into two parts:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/174/1*VfVai89UGDR_FaCQEIyKcw.gif" /><figcaption>Equation (1)</figcaption></figure><p>Where <em>p(w)</em> is our initial probability distribution of <strong>w</strong>, called the <em>prior distribution</em>, and <em>p(D|w)</em> is called the <em>likelihood function</em>. We begin by examining <strong>w</strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/120/1*ZG7hTDpNuziMzAi6RURB0A.gif" /><figcaption>Equation (1)</figcaption></figure><p>Let us consider what Equation (1) means. We first assume, for the sake of simplicity, that <strong>w </strong>follows a multivariate normal distribution. If a vector follows a multivariate normal distribution, then all of its components — and any linear combination of those components — are normally distributed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*A97sbQvl-fyXyFbko370Ug.png" /><figcaption>A three-dimensional multivariate normal distribution</figcaption></figure><p>We also assume that the mean of <strong>w</strong> is 0, and that each component of <strong>w</strong> is independently and identically distributed (i.i.d.). Finally, we take the covariance matrix as given.</p><p>To find <em>p(w)</em>, we can express <strong>w</strong> in terms of its multivariate normal PDF:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/260/1*r8P9LaKSjCFTpImaceW7jw.gif" /></figure><p>Where <em>Σ </em>is the covariance matrix. Next, we can substitute in for <em>Σ </em>and remove the constant in the front of our expression, since constants become irrelevant for probabilities once the probability is normalized:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*6cyEM9t29r27EXE4IKz7yg.gif" /></figure><p>Lastly, we observe that <em>μ</em>, the mean of <strong>w</strong>, is zero, as assumed in Equation (1). Substituting in, we have:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/133/1*zl5rnbPEOsm5BrvNVY52wg.gif" /><figcaption>Equation (2)</figcaption></figure><p>Here, we note that some textbooks replace the term <em>1/σ²</em> as a constant, known as the “precision”; for clarity, we will leave Equation (2) as is.</p><p>Now that we’ve derived <em>p(w)</em>, our next task is to derive <em>p(D|w)</em>. We observe that, since our response variables <strong>Y</strong> are all normally distributed (an assumption of linear regression), then <strong>Y </strong>also follows a multivariate normal distribution. Therefore, <em>p(D|w)</em> can be expressed as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/503/1*TMjND3XdP-5g4YtDoWt5rA.gif" /></figure><p>Our final step in deriving <em>p(w|D)</em> is to multiply our expressions for <em>p(w)</em> and <em>p(D|w)</em> together:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/293/1*i4cUplRTnJW0_vTNZu3PHQ.gif" /><figcaption>Equation (3)</figcaption></figure><p>Now that we have <em>p(w|D)</em>, we can solve for <strong>w</strong> by finding the mode of <em>p(w|D)</em>. Here, we observe that <em>p(w|D)</em> also follows a multivariate normal distribution by the theorem that a normal prior multiplied by a normal likelihood function produces a normal posterior distribution (Wikipedia article for reference: <a href="https://en.wikipedia.org/wiki/Conjugate_prior">https://en.wikipedia.org/wiki/Conjugate_prior</a>), and that the mode of any normal distribution is its mean (visualize the bell curve). Since <em>p(w|D)</em> is multivariate normal, we can therefore set the exponent of Equation (3) equal to the exponent of a generic multivariate normal distribution:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/427/1*3zviHCWYSu8MMfQT5srMPQ.gif" /></figure><p>Note how we removed the <em>-1/2</em> from both sides of the equation. Solving for<strong><em> </em></strong><em>μ</em>, we find:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/123/1*PI47ftm-F6c62K-ynvdJhw.gif" /></figure><p>Where <em>Λ </em>is the inverse covariance matrix. Solving for <em>Λ</em>, we find:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/156/1*_850wPMKy0C9SYMhLuqCYA.gif" /></figure><p>Substituting our expression for <em>Λ </em>into our expression for <em>μ</em>, we find:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/223/1*NeQNaM30ggjkJJfd0kl2ZA.gif" /><figcaption>Equation (4)</figcaption></figure><p>From Equation (4), we see something interesting: the weight vector selected by a Bayesian regression is a special case of the weight vector chosen by a Ridge regression.</p><h4>Thoughts and Conclusions</h4><p>Well, that’s all for now. This chapter took me much longer to get through than I had expected, especially since I had to rewrite this entire blog post after accidentally deleting my entire account (oops). Unfortunately, Chapter 3.3 still contains two more subsections that I have yet to read. I’ll attempt to get through more of Chapter 3.3 tomorrow.</p><p>Until then, signing off,</p><p>12/23/2017 at 11:10 PM</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f160c4eaef99" width="1" height="1" alt=""><hr><p><a href="https://medium.com/ming-learns-thing/machine-learning-bayesian-linear-regression-f160c4eaef99">Machine Learning: Bayesian Linear Regression</a> was originally published in <a href="https://medium.com/ming-learns-thing">Ming Learns Thing</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Machine Learning: Regression I]]></title>
            <link>https://medium.com/ming-learns-thing/machine-learning-regression-i-ef7c0d8552db?source=rss-332a8a41840d------2</link>
            <guid isPermaLink="false">https://medium.com/p/ef7c0d8552db</guid>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Ming Ying]]></dc:creator>
            <pubDate>Sun, 24 Dec 2017 02:30:42 GMT</pubDate>
            <atom:updated>2017-12-24T03:25:06.193Z</atom:updated>
            <content:encoded><![CDATA[<p>After completing my Artificial Intelligence course this past fall semester, the next step in my computer science journey — and the final course of my computer science minor — is CS 181: Machine Learning. I’m set on taking this class in the upcoming spring, but it’s a challenging course, and I have a busy schedule ahead. Therefore, the first goal of my one-month winter holiday is to finish learning the course independently.</p><p>Luckily, the course website (<a href="http://cs181.fas.harvard.edu/">http://cs181.fas.harvard.edu/</a>) — replete with reading assignments and problem sets — is publicly available. After a few minutes on Library Genesis, I’ve downloaded the required textbooks onto my computer. I’m ready to go.</p><p>Unit One focuses on linear regression. Coming directly out of two regression-oriented classes — Linear Modeling (Stat 139) and Econometrics (Econ 1126) — I feel fairly confident in my understanding of regression. I’ll begin by summarizing what I learned from the required reading, Chapter 3.1 in the Bishop textbook.</p><p><strong>Linear Basis Function Models</strong></p><p>A linear regression is an equation that takes in a vector of input variables <strong>X </strong>and attempts to predict the value of a response variable <strong>Y </strong>(or, as the textbook calls it, a target variable <strong>t</strong>). The general form of a linear regression is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/344/1*gpPzPtalJXZhaKzCqJb8iQ.png" /><figcaption>Equation (1)</figcaption></figure><p>Where wⱼ is simply a constant, referred to as a weight, and <em>ϕ(X</em>ⱼ<em>) </em>is a function, referred to as a basis function. Popular choices for <em>ϕ(X</em>ⱼ<em>) </em>include polynomial, <em>tanh</em>, and logistic functions. We can see from Equation (1) why <em>Y|X₁…Xₙ </em>is called a linear regression — even if <em>ϕ(X</em>ⱼ<em>) </em>is itself nonlinear, if we treat <em>ϕ(X</em>ⱼ<em>) </em>as a single variable, then <em>Y|X₁…Xₙ </em>becomes a simple, linear equation.</p><p>If we denote <strong>W </strong>as the vector of weights <em>{w₁…wₙ}</em>, denote <em>ϕ(X) </em>as the vector of basis functions, and let <em>ϕ(X</em>ₒ<em>) = 1, </em>then we can rewrite Equation (1) using summation notation or vector notation:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/296/1*ngWqYm54cPsGobYlAYNbPw.png" /><figcaption>Equation (2)</figcaption></figure><p>In practice, our predictor variables <strong>X</strong> and our response variables <strong>Y</strong> will be given to us — this set of provided data is called the <em>training set</em>. Our next logical step, then, is to solve for the remaining unknown: the vector of weights <strong>W</strong>. Hypothetically, these weights could take on any value; however, we want to set them equal to values such that our regression can accurately predict any <strong>Y</strong>ⱼ given any <strong>X</strong>ⱼ.</p><p>How do we determine if the weights in our model are “accurate?” One way to test their accuracy is by examining our model’s sum of squared residuals (SSR). Recall that a residual is defined as the difference between our model’s prediction for <strong>Y</strong>ⱼ and the true value of <strong>Y</strong>ⱼ given to us in the training set:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/151/1*9vADSRTEoNZPJHZkxkYvEw.png" /></figure><p>Here, we observe that, if our predicted value of <strong>Y</strong>ⱼ is greater than the true value, then our residual will be negative. Since we only care about the distance between our predicted value and the true value and less about the sign of that distance, we square each residual and then sum them all up:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/165/1*iU1rfXrsAkVb2KeFKypshA.png" /><figcaption>Equation (3)</figcaption></figure><p>One may wonder why we square each residual rather than taking its absolute value. Both methods are, in fact, valid; however, squaring the residual is the more commonly-used technique due to the ease with which we can take the derivative of the SSR, which would be the next step in our approach if our model contained only one parameter.</p><p>In this case, our regression contains multiple parameters (i.e. a multiple regression). To proceed, we use a geometric interpretation of our residual minimization problem:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7HJ0AKwcVyLKhTRm1dZo4g.png" /></figure><p>The above diagram is an illustration of our regression problem. We observe that Ŷ (equivalent to <em>Y|X₁…Xₙ</em>) is a linear combination of vectors in the column space of <strong>X</strong>. Since our regression likely produces some errors in prediction, the vector of true values <strong>Y </strong>lies on a separate plane. The residual of any prediction <strong>Y</strong>ⱼ, denoted by <em>r</em>, is simply the distance between <strong>Y </strong>and its projection onto <strong>Y</strong>ⱼ. Therefore, every residual must be <strong>orthogonal </strong>to Ŷ (note from the illustration above that the pink line forms a right angle with Ŷ).</p><p>Recalling that the dot product of two orthogonal vectors equals zero, we can rewrite Equation (3):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/174/1*uJ3DweAijxvmLObp0MH2ag.png" /></figure><p>Solving for <em>w</em>, we obtain:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/142/1*INzF18OrXY3Dc0L7ZcBYQQ.png" /><figcaption>Equation (4)</figcaption></figure><p>Where Φ represents an n × j matrix of all possible combinations <em>ϕ</em>ⱼ<em>(X</em>ₙ<em>). </em>We have now arrived at the solution for our linear regression.</p><p>By substituting our training set values for <strong>X </strong>and <strong>Y</strong> into Equation (4), we can find the appropriate, SSR-minimizing coefficients.</p><p>A second way to derive Equation (4) is through an approach called maximizing the likelihood function. This is the textbook’s preferred method, and is slightly less intuitive but nevertheless useful. To understand what a likelihood function is, we first note that all linear regressions assume a normal distribution of response variables <strong>Y</strong>ⱼ (for reference, the four basic assumptions of any ordinary least squares regression are constant variance of residuals, independence of residuals, linearity, and normality of response variables/residuals). We can therefore express the distribution of any predicted value <strong>Y</strong>ⱼ as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/179/1*CFlqUfPUl8BDUTR_mlaQVA.png" /></figure><p>Or, in terms of the probability density function (PDF) of a normal distribution:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/280/1*uBXh1aMGOjmMtFWSXaMkaw.png" /><figcaption>Equation (5)</figcaption></figure><p>If Equation (5) gives us the PDF of a single response variable <strong>Y</strong>ⱼ, then the PDF of our entire training set of response variables, <strong>Y</strong>₁…<strong>Y</strong>ₙ, can be expressed as the product of individual PDFs:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/293/1*MWNsgUqwuKNJ4bgNyE1RKQ.png" /></figure><p>We simplify this expression by taking the natural logarithm of both sides:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/357/1*Lb0apUN0utj_Ze8WumdA0w.png" /></figure><p>Then, using the property of log multiplication, we can simplify further:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/464/1*_n5AZ0Q2LWTWNJcUWBgw5w.png" /><figcaption>Equation (6)</figcaption></figure><p>Where:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/103/1*WUfPtwj4o29ZXOmZaZJ3Qw.png" /></figure><p>Recall that our goal here is to find <em>w </em>such that <em>P(Y₁…Yₙ|X) </em>is maximized. Since the natural logarithm is a monotonic transformation, we can also maximize the log of this probability function; or, equivalently, minimize the negative log of the probability function. Multiplying Equation (6) by <em>-1</em> and taking the gradient with respect to <em>w</em>, we observe that we arrive at the same residual-minimization problem as that of Equation (3).</p><p><strong>Stochastic Gradient Descent</strong></p><p>Often, computing the weights of extremely large data sets using Equation (4) can be computationally tasking. An alternative strategy, called stochastic gradient descent, works by computing an initial vector <em>w</em> from a small, randomly-selected subset of the training data and updating <em>w</em> with each additional data point:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/144/1*GWthJ-fcwt9_8fwNLY-6Fw.png" /><figcaption>Equation (7)</figcaption></figure><p>Where <em>t </em>is the number of iterations thus far, <em>n</em> is the learning rate parameter (a constant), and <em>Eⱼ</em> is the error function that we are trying to minimize (in this case, the sum of squared residuals).</p><p>Intuitively, Equation (7) works by finding a local minimum of <em>w</em>. When we subtract off the gradient from <em>w</em>, we are essentially moving in a direction in which <em>w</em> is decreasing. Hopefully, after a certain number of iterations, our value of <em>w</em> will converge to a point at which the gradient of <em>Eⱼ </em>equals to zero.</p><p><strong>Ridge and Lasso Regularization</strong></p><p>Whereas we use stochastic gradient descent when our training data set is too large, another issue arises when our data is too small relative to the number of weights we are trying to estimate. This problem, called the overfitting problem, causes our regression model to lose generalizability — that is, the ability to accurately predict response variables that lie outside the training data set.</p><p>As a general rule, our data set should contain at least 10–15 observations for every predictor variable that we try to include in our regression. If our predictor variables are multicollinear — that is, if two or more predictors are strongly correlated with one another — then we require an even larger sample size. This is due to the fact that interdependence among our predictor variables makes it difficult to estimate their respective coefficients, since we can no longer isolate the influence of the predictor variable on the response variable.</p><p>Therefore, we introduce two similar techniques to resolve the overfitting problem: the Ridge and Lasso regressions.</p><p>Whereas the ordinary least squares regression is a pure optimization problem that tries to minimize the SSR, we can think of the Ridge and Lasso regressions as a constrained optimization problem that subjects our weights <strong>w </strong>to a certain upper limit.</p><p>More specifically, the Lasso regression says:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/91/1*MjsT7wC9GG9QldzCNKaHkw.png" /></figure><p>While the Ridge regression says:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/75/1*quE2XnNFmQZ-tk4DL1G-6w.png" /></figure><p>Where <em>c</em> is an arbitrary constant. Focusing on the Ridge regression, we can express our Lagrangian as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/381/1*LhgfBPGvFOOiEIw3Kx5cag.png" /></figure><p>Differentiating and solving this Lagrangian, we find our new expression for <strong>w</strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/183/1*8O-hUz6dhtoalaCDcsaI6Q.png" /></figure><p>Here, we can derive some intuition as to what Ridge does to our parameters. If the value of λ is large, then the ΦᵀΦ term becomes irrelevant. The inverse of λI is I/λ; therefore, as λ grows larger, the fraction I/λ shrinks, so the weights inside our vector <strong>w</strong> shrink as well. This penalizes weights that are unreasonably large, decreasing them to a more reasonable value.</p><p><strong>Thoughts and Conclusions</strong></p><p>The reading was certainly more dense than I expected. While I had covered most of this material in my other classes, I learned quite a bit of new material and gained an in-depth appreciation of the mathematics and intuition behind methods like regularization and ordinary least squares.</p><p>I’m falling slightly behind schedule; I had planned to finish reading the Scikit-Learn documentation today and to start the assigned problem set. By the end of tomorrow, I should be able to finish reading all the Unit One material and begin the problem set.</p><p>Until then, signing off!</p><p>12/22/2017 at 12:48 AM</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ef7c0d8552db" width="1" height="1" alt=""><hr><p><a href="https://medium.com/ming-learns-thing/machine-learning-regression-i-ef7c0d8552db">Machine Learning: Regression I</a> was originally published in <a href="https://medium.com/ming-learns-thing">Ming Learns Thing</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>