Linear Regression in Two Minutes
Hi technocrat! In this blog, I’m going to explain linear regression in the easiest way you’ve ever learned it.
Before we start, I’ll assume you know a bit of machine learning already — things like what training and testing mean, what a train-test split is, and the difference between classification and regression.
Let’s Start Our Journey
I want you to recall a small concept from geometry: the line equation.
ŷ = mx + c
This simple line is our machine learning model. In formal terms, this is called the hypothesis function — the function our model uses to make predictions.
Now you might ask, “How can a simple line be a machine learning model?” But yes — it is one. Let me walk you through it.
This line, y = mx + c, tries to pass through all the data points as closely as possible. Basically, in machine learning, what do we want? We want our model to predict values exactly, or as closely as possible, given some input.
Here’s the key word: closely. Given an input, our model (the line) tries to pass through all the data points closely, because that’s what ensures the model is making good predictions.
But how do we know which line is the best line?
Choosing the Best Line
Look at the line equation again: ŷ = mx + c
- ŷ is the prediction
- x is the input
- m is the slope
- c is the intercept
Together, m and c are called the parameters (or weights) of the model — the values the model actually “learns” during training.
Get Medisumeet’s stories in your inbox
Join Medium for free to get updates from this writer.
Think about it this way: we can’t change the input, because without input there’s no model. And since the output depends on the input, we can’t directly change y either.
But there are two other terms in the equation — m and c. We can change these, because they don’t depend on any other value. If you remember basic geometry, changing m and c changes the orientation and position of the line, letting it shift anywhere in the plane.
So the idea is: we need to choose m and c in such a way that our line (our model) passes as closely as possible through all the data points. This ideal line is called the best-fit line, and the overall approach of finding it this way is known as Ordinary Least Squares (OLS).
Which raises the next question — how do we actually choose m and c?
Finding m and c
Here, y is the actual value and ŷ is the predicted value. We want the model to have zero error, or as little error as possible.
So, y − ŷ is the error for a single data point — this is called the residual. Let’s square it: (y − ŷ)² — squaring cancels out the effect of positive and negative errors so they don’t cancel each other out.
When we do this for every data point and average it out, we get the model’s cost function (also called the loss function). For linear regression, this specific cost function has a name too — it’s called Mean Squared Error (MSE).
We want this cost to be as small as possible. If we expand the cost function using our model:
J(m, c) = (y − mx − c)²
Here, y and x are known (they come from our data), while m and c are the unknowns we get to choose. This whole process of adjusting m and c to bring the cost down is what we call training the model.
So how do we find the m and c that minimize this cost? Differentiation is the concept behind this — this is where optimization comes into the picture.
We take the partial derivative of the cost function with respect to m and with respect to c, and set both to zero. This gives us the m and c values where the cost is at its minimum. Solving these equations directly is called the Normal Equation method (one of the ways to find the optimal parameters, alongside iterative methods like Gradient Descent).
Once we find those values and plug them back into our model, ŷ = mx + c, our machine learning model is ready and trained.
That’s all from my side. Thank you!
