Derivatives & Gradients — How a Little Calculus Shapes Machine Learning
When I first heard the phrase “machine learning”, I imagined a sleek, mysterious black box — something far too technical for someone like me.
Then I learned that, at its heart, machine learning is a little like teaching a child to ride a bicycle. You give gentle pushes, small adjustments, and gradually they find their balance.
Those “adjustments” in machine learning? That’s where derivatives and gradients quietly work behind the scenes.
Why Calculus Matters in Machine Learning
In a neural network, we have “weights” — numbers the model uses to make decisions.
We want those weights to be just right, so the model gets better over time.
The process of tuning them is called gradient descent — taking small steps, again and again, until we’re as close as possible to the right answer.
To make those steps, we need to know how changing something will change the outcome.
That’s exactly what derivatives tell us: the rate of change.
What Exactly is a Derivative?
Imagine you’re hiking up a hill.
At every point, the slope tells you how steep the path is — that slope is your derivative.
In math language:
- Geometry’s view: the slope of a curve at a point.
- Physics’ view: the rate at which something changes.
Simple Rules for Derivatives
Even if math feels like a distant memory, these are easy patterns to learn:
- Constant Rule: The slope of a flat road is zero.
- Power Rule: Multiply by the exponent, then subtract one from it.
- Example: If f(x) = 3x⁵, then f’(x) = 15x⁴.
- Sum Rule: The slope of a sum is the sum of the slopes.
- Product Rule: If two things multiply, their slope involves both parts changing.
- Chain Rule: For “functions inside functions,” take it step by step.
- Quotient Rule: For division, there’s a special pattern (we won’t dive deep here).
Partial Derivatives — Change in One Direction
What if we have more than one variable?
A partial derivative is like asking:
“If I walk only east or west, ignoring north and south, how steep is it?”
Get Wendy’s Log from Neverland’s stories in your inbox
Join Medium for free to get updates from this writer.
Example: If f(x, y) = 3x²y:
- ∂f/∂x = 6xy (change when moving along x)
- ∂f/∂y = 3x² (change when moving along y)
This is the first step toward gradients.
The Gradient — A Compass for Change
A gradient is simply all the partial derivatives, gathered into a vector:
∇f(x, y) = [∂f/∂x, ∂f/∂y]
Think of it as a compass that points in the direction of the steepest climb — or, in machine learning, the fastest improvement.
Gradient Descent — Gentle Steps Toward Better Answers
If the gradient points “uphill,” we can take steps “downhill” to find the lowest point — the place where our model makes the smallest mistakes.
We don’t jump; we take small, careful steps, adjusting each time.
A Gentle Closing Thought
Derivatives and gradients are not just cold equations.
They are the quiet guides, showing us how to improve — one step, one adjustment at a time.
And once you start seeing them this way, the math becomes less about fear, and more about understanding the language of change.
