Heading Straight for the Peak: Why the Gradient Vector Points to Steepest Ascent
Imagine you are climbing a mountain and want to find the steepest (quickest) path to the peak. Which direction should you step? This is the classic math problem of finding the gradient vector of a function.
When I set out to explain this to my kid, who is currently studying calculus, the journey ended up reinforcing my own understanding, specifically why the gradient vector always points in the direction of steepest ascent.
You see this idea everywhere in optimization, notably as the driving engine behind training artificial neural networks that powers the AI revolution.
The first step is to model the mountain mathematically.
Modeling the Problem Mathematically
Like an ancient Greek philosopher seeking an ideal form, I can picture the mountain as a symmetric cone whose peak points straight up from its flat circular base. Because the climber only navigates the terrain, not the solid interior, I only need the outer boundary.
A natural starting point is the equation that defines a conical surface. The surface of my idealized mountain cone can be described by relating elevation z to horizontal distance (radius) from the center:
where H is the peak elevation at (0, 0) and k represents the steepness of the slope.
To make it simple, let’s set k = 1. That means my idealized mountain has a 45-degree slope (a 1:1 rise-to-run ratio), where the height of the cone equals its base radius.
At the peak, x=0 and y=0, z is the same as H. At any fixed elevation, z = a, rearranging the equation:
gives the circular cross-section expressed in x² + y² = r² where the radius (r) shrinks linearly as you climb toward the peak.
Once the mountain is modeled, I can examine the surface under the climber’s feet. Their location is tracked in 3D coordinates (x, y, z) where (x, y) pinpoints their location on the flat base and z measures their elevation above it. Basically, the climber stands at coordinates (x, y) on the base, lifted to height z on the mountain surface. This conical function returns the climber’s elevation (z) given the climber’s position in x and y, which is suitable to describe the position of the climber in 3D coordinates.
In this coordinate system, I want to analyze the surface slope along the y-direction while holding the x-direction constant. Geometrically, this means slicing the cone with a vertical plane parallel to the y-z plane, revealing a 2D cross-sectional curve along the mountain’s surface.
The next step is to determine the steepness of that cross-sectional curve at the climber’s position. Imagine a straight line touching the curve at only that single point where the climber is standing, as shown in the far-right image below. The tilt of that line, also know as tangent line, tells me the exact steepness under the climber’s feet.
Here lies my kid’s dilemma: grounded in middle school algebra, his instinct was that calculating a slope strictly requires two points to compute the “rise over run” as shown in the far-left image above. If the line only touches a single point on the curve, as seen in the far-right image, the two points collapse into the exact same spot. Moving from left to right, you can see the horizon distance between the points, y₀ and y₁, shrinking to zero, yielding 0/0. The formula breaks down, because you cannot divide by zero.
To address this dilemma, I first need to explain a fascinating paradox to my kid: Zeno’s arrow. Zeno’s paradox is resolved by the concept of the derivative, making a perfect bridge to the next concept in my kid’s differential calculus journey.
Zeno’s Arrow
Zeno, an ancient Greek philosopher, argued that to measure steepness, you need a horizontal step forward (run) and the vertical change in height (rise), which is exactly what my kid thought. But he went even farther by arguing that if you stand on one exact spot without moving, your step forward is zero, and consequently, your change in elevation (rise) is also zero. So, he concluded that a single point has no width; therefore, it cannot have a slope. This is completely paradoxical. If no point has steepness, how can the hill as a whole be steep?
By the way, this is not exactly what Zeno said, as his original paradox was about an arrow in flight and the illusion of motion. I simply adapted his logic for the purpose of this problem.
Differential Calculus in the Rescue
“Think of it as a ratio”, I told my kid. Both rise and run shrinks simultaneously. Instead of freezing at a single point, see what happens to the steepness across smaller and smaller gaps. As the gap continuously approaches zero, the ratio stabilizes toward a specific, non-zero number. This behavior, the trend of the shrinking ratio, is the foundation of the derivative.
For example, the ratio of two microscopic numbers close to zero, such as, 0.0000000000001/0.0000000000002, simplifies cleanly to 1/2. Even as these quantities become infinitely small or “infinitesimal”, I can still calculate their proportion or ratio without breaking math rules.
By examining this progression, I can ask, what value the ratio targets as the gap vanishes? In other words, what is the limit of that ratio as the gap shrinks? As an example, the table below illustrates this predictable trend as the horizontal gap (run) approaches zero, showing the steepness targeting a clear destination: 1.
Derivative
Let me briefly remind you of the earlier setup. For this cross section, since I am looking only at the slice parallel to the y-z plane, x stays locked in place. That means elevation z depends entirely on y, giving me a simple single value curve, z =f(y).
The derivative of the function is formally defined as:
where the gap between the two points is represented by h. As h approaches zero, the shrinking gap reveals the exact steepness at that single point.
Let see how this limit is evaluated algebraically, since the equation defining a conical surface makes the derivative relatively easy to derive.
Given the equation,
I can find the derivative of the function with respect to y first. This is called a “partial derivative” (∂) since the function depends on two input variables (x and y) and x is treated as a constant.
Perfect! I have the steepness under the climber’s feet in the y direction.
Next, I need to find the steepness in the x direction by analyzing the slice parallel to the x-z plane, revealing the cross-sectional curve along the mountain’s surface.
Get Jae H. Park’s stories in your inbox
Join Medium for free to get updates from this writer.
Moving forward, I would not use the formal limit definition every time I find a derivative. There are well-defined calculus rules that make this calculation much faster. I am going to use both “Power Rule” and “Chain Rule” as shown below.
Once the power and chain rules are applied, finding the derivative with respect to x becomes much easier. By the way, the derivative of any constant is zero, which applies directly to both H and y² here.
I can package these two rate of changes (partial derivatives) into a single vector, placing the derivative with respect to x as a first element and the derivative with respect to y as a second. It is known as the gradient vector of the function.
Now I can pinpoint the path of steepest ascent toward the peak. However, I have not yet proved why the vector defined by both directional slopes guarantees the steepest path. I will tackle that proof next.
The Dot Product Proof
Now that I have a vector built from these two partial derivatives, let’s tackle the main question: why does it point to the direction of steepest ascent? I need two additional insights to prove this.
Additional Insights
First, I need a key insight from circle geometry: a tangent line to a circle is always perpendicular to the radius at the point of contact. This holds true because the radius represents the shortest possible distance from the circle’s center to that tangent line.
Second, if I slice the conical surface horizontally using a plane parallel to the x-y plane, each elevation z produces a circle. This cross-section is known as a level curve or contour line, representing a path of constant elevation.
Given that the climber stands on the contour line, walking along its tangent line gives no change in height. If the gradient vector is perpendicular to this tangent line, it must point along the radius straight toward the center of the circle (0, 0). Because the peak sits directly at the center, this proves that the gradient vector points to the direction of steepest ascent. Note that while elevation z changes, the gradient vector itself naturally lives in the horizontal x-y plane.
Tangent Line on the Circle
Now I need to determine the equation of this tangent line at the climber’s position on the circle. Once the line is determined, a vector of any length along it (specifically, a unit vector) can be used to evaluate its relationship with the gradient vector. I can verify their perpendicularity directly by computing the dot product of the two vectors. The dot product is another topic I need to cover as part of this proof.
Finding the equation of the circle is relatively simple. Just think of the Pythagorean theorem:
r² = x² + y²
where r is the radius of the circle. As mentioned earlier, at any fixed elevation, z = a, I can rearrange the cone equation:
x² + y² = (H — a)² where (H — a) = r
which gives the circular surface expressed in:
x² + y² = r²
where the radius (r) shrinks linearly as the climber moves toward the peak.
To find the slope of the tangent line to the circle, r² = x² + y² at the climber’s position (x, y), I take the derivatives of both sides with respect to x, treating y as an implicit function of x.
To clarify, equations usually express the relationship explicitly by isolating y completely on one side:
But in this balanced equation, r² = x² + y², y is not isolated, but it still depends on x. Think of y as an inner function. y is effectively a placeholder for some function of x or y(x). Basically, the relationship between x and y is implied by the equation rather than explicitly expressed as y = …
So, I differentiate both sides with respect to x:
Then, I am going to apply the power rule to x² and the chain rule to y(x)². Keep in mind that r is a constant and its derivative is zero.
To solve for the tangent slope,
The slope of this tangent line is defined as rise (Δy) over run (Δx). Think of the run as horizontal steps and the rise as vertical steps. Since the slope is
we can write this in vector notation as:
For example, if the climber stands at (x, y) = (3, 4) on a circle of radius 5, which results in 3² + 4² = 5², the tangent slope is -3/4. This means taking 4 steps to the right and 3 steps down, giving the tangent vector, <4, -3>.
I am going to divide each element of the vector by the circle radius, r, resulting in the unit tangent vector. This is called, normalization.
To normalize the vector, I need to divide the vector by its length. To calculate the length of the vector, I am going to use the Pythagorean theorem. Since the point is on a circular cross-section with radius, r, the length simplifies to r:
Note that the normalized tangent vector matches with the polar derivative of the circle function, but I will not cover that here.
Now I have two vectors, the gradient vector of the function and the unit tangent vector, to validate their perpendicularity. Let me introduce the concept of the dot product.
Dot Product
The dot product of two vectors is a scalar value that shows how much the two vectors point in the same direction. It can be thought of as measuring the shadow one vector casts along the second, then multiplying that shadow by the second vector’s length.
The result of multiplication is positive when the two vectors generally point in the same direction and reaches its maximum when they are parallel, zero when they are perpendicular and negative when they oppose each other.
Mathematically, you express the dot product as:
and in this example, I am going to calculate using the elements of the two vectors:
If I calculate the dot product of the gradient vector of the function and the tangent directional vector, as shown below,
you will see that the result of this operation is zero, which indicates that they are perpendicular. It proves that the gradient vector points directly along the radius toward the center of the circular level curve, representing the steepest path to the peak.
Before You Go …
I decided to simplify some of the more advanced topics and dive deeper into the basics because this is meant for anyone starting out on their differential calculus journey, especially those who, like my kid, need a good algebra and geometry refresher. I hope this walkthrough is helpful and sparks some curiosity along the way!
