Hessian, Jacobian & Taylor: The OGs of Mathematics

Hessian for dummies

The Hessian is like a roadmap that helps us understand how a function changes in different directions.

Get Anirudh Singh’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Think of a landscape with hills and valleys. If you’re standing at a point on this landscape:

1. The slope (gradient) tells you which direction is steepest uphill or downhill. It’s like saying, “If I take a step this way, will I go up or down?”
2. The Hessian gives more detailed information: it tells you how the slope itself changes in every direction. It’s like asking, “If I take a step in this direction, will the ground keep getting steeper, or will it flatten out?”

Mathematically speaking

The Hessian is a square matrix (a grid of numbers) that collects the second derivatives of a function. A second derivative is just a way of measuring how quickly something is changing — like asking, “Is the slope getting steeper or less steep?”

For example:

• If the Hessian shows all positive values, you’re at the bottom of a valley (a minimum point).
• If it shows all negative values, you’re at the top of a hill (a maximum point).
• If it’s a mix, you’re at a saddle point — a spot where the land curves up in one direction but down in another.

The Hessian helps us:

• Predict how a system behaves, like finding the best (or worst) outcome in optimization problems.
• Understand the stability of solutions in engineering, physics, and machine learning.

In short, the Hessian is a mathematical tool to analyze the “shape” of a function, especially when things get multidimensional or complex.

Jacobian for Dummies

• The Jacobian is like a multi-directional compass that tells you how something is changing in many directions at the same time. Let’s simplify this: Think of a rubber sheet with a grid drawn on it. Now, stretch or squash the sheet in different ways.
• At each point on the sheet, the grid might stretch more in one direction, less in another, or even twist.
• The Jacobian is the mathematical tool that describes how the grid at a specific point is being transformed — stretched, squashed, or rotated.

Mathematically Speaking

The Jacobian is a matrix (a table of numbers) that captures how each input affects each output for a function. It measures the rate of change in a system with multiple inputs and outputs.

• If you have one input and one output: this is just the derivative.
• If you have multiple inputs and outputs, the Jacobian is a matrix showing how each input influences each output.

Example: A Fan Blowing Air

Imagine a fan blowing air into a room:

• Inputs: The speed and angle of the fan.
• Outputs: The airflow’s direction and force at a specific spot.

The Jacobian would tell you:

• How changing the fan’s speed affects the airflow’s direction.
• How adjusting the angle of the fan changes the force of the air.

The matrix summarizes all these relationships.

How is it Useful?

1. Robot Movements: Robots with arms and joints use the Jacobian to calculate how moving one joint affects the overall position of the hand.
2. Transformations: If you’re mapping a shape from one coordinate system to another, the Jacobian shows how the shape gets stretched or compressed.
3. Optimization: In machine learning, it helps understand how tweaking inputs (like model parameters) affects the output predictions.

The Jacobian is like a control panel for a system, showing all the knobs you can turn and how they influence the system’s behavior.

Taylor (Series) for dummies

The Taylor series is like a recipe for building a function using just its derivatives (rates of change) at a single point. Imagine Drawing a Curvy Path:

1. You’re trying to draw a smooth curve (like a hill or a wavy road) but you only know how it behaves at one spot.
2. The Taylor series helps you figure out the rest of the curve by adding one piece at a time:

• Start with the point you know.
• Add a straight line to show the slope (first derivative).
• Add a bit of curve to show how the slope changes (second derivative).
• Keep adding more corrections to make it more accurate.

The more pieces you add, the closer you get to the real curve. In Plain Words the Taylor series is like a puzzle where you:

• Use the starting value of the function (what it equals at one point).
• Use the slope (how fast it’s changing).
• Use the curvature (how it bends).
• Add even finer details the more pieces you include.

This looks like:

A Real-Life Analogy

Imagine you’re tasting a soup:

1. The first spoonful gives you a basic idea of its flavor (like the first term in the Taylor series).
2. Then, you notice the spices and seasonings (like the slope and curvature).
3. The more you taste, the better you understand all the subtle flavors (higher-order terms).

Why Use It?

1. Quick Predictions: You can approximate a complicated function near a point using just a few terms.
2. Engineering & Physics: It’s used to simplify and solve real-world problems, like calculating orbits or designing machines.
3. Computers: Helps computers break down tough problems into simpler calculations.

In short, the Taylor series is a tool for turning complex functions into easier, bite-sized chunks you can work with.