Understanding Vector Norms. L0, L1, L2, and Beyond…
The Mathematics of Size and Distance.
The term “norm” in mathematics comes from the concept of “normal” or “standard” way of measuring the size or length of objects in a vector space. Conceptually, a norm provides a way to quantify the “magnitude” or “length” of a vector, giving a consistent measure of its size. This is analogous to the way we measure the length of a physical object using a ruler.
A norm provides a standardized way to measure vectors. Regardless of the vector’s direction or the specific components, the norm offers a consistent measure of its magnitude. This standardization is crucial for comparing vectors and performing various mathematical operations.
To be a norm, a function must satisfy specific properties that align with our intuitive understanding of “length” or “size”:
- Non-negativity: The norm of a vector is always non-negative. ∥x∥ ≥ 0
- Definiteness: The norm of a vector is zero if and only if the vector is the zero vector. ∥x∥=0 ⟺ x=0
- Scalability: Scaling a vector by a positive constant scales the norm by the same constant. ∥αx∥=∣α∣∥x∥ for any scalar α
- Triangle Inequality: The norm of the sum of two vectors is less than or equal to the sum of the norms. ∥x+y∥ ≤ ∥x∥+∥y∥
These properties ensure that norms behave in a predictable and intuitive way, much like how we expect measurements of physical objects to behave.
Norms generalize the concept of length from Euclidean space to more abstract vector spaces. In Euclidean space, the length of a vector is given by the Euclidean norm (L2 norm). Norms extend this idea to other spaces and contexts, allowing us to measure sizes in a broader range of mathematical settings.
Norms are functions that assign a non-negative length or size to vectors in a vector space.
Types Of Norms
L0 Norm
The L0 norm, also known as the “zero norm,” is defined as the number of non-zero elements in a vector. Despite its name, it is not actually a norm in the strict mathematical sense because it does not satisfy certain properties like the triangle inequality. However, it is widely used in areas such as compressed sensing and sparse coding.
Mathematically:
L1 Norm
The L1 norm, also known as the “Manhattan norm” or “taxicab norm”, is defined as the sum of the absolute values of the vector components. It is commonly used in various optimization problems, including those in machine learning and statistics.
Get MANOJ PILLAI’s stories in your inbox
Join Medium for free to get updates from this writer.
Mathematically:
L2 Norm
The L2 norm, also known as the “Euclidean norm,” is the most common norm. It is defined as the square root of the sum of the squares of the vector components. This norm corresponds to the usual notion of distance in Euclidean space.
Mathematically:
Lp Norm
The Lp norm is a generalization of the L1 and L2 norms. It is defined for any real number p ≥ 1 and is given by:
Special cases include:
- L1 norm when p = 1
- L2 norm when p = 2
- L∞ norm when p → ∞
numpy.linalg.norm() calculates the Lp Norm with L2 being the default.
import numpy as np
# Calculate the L2 norm (default)
l2_norm = np.linalg.norm(v)
print(l2_norm)
# Calculate the L1 norm
l1_norm = np.linalg.norm(v, ord=1)
print(l1_norm)L∞ Norm
The L∞ norm, also known as the “maximum norm” or “Chebyshev norm,” is defined as the maximum absolute value of the vector components.
Mathematically:
Examples
Let’s consider a vector x=[3,−4,0,5] and compute its norms.
- L0 norm: ‖x‖₀ = 3 (since there are three non-zero elements: 3, -4, and 5)
- L1 norm: ‖x‖₁ = |3| + |-4| + |0| + |5| = 3 + 4 + 0 + 5 = 12
- L2 norm: ‖x‖₂ = √(3² + (-4)² + 0² + 5²) = √(9 + 16 + 0 + 25) = √(50) ≈ 7.07
- L∞ norm: ‖x‖∞ = max(|3|, |-4|, |0|, |5|) = 5
