Introduction to Vector Spaces

A vector space V over a field F is a collection of vectors that is closed under vector addition and scalar multiplication. These operations satisfy certain axioms that ensure the structure is well-defined and widely applicable in various mathematical and real-world contexts, such as linear algebra, geometry, physics, and computer science.

• Vector Addition - An operation in which two vectors u and v are added to produce a new vector u + v that also belongs to the vector space V (u + v ∈ V).
• Scalar Multiplication - An operation in which a scalar c and a vector v are multiplied to produce a new vector cv that also belongs to the vector space V (cv ∈ V).

Vector Space Axioms

Ten axioms can define a vector space. Let x, y, & z be the elements of the vector space V, and a & b be the elements of the field F.

1. Closed Under Addition

For every element x and y in V, x + y is also in V.

2. Closed Under Scalar Multiplication

For every element x in V and scalar a in F, ax is in V.

3. Commutativity of Addition

For every element x and y in V, x + y = y + x.

4. Associativity of Addition

For every element x, y, and z in V, (x + y) + z = x + (y + z).

5. Existence of the Additive Identity

There exists an element in V which is denoted as 0 such that x + 0 = x, for all x in V.

6. Existence of the Additive Inverse

For every element x in V, there exists another element in V that we can call -x such that x + (-x) = 0.

7. Existence of the Multiplicative Identity

There exists an element in F notated as 1 so that for all x in V, 1x = x.

8. Associativity of Scalar Multiplication

For every element x in V, and for each pair of elements a and b in F, (ab)x = a(bx).

9. Distribution of Elements to Scalars

For every element a in F and every pair of elements x and y in V, a(x + y) = ax + ay.

10. Distribution of Scalars to Elements

For every element x in V, and every pair of elements a and b in F, (a + b)x = ax + bx

Vector Space Examples

Real Numbers (ℝ): The Set of all real numbers forms a vector space under standard addition and scalar multiplication. For example, any two real numbers can be added together (resulting in another real number), and any real number can be multiplied by a scalar (another real number) to give another real number.

Euclidean Space (ℝⁿ): This is the classic n-dimensional vector space where vectors are represented as n-tuples of real numbers. For example, in ℝ³ (3-dimensional Euclidean space), vectors could be defined as (x, y, z), where x, y, and z are real numbers.

Polynomials: A Set of all polynomials with coefficients from a field (like ℝ or ℂ) forms a vector space.

For example, the set of all quadratic polynomials ax² + bx + c, where a, b, and c are real numbers, is a vector space under polynomial addition and scalar multiplication. This set includes the zero polynomial 0 = 0x² + 0x + 0, and for any p(x), 0·p(x) = 0 ∈ P₂, ensuring closure under scalar multiplication.

Matrices: A Set of all matrices of a fixed size (e.g., m x n matrices) with entries from a field forms a vector space. Matrices can be added together element-wise, and scalar multiplication involves multiplying each element of the matrix by a scalar.

Solved Example

Is the set of all 2×2 matrices with real entries a vector space? Explain by verifying if the set satisfies the vector space axioms.

Solution:- Let two Matrices A = \begin{bmatrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{bmatrix} and B = \begin{bmatrix} b_{11} & b_{12} \\ b_{21} & b_{22} \end{bmatrix} then

1. Closure under Addition:

A+B= \begin{bmatrix} a_{11} + b_{11} & a_{12} + b_{12} \\ a_{21} + b_{21} & a_{22} + b_{22} \end{bmatrix}

Since the sum of two 2×2 matrices is also a 2×2 matrix, the set is closed under addition.

2. Closure under Scalar Multiplication:

For a matrix A and scalar c∈R , scalar multiplication results in
cA=\begin{bmatrix} c \cdot a_{11} & c \cdot a_{12} \\ c \cdot a_{21} & c \cdot a_{22} \end{bmatrix}

Since multiplying a 2×2 matrix by a scalar produces another 2×2 matrix, the set is closed under scalar multiplication.

3. Commutativity of Addition

In matrix addition, the dimension does not change and always remains the same, so matrix addition is commutative.

A + B = B + A

4. Associativity of Addition:

For any three matrices A, B, and C, matrix addition is associative,
(A + B) + C = A + (B + C)
This property holds because matrix addition satisfies the associativity property by the definition of matrix addition.

5. Existence of the Additive Identity:

The additive identity in the set of 2×2 matrices is the zero matrix, so the matrix remain same .

6. Existence of the Additive Inverse:

For a matrix A , its additive is, - A
so , A+(-A)= \begin{bmatrix} 0 & 0 \\ 0 & 0 \end{bmatrix}

Thus, every matrix has an additive inverse.

7. Existence of the Multiplicative Identity:

The multiplicative identity in the scalar multiplication of matrices is 1. For any matrix A, multiplying it by 1 results in the original matrix

1⋅A = A, So, the set is closed under scalar multiplication by 1.

8. Associativity of Scalar Multiplication:

For any scalar c,d∈R and matrix A, scalar multiplication is associative
c (dA) = (cd)A

This property holds because scalar multiplication satisfies associativity by the definition of scalar operations on matrices.

9. Distributive Property of Scalar Multiplication over Vector Addition:

For any scalars c and matrices A and B,
c(A + B) = cA + cB

This property is satisfied because matrix addition and scalar multiplication are defined in such a way that this distributive property holds.

10. Distributive Property of Scalar Multiplication over Scalar Addition:

For any scalars c and d and matrix A,
(c + d)A = cA + dA

This property holds due to the distributive property of scalar multiplication with respect to scalar addition.

Since the set of all 2×2 matrices with real entries satisfies all 10 axioms of a vector space, it forms a vector space under matrix addition and scalar multiplication.

Dimension of a Vector Space

Number of vectors in a basis for V is called the dimension of V.

For example, the dimension of Rⁿ is n. The dimension of the vector space of polynomials in x with real coefficients having degree at most two is 3.

Basis of Vector Space

Let V be a subspace of Rⁿ for some n. A collection B = {v₁, v₂, …, v_r} of vectors from V is said to be a basis for V if B is linearly independent and spans V. If either one of these criteria is not satisfied, then the collection is not a basis for V.

If a collection of vectors spans V, then it contains enough vectors so that every vector in V can be written as a linear combination of those in the collection. If the collection is linearly independent, then it doesn't contain so many vectors that some become dependent on the others.

Vector Addition and Scalar Multiplication

Vector addition and scalar multiplication are two main concepts in vector space:

Vector Addition: When you add two vectors, you add their corresponding components.

For example, if you have two vectors v = ⟨v₁, v₂, v₃⟩and w = ⟨w₁, w₂, w₃⟩ their sum v+ w is ⟨v₁+w₁, v₂+w₂, v₃+w₃⟩. Geometrically, vector addition represents the process of moving one vector's endpoint to the other vector's endpoint, forming a new vector from the initial point of the first vector to the final point of the second vector.

Scalar Multiplication: Scalar multiplication involves multiplying a vector by a scalar.

For example, if you have a vector ⟨v = v₁, v₂, v₃⟩ and a scalar k, then the scalar multiple k is ⟨kv₁, kv₂, kv₃⟩. Geometrically, scalar multiplication stretches or compresses the vector without changing its direction, depending on whether the scalar is greater than 1 or between 0 and 1

Linear Combinations and Span

Let v₁, v₂,…, v_r be vectors in Rⁿ. A linear combination of these vectors is any expression of the form

k₁v₁ + k₂v₂ + ......... + k_rV_r

where the coefficients k₁, k₂,…, k_r are scalars.

Vector Space Properties

• Closure under Addition: The the Sum of any two vectors in the vector space is also a vector in the vector space.

• Closure under Scalar Multiplication: Multiplying any vector in the vector space by a scalar yields another vector in the vector space.

• Associativity of Addition: Vector addition is associative, meaning (u + v)+ w = u + (v + w) for all vectors u, v, and w in the vector space.

• Commutativity of Addition: Vector addition is commutative, meaning u + v = v + u for all vectors u and v in the vector space.

• Existence of Additive Identity: There exists a vector, denoted by 0 or 0, called the zero vector, such that u + 0 = u for all vectors u in the vector space.

• Existence of Additive Inverse: For every vector u in the vector space, there exists a vector -u such that u + (-u) = 0.

• Distributive Properties: Scalar multiplication distributes over vector addition, meaning α(u + v) = αu + αv and (α+β)u = αu + βu for all scalars α and β, and vectors u and v in the vector space.

• Multiplicative Identity: Scalar 1 acts as the multiplicative identity, meaning 1⋅u = u for all vectors u in the vector space.

Subspaces

A subset W of a vector space V is called a subspace of V if W is itself a vector space under the addition and scalar multiplication defined on V.

Subspaces are subsets of a vector space that themselves form vector spaces. Operations of vector addition and scalar multiplication from the larger vector space are applicable to the vector space. Subspaces satisfy all axion/properties of a vector space.

• Contain the zero vector.
• Is closed under addition
• Is closed under scalar multiplication

They can be lower-dimensional spaces within the larger vector space and can provide insights into the structure and properties of the vector space as a whole.

Vector Space vs Euclidean Space

Applications of Vector Spaces

Vector Spaces are also used in Machine Learning, and their various other uses are:

• Data Representation – Represent images, text, and numerical datasets as vectors in an n-dimensional space.
• Machine Learning & Data Analysis – Use PCA, SVD, and similar methods for dimensionality reduction, clustering, and pattern discovery.
• Feature Vectors – Create engineered feature representations where each dimension corresponds to a measurable attribute for model input.
• Vector Operations in Algorithms – Perform addition, subtraction, dot products, and norms for tasks like similarity search or centroid computation.
• Search Engines – Represent documents as vectors in high-dimensional word space to measure similarity and relevance.

Practice Problem

Question 1. For scalars α = 3, β = 4, and vectors u = (1, 2) and v = (3, 4), verify the following properties of scalar multiplication
i) α(u + v) = αu + αv and
ii) (α+β)u = αu + βu.

Question 2. Let V = {(x, y) ∈ R²∣x ≥ 0,y ≥ 0} be the set of vectors in the first quadrant. Is this set closed under addition?

Question 3. Let v = (3, −4) be an element of R². Find the additive inverse of v and verify that v + (−v) = 0, where 0 is the zero vector in R².

Question 4. Consider the set V =\left\{ \begin{bmatrix} a & b \\ c & d \end{bmatrix} \mid a, b, c, d \in \mathbb{R} \right\} of all 2 × 2 matrices with real entries. Is this set a vector space?