Understanding Tensors: The Mathematical Language of AI

Artificial Intelligence (AI) has become part of our everyday lives. From recommendation systems and voice assistants to self-driving cars and medical image analysis, AI is changing how we interact with technology. Behind many of these breakthroughs is a mathematical concept that every machine learning practitioner eventually encounters: the tensor.

If you’re just getting started with deep learning, you’ve probably seen the word tensor in frameworks like TensorFlow or PyTorch. At first, it might seem like another intimidating mathematical term. In reality, it’s much simpler than it sounds. A tensor is simply a way of organizing numerical data so computers can process it efficiently. Once you understand tensors, many concepts in machine learning become much easier to follow.

So, What Is a Tensor?

A tensor is a mathematical structure used to represent data with one or more dimensions. You can think of it as a container that holds numbers in an organized way.

The easiest way to understand tensors is to start with familiar objects.

A single number is called a scalar, which is a 0-dimensional tensor.

5

A list of numbers is called a vector, or a 1-dimensional tensor.

[2, 5, 8]

When numbers are arranged in rows and columns, they form a matrix, which is a 2-dimensional tensor.

[
  [1, 2, 3],
  [4, 5, 6]
]

If you stack several matrices together, you get a 3-dimensional tensor. You can imagine this as a collection of pages in a book, where each page represents one matrix.

Rather than treating scalars, vectors, and matrices as separate concepts, mathematics views them as different forms of tensors with different numbers of dimensions.

Why Do We Need Tensors?

Almost every type of information used in AI can be represented as numbers. Images, videos, audio recordings, text, financial data, and sensor measurements all need to be converted into a numerical format before a machine learning model can process them.

Tensors provide a universal way to store that information.

Instead of creating different data structures for different types of data, deep learning libraries represent everything as tensors. This makes it possible to use the same mathematical operations regardless of whether the model is processing an image, a sentence, or a time series.

For example:

• A photograph is stored as a tensor.
• A sentence is converted into a tensor of numerical values.
• Audio signals become tensors.
• Stock market data is represented as tensors over time.

This consistency is one of the reasons modern deep learning frameworks are so powerful.

Understanding Tensor Dimensions

Every tensor has a rank, which simply refers to the number of dimensions it has.

A scalar has no dimensions.

42

Shape:

()

A vector has one dimension.

[4, 8, 15, 16]

Shape:

(4)

A matrix has two dimensions.

[
  [1, 2, 3],
  [4, 5, 6]
]

Shape:

(2, 3)

A common example of a 3-dimensional tensor is a color image. Since most digital images use three color channels — red, green, and blue — their shape is typically written as:

(Height, Width, Channels)

For an image with a resolution of 224 × 224 pixels, the tensor shape would be:

(224, 224, 3)

As models become more complex, tensors with four or even five dimensions become common, especially when processing multiple samples at once.

Tensors in Deep Learning

Deep learning models rarely process one example at a time. Instead, they usually work with batches of data because modern GPUs can perform many calculations in parallel.

Get M.Rasoul Zahmatkesh’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Imagine training an image classification model using a batch of 32 RGB images. Each image has a size of 224 × 224 pixels and contains three color channels.

The resulting tensor would have the following shape:

(32, 224, 224, 3)

Here:

• 32 represents the number of images.
• 224 is the image height.
• 224 is the image width.
• 3 represents the RGB color channels.

This batch-based representation allows deep learning frameworks to train models much faster than processing each image individually.

Common Tensor Operations

Neural networks work by repeatedly applying mathematical operations to tensors. During training, millions — or even billions — of these operations are performed every second.

Some of the most frequently used operations include:

• Addition
• Subtraction
• Element-wise multiplication
• Matrix multiplication
• Reshaping
• Transposing
• Concatenation
• Slicing
• Broadcasting

Every layer in a neural network receives one or more tensors, transforms them mathematically, and produces new tensors that are passed to the next layer.

Why Is TensorFlow Called TensorFlow?

The name TensorFlow reflects exactly how the framework works.

The word Tensor refers to the multidimensional data structures used throughout the model, while Flow describes how those tensors move through a computational graph as different mathematical operations are applied.

Although TensorFlow introduced many developers to the concept, tensors themselves have been used in mathematics and physics for decades. Today, they form the foundation of virtually every major deep learning framework, including PyTorch, TensorFlow, JAX, and MXNet.

Creating a Tensor in Python

In PyTorch, creating a tensor is straightforward.

import torch

x = torch.tensor([
    [1, 2],
    [3, 4]
])
print(x)
print(x.shape)

Output:

tensor([[1, 2],
        [3, 4]])

torch.Size([2, 2])

TensorFlow offers a very similar interface.

import tensorflow as tf

x = tf.constant([
    [1, 2],
    [3, 4]
])
print(x)

Although the syntax differs slightly, both libraries use tensors as their primary data structure.

Where Are Tensors Used?

Once you begin looking for them, you’ll realize tensors appear in nearly every AI application.

In computer vision, images and videos are represented as tensors. In natural language processing, words and sentences are converted into numerical tensors before being processed by language models. Speech recognition systems transform audio signals into tensors, while medical imaging applications analyze MRI and CT scans stored as high-dimensional tensors.

Even financial forecasting systems use tensors to represent historical market data, and autonomous vehicles rely on tensor-based sensor information from cameras, LiDAR, and radar to understand their surroundings in real time.

Final Thoughts

Tensors are one of the most important concepts in artificial intelligence because they provide a consistent way to represent data of almost any kind. Whether you’re working with text, images, audio, or time-series data, the information is ultimately converted into tensors before it reaches a machine learning model.

The good news is that you don’t need an advanced mathematics background to start using tensors. If you’re comfortable with numbers, lists, and matrices, you’re already familiar with the basic ideas. As you continue learning deep learning, tensors will quickly become a natural part of your workflow. Understanding tensors is one of the first milestones on the journey toward building and understanding modern AI systems.