Understanding Matrix Multiplication Errors in TensorFlow: A Beginner’s Guide

Introduction

TensorFlow is one of the most popular open-source frameworks for building and training machine learning and deep learning models. It provides powerful mathematical operations that allow developers to perform computations efficiently using tensors.

However, when working with TensorFlow, one common challenge beginners encounter is matrix dimension mismatch, particularly when using the tf.matmul() operation.

In this article, we will explore a simple TensorFlow program that demonstrates this problem, understand why the error occurs, and learn how to fix it.

1. Understanding the Program

Let’s begin by examining the original Python code:

python

import tensorflow as tf
import numpy as np

data = np.array([0.1, 0.2])
x = tf.placeholder("float", shape=[2])
T1 = tf.Variable(tf.ones([2, 2]))
l1 = tf.matmul(T1, x)
init = tf.initialize_all_variables()
with tf.Session() as sess:
    sess.run(init)
    print(sess.run(l1, feed_dict={x: data}))

The intention of this program is to multiply a 2 × 2 matrix by a vector containing two elements.

However, the code encounters a dimensionality issue because tf.matmul() expects both inputs to be matrices (2-D tensors), whereas x is defined as a one-dimensional tensor.

Let’s examine each component.

2. Understanding the Input Data

First, we define a NumPy array:

python

data = np.array([0.1, 0.2])

This array contains two values:

text

[0.1, 0.2]

Its shape is:

python

(2,)

This means that data is a one-dimensional vector with two elements.

Next, we create a TensorFlow placeholder:

python

x = tf.placeholder("float", shape=[2])

The placeholder represents an input tensor that will receive data when the TensorFlow session executes.

Its shape is [2], meaning it is a one-dimensional tensor containing two elements.

3. Understanding the Matrix Variable

Next, we define a TensorFlow variable:

python

T1 = tf.Variable(tf.ones([2, 2]))

The tf.ones() function creates a matrix containing only ones.

Since its shape is [2, 2], the resulting matrix is:

T1=[1111]

This is a two-dimensional tensor with two rows and two columns.

4. Why Does the Error Occur?

The main problem appears in this line:

python

l1 = tf.matmul(T1, x)

The tf.matmul() operation performs matrix multiplication. It expects both input tensors to have two dimensions.

However, our inputs have different shapes:

• T1: Shape (2, 2) — a two-dimensional matrix.
• x: Shape (2,) — a one-dimensional vector.

TensorFlow does not automatically convert a one-dimensional vector into a two-dimensional matrix when using tf.matmul().

Therefore, the operation produces a dimension-related error.

5. How to Fix the Problem

There are two simple ways to solve this issue, depending on the intended mathematical operation.

Solution 1: Reshape the Vector into a Column Matrix

If we want to multiply a 2 × 2 matrix by a column vector, we can reshape x into a 2 × 1 matrix.

The corrected code is:

Get Prathik Karthikeyan’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

python

import tensorflow as tf
import numpy as np

data = np.array([0.1, 0.2])
x = tf.placeholder("float", shape=[2])
T1 = tf.Variable(tf.ones([2, 2]))
x_reshaped = tf.reshape(x, [2, 1])
l1 = tf.matmul(T1, x_reshaped)
init = tf.global_variables_initializer()
with tf.Session() as sess:
    sess.run(init)
    result = sess.run(
        l1,
        feed_dict={x: data}
    )
    print(result)

Now the dimensions are compatible:

(2×2)×(2×1)

The multiplication produces a 2 × 1 matrix.

The expected output is:

text

[[0.3]
 [0.3]]

Solution 2: Expand the Dimensions of the Vector

Another approach is to use tf.expand_dims():

python

x_reshaped = tf.expand_dims(x, axis=1)

l1 = tf.matmul(T1, x_reshaped)

This operation adds an additional dimension to the original vector, converting its shape from (2,) to (2, 1).

The resulting matrix multiplication works correctly.

6. Understanding Matrix Multiplication

An important rule of matrix multiplication is that the number of columns in the first matrix must equal the number of rows in the second matrix.

For example:

A(2×2) * B(2×1)

is valid because the inner dimensions match.

The result has the outer dimensions:

C(2×1)

This rule is essential when working with TensorFlow because tensor dimensions determine whether mathematical operations can be performed.

7. A Note About TensorFlow Versions

The original program uses TensorFlow 1.x syntax, including:

• tf.placeholder()
• tf.Session()
• tf.initialize_all_variables()

These APIs belong to TensorFlow’s older graph-based execution model.

In TensorFlow 2.x, eager execution is enabled by default, and the recommended approach is to use tf.Tensor objects directly.

Here is an equivalent TensorFlow 2.x implementation:

python

import tensorflow as tf

data = tf.constant([[0.1], [0.2]], dtype=tf.float32)
T1 = tf.Variable(tf.ones([2, 2]))
result = tf.matmul(T1, data)
print(result.numpy())

Output:

text

[[0.3]
 [0.3]]

This version is shorter and does not require placeholders or sessions.

Conclusion

Matrix dimension mismatch is a common issue when performing mathematical operations in TensorFlow. The example in this article demonstrates why multiplying a two-dimensional matrix by a one-dimensional vector using tf.matmul() causes an error.

The key lessons are:

1. Understand the shape of every tensor before performing mathematical operations.
2. Remember that tf.matmul() requires compatible matrix dimensions.
3. Use tf.reshape() or tf.expand_dims() to convert a vector into a matrix when necessary.
4. Check the inner dimensions of matrices before performing multiplication.
5. Remember that TensorFlow 1.x and TensorFlow 2.x use different programming approaches.

Understanding tensor shapes and matrix multiplication is an important foundation for learning neural networks, deep learning, and machine learning with TensorFlow.