Can We Watch a Neural Network Think ??

Visualizing What Happens Inside a Neural Network, Layer by Layer

We usually see a neural network like this:

Image → Model → Prediction

An image goes in, a prediction comes out, and somewhere in the middle there is a giant mathematical black box.

But what if we could actually look inside?

What happens to an image as it moves through a neural network?

Does the network first see edges?
Then textures?
Then shapes?
And eventually something that looks like a recognizable object?

I wanted to find out.

So I built a small experiment to watch a pretrained neural network process an image layer by layer.

The Experiment

For this experiment, I used:

• Python
• PyTorch
• Torchvision
• ResNet18
• ImageNet pretrained weights
• PyTorch forward hooks
• Matplotlib

The model was given an image of a toucan.

The final prediction was:

Toucan → 99.92% confidence

But the prediction itself wasn’t the interesting part.

The interesting part was what happened before the prediction.

Looking Inside the Network

Instead of only looking at the final output, I captured intermediate feature maps from different parts of ResNet18.

The experiment focused on three stages:

Input Image
     ↓
Conv1
     ↓
Layer 2
     ↓
Layer 4
     ↓
Prediction

Each stage represents a different depth inside the network.

And the deeper we go, the representation becomes more abstract.

What Does the Network See First?

The first convolutional layer, Conv1, operates very close to the original image.

At this stage, the network has not yet built complex concepts.

Instead, early convolutional filters can respond strongly to basic visual structures such as:

• Edges
• Lines
• Brightness changes
• Simple textures
• Basic orientations

Think of this as the network learning the visual building blocks of the image.

Going Deeper

After the early layers, the network starts combining simpler features into more complex patterns.

By the time we reach deeper layers, the spatial resolution has decreased while the number of feature channels has increased.

For ResNet18, our captured representations look like this:

Conv1
64 channels
112 × 112

Layer 2
128 channels
28 × 28
Layer 4
512 channels
7 × 7

Something interesting is happening here.

The network is losing some spatial detail while gaining a much richer collection of learned feature representations.

The Full Visualization

This is the main visual from the experiment:

The left side starts with the original image.

Get Hasher Amin’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Moving deeper into the network, we are no longer looking at the image itself.

We are looking at the network’s internal representations of that image.

This is one of the reasons convolutional neural networks are so powerful.

They don’t simply memorize pixels.

They progressively transform the input into representations that are more useful for the final task.

From Pixels to Patterns

A simplified way to think about the process is:

Pixels
  ↓
Edges
  ↓
Textures
  ↓
Shapes
  ↓
Complex patterns
  ↓
Object representation
  ↓
Prediction

This isn’t a strict rule where every individual layer has exactly one job.

Real neural networks are much more complicated than this.

But it is a useful mental model for understanding hierarchical feature learning.

So, Did We Actually Watch It Think?

Not exactly.

A neural network doesn’t “think” in the human sense.

What we actually visualized were intermediate activations produced as the input moved through the network.

But this gives us something extremely valuable:

A way to make part of the model’s internal computation visible.

Instead of treating the network as:

Input → Black Box → Output

we can start thinking about it as:

Input
  ↓
Transformation
  ↓
Transformation
  ↓
Transformation
  ↓
Representation
  ↓
Prediction

That shift in perspective is important when working with deep learning.

Why This Matters

Feature visualization is more than just making cool images.

It can help us:

• Understand learned representations
• Debug neural networks
• Investigate model behavior
• Explain CNN architectures
• Explore why certain inputs produce certain predictions
• Build intuition about deep learning

It also reminds us that the final prediction is only the end of a much larger process.

The Interesting Part

The most interesting thing I learned from this experiment wasn’t that ResNet18 could classify a toucan with 99.92% confidence.

It was seeing how different the representation becomes as we move deeper into the network.

The original image is something humans can immediately understand.

The feature maps are different.

They don’t look like “a toucan” to us.

Yet these representations are exactly what the network uses to eventually produce the prediction:

Toucan.

That gap between what humans see and what neural networks represent is one of the most fascinating parts of deep learning.

Final Thought

We often say:

“The model learned the image.”

But that’s an oversimplification.

The model learned a hierarchy of representations.

And by extracting intermediate activations, we can get a small window into that process.

We may not be literally watching a neural network think.

But we can certainly make some of its hidden transformations visible.

And sometimes, that’s enough to turn a black box into something we can start to understand.

What’s Next?

This experiment can be pushed much further.

Instead of manually selecting a few layers, we could visualize the network as a continuous pipeline and create an animation showing the image moving through the network.

That would make the question even more interesting:

What would a neural network look like while it is processing an image?