Pattern Recognition Part 4: Neural Networks Step by Step

How neural networks learn patterns, make predictions, and apply to everyday recognition tasks.

It’s Monday morning again. Maya appears on the customer retention list. In Part 3, we followed a decision tree through her inactivity and unresolved support ticket to a churn prediction.

Now suppose we keep the same question: which customers will cancel within the next 30 days? This time, we’ll train a neural network to learn from the customer snapshots. We’ll follow its calculations, see how it learns, and ask what its prediction means for Maya.

The idea we’re building towards is simple: a neural network makes a prediction, measures its mistake, and adjusts the numbers inside it to improve future predictions.

The subscription business and the numbers below are fictional learning examples. We’ll keep the calculations small enough to follow one step at a time.

How neural networks fit into pattern recognition

Infographic showing how neural networks learn patterns and make predictions.

Our series has moved from recognising useful patterns in Part 1, to preparing data and evaluating predictions in Part 2, to learning branching rules in Part 3. Neural networks give us another way to learn from the same kinds of observations.

A neural network is one method we can use to learn those regularities.

For example, a recognition task might ask whether a payment looks fraudulent, which digit appears in an image, or which command someone spoke.

Using a neural network for pattern recognition means training it to turn observations into a useful prediction about what those observations represent.

We choose the inputs, the outcome we want to predict, and the training objective. The network learns weights and biases that help connect them.

For the supervised examples in this article, we provide labelled examples during training. Later, the trained network makes predictions on new inputs without being given their answers.

The goal is to recognise useful relationships across different examples. Seeing the same training example again is not enough.

Let’s return to our subscription business and build the calculation from one neuron upwards.

Step 1 Give the network a prediction task

Our target stays the same as in Part 3: whether a customer cancels within the next 30 days.

To keep the arithmetic simple, we’ll begin with two inputs for each customer snapshot:

• Days since the customer’s last login.
• Whether the customer has an unresolved support ticket, encoded as 1 for yes and 0 for no.

After observing the full 30-day outcome, we record 1 for churned and 0 for stayed. All inputs must come from information available at the prediction date.

The login inactivity and ticket indicator are our features. Churn within 30 days is our target, or label. During training, we show the network many examples containing both the inputs and the correct answer.

This is called supervised learning.

Eventually, we want to predict outcomes for later customer snapshots whose labels the model hasn’t used for training.

We could include tenure and recent plan changes, as in Part 3. Two inputs are enough to show how a network works. This is a teaching example, not a claim that two features would make a reliable churn model.

A tree learns questions and branches that route a customer to a leaf. Our network will learn numerical transformations that combine the inputs into a prediction. Both can learn interactions between features; they express them differently.

The target and evaluation should stay consistent when we compare them. A neural network does not automatically improve on the tree simply because it has more adjustable numbers.

Step 2 Start with one artificial neuron

A visual breakdown of how a single artificial neuron calculates a score by multiplying customer inputs by learned weights and adding a bias.

A neuron is a small calculation unit.

It takes input numbers, multiplies each one by a weight, adds the results together, and adds another number called a bias.

For our two inputs, that looks like this:

score = scaled inactivity × weight₁ + ticket indicator × weight₂ + bias

A weight controls how an input contributes to this particular neuron. A positive weight pushes its score up as that input increases. A negative weight pushes it down.

The bias shifts the score even when the inputs stay the same.

As we saw in Part 2, scaling helps control the numerical ranges of our inputs. We’ll scale days since login using values learned from training data. The ticket indicator is already 0 or 1. Scaling can help neural network optimisation even though the tree in Part 3 generally did not need it.

Maya has been inactive for 19 days and has an unresolved ticket. Suppose, for this illustrative calculation, the training minimum and maximum inactivity values are 3 and 23 days. Min-max scaling gives (19 − 3) / (23 − 3) = 0.8. Let’s use:

• Scaled inactivity: 0.8.
• Unresolved ticket indicator: 1.
• Inactivity weight: 0.5.
• Ticket weight: 0.24.
• Bias: −0.3.

The calculation becomes:

score = (0.8 × 0.5) + (1 × 0.24) − 0.3
score = 0.34

That’s our first calculation done.

These weights are just illustrative starting values. During training, the network learns better values from the examples.

Step 3 Apply an activation function

A visual explanation of how a neuron’s raw score is transformed by activation functions, using Sigmoid for final probability predictions and ReLU for hidden layers to capture complex patterns.

After calculating the score, the neuron passes it through an activation function.

For an output that predicts between two choices, a common option is sigmoid. It converts any score into a number between 0 and 1.

Our score of 0.34 becomes approximately 0.58.

We can read that as an estimated 58% probability of churn within 30 days. As in the earlier parts, the numerical probability needs calibration checks before we rely on it for decisions.

With a decision threshold of 0.5, this would flag Maya as a predicted churner. We would choose the operating threshold using validation data and the consequences of acting on the prediction.

If this calculation looks familiar, it should. Part 2 introduced logistic regression as a weighted score followed by sigmoid. A single sigmoid output neuron connected directly to the inputs has that same model form.

The next step is to add hidden neurons with nonlinear activations. They let the network learn a more flexible relationship between the inputs and the final score.

Hidden neurons often use a different activation function called ReLU:

ReLU(score) = max(0, score)

It keeps positive scores and replaces negative scores with zero.

For example, ReLU turns a score of −0.7 into 0. A score of 0 stays at 0, while a positive score such as 0.34 stays at 0.34.

Why do we need these functions?

Nonlinear activation functions let a network learn more complex relationships. If we stacked layers using only linear calculations, the whole stack would still collapse into one linear calculation.

So adding layers alone isn’t enough. The activation functions matter too.

Step 4 Connect neurons into layers

A diagram of a Multilayer Perceptron (MLP) showing how an input layer connects to a hidden layer of three neurons, which then combine their learned patterns into a single output neuron to predict churn probability.

Now let’s move from one neuron to a small network.

We’ll give it two input values, one hidden layer containing three neurons, and one output neuron.

The input layer holds our scaled inactivity and ticket indicator.

The hidden layer calculates three learned combinations of those inputs.

The output layer uses those results to produce the estimated probability of churn within 30 days.

Each hidden neuron has its own weights and bias. Even though the neurons receive the same inputs, they can produce different outputs.

The output neuron then combines those hidden outputs to make the final prediction.

We call the middle layer “hidden” because the data doesn’t provide correct answers for its individual neurons. We only provide the final target: churned or stayed.

We also don’t assign each hidden neuron a specific job. The useful combinations emerge during training.

This type of network is commonly called a multilayer perceptron, or MLP. In the fully connected version we’re using, every neuron in one layer connects to every neuron in the next.

We’ll stay with this small network. It gives us everything we need to understand the learning process.

What the hidden layers learn to recognise

How hidden layers automatically extract features to form learned representations, along with a warning about “shortcut bias” — where a network might mistakenly learn misleading background patterns instead of the true features.

This connects to the distinction between feature selection and feature extraction in Part 2.

We still choose the initial inputs, such as inactivity and unresolved tickets. The hidden layer then learns combinations of those features that help predict churn. These combinations form a learned representation.

With images, a network can learn useful representations from pixel values. In some image networks, earlier layers respond to simple visual patterns, while later layers combine them into more complex features.

This is part of the appeal of neural networks for pattern recognition: learning a useful representation can happen alongside learning the final prediction.

Remember the dark inspection mat in Part 2? A neural network can learn that misleading background shortcut too. Learning features automatically does not guarantee that it learns the right ones.

Step 5 Make a prediction with a forward pass

Performing a “forward pass” in a neural network, showing how data moves from left to right through inputs, weighted hidden layer calculations, activation functions, and output layers to produce a final churn prediction of 0.58.

A forward pass means calculating the output by moving through the network from inputs to prediction.

For our churn example:

1. Supply scaled inactivity and the ticket indicator.
2. Calculate each hidden neuron’s weighted score.
3. Apply its activation function.
4. Pass those hidden outputs to the output neuron.
5. Apply sigmoid to get the final probability.

That’s all a forward pass does: calculate a prediction using the current weights and biases.

The 0.58 earlier came from our single-neuron calculation. Adding a hidden layer changes the computation, so it does not automatically preserve that output. We’ll use 0.58 again below as an illustrative network prediction to explain the loss.

At the beginning of training, weights are typically initialized randomly using a suitable initialization method. The first predictions may be poor.

Get Jarebb Ng’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

The forward pass alone doesn’t improve them. For that, we need to measure the mistake.

Step 6 Measure the mistake with a loss function

Step 6 of a neural network process, showing how a loss function compares a prediction to the actual outcome to guide training adjustments.

Suppose our small network produces a churn estimate of 0.58 for a historical customer snapshot. After the full 30-day window, we know the customer cancelled.

The target is 1. The prediction points towards the right choice, but the model still assigns a substantial probability to the other choice.

A loss function turns the prediction and target into a number that measures how wrong the prediction was.

For this two-choice task, a common loss is binary cross-entropy.

We don’t need its formula to understand the idea. When the true answer is 1, a prediction closer to 1 receives a lower loss. A prediction confidently close to 0 receives a much higher loss.

This also explains why loss and accuracy are different.

With a 0.5 threshold, predictions of 0.58 and 0.95 both count as correct for this snapshot. Their losses differ because they assign different probabilities to the correct answer.

Training uses the loss to guide adjustments.

Step 7 Work backwards with backpropagation

Step 7 of a neural network process, detailing how backpropagation works backwards from the loss function to calculate gradients for weights and biases.

We now know how wrong the prediction was. Next, we need to work out how to adjust the network.

That’s where backpropagation comes in.

It works backwards from the loss through the network, calculating how sensitive that loss is to each weight and bias. These sensitivities are called gradients.

Think of each weight as an adjustable dial. A gradient tells us how a small turn of that dial would affect the loss near its current setting.

This calculation reaches the hidden layers too. Even though we never supplied target values for the hidden neurons, we can calculate how their parameters affect the final loss.

There’s one distinction that helped me make sense of the process:

Backpropagation calculates the gradients. An optimizer uses those gradients to update the weights and biases.

Step 8 Update the weights with gradient descent

Step 8 of a neural network process, explaining how the gradient descent optimizer updates weights using a learning rate.

A simple optimizer is gradient descent.

Its update rule is:

new weight = old weight − learning rate × gradient

The learning rate controls the size of the adjustment.

Let’s use another made-up example. Suppose a weight is 0.5, its gradient is 0.2, and the learning rate is 0.1.

new weight = 0.5 − (0.1 × 0.2) = 0.48

The weight moves slightly down. A negative gradient would move it up instead. Biases get updated in the same way.

If the learning rate is too small, training can take a long time. If it’s too large, updates can overshoot and make training unstable.

The aim is to make repeated useful adjustments. A single update doesn’t guarantee that every prediction will improve.

Step 9 Repeat the training loop

Step 9 of a neural network process, visualizing the repetitive training loop and key training terminology such as mini-batch and epoch.

Now we can put the pieces together:

1. Make predictions with a forward pass.
2. Calculate the loss using the known targets.
3. Use backpropagation to calculate gradients.
4. Use the optimizer to update weights and biases.
5. Repeat.

In practice, we often process a small group of examples at a time. This group is a mini-batch, and its combined loss guides an update.

One complete pass through the training dataset is called an epoch.

If we have 1,000 examples and a batch size of 100, one epoch contains 10 batches and usually 10 updates.

Over multiple epochs, the network gets repeated opportunities to improve. But how do we tell whether it has learned something useful?

Step 10 Check whether it works on new examples

Step 10 of neural network training, detailing how to check model performance on new examples to prevent overfitting using three datasets and early stopping.

A network can become very good at its training examples and still perform poorly on new ones. This is called overfitting.

For our churn model, the useful question is whether it works on later customer snapshots, including cases that differ from the training examples.

We keep the same roles for the three datasets introduced in Part 2:

• Training set: The examples the network uses to learn its weights and biases.
• Validation set: Separate examples that help us choose the network size and decide when to stop training.
• Test set: Examples we keep aside to evaluate the chosen model after those decisions are finished.

We split the data before fitting preprocessing steps. For example, we learn scaling values from the training set and apply those same values to validation and test data.

This keeps information from the held-out examples out of training.

One practical way to limit overfitting is early stopping. We monitor validation loss and stop when it stops improving for a chosen number of checks, often keeping the best-performing weights.

We can also try a smaller network or regularization, which discourages overly complex solutions.

Adding more neurons, more layers, or more training epochs doesn’t automatically give us a better model.

For this business, we would train on earlier snapshots and evaluate on later ones, allowing enough time for the 30-day labels to become known. Repeated snapshots of the same customer need a split design that matches the intended use, just as in Part 3.

We would compare the network with the tree and a simple baseline under the same evaluation setup. Precision, recall, and the churn rate among the top customers the team can contact remain more informative than accuracy alone when churn is rare.

A note to remember: A different learning method changes how the prediction is calculated. It does not remove the need for representative data, honest evaluation, or a useful action.

Applying the idea to payment screening

Let’s return to the payment example from Part 1.

Suppose we want to estimate whether a new payment needs additional verification. A neural network could take inputs such as:

• The amount relative to the customer’s usual spending.
• The number of attempted payments in the last ten minutes.
• Whether the device is familiar.
• The time since the previous payment.

We would train it using past payments labelled fraudulent or legitimate.

A large amount alone might tell us little. The network can learn how combinations of amount, recent activity, and device information relate to the labels.

When a new payment arrives, we prepare the same features and run a forward pass. The trained network produces a score, and the surrounding system uses a chosen threshold to decide whether to request verification.

Training changes the weights. Using the trained network means applying those weights to a new input. This second stage is called inference.

The score is not proof of fraud. As we discussed in Part 1, we still need to check missed fraud, false alarms, and whether the threshold makes sense for the action.

Other practical pattern recognition examples

Reading handwritten digits

Think back to the different handwritten versions of “7” in Part 1.

The input is an image represented by pixel values. The target is the digit, from 0 to 9. The network learns visual relationships that help distinguish one digit from another despite differences in handwriting.

A digit classifier commonly has ten output scores, one for each possible digit, rather than the single churn output in our subscription example.

This is a building block for tasks such as reading numerical fields on scanned forms. The Keras handwritten digit example shows a small neural network trained on MNIST images. A complete document-reading system would also need to locate and prepare the relevant image regions.

Recognising spoken commands

For a command recogniser, the input is a short audio recording and the target might be “stop,” “go,” or “yes.”

The network learns sound patterns that help distinguish the commands across the voices and recording conditions represented in training.

TensorFlow’s audio recognition example demonstrates this with a small set of spoken words. The practical goal is to recognise a command so a device can respond.

Inspecting products for defects

Return to the camera on the factory line in Part 2. This time, consider giving an image network the product photograph rather than only measurements such as weight and surface roughness.

A neural network could be trained on images labelled acceptable or defective. Its task would be to recognise visual patterns associated with defects, such as scratches or damaged edges, and flag products for inspection.

Here, the useful question is whether it catches relevant defects on new products under actual factory lighting. A model trained only on clear, carefully staged images may struggle when conditions change.

Across these examples, the input changes from customer or transaction features to pixels or audio. The common task is learning relationships that help interpret a new observation.

The network architecture can change too. A small MLP helps us understand the mechanics, while image and audio systems may use architectures designed for those inputs.

When a neural network earns its place

For our customer measurements, the network needs to demonstrate an improvement over simpler alternatives. It may require more tuning, and its prediction is harder to explain as a short path of questions than the compact tree in Part 3.

For images, audio, and text, a suitable neural network can also learn representations from those inputs. That extends the feature-learning idea introduced in Part 2. Architecture and data preparation still need to suit the task.

Multiple hidden layers are commonly associated with deep learning. We don’t need a large or deep network to understand the learning loop, and adding depth is a choice to evaluate rather than a requirement.

Putting it all together

Return to Monday morning. Maya is still on the retention list, but this time her score comes from a neural network.

The network combines the inputs through layers of neurons, learns useful representations, and produces a prediction. Activation functions let those layers model nonlinear relationships.

During training, we compare predictions with the observed 30-day outcomes. Backpropagation calculates gradients, and the optimizer adjusts the weights and biases.

Before the team acts, we ask the questions carried through the series: did the model work on later examples, are its probabilities useful, and does the chosen threshold fit the team’s capacity? Predicting that Maya may leave still does not tell us whether a particular offer will persuade her to stay.

When I talk about using neural networks for pattern recognition, this is what I mean: learning useful relationships from examples so we can interpret new observations and support a decision.