My Neural Network Scored 97.8%. It Still Missed One in Three Rare Heartbeats.
What a small classifier built from scratch in PyTorch taught me about trusting a single number
A heart monitor that says “everything looks normal” is right most of the time, because most heartbeats are normal. That is exactly why it can be dangerous. The beats that matter are the rare, abnormal ones, and a system can look excellent while quietly missing them.
For our Applied Machine Learning project, my teammate Tayyba Bashir and I built a neural network that looks at a single heartbeat and decides what type it is. It reached 97.82% test accuracy. This post is about why that number is less impressive than it sounds, and what the network was really doing underneath it.
The problem: one heartbeat, five possible types
An electrocardiogram (ECG) records the heart’s electrical activity. Every heartbeat makes a characteristic wave, and abnormal beats (arrhythmias) have different shapes.
We used the ECG Heartbeat Categorization Dataset from Kaggle, built from the MIT-BIH Arrhythmia Database. Each row is one heartbeat: 187 measurements taken 125 times per second, which is about 1.5 seconds of signal. Shorter beats are padded with zeros at the end. Every beat belongs to one of five classes:
- N — normal beat
- S — supraventricular ectopic beat
- V — ventricular ectopic beat
- F — fusion beat
- Q — unknown or paced beat
The catch is the class balance. Of the 87,554 training beats, 72,471 are normal. The fusion class has only 641.
Looking at one example of each, the normal beat falls quickly, stays low and then shows a thin, tall spike. The S and F examples share that sharp fall and thin spike, but S has a higher middle section and F has an earlier spike. The V and Q examples look most different, staying high in the middle. Keep that similarity between N and F in mind. It comes back later.
What we built
The network is small and fully connected: 187 inputs, two hidden layers of 128 and 64 units with ReLU, and 5 outputs, one score per class. That is 32,645 trainable parameters.
The rule of the project was that we build everything ourselves. No ready-made layers, no built-in loss function and no built-in optimiser. We created each weight matrix by hand, wrote the forward pass, wrote the softmax cross-entropy loss and wrote the update rule. PyTorch only computed the gradients, and we checked one of those by hand too.
Get Moomaleman’s stories in your inbox
Join Medium for free to get updates from this writer.
We split the training file into 78,799 training rows and 8,755 validation rows. The 21,892 test rows were locked away and used exactly once, at the very end.
Training: choosing how big a step to take
We trained with mini-batches of 128 beats, momentum of 0.9 and a small weight decay of 0.0001, for 30 epochs. The one thing we had to choose was the learning rate, from 0.01, 0.1 and 0.5. We trained a fresh network for each and picked the one with the lowest validation loss.
Learning rate Lowest validation loss Validation accuracy at that epoch
0.01 0.1618 95.79%
0.1 0.0862 97.64%
0.5 0.0722 97.93%
A learning rate of 0.5 won. Its lowest validation loss came at epoch 23, so we saved the network from that epoch. Training took about 51 seconds on an ordinary laptop CPU. No GPU was needed.
The results, and the number that hides the problem
Here is the headline:
Test accuracy 97.82%
Balanced accuracy 83.65%
Always-"N" accuracy 82.76%And here is the part that matters:
Class N S V F Q
Per-class accuracy 99.60% 66.37% 91.37% 62.96% 97.95%The network is almost perfect on normal beats and very good on paced beats. But it recognises only about two in three S beats and fewer than two in three fusion beats.
What the results taught us
Is accuracy alone enough? No. A “model” that always answers “N” would get 18,118 of the 21,892 test beats right, which is 82.76%, without detecting a single abnormal beat. Our network beats that comfortably, but its 97.82% is still dominated by normal beats. Balanced accuracy, the average of the five per-class accuracies, drops to 83.65%. That gap is the honest summary of the project.
Which class is hardest? Fusion. The network got only 102 of 162 fusion beats right. Its most common mistake was calling them normal (39 beats), followed by ventricular (21 beats). That makes some sense: a fusion beat happens when a normal beat and a ventricular beat occur together, so it can share parts of both shapes. With 113 times more normal beats than fusion beats in training, the network also had far more reason to learn “normal” well.
Did it overfit? Only mildly. Training and validation loss start together, then validation loss settles slightly above training loss and stops improving after about epoch 20. At the best epoch, training loss was 0.0560 and validation loss was 0.0722, and the accuracies differed by only 0.33 percentage points. Saving the best epoch protected us from the later drift.
Why not start all the weights at zero? Because then every unit computes the same thing, gets the same gradient and learns the same thing. When we tried it on our network, every weight gradient was exactly zero. Random starting weights break that symmetry. We used the He rule, which draws weights with standard deviation √(2/n), where n is the number of inputs to the layer. The n keeps the signal from growing as many inputs are added, and the 2 makes up for ReLU zeroing out about half the values.
What do momentum and weight decay do? Momentum averages recent gradients, so when two consecutive gradients point in opposite directions they largely cancel and the step stays small. Starting from zero, a gradient of +1 gives a velocity of 0.1, and a following gradient of −1 brings it to −0.01. Weight decay pulls every weight slightly towards zero. A weight that gets no gradient from the loss still shrinks, by roughly 3% per epoch with our settings.
How could the network do better on rare beats? We would try a class-weighted loss, where a mistake on a rare beat costs more. Weighting each class by how rare it is would give fusion beats a weight of about 27.3 and normal beats about 0.24. The likely trade-off is more false alarms on normal beats. We did not run this. It is our suggestion for next time.
Try it yourself
We also built a small Streamlit app. You pick a test beat, or ask for a random beat of a chosen class, and see the signal, the predicted class, the probability of all five classes and whether the network was right.
My favourite way to use it is to ask for a random fusion beat. It makes the 62.96% feel very real.
The lesson
The most useful thing this project taught me was not how to build a neural network. It was how to doubt one. A single accuracy figure can be true and still mislead you. When one class is far more common than the rest, look at every class separately, because a monitor that misses the rare beat is the one that matters.
The code, results and app are on GitHub: [https://github.com/MoomalEmaan/Classifying-Heartbeats-from-ECG-Signals]
