How machines learn: Classification, Clustering and Beyond

In our map of AI, ML, deep learning and reinforcement learning we grouped machine learning by the kind of feedback a model gets: right answers, no answers, answers hidden in the data itself, or a score after acting. This post opens each of those boxes. For every one we look at the typical tasks, a few well-known algorithms, and how you would measure whether it works.

You do not need any maths to follow along. Where a formula would normally go, we use a picture.

Supervised learning: learning from the answer key

In supervised learning every training example is a pair: an input (an email, a photo, a row of numbers about a house) and the label we want the model to produce (spam or not, “cat”, a sale price). Training adjusts the model until its outputs match the labels as closely as possible. Then we hope it does the same on inputs it has never seen.

Supervised problems come in two main shapes, depending on what the label is.

Classification: predicting a category

The label is one of a fixed set of classes.

• Binary classification has two classes: fraud or not fraud, will churn or will stay, tumour or healthy tissue.
• Multi-class classification has more than two, but each example belongs to exactly one: which digit is in this image, which language is this sentence in.
• Multi-label classification lets an example carry several labels at once: a photo can contain both “beach” and “dog”.

Common algorithms include logistic regression (despite the name, a classifier), decision trees and their ensembles such as random forests and gradient boosting (XGBoost, LightGBM), support vector machines, k-nearest neighbours, and neural networks for images, audio and text. On tabular business data, gradient-boosted trees are very often the strongest choice.

How to measure it. Accuracy, the share of correct predictions, is the obvious metric and often a misleading one. If 1% of transactions are fraud, a model that always says “not fraud” is 99% accurate and completely useless. That is why teams look at the confusion matrix and two numbers from it:

• Precision: of everything the model flagged, how much was really positive? Low precision means many false alarms.
• Recall: of everything that was really positive, how much did the model catch? Low recall means missed cases.

The two pull against each other, and the right balance is a product decision. A spam filter should favour precision (never hide a real email); a cancer screening tool should favour recall (never miss a case). Google’s Machine Learning Crash Course has a good interactive section on this trade-off.

Regression: predicting a number

The label is a continuous value: the price a flat will sell for, tomorrow’s electricity demand, how many minutes a delivery will take.

Linear regression fits a straight line (or a flat surface, in many dimensions) through the data and is still the first model to try. Trees, gradient boosting and neural networks also do regression, and can capture curves and interactions a straight line cannot. Time-series forecasting is a special case of regression where the order of the data matters.

How to measure it. Typical metrics are mean absolute error (on average, how far off are we, in the same units as the label) and root mean squared error (similar, but punishes large misses more). Always compare against a naive baseline, such as “predict last week’s value”. A complex model that cannot beat the baseline is not worth shipping.

The catch: labels are expensive

Supervised learning works extremely well, but someone has to produce the labels. Doctors annotating scans, lawyers tagging contracts, crowd workers drawing boxes around cars: labelling is often the slowest and most expensive part of a project. That cost is exactly what the other paradigms try to avoid.

A middle path worth knowing is semi-supervised learning: label a small part of the data, then let the model use the structure of the large unlabelled part to improve. Many real systems quietly work this way.

Unsupervised learning: finding structure without answers

In unsupervised learning there are no labels at all. The model is given raw data and asked to describe it: what groups exist, what is normal, what can be summarised. The results are usually less “correct or wrong” and more “useful or not”, which makes them harder to evaluate but often very valuable for exploration.

Clustering: grouping similar things

Clustering puts examples that are alike into the same group, without being told what the groups are. Typical uses are customer segmentation, grouping news articles by topic, or finding similar products.

The classic algorithm is k-means, and it is simple enough to explain in three steps:

1. Pick k centres at random (you choose k, the number of clusters).
2. Assign every point to its nearest centre.
3. Move each centre to the average position of its points, then go back to step 2. Stop when the assignments no longer change.

K-means is fast, but it assumes round clusters of similar size and needs k up front. Other methods relax those assumptions. Hierarchical clustering builds a tree of nested groups, so you can choose the level of detail afterwards. DBSCAN finds clusters of any shape as dense regions of points, and labels isolated points as noise. Google’s free clustering course and the scikit-learn clustering guide compare these well, with pictures of where each one succeeds and fails.

How to measure it. Without labels there is no accuracy. Teams use internal scores such as the silhouette score (are points closer to their own cluster than to others?), but the real test is whether the groups make sense to people and lead to better decisions.

Anomaly detection: spotting what does not fit

Anomaly detection learns what “normal” looks like and flags what is far from it: an unusual card payment, a server whose metrics suddenly drift, a faulty part on a production line. Because real anomalies are rare and varied, it is often impossible to label enough of them, which makes this a natural unsupervised task.

Get Gilaburu’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Approaches range from simple statistics (how many standard deviations from the mean?) to isolation forests, which exploit the fact that odd points are easier to separate from the rest, and to autoencoders, neural networks that learn to compress and rebuild normal data and struggle to rebuild anything unusual.

Dimensionality reduction: keeping what matters

Real data often has hundreds or thousands of columns. Dimensionality reduction squeezes it into a few while keeping as much of the structure as possible. Principal component analysis (PCA) finds the directions along which the data varies most. t-SNE and UMAP are popular for drawing high-dimensional data, such as text embeddings, as 2D maps where similar items land near each other. Reduced data is also faster and often easier for other models to learn from.

Association rules: what goes together

The last classic family looks for items that tend to appear together: people who buy pasta often buy tomato sauce. This “market basket analysis” drives simple recommendations and shop layouts, and algorithms such as Apriori have been around since the 1990s.

Self-supervised learning: the data writes its own labels

Self-supervised learning is the idea behind almost every modern foundation model. It uses unlabelled data, like unsupervised learning, but trains the model with a supervised-style task whose answers come from the data itself. You hide part of the input and ask the model to fill it in.

• Next-token prediction. Show the start of a sentence, ask for the next word. This is how GPT-style language models are pre-trained, on enormous amounts of text.
• Masked prediction. Hide a word in the middle and ask the model to recover it from both sides. Google’s BERT made this popular in 2018. The same idea works for images, by hiding patches.
• Contrastive learning. Show two altered versions of the same photo (cropped, recoloured) and teach the model that they belong together while other photos do not. SimCLR is a well-known example.

None of these tasks is useful in itself. Nobody needs a machine to guess hidden words. The point is that, to do them well, the model has to learn grammar, facts, shapes and context. The result is a general-purpose representation that can then be fine-tuned for a real task with only a small labelled dataset. That two-step recipe, pre-train with self-supervision, then adapt with a little supervision, is the main reason AI capabilities jumped so quickly in recent years.

Reinforcement learning: learning from consequences

Reinforcement learning (RL) has no answer key and no fixed dataset. An agent acts in an environment and receives a reward. Its job is to learn a policy, a rule for choosing actions, that collects the most reward over time. We introduced the loop in the first post; here are the main families of methods.

• Multi-armed bandits are the simplest case: one decision, no long-term state. Which of four headlines should we show? Try them, watch the clicks, gradually show the winner more often. Bandits are widely used for testing and recommendations.
• Value-based methods learn how good each action is in each situation, then pick the best one. Q-learning is the classic example. DeepMind’s DQN combined it with a deep neural network and learned to play dozens of Atari games from raw pixels.
• Policy-based methods skip the scores and adjust the policy directly in the direction that earned more reward. Policy gradient methods, and their popular stable variant PPO, are common in robotics and in tuning language models.
• Model-based methods first learn how the environment behaves, then plan inside that learned model. They need far fewer real-world trials, which matters when each trial is slow or expensive.

Every RL method has to balance exploration (trying new actions to learn more) and exploitation (using what already works). A simple, widely used rule is epsilon-greedy: usually pick the best known action, but now and then pick a random one.

RL is powerful but fussy. It needs a reward that really captures what you want, a lot of trial and error, and usually a simulator. In practice it shines in games, robotics, resource scheduling, and in reinforcement learning from human feedback (RLHF), where human preferences become the reward that makes chat assistants more helpful. Sutton and Barto’s Reinforcement Learning: An Introduction remains the best place to go deeper.

Choosing an approach

Most real projects start from the data you have and the decision you need to make. This rough guide covers the common cases:

Scikit-learn’s Choosing the right estimator map is a longer version of the same idea, and worth bookmarking.

A few habits matter more than the choice of algorithm:

• Start with a baseline. A simple rule or a linear model tells you how much the fancy approach is really adding.
• Keep a test set the model never sees during training, and evaluate on it only at the end.
• Pick the metric with the people who will use the model. Precision or recall, average error or worst case: these are product questions, not just technical ones.
• Watch it after launch. Data changes over time, and a model that was right last year can quietly drift.

Summary

In the next post in this series we will open the most popular box of all: how a neural network actually learns from its mistakes.

Sources and further reading

• Google, Machine Learning Crash Course: Classification
• Google, Clustering course
• scikit-learn, User guide: clustering and Choosing the right estimator
• J. Devlin et al., BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (2018)
• T. Chen et al., A Simple Framework for Contrastive Learning of Visual Representations (2020)
• V. Mnih et al., Human-level control through deep reinforcement learning, Nature 518 (2015)
• R. Sutton, A. Barto, Reinforcement Learning: An Introduction, 2nd edition (2018)

Originally published at https://gilaburu.co on November 4, 2026.