How Machine Learning Learns from Data

From Raw Data to Intelligent Decisions — by Aman Kumar Singh

When Netflix suggests your next show or a bank flags a suspicious transaction within seconds, nobody has written a rule for that exact situation. The system learned it from data. But what does “learning” mean for a machine?

Machine learning (ML) is the science of learning patterns from data using statistics, linear algebra, programming and optimization, so that a system can predict or classify things it has never seen before. Think of it as a student preparing for an exam. The student studies past papers (data), practises (training), takes mock tests (validation), and finally faces the real exam (unseen data). This blog follows that journey from raw data to a decision, and ends with a small experiment you can run yourself.

The pipeline: Collect data → Prepare & explore → Choose model → Train → Evaluate → Improve → Deploy. Results at each stage send us back to improve earlier stages: ML is iterative.

Step 1: Raw data, the starting point

• Structured data: neat rows and columns, like a bank’s customer table.
• Unstructured data: images, audio and text, such as ECG signals, speech or emails.
• Cross-sectional data: many subjects at one point in time, like a survey of 1,000 students.
• Time-series data: one subject over time, like daily stock prices or sensor readings.

The kind of data you have decides which techniques you can use. Data is also the biggest limit on any model: a model trained on poor data will give poor decisions.

Step 2: Choosing how the machine learns

TypeLearns fromExampleSupervisedLabelled data (input + correct answer)Predicting house prices, spam detectionUnsupervisedUnlabelled data, finds hidden structureGrouping customers by behaviourReinforcementRewards and penalties from trial and errorA robot learning to walk, game-playing agents

Supervised problems split further into regression (predicting a number, like tomorrow’s temperature) and classification (predicting a category, like spam or not spam). Classification can be binary or multi-class (like recognising digits 0 to 9).

Step 3: Cleaning and preparing the data

Real data is messy, and preparation often takes more time than the modelling itself.

1. Exploratory Data Analysis (EDA): Bar charts, histograms, box plots, scatter plots and heat maps show distributions, outliers and relationships before any model is built.
2. Handling missing values: Fill gaps with the mean or median (use the median when outliers exist), or use k-NN to estimate them from similar records.
3. Transforming data: One-hot encoding turns categories into numbers. Log transforms tame skewed values like income. Normalization puts features on the same scale, so a feature measured in lakhs does not drown out one measured in single digits.

Min-max scaling maps values into 0 to 1; standardization (z-score) centres data at 0 with unit spread:

x′ = (x − x_min) / (x_max − x_min) (1)

z = (x − μ) / σ (2)

Step 4: Learning the pattern

Learning means finding the model parameters that minimize a loss function, a number measuring how wrong the predictions are. For regression, two common measures are MAE and MSE:

MAE = (1/n) Σ |yᵢ − ŷᵢ| (3)

MSE = (1/n) Σ (yᵢ − ŷᵢ)² (4)

MAE is the average size of the errors, while MSE squares each error first, so large mistakes are punished much more heavily. Example: true values 3, 5, 7, predictions 2, 5, 9. MAE = (1 + 0 + 2) / 3 = 1.0. MSE = (1 + 0 + 4) / 3 ≈ 1.67. The single big mistake dominates the MSE, which is why MSE is more sensitive to outliers.

The most common way to minimize the loss is gradient descent. It adjusts the parameters step by step in the direction that reduces the error, like walking downhill in fog by always stepping where the ground slopes down:

θ(t+1) = θ(t) − α ∇J(θ(t)) (5)

where J(θ) is the loss, ∇J(θ) its gradient, and α the learning rate (step size). If α is too large we overshoot the valley; if too small, training takes forever.

Linear and logistic regression

Linear regression fits a straight line (or a plane) to the data:

ŷ = β₀ + Σ βⱼxⱼ (6)

Logistic regression passes the same weighted sum z through an S-shaped sigmoid, so the output is a probability between 0 and 1, suitable for classification:

P(y = 1 | x) = 1 / (1 + e^(−z)), z = β₀ + Σ βⱼxⱼ (7)

To stop a model from becoming too complex, regularization adds a penalty on large weights. Ridge adds the sum of squared weights and shrinks all weights smoothly, while LASSO adds the sum of absolute weights and can push some weights exactly to zero, which also performs feature selection. λ controls the penalty strength:

J_ridge = MSE + λ Σ βⱼ² (8)

J_lasso = MSE + λ Σ |βⱼ| (9)

k-Nearest Neighbours (k-NN)

k-NN classifies a new point by looking at the k closest known points and taking a vote, like asking your nearest neighbours for their opinion. Closeness is the Euclidean distance:

d(x, x′) = √( Σ (xⱼ − x′ⱼ)² ) (10)

Example: predict whether a new student who studies 4 hours a day with 75% attendance will pass.

StudentHoursAttendance (%)ResultDistanceA260Fail15.1B370Fail5.1C685Pass10.2D790Pass15.3E580Pass5.1

For B: √((4 − 3)² + (75 − 70)²) = √26 ≈ 5.1. The three nearest neighbours are B (Fail), E (Pass) and C (Pass). Two votes to one, so the prediction is Pass. Notice that attendance is measured in tens while hours are single digits, so attendance dominates every distance. This is exactly why normalization matters. k-NN also struggles with too many features (the curse of dimensionality), because in high dimensions every point ends up roughly equally far from every other.

Decision trees and Naive Bayes

A decision tree is a flowchart of yes/no questions, such as “Is attendance above 75%?”. It is easy to read and explain, but an unrestricted tree can grow until it memorizes the training data, so it is controlled by pruning.

Naive Bayes uses Bayes’ rule to compute the probability of each class C given features x:

P(C | x) = P(x | C) P(C) / P(x) (11)

It is “naive” because it assumes features are independent once the class is known, so the likelihood becomes a product and the predicted class is the one with the largest value:

P(C | x₁…xₚ) ∝ P(C) Π P(xⱼ | C) (12)

The assumption is rarely true, yet the method is fast and works surprisingly well for spam filtering and text.

Support Vector Machines (SVM)

Many lines can separate two classes, but an SVM picks the one with the maximum margin, the widest empty street between the classes. The points touching the edge of the street are the support vectors, and only they decide where the boundary goes. For a boundary w · x + b = 0:

margin = 2 / ‖w‖ (13)

Maximizing the margin means minimizing the length of w while keeping every training point (label yᵢ = +1 or −1) on the correct side:

min ½‖w‖² (14)

subject to yᵢ(w · xᵢ + b) ≥ 1 for every training point i.

Real data is rarely perfectly separable, so a soft margin allows a few mistakes using slack variables, and a parameter C controls the trade-off between a wide margin and few errors. When classes cannot be separated by any straight line, such as one class forming a ring around another, the kernel trick lets the SVM behave as if the data had been lifted into a higher dimension where a flat boundary works, without ever computing that transformation explicitly.

Neural networks

A neural network is built from layers of simple units (neurons). Each neuron takes a weighted sum of its inputs and passes it through a non-linear activation function f:

a = f( Σ wᵢxᵢ + b ) (15)

A single neuron is a perceptron; stacking layers lets the network learn very complex patterns. Training uses backpropagation in four repeating steps: (1) a forward pass produces a prediction, (2) the loss measures the error, (3) the chain rule sends the error backward through the layers to find how much each weight contributed, and (4) gradient descent updates every weight:

Get Aman Ks’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

w ← w − α ∂L/∂w (16)

This is the foundation of deep learning.

Step 5: Learning without answers

Sometimes there are no labels. k-means clustering repeats two steps until nothing changes: assign every point to its nearest centre, then move each centre to the average of its points. It minimizes the total squared distance between points and their cluster centres μₖ:

J = Σₖ Σᵢ ‖xᵢ − μₖ‖² (17)

Hierarchical clustering builds a tree of merges, so you can cut it at any level to get the number of groups you want. Gaussian Mixture Models (GMM) allow soft grouping, where a point can belong 70% to one cluster and 30% to another, modelling the data as a weighted mix of K Gaussian distributions:

p(x) = Σₖ πₖ N(x | μₖ, σₖ²) (18)

where the mixing weights πₖ add up to 1. The model is fitted with Expectation-Maximization (EM), which alternates between estimating each point’s membership probabilities (E-step) and updating cluster means and spreads (M-step). A telecom company, for example, can discover distinct customer segments without ever being told what they are.

Step 6: Is the model actually good?

A model that scores 100% on training data may still fail in the real world. This is overfitting: the student memorized the past papers but did not understand the subject. The opposite, underfitting, means the model is too simple to capture the pattern. The balance between the two is the bias-variance tradeoff:

Expected error = Bias² + Variance + Noise (19)

High bias means the model is too rigid; high variance means it is too sensitive to the particular training sample.

Figure 2 (not shown here): polynomials of increasing degree fitted to 25 noisy points from a sine curve, with training error vs error on 200 unseen points. Experiment by the author using NumPy. The straight line (degree 1) underfits. The degree-4 curve follows the true pattern. The degree-15 curve passes close to every training point but swings wildly in between, and its error on unseen data explodes. To catch this, we test on data the model has not seen:

• Validation set method: hold back part of the data for testing.
• k-fold cross-validation: split the data into k parts, rotate which part is held back, then average the k scores.
• Leave-One-Out: an extreme version where each data point takes a turn as the test set.

CV = (1/k) Σ Eᵢ (20)

Measuring classification performance

The confusion matrix counts true positives (TP), false positives (FP), false negatives (FN) and true negatives (TN). A fraud detector tested on 1,000 transactions, of which 10 are truly fraud:

Predicted fraudPredicted genuineActually fraud (10)TP = 8FN = 2Actually genuine (990)FP = 20TN = 970

Accuracy = (TP + TN) / (TP + TN + FP + FN) (21)

Sensitivity = TP / (TP + FN) (22)

Specificity = TN / (TN + FP) (23)

Precision = TP / (TP + FP) (24)

For our detector: accuracy = 978 / 1000 = 97.8%; sensitivity = 8 / 10 = 80% of real frauds caught; specificity = 970 / 990 ≈ 98% of genuine transactions left alone; precision = 8 / 28 ≈ 28.6%, meaning only about 3 in 10 fraud alerts are real frauds. The ROC curve plots sensitivity against (1 − specificity) as the decision threshold is varied.

A critical point: accuracy can mislead

A lazy model that labels every transaction “genuine” would score 990 / 1000 = 99% accuracy, higher than our fraud detector, and yet catch zero fraud (sensitivity = 0%). The right metric depends on the cost of each mistake. In cancer screening, a missed case (false negative) is far worse than a false alarm, so sensitivity matters most. For a spam filter, blocking an important email (false positive) is the bigger problem.

Step 7: Combining models and simplifying data

• Ensemble methods: Many weak models together can beat one strong model. Bagging (as in random forests) trains models on different random samples and averages them, which reduces variance. Boosting trains models one after another, each focusing on the errors of the last, which reduces bias. It is like asking a committee of experts rather than a single one.
• Feature selection: Forward selection starts with no features and adds the most useful one at a time; backward selection starts with all features and removes the least useful.
• Dimensionality reduction (PCA): Principal Component Analysis finds new axes, called principal components, along which the data varies the most, and keeps only the first few. It compresses many correlated features into a few while keeping most of the information, and helps fight the curse of dimensionality.

The share of total variance captured by the i-th principal component is its eigenvalue λᵢ divided by the sum of all eigenvalues:

rᵢ = λᵢ / Σ λⱼ (25)

Putting it to the test: a small experiment

Five classic models were compared on the Iris dataset (150 flowers, 3 species, 4 measurements each) using scikit-learn. Features were standardized, and each model was scored with 5-fold cross-validation (shuffled, random_state = 42).

from sklearn.datasets import load_iris
from sklearn.model_selection import cross_val_score, StratifiedKFold
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier

X, y = load_iris(return_X_y=True)
cv = StratifiedKFold(5, shuffle=True, random_state=42)
model = make_pipeline(StandardScaler(), KNeighborsClassifier(5))
print(cross_val_score(model, X, y, cv=cv).mean())  # 0.973

ModelMean CV accuracy (%)Std. deviation (%)k-NN (k = 5)97.32.5SVM (RBF kernel)96.03.9Logistic Regression95.34.5Decision Tree95.33.4Random Forest94.72.7

What the results teach us

• No model is automatically best. The simple k-NN beat the Random Forest here. On a small, clean dataset with only 150 samples, a complex ensemble has little room to help. All five models lie within about 2.6 percentage points of each other, and the standard deviations are 2.5 to 4.5 points, so these differences are not conclusive.
• Overfitting is visible even on easy data. A fully grown decision tree scored 100% on its training split but 93.3% on the held-out test split (70/30 stratified split).
• The confusion matrix shows where the errors are. The k-NN model reached 91.1% on the test split, and all 4 mistakes were versicolor flowers predicted as virginica, two species whose measurements overlap. Setosa was classified perfectly. A single accuracy number hides this.
• PCA shows the data is simpler than it looks. After standardizing, the first two principal components capture 95.8% of the variance (about 73% + 23%). Four measurements can be reduced to two with very little information lost.

Step 8: From model to decision

• Healthcare: classifying ECG signals as normal or abnormal.
• Finance: predicting loan default or detecting card fraud.
• Telecom: predicting customer churn and grouping users into segments.
• E-commerce: recommending products.

At this point the model turns raw data into an intelligent decision.

The bigger picture

Machine learning follows one pipeline: collect data → prepare it → choose a model → train it → evaluate it → improve it → deploy it. It is also not magic. A model reflects its data, so biased or incomplete data leads to biased decisions. Our experiment showed that even on a tiny, clean dataset, the “best” model depends on how it is measured, and a single score can hide the mistakes that matter. Good ML practice needs good data, the right metrics, and human judgement about what the predictions mean. The best models are the ones we understand well enough to trust, and to question.

Sources and References

The topics follow the Machine Learning course (216U03E515) in the T.Y. B.Tech EXTC curriculum. The figures and Iris results come from the author’s own experiments using Python (NumPy, scikit-learn, Matplotlib).

1. Somaiya Vidyavihar University, K. J. Somaiya College of Engineering. (2025). T.Y. B.Tech EXTC Scheme and Detailed Curriculum (SVU-KJSCE 2023_2.0), BoS dated 17–04–2025. Course 216U03E515: Machine Learning.
2. Hastie, T., Tibshirani, R., and Friedman, J. (2009). The Elements of Statistical Learning (2nd ed.). Springer.
3. Alpaydin, E. (2015). Introduction to Machine Learning (3rd ed.). PHI Learning.
4. Gopal, M. (2019). Applied Machine Learning (1st ed.). PHI Learning.
5. Shalev-Shwartz, S., and Ben-David, S. (2014). Understanding Machine Learning: From Theory to Algorithms. Cambridge University Press.
6. Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning. MIT Press.
7. Géron, A. Hands-On Machine Learning with Scikit-Learn and TensorFlow. O’Reilly Media.
8. Pedregosa, F., et al. (2011). Scikit-learn: Machine Learning in Python. JMLR, 12, 2825–2830.

Written by Aman Kumar Singh