Advanced Learning Algorithms
In my previous article, I explained the fundamentals of machine learning, including linear regression, cost functions, and gradient descent.
After learning those concepts, I wanted to understand how computers solve more complicated problems, like recognizing faces, understanding speech, and making predictions using large amounts of data.
That led me to the second course in Andrew Ng’s Machine Learning Specialization: “Advanced Learning Algorithms.”
At first, topics like neural networks, backpropagation, and decision trees sounded a little complicated. But once I started learning them, I realized that the basic ideas were not as difficult as they seemed.
So, just like my previous article, I decided to explain what I learned in the simplest way possible.
This article covers four main topics:
- Neural Networks: How computers learn complicated patterns.
- Training Neural Networks: How they learn from mistakes.
- Improving Machine Learning Models: How to identify problems and fix them.
- Decision Trees: How computers make decisions by asking questions.
Neural Networks
What is a neural network?
Think about how our brain works.
Our brain contains billions of tiny cells called neurons. These neurons communicate by sending signals to each other.
A single neuron cannot do much on its own. But when billions of them work together, we can recognize faces, understand languages, and solve problems.
A neural network is a machine learning model loosely inspired by this idea.
Instead of biological neurons, it uses small mathematical calculations connected.
Each calculation is simple, but when thousands or millions of them work together, they can learn complicated patterns.
Let’s discuss with an example
Imagine you have an online T-shirt business.
You want to predict whether a particular T-shirt will become a top-selling product.
You have information about four things:
- Price of the T-shirt
- Shipping cost
- Marketing budget
- Quality of the material
Now, how can a neural network use this information?
Instead of directly predicting whether the T-shirt will sell well, it can learn some useful patterns from the data.
For example:
Affordability: Is the T-shirt affordable when we consider its price and shipping cost?
Awareness: Do enough people know about the T-shirt through marketing?
Perceived Quality: Will customers consider the T-shirt a good-quality product?
These are examples of intermediate patterns a neural network might learn.
The interesting part is that we do not have to manually tell the neural network to calculate these things.
During training, it can learn useful intermediate patterns by itself.
Using those patterns, the network makes its final prediction.
How are neural networks organized?
Neural networks are organized into groups called layers.
There are three main types of layers.
1. Input Layer
This is where we provide information to the model.
For our T-shirt example, the inputs are price, shipping cost, marketing budget, and material quality.
2. Hidden Layers
These layers process information and learn patterns from it.
For example, a hidden layer might learn relationships connected to affordability, awareness, and quality.
They are called hidden layers because their outputs are internal values learned by the network rather than the answers directly provided in our dataset.
3. Output Layer
This layer gives us the final prediction.
In our example, it could predict the probability that the T-shirt will become a top seller.
A neural network can have multiple hidden layers. When a network contains many layers, we commonly call it a deep neural network.
This is where the term Deep Learning comes from.
What Does a Single Neuron Actually Do?
Remember the linear regression formula from my previous article?
f(x) = wx + bA neuron uses a similar calculation.
It takes the input values, multiplies them by weights, and adds a bias.
Then it passes the result through something called an activation function.
The general formula is:
a = g(w · x + b)Here:
- x: Input values
- w: Weights
- b: Bias
- g: Activation function
- a: Output of the neuron, called its activation
An activation function decides how the calculated value should be transformed before passing it to the next layer.
For example, the sigmoid activation function converts any real number into a value between 0 and 1.
So, a neuron is basically a small calculator.
A layer contains many of these calculators working together.
How Does a Neural Network Make Predictions?
This process is called Forward Propagation.
Imagine a factory assembly line.
At the first station, workers receive raw materials and process them. Then they pass the results to the next station.
The next station performs another operation, and the process continues until the final product is ready.
A neural network works similarly.
- The input layer receives the data.
- The first hidden layer performs calculations.
- Its outputs are passed to the next layer.
- The process continues through the network.
- The output layer produces the final prediction.
The information flows forward through the layers. That is why we call it forward propagation.
Building a Neural Network with TensorFlow
The good news is that we do not have to write all those calculations manually.
Using TensorFlow, we can create a neural network with just a few lines of Python.
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
model = Sequential([
Dense(units=3, activation='sigmoid'),
Dense(units=1, activation='sigmoid')
])Here, Sequential arranges the layers one after another, Dense creates the connected neurons, and units tells us how many neurons each layer has.
That is our basic neural network! We still need to train it before making useful predictions.
How Do Neural Networks Learn?
Remember gradient descent from my previous article?
Neural networks learn using a similar idea.
Make a prediction → Check the mistake → Adjust the weights → Repeat.
In TensorFlow, training involves three steps:
- Build: Create the neural network.
- Compile: Choose how to measure mistakes and update the parameters.
- Fit: Give the network training data so it can learn.
model.compile(
optimizer='adam',
loss='binary_crossentropy'
)
model.fit(X, y, epochs=100)An epoch means going through the entire training dataset once.
Get Sahan Dewasinghe’s stories in your inbox
Join Medium for free to get updates from this writer.
So, 100 epochs means the network goes through the dataset 100 times.
But how does it know which weights to change?
That is where Backpropagation comes in. It calculates how much each weight contributes to the error. Then an optimizer uses that information to adjust the weights.
A Few More Important Concepts
ReLU: An activation function that keeps positive numbers and turns negative numbers into zero. It is commonly used in hidden layers.
Softmax: Converts output scores into probabilities when we have multiple categories, like recognizing digits from 0 to 9.
Adam: A popular optimizer that automatically adapts the step sizes used to update the model’s parameters.
You do not have to understand every mathematical detail right now. The important thing is knowing what each part does.
What If Our Machine Learning Model Is Not Good Enough?
This was one of my favorite parts of the course.
Imagine a student who memorizes every question in a practice paper.
When the same questions appear in an exam, the student gets 100%.
But when the questions change, the student struggles.
Machine learning models can have the same problem.
Underfitting vs. Overfitting
Underfitting (High Bias):
The model is too simple to understand the patterns.
For example, trying to fit a straight line through data that follows a complicated curve.
Overfitting (High Variance):
The model memorizes the training data instead of learning patterns that work on new data.
So how do we identify these problems?
We split our data into three parts:
- Training Set: The model learns from this data.
- Cross-Validation Set: We use this to compare models and make improvements.
- Test Set: We use this to evaluate the final model.
We can compare training error and validation error to understand how well our model is learning.
- High Training Error + High Validation Error: The model is probably underfitting. It is too simple to understand the patterns.
- Low Training Error + High Validation Error: The model is probably overfitting. It performs well on training data but struggles with new data.
- Low Training Error + Low Validation Error: The model may be performing well on both training and new data.
How Can We Improve the Model?
If the model is underfitting, we can try adding useful features or using a more complex model.
If the model is overfitting, we can try collecting more data, simplifying the model, or using regularization.
Another useful technique is Error Analysis.
Instead of randomly changing the model, we look at the predictions it got wrong and try to understand why.
There are also two useful techniques when we have limited data:
Data Augmentation: Create additional training examples by modifying existing ones, such as rotating or adjusting images.
Transfer Learning: Start with an already trained model and adapt it to our task instead of learning everything from scratch.
Is Accuracy Always Enough?
Not really.
Imagine only 1 out of 100 patients has a disease.
A model that predicts everyone is healthy would have 99% accuracy, but it would miss every patient with the disease.
That is why we use other measurements.
Precision: Out of all the positive predictions, how many were actually positive?
Recall: Out of all the actual positive cases, how many did the model identify?
F1 Score: Combines precision and recall into one measurement.
Decision Trees
Now let’s look at a completely different machine learning algorithm.
Imagine playing a guessing game.
You are trying to identify an animal, so you ask questions:
- Does it have pointy ears?
- Does it have a round face?
- Does it have whiskers?
Depending on the answers, you reach a final decision.
A Decision Tree works similarly.
It makes predictions by asking a series of questions.
The first question is called the Root Node, the questions in between are Decision Nodes, and the final answers are Leaf Nodes.
How Does It Choose the Questions?
Imagine two boxes.
One contains 10 cats.
The other contains 5 cats and 5 dogs.
The first box is easier to classify because everything belongs to the same category.
Decision trees measure how mixed a group is using something called Entropy.
A group containing only one class has low entropy. A group evenly divided between two classes has high entropy.
The tree tries to choose questions that reduce this uncertainty.
The reduction is called Information Gain.
So, the tree keeps asking questions that separate the data into clearer groups.
Why Use Multiple Decision Trees?
Imagine asking one person for advice versus asking 100 people and combining their opinions.
This is the basic idea behind Tree Ensembles.
Two popular techniques are:
Random Forest: Builds many different decision trees and combines their predictions.
XGBoost: Builds boosted trees one after another, with each new tree helping improve the overall prediction.
Decision Trees or Neural Networks?
Both are useful, but they work well for different problems.
Decision Trees:
- Often great for structured, table-like data.
- Small decision trees are easier to understand and explain.
- Often faster to train on tabular data.
Neural Networks:
- Often great for images, audio, and text.
- Usually more difficult to interpret.
- Can require more computing power to train.
A useful starting point: For table-like data, try tree-based models. For images, audio, and language, consider neural networks.
Conclusion
When I started learning these topics, words like neural networks, backpropagation, and decision trees sounded complicated.
But after understanding the basic ideas, I realized they are not as scary as they seem.
A neural network is a group of small calculations working together.
Training means learning from mistakes.
Improving a model means understanding what went wrong.
And a decision tree is simply a model that learns which questions to ask.
All these topics helped me understand how machine learning models learn, make predictions, and improve.
This article was based on what I learned from “Advanced Learning Algorithms,” the second course of Andrew Ng’s Machine Learning Specialization on Coursera.
Next, I plan to explore Unsupervised Learning, Recommender Systems, and Reinforcement Learning.
If this helped, follow along. I will continue sharing what I learn.
Machine learning might look complicated at first, but the ideas become simpler once you break them down.






