From Market Baskets To Models: Association Rule Learning With Apriori
Introduction
Association rule learning is an unsupervised machine learning technique used to discover interesting relationships or associations between variables in a dataset. It identifies frequent patterns, connections, and dependencies among groups of items, often called itemsets. This method is widely used in retail, healthcare, finance, and entertainment — especially in market basket analysis, web usage mining, and recommendation systems.
In this article, we’ll introduce the Apriori algorithm, a classical method for association rule learning. We’ll explore three key statistical metrics — support, confidence, and lift — and use Apriori to perform Market Basket Analysis, a technique commonly used in supermarkets to uncover item-purchase relationships. For example, if customers often buy bread and butter together, placing them side by side could boost sales.
The Apriori Algorithm: A Brief Overview
Apriori is a fundamental algorithm that helps uncover frequent itemsets and generate association rules. It is based on the principle that:
If an itemset is frequent, then all its subsets must also be frequent.
For instance, if {egg, ground beef, herbs & pepper} is a frequent itemset, then {egg, ground beef} must also be frequent.
Understanding Association Rules
An association rule is expressed as:
X ⇒ Y, meaning if X occurs, then Y is likely to occur.
Let’s consider the following transaction data:
Transactions Items
t1 {Light Cream, Chicken}
t2 {pasta, escalope}
t3 {pasta, shrimp}
t4 {egg, ground beef, herb & pepper}
t5 {herb & pepper, ground beef, spaghetti}
t6 {herb & pepper, mineral water, ground beef}
t7 {tomato sauce, ground beef}From this, a rule could be:
{Herb & Pepper, Ground Beef} ⇒ {Spaghetti}
This implies that customers who buy herb & pepper and ground beef are likely to also buy spaghetti.
Key Metrics in Association Rule Mining
1. Support
Support measures how frequently an item appears in the dataset:
Support(X) = (# of transactions containing X) / (total # of transactions)
Example:
Support(Pasta) = 2/7 ≈ 28.6%
2. Confidence
Confidence evaluates the likelihood that Y is bought when X is bought:
Confidence(X ⇒ Y) = Support(X ∪ Y) / Support(X)
Example:
Confidence(Ground Beef ⇒ Herb & Pepper) = (3/7) / (4/7) = 75%
This means that 75% of the time ground beef is purchased, herb & pepper are also purchased.
3. Lift
Lift measures the strength of a rule compared to random co-occurrence:
Lift(X ⇒ Y) = Support(X ∪ Y) / (Support(X) × Support(Y))
- Lift = 1: X and Y are independent
- Lift > 1: Positive association (bought together more often than by chance)
- Lift < 1: Negative association
Example:
Lift(Ground Beef ⇒ Herb & Pepper) = (3/7) / [(4/7) × (3/7)] ≈ 1.8
Hands-on implementation in R
Loading the dataset
We will be working with a dataset of a supermarket with 7500 observations involving different products that customers bought from the supermarket. The apriori algorithm doesn’t take in a csv file as an argument when creating association rules. It takes in a sparse matrix. what is a sparse matrix? This is a matrix that contains a lot of zeros. Now how does this work? with the help of the arules package our dataset is converted into a sparse matrix where the columns will be the represented with the transactions i.e., the products making up the itemset. The rows will be represented with 1s to indicate that the product was purchased and 0s to indicate otherwise. Now you can imagine that we will have a lot of zero values and there you have your sparse matrix object.
Get Malachy K. Madu’s stories in your inbox
Join Medium for free to get updates from this writer.
for example;
Light Cream, Chicken, Pasta, Escalope, Shrimp, Eggs, Ground beef, Herb & pepper, Spaghetti, Mineral water, tomato sauce
t1 1 1 0 0 0 0 0 0 0 0 0
t2 0 0 1 1 0 0 0 0 0 0 0
t3 0 0 1 0 1 0 0 0 0 0 0
t4 0 0 0 0 0 1 1 1 0 0 0
t5 0 0 0 0 0 0 1 1 1 0 0
t6 0 0 0 0 0 0 1 1 0 1 0
t7 0 0 0 0 0 0 1 0 0 0 1 library(arules)
dataset = read.csv('Market_Basket_Optimisation.csv')
dataset = read.transactions('Market_Basket_Optimisation.csv', sep = ',', rm.duplicates = TRUE
summary(dataset)
#Item Frequency plot
itemFrequencyPlot(dataset, topN = 10)
The Apriori Algorithm
Choice of support and confidence
Selecting appropriate support and confidence values is crucial. Low values may generate too many rules, while high values may miss interesting patterns. Let’s experiment with different thresholds.
#Support and confidence values
supportLevels = c(0.1, 0.05, 0.01, 0.005)
confidenceLevels = c(0.9, 0.8, 0.7, 0.6, 0.5, 0.4, 0.3, 0.2, 0.1)
# Empty integers
rules_sup10 <- integer(length=9)
rules_sup5 <- integer(length=9)
rules_sup1 <- integer(length=9)
rules_sup0.5 <- integer(length=9)
# Apriori algorithm with a support level of 10%
for (i in 1:length(confidenceLevels)) {
rules_sup10[i] <- length(apriori(dataset, parameter=list(sup=supportLevels[1],
conf=confidenceLevels[i], target="rules")))
}
# Apriori algorithm with a support level of 5%
for (i in 1:length(confidenceLevels)){
rules_sup5[i] <- length(apriori(dataset, parameter=list(sup=supportLevels[2],
conf=confidenceLevels[i], target="rules")))
}
# Apriori algorithm with a support level of 1%
for (i in 1:length(confidenceLevels)){
rules_sup1[i] <- length(apriori(dataset, parameter=list(sup=supportLevels[3],
conf=confidenceLevels[i], target="rules")))
}
# Apriori algorithm with a support level of 0.5%
for (i in 1:length(confidenceLevels)){
rules_sup0.5[i] <- length(apriori(dataset, parameter=list(sup=supportLevels[4],
conf=confidenceLevels[i], target="rules")))
}In the following graphs we can see the number of rules generated with a support level of 10%, 5%, 1% and 0.5%.
# Combine with confidenceLevels into a data frame
df <- data.frame(
Confidence = confidenceLevels,
Rules = rule_counts
)
library(ggplot2)
# Number of rules found with a support level of 10%
plot1 <- qplot(confidenceLevels, rules_sup10, geom=c("point", "line"),
xlab="Confidence level", ylab="Number of rules found",
main="Apriori with a support level of 10%") +
theme_bw()
plot1 <- ggplot(df, aes(x = confidenceLevels, y = rules_sup10)) +
geom_point(color = "steelblue", size = 3) +
geom_line(color = "steelblue", linetype = "solid") +
labs(
title = "Apriori with a Support Level of 10%",
x = "Confidence Level",
y = "Number of Rules Found"
) +
theme_bw() +
theme(
plot.title = element_text(face = "bold", hjust = 0.5),
axis.title = element_text(size = 12)
)
# Print the plot
print(plot1)
# Number of rules found with a support level of 5%
plot2 <- qplot(confidenceLevels, rules_sup5, geom=c("point", "line"),
xlab="Confidence level", ylab="Number of rules found",
main="Apriori with a support level of 5%") +
theme_bw()
plot2 <- ggplot(df, aes(x = confidenceLevels, y = rules_sup5)) +
geom_point(color = "steelblue", size = 3) +
geom_line(color = "steelblue", linetype = "solid") +
labs(
title = "Apriori with a Support Level of 5%",
x = "Confidence Level",
y = "Number of Rules Found"
) +
theme_bw() +
theme(
plot.title = element_text(face = "bold", hjust = 0.5),
axis.title = element_text(size = 12)
)
# Print the plot
print(plot2)
# Number of rules found with a support level of 1%
plot3 <- qplot(confidenceLevels, rules_sup1, geom=c("point", "line"),
xlab="Confidence level", ylab="Number of rules found",
main="Apriori with a support level of 1%") +
theme_bw()
plot3 <- ggplot(df, aes(x = confidenceLevels, y = rules_sup1)) +
geom_point(color = "steelblue", size = 3) +
geom_line(color = "steelblue", linetype = "solid") +
labs(
title = "Apriori with a Support Level of 1%",
x = "Confidence Level",
y = "Number of Rules Found"
) +
theme_bw() +
theme(
plot.title = element_text(face = "bold", hjust = 0.5),
axis.title = element_text(size = 12)
)
# Print the plot
print(plot3)
# Number of rules found with a support level of 0.5%
plot4 <- qplot(confidenceLevels, rules_sup0.5, geom=c("point", "line"),
xlab="Confidence level", ylab="Number of rules found",
main="Apriori with a support level of 0.5%") +
theme_bw()
plot4 <- ggplot(df, aes(x = confidenceLevels, y = rules_sup0.5)) +
geom_point(color = "steelblue", size = 3) +
geom_line(color = "steelblue", linetype = "solid") +
labs(
title = "Apriori with a Support Level of 0.5%",
x = "Confidence Level",
y = "Number of Rules Found"
) +
theme_bw() +
theme(
plot.title = element_text(face = "bold", hjust = 0.5),
axis.title = element_text(size = 12)
)
# Print the plot
print(plot4)
# Subplot
grid.arrange(plot1, plot2, plot3, plot4, ncol=2)
now we have been able to generate different number of rules at support levels of 10%, 5%, 1% and 0.5%.
Let’s combine the four lines to improve the visualizations.
# Data frame
num_rules <- data.frame(rules_sup10, rules_sup5, rules_sup1, rules_sup0.5, confidenceLevels)
# Number of rules found with a support level of 10%, 5%, 1% and 0.5%
ggplot(data=num_rules, aes(x=confidenceLevels)) +
# Plot line and points (support level of 10%)
geom_line(aes(y=rules_sup10, colour="Support level of 10%")) +
geom_point(aes(y=rules_sup10, colour="Support level of 10%")) +
# Plot line and points (support level of 5%)
geom_line(aes(y=rules_sup5, colour="Support level of 5%")) +
geom_point(aes(y=rules_sup5, colour="Support level of 5%")) +
# Plot line and points (support level of 1%)
geom_line(aes(y=rules_sup1, colour="Support level of 1%")) +
geom_point(aes(y=rules_sup1, colour="Support level of 1%")) +
# Plot line and points (support level of 0.5%)
geom_line(aes(y=rules_sup0.5, colour="Support level of 0.5%")) +
geom_point(aes(y=rules_sup0.5, colour="Support level of 0.5%")) +
# Labs and theme
labs(x="Confidence levels", y="Number of rules found",
title="Apriori algorithm with different support levels") +
theme_bw() +
theme(legend.title=element_blank())
let’s analyze the results.
Support level of 10% we identified 7 rules at 10% confidence level and just 1 rule at 20% confidence level. We can’t use this value because there is no frequent associations in our dataset hence making the rules unrepresentative.
Support level of 5% we identify just 2 rules with confidence level as high as 30%.
Support level of 1% we only identify just 2 rules with confidence level of 50%.
Support level of 0.5% we get to discover so many rules with 20 of them having a confidence level of 50%.
In conclusion, we will be going with a support level of 0.5% and a confidence level of 50% to properly represent the dataset.
Execution
We will execute the apriori algorithm with the values we chose earlier — Support level of 0.5% and confidence level of 50%.
# Apriori algorithm execution with a support level of 0.5% and a confidence level of 40%
rules_sup0.5_conf50 <- apriori(dataset, parameter=list(sup=supportLevels[4],
conf=confidenceLevels[5], target="rules"))We can check out the first 10 rules.
inspect(sort(rules_sup0.5_conf50, by = 'lift')[1:10]) lhs rhs support confidence coverage lift count
[1] {ground beef, shrimp} => {spaghetti} 0.005999200 0.5232558 0.011465138 3.005315 45
[2] {frozen vegetables, ground beef} => {spaghetti} 0.008665511 0.5118110 0.016931076 2.939582 65
[3] {frozen vegetables, olive oil} => {spaghetti} 0.005732569 0.5058824 0.011331822 2.905531 43
[4] {frozen vegetables, soup} => {mineral water} 0.005065991 0.6333333 0.007998933 2.656954 38
[5] {olive oil, soup} => {mineral water} 0.005199307 0.5820896 0.008932142 2.441976 39
[6] {frozen vegetables, olive oil} => {mineral water} 0.006532462 0.5764706 0.011331822 2.418404 49
[7] {milk, soup} => {mineral water} 0.008532196 0.5614035 0.015197974 2.355194 64
[8] {chocolate, soup} => {mineral water} 0.005599253 0.5526316 0.010131982 2.318395 42
[9] {cooking oil, eggs} => {mineral water} 0.006399147 0.5454545 0.011731769 2.288286 48
[10] {frozen vegetables, ground beef} => {mineral water} 0.009198773 0.5433071 0.016931076 2.279277 69 Let’s interpret a few rules;
63% of the customers who bought frozen vegetables and soup also bought mineral water.
52% of the customers who bought ground beef and shrimp also bought spaghetti.
56% of the customers who bought milk and soup also bought mineral water.
It makes so much sense seeing mineral water as the common product on the rhs.
Real-World Applications and Limitations
Big businesses leverage the Apriori algorithm primarily for market basket analysis, which helps them identify frequent itemsets and association rules within their transaction data. This knowledge is then used to improve operations, boost sales, and enhance customer satisfaction. Specifically businesses can use Apriori to discover customer buying patterns, optimize product placement, and personalize recommendations.
Limitations
- The Apriori algorithm requires a large amount of memory, which can be a problem when dealing with very large datasets.
- The efficiency of this algorithm goes down when there is a large number of transactions going on through a limited memory capacity.
Conclusion
In this article, we’ve explored the Apriori algorithm, its core metrics — support, confidence, and lift — and how it is used in market basket analysis. With hands-on R implementation, we saw how adjusting support and confidence affects the number of rules generated. Apriori is a powerful technique for uncovering hidden patterns in transactional data, useful in everything from retail strategies to recommendation systems. I would encourage you to practice the hands-on implementation with a different dataset and different values for support and confidence and see where it takes you.
