ML Capstone Project Day 4: Funnel Analysis with Machine Learning
Predicting Conversions and Personalizing Offers
In the dynamic world of e-commerce, understanding how users progress through the buying journey — from browsing to purchasing — is essential. This article presents an end-to-end funnel analysis framework powered by machine learning, enabling platforms to not only understand user behavior but also predict conversions and personalize their offering for higher ROI.
The primary goal of this project was to:
Understand user behavior across funnel stages — View → Cart → Purchase.
Identify drop-off points and segment funnel performance by category and brand.
Predict the likelihood of purchase using machine learning models.
Use these predictions to personalize marketing strategies and product offers.
Dataset Overview
We used a public e-commerce dataset: “E-Commerce Behavior Data from Multi-Category Store”. It captures user interactions over time with the following structure:
Key Columns:
Tech Stack Used
Platform: Databricks (Delta Lake, PySpark)
Language: Python 3
Libraries: Pandas, Scikit-learn, Matplotlib, Seaborn, Imbalanced-learn (SMOTE)
ML Models: Logistic Regression, Random Forest
Get THE BRICK LEARNING’s stories in your inbox
Join Medium for free to get updates from this writer.
Data Storage: Delta Table on Unity Catalog
Machine Learning Implementation
Let’s now walk through the full machine learning pipeline for funnel analysis and prediction.
1. Data Preparation and Funnel Construction
We start by pivoting the dataset to capture session-level funnel behavior.
# Create binary indicators for each event type
funnel = df.pivot_table(
index=['user_id', 'user_session'],
columns='event_type',
values='event_time',
aggfunc='min'
).reset_index()
funnel['viewed'] = funnel['view'].notna().astype(int)
funnel['carted'] = funnel['cart'].notna().astype(int)
funnel['purchased'] = funnel['purchase'].notna().astype(int)We then enrich the funnel with product metadata like brand, price, and category_code, and split category_code into hierarchical levels (e.g., cat_level_1 = electronics).
2. Logistic Regression for Conversion Prediction
Our first model is a baseline logistic regression, which provides interpretability and speed.
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import LabelEncoder
# Encode brand and category
funnel['brand'] = funnel['brand'].fillna('unknown').astype(str)
funnel['cat_level_1'] = funnel['category_code'].str.split('.').str[0].fillna('unknown').astype(str)
le_brand = LabelEncoder()
le_cat1 = LabelEncoder()
funnel['brand_enc'] = le_brand.fit_transform(funnel['brand'])
funnel['cat1_enc'] = le_cat1.fit_transform(funnel['cat_level_1'])
# Feature set
X = funnel[['viewed', 'carted', 'price', 'brand_enc', 'cat1_enc']]
y = funnel['purchased'].values.ravel()
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)
# Train logistic regression
lr_model = LogisticRegression(class_weight='balanced', max_iter=200)
lr_model.fit(X_train, y_train)Logistic Regression Results
from sklearn.metrics import classification_report
y_pred = lr_model.predict(X_test)
print(classification_report(y_test, y_pred))Result: Improved recall for class 1 (purchases) from 6% (baseline) to ~46% with
class_weight='balanced'.
3. Random Forest for Performance Boost
We now shift to a Random Forest classifier to capture non-linear relationships and complex feature interactions.
from sklearn.ensemble import RandomForestClassifier
from imblearn.over_sampling import SMOTE
# Apply SMOTE to balance class distribution
smote = SMOTE(random_state=42)
X_train_bal, y_train_bal = smote.fit_resample(X_train, y_train)
# Train Random Forest
rf_model = RandomForestClassifier(
n_estimators=200,
max_depth=10,
min_samples_leaf=4,
class_weight='balanced',
random_state=42
)
rf_model.fit(X_train_bal, y_train_bal)
# Evaluate
y_pred_rf = rf_model.predict(X_test)
print(classification_report(y_test, y_pred_rf))Random Forest Results
MetricClass 1 (Purchase)Precision1.00Recall0.99F1-score0.99Accuracy99%
We also visualized feature importances:
import seaborn as sns
import matplotlib.pyplot as plt
importances = rf_model.feature_importances_
feat_names = X.columns
plt.figure(figsize=(8,5))
sns.barplot(x=importances, y=feat_names)
plt.title("Feature Importances from Random Forest")
plt.show()Use Case: Personalization with ML Predictions
After building a powerful model, we used it to score all sessions and segment likely buyers:
# Score sessions
funnel['purchase_proba'] = rf_model.predict_proba(X)[:, 1]
# Filter likely buyers (probability ≥ 0.7)
likely_buyers = funnel[funnel['purchase_proba'] >= 0.7]Then, we personalized the experience by recommending top-performing brands in the same category:
# Find top brand per category_code from purchases
top_brands = df[df['event_type'] == 'purchase'] \
.groupby('category_code')['brand'] \
.agg(lambda x: x.value_counts().idxmax()) \
.reset_index()
likely_buyers = pd.merge(likely_buyers, top_brands, on='category_code', how='left')This output is used for real-time offer recommendations and retargeting via email/push notifications.
Business Benefits
Conclusion
Funnel analysis is a powerful method to quantify user journeys — but machine learning takes it further. By predicting user intent and conversion likelihood, businesses can drive smarter interventions, better personalization, and measurable revenue uplift.
This project showed that:
- Logistic regression is a strong baseline for explainability.
- Random Forest + SMOTE delivers state-of-the-art accuracy.
- Real-time scoring enables AI-driven decision-making for offers, UI, and marketing.
If you’re building an e-commerce analytics platform or marketing intelligence dashboard, funnel prediction models are low-hanging fruit with high strategic ROI.
