Building an Anonymized Recommender System

This post features Part 2 of my mini-series The Azubi Experience. I will be describing my experience building an anonymized recommender system. I will mostly describe how I cleaned messy e-commerce data, handled missing products, built a sparse CF model, and deployed a hybrid recommender for real-world use.

Recommendation systems quietly power your favourite shopping platforms. Whether it’s Amazon, Jumia or Jiji, the goal is always the same:

Recommend the right item to the right user at the right time.

I built my first recommendation engine and let me be honest from the start:

This was the toughest machine-learning project I’ve attempted so far.

Why?
Messy data, anonymized products, missing items, and incredibly sparse interactions set the stage for a deep dive into real data science work.

Let me walk you through it.

1. Working With Anonymised Data

The dataset had:

• Hashed user IDs
• Hashed product IDs
• No product names
• No images
• No real-world metadata

Data of such kind are great for privacy, but a huge limitation for:

• Sanity checks
• Interpretability
• Demo visualisation
• Validating relevance

Still, anonymisation is common in the real world, so learning to work around it was valuable.

2. The Three Datasets (and Their Problems)

i. Events dataset

This was the core dataset containing the following interactions:

• view
• add-to-cart
• transaction or purchased items

ii. Item properties dataset

Completely anonymized. Couldn’t use attributes at all.

iii. Categories dataset

Also anonymized beyond usability.

After exploring the items, I realised something serious:

Most products in the catalogue had no interactions at all.

This meant:

• A huge portion of products were “dead items”
• Only a tiny percentage were ever viewed or purchased
• This amplified the sparsity problem even further

Example:

Total products: 20,000,000
Available products: 650,000
Products with at least 1 interaction: ~5,000
Products with purchases: <1,000

This imbalance makes personalization much harder.

3. Extreme Sparsity and Cleaning

Even among the active items, interactions were extremely repetitive:

Get Andrew Mensah-Onumah’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

For example:

User A viewed item 1331 (60 times)
User B viewed item 2119 (45 times)

The following were clear after exploration:

• Bots generated many interactions
• Some users viewed hundreds of items in seconds
• Duplicates were everywhere
• Purchases (the most valuable signal) were rare
• Most users had 1–3 interactions total

Cleaning involved:

• Removing bots
• Removing exact duplicates
• Filtering unrealistic activity
• Collapsing repeated events
• Removing users with fewer interactions

This improved the matrix but didn’t eliminate sparsity, only 1–2% of the matrix had any data at all.

4. Choosing a Modelling Strategy

My first approach was to use a deep learning model, which I felt will scale well to the dataset. I opted for the Alternating Least Squares (ALS) model from the Python Implicit library. The model did not converge due to:

• Too many dead items
• Too few meaningful interactions
• Implicit models generally struggle with extreme sparsity

So I moved toward building a simpler model manually. I built a user-item matrix and followed up with a model to utilise the matrix for predictions.

5. Building the Sparse User–Item Matrix

This matrix became the backbone of the project. Below is the matrix makeup:

• rows = users
• columns = items
• values = count/ weight of interactions

But because most items had zero interactions, the matrix looked like:

user1: [0,0,0,1,0,0,0,0,...]
user2: [0,0,0,0,2,0,0,0,...]

It was extremely sparse and noisy — but workable.

6. The Baseline Model: Sparse Collaborative Filtering (CF)

I implemented the simplest collaborative filtering method:

Dot-product similarity on the sparse matrix

score(item) = user_vector · item_vector

It worked, but there were two big issues:

1. All Users Got the Same Recommendations

Due to sparsity and repetitive patterns, many users had identical interaction vectors. Thus, the model produced identical results. Every user saw the same “Top 5 items” and there was no user personalization at play.

2. No Handling for New Users (Cold-Start)

What happens if a new user arrives with zero interactions? Pure CF can’t compute anything in such situations.

Solution: A Hybrid (CF + Popularity + Randomness) Model

To handle both problems, I built a hybrid model combining:

i. Collaborative Filtering (CF)

Personalized scoring when user history exists.

ii. Popularity Model

When users have no history, recommend:

• most interacted items
• most purchased items
• most co-occurring items

This solves cold-start.

iii. Controlled Randomness

Even with CF, sparsity caused repetitiveness. So I blended the CF scores with a small random factor:

scores = alpha * scores + (1 - alpha) * noise

Where:

• alpha = 0.9 → 90% deterministic, 10% noise
• Prevents identical recommendations
• Increases diversity
• Makes the system feel more realistic

The deployed app uses:

• Pure popularity when user has no history.
• CF + randomness when user has history.

7. Solving the Demo Problem: Missing Product Images

Since real item metadata was anonymized, I could not show real product names nor images. A deployed app would look boring and unusable. To address these problems, I downloaded a Kaggle e-commerce product dataset, sampled 50 images, and mapped my anonymized items to these.

This allowed me to showcase product cards, image-based UI and a real-feeling of recommendations. These inclusions made the demo more realistic and meaningful.

8. Final Lessons

What worked well

• Sparse collaborative filtering for personalization
• Popularity model for cold-start
• Controlled noise for diversity
• Mapping anonymized items to sample images
• Realistic user experience without full metadata

What was unexpectedly difficult

• Extreme sparsity
• Bot-generated interactions
• Lack of real item attributes

Final Thoughts

This project wasn’t just about building a recommender. It was about learning how recommendation systems actually behave under the rough, imperfect conditions of real-world data.

I learned that:

• Good systems are hybrids, not single algorithms
• Deployment exposes weaknesses you didn’t know your model had
• Sometimes the smartest engineering trick is adding a tiny bit of randomness

For more detailed insights, including notebooks and a local host app version, you can explore the GitHub repository . It is my hope that this post will prove useful to anyone who comes across it. Looking forward to my final post on The Azubi Experience, stay tuned.