Sitemap

Software Engineering Dose #6

6 min readApr 1, 2023

--

Hey folks, Back with another edition of Software Engineering Dose.

Link to the previous edition:

In this edition we will be discussing:

  1. Optimizing Myntra’s Pricing System for Serving Millions of Traffic
  2. Fantasy Service: The Great Cache Game
  3. How ads indexing works at Pinterest

Optimizing Myntra’s Pricing System for Serving Millions of Traffic:

In this article, Myntra Tech shares how they scaled up their pricing system and eventually reduced the latency from 150ms to 30ms.

Press enter or click to view image in full size
Pricing system archi

Looking at the above diagram, we have sellers creating discounts for their products and the created events flow to the Discount injector service which injects them into Redis and Mongo which will eventually be used by the Discount fulfillment service to serve downstream services.

The main hurdle in scaling this issue is that currently fetching discounts for multiple products, we make a synchronous call for each product to Redis. So, for bulk operations, this is causing latency to increase. As the total time would be the sum of all sync operations. One more issue with this is too many network calls which will have a total round trip time(RTT) of n*(RTT)

So, the first approach they discussed is Command Batching and Flushing. In this, we turn off Redis auto flush mode in the connection to configure a batch size and flush manually thus reducing the network calls.

But since the connection is shared across threads, if one thread invokes flush, the commands added by other threads are also flushed which results in sending more than the configured batch size. This causes more load on Redis servers as we have no control over the batch size

So, in the second approach, Command Batching and Flushing with connection pooling is discussed. So, instead of sharing the connection across the threads, we create a pool of threads for our use case. This mitigates the above issue but the new problem is at high loads we would run out of threads and if we increase the thread count, Redis has to manage too many connections which result in higher IO and CPU.

In the last approach, Command Batching Through Lettuce Command Interfaces is used. In this, we maintain a queue of commands to be executed using Lettuce’s commands interface and we flush it when required or configure a batch size. This solves the problem in the first approach where we might be sending more than the required batch size.

As in, the queue is automatically configured to flush when the size reaches a specific batch size.

Ref :- https://lettuce.io/core/release/reference/#command-interfaces.batch

Considering all, the last approach was selected.

Fantasy Service: The Great Cache Game

The Winzo tech blog discusses the challenges they faced while scaling the application for their specific pattern. We all know that in fantasy games, the pre-match time when we get info regarding toss and squads, we quickly log in and update our teams based on that. This surge lasts only for a while but the traffic pattern can go from normal to scary in seconds.

Get Sriteja Parimi’s stories in your inbox

Join Medium for free to get updates from this writer.

So, handling that part is really crucial. At the start, Winzo just had this.

Press enter or click to view image in full size
Initial architecture

Here they faced challenges with cache stampede on DB which we all know as the Thundering herd problem. Simply put, when the key is not available on Redis, all requests come to the DB at the same time putting a load on it.

Based on their query pattern, they figured most requested APIs are the homepage, match info and contests list. As this info is mostly static, adding a cache layer before DB makes sense and that’s what they did.

Press enter or click to view image in full size
After adding Cache before DB

Here, Redis started to break. And we can’t just throw money at it as Redis scaling has limitations.

Each shard in a replication group has a single read/write primary node and up to 5 read-only replica nodes, which is the maximum number of replicas allowed.

Then came the next great idea, IN-MEMORY CACHING.

Press enter or click to view image in full size
With in-memory cache added

Then cache stampede again at Redis. But why?? Well, all servers have pretty much the same TTLs and they all are expiring at the same time. What can we do? Just add some randomness. Every server sets its TTL at TTL+(random 0–10%) of TTL. It solved the problem for then.

But it introduced another problem. Data inconsistency. When a user refreshes, it may go to a server that still has old data and that is not a good UX.

So, we have to figure out a way to communicate changes back to servers as soon as they happen. Like using a Queue. They have used Redis Pub/Sub for this.

They added a cron with schedule X — jitter where X is TTL at the server. This ensures local cache is updated before it could expire.

Press enter or click to view image in full size
Cron architecture

This article felt like this is how a typical application should work its way through. If we want to build a fantasy app tomorrow, we can’t just copy the same architecture as the cost would be too high to maintain. If there are not that many users, you would be losing money on the infra. So, always look at the traffic pattern and query pattern.

How ads indexing works at Pinterest

Pinterest Engineering shares how they improved Ads indexing to reduce the update-to-serve latency of the system. Update-to-serve latency means the time it takes for an update to be ready to serve. It’s not like a traditional system where we update and it shows up already.

Ads are served based on different params set by the Ads business like filtering criteria and ad budget. So, the update has to be processed before we serve that ad.

So, they built a Real time incremental pipeline and batch processing pipeline.

Press enter or click to view image in full size

Rest of the article explains each of these components in detail. Interestingly, they also shared some of the techincal discussions they had behind the scenes. For example, why not merge some of these components etc.

That’s it for this time folks.

Thanks for reading!

--

--

Sriteja Parimi
Sriteja Parimi

Written by Sriteja Parimi

Software Developer. Writes about software engineering. Constant learner.