Author: Aarav Nigam

Introduction

Every search box eventually hits a wall where traditional search stops being useful.

Searching tide powder 100g on Instamart works seamlessly. Exact keywords match product titles — a problem classical search solved decades ago.

Then someone types this:

• Get me ready for beach holidays
• Snacks for a 5-year-old birthday party
• No added sugar biscuits for my diabetic grandfather
• I’m tired

These natural queries derail traditional keyword search:

• No exact match: There is no product named “i’m tired,” nor does the catalog metadata capture such situational intent.
• Misleading keywords: “Get me ready for beach holidays” matched “holidays,” returning party supplies instead of beach items.
• Contextual intent: A query like “snacks for a 5-year-old birthday party” specifies an occasion and an age — neither of which appears on a packet of crisps.

The engine isn’t broken. It’s answering a question nobody asked.

To bridge this gap, we replaced Instamart’s query-understanding layer with an AI search agent that interprets the underlying intent in real time and translates it into relevant results.

Handling intent-driven queries at scale daily, this is India’s first quick-commerce search stack with a live large language model (LLM). It unlocks three capabilities:

• Situational queries: Interprets complex situational requests like “Snacks for a 5-year-old birthday party” without needing explicit product or department keywords.
• Multiple constraints: Narrows results with complex criteria like “No added sugar biscuits for my diabetic grandfather.”
• Multilingual support: Processes Hindi, Hinglish, Kannada, and Telugu in native scripts directly.

Understanding vague queries is straightforward. The real architectural challenge was validation: we assume the model is wrong until it’s verified. One LLM call interprets; subsequent passes verify.

We start with what a shopper sees, then the designs that failed, and the idea most of this system leans on: turning words into numbers so meaning can be looked up.

What the shopper actually sees

Before diving into the underlying machinery, let’s examine the user experience.

The category chips dynamically display broad results across category shelves (e.g., Swimwear, Beach Gear, Sun Care), allowing immediate navigation instead of endless scrolling as shown in the figure.

The structure underneath that right-hand screenshot looks like this:

The harder case is a query with no relevant product in it at all.

The search bar is natively multilingual with AI Search. Because the agent processes semantic meaning rather than English string matches, queries in Hinglish or native scripts like Kannada work out of the box.

Unshoppable queries (e.g., “What is the capital of India?”) receive a conversational reply without product results, maintaining catalog quality and reducing unnecessary overhead.

That’s the third shape of answer this section has shown. First, chips — a broad query fanned out into browsable categories. Then, a single ranked list with a line of explanation — for “i’m tired,” the catalog can’t name the need, but the shelves still can stock it. Now, a plain conversational reply — for “What is the capital of India?” there is nothing to shop, so no products appear at all. The search box never changed; the query did.

Which shape a query gets is decided before a single product is fetched. That decision is where the design starts.

Two decisions before anything else

Not every query should go to the language model, and not every query is a request to buy something. Before anything is fetched, the system settles two questions about each query. They are independent of each other.

Question 1: Which path does this query take? There are three.

Plain search. “milk” and “almond milk” name the thing the shopper wants. Keyword search handles these quickly and well, and there is nothing for a model to interpret, so they stay on the classic path.

AI Search. “Get me ready for beach holidays” and “i’m tired” do not name a product. The shopper describes a need, and someone has to work out which products meet it. This is the conversational search the agent was built for.

Remark. “Who is the president of India” has nothing to shop for, so the shopper gets a conversational reply and no products. The bar for this path is deliberately high. “My friend is in hospital” is not a purchase request on its face, but flowers and fruit are exactly right, so it goes to AI Search. A remark is the last resort, not the first.

Question 2: How broad is the answer? This applies to plain search and AI Search alike. A broad query covers many kinds of product, so the results come with category chips that let the shopper jump to the one they meant. A narrow query points at one kind of product, so the shopper gets a single list and no chips.

“milk” is broad: protein milk, almond milk, and flavoured milk are different shelves, so it gets chips. “almond milk” is narrow, so it gets one list.

“Get me ready for beach holidays” is broad: swimwear, beach gear, and sun care all belong to it, so it gets chips.

“no added sugar biscuits for my diabetic grandfather” is narrow, because the shopper needs one kind of product, so it gets one list.

Conversational or not is about how the shopper asks. Broad or narrow is about how much ground the answer covers. Because the two are independent, both plain search and AI Search can end up with chips or with a list.

Both questions are policy, not linguistics, and both lean deliberately. Routing leans towards shopping: a remark is the last resort. Breadth leans towards restraint: for a described need, extra product types are added only when the query is a real plan, like a trip or a party, and not just because something else could go with it.

Classic search would have served ‘milk’ and ‘almond milk’ all along. The other two need the agent — and they are why it exists.

Knowing which path a query should take is the easy half. Making AI Search answers reliable is harder, and our first three designs for doing that did not survive testing.

Why the obvious approaches did not work

We evaluated three alternative designs that failed in testing:

1. Separate model calls per task: Introduced unacceptable network latency.
2. Displaying raw model outputs: Caused broken UI rails due to hallucinated non-existent catalog terms.
3. Passing full category trees: Led to incorrect pairings between valid shelves and parent categories.

Additionally, early prompts over-indexed on grocery examples, causing the agent to skew heavily toward grocery results even for non-grocery items. Re-framing prompt context quickly unlocked the entire catalog.

Key takeaways: Request minimal model output, verify everything against catalog truth, and ensure framing doesn’t inadvertently restrict capabilities.

What replaced those designs leans on one underlying trick: turning words into numbers, so that meaning can be compared and looked up. It shows up so often in this system that it deserves its own explanation.

Turning words into numbers

Most of our verification pipeline relies on vector embeddings to convert semantic intent into actionable numbers.

An embedding maps text into a numeric vector space based on semantic meaning rather than spelling. Our in-house embedding model understands our exact catalog vocabulary, brand names, and pack sizes.

Storing these vectors enables fast proximity lookups in live requests via a vector store.

That single mechanism does three different jobs here:

1. Map invented/typo terms: Proposed terms map directly to the nearest catalog terms, bypassing explicit spell-checking or exact string matching.
2. Dynamic few-shot retrieval: Fetches the top-3 closest query-answer pairs to populate the prompt with relevant context dynamically.
3. Semantic caching: Reuses verified historical responses for close vector matches.

With that lookup in hand, we can put the pieces together and follow a single query through the system.

One call to understand, two passes to verify

Our production architecture balances deep semantic understanding with strict performance budgets:

1. Single Understanding Call: A primary model call handles routing, intent sizing, query expansion, and attribute extraction in one pass.
2. Parallel Verification: Two independent validation routines verify attributes and expansion terms simultaneously against catalog data.
3. Response Assembly: A lightweight final pass synthesizes validated results into client-ready category rails.

Here is one query, “get me beach wear for men”, moving through all three steps. The last box is what the shopper finally sees:

Consolidating understanding into one primary LLM call reduces latency. Downstream verification uses lightweight lookups or dedicated micro-prompts.

Every proposed term and attribute is checked against inventory; unmatched terms are discarded.

This part is unglamorous and it is the load-bearing wall. The model is allowed to be creative about what to suggest. It is never allowed to be creative about what exists.

Checking terms against the catalog settles what exists. It does not settle where each term belongs in our category tree, and that turned out to be the model’s weakest spot.

Keeping the model out of the hierarchy

This is the piece we’d most want another engineer to take away.

To assign terms to our 3-level category tree safely, we avoid exposing the full hierarchy to the LLM.

Instead, the model evaluates a flat list of shelves solely to remove invalid options. Deterministic catalog mappings then apply the correct parent categories.

The model sees a flat list of shelves and answers exactly one question: which of these don’t belong with this term? It returns things to delete. A lookup table built directly from the catalog’s own data then supplies the parent categories for whatever survived.

This structural separation prevents invalid pairings, minimizes prompt size, and adds virtually no latency overhead.

Takeaway: Give LLMs smaller, evaluative tasks rather than structural data responsibilities. Rebuild structure deterministically.

So far every answer has been built fresh. But the fastest answer is one we have already verified, which brings us to reusing past answers, and to the ways that can go wrong.

Reusing answers without repeating mistakes

The fastest possible response occurs when the model call is bypassed entirely via semantic caching. However, relying solely on similarity scores introduces specific failure modes where distinct intents appear vector-adjacent:

To prevent incorrect cache hits, queries pass through strict deterministic pre-filters:

• Exact match validation: Exact string matches pass through immediately.
• Language consistency: Mismatched scripts or writing systems are rejected.
• Length disparity checks: Large length differences indicate distinct query specificity, and such pairs are discarded.
• Negation handling: Queries containing asymmetric negations (e.g., “sugar free”) are immediately rejected.
• Numeric matching: Cached queries containing numbers (model versions, pack sizes) require exact numerical agreement.

Only candidates passing all pre-filters are evaluated by similarity scoring.

The ordering matters more than any individual rule. A wrong cache hit doesn’t degrade the experience quietly — it serves one shopper another shopper’s answer. So the cheap, categorical checks go first, and the fuzzy one only gets to say yes after all of them have failed to say no.

Whether an answer is fresh or reused, one thing is left to get right: the order in which its results appear.

Ordering results so the breadth shows

Raw model outputs group results sequentially by category. Showing one full category first pushes the remaining broad results off-screen.

We apply diversity-aware interleaving by cycling through category rails to immediately showcase overall result breadth.

This reordering maximizes visual diversity on mobile viewports without altering item relevance.

Ordering shapes what the shopper sees first. The other thing they experience is the wait before they see anything at all.

Designing around the wait

To reduce perceived latency during LLM reasoning, a fast micro-model streams dynamic status updates (e.g., “Searching for cute small clothes for little children…”).

This makes the brief wait legible and reassuring to the shopper without significant system cost.

The lesson generalises beyond search. When you can’t remove a wait, make it legible.

Making the wait legible helps when the system is merely slow. What should happen when part of it fails outright?

Failing in the right places

The whole pipeline runs against a hard latency ceiling, and the budget is nested: each stage asks for what it needs but only ever receives what’s actually left. A slow first call doesn’t blow the deadline — it squeezes everything downstream — and if the clock runs out, whatever finished in time is assembled into a response rather than the request collapsing.

Under strict latency budgets, sub-tasks degrade gracefully based on criticality:

Non-critical failures: Dropped cleanly (e.g., missing status captions or attributes) to deliver partial valid results.

Core understanding failure: Fails explicitly rather than silently serving confident empty states.

The rule that came out of it: for each component, ask whether a wrong answer is worse than no answer. Where it isn’t, degrade quietly. Where it is, fail where someone will notice.

To see how these pieces work together, let’s trace one query from the moment it is typed to the moment the results appear.

A query from start to finish

Tracing “get me beach wear for men” through the full pipeline illustrates how these components interact:

1. Cache Lookup: The query misses the semantic cache, triggering live execution.
2. Understanding Pass: The primary model classifies the query as shoppable and broad, outputting three rails (Swimwear, Beach Gear, Sun Care), six expansion terms, and the attribute “men.”
3. Parallel Verification: The attribute “men” is validated against catalog metadata. Expansion terms are verified against real inventory. Ambiguous terms like “flip flops” have their candidate shelves pruned via the lightweight verification prompt.
4. Interleaving and Cleanup: Results are interleaved across category rails to ensure UI diversity, with any empty rails dropped.
5. UI Delivery: Three populated category chips ship to the mobile app alongside curated product listings.

That is the design in motion. The next question is whether it actually works.

How we judge whether it works

Evaluation relies on two complementary feedback loops:

LLM-as-a-judge: Blind side-by-side scoring across specific rubrics catches regressions before deployment.

A blind judge. When a prompt or model changes, old and new outputs for the same query are scored side by side by a separate model, the two candidates are shuffled and scored against a per-criterion rubric. Routing, keywords, and the explanation line each get their own rubric, because “good” means something different for each. It’s an imperfect grader but a consistent one, and it catches regressions that hand-checking a dozen queries does not.

Funnel metrics: Real-world CTR, add-to-cart, and order conversions validate actual relevance.

Judge scores decide whether to ship a change. The funnel decides whether we were right.

Judges and funnels guided each change along the way. The larger question was whether the agent as a whole was better for shoppers and for the business, which we tested in an A/B experiment.

Impact

A/B testing demonstrated significant performance and business improvements:

1. Order volume boost: Reduced null search results to nearly zero by resolving conversational requests.
2. Latency reduction: Single-pass LLM routing, parallel validation, and semantic caching significantly reduced response times.
3. Multilingual engagement: Organic adoption across Indic languages without explicit marketing.
4. Architectural stability: Deterministic hierarchy mapping eliminated misclassification bugs entirely.

Along the way we picked up a handful of lessons that apply well beyond search.

What we learned

Building and scaling this architecture provided several key technical takeaways for production AI systems:

• Latency lives in round trips, not work: Consolidating multiple model tasks into a single rich prompt yields significant latency savings compared to chaining smaller calls.
• Model output is a proposal, not a fact: Generative outputs must always be grounded and verified against hard catalog constraints before reaching the user interface.
• Scope the model tightly: Use AI for intent interpretation while relying on deterministic lookups for static data structures like category hierarchies.
• Similarity is not equality: Guard vector search and caching mechanisms with explicit categorical rules to catch fine-grained distinctions like negations and numbers.
• Design explicit failure modes: Allow non-critical UI elements (such as captions) to degrade gracefully while failing loudly on foundational understanding calls.
• Verify prompt assumptions: Ensure prompt examples reflect the entire catalog breadth rather than narrow product subsets that unintentionally bias model output.

None of this is finished, of course. Here is what we are working on next.

Limitations and what comes next

Current focus areas for ongoing development:

• Feature discoverability: Guiding shoppers to use conversational natural language queries.
• Sharpening the routing between the language model and classic semantic search, so each gets the queries it’s genuinely better at.
• Voice integration: Enabling complex, hands-free queries.
• Conversational assistant: Evolving into a full-featured in-app shopkeeper (dukaandaar).

Further out, the thing we actually want to build is an in-app shopkeeper — the dukaandaar who knows your household. Not a search box but an assistant you can hand a whole task to: “plan my week’s groceries,” or “top up the staples I usually buy.” Every piece described in this post — reading intent, grounding it in a real catalog, verifying before showing — is a prerequisite for that. A shopkeeper who confidently hands you something you didn’t ask for isn’t one you go back to.

Conclusion

Closing the semantic gap in quick commerce requires combining LLM natural language understanding with strict deterministic validation against ground-truth catalog data.

The model does the reading. The system keeps it honest.

Special thanks to Rutvik Reddy, Devendra Kumar Sahu, Yatish Chandra, Aakarsh Nambiar, Aaditya Khetan, Saurav Singh, Mayank Solanki, and Sunil Rathee for their contributions and guidance throughout this project.

References

• Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
• Query2doc: Query Expansion with Large Language Models
• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

Search That Gets You: Inside Instamart’s LLM-Powered Search was originally published in Swiggy Bytes — Tech Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

1. Introduction

In the on-demand food delivery ecosystem, an app’s Expected Time of Arrival (ETA) is the foundation of customer trust. Promise 25 minutes for a 40-minute delivery, and you leave a customer frustrated. Show 50 minutes for a 30-minute delivery, and that customer might abandon their cart entirely. Balancing under- and over-prediction is a delicate trade-off. To maintain operational efficiency and a competitive edge, the business requires granular, real-time control over this trade-off. The machine learning system powering our ETAs must be capable of supporting that level of precision.

Historically, our system relied on pointwise estimation trained primarily on custom Mean Absolute Error (MAE) loss function. While this setup successfully met global business targets, it struggled to maintain the business metric consistency across regions and segments. Furthermore, it lacked the mechanism needed to optimally manage the tradeoff between under and over prediction errors across segments as per evolving business needs.

To solve this, we built a deep probabilistic modelling system that predicts the entire distribution of delivery time for every order. Using distribution estimation, we gained precise control over regional business metrics and significantly boosted regional compliance consistency without compromising core prediction accuracy. In this post, we will explore why regional metric consistency matters, examine the challenges of our legacy point-estimation setup, and dive deep into how we engineered our deep probabilistic system and overcame its real-world implementation challenges.

2. Background

To evaluate the quality of our delivery ETA, we rely on two primary business metrics as below which must be maintained consistently across regions and segments:

1. Compliance: The percentage of delivered orders where Actual Time of Arrival (ATA) < ( Estimated Time of Arrival (ETA) + Buffer Time). High compliance protects customer trust by preventing delay-induced frustration.
2. Difference of Means: The average prediction bias (mean of ETA — ATA), serving as our primary metric for absolute accuracy. A significantly positive difference indicates systematic over-prediction, inflating promises, triggering demand shaping, and driving customers to competitors.

These metrics have a trade-off with each other. While ML models strive to improve this Pareto frontier overall, business operations require the capability to navigate this trade-off dynamically at a micro level (by region, time-slot, or customer cohort) because:

1. Customer Experience Consistency: A single national compliance average can mask severe regional disparities. Ensuring consistent compliance across every micro-market protects local customer trust and long-term retention.
2. Better Competitiveness: Granular control enables the business to selectively tighten or relax ETA promises in specific regions/cohorts to counter aggressive local competitor moves without risking ETA promise breaches elsewhere.
3. Managing Refund Costs: Severe delays trigger expensive customer refunds due to cancellations. By stabilizing consistency across different regions, the business can directly curb these financial leakages to stay strictly within regional budget constraints.

3. Architectural Choices

Historically, we managed regional-level compliance using static city-level buffers applied on top of pointwise ETA predictions. While this approach was easy to understand and simple to deploy, its coarse granularity introduced severe inefficiencies. Applying a blanket time buffer across an entire city heavily penalizes regular, on-time orders, just to safeguard a small fraction of long-tail delayed orders, drastically degrading our global Compliance vs. Difference of Means trade-off. To achieve a finer control, we conceptualized several candidate solutions before choosing our final approach.

1. Multi-Task Learning with Specialized Heads

A natural evolution from static buffers is Multi-Task Learning, using a shared neural network backbone that branches into dedicated heads:

City-Level Heads: Adding dedicated heads per city captures broad regional baselines (e.g., traffic speeds in Mumbai vs. Chandigarh). However, each head still outputs a single point estimate, forcing us right back to static, manual buffers to manage regional compliance, while loss gradients from one city still interfere with shared representations for others.

Fixed Quantile Heads (e.g., p75, p80, p85 via Pinball Loss): While this predicts global percentiles, a global p80 estimate does not yield 80% compliance in every city due to local variance in delays. If a city requires a p83 estimate to hit its 80% compliance, fixed quantile heads cannot provide it without crude interpolation between p80 and p85.

City x Quantile Heads: Combining both creates a combinatorial explosion (e.g., 100 cities x 5 quantiles = 500 heads). The resulting gradient competition across hundreds of losses severely corrupts shared backbone representations, while smaller cities suffer from sparse data.

The Bottom Line is Multi-task heads remain fundamentally rigid. Any shift in business targets or expansion into new cities requires structural code changes and an expensive, full model retraining cycle.

2. Deep Probabilistic Modeling (Chosen Solution)

Rather than predicting discrete quantiles, we transitioned to Deep Probabilistic Modeling, where the network predicts the parameters of a probability distribution for every order. Probabilistic modeling emerged as the winning paradigm for two fundamental reasons:

1. Loss Decoupling: The model’s training objective (which is Negative Log-Likelihood) is completely decoupled from downstream business metrics and devoid of manual weight tuning.
2. Post-Hoc Optimization across use cases: Because the output is a continuous distribution, business objectives can be optimized from the single model independently after training. We can dynamically extract any percentile (p54, p77 or p89) at inference time for any city or cohort on the fly, achieving precise compliance targeting without modifying the underlying model.
3. Variability Learning: Unlike point-wise models that output a single deterministic estimate, probabilistic models explicitly quantify and learn order-level uncertainty. By modeling spread alongside expected mean, the system dynamically adjusts predictions to reflect fluctuating real-world conditions like weather, traffic or network stress spikes.

4. Data Analysis

Before jumping directly into the modelling, we performed extensive Exploratory Data Analysis (EDA) to understand the delivery time data better. Given delivery times are strictly positive and right skewed, potential distributions to fit were lognormal and gamma. We also added gaussian and exponential for baselining.

The Lognormal distribution secured the best fit because food delivery is an environment of compounding delays between interdependent legs of order journey. The Gamma distribution took a close second, since it models delivery time as a sum of independent stage-wise waits — cooking, driving, hand-off, etc. In practice, though, these stages aren’t fully independent: a delay in cooking often cascades into a delayed pickup and a rushed, less predictable ride. Gamma’s independence assumption misses this compounding effect, which is exactly what Lognormal distribution captures better.

Beyond single parametric distributions, we also benchmarked against Mixture Density Networks (MDNs) using Gaussian components. Surprisingly, a single parametric Lognormal distribution consistently outperformed Gaussian MDNs up to 10 mixture components, while requiring a fraction of the parameter complexity.

Finally, we validated distribution fitment across diverse business-relevant segments (city tiers, distance buckets, peak order hours, etc). Lognormal consistently delivered superior fitment metrics across all micro-cohorts, confirming it as the winner for our deep learning pipeline.

5. Model Design

We modified our existing point-estimation model by updating the output layer to predict the two parameters of the Lognormal distribution: mu (location) and sigma (scale).

To enforce sigma > 0, the model predicts log sigma, which is exponentiated to obtain sigma = exp(log(sigma)). We chose exponentiation over ReLU because ReLU can floor at zero, causing division-by-zero errors, undefined log-likelihoods, and dead gradients. Exponentiating log sigma ensures a smooth, strictly positive with well-behaved gradients everywhere.

We replaced our legacy MAE-based custom loss with Negative Log-Likelihood (NLL). For an observed delivery time x > 0 and predicted parameters (mu, sigma), the point-wise NLL loss function is expressed as:

6. Challenges

Transitioning to probabilistic loss functions in production is rarely plug-and-play. During initial trials with vanilla Lognormal NLL loss, we encountered two immediate training hurdles:

1. Numerical Instability (NaN Predictions): Unconstrained sigma values quickly blew up exponentiation bounds, generating floating-point NaN losses.
2. Variance Inflation (sigma-Inflation): Even after applying basic max clipping to prevent numerical crashes, the model consistently over-predicted sigma. Extremely high predicted variance flattened the distribution curves, rendering point-estimate predictions far less useful.

To understand why the network favored high variance over accurate mean estimates, we analyzed the geometry of the 3D NLL loss surface.

When starting from a high initial loss state, the optimizer sees two distinct trajectories to minimize NLL:

• Path 1 (True Learning): Adjust to minimize | log x — mu | (learning the true location parameter)
• Path 2 (The "Escape Route"): Inflate Sigma, artificially spreading probability mass to lower the loss penalty without refining.

Because gradient descent greedily follows the steepest directional slope, the optimizer takes Escape Route. In early training iterations—when mu is far from optimal, the partial derivative of sigma presents a significantly steeper gradient than the partial derivative of mean. The optimizer takes this escape route, reducing the loss by inflating sigma rather than performing the harder, more valuable work of updating mu.

We explored three candidate techniques to close this escape route:

1. Smarter Capping: Implemented EDA-backed thresholds on Sigma that zero out gradients when sigma enters unphysically high ranges. This blocks the escape route and moves the backpropagation towards true learning.
2. Sigma — Regularization: Added an explicit variance penalty term based on sigma to the NLL loss to penalize over-dispersion.
3. Two-Stage Training: Cold-started the network using MAE loss to converge on mu first, followed by a second phase of NLL training to learn sigma.

While all three approaches successfully eliminated sigma-inflation, we selected Smarter Cappings for production release. It delivered similar stability without adding extra hyperparameter or complicating the training pipeline with multi-stage execution.

7. Offline Evaluation

Evaluating a probabilistic model requires assessing two distinct dimensions: Calibration (are the predicted probabilities mathematically reliable?) and Accuracy (does the model outperform our point-estimate baseline on business metrics?).

1. Evaluating Calibration

To verify that predicted percentiles reflect real-world outcomes, we evaluated two complementary diagnostic tools across global and regional cohorts:

Calibration Curve plots the predicted percentile on X-axis against the actual observed fraction of orders arriving within that percentile threshold on Y-axis. A perfectly calibrated model follows a 45-degree diagonal line.

Probability Integral Transform (PIT) Plot carries a similar underlying information, but it is uniquely sensitive to specific calibration failure modes — such as systemic over-dispersion (U-shaped), under-dispersion (inverted-U), or directional bias. Observing both plots ensured complete calibration confidence.

2A. Benchmarking Accuracy Against a Point-Estimate Baseline

While Continuous Ranked Probability Score (CRPS) is the gold standard accuracy metric for distributional forecasts, our legacy point model outputs a single scalar — making direct CRPS comparison impossible. Furthermore, because the baseline was trained on a custom loss blending MAE with business heuristics, standard point metrics (MAE/MSE on mean or median predictions) yielded inconclusive comparisons.

To construct a like-for-like benchmark, we converted our continuous distribution into single-point predictions by using a segment-level greedy search. For each segment, we selected the exact distribution percentile that matched the baseline’s compliance level while optimizing the Difference of Means (and vice versa).

2B. Shift of Frontier for Compliance vs. Difference of Means Tradeoff

To evaluate performance across all operating percentiles, we introduced a Compliance vs. Difference of Means Trade-off Curve. By sweeping across all input percentiles (p1, …, p99), we plotted the new model’s continuous trade-off frontier. For the legacy model, we generated a comparative baseline curve by adding constant positive/negative time buffers to its point predictions. As shown in figure, the probabilistic model’s trade-off curve strictly dominates the baseline across the entire frontier, delivering a lower Difference of Means at every target compliance level.

2C. The Core Business Goal: Achieving Regional Compliance Consistency

With accuracy and calibration established, we turned to the primary objective of this initiative: enforcing consistent regional compliance. We designed a customized operational metric that measures the percentage of micro-segments (e.g city/time-slot cohorts) hitting their target compliance within a +/- 0.5% margin of error.

The probabilistic model drove a 13% relative increase in segments meeting compliance targets, eliminating regional ETA volatility and giving business operations precise, predictable control over local delivery promises.

8. Productionization and Experimentation

Deploying a probabilistic ML model into a real-time production system requires careful operational alignment and robust online validation before full rollout.

Operational Percentile Mapping: We defined segments that reflect closely with physical operations and business needs and we applied our offline greedy quantile search on those micro-segment level. With this, we achieved fine-grained control to inspect local performance or adjust safety thresholds dynamically.

Experimentation Setup: Validating ETA predictions via traditional user-level A/B testing is notoriously prone to marketplace spillover effects as it violates the Stable Unit Treatment Value Assumption (SUTVA). Because all customers within a geographic zone share the same finite fleet of delivery partners, showing altered ETAs to a treatment group reshapes order distribution, affecting delivery partner availability for the control group.

Standard user-level A/B tests fail in logistics marketplaces due to network interference. Because Treatment and Control users share the same courier pool, changes in Treatment ordering behavior distort logistics conditions for Control users.

The Solution: Shadow Production Mode:

We ran our probabilistic model in Shadow Mode, generating real-time predictions on live order streams without displaying them to users. Since average predicted ETA were held almost similar to production predictions, customer ordering behavior remained unbiased, allowing us to evaluate real-world prediction accuracy without marketplace spillover.

Shadow Validation: Live shadow metrics matched offline backtesting results, achieving target regional compliance consistency.

Pan-India Deployment: The probabilistic pipeline was promoted to serve customer-facing ETA nationwide.

Reverse Shadow Mode: The legacy point-wise model was shifted into shadow mode alongside the live probabilistic engine. This creates an automated, continuous counterfactual predictions — proving daily that the new model sustains superior regional compliance over the old approach.

9. Key Conclusions and Future Outlook

The transition from traditional point-estimate predictions to Deep Probabilistic Modeling marks a fundamental evolution in how we manage delivery time expectations at scale. By predicting full probability distributions rather than single scalar values, we decoupled downstream business objectives from model training, giving us an elegant, zero-retrain control mechanism over regional compliance.

Probabilistic modeling is now the default paradigm across our time prediction ML ecosystem. Following its success in end-to-end ETA estimation, we have migrated all other time-prediction models, from restaurant kitchen prep times to last-mile transit estimators onto probabilistic frameworks.

The ability to query full probability density curves in real time has opened up rich downstream possibilities for delivery logistics. In our upcoming blog posts, we will share the details about how we unlocked multitude business use cases using probabilistic framework.

10. Credits:

Special thanks to Nishant Agrawal for his guidance and Sonakshi Gupta and Prince Raj for their contribution in productionization.

11. References:

Enhancing Uber’s Guidance Heatmap with Deep Probabilistic Models

Improving ETAs with multi-task models, deep learning, and probabilistic forecasts — DoorDash

Using Gamma Distribution to Improve Long-Tail Event Predictions

ETA (Estimated Time of Arrival) Reliability at Lyft | by Rachita Naik

Improving ETA Prediction Accuracy for Long-tail Events — DoorDash

Precision in Motion: Deep learning for smarter ETA predictions — DoorDash

Solving Regional Consistency of Delivery ETA with a Deep Probabilistic Framework was originally published in Swiggy Bytes — Tech Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

Introduction

Predicted lifetime value is one of the most fundamental signals in e-commerce. It quietly powers the long term growth by influencing: how platforms prioritize acquisition, how ad systems optimize bids, how retention programs decide whom to invest in, how incentives are allocated, and even how risk and fraud systems reason about customer quality over time. Underneath many of these decisions is the same question: not just will this user act, but how valuable are they likely to become?

Most growth systems are optimized for the next action: a click, a cart, a first order. But for a marketplace like Swiggy — operating across two distinct verticals, food delivery and quick commerce (Instamart) , that is often too narrow. Two users can look nearly identical in the first few days after signup and still end up contributing very different long-term values. The install to purchase conversion rate in e-commerce is 1%-3%. This number is even smaller in the initial week or first few days. The system had to make high-stakes decisions in a setting where most users generated weak/no early signals, and where the business outcome we actually cared about ‘Lifetime Value’ appeared much later and was concentrated in a relatively small share of users.

Our early attempts made this gap painfully visible. We started with an existing conversion-prediction model - the model produced predictions with a very high MAPE, meaning the predicted numbers were too noisy to put in a production system. This is a well-known challenge in the pLTV literature; point-estimate accuracy on heavily zero-inflated, long-tailed targets is notoriously difficult, and models trained for binary classification objectives (will this user convert?) don’t generalize to continuous value prediction (how much will this user be worth? - regression). The nuances of the Swiggy marketplace significantly amplify this challenge.

The problem wasn’t producing a score; it was producing a score that could survive sparse early signals, distinguish high-value users from low-value ones early enough to change downstream decisions, and remain robust enough to plug into bidding systems.

The Constraints: Why This Problem Is Hard

The zero-inflation challenge. A large share of users place zero orders in their first 30 days. Any model that doesn’t explicitly handle this zero-mass will be systematically wrong, and a model that predicts “everyone is zero” will have a deceptively low MAE but zero business value. You can ace the loss function and still fail the business problem.

Multi-vertical complexity. A user’s value isn’t one number. They might be a weekly Instamart shopper who never orders Food, or a daily Food orderer who occasionally adds Instamart. The model needs to predict both independently. These two verticals have fundamentally different purchase patterns, zero-rates, and spend distributions but are also correlated by the same user behaviours.

Cold-start at the worst possible moment. We predict soon after the user registers, much before the first order, with zero transaction history. The only signals possibly available are acquisition metadata, device characteristics, and geo-context. This is the hardest possible moment to make a lifetime value prediction, and also the moment when it matters most (ad platforms need the signal at install time to optimize bidding).

The Swiggy market reality: two populations, one prediction target. Unlike mature e-commerce markets where most acquired users follow a broadly predictable spending trajectory, the Swiggy market is structurally different. A large share of new users are trial-motivated, they arrive through cashback offers, first-order discounts, or coupon-driven campaigns. They may look identical to genuine repeat buyers at day 1: same cart behavior, same session patterns, but their long-term trajectories might diverge.

This means the model isn’t just predicting a continuous value from noisy features. It’s fundamentally trying to resolve two populations that are observationally similar at prediction time but behave completely differently over the 30-day horizon. If you get this wrong, you either bid aggressively for coupon-burners who will never return, or miss the genuine high-value users because their early signals looked unremarkable.

Data richness as a double-edged sword. Swiggy’s data infrastructure is unusually rich, we don’t just have order history. We have address verification tags, phonetic name matching, geo-hierarchy competition metrics, post-order complaint dispositions by category, SKU-level Instamart category affinity, premium payment instrument usage, fraud signals, and acquisition channel attribution. All available before a single order is placed. The question wasn’t whether there was a signal; it was whether we could extract it without drowning in noise.

The Feature Engineering Philosophy: Everything We Know, Encoded as Numbers

We started with an audit. For every signal available before a user’s first order, we asked: does this correlate with LTV? If it did, it went in. The result was 350+ features across 12 groups. Our guiding principle was: everything we know, encoded as numbers. Here’s a look into some signals:

1. Acquisition Context
Key Signals: Channel (organic, paid Food/IM, alliances, media partners).
Strongest proxy for user intent — organic vs. paid-campaign installs tell fundamentally different stories about motivation.

2. Device & Fraud
Key Signals: Jailbreak/GPS spoof/clone app/screen-sharing flags; sessions per device in 24h; payment carts per session..
Fraud signals double as LTV signals — a GPS spoofer is also a low-LTV user whose geo features are unreliable.

3. Geo-Hierarchy
Key Signals: 4 spatial resolutions (Geohash-8 → Geohash-7 → Geohash-6 → city (building to neighbourhood to block to city): order share, discount rate, new-user density, competitor presence.

4. Post-Order Complaints
Key Signals: Issue rates (missing items, quality, wrong items) at customer, device, and neighborhood level; 7/30/60-day lookbacks.

5. IM Category Affinity
Key Signals: Order flags, counts, bills, cancellations, complaints per SKU category (F&V, Dairy, Cooking Essentials, Munchies, etc.) at customer + neighborhood level.
Category mix is a retention signal — a weekly fresh-produce buyer has fundamentally different LTV than an occasional snack buyer, even at identical 7-day spend.

6. Last-X-Order Behavior
Key Signals: Per-position features: vertical (IM vs Food), outcome, delivery fee paid, tip, inter-order intervals, complaint type, bill amount.
Encodes behavioral trajectory without manual pattern definition i.e the shared trunk learns the sequences.

7. Affluence & Socioeconomic
Key Signals: Premium/gourmet restaurant orders; delivery fee paid without subscription; address tags (apartments, tech parks); premium payment instruments; corporate meal cards.

8. Phonetic Name & Address Matching
Key Signals: Same-sounding names in neighborhood: order volume, cancellation rate, complaint rate; similar-sounding address landmarks.
Catches household-level signals that individual-level features miss — shared buildings where address strings vary (“Flat 3B, XYZ Apts” vs “XYZ Apartments, 3rd Floor”).

9. Cart Drop-Off & Misc
Key Signals: COD abandonment ratio; lifetime order aggregates; temporal features (hour of day, day of week).
COD drop-off signals payment friction that directly affects completion rates and therefore LTV.

In total: 10+ feature groups, 350+ features, every signal we could extract before the first order.

Spotlights

Geo-Hierarchy : Four Resolutions, Four Different Questions — A user’s location is represented at four spatial scales simultaneously.

Geohash-8: new-user intensity at the apartment. Geohash-7: Instamart order share, discount rate, new-user density, 7-day transacting users. Geohash-6: total and completed orders at 7- and 30-day windows, competitor order share. City: new-user order volume, 7-day window.

The ratio of block-level to neighborhood-level orders tells the model how “concentrated” demand is around this specific user. A micro-cluster signal that city-level trends completely miss. A user in a building with high Instamart penetration surrounded by a neighborhood with low penetration is a different signal than either dimension alone.

Post-Order Complaints : Three Levels, One Powerful Signal — Complaint data (missing items, quality issues, wrong items) aggregated at three levels:

Customer: What share of this user’s past orders had issues?
Device: What’s the issue rate on this specific device? Catches shared-device patterns if this phone has complaints across multiple Swiggy accounts, that’s a signal about the user behind it.
Neighborhood: What’s the local issue rate, broken down by Food vs. IM?

A user with a complaint history will likely churn regardless of individual spend as bad experiences drive abandonment. A neighborhood with high refund-to-bill ratios signals systematic delivery quality issues that depress LTV for everyone there.

The Model Architecture:

Given the richness of the features, we deliberately kept the model architecture simple.

Three shared hidden layers, two task-specific heads. No attention, no embeddings, no mixture-of-experts, no transformers. Trained with a custom MAE loss, Adam optimizer.

The 4-Head Insight: Auxiliary Tasks as Free Regularization

As we iterated, we experimented with expanding to four heads, adding order-count predictions alongside LTV for each vertical. The intuition was that predicting how many orders a user would place (a frequency signal) might help the shared trunk learn richer representations than predicting how much LTV (a magnitude signal) alone. We also experimented narrowing the trunk as order-count heads would provide intermediate supervision. The results surprised us: the 4-head model achieved marginally better performance while using 63% fewer parameters (135K vs 363K)

(Part 2 of this series will cover the evolution to probabilistic model architectures)

Results

Why Not Just Report MAE?

A natural first instinct is to by MAE, MAPE, R². The problem is that these metrics are deeply misleading on zero-inflated LTV targets.
>50% of new users place no orders in a typical e-commerce market. A model that simply predicts zero for everyone achieves a better MAE than most non-trivial models while being completely useless for campaign targeting. High MAPE on non-zero actuals is almost guaranteed even for a good model, because LTV distributions have heavy tails where a single outlier user can swing a percentage error by orders of magnitude.

Practically Meaningful Eval
Can the model correctly rank users by their relative value? Ad platforms don’t need the exact value but to say “this user is in the top 20% so bid higher for them.”
So we evaluate via quantile matching and diagonal coverage along with a spearman coefficient: sort users by predicted and true labels into 10 equal buckets; check how much of the population lands in the right bucket. A random model scores 10% diagonal. The goal is to push that as high as possible, especially at the extremes.

We are able to achieve >0.75 spearman in IM and Food respectively, with diagonal coverage of 70–80%.

From Model Score to Campaign Decision

The model’s role in production is to feed directly into Google’s tROAS bidding. The causal chain is: better rank ordering → correct bid differentiation → more budget allocated to acquirable high-value users → better cohort quality at the same spend.

vs. Third-Party Platform

The real test came when we ran A/B experiments pitting against an external pLTV bidding platform. The results:

Our in-house model delivered better retention and higher gross order value per acquired user at zero incremental platform cost. The third-party platform acquired more new users per dollar spent (their campaign was more volume-efficient).

What’s Next (Part 2 Preview)

In Part 2, we’ll cover the model’s evolution from MAE-based point predictions to a probabilistic architecture. This moves the model from “what’s the expected value?” to “what’s the full probability distribution?” enabling the ad platform to make risk-aware bidding decisions in real time. We’ll also dive deeper into the trade-offs, the Kendall uncertainty weighting scheme for multi-task balancing and model metrics.

Decoding Customer Value: pLTV Models for India’s Quick Commerce Delivery Wars — part 1 was originally published in Swiggy Bytes — Tech Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

For a new Delivery Executive (DE) joining Swiggy, the experience is often overwhelming, throwing a world of SOPs, incentive structures, zone logistics, and app workflows at them all at once. This friction is evident in our metrics, but the drop-off isn’t due to unavailable information, as the DEs are exposed to this information in their onboarding and the information is present in sections of the Swiggy DE app (the dedicated mobile app for delivery partners, separate from the consumer-facing Swiggy app); it’s because the information fails to reach them in a way that builds confidence and delivers instant answers.

“The ones who survive the first two weeks? They almost always had an experienced DE nearby who could say — Ignore that for now. Focus on this. Here’s how I did it.”

That’s not a search problem. That’s mentorship. And mentorship doesn’t scale. Until now.

DE Dost is Swiggy’s answer: a GenAI-powered onboarding companion embedded inside the DE app—not a support bot, but a friend with infinite patience, consistent tone, and real-time personalized guidance.

What is DE Dost?

DE Dost lives inside the Swiggy DE app. New delivery executives can ask questions in any language (English, Hindi, Kannada, Marathi, Malayalam, Telugu, Tamil, Gujarati) and get real-time, personalized answers spanning delivery SOPs, earnings and payouts, app navigation, shift management, and support channels.

But what sets Dost apart from a typical support bot is a deliberate product choice: it’s positioned as a companion, not a helpdesk. The bot doesn’t just answer questions; it anticipates what a confused new DE needs to ask next. It adapts its language, escalates its level of detail when you ask the same thing twice, and maintains a conversational tone that feels like chatting with an experienced colleague rather than navigating a knowledge base.

This companion approach means the system must dynamically adjust its response based on the user’s immediate context and the complexity of their question.

Not all questions need the same depth

A DE asking “What is an incentive?” needs a very different answer than “Why did I miss my incentive yesterday?” The system must know the difference and handle each appropriately.

We designed Dost around a 3-level personalization framework that progressively deepens. The level determines the pipeline, the data sources, and the latency budget.

This framework enabled us to support multiple pipelines, diverse data sources, and varying latency budgets. The routing decision between L1/L2 and L3 is made before the first LLM call.

The Graph State: memory backbone of every conversation

When a DE opens Dost, the system assembles their full context before a single LLM call is made. This is why two DEs asking the exact same question can get different answers.

The Knowledge Base is indexed at the Fleet × Payout level. This indexation helps us personalize starting in the cache, not the LLM.

The Preprocessing Pipeline

Before anything reaches the retrieval system, the raw query goes through a cleanup pipeline. This sounds like plumbing — but it’s actually where a lot of the intelligence lives.

Consider what DEs actually type. Conversations are messy. A DE might ask, "When do I get paid?” followed by "What if I miss it?” — the second query is completely unresolvable without knowing what “it” refers to. To a human colleague, it’s obvious from context. To a retrieval system, "What if I miss it?” would return garbage results about missing orders, missed deliveries, anything but payouts. This is the anaphora problem, and it destroys retrieval quality in multi-turn conversations.

For tackling this problem, every raw query is cleaned in three steps:

Why we used embedding similarity instead of an LLM classifier

The routing problem: distinguish general KB questions (L1/L2) from personal diagnostic questions (L3)—without burning latency and cost on an extra LLM call for every single query.

The naive solution is obvious: just ask the LLM to classify the query. “Is this a general question or a personal one?” Add a short system prompt, get a label, route accordingly. Takes maybe 800ms and $0.0001. Not a big deal per query—but multiply by millions of queries a month, and you’ve added a full second of latency to every conversation, plus non-trivial cost, for a task that doesn’t need LLM intelligence at all.

There’s also a subtler problem: LLM classifiers can hallucinate routing decisions. An edge case query like "Kya mera paisa aaya?" is personal—it needs real DE data. But a poorly prompted classifier might see “money” and confidently route it to the general KB about payouts. The DE gets a generic answer when they deserve a personalized one. Trust broken.

The answer was semantic routing with a sentence-embedding encoder. Two routes are defined with seed utterances. At query time, similarity is computed against both, and the highest match wins deterministically, fast, with no hallucination risk.

The two routes are seeded very deliberately:

Route A gets ~140 general onboarding questions spanning every topic in the KB, plus common greetings.

Route B gets a focused set of L3 utterances specifically about incentives and minimum guarantee

The two areas where DEs need real-time personal data. The contrast between routes is sharp enough that the encoder can pick confidently even on paraphrased or informal phrasing.

And because it’s a standalone embedding model — not part of the main LLM pipeline — it can be retrained independently if query patterns shift. No full model retraining, no prompt engineering. Just update the seed utterances and re-embed.

Answer-centric retrieval: find answers, not text

Traditional RAG retrieves text passages and asks the LLM to stitch them into an answer. The LLM does too much synthesis. We flipped the structure.

Imagine you’re looking for “how to book a shift” in a large Swiggy operations manual. A text-chunked RAG system might return the paragraph that mentions shift booking in the context of zone eligibility, another chunk that mentions shift slots in the context of payment tiers, and a third about shift cancellation rules. None of them directly answers the question. The LLM has to read all three, figure out which parts are relevant, and stitch a coherent answer, which is exactly where hallucinations creep in. The model fills the gaps with confident-sounding guesses.

DE Dost’s knowledge base isn’t built that way. Instead of chunking documents into paragraphs, every piece of knowledge is authored as a QnA pair: a question, a direct answer, and a section tag. When a DE asks, "How do I book a shift?”, the retriever finds the chunk that literally answers that question—and the LLM’s job becomes much easier: adapt this answer to the DE’s context and tone, not synthesize something from scratch.

This also makes knowledge base maintenance cleaner. When a policy changes, you update one Q&A pair, not a paragraph buried in a long document.

Hybrid Retrieval & Reranking

Here’s the honest problem with standard semantic search: embeddings are great at understanding meaning but terrible at exact words. Ask for "MG," and the embedding model might happily return results about “milestones” or “minimum orders”—because they’re semantically close. But a DE who typed “MG” means one very specific thing: Minimum Guarantee, the floor earnings amount Swiggy promises. Missing that is not a retrieval failure—it's a trust failure.

So we fused two completely different signals together.

Semantic scoring (cosine similarity) captures intent and meaning. It handles paraphrasing beautifully—"How much will I earn?" and "What is my pay?" point to the same answers even though they share no words.

Lexical scoring captures exact vocabulary. It’s split into two sub-signals: containment measures what fraction of the query’s words appear in the chunk (precision-focused), and Jaccard similarity measures the overlap of unique terms between query and chunk (balance-focused). Together they give exact-keyword matching without requiring a full text search index.

The β parameter is the secret weapon here. Tune it higher when the domain has lots of precise jargon (like Swiggy operations) and tune it lower when queries are more conversational. It’s not a static hyperparameter—it can be adjusted as query patterns evolve, without retraining anything.

Responding in the language DEs actually speak

DEs speak in a fluid mix — a Hindi sentence with the word “incentive” in English. A direct translation of “incentive structure” into Hindi sounds bureaucratic and alien. That’s not how DEs talk.

Dost detects language per turn — not a one-time setting. If a DE switches from Hindi to Kannada mid-conversation, Dost follows seamlessly. All internal processing stays in English. Only the final response gets translated back.

Domain-aware dynamic dictionaries: Swiggy-specific terms — “base pay”, “minimum guarantee”, “incentive”, “milestones”, “onboarding fees” — are tagged before translation so they pass through untranslated. Deep-links to in-app screens are preserved too. The DE hears their language, but the operational vocabulary stays exact.

This translation layer supports 8 languages: English, Hindi, Kannada, Marathi, Malayalam, Telugu, Tamil, and Gujarati—all operating on a single English-internal pipeline.

Teaching DEs what to ask next — without hallucinating

New DEs don’t know what to ask next. A user asking about incentive rules may also need payout timelines, qualification logic, or overlapping incentive math—but doesn’t have the mental model to ask those questions yet.

That anticipatory layer is what follow-up suggestions are supposed to encode. Not UI convenience — actual mentorship logic baked into the product. When Dost answers, "How do incentives work?”, it should already know that the natural follow-up is usually, "Okay, but when do I actually get paid?” or "What happens if I miss the milestone by one order?" ”.

Follow-ups encode mentorship. But they’re dangerously easy to get wrong. Early versions generated plausible follow-ups the KB couldn’t answer. DE clicks → vague response → trust broken immediately.

The damage from a bad follow-up is disproportionate. It’s not just one bad answer—it's the DE learning that Dost’s suggestions can’t be trusted. After that, they stop clicking them. The whole mentorship layer collapses. We treated follow-up quality as a hard correctness constraint, not a nice-to-have.

Hard constraint enforced: every follow-up must be grounded in the KB. And computing this without an extra LLM call required an information-theoretic approach.

The decay is natural: if a DE keeps asking about incentives, incentive-related chunks get consumed turn by turn — the follow-up pool shrinks. When they switch to “shift booking”, the pool resets with fresh, relevant suggestions. Mentorship encoded as math.

From 15 seconds to 4.5 seconds

In the early pilot, Dost’s end-to-end latency sat at 15 seconds. For a DE standing in the sun between deliveries, that’s an eternity. The target was sub-6 seconds.

The original architecture made 5 separate micro LLM calls: intent classification, rephrasing, response generation, follow-up selection, and translation. Each added its own network and inference overhead.

Evaluating what matters in onboarding support

A massive effort went into ensuring not just accuracy, but safety and contextual correctness at scale. Our evaluation framework was designed to replicate the complexity of real-world DE interactions, while our guardrails specifically addressed the critical challenges of latency and hallucination.

LLM-as-a-Judge and Feedback Loop

Responses are scored using an LLM-as-a-Judge setup, providing a quantitative measure for quality:

• Response Correctness: Scored on a 0–5 similarity scale against expected gold-standard answers.
• Follow-up Relevance: Scored on a 0–3 relevance scale, with a hard constraint that all follow-ups must be grounded in the knowledge base.

Beyond the automated pipeline (which tracks macro accuracy trending at 90%+), we maintain a tight feedback loop:

• Daily Review: We hold daily cadence calls with product stakeholders to review live bot responses.
• Micro Tracking: We drill down into failure categories, tracking specific issues like knowledge base gaps, bot misunderstandings, and language-related errors.
• Weekly Insights: Weekly reports highlight frequent DE concerns and poorly performing categories.

This rigorous feedback loop fueled the improvements that we see today in DE Dost.

The Comprehensive Evaluation Framework

To rigorously validate every update, we developed a comprehensive test suite based on a cross-product matrix:

• Breadth and Depth: The suite covers ~200 unique questions across 2 fleet types (shared, IM) and 2 payout types (daily, weekly), creating ~800 test cases per language.
• Multilingual Validation: These tests are run end-to-end across 9 languages.
• Adversarial Testing and Safety: We include contextual queries, paraphrased variants for semantic robustness, and adversarial inputs (such as abusive or off-topic questions) to validate the system’s safety guardrails and content filtering.
• End-to-End Logic Check: Each test case dynamically routes through the correct Knowledge Base path, validating the crucial semantic routing logic as well as the final response quality.

Key Challenges and Engineering Guardrails

The evaluation process exposed several core challenges that required dedicated architectural guardrails:

1. Latency Guardrail: The initial pilot latency of 15 seconds was unacceptable. The fix involved consolidating multiple micro-level LLM calls (for rephrasing, classification, response, follow-ups, and translation) into broad, structured prompts to achieve a target P50 latency of 4.5 seconds.

2. Follow-up Hallucination Guardrail: The system struggled with generating non-answerable follow-ups. This was fixed by enforcing that every follow-up must be derived from the existing, unconsumed knowledge base chunks, a process managed by an information-theoretic approach using KL divergence.

3. Retrieval Accuracy: Traditional text-centric RAG was not effective for operational support. The guardrail was a fundamental switch to a QnA-structured knowledge base (where each chunk is a self-contained answer) combined with a hybrid retrieval score (semantic + lexical matching) to handle Swiggy-specific terms like “MG” and "COD."

The future of onboarding isn’t manuals. It’s conversations that never get tiring.

What DE Dost taught us is that building an AI companion isn’t primarily a model problem. The model is a commodity. The hard part is encoding tacit knowledge — the kind that lives in conversations between experienced and new DEs — into a system that can reproduce it at scale, in the right language, at the right moment, with the right tone.

Every technical decision in this system was ultimately a product decision in disguise. QnA-structured retrieval isn’t just an engineering optimization—it's a commitment to never giving a DE an incomplete answer. Semantic routing isn’t just latency management—it's the difference between a DE getting a generic payout explainer versus a diagnosis of why their specific incentive didn't get credited last Tuesday. Language adaptation isn’t localization—it's respect for how people actually talk at work.

A good LLM answers questions. A great one teaches users what to ask next.

DE Dost is live on the Swiggy DE app, supporting new delivery executives across India.

Author: Rishabh Patil and Aishwary Yadav

Special Thanks to Farzan, Prateek Saxena, Suryansh, Manav K for their contribution and Meghana Negi, Vaishnavi for their guidance throughout this project.

DE Dost: Teaching Machines to Be a Friend on the Last Mile was originally published in Swiggy Bytes — Tech Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

Engineering across all Swiggy Apps has always had a simple problem with a very large surface area: we ship fast, but every release, every production signal, and every operational review touches too many systems.

For a single release manager or on-call engineer, the day could start across Jira, Confluence, Bitrise, Play Console, Crashlytics, New Relic, Google Sheets, ImageKit dashboards, and Slack threads. None of these tools were wrong. Each had the data it was supposed to have. The problem was that the human had become the integration layer.

That does not scale.

Over the last few months, we have been rebuilding ways of working across all Swiggy Apps around a different idea: let engineers stay where the work is happening, and let agentic AI do the repetitive context gathering, cross-tool correlation, formatting, reminders, and first-level analysis.

The result is not one chatbot. It is an operating layer for mobile engineering. Release checklists, app vitals, New Relic observability, CDN and ingestion cost tracking, TRM follow-ups, rollout health, QA automation, and on-call analysis are now increasingly handled through AI skills, MCPs, scheduled jobs, and Slack-native agent workflows.

This is the story of what changed.

The Problem: Operational Load Hidden In Plain Sight

The Swiggy Apps ecosystem is not one app and one release train. It spans multiple business lines and shell apps across Android and iOS. Each team cares about stability, performance, release quality, app size, cost, experimentation safety, and user experience. Each of those has its own telemetry and review rhythm.

Before automation, a lot of high-leverage engineering time was being consumed by low-leverage work:

• Pulling the same vitals every morning.
• Checking whether release tickets had missing QA owners or blockers.
• Generating release notes and checklist summaries.
• Looking up Crashlytics and Play Vitals before ramping rollout.
• Mapping New Relic alerts to owning teams, widgets, banners, APIs, or configs.
• Preparing TRM notes and chasing action item updates.
• Explaining bandwidth or ingestion cost spikes after the bill had already moved.

The team did not need another dashboard. We already had dashboards. What we needed was an assistant that could connect the dashboards, understand our workflows, and show up inside the channel where decisions were already being made.

The Shift: From Dashboards To Agents

We built around a few operating principles.

First, AI should work inside Slack, not as a separate destination. Engineers already coordinate releases, incidents, and TRMs in channels and threads. If the agent can read the same context, run the same checks, and post structured output there, adoption becomes natural.

Second, AI should be connected to real tools. A generic model cannot run release operations. Our workflows use domain skills and MCPs for Jira, Confluence, New Relic, Firebase, Play Vitals, Bitrise, Google Sheets, Appium, Playwright, Databricks, ImageKit, and internal mobile tooling.

Third, humans should stay in control of judgement-heavy decisions. The agent can recommend RAMP, HOLD, or ROLLBACK. It can summarize freshness of crashes, ANRs, adoption, and vitals. But rollout approval remains a human gate.

Fourth, automation must be reusable. A one-off prompt is useful once. A skill, cron, or MCP is useful every week.

That mindset changed our execution speed more than any individual tool did.

1. Release Management From Slack

Release management was one of the clearest places to start because the workflow had a lot of repeatable structure.

We automated major RM tasks through AI agent skills for Android and iOS release readiness. A release manager can now ask for release status, bug bash readiness, blockers, unassigned tickets, missing QA owners, BL-wise grouping, release notes, or release checklist status directly from Slack.

The old flow required a release manager to open Jira, filter by fixVersion, manually group tickets, check status transitions, look at QA ownership, scan blockers, and then format an update for the team. The new flow is intent-driven:

Run Android release readiness for this version.

The agent pulls from Jira, groups the work, identifies gaps, and posts a structured update. It is not just a summary; it is a decision aid.

We also moved repetitive release hygiene into automation:

• Baseline profile updates no longer need manual PR verification in the common path.
• Release checklists centralize APK size, memory leak reports, benchmark results, and other pre-release quality signals.
• Release notes can be generated from the release scope.
• Build creation can be triggered through the assistant.
• TRM action items can be followed up through a skill and cron instead of individual manual pings.

This changes the release manager’s role. The RM is no longer a person doing clerical integration across tools. The RM becomes the reviewer of an already-prepared operational picture.

2. AI-Gated Rollouts

The most visible release automation is rollout health.

For staged Play rollouts, the agent correlates Crashlytics, Play Vitals, and rollout adoption to recommend whether the release should ramp, hold, or roll back. This has already been used live in rollout discussions where health checks at early percentages informed the next ramp decision. One example is a 1 percent to 5 percent ramp on a release version after the assistant found no fresh fatal signal.

The important part is not that AI clicks the button. It does not. The important part is that the agent removes the manual pre-work before the decision:

• What is current adoption?
• Are there fresh fatal crashes?
• Are ANRs moving?
• Are vitals within expected range?
• Are there carry-over issues from earlier releases?
• Does the signal justify a ramp, hold, or rollback?

The final action still sits behind approval. That is the right balance. AI handles the repetitive analysis; humans retain accountability.

3. Daily Observability Without Daily Pulls

Observability used to be highly dependent on who remembered to check what.

Today, teams across all Swiggy Apps have moved toward recurring AI-driven observability crons that post daily updates into team channels. These cover app vitals such as crash rate, ANR rate, and startup time; UX zippiness metrics; top-of-funnel API RTT; New Relic ingestion; ImageKit CDN bandwidth; and scale-up health updates.

This sounds simple, but it changes behavior.

When vitals arrive every day in the channel, observability becomes ambient. The team does not need to wait for a TRM to rediscover a trend. If a startup metric, page load, API latency, or bandwidth number moves, the conversation can start immediately and with context.

We also invested in deeper signals. For example, warm and hot start debugging now has fragment-level information in New Relic, debug traces similar to cold start traces, and lifecycle timing breakdowns from create to resume to first draw. That means startup regressions can be traced to specific screens or fragments rather than discussed as a vague app-level number.

This is the difference between “startup is bad” and “this screen path is responsible, here is the owner, here is the metric, here is the next action.”

4. Alert Triage With Context, Not Just Noise

Alerts are cheap to create and expensive to interpret.

New Relic alerts used to trigger the same set of questions repeatedly: what changed, which widget or API is involved, who owns it, whether it is a real regression, and what the historical baseline says. The agentic workflow now does the first-level investigation.

For ImageKit 404 alerts, the assistant has mapped alerts down to the widget, banner ID, missing asset, and config owner. For frozen-frame alerts on collection pages, it has traced top collection IDs to Gandalf layouts, carousel widgets, banner IDs, and Lottie asset URLs. For network issues, the agent can facet signals by dimensions such as ASN and produce an investigation-ready summary.

Internally, we had already seen the same need in earlier tooling across all Swiggy Apps: New Relic alerts, crashes, and ANRs required manual context gathering across NR, code, logs, Jira, and Slack. The new MCP-backed flow reduces the alert-to-analysis loop from a manual hunt to a structured first response.

The goal is not to pretend every RCA is solved by AI. The goal is to make sure an engineer starts from the second page of debugging, not the first.

5. Cost Tracking As An Engineering Signal

Cost used to appear late in the cycle. A bandwidth or ingestion spike would show up in a bill, someone would ask why, and then engineers would reconstruct the story from dashboards and release timelines.

We changed that by treating cost as part of daily observability.

ImageKit bandwidth, storage, and video processing signals are now tracked in the same operating rhythm as vitals. New Relic ingestion is also tracked and compared so teams can catch volume changes earlier. Our in house Mobile Toolkit adds a self-serve NR cost optimisation suite with drop filters and cost tracking. The broader release platform now brings CI/CD, Play/App Store release tracking, release quality, and cost visibility into one place.

This matters because mobile cost problems often hide inside engineering details:

• A media transform change can increase CDN processing.
• A logging or event schema change can increase New Relic ingestion.
• A missing image transform can inflate bytes received.
• A noisy 404 path can create both alert fatigue and bandwidth waste.

When the daily report includes the why, not just the number, cost becomes actionable. The discussion moves from “the bill went up” to “this version, transform, event table, or asset path moved.”

That is a better operating model.

6. The Long Tail: Skills For Repeated Work

Once the team saw the pattern, the automation surface expanded quickly.

We now have or are actively using skills and MCPs for:

• New Relic query validation and alert investigation.
• Play Vitals analysis.
• Firebase and Crashlytics reports.
• Weekly TRM generation.
• Bitrise build and CI/CD stats.
• Appium and Playwright test generation and execution.
• Perfetto-based performance debugging.
• Google Sheets and Docs automation.
• Figma user journey mapping.
• Engineering code review checks.
• Specialized Swiggy mobile agents for screen exploration, DI wiring, analytics, network layer, and screen-specific code tasks.

The important cultural change is that engineers are no longer waiting for a central platform team to automate every workflow. The people closest to the pain are building skills for their own repeated work, and those skills are being shared across all Swiggy Apps.

That is how automation becomes contagious.

What Changed Operationally

The most obvious improvement is speed. Tasks that earlier required opening several tools now finish in a Slack thread. Some internal examples show New Relic alert analysis in under a minute, UXZ or P0 metric summaries in under two minutes, and custom New Relic metric crons in minutes instead of a manual setup cycle.

But speed is only one part of the story.

The bigger change is reduction in cognitive load. Engineers no longer need to remember every dashboard, query, ownership map, release checklist field, or TRM follow-up format. The agent brings the checklist to them.

We also improved consistency. A manually written release update depends on who is writing it and how much time they have. An automated readiness check follows the same structure every time. Daily vitals reports create the same operating rhythm every morning. TRM action items no longer depend on someone remembering to chase every owner.

And finally, we improved accountability. When observability, release readiness, rollout health, and cost signals are posted in shared channels, gaps become visible early. The team can act before an issue turns into a production surprise.

What We Learned

The first lesson is that the best AI workflows are boring in the right way. They do not need to feel futuristic. They need to save X minutes every day, reduce one avoidable miss every release, and remove one manual chase every TRM.

The second lesson is that AI needs domain tools. A model without access to Jira, New Relic, Play Vitals, Crashlytics, Bitrise, Confluence, and internal release systems is only a writer. A model connected through skills and MCPs becomes an operator.

The third lesson is that approvals matter. For release rollouts, the safest pattern is recommendation plus human gate. The agent prepares the operational view; the owner approves the action.

The fourth lesson is that visibility creates adoption. When an automation posts a useful daily report in a channel, everyone sees the value. When a release manager asks a question in Slack and gets a structured answer in seconds, the next RM tries it too.

The fifth lesson is that small automations compound. A release checklist skill, a vitals cron, a New Relic alert analyzer, a TRM follow-up bot, and a CDN cost report may look small individually. Together, they change how a team operates.

What Is Next

We are still early.

The next phase is to expand AI-gated rollouts to more surfaces across all Swiggy Apps, bring more approvers and guardrails into the approval flow, extend daily observability coverage across shell apps, add per-session network data consumption tracking, and build stronger business-impact overlays for performance and reliability signals.

We also want every repeated operational workflow to have the same question attached to it:

Why is a human still doing this manually?

Not because humans are unimportant. The opposite. Human judgement is too valuable to spend on copying metrics, chasing owners, formatting reports, and switching tabs.

Our goal is simple: make the Swiggy Apps operations fast, visible, consistent, and AI-native. Let agents handle the mechanical load. Let engineers focus on judgement, product quality, architecture, and execution.

That is how we improve speed without losing control. That is how we reduce operational load without reducing ownership. And that is how the Swiggy Apps ecosystem is turning AI from a productivity experiment into a real operating model.

Acknowledgements

This shift wouldn’t have happened without a lot of people quietly doing the hard work. Pratik Sahu ideated and built the agentic AI-driven flows that made this real, with Sambuddha Dhar guiding him. Tushar Tayal put his trust in the team to experiment and take it live — and that trust is what let this move from an idea to something running in production.

And the entire Apps team — the engineers, QA partners, release managers, on-call owners, and platform contributors who kept questioning repetitive work, turned operational pain into reusable automations, and kept building more skills and workflows around it. Their ownership across releases, observability, cost, and quality is what turned AI from an experiment into how Consumer Apps actually works now.

How We Made All Swiggy Apps Operations AI-Native was originally published in Swiggy Bytes — Tech Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

Picture this: You’re a responsible Android developer. You’ve done everything right. You’ve replaced those bulky PNGs with lighter, elegant, scalable SVGs. You’ve patted yourself on the back. You’ve probably even tweeted about it.

Then one day, you open the APK Analyzer and see a familiar face staring back at you — a PNG you never invited to the party.

The Crime Scene

We recently discovered multiple instances where our APK analyzer reported mysterious size bumps. Upon investigation, we found PNG files that we absolutely, definitely, 100% did not add to our codebase.

Alongside our pristine SVG files, there were now multiple high-resolution PNG versions — one for each screen density (mdpi, hdpi, xhdpi, xxhdpi, xxxhdpi).

Lets try to understand this better with one of our SVGs

The Case of random.png

We discovered a PNG, lets call it random.png had snuck into our APK like that one person who wasn’t invited to the party but somehow ends up in all the photos.

Plot twist: We never added random.png. We added random.xml.

So how did it end up in the APK?

🔍 The Root Cause (A.K.A. “It’s Not a Bug, It’s a Feature™”)

The automatic generation of these PNG assets comes from how the Android Asset Packaging Tool (AAPT2) handles vector drawables during the build process.

Here’s what the Android documentation says:

Android 5.0 (API level 21) and higher provides vector drawable support. If your app has a minimum API level that’s lower, Vector Asset Studio adds the vector drawable file to your project; also, at build time, Gradle creates PNG raster images at various resolutions.

Alright, that sounds reasonable… and with minSdkVersion 23, we should be safe. Right?

Upon deeper investigation into the VectorDrawable documentation (you know, the documentation nobody reads until something goes horribly wrong), we discovered the truth:

While basic vector drawable support begins at API 21, certain tags and attributes — like android:fillType and gradients in android:fillColor — were only introduced in API level 24.

If these newer attributes are present in your vector XML, AAPT goes:

> “Oh, you’re using fancy features? Let me just generate PNGs for you.

Let’s look at our random.xml:

See that gradient in android:fillColor? That feature was introduced in API 24. Our minSdkVersion is 23.

Since gradients were used heavily throughout the apps, all those assets automatically qualified for raster generation

Impact

While vector drawable XML files are typically lightweight, often just a few kilobytes, but they can significantly bloat the APK when they trigger automatic rasterization. The problem lies in the fact that rasterization process is opaque and we don’t know in advance how large the generated PNGs will be, and their size can vary significantly based on the complexity of the vector

For example, raster_prone.xml is only ~1KB as a vector asset. However, when rasterized by the build system, it results in a PNG file of approximately ~50KB.

As XML,

In Bundle,

For few assets, it generated PNGs worth ~200KB

The Plot Twist

AAPT2 generates rasterized PNG assets even for unused vector resources.

That’s right. You could have an SVG sitting in your drawable folder, referenced by absolutely nothing, forgotten by everyone, and AAPT will generate PNG versions of it even if you have enabled shrinkResources.

Solution?

The solve is to use VectorDrawableCompat by referencing vector assets via app:srcCompat, particularly for SVGs that include attributes not supported below API 24 and are prone to triggering rasterization.

Why VectorDrawableCompat?

VectorDrawableCompat is part of the AndroidX support library and renders vector drawables at runtime on devices running API levels below 24 without relying on pre-generated PNGs.

Also, we need to explicitly inform Gradle and AAPT that our project intends to use the support library for vector drawables to ensure that Gradle avoids generating density-specific PNGs and relies on the support library to render vector assets at runtime.

🧹 What We Actually Did

Phase 1: Removed unused assets — especially vector files that were triggering rasterization. With these gone, they no longer made it into the build, preventing AAPT2 from generating those unnecessary PNG variants and reducing APK size.

Result — Download size reduced by > 1 MB.

Phase 2: minSDK bump to 24

Result — Download size reduced by 2.7MB

We observed a similar improvement in our shell apps (Toing-2.1MB, Instamart — 2.3MB) also.

References

• Vector Asset Studio — PNG Generation https://developer.android.com/studio/write/vector-asset-studio#pngoption
• VectorDrawable API Reference
  https://developer.android.com/reference/kotlin/android/graphics/drawable/VectorDrawable

About the Author:

An Android developer who has spent too many hours in APK Analyzer.

Acknowledgements

Big thanks to Sambuddha Dhar, Raj Gohil, and Tushar Tayal for always having my back. This milestone wouldn’t have happened without you all! 🙌

How Swiggy Saved 3.7 MB by Solving a Vector Drawable Mystery was originally published in Swiggy Bytes — Tech Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

At Swiggy, we use Protocol Buffers everywhere. Nearly every message exchanged between our backend services and our mobile apps is a protobuf. On Android, the default and most widely adopted way to work with those schemas was protobuf-java, Google’s official Java runtime, with Java classes generated by protoc.

It worked. But when we started profiling our app size, we found that protobuf-java and the code generated by protoc collectively were consuming 6.735 MB of our overall app size. That is a significant chunk, and it became the reason we started planning a migration away from protobuf-java.

App size is not just a vanity metric for us. Every additional MB affects install conversion, update reliability, and the size budgets of multiple Android apps built on the same platform stack. Since protobuf sits in a shared layer, any improvement here would not be limited to one screen or one feature. It had the potential to reduce size across Swiggy, Instamart, and Toing.

Why Not Just Switch to protobuf-javalite?

The obvious answer to a bloated protobuf runtime is protobuf-javalite — Google's trimmed-down version built specifically for Android. Smaller runtime, less generated code, same wire format.

We could not use it. The Swiggy app uses google.protobuf.Any extensively across its schema — it is how our APIs handle dynamic, polymorphic payloads. protobuf-javalite strips well-known types, including Any, entirely from the runtime. Switching to lite would have required rearchitecting a significant portion of our serialization layer.

We needed a library that was smaller than protobuf-java on Android but still supported everything protobuf-java supports, including Any. That led us to Square Wire.

What Is Square Wire?

Wire is an open-source protobuf implementation maintained by Square. It is byte-for-byte wire-format compatible with protobuf-java — same .proto schemas, same encoding. What changes is the code generator and the runtime library.

Instead of Java Builder-pattern classes, Wire generates lean, immutable Kotlin objects. The runtime is significantly smaller than protobuf-java. And unlike protobuf-javalite, it supports google.protobuf.Any and the rest of the well-known types.

Wire also supports JSON serialization out of the box, which is a useful addition — though worth noting that Wire’s JSON parsing speed is significantly slower than its binary parsing. It works, but if you have a high-throughput JSON path, benchmark it before relying on it.

The Generated Code: Before and After

Take this simple proto:

syntax = "proto3";
message CartItem {
    string item_id = 1;
    string name = 2;
    int32 quantity = 3;
}

What protobuf-java generates (Java):

public final class CartItem extends com.google.protobuf.GeneratedMessageV3 {
private String itemId_ = "";
private String name_ = "";
private int quantity_ = 0;
public String getItemId() { return itemId_; }
public String getName()   { return name_; }
public int getQuantity()  { return quantity_; }
public static Builder newBuilder() { ... }
public static final class Builder extends GeneratedMessageV3.Builder<Builder> {
    public Builder setItemId(String value) { ... return this; }
    public Builder setName(String value)   { ... return this; }
    public Builder setQuantity(int value)  { ... return this; }
    public CartItem build() { ... }
    }
}

// To create, modify, encode, and decode:
CartItem item = CartItem.newBuilder()
    .setItemId("item_001")
    .setName("Masala Dosa")
    .setQuantity(2)
    .build();

// Modify (must go through builder again)
CartItem updated = item.toBuilder()
    .setQuantity(3)
    .build();

// Encode
byte[] bytes = item.toByteArray();

// Decode
CartItem item = CartItem.parseFrom(bytes);

What Wire generates (Kotlin):

class CartItem(
    @field:WireField(tag = 1, adapter = "com.squareup.wire.ProtoAdapter#STRING")
    val item_id: String = "",

    @field:WireField(tag = 2, adapter = "com.squareup.wire.ProtoAdapter#STRING")
    val name: String = "",

    @field:WireField(tag = 3, adapter = "com.squareup.wire.ProtoAdapter#INT32")
    val quantity: Int = 0,

    unknownFields: ByteString = ByteString.EMPTY
) : Message<CartItem, CartItem.Builder>(ADAPTER, unknownFields) {

    fun copy(
        item_id: String = this.item_id,
        name: String = this.name,
        quantity: Int = this.quantity
    ): CartItem = CartItem(item_id, name, quantity)

    companion object {
        @JvmField
        val ADAPTER: ProtoAdapter<CartItem> = ...
    }
}

To create, modify, encode, and decode:

// Create
val item = CartItem(
    item_id = "item_001",
    name = "Masala Dosa",
    quantity = 2
)

// Modify
val updated = item.copy(quantity = 3)

// Encode
val bytes: ByteArray = CartItem.ADAPTER.encode(item)

// Decode
val item: CartItem = CartItem.ADAPTER.decode(bytes)

The difference in generated code is visible immediately. Wire produces a straightforward Kotlin class with public properties. No builder boilerplate, no getter methods, no intermediate allocation for construction or modification.

What the Migration Actually Looks Like

Field Names

protobuf-java generates camelCase Java getters. A field called user_id in your .proto becomes getUserId() in Java. Wire keeps the original snake_case name as a Kotlin property, so getUserId() becomes user_id.

Every field access in the codebase needs to be updated. A regex or IDE structural search handles most of it, but budget real time for this — it is the most labour-intensive part of the migration.

Building and Copying Messages

With protobuf-java, every message is constructed through newBuilder(). Wire generates a plain Kotlin constructor instead, so you pass fields as named arguments directly. Modifying a message is a copy() call rather than a round-trip through a builder. The resulting code is noticeably cleaner.

Encoding and Decoding

With protobuf-java, messages serialise themselves: message.toByteArray(), MyMessage.parseFrom(bytes). Wire moves this to a static ADAPTER on each generated class: MyMessage.ADAPTER.encode(message), MyMessage.ADAPTER.decode(bytes). Every serialisation call site in the codebase needs updating.

The Packed Encoding Gotcha

This is the issue to watch out for most carefully, because it produces no compile errors and no runtime exceptions — just a silent wire format mismatch.

The proto3 spec says that repeated scalar fields — integers, enums, and similar — should be packed by default. protobuf-java follows this automatically. Wire does not enable packed encoding for repeated enum fields unless you explicitly annotate the field in the .proto schema with [packed = true].

We ran into this when sending requests that had a proto as the request body. The server was rejecting or misreading the payload — the repeated enum fields were being encoded unpacked by Wire on the client, while the server expected packed encoding as per the proto3 spec. No crash, no obvious error, just the server not seeing the data it expected.

The fix was straightforward: add [packed = true] to the relevant repeated fields in the .proto file. But the issue itself is subtle enough that you will not find it without specifically testing your request payloads end-to-end after the migration.

ProGuard Rules

Getting ProGuard right with Wire took us several rounds of trial and error. The challenge in our case is that we use both binary protobuf and JSON — Wire’s JSON serialization goes through a different code path than binary, and R8 will aggressively strip things that look unreachable from static analysis but are actually needed at runtime.

After going back and forth, these are the rules that give us the leanest app size without breaking anything on the Wire side:

# Wire runtime
-keep class com.squareup.wire.** { *; }
-keepclassmembers class * extends com.squareup.wire.Message {
    public static com.squareup.wire.ProtoAdapter ADAPTER;
    @com.squareup.wire.WireField <fields>;
    <init>(...);
    public *** get*();
    public *** is*();
}
-keepnames class * extends com.squareup.wire.WireEnum
-keepclassmembers class * extends com.squareup.wire.WireEnum {
    public static com.squareup.wire.ProtoAdapter ADAPTER;
    public static ** fromValue(int);
    ** INSTANCE;
    <init>(...);
}
# EnumJsonFormatter checks simpleName == "Companion" to skip companion objects.
# Without this, R8 renames the companion and the check breaks, producing corrupt JSON.
-if class * extends com.squareup.wire.WireEnum
-keepnames class <1>$Companion

A few things worth explaining:

ADAPTER is accessed reflectively at runtime — not through a direct static reference in your own code, but by Wire internally. If R8 removes or renames it, serialization silently breaks.

@WireField annotations are how Wire knows which fields to include when encoding a message and how to map bytes back to fields when decoding. Strip them and your messages come out empty.

fromValue(int) is called during enum deserialization to resolve an integer back to the right enum constant. R8 will remove it if you do not keep it, causing a crash at decode time.

$Companion keepnames is the one that caught us off guard. Wire's EnumJsonFormatter checks whether a class's simpleName is "Companion" to skip it when iterating enum members. R8 renames companion classes when they are not otherwise referenced by name, which causes the formatter to fail and crash. The conditional keep rule targets only companions of WireEnum subclasses, so you are not keeping more than you need.

The Code Generation Pipeline

At Swiggy, we do not use the standard Wire Gradle plugin directly. Our proto schemas live in a monorepo organised into packages, and we wanted to publish one AAR per package to our internal Maven repository on AWS CodeArtifact.

To support that, we built a custom generation pipeline.

The build script scans all proto packages, normalises the file structure (Wire requires that a file’s filesystem path matches its package declaration), runs the Wire compiler once across the entire schema, maps the generated Kotlin files back to their source packages, builds an AAR per package, and publishes to CodeArtifact.

Running the Wire compiler once for everything — rather than per package — is important. Wire needs to resolve all cross-package type references at compile time, so isolating packages causes unresolved type errors. A single pass over the full schema avoids this.

We also run incremental builds: only packages with changed files, and their transitive dependents, are rebuilt on each CI run.

The Result

Switching from protobuf-java to Wire reduced the size of the Swiggy app by 3.52 MB. The savings came from replacing the full protobuf-java runtime and the heavier protoc-generated classes with Wire’s leaner runtime and generated Kotlin models.

The impact was not limited to the Swiggy app. Instamart and Toing also depend on the same protobuf-generated artifacts, so the migration gave us measurable size reductions across those apps as well. That made this more than a single-app optimization; it became a platform-level cleanup across our Android apps.

If you are on protobuf-java and looking to reduce your Android app size without giving up features like google.protobuf.Any, Wire is the right move.

Acknowledgements

I want to thank Sambuddha Dhar Raj Gohil Tushar Tayal for your continuous support throughout this migration. This migration wouldn’t have been possible without you.

How Swiggy Migrated from protobuf-java to Square Wire was originally published in Swiggy Bytes — Tech Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

Android monorepos get expensive and slow as they scale. We re-architected CI/CD caching, trimmed the Gradle graph, tuned the JVM, modernized the toolchain, and finally moved to Apple Silicon — plus specific configs and numbers you can replicate.

Context and Baseline

Our CI/CD pipeline started conventional: git clone → fetch npm and Android dependencies → execute Gradle build → notify; the build phase alone averaged ~35 minutes and repeatedly re-downloaded dependencies with no cache restore while configuration re-ran due to environment-driven invalidations. Each CI minute cost 2 credits.

After targeted optimizations, we cut total time to P90 = 11 minutes and P50 = 10 minutes, and reduced average credits per build from 55 to 43 (~20%).

At ~460 builds/month, we projected ~37% fewer credits and ~37% less wall-clock time, avoiding an imminent $22k/year tier-upgrade by staying on the current plan.

What Was Holding Us Back

• Archive cache irrelevance: as many modules changed, CICD cache decayed fast and reuse dropped.
• Configuration invalidations from system properties and env inputs caused re-configuration even when tasks didn’t change.
• Dependency resolution overhead: Gradle probed multiple repositories for each coordinate, adding latency at our scale.
• Variant bloat: CI compiled non-required variants, inflating the task DAG by thousands of tasks.

Machine constraints created OOMs and noisy variability that masked true performance.

Strategy Overview

We focused on four compounding pillars:

• Cache architecture with branch-aware keys, release/develop fallbacks, and a Gradle Remote Cache for task-level reuse.
• Build graph control to shrink the DAG: reduce inter-module coupling and enforce only required variants.
• Dependency resolution hardening via repository content filtering to avoid redundant lookups.
• Runtime tuning: right-size JVM heap/Metaspace for CI and disable work that helps locally but wastes time on ephemeral agents (e.g., configuration cache).

Optimization 1: Deterministic Caching with Branch-Scoped Keys

We made caching branch-aware and OS/arch-aware, and cached more than artifacts: Android SDK components, project dependencies, transformed artifacts, NPM cache, and NDK.

Restore priority was: current branch → previous release → develop.

# Gradle restore (priority order)
{{ .OS }}-{{ .Arch }}-gradle-cache-v3-{{ getenv "SWIGGY_CACHE_VARIANT" }}-{{ getenv "GIT_BRANCH" }}
{{ .OS }}-{{ .Arch }}-gradle-cache-v3-{{ getenv "SWIGGY_CACHE_VARIANT" }}-release
{{ .OS }}-{{ .Arch }}-gradle-cache-v3-{{ getenv "SWIGGY_CACHE_VARIANT" }}-develop
# Gradle save
{{ .OS }}-{{ .Arch }}-gradle-cache-v3-{{ getenv "SWIGGY_CACHE_VARIANT" }}-{{ getenv "GIT_BRANCH" }}
# npm restore/save
{{ .OS }}-{{ .Arch }}-npm-cache-{{ checksum "**/package-lock.json" }}

We normalized cache variants (release, releaseDebuggable) and gated cache saves to the default branch to reduce fragmentation and cost overhead:

# Decide cache variant and whether to save cache
cache_variant="prod"
if [ "$SWIGGY_APK_APP_VARIANT" = "release" ] || [ "$SWIGGY_APK_APP_VARIANT" = "benchmark" ]; then
  cache_variant="release"
elif [ "$SWIGGY_APK_APP_VARIANT" = "releaseDebuggable" ]; then
  cache_variant="releaseDebuggable"
fi
envman add --key SWIGGY_CACHE_VARIANT --value "$cache_variant"

save_cache=false
default_branch=develop
if [ "$GIT_BRANCH" == "$default_branch" ]; then
  save_cache=true
fi
envman add --key PROJECT_DEFAULT_BRANCH --value "$default_branch"
envman add --key SWIGGY_SAVE_CACHE --value "$save_cache"

Early lesson: continuously saving branch-level cache added overhead that often outweighed benefits on our credit-per-minute plan; using develop/release caches (refreshed automatically on back-merges via GitHub Actions) gave better ROI.

Pricing plan tradeoff

Under the credit-per-minute plan (2 credits/min), the 1–1.5 minutes (sometimes ~3 minutes) to save cache increased direct cost, so we avoided branch-level cache saves. Once we moved to a per-build plan, time overhead wasn’t billed linearly, and we started saving branch cache post-build.

Post-Apple Silicon refresh

After moving to Apple Silicon, we set the base branch cache as the primary restore, then save a branch-level cache after the first run so subsequent, same-branch builds consistently hit single-digit minutes when changes are localized.

Optimization 2: Gradle Remote Cache (Module-Level Reuse)

CICD archive cache staled quickly as more modules changed, so we piloted a Gradle Remote Cache on a small EC2 (Linux, 8GB RAM, 256GB SSD) serving GET/PUT under /cache.

Results from the POC:

• Incremental builds: cache hit is now P50 = 44% and P90 = 56% (latest),

This raised incremental hit rates and stabilized day-to-day developer builds.

Optimization 3: Build Only What Matters (Variants + DAG)

We enforced building only required release variants in CI, cutting total Gradle tasks by ~26% (3399 → 2745).

We also reduced inter-module coupling across 100+ modules to widen parallelism and shorten the critical path — an ongoing effort with tangible impact.

Optimization 4: Repository Content Filtering

On clean builds, Gradle’s multi-repo probing added up. We introduced repository content filtering so Gradle fetched coordinates from the correct repo without trial-and-error, reducing resolver latency for our large graph.

Optimization 5: Right-Size the JVM for CI

We ran controlled trials and converged on Heap = 16GB and Metaspace = 12GB for 32GB runners, balancing GC frequency and pause time while avoiding OOMs.

Representatively, under the develop/release cache plan, end-to-end times routinely landed around ~20–22 minutes when cache hit was healthy.

We also disabled configuration cache in CI after build scans showed ~30 seconds spent saving a cache that’s not reused in ephemeral CI — useful locally, wasteful in pipelines.

Optimization 6: Machines That Don’t Fight You

We moved from Ubuntu 22.04 L to XL to eliminate OOMs; Bitrise’s SOC update (Google → AMD) improved times, and M1 Max trials reached ~15 minutes with availability caveats at the time.

We’ve since moved critical workflows to Apple Silicon (M4 Pro, 54GB), which tightened variance and removed memory headroom issues seen on earlier x86 profiles.

Toolchain Modernization

We bumped Kotlin, AGP 8.10.1, and the Gradle wrapper 8.11.1. Combined with the refreshed caching approach on Apple Silicon, this pushed subsequent, small/localized builds into single-digit minutes.

Results

Baseline vs improvements:

• Before optimizations on x86: P90 ≈ 44m, P50 ≈ 35m.
• After pipeline optimizations on x86: P90 ≈ 26m, P50 ≈ 23m; average credits per build 55 → 43 (~20% lower); ~37% monthly reductions in both credits and minutes; avoided a $22k/year tier-upgrade.
• After Apple Silicon (M4 Pro, 54GB) and cache/toolchain refresh: P90 = 11m, P50 = 10m.

Additional visible effects:

• “Time to start build” dropped from ~4–5 minutes to 30–90 seconds depending on cache source.
• In a comparable live window after rollout, we saved ~5800 credits (~45%).

What We’d Do Again (Best Practices You Can Lift)

• Use branch-aware, tiered cache keys and avoid saving branch caches on every run under per-minute billing; refresh authoritative develop/release caches automatically on back-merges.
• Adopt a Gradle Remote Cache for large modular repos to stabilize incremental builds and raise hit rates.
• Trim the DAG ruthlessly: enforce required variants only; stub or disable non-essential modules in CI to cut tasks.
• Constrain dependency lookups with repository content filtering to reduce resolver overhead.
• Tune the JVM to the runner and validate with CI trials; target predictable GC over edge-of-OOM settings.

Right-size machines and prefer Apple Silicon for Kotlin/AGP-heavy builds where available to compress build times.

Acknowledgements

I am Pratik Sahu from the Android Mobile team at Swiggy and I’d like to thank Sambuddha Dhar, Raj Gohil and Tushar Tayal. This milestone wouldn’t have been possible without their constant help and support

Scaling Android CI: From 44 → 10 minutes — A Deep Dive into Build Time Optimization and Best… was originally published in Swiggy Bytes — Tech Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

Your heart is going faster than it ever has. You’re calculating exit routes. And then you hear it: the slap of a chappal being picked up. You’ve accepted your fate. You’re bracing for impact. And then it lands. BOOM.

Every Indian kid has lived this moment. That cocktail of dread and love that only moms can brew. This Mother’s Day, Swiggy wanted to bottle that feeling. Not as a greeting card, not as a discount banner, but as something you could actually play with.

What We Were Actually Building

We built a game around mom’s love and nostalgia. Land on the page, click a photo of the iconic mom weapons from your childhood — the jhadu, the chappal, the wooden spoon, the belt — scan them, and unlock coupons. The more you find, the more you unlock. Share the fun with friends and get cashback straight to your account. Simple concept, deeply satisfying to play, especially when you’re the one holding the chappal for once.

Simple to play. Not so simple to build.

The moment you land, a fully animated mom is already running behind her kid, ground moving, clouds drifting, the whole scene alive. Every weapon you scan triggers a unique state. Every coupon unlocked reflects a real value pulled from the backend. The share page at the end isn’t a static image either; it assembles itself dynamically based on the amount you unlocked.

Everything is live. The coupon count, the reward values, how much of the final reveal to show, all driven by real-time backend data. Nothing is static, nothing is pre-baked.

When your product manager first walks you through a brief like this, you start thinking: this is going to be a very long sprint. The kind of project that sounds deceptively simple until you start pulling the thread.

Rive is why it isn’t.

What Is Rive exactly?

Before we get into how we used it, a quick primer for anyone who hasn’t encountered it yet.

Rive is a tool for building interactive animations that run in real-time — not pre-rendered videos, not GIFs, not something that just plays on loop and calls it a day. Animations that actually listen. To user input, to backend data, to whatever state your product is in at that exact moment.

The thing that makes Rive genuinely different is the state machine baked into its core. Most animation tools give you a timeline. You choreograph what happens, export it, and it plays back exactly as designed. Rive gives you logic. You can define states, you can define what triggers a transition between them, and the animation responds accordingly — at runtime, in real-time, based on what’s actually happening in your product.

Rive is really two things working together:

• Rive Editor: is the designer’s territory. The interface will feel familiar if you’ve spent time in Figma, with a bit of After Effects mixed in. But the job here isn’t prototyping, it’s building animations that are production-ready. Designers own the motion logic: when something appears, what triggers it, how it responds to state changes. By the time a Rive file reaches engineering, it isn’t a reference to be recreated. It’s the real thing.

• Rive Runtimes: is where engineering picks it up. Lightweight libraries for whatever platform you’re building on — web, iOS, Android, Flutter. The integration is genuinely minimal. You load the file, connect your data to the right properties, and Rive handles the rest.

What this creates in practice is something that’s surprisingly rare on a product team: design and engineering moving independently without blocking each other. Design owns the visual logic and states. Engineering owns the integration. The file is the handoff, and it’s a two-way street.

Why Rive, and Why Not Lottie?

We’ve worked with Lottie before. It’s a great tool for what it does. You export an animation and it plays back on screen. Clean, reliable, predictable.

Lottie is linear by design. It plays a timeline, and that’s the entirety of what it does. The second your animation needs to reflect state, respond mid-interaction, or branch on a condition, you’re no longer animating; you’re manually managing playback as a substitute for logic. Lottie works beautifully when the world is static. When the world isn’t, it doesn’t bend with you.

Rive is built differently. It has a state machine at its core. Animations aren’t just timelines, they’re logic. And it supports real-time data binding natively through something called ViewModel properties: named fields inside the animation that your code can write to directly.

// autoBind: true is the magic flag that connects your ViewModel to Rive
const { rive, RiveComponent } = useRive({
  src: riveSrc,
  stateMachines: "State Machine 1",
  autoplay: true,
  autoBind: true,
});

const viewModel = useViewModel(rive, { name: "ViewModel1" });
const viewModelInstance = useViewModelInstance(viewModel, { rive });
// From here, it's just: get the property, set the value. Rive does the rest.
const totalWinText = useViewModelInstanceString("winSubtext", viewModelInstance);
totalWinText.setValue(data.totalWinText);

Pass it a value. Rive handles the rest. That simplicity is deceptive; there’s a lot of sophistication underneath it.

Beyond the state machine, Rive also handled cross-platform consistency and dynamic asset swapping out of the box — two things that would have added significant complexity had we rolled our own solution.

What It Actually Changed for Our Team?

Here’s the shift that still surprises us most: design doesn’t need to come back to engineering for every visual tweak.

Normally, if something looks slightly off, a label in the wrong place, a transition that feels too fast, an element that needs to hide under a specific condition, that requires dev work. Back-and-forth. Context switching. Waiting.

With Rive, design and engineering each have a clear lane, and neither is blocked waiting on the other. The visual logic lives inside the Rive file. Engineering focuses on passing the right data into the right property names. Both sides move independently.

And when we say visual logic, we mean a lot more than tweaking colors or timing. Design can control when the coupon value appears above a brick on the landing page, how much of the mother’s glare gets revealed as you progress, even when to pause the first time user experience (FTUE) animation if another gratification was happening on screen. All of this sitting cleanly inside the Rive file, zero engineering involvement.

The back-and-forth that typically eats a campaign timeline mostly disappears. It is, genuinely, plug and play.

That’s not us being generous. That’s what it is.

Getting Here Wasn’t Instant

The first time our team used it, on a Holi Blast campaign before Mother’s Day, we learned that the hard way.

Holi Blast Rive implementation — our first attempt at wiring motion to logic

The shift is that Rive asks your motion designer to think like an engineer. State machines, conditions, logic branches — these aren’t concepts that come naturally from a timeline-based animation background. On our Holi Blast campaign, design wasn’t just choreographing motion anymore; the team was building logic. Mapping out when a label should appear, what state triggers it, what happens when the data arrives out of order. That’s a real transition, and it takes time.

On the engineering side, the integration itself is lightweight, but you need to understand how ViewModels and state machines connect before things start making sense.

Early on, we hit a wall: assets loading, data flowing, code clean, but nothing showing on screen. After hours of going back and forth, we found it. The ViewModel wasn’t linked to the artboard inside the Rive file itself. One missed connection, completely invisible from the code. We linked it, loaded the new file, hit refresh, and everything appeared. Thirty seconds to fix. Hours to find.

That kind of problem only gets solved when both teams understand the tool together. Which is probably the most important thing we took from that first project. The Holi Blast pain was, in hindsight, exactly the investment the Mother’s Day experience needed.

One Technical Thing Worth Knowing

There was one specific gotcha worth passing on. Certain triggers, specifically label show/hide behaviors, weren’t updating in real-time the way we expected when wired to numerical inputs.

Before:

const revlabel4 = useViewModelInstanceNumber("revealLabel4", viewModelInstance);
const earned = landingPageData.progress?.cashback?.earned;

revlabel4?.setValue(earned); // raw number — label never responded

The label bound to revealLabel4 wasn't responding live. The value was updating correctly on the React side, but the state machine transition conditioned on that boolean wasn't picking it up in real-time — because earned was a number, and we were implicitly coercing it at the point of setValue rather than binding a proper boolean input from the start.

After:

const revlabel4 = useViewModelInstanceBoolean("revealLabel4", viewModelInstance);
const targetscore4 = useViewModelInstanceString("label4Text", viewModelInstance);

useEffect(() => {
    if (!targetscore4 || isEmpty(landingPageData)) return;
    const earned = landingPageData.progress?.cashback?.earned;
    if (isEmpty(earned)) return;

    revlabel4?.setValue(earned !== 0); // boolean — transition fires correctly
    targetscore4.setValue(format("₹{0}", earned));
}, [targetscore4, landingPageData, revlabel4]);

The fix was straightforward once we saw it. The state machine transitions inside Rive are typed against whatever the designer wired in the editor. Pass the wrong type and the transition silently never fires.

Small in isolation, but it’s the kind of edge case that bites you at 11pm before a campaign launch. Map your trigger types carefully before you wire everything up.

What We’d Have Built Instead (And Why I’m Glad We Didn’t)

The running mom animation alone — character in motion, parallax background, clouds, ground, would have been a significant custom engineering effort without Rive. Sprite sheets, canvas rendering, frame management. A few weeks, realistically, not days.

Stack the dynamic share page reveal, the real-time coupon states, and the personalized reward image on top of that, and you’re looking at a completely different project scope. A different timeline. A different sprint.

With Rive, design authored all of it. We connected the data. The experience shipped feeling polished, reactive, and considered in ways that the actual code behind it wouldn’t suggest. A lot of what would typically live in engineering — the transitions, the state logic, the micro-interactions — lived inside the Rive file instead. That’s the best compliment you can give a design tool.

If Your Team Is Thinking About Rive

Here are the tips we’d share with teams starting out:

Designers: Go through the Rive 101 tutorials on YouTube before opening your production file. Rive will ask you to think about logic, not just how something looks, but when it appears, what triggers it, what state it’s in. That’s a real shift. The tutorials prepare you for it.

Developers: Read the Rive runtime docs for your platform. The integration is genuinely lightweight, but understanding how ViewModels and state machines connect will save you from staring at a blank screen wondering why nothing is rendering.

Both: Spend time on the Rive Marketplace before you start. It shows you the ceiling of what’s possible and often sparks ideas you wouldn’t have had otherwise.

Would We Do It Again?

Yes. And we say that having felt the learning curve.

The first integration is always the hardest. But what’s on the other side is worth getting to: a design team that owns the motion logic, an engineering team that just passes data, and an experience that updates in real-time without either side blocking the other.

The day we shipped, the devs were only sending numbers and strings from the backend. Everything else, the animation, the logic, the feel, was Rive. And somewhere in the world, a kid is picking up a virtual chappal and feels that old familiar dread.

For every mom who ever made us run — this one was for you. Happy Mother’s Day.

To the team that made it possible❤️🙏:

Engineering: Dalchand Sharma, Taruun R Malik, Rooheen Garg, Rahul Dhawani
Design: Srushti Haware, Ezaz Ansari, Spardha, Vishnu Madhavan and the rest of the team

Nostalgia, Coupons, and a State Machine: The Swiggy Mother’s Day Story was originally published in Swiggy Bytes — Tech Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

Introduction

If you have ever filed a complaint on an e-commerce app — uploaded a photo of a damaged packet, reported a wrong item, or flagged an expired product — you were probably expecting a quick, fair resolution. You were not thinking about what happens on the other side. But at scale, every wrong call on the other side either costs the business money or costs the customer their trust, and both compound fast.

At Swiggy Instamart, that single action triggers a chain of decisions that touches customer trust, financial exposure and fraud risk — all at once. We process more than 40,000 such claims every day across five complaint categories: wrong items, expired items, bad quality, damaged items, and packaging issues.

Each claim follows the same pattern: a customer reports an issue, uploads a photo, and waits. Sometimes patiently. Usually not. Before automation, every claim that our rule-based checks and fraud detection models could not resolve was routed to a human agent. The agent would visually compare the customer’s image against the catalog, read the customer’s description, cross-reference order details, and make a judgment call — approve the claim, or deny it.

This worked, but it did not scale: agent costs grew linearly with claim volume, resolution times stretched during peak hours, and inconsistency between agents meant identical claims could get opposite outcomes. There was also a second problem downstream. For claims that required returns, delivery executives (DEs) were expected to verify whether the returned item actually matched the customer’s complaint, often while rushing between deliveries. In practice, most returns were accepted without meaningful verification — leading to inventory leakage.

This is the story of how we built two Vision Language Model (VLM) powered systems that solved both sides of this problem — while being deliberate about where we allowed the model to reason and where we kept the system deterministic.

The Operational Challenges Behind Claims and Returns

On the Claim Side

The existing system had a clear gap. Rule-based checks (fraud signals, internal policies, compliance filters) covered a minority of claims. But for the rest — especially those involving visual evidence , the only path was a human agent.

That created three recurring failure modes:

• Inconsistency — Two agents looking at the same blurry photo of a dented can would reach different conclusions.
• Latency — Customers waited minutes for decisions on a platform designed for instant delivery.
• Scale — At the scale we operate, even marginal improvements in bot resolution rates translate to meaningful cost savings.

The opportunity was clear: If a model could reliably identify the obviously valid cases, we could provide instant resolution without agent intervention — and only route genuinely ambiguous claims to humans.

On the Return Side

The problems on the return side were equally structural but different in nature:

• Operational friction — DEs are optimised for speed and delivery, not quality assurance. Asking them to cross-reference and verify returned items adds complexity to their expectations.
• Incentive misalignment — DEs are primarily measured and rewarded on how quickly they complete pickups and deliveries. Verifying returned items requires additional time and effort, but the workflow does not explicitly incentivise that behaviour. As a result, speed naturally takes precedence over thorough inspection.
• Fraud exposure — Without verification, customers could return incorrect or used items, and DEs could accept items without accountability.

Approximately 10–15% of returned orders were incorrect or in lesser quantity. That is real inventory leakage at scale.

The old RQC process relied on the DEs making a subjective decision — which, given the constraints above, was not working. We needed to move the verification decision away from the DEs entirely, into a system that could verify visually without requiring training or judgment from the person on the ground.

Scope: Two Systems, One Loop

We built two systems that work in tandem across the claim lifecycle:

Image Validation Model (IVM) runs when a customer files a claim. It analyses the uploaded images against catalog images (with context of other signals too, like the comments which the customer has written) and returns a structured verdict: valid, invalid, or ambiguous. That verdict feeds into a business rules engine alongside other signals (fraud scores, confidence levels etc.) to make the final approve, reject, or escalate decision.

Reverse Quality Check (RQC) fires after a claim has been approved and a return pickup is triggered. When the DE arrives at the customer’s doorstep and photographs the item, RQC compares those photos against the customer’s original claim images. If they match, the pickup proceeds. If they do not, the DE receives a specific nudge message explaining what to fix and can retry.

High-Level Architecture

The diagram below shows how both systems fit into the overall claim lifecycle. IVM and RQC share a common design philosophy: the model handles visual understanding, while deterministic code handles all logic, routing, and business decisions.

IVM: The Image Validation Model

Why Visual Evidence Must Come First

The central design principle for IVM was simple but critical: the customer’s uploaded image is the primary evidence. Everything else — comments, product name, category — is context, never proof.

This principle emerged from studying how the old system failed. Earlier versions of IVM would sometimes approve claims based on a compelling written narrative alone, without sufficient visual evidence to support it. A customer claiming “the yogurt tasted sour” with a photo of a perfectly sealed container tells us nothing visually. Taste is not verifiable from an image. A customer claiming “item was damaged” with a dark, blurry photo tells us the evidence is insufficient, not that the claim is false.

The model needed to be more disciplined than a human in this specific dimension — it should never validate what it cannot see.

Why One Prompt Cannot Fit All Five Categories

Early on, we tried a single generic prompt; something on the lines of “look at this image and decide if the claim is valid.” It performed poorly across the board.

The reason became clear quickly. What constitutes valid evidence for a wrong item claim (visual identity mismatch between two products) is fundamentally different from what constitutes valid evidence for an expired item claim (a date printed on a label). These are different visual reasoning tasks that require different evaluation criteria, different failure modes to watch for, and different output structures.

So we decomposed IVM into a LangGraph StateGraph — a directed workflow that routes each claim to a specialised handler based on its category. The routing is deterministic; only the visual assessment within each handler uses the LLM.

Every claim flows through four stages:

1. Ingesting and pre-processing the request
2. Downloading customer and catalog images
3. Routing the claim to the appropriate handling mechanism
4. Finally aggregating the outputs into a standard decision format.

What Each Category Handler Does Differently

Bad Quality, Damaged and Packaging — These three share a common architecture: compare the customer image against the catalog image to assess physical condition. The core principle is that visual evidence always takes precedence over customer narrative. Non-visual claims (smell, taste, “device not powering on”) are classified as out of scope.

Expired Items — Focused on extracting manufacturing and expiry dates from product labels. The hardest vision task in the system — labels are rotated, printed in small fonts, in regional languages, and often partially obscured.

Wrong Items — Compares visual identity between what was ordered (catalog image) and what arrived (customer image). Special handling for categories where catalog images may not be representative of the product, and explicit rules against marking claims valid for quantity, ripeness, or quality issues that are not actually wrong-item scenarios.

The Expiry Challenge: Why Two Extraction Paths Are Better Than One

Extracting dates from product labels turned out to be unreliable with a single approach. VLMs are impressive at reading text in images, but they occasionally hallucinate dates or misread characters in small, low-contrast label text. This matters because for expiry claims, the entire decision hinges on correctly reading a printed date. A single wrong character changes “2026” into “2028” and transforms an expired product into a valid one.

To maximise both precision and recall, we run two extraction paths in parallel for every expiry claim:

Path 1 (VLM Direct): GPT-4o receives the customer image and attempts to read manufacturing and expiry dates directly from the label.

Path 2 (OCR + LLM): AWS Textract runs OCR on the image, producing a raw text string. A second GPT-4o call then parses this text with a specialised prompt that handles misspelled labels, computes expiry dates, also helps in disambiguating when multiple dates are present.

Merging Logic: OCR-extracted dates take priority over VLM-extracted dates, because OCR is empirically more reliable for reading small printed text, which is also confirmed by our internal audits of the extracted dates.

Multi-Image Aggregation: Where Deterministic Code Meets LLM Output

Customers often upload multiple photos for a single claim. Rather than asking the LLM to reason across all images simultaneously (which increases hallucination risk and prompt complexity), we process each image independently through the model and then apply deterministic aggregation rules in code:

• If any image shows valid evidence, the claim is treated as valid.
• Image relevance guardrail: if the aggregated outcome is “valid” but none of the images were deemed relevant to the claimed issue, it overrides to invalid.
• Cross-category detection: if the model consistently classifies all images as belonging to a different category (e.g., filed as “bad quality” but every image shows a wrong item), the claim is flagged as miscategorised.

This was a deliberate architectural choice. The LLM does what it is uniquely good at — visual understanding and nuanced judgment. Code does what code is good at — aggregation, boundary conditions, and consistency. The aggregation logic is testable, debuggable, and perfectly deterministic. Putting it inside an LLM call would make it probabilistic and harder to audit. In short, we use the LLM as the artist and the code as the accountant — because letting the artist handle the taxes never ends well.

Model Selection: The Experiment That Shaped Our Architecture

IVM started with GPT-4o across all categories. As we accumulated production data, we benchmarked Gemini 2.5 Flash to evaluate whether we could reduce cost and latency without sacrificing accuracy.

The results were clear for visual categories: Gemini matched or exceeded GPT-4o at significantly lower cost and latency. The table below shows production error rates across all visual categories:-

Gemini consistently delivered lower error rates across both FP and FN, while also being cheaper and faster per call.

For Expired Items, the story was different: GPT-4o proved more reliable on expiry-date extraction, where reading small printed text accurately mattered more than speed. A model that confidently reads ‘MFD: 2022’ as ‘MFD: 2026’ is not helping anyone, least of all the customer who just ate that yogurt

The current routing is:

• Gemini 2.5 Flash → Bad Quality, Damaged, Packaging, Wrong Items
• GPT-4o → Expired Items

All of this matters only if the system can respond fast enough in real-world conditions. Customers expect near-instant resolutions, and DEs cannot stand outside a doorstep waiting for a slow model response.

Deployment

IVM runs as a real-time serving endpoint. The moment a customer submits a claim, it handles fetching images, running the model on each one in parallel, aggregating the results, and returning a decision, all within seconds. Full tracing (through MLFlow) gives us visibility into every step of that process: which category handler ran, what the model saw in each image, how much it cost, and where it failed when something went wrong. That observability is what makes continuous improvement possible at this scale. We have also set up alerting jobs that trigger when the error rate exceeds a predefined threshold within a specified time period. This is essential for aggregate-level monitoring.

RQC: Reverse Quality Check

The Verification Problem

After a claim is approved and a return is initiated, the DE arrives at the customer’s doorstep to pick up the item. The DE photographs the product. And now the system has to answer one question:

Is the item the DE is picking up the same physical product the customer showed in their claim images?

This is not a trivial image comparison. The customer’s photo was taken indoors, from one angle. The DE’s photo is taken outside — often in harsh sunlight or a dim stairwell — from whatever angle their back pain and delivery schedule allow. The same blue towel looks navy in one photo and sky blue in another. The same sealed packet looks flawless in customer lighting and dented under the DE’s flash. A lot of deliveries basically turn into that viral “is the dress blue or gold” debate from 2015.

The system needs to distinguish between:

• Genuine mismatches — the customer is trying to return a different (or older or cheaper) item
• Apparent mismatches caused by environment — same item, different lighting, angle, or packaging state
• Genuinely unverifiable scenarios — DE photographed a sealed opaque bag, or the image is too blurry to verify

How a Return Verification Flows Through RQC

RQC runs as an AWS Lambda function, triggered the moment the DE uploads all the photos:

1. We download both image sets (customer images and DE images), and construct a single prompt with all images.

2. The VLM evaluates and returns a structured response: Match, Not Match, or Ambiguous — along with a confidence score and reasoning.

3. The result is returned to the DE’s app in real time.

4. If Not Match or Ambiguous: the DE receives a contextual nudge message and can retry with a new photo.

Model Selection

We benchmarked three models for RQC on an evaluation set with human-verified labels. Gemini 2.5 Flash led on accuracy while also being the cheapest and fastest option. GPT-4o came close on accuracy, but at 13x the cost and significantly higher latency, while Claude Sonnet 4 trailed on both accuracy and cost efficiency.

Gemini 2.5 Flash won on the accuracy-cost-speed trifecta. It handled multi-angle and partially visible product images well by relying on colour, shape, and text cues together.

The caveat: it occasionally overanalyses subtle visual noise — lighting reflections or glossy textures can tip it into a wrong call.

GPT-4o was reliable and consistent but too slow at P90 (30.6s) for real-time use, and 13x costlier per claim. It also follows stricter content policy filters — refusing to process certain categories (e.g., tobacco items) — which creates coverage blind spots.

Claude Sonnet 4 was the fastest overall, but its visual reasoning fell well short — frequently misidentifying objects entirely (e.g., a badminton racket as a stick), making it unsuitable for precision product matching. Our confusion matrices confirmed this: Claude produced nearly 3x more false “Not Match” calls than Gemini on the same evaluation set.

The choice was clear: Gemini 2.5 Flash went to production.

The Prompt: 11 Core Principles

Unlike IVM (which compares against a canonical catalog image), RQC compares two real-world photos — both imperfect, both taken in uncontrolled environments. The prompt encodes 11 principles that emerged directly from production cases. A few key ones:

• Focus on core invariants — Branding, logos, shape, and fundamental product design. Not colour (which shifts with lighting) or minor text (often unreadable in phone photos).
• Ignore low-confidence features — Colour differences from lighting, partial text, condensation, and minor packaging creases must not trigger a “Not Match.”
• Packaging state invariance — Product inside packaging in one photo and outside in another is still a Match if the core item is identifiable.

Each of these principles exists because of a specific failure pattern we observed in early production. Lighting differences alone accounted for the majority of initial false “Not Match” decisions.

The DE Nudge System

When RQC cannot confirm a match, the DE does not just see a generic failure. They receive a specific and actionable instruction telling them exactly what went wrong and how to fix it:

“Photo is dark or blurry. Go to a bright place and focus the image.”

“Item is inside packaging. Take it out and click again.”

“Item is inside delivery bag. Take it out and click again.”

“Photo angle not match. Click from the same angle as customer’s photo.”

“Item is different. Click photo of the same item customer reported.”

After receiving a nudge, the DE can retry — take a new photo following the guidance and submit again. This loop continues up to 2 attempts before escalating to manual review.

The key insight: around 60% of initial “Not Match” or “Ambiguous” results aren’t caused by actual product mismatches — they’re caused by bad photos. Giving the DE specific feedback and another chance dramatically improves the final match rate, which directly affects the linked customer experience.

Automated Prompt Optimisation

The 11-principle prompt was not written once and frozen. As we accumulated labeled production data (human-verified Match/Not Match decisions), we adopted an Automated Prompt Optimisation pipeline to systematically improve RQC accuracy.

The approach: we define a metric which we want to improve upon (False Not Match % — cases where the system incorrectly flags a legitimate return as a mismatch), compile a labeled eval set from production, and let the optimiser iterate on prompt instructions. It refines principles against the eval set until the metric improves.

This moved our False Not Match rate from 23% down to 17% — a meaningful reduction, because every false “Not Match” means a DE is incorrectly told the item does not match, triggering unnecessary retries and friction in the return flow, at the same time impacting the customer experience too.

The key insight here is that, at a certain scale, manually iterating on prompts hits diminishing returns. You stop being able to intuit which phrasing change will move the needle. Automated optimisation treats prompt engineering as a search problem (because letting gradient descent do the heavy lifting is much easier than pretending you can hallucinate the perfect adjective.)

Operating at Swiggy’s Scale

Building the model is only one part of the job. Operating it reliably at 40K+ decisions per day is equally important.

Observability is what makes continuous improvement possible at this scale. Every IVM and RQC decision is fully traced: which category handler ran, what each image returned, confidence scores, cost per call, and end-to-end latency. When a wrong decision surfaces through a customer complaint or an audit, the trace gives us a complete reconstruction of exactly what the model saw and why it decided what it did. Without that, debugging at this volume is guesswork.

Cost is a first-class concern. At 40K+ decisions a day, even small per-call differences compound significantly. The Gemini/GPT-4o routing split in IVM and Gemini selection for RQC were not purely accuracy decisions; each was an accuracy-cost-latency tradeoff validated against real production data, not a benchmark.

Resilience is non-negotiable for RQC specifically. A DE standing at a customer’s doorstep cannot wait for a slow or failed model response. If the system is unavailable, the claim escalates to manual review rather than blocking the DE in place. The failure path is designed to be invisible on the ground.

Finally, neither system was deployed and frozen. Production data continuously feeds back into prompt refinement, eval sets, and model benchmarking. RQC launched with a False Not Match rate of 60% — the model was overcautious, flagging legitimate returns as mismatches. Iterative prompt work brought that to 23%, and automated optimisation brought it further to 17%. That trajectory is only possible if the infrastructure to capture, label, and act on production failures is built from week one.

Results

• Customer Satisfaction score: up by 6%
• Image Validation tickets: weekly ticket volume for Image Validation has decreased by 20,000. This significant reduction directly improves our time-to-resolution and elevates the overall customer experience.
• RQC Accuracy: 98.5%
• RQC Tickets solved without human intervention: >5000 per day
• RQC Business Impact: The reverse QC system has significantly reduced fraud-related losses and inventory leakage by ensuring returned items are verified before being accepted.

The system has been processing 40K+ claims daily in production for over 6 months, with continuous prompt and model improvements driving incremental accuracy gains.

Conclusion

The biggest shift was not that we added a VLM to the claim flow. It was that we changed the architecture from a binary “send to agent or auto-resolve by rule” into a visual evidence system with a clear division of responsibility. Different claim types get fundamentally different evaluation criteria. The model’s output is one input among many into a financial decision, not the decision itself. DEs receive actionable feedback and can retry rather than hitting a dead end. And the model stays in its lane — vision stays probabilistic, while logic, routing, and financial decisions remain deterministic.

The old system relied on humans to perform visual verification at scale. The new system relies on AI for visual reasoning, while reserving humans for genuinely ambiguous cases. The key architectural decision was knowing where not to use the model.

Author: Harshit Juneja

Contributors: Yash Khandelwal, Aishwary Yadav and Jayanth

Special Thanks to: Prateek Saxena, Meghana Negi, Rachna Singh and Shaurya Sindhu for their guidance throughout this project.

Building Production-Scale Visual AI for Claims and Returns at Swiggy Instamart was originally published in Swiggy Bytes — Tech Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.