Beyond the noise

Every day, millions of people scroll through the social media feeds, the recent research from June 2026 in Demandstage suggests “An average individual spends about 2 hours and 21 minutes daily on social media” and our perfect customer is one among them, but how would we find the “Signal in the Noise”?

“Effective Advertising” is to cut through the digital clutter and connect with the right people at the exact right moment and transform a casual scroll to real, measurable results!

Effective social media targeting for reaching the right users

For an analogy, imagine you’re hosting a party, and you need to make a guest list. Curating the guest list is targeting and sending out the invites refers to activations.

Similarly, social media platforms use first-party (1P) data like videos/reels, content interactions, and keywords to track users’ behavior and interests. And this data is used by advertisers to activate ad campaigns to target specific curated audiences.

Campaigns for effective targeting at MiQ ,use third-party (3P) data which helps us find the relevant audience. This 3P information can be captured by pixels embedded on their websites or apps they interact with or from user-declared data like their home-city or the IP address that every internet connection is tagged on to. For example Meta Pixel or LinkedIn Insight Tag and Geo-Location for Snapchat, Instagram and TikTok.

For effective social advertising, advanced platforms like MiQ Sigma takes this a step further. By ingestion and analysis of over 700 trillion multi-dimensional signals across watching, browsing, and buying modes, cross-channel technology removes the guesswork from social activation. This bridges the gap between raw data and fluid, platform-agnostic campaign deployment.

Choosing the right social media platform for your business

Did you know that each social platform affects the user’s mindset differently?

And by adapting your ads to those mindsets can drive better results to your business?

There are various types of targeting provided by multiple social platforms . Meta provides targeting based on behavior, retargeting and lookalike audiences because you’re reaching the audience in a relaxed time. LinkedIn provides professional identity and skill-based targeting when they are browsing about their work industry or looking out on career growth. X provides keyword targeting and event targeting. Reddit provides subreddit and community-based targeting to target people while they are tracking real time news or exploring niche communities. TikTok and Snapchat provide engagement and real-world signal and Snap Map (Geo-Location) based targeting.

User personalization in social media targeting

YES, user personalization is mandatory! Because what is an advertisement if it doesn’t feel like a mirror of the user’s interests? To engage the user effectively, creative needs a conversational,1:1 tone that directly communicates the value of purchasing the product.

The research from WhiteHat says “Companies excelling at personalization generate 40 percent more revenue from these activities than average performers”.

To support personalization and keep relevance as a ranking factor, targeting is moved from manual to automatic with advanced social media algorithms.
Meta’s Advantage+ or TikTok’s Smart+ algorithms act as strict gatekeepers to decide which ads get seen and when.

By personalizing at a user level, matching their immediate intent, you unlock a “Relevance Bonus” which means, social media awards highly relevant ads at better placements at lower costs . Which results in lower Cost Per Click (CPC), maximizing Ad budget and fostering brand loyalty. This level of ad personalization is where MiQ excels, turning outreach to high converting 1:1 engagement.

The platform playbook: Matching use cases to mindsets of users

According to a research blog from MarinSoftware, “Modern marketers must understand how each social media platform differs from its competitors and find the appropriate channel to reach their intended audience.”

According to the data from Sprout Social, the social landscape is more crowded than ever, with Facebook alone hosting over 3 billion users. This makes precision targeting a financial necessity.

For B2B roles, LinkedIn’s 1 billion professional members offer a high-intent environment where 80% of users drive business decisions.

Conversely, when we talk about B2C roles, Instagram’s high follow rate makes it a premier platform for product discovery.

Reddit users are more interested in new product information and 41% of these users are more likely to be early adopters of new tech.

X serves as “Digital Town Square” where users spend about 26% more time engaging with ads that align with real-time news.

In brief, consumers change their mindsets entirely depending on which app they engage with. At MiQ, we align these shifting mindsets with the interconnected programmatic to social pipeline divided into watching, browsing and buying.

These clusters are categorized based on the mind space of users:

• Watching mode (TikTok/Instagram): Users are in an entertainment-first state of mind.
• Browsing mode (Reddit/X): Users enter into the active research and news gathering mode.
• Buying mode (Meta/Commerce Ecosystems): In this space the journey of users is from discovery to purchase mode.

By targeting specific users within these clusters, rather than treating the platforms as monolithic blocks, advertisers can move from ‘interrupting’ to ‘Integrating’ into their most trusted communities.

References:

1. Mckinsey Official Document:
   https://whitehat-seo.co.uk/blog/boosting-sales-with-personalization#:~:text=According%20to%20McKinsey's%20January%202025,fail%20to%20deliver%20personalised%20experiences.
2. Tiktok Official Document:
   https://posteverywhere.ai/blog/how-the-tiktok-algorithm-works
3. SproutSocial Document:
   https://sproutsocial.com/insights/social-media-statistics/
4. Demandstage Document:
   https://www.demandsage.com/average-time-spent-on-social-media/
5. Research Blog for different social media channels from Marin:
   https://www.marinsoftware.com/blog/understanding-the-different-social-media-platforms-which-one-fits-your-business

Akshata Ubrani is a Product Analyst at MiQ by profession and a curious explorer by nature. When she’s not turning data into insights, she enjoys travelling, connecting with people, and bringing colour to life through art, craft, and painting.

The “Why, What, and How” of modern social media targeting was originally published in MiQ Tech and Analytics on Medium, where people are continuing the conversation by highlighting and responding to this story.

The real problem isn’t a lack of data

In programmatic media planning, the hard part is rarely finding data. There is plenty of it. The real bottleneck sits in the workflow, in turning scattered signals into a recommendation someone can act on before the moment passes.

A lot of planning time still goes into gathering inputs, working out what actually matters, and stitching everything back together into something practical. That is why retrieval-augmented generation, or RAG, only solves half the problem. RAG is the technique of pulling in relevant documents so a model can ground its answer in real context, and it is very good at that. However, planning is not just about finding information. It is about deciding what to do with it.

Why RAG alone hits a ceiling

When a question is simple, retrieval is often enough. Planning questions are rarely simple. A good system has to judge whether a request is straightforward or messy, whether it needs external tools, whether it should dig deeper across several rounds, or whether a full strategic workflow is warranted.

It is less about a model finding the right paragraph and more about the wider system choosing the right path. Put plainly, RAG helps a model read, while a thinking layer helps a system choose.

From information retrieval to decision support

The difference is easier to see side by side.

Media planning is rarely a one-shot lookup. It is usually a chain of smaller decisions tucked inside one bigger question: what matters, which tools to use, how deep to go, and what the final output should look like.

What a “thinking layer” actually does

Think of a thinking layer as the coordinator between a question and its answer. It reads the request, picks the right path, pulls in whatever tools are needed, and turns raw outputs into something usable.

The key idea is that not every prompt deserves the same treatment.

The agent first checks the request’s intent, specificity, and required evidence: a clear one-step question is treated as Simple, a strategic brief as Complex, and a request that depends on a tool as data/tool lookup. For example, “What is RAG?” stays simple, “Build a media plan for Freshpet” is complex, while “Compare Freshpet buyers using purchase data” routes to the relevant data tool.

A simple ask stays light. A strategic ask triggers a richer reasoning flow.

In the deeper flow, a supervisor coordinates the reasoning. It takes the brief, breaks it into smaller research tasks, and spins up helper agents on the fly to work through them in parallel. The number of helpers is not fixed; it depends on how complex the question is. A narrow question should not fire up the whole machine, while a broad planning brief usually should. This pattern is not just theory either. Each sub-agent works from a shared brief that carries the goal, key evidence, decisions, and unresolved questions forward. The supervisor compares disagreements against the available evidence and confidence levels with citations and human approval provides a final safety net.

Why this matters for the business

This is more than a prettier diagram. It cuts wasted effort. When the system picks the right path early, people stop reformulating prompts, rerunning workflows, and gluing outputs together by hand. The payoff is concrete: a planning brief that takes a lot of time and involves plenty of back-and-forth, turns into a clear opportunity for real time savings and measurable business impact.

It also makes the output more useful. A retrieved answer is fine, but a guided recommendation is better, and speed only counts when the result is clear enough to act on. On top of that, it sets a stronger base for scale, because systems built on memory, delegation, and structured coordination hold up far better as workflows get longer and messier.

The hard part isn’t the model

A lesson that keeps coming back is that the model is only one piece. The harder work lives around it: coordination, keeping track of state, staying reliable, and being able to explain itself. That is the gap between a slick demo and something people actually trust when the task is ambiguous, context-heavy, and operationally complex.

Summary for sellers

The example below shows the final response the solution produces: a seller-ready summary generated at the end of the planning flow.

Closing thought

For complex programmatic media planning, the real leap beyond RAG is not about pulling in more context. It is about building systems that can read a planning problem, judge how deep to go, and hand back something closer to strategic guidance than raw output.

That is what an agentic thinking layer is here for. It turns AI from a smarter search box into a more dependable planning partner, and when teams are asked to do more with less, that is worth far more than incremental gains in retrieval alone.

References

• LangChain: Open Deep Research
• LangChain: Deep Agents
• LangChain Docs: Subagents

Bala Saatvik is a Data Scientist at MiQ interested in AI and Agentic AI. He enjoys exploring how intelligent systems can solve real-world challenges. Outside work, he loves travelling, spending time with his pets, and reading about technology and new ideas.
Prakhar Yadav is a Lead Data Scientist at MiQ, focusing on data science, AI, and scalable product solutions. Outside work, he enjoys gaming, riding, building DIY projects with microcontrollers, and exploring new technologies.

Beyond RAG: Engineering an Agentic Thinking Layer for Complex Programmatic Media Planning was originally published in MiQ Tech and Analytics on Medium, where people are continuing the conversation by highlighting and responding to this story.

In most of the design discussions, the moment someone says, “We need to build this for scale,” the conversation quickly shifts to technology. Should we use Kafka or microservices? Do we need Redis? What about Kubernetes?

While those are valid discussions, systems rarely fail because teams pick the wrong technology. More often, the real problem lies in the initial assumptions: the system will only handle a few thousand records, APIs will always respond quickly, reports will generate in seconds, or a single database will be enough.

Those assumptions are usually reasonable at the time. But as products evolve and new requirements emerge, they begin to break down. That’s when systems start showing their limits.

Over time, we realise that building for scale is less about choosing the right technology and more about developing the right engineering mindset.

Here are a few thought processes that are good to consider when starting architecture design:

1. Think about what is actually growing

When we talk about building for scale, we often use the word growth without stopping to ask what is actually growing. The answer varies from one system to another. For a consumer application, it might be millions of users. For a data platform, it could be terabytes or even petabytes of data.

We ran into this while building the report scheduling module. Initially, all our attention was on the APIs serving the UI. As adoption grew, however, the real bottleneck wasn’t user traffic. It was the increasing volume of reports being generated and scheduled every hour. What started at roughly 100’s of scheduled reports per hour grew to ~1000, putting sustained pressure on the reporting pipeline.

That shift changed how we approached scalability, leading us to introduce back-pressure mechanisms to regulate asynchronous processing and maintain system stability.

2. Every design has an expiry date

“What happens if the usage becomes ten times bigger?” Not because we expect 10x growth anytime soon, but because it forces us to question the assumptions we’ve made. A database query that completes in 100 milliseconds today with 10 million rows may take several seconds once that table grows to hundreds of millions of rows.

The same applies to service dependencies. Calling a notification service synchronously may work at 100 requests/second, but at 1,000 requests/second, even a temporary slowdown can create backlogs and increase application latency. We moved our notification service to an asynchronous queue with dedicated scalable workers, which led the application to absorb spikes and process events at a controlled rate.

The goal isn’t to solve tomorrow’s problems today. It’s to understand where the design is likely to break so we’re ready to evolve it when the time comes.

3. The bottleneck is usually somewhere else

One of the most common mistakes is optimizing the wrong thing. It’s easy to look at a slow system and conclude that we need more servers, but the application itself often isn’t the problem. The real bottleneck could be a missing database index, a slow third-party dependency, network latency, or a queue that’s backed up.

A system’s performance is usually determined by its weakest component.

While optimising our APIs, we initially focused on the application layer. However, after measuring each stage of the request, it showed that query execution accounted for ~80% of the response time. Optimising table partitioning and indexes therefore delivered a much larger performance improvement than application-level optimisations.

4. Assume things will fail

When designing a system, it’s easy to assume everything will work as expected, that the database will always be available, APIs will respond on time, and the network will behave. Production environments have a way of proving those assumptions wrong.

Today, I almost automatically think about failure scenarios:

• What happens if the API times out?
• What happens if the queue is unavailable?
• What happens if the job fails halfway through?

Good systems aren’t the ones that never fail. They’re the ones that fail gracefully and recover quickly.

We learned this while re-architecting our campaign event-processing pipeline. Rather than assuming every event would succeed, we designed for failure from the start, that is adding retries, prioritising user-triggered events, ensuring ordered processing, and automatically cleaning up stale data. This made the system far more resilient and predictable under load.

5. Scaling isn’t just about traffic

Most of the scalability challenges don’t come from traffic, they come from changing business requirements. A new customer needs a custom workflow, another integration gets added, and the once-clean architecture starts carrying the weight of those decisions.

Designing configuration-driven systems, using feature flags, or defining integration extension points makes it much easier to accommodate new requirements without changing the core business logic.

In the reporting module, new data sources and insight types were being introduced regularly. Instead of building custom code for each one, we moved to a configuration-driven architecture. This reduced the effort required to onboard new report/insight types while keeping the core reporting engine largely unchanged.

6. Observability before optimization

We often try to optimize systems before understanding how they’re actually behaving. Before improving performance, we need visibility into latency, throughput, error rates, queue depth, and resource utilization. Without that, we’re mostly guessing.

Observability isn’t just an operational concern; it’s a scalability concern.

As part of an internal re-architecture initiative, we built a real-time observability dashboard for our event-processing pipeline. Previously, identifying where failures or backlogs were occurring could take hours of investigation. With visibility into incoming, in-progress, completed, and failed tasks, we could quickly identify bottlenecks and focus optimization efforts where they had the most impact.

You can’t effectively scale a system you can’t see.

In summary, the technologies will change. The frameworks will change. The architectural trends will change. But the habit of questioning assumptions is timeless, and that’s the real foundation of building applications that not only scale but continue to evolve as the business grows.

Chandan is an Associate Engineering Manager based in MiQ’s Bengaluru office. Outside of work, Chandan enjoys traveling, going on long drives, and is an avid photographer. He loves exploring new places and capturing those experiences through photography.

Building applications that scale: an engineering mindset was originally published in MiQ Tech and Analytics on Medium, where people are continuing the conversation by highlighting and responding to this story.

In Programmatic Advertising, Bidding is one of the most important levers when it comes to optimizing the campaign. As campaigns scale across the market even a small inefficiency in setting bids can lead to huge cost leakage and can directly impact media cost and return on investment.

One of the world’s largest independent Demand-Side Platform — The Trade Desk (TTD)offers Value-Based Bidding (VBB) which transitions campaigns from volume-based to value-based metrics, and Predictive Clearing (PC), a bidding feature designed to win ad impressions at the lowest possible cost. The fee incurred to reduce the gap between what you would have paid and what actually cleared in the auction is known as Predictive Clearing fees. If this PC fee is not managed correctly, this leakage can scale across campaigns and markets.

In Kokai, TTD’s AI-driven media buying and programmatic advertising platform, VBB can lift the effective bid closer to max bid. Max bid (ceiling bid) is one of the levers that is set by the trader with base bid (floor bid). The inflated effective bid is then reduced by PC. The difference between the elevated bid and clearing bid is called savings, and the platform charges a small portion as PC fees.

Built on MiQ’s analysis of thousands of TTD campaigns spanning multiple verticals and campaign types, we built BOAT, an adaptive bid optimization solution that recommends bid changes to reduce PC fees while protecting delivery.

The model operates on a fixed cadence, a learning loop and not a one time application system.

Pipeline stages:

• Stage A — Ingest Data: With delivery reports and bid settings we focus on those entities where PC fees have a decent share in media cost associated with that specific ad group.
• Stage B — Feedback: Feedback reverts the action if a loss in delivery is detected, otherwise the measured steps are continued in the next iterations. The learning/history is used to auto tune coefficients to make optimization engines adaptable to market behavior.
• Stage C — Optimization: Optimization engine takes into account current bid setup, minimum floors, bid cut caps per iteration, and relation with observed media cost per mille of impressions (mCPM) to recommend smart bids.
• Stage D — Activation: Recommended bids from optimization and feedback decisioning stage is taken in account and activated via TTD’s DSP api.

Technical backbone:

Derived Metrics: Spread (max bid — base bid), Headroom (max bid — mCPM), PC fee is approximately 15% of PC savings, PC impact ratio = PC fees / Media cost.

Optimization engine

The two branch recommendations are based on the relationship between base bid and clearing CPM:

Clearing CPM is below base bid (Overbidding):

Action: Trim base bid towards clearing CPM maintaining some buffer, and scale down the spread by trimming max bid keeping PC fee severity in consideration.

Algorithm:

New base bid = clearing CPM × (1 − α)

New max bid = min (clearing CPM + β × spread × (1 − pc impact ratio) , old max bid)

Clearing CPM is at or above base bid (VBB Inflation):

Action: Instead of cutting both bids, shrinking max bid according to headroom and PC impact ratio while keeping the base bid unchanged to reduce PC savings.

Algorithm:

New base bid = old base bid (unchanged)

New max bid = old max bid − (γ × headroom × pc_ratio)

Feedback and guardrails

Feedback: The internal coefficients (α, β and γ) in Cases 1 and 2 control the aggressive and conservative nature of the engine and these are auto-tuned against each iteration. Feedback decisions include revert (impression drop versus threshold), reduce further (next iteration) or hold (hold bids as impact is minimized).

Guardrails: These include policies to protect ad groups by considering bid cut caps, a base bid floor, rate limits, partial data and valid ad group setup.

Machine Learning based extension to BOAT

As BOAT is built on pure heuristics which doesn’t take into account the actual relationship of PC fees with campaign setup. To overcome this we have this extension whose primary goal is to learn how PC impact relates to bid settings.

Pipeline Stages:

Feature engineering is the crucial part of this approach. Derived features like win rate (impressions/bids), deal type, headroom (max bid- mCPM), spread share of base bid ([max bid-base bid]/base bid) and so on help the model to understand the PC fee relation better. The data is used as training data for model keeping supervised targets as PC impact ratio.

yᵢ = f(x₁ᵢ, x₂ᵢ, x₃ᵢ, …, xₙᵢ)

Where,

x₁ᵢ, x₂ᵢ, x₃ᵢ, …, xₙᵢ = win rate, deal type, headroom, spread share, and so on

yᵢ = PC impact ratio

Optimize/Inference:

After the extra trees regressor model is trained, using predictions the optimization stage evaluates candidate bids defined by intelligent sampling. The bid pair with least predicted PC fee is chosen while other features like clearing CPM and spend are fixed or held constant. Like in the figure above we can see the bid pair of $2.83 and $5.3 is the most optimal as predicted PC fee is the least.

Success for both approaches was defined practically by lowering PC impact ratio and sustaining delivery.

Observations based on live testing

To conclude, while the current approach can be refined, the results have shown that smarter bid tuning can significantly help reduce PC fees while protecting delivery. Since this solution is designed for all campaign types, after thoroughly analyzing the performance, we can extend the solution to handle each campaign type separately, thereby achieving better performance.

Bhushan is a Data Scientist working in MiQ’s Bengaluru office. Outside of work, he enjoys bike rides and gaming.

Bid Optimization with Adaptive Tuning (BOAT) was originally published in MiQ Tech and Analytics on Medium, where people are continuing the conversation by highlighting and responding to this story.

Our data lake had crossed the multi-petabyte threshold and the cloud bill was scaling rapidly. Legacy compression was padding every file with redundant bytes, and hand-tuned clustering strategies were quietly forcing full table-scans on queries that should have returned in seconds.

At MiQ, data is the fundamental currency of programmatic media. For years, standard lakehouse patterns carried us through rapid growth. But as any engineer knows, the architecture that gets you to the first petabyte is rarely the one that gets you to the tenth.

We had hit what we call the Scalability Ceiling. Our legacy configuration, Snappy compression and human-led data layout strategies, was imposing two invisible taxes: a storage tax and a compute tax i.e., paying to scan irrelevant data. Hence, we set out to eliminate both.

The problem: high cost of speed

Years ago, when the Lakehouse paradigm was young, CPU was the bottleneck. Engineers everywhere defaulted to Snappy compression because it was fast to write. It prioritized ingestion speed over storage density.

Today, that equation has inverted. Modern cloud compute is abundant and efficient. But high-performance object storage is a compounding cost that never stops growing. By sticking with Snappy, we were paying premium rates to store redundant bit sequences that ZSTD would have eliminated during writes.

Optimization drift

The compute side was equally problematic. Traditionally, data engineers would manually configure Z-Order clustering based on intuition: “Analysts usually filter by region, so let’s cluster by that.” But business questions change.

A table optimized for the queries of 2022 forces the queries of 2026 into brute-force full-table scans. We called this the “Compute Tax”. This results in us burning money on I/O operations that deliver zero value.

The solution: better math, smarter AI

To dismantle these taxes, we engineered a migration strategy built on two pillars: Data Physics and Intelligence.

The physics: switching to Zstandard (ZSTD)

We moved from Snappy (LZ77-based) to Zstandard. ZSTD represents a genuine mathematical breakthrough, going far beyond a simple version upgrade. Where Snappy focuses on byte-oriented speed, ZSTD uses a method called Finite State Entropy (FSE — it looks at which pieces of data appear most often and gives them the smallest representations, while less common pieces get larger ones.) to statistically find patterns in data and compress it very efficiently, approaching the maximum compression that’s theoretically possible.

The impact is immediate and measurable. ZSTD compresses files significantly tighter than Snappy while maintaining the read speed required for interactive analytics. We are no longer paying to store air.

Note: ZSTD’s compression level is tunable. At level 3, write speed is within 15% of Snappy while compression density improves by 37%. At higher levels, density improves further at the cost of write latency.

The intelligence: “Genie-in-the-loop”

Changing compression handles the size of the files, but it does not solve read efficiency. For that, we needed better Z-Ordering.

We integrated Databricks Genie Research as an automated analyst. Genie connects directly to our Unity Catalog and system tables to perform semantic reasoning. It does not just inspect schema, it analyzes the intent behind thousands of production queries.

Genie reviews query lineage, identifies slow full-table scans, and correlates them with filter predicates. It then synthesizes this data to recommend the mathematically optimal Z-Order columns based on actual usage patterns, not anecdotal evidence.

This moves us from a static architecture to a dynamic governance model that adapts to shifting query patterns automatically.

The migration: surgical precision at petabyte scale

Migrating a live, petabyte-scale data lake is not something you do on a Friday afternoon. We employed a “Shadow Partition” backfill methodology. A Write-Audit-Publish pattern that ensured zero downtime for downstream consumers.

The impact: efficiency at scale

The results of this modernization have been immediate and profound.

• 50% storage savings: By combining ZSTD with aggressive small-file compaction, we halved the footprint of our heaviest tables.
• 25% compute efficiency: By letting Genie dictate layout, we maximize data skipping. Our engines now ignore 90% of files for high-selectivity queries, shifting workloads from I/O-bound (slow) to CPU-bound (fast).
• Elimination of drift: The platform now adapts. If query patterns shift next month, Genie detects it and the layout strategy evolves automatically. This will power our Liquid Clustering system in the next phase.

Key takeaways

1. Snappy served its era, but ZSTD’s Finite State Entropy delivers 37% tighter compression with negligible speed trade-off. Stop paying to store air.
2. Human-led Z-Order decisions drift within months. AI-driven layout optimization (Genie + query lineage) turns static tables into self-tuning data structures.
3. Use the Write-Audit-Publish (shadow partition) pattern for zero-downtime migrations. Never bet your SLA on a big-bang cutover.
4. Combine compression and clustering improvements together. The compounding effect (50% storage + 25% compute savings) is greater than either alone.

Conclusion

Modern data engineering means designing systems intelligent enough to manage themselves. These gains allow us to process data faster, operate more reliably at scale, and ultimately deliver quicker, more actionable insights to our clients.

That matters more at MiQ than almost anywhere else, because data is the fundamental currency of programmatic media. Every audience we build, every campaign we optimize, and every outcome we measure is powered by our ability to turn massive, fragmented datasets into clarity at speed.

Interested in solving complex data challenges at the petabyte scale? Follow the MiQ Tech Blog for more insights into our architecture and engineering practices.

Milan is a Software Engineer II in MiQ’s Data and AI Platforms team. Outside of work, he enjoys reading, writing, and exploring ideas at the intersection of life and engineering.

How MiQ uses AI to cut storage costs at petabyte scale was originally published in MiQ Tech and Analytics on Medium, where people are continuing the conversation by highlighting and responding to this story.

Every campaign starts with a plan. The challenge is that markets, audiences and inventory rarely behave according to plan. As campaigns evolve, advertisers often find themselves underdelivering, overspending or manually adjusting budgets across dozens of line items just to stay on track. At scale, pacing is no longer an execution challenge — it becomes a decision challenge.

Campaign performance and delivery are highly volatile. Without active management , even a well structured initial setup will fall short of its targets.

Today, campaign pacing is largely solved through manual, rule-based approaches. This creates significant operational overload 30–50 hours per month per trader, while still producing under delivery and inconsistent performance across campaigns.

Built on MiQ’s analysis of thousands of campaigns spanning multiple DSPs, verticals and campaign types, we built the Pacing recommendation tool: a scalable, data-driven system that replaces manual pacing decisions with intelligent, ongoing recommendations.

For trading teams and advertisers alike, the tool delivers impact across three dimensions. First, it cuts operational overhead: decisions that currently consume 30–50 hours per trader every month are automated. Second, it reduces both under delivery and over delivery, keeping campaigns in the optimal zone, typically 100–110% of target — which maximizes both performance and profitability. Third, it protects media spend: proactive, data-driven budget reallocation has already demonstrated approximately $840 in media wastage savings during a single week of testing.

What is pacing, and why does it matter ?

Pacing describes how evenly a campaign delivers impressions or spend relative to its target timeline. Think of it like fuel on a road trip, spend too fast and you run out before the destination. Spend too slowly and you arrive with a budget unspent and goals unmet. MiQ tracks four states.

Even pacing means the campaign is on track for example, 50% delivered by day 15 of a 30- day flight. Under-pacing means the campaign is falling behind target. Optimal pacing is the sweet spot: slightly ahead at around 100–110% of target. And over-pacing means the campaign is burning through budget too quickly and risks exhausting spend before the flight ends.

How the pacing recommendation tool works

Step 1: Assess pacing effectiveness
The tool begins by pulling campaign data from all active DSPs (Demand-Side Platforms — the technology advertisers use to buy digital ad inventory) into a single system. It evaluates delivered impressions, revenue, client impressions from the ad server, and days remaining in the flight.

Step 2: Projecting costs and impressions
Next, the tool forecasts how impressions and costs will develop for the remainder of the campaign. It accounts for media spend, CPM (cost per thousand impressions — the standard unit for measuring ad buying efficiency), data fees, ad serving charges and vendor fees. A key nuance is reconciling the difference between impressions recorded by the client’s ad server and those recorded by the DSP, these rarely match exactly. The tool adjusts projections to account for this discrepancy, ensuring budget recommendations are grounded in accurate numbers.

Step 3: Budget recommendation

The goal here is simple: Allocate funds to areas where they are most needed. If certain line items are performing well, we allocate more budget there. For underperforming items, we recommend reducing the budget allocation.

For under-pacing campaigns: We use a technique called Multi-Armed Bandit (MAB) with Upper Confidence Bound (UCB) to shift budget to where it’s most likely to improve performance. Each line item is treated as an arm with rewards derived from normalized performance signals such as delivery velocity and cost effectiveness.

The budget is then incrementally reallocated toward line items with higher reward scores, maximizing the expected marginal improvement in pacing while still allowing controlled exploration of less-sampled items. This approach supports convergence towards optimal spend allocation.

At each decision interval, the system computes a UCB score for every line item:

For over-pacing campaigns: We look at the overall performance and adjust the budget to avoid overspending. The system redistributes excess funds from high-performing areas to underfunded ones until everything balances out.

Observations based on live campaigns

MiQ tested the tool across 24 campaigns within a week, covering Display, Online Video (OLV) and Audio formats across multiple DSPs. Success was defined practically: for under-pacing campaigns, it meant an increase in delivery, for over-pacing ones, a reduction in over-delivery, in both cases with CPM held approximately constant.

The results were clear: an 83.3% success rate across those 24 campaigns, with an average pacing uplift of 8% and approximately $800+ in media wastage savings over a single week. The tool consistently moved campaigns in the right direction without inflating costs to get there.

Pacing at scale, without the manual grid
What makes this tool different from a simple automation script is that it never stops learning. As market conditions shift and audience behavior changes, recommendations adapt in real time. Trading teams no longer need to spend 30–50 hours a month firefighting pacing issues manually. Instead, MiQ’s system does the heavy lifting. For advertisers, that translates directly into campaigns that deliver on their promise: the right impressions, at the right pace, with every dollar working harder.

Bhushan is a Data Scientist working in MiQ’s Bengaluru office. Outside of work, he enjoys bike rides and gaming.
Adnan is a Data Scientist at MiQ who loves turning complex data into clear insights and meaningful impact.

Why campaign pacing is one of digital advertising’s biggest hidden challenges was originally published in MiQ Tech and Analytics on Medium, where people are continuing the conversation by highlighting and responding to this story.

Introduction

Expanding audience reach without losing relevance is a tricky balance. Brands desire scale, precision, and speed, but traditional look-alike modeling generally compromises one for the other.

What if we could model similarity directly using network-level signals, based on real online behavior?

Our machine learning-based system designed for scale does just that, creating audiences that actually mirror a seed group without relying on invasive PII (Personally Identifiable Information).

The Objective & Landscape

Our goal was to create and deploy a look-alike modeling system that works natively with anonymized, network-level behavioral data. In order to improve accuracy and velocity, this bypasses traditional proxies.

The traditional modeling method is proxy-driven, based on broad interest groups, ZIPs, or demographics. But this method has terminal weaknesses:

• Proxy Dependence: The nuances of behavior are not considered in demographics.
• Assumption-Based Targeting: ZIPs, age ranges, or interests as a proxy for behavior are inherently flawed.
• Lack of Granularity: The most important behavioral signals of relevant audiences are abandoned when granular activity indicators are abandoned.

Look-Alike Workflow: From Noise to Insights

Our approach provides readable audience insights in real-time at scale, from noise to insights, as shown in Figure 1.

• Clean and Organize Data: We take messy, unstructured data and turn it into tidy, structured user profiles.
• Richer Audience Insights: We increased reach by processing 3x more unique website domains. Using LLM-based categorization, we reduced data “blind spots” (uncategorized traffic) from 75% to just 25%
• High-Density Profile Generation: We transform thousands of behavioral signals into compact, high-value embeddings (the “essence” mentioned earlier).
• Effortless scaling: Fine-grained, precise information can be taken advantage of at a large scale with gigantic simplicity.
• Instant matching: Our rapid search finds exact fit lookalikes in a few seconds, even through billions of records.
• Intelligent targeting: Teams can pinpoint exact audiences to refine campaigns and enhance ROI.

Step by step with a closer look at how our approach systematically solves the problem.

• Data Ingestion and Categorization — We started with a massive DSP feed of site-level browsing linked to anonymized network identifiers. Raw data was biased and difficult to process. Using LLMs, we automated categorization across 75,000+ websites tripling usable coverage, cutting ~98% of manual work, and converting messy data into structured, privacy-safe profiles.
• Feature Engineering using Autoencoders — Having rich, categorized data, we constructed user profiles that extracted thousands of behavioral traits. The challenge, however, is that a profile with thousands of features is computationally slow and expensive to compare against billions of other users in real-time.
  To solve this, we needed to compress these rich profiles into a smaller, manageable format while preserving their behavioral essence. This is the primary motivation for using an autoencoder. An autoencoder is a neural network designed for efficient dimensionality reduction. We trained an autoencoder in an incremental manner on various partitions of the data. This procedure reduced the profiles into concise 30-point embeddings, capturing user behavior “essence” while sustaining 99% information integrity.
• High-Speed Retrieval using FAISS — The last one was to facilitate real-time seed audience matching. We utilized FAISS (Facebook AI Similarity Search), the high-speed retrieval-optimized vector database, to conduct the nearest-neighbor search on our user embeddings. With this, we can determine the most suitable lookalike audiences in an instant, making high-scale analysis possible and effective.

Our solution transforms high-volume complex data into actionable and meaningful audience profiles, facilitating quick lookalike discovery and driving accurate personalization straightforwardly increasing campaign ROI.

Impact and Takeaways

This new approach offers immense technical benefits and immediate business effects for Engineering leaders and Product owners alike.

• Unlock Scale with Intelligence (LLMs): LLMs automatically sort and process unstructured datasets with ease, eliminating thousands of man-hours of work, lowering the cost of processing, and building richer user profiles — leading to more intelligent targeting and higher ROI.
• Speed and Scale (Autoencoders + FAISS): Scaling with vector searches in neural embeddings, scaling hundreds of millions of users with zero decrease in speed. Enabling quicker campaign deployment or finding lookalike audiences, making it more agile.
• Simplify and Succeed (Centroid Querying): Instead of querying for every single identifier in a seed audience, which could amount to thousands of queries, we precalculate what we call the “centroid,” a single vector representing the mathematical average of all users. We then conduct a single search for the identifiers closest to this average profile. This cuts computational costs exponentially and simplifies the query protocol, making it easy for teams to test, iterate, and refine campaigns quickly.

Why This Matters Now

In a world where third-party cookies are falling apart and privacy laws are becoming increasingly stringent, the ad industry is finding itself losing its traditional sources of signals. The ad industry is in dire need of a privacy-aware and scalable solution that can replace user-level tracking. This solution acts as that bridge.

Conclusion

Look-alike modeling is evolving beyond blunt geographic or demographic proxies into behavior-driven intelligence. Our IP-to-IP pipeline shows that advertisers can reach not just more people but the right people.

This system fundamentally changes the trade-off between reach and relevance, providing advertisers with a clear path to enhanced ROI, greater campaign agility, and the behavioral precision needed to win in a competitive landscape.

References

• Facebook AI Similarity Search (FAISS)
• Autoencoders for Dimensionality Reduction
• LLMs in Feature Engineering
• Adversarial Factorization Autoencoder for Look-alike Modeling

I am Kuntal Mudi, I work as a Senior Data Scientist(PDS) in MIQ. Outside of work, I enjoys playing computer games and cricket and likes up-skilling regularly on the technology space. I also love listening to music and playing chess.

I’m Bala Saatvik. I’m really interested in AI and intelligent systems, particularly Agentic AI, and I enjoy exploring how these technologies can be used to build smarter, more autonomous solutions for real-world challenges.Outside of work, I love traveling, spending time with my pets, and reading Medium blogs in my free time to learn about new ideas, technology trends, and different perspectives.

From Proxies to Behavior: Building Scalable Look-Alike Audiences with IP-Level Intelligence was originally published in MiQ Tech and Analytics on Medium, where people are continuing the conversation by highlighting and responding to this story.

What if your ads knew where your customers were and how to address them at the right time?
The Power of Being in the Right Place at the Right Time

In today’s hyper-digital age, attention is money. Brands are constantly battling for a few seconds of focus, and traditional targeting strategies like demographics or interests simply aren’t enough.

To cut through the noise, marketers require an emerging superpower: location intelligence. That’s where geofence targeting comes in. It blends physical presence with digital behavior and allows brands to appear not only online but offline too, exactly where their audience is.

What is Geofence Targeting?

It’s a location-based targeting that places a virtual fence, “geofence”, around a physical space. When a person enters or is present within that space with their phone or other device, it triggers an action, such as showing them an ad, placing them in an audience segment, or sending them a notification.

Example: Geofencing a block near a popular café in Surry Hills at 9 AM on a workday.

How It Works

• You target a specific location, eg, a stadium, a shop, or even a hand-drawn polygon.
• The user enters this place with location services enabled, and the system identifies them (through GPS or Wi-Fi).
• Your ad is served in the moment, or the user is put into a pool to be retargeted later.

The best part? It’s invisible, automatic, and very powerful.

Why Geofencing Works: Marketing, But Smarter

Let’s be real, attention spans are decreasing. If you can’t reach someone at the right moment, you’ve lost them.

Geofencing delivers:

• Hyper-Relevance
  Individuals are more likely to engage with a message that reflects their perspective. A lunch offer when they’re 100m from your store? Perfect timing.
• Real-World Outcomes
  It connects the dots between digital impressions and physical footfall. Geofencing influences store visits, reservations, and conversions.
• Competitive Edge
  You can target people near competitor locations, target footfall, or build brand visibility.

Why now?

With third-party cookies disappearing and advertisers shifting toward more context-driven strategies, geofencing has become increasingly important. Precise location intelligence is now one of the most dependable ways to reach the right audience in today’s privacy-focused advertising landscape.

Real-World Use Cases: From Retail to Politics

The potential for uses is limitless. Here’s how brands are applying geofencing today:

How MiQ is Using Geofence Targeting in Real Life?

Standard DSP targeting has a ceiling. When restricted to basic postcodes, brands often bleed budget on irrelevant audiences. We refused to accept those limits.

In our recent campaigns in Australia, we worked around the usual DSP geo limits by building our own setup that feeds detailed polygon files straight into the platforms. This lets us create precise geofence rail segments that standard tools don’t normally allow. This didn’t just tweak our targeting; it revolutionized it.

From high-stakes Federal Elections to granular regional campaigns, we unlocked the ability to target down to the electorate level.

Result of the Geo-Fence Campaigns

1. Electorate Level Campaign Performance
   The setup made it really easy for the client to check their main KPIs, for both the candidate and the electorate. We instantly tell which areas were performing well and which need a bit more push or support. Using these insights, we fine-tuned our strategy and focused resources and budget where they would make the biggest difference.

2. Precise Targeting

• Hyper-local Precision: Instead of using wide targeting zones, we narrowed things down to particular local areas so the ads reached the exact people we wanted.
• Smarter Efficiency: This approach helped us deliver almost 90% of our impressions inside the exact polygons we set up.
• Cutting the Waste: Using our own custom polygons (geofence) targeting helped us trim more than 30% of ad spend that would’ve otherwise been wasted.

Best Practices for Geofence Campaigns

Want to win with geofencing? Here’s what you need to get right:

1. Be Specific, But Not Too Narrow
A 200m radius around your store may be too narrow. A 3km radius may be too wide. Experiment with different shapes and sizes of geofences — custom polygons will usually outperform circles.

2. Time It Right
Serve breakfast offers in the morning. Push promos when competitors are busiest. Align your ad timing in sync with user behavior.

3. Measure What Matters
Monitor outcomes not only by impressions and clicks, but by visits to the store, conversions, and footfall lift.

A Word on Privacy (Because It Matters)

Privacy isn’t a checklist for us; it’s a priority. Every tool that MiQ creates is designed with privacy in mind. To ensure everyone’s safety, we use only anonymised, opt-in location data for our geofencing and adhere strictly to key international laws such as the CCPA and GDPR.

Final Thoughts

Geofence targeting is a tactical layer that enhances your audience targeting, bidding strategy, and creative planning; it is not an extra line in your media plan.

“On a map, location is more than just a point. It’s a moment, and marketing magic happens in moments.”

I’m Rhushikesh Chavan, work as a Product Analyst at MiQ, where I build and optimize ad-tech products that solve real business challenges. Outside of work, you’ll usually find me traveling, exploring new places, or enjoying outdoor games. I love anything that gets me moving and discovering something new.

How Geofence Targeting Unlocks Precision for Modern Ad Campaigns was originally published in MiQ Tech and Analytics on Medium, where people are continuing the conversation by highlighting and responding to this story.

There’s an old saying: “If you don’t schedule time for maintenance, your systems will schedule it for you and usually at the worst possible time.”

As engineering teams scale rapidly and AI accelerates product cycles, the tug-of-war between shipping fast and staying stable has never been sharper. Every release brings a choice — deliver now or invest in tomorrow’s reliability.

Too much focus on delivery, without correct design architecture, may end up with a brittle codebase that’s painful to work with. And, too much refactoring makes our product roadmaps stall while the business waits.

Tech Debt Isn’t Always Bad

We used to think of tech debt as a huge task and heavy on resources. But in reality, it’s often a deliberate choice, a trade-off to move fast. Like financial debt, where you pay interest. The more you delay paying the interest, the interest compounds.

• Features take longer to ship.
• Bugs multiply and get harder to debug.
• Developers start dreading the codebase.
• Stability takes a hit.

On the flip side, we obviously can’t let product delivery come to a halt while we clean every corner of the system. So the question becomes: how do we make this work to achieve both?

At MiQ, our engineering teams have faced this exact tension across products like TV Intelligence. TV Intelligence is a comprehensive TV planner tool for Sellers, where clients can get detailed insights about top ads, reach of their brands across various channels, and compare with competitors.

What follows are the habits and frameworks that helped us stay balanced:

1. Create a Shared Language

We started by aligning the entire team on what “tech debt” means. Vague complaints like “this code is messy or complex” don’t help. What did work better was documenting specific, actionable issues like:

• The impact (slower velocity, onboarding friction, more bugs)
• The risks or bottlenecks to future roadmap items and classifying them into foundational, localized, or blocking categories.

We built a simple debt registry with a clear flag on what and how it could affect other features every quarter.

2. Considering Tech Debt in Every Sprint

We’ve stopped treating tech debt as a major separate epic and instead made it part of our regular sprint planning. Here’s how:

• Each sprint can have tech debt bandwidth (for example, 15–20% of story points)
• A few of them can be coupled with the new feature tickets when feasible

This ensured that progress is continuous without requiring long “debt-only” cycles or quarters.

3. Use Feature Development as a Refactoring Trigger

Every new feature we build is a chance to “leave the codebase better than we found it”.

For example:

• Refactor a service module if you’re touching it for a new feature
• Improve test coverage when adding new business logic
• Move towards a shared design pattern when extending components
• Anything that could lead to the rewriting of the code later due to other features needs to be refactored now.

This strategy reduced resistance to cleanup and allowed small wins to accumulate.

4. Connect Tech Debt Outcomes to Business Outcomes

The fastest way to get the Product team on board with tech debt cleanup is to link it directly to business value.

For example:

• Fixing the code that helps cut dashboard load time by 80%.
• Restructuring a module that will unblock new features to be implemented faster.
• Removing duplicate logic will speed up new data onboarding by 30%.
• Failing to decouple a module would result in data inconsistency and a complex implementation.

When framed this way, tech debt stops sounding like an engineering-only problem and becomes a shared priority across teams.

5. Create Space for Larger Refactor Projects

Some types of debt can’t be incremental, such as moving to a new architecture, consolidating APIs, or switching libraries. For these, the approach is to:

• Break the entire effort into smaller and testable chunks that can be deployed safely, without affecting the whole.
• Use feature flags to roll out changes gradually.
• Plan these projects in buffer periods after major releases.

This ensures even large refactors can be delivered without putting timelines at risk.

6. Celebrate the Invisible Wins

Refactoring, cleanup, and architecture improvements often go unnoticed — unless you highlight them.

In our case:

• We highlight them as “Tech Wins” for every quarter.
• Demo important refactorings in sprint reviews (just like features).
• Calling out developers who improved shared infra/tools in monthly shoutouts.

Over the past few quarters, we at the TV Intelligence team of MiQ have tackled some long-standing tech debts, which improved our stability and performance:

1. Domain-Driven Design for TV Intelligence:
   We refactored the backend into modular domains, making it more scalable, easier to manage, and faster to onboard new geographies.
2. TV Intelligence UI Migration to TanStack Query:
   We moved UI state management from Redux to TanStack Query, cutting dashboard load time from 10 to 2 seconds for a faster, smoother experience.
3. Unified Brand Registry Integration:
   TV Intelligence now connects to a central brand registry, removing duplicate data and streamlining workflows.
4. Unified Custom Ads Manage Service:
   We unified political and non-political ad uploads into one service, eliminating extra pipelines and simplifying maintenance.

Conclusion:
Balancing tech debt and delivery isn’t a one-time act; it’s a cultural muscle. For us at MiQ, the goal isn’t to have zero debt, but to make every line of code a little stronger than the one before.

Chandan is an Associate Engineering Manager at MiQ, working from the Bangalore office. Delivery manager for Advanced TV capability. Super adventurous, he loves challenging himself through trekking, sports, and bike rides. Skilled photographer.

Balancing Tech Debt and Product Delivery was originally published in MiQ Tech and Analytics on Medium, where people are continuing the conversation by highlighting and responding to this story.

How MiQ leverages Agentic AI for natural language conversations with SQL Databases

In the high-speed world of digital advertising, an account manager needs answers, now. Why isn’t a campaign working? What’s underperforming and where? The clock’s ticking, and delays cost impact. Traditionally, getting to the root of campaign performance meant asking an engineer for a deep dive into complex databases, from traditional SQL servers to real-time systems like Apache Pinot. That process — and the waiting — is no longer good enough. At MiQ, we’re building a new way for our teams to get insights, driven by a specific use-case: a conversational chat agent to derive insights from our data.

With Agentic AI, we’re putting intelligent, conversational assistants in the hands of our stakeholders, no code, no SQL, and no waiting. These AI agents don’t just answer, they understand.

The Challenge: From Natural Language to Pinot

The fundamental hurdle is translating language used by our teams into precise SQL queries that Pinot can execute. While LLMs excel at understanding human language, directly asking them to generate SQL can be unreliable. For instance, if a user asks for “active campaigns,” a generic LLM might not know whether “active” refers to campaigns with recent ad spend or those simply marked as “live” in the database, leading to an incorrect query.

We need a system that not only grasps the intent behind the question but also understands the intricacies of our Pinot database — its structure, the type of data in each column, and how different pieces of information relate to each other. Furthermore, the system needs to handle follow-up questions, refine queries based on initial results, and gracefully recover from any errors that might occur.

From Language to Action

To solve this, we employ an LLM agent framework. An LLM agent is a system that combines a Large Language Model (LLM) with tools it can use to find answers and take action. Think of it as an intelligent assistant that can understand a request, decide which tool to use (like a query generator), execute that tool, and determine the next step.

To build this, we utilize Google’s Agent Development Kit (ADK). The ADK is a set of tools and libraries that helps developers build and customize these agents more efficiently, providing powerful orchestration capabilities to manage the conversation and tool use.

What is an LLM Agent?

An LLM agent is a system that combines an LLM with orchestration and tool-calling capabilities. In practice, this means the agent can:

• Understand your requests.
• Decide which tools to use to fulfill your request.
• Execute those tools with the correct arguments.
• Process the output from the tools.
• Determine the next steps, which might involve calling another tool, refining the query, or providing a final answer.

Here’s a diagram illustrating the basic concept (Fig 1):

Workflow

The following diagram illustrates the overall flow of the system (Fig 2):

Here’s how our chat agent solves the problem, step-by-step. The entire process is managed by the brain of the operation: an LLM Agent built using Google’s ADK. This agent orchestrates the entire workflow, manages conversational memory, and decides which tools to use and when.

1. User Query: An account manager asks in plain English: “Show me the top 10 performing campaigns in the last week.”
2. Agent Calls the NL2SQL Tool: The LLM Agent receives the question and determines it needs a SQL query. It calls the first tool in its arsenal: the NL2SQL Tool. This tool takes the user’s natural language and our Pinot database metadata as input and, using Gemini, generates the precise SQL query needed.
3. Agent Calls the Verification Tool: With the SQL query now in hand, the agent knows it must be executed. It calls the next tool: the Verification Tool. This tool first validates the query’s parameters using the sql-rail python package. If the parameters are invalid, it provides feedback so the agent can orchestrate a correction. If valid, the tool executes the query against our Apache Pinot database.
4. Agent Delivers the Answer: The Verification Tool sends the data from Pinot back to the LLM Agent. The agent processes these results and formulates a clear, concise answer that directly addresses the account manager’s original question.

This cycle of understanding the user, figuring out the right tools, using those tools, and then responding continues until the agent has provided a satisfactory answer.

Key Observations and Benefits

• Automated Error Resolution: The LLM agent can automatically detect and correct errors in the generated SQL, leading to more robust query execution.
• Follow-up Query Handling: The agent’s conversational memory allows it to understand and correctly interpret follow-up questions, providing a more natural and interactive experience.
• Complex Query Breakdown: The agent can break down complex user requests into multiple, simpler SQL queries, improving accuracy and efficiency.

Conclusion

By combining LLM agents with Apache Pinot, we’re enabling faster, more natural data exploration across our business. This capability is a key step towards our broader Agentic AI vision at MiQ.

References:

https://google.github.io/adk-docs/

https://pypi.org/project/sql-rail/

Abhay leads the Data Science team at MiQ’s Centre of Excellence in Bangalore. When he’s not building AI-driven solutions, you’ll find him pursuing his social entrepreneurial ventures or exploring the cosmos from his terrace on any given night.
Ajay is a Data Scientist at MiQ’s Centre of Excellence in Bangalore, where he focuses on developing generative AI solutions. In his free time, he loves to travel and explore new places.

Building a Natural Language Interface for Apache Pinot with LLM Agents was originally published in MiQ Tech and Analytics on Medium, where people are continuing the conversation by highlighting and responding to this story.