AI agents are no longer just suggesting the next line of code. They’re starting to plan, build, test, and ship work alongside engineers. Last month, we explored that shift in the Agoda AI Developer Report 2026. On 21 October 2026, we took the conversation further by inviting Bangkok’s tech community to our office at One Bangkok for an evening of talks and discussions. Our theme was “From Assistants to AI Agents,” and leaders from Agoda, Microsoft, SCB 10X, OpenAI, and Omise shared where agents are already making a difference, where the bottlenecks have moved, and how teams can stay in control.

Here are the key takeaways.

1. Speed moves the bottleneck

AI coding tools have made writing code dramatically faster, but delivery hasn’t kept pace. Once coding gets faster, the slow parts shift to planning, review, testing, release, and understanding what customers actually need. The teams seeing the biggest gains are the ones looking at the whole lifecycle, not just the editor.

2. AI belongs in every stage, not just coding

The next gains come from applying AI beyond code generation: reviewing changes, triaging support requests, checking specs before work begins, and running agents for hours in the cloud rather than on a laptop. As these tools start working together, the engineer’s role shifts from handing out tasks to setting goals and reviewing results.

3. Good context and clear goals make agents useful

Agents perform best when they’re treated like new team members. They need clear instructions, curated context, and a precise definition of what “done” looks like. A vague goal or a loose definition of “good” leads agents to repeat one pattern everywhere, or even to optimize for the score instead of the outcome. Framing the goal well and adding independent checks matter as much as the model itself.

4. Match autonomy to risk

Not every decision needs a human, but some always should. A useful test is whether a mistake can be reversed and how much damage it could do. Low-risk, reversible work can be delegated, while high-stakes or irreversible decisions should be paused for a person. Many teams start small, automating narrow, well-defined tasks and keeping a human approval step before anything goes live.

5. Accountability stays with people

Agents can take on more of the work, but they can’t take on the responsibility. When AI writes more code than anyone can read line by line, teams need agents that flag where the risk lies, plus strong observability and audit trails. Rather than making engineering skills less important, agents make sound judgment and fundamentals matter more.

Conclusion

AI agents are already changing how software gets built, but the evening made one thing clear: the biggest gains don’t come from faster code alone. They come from rethinking the whole delivery process, giving agents the right context and goals, and keeping people in charge of the decisions that matter. As agents take on more of the work, engineering judgment becomes more valuable, not less.

We’re continuing to explore what this shift means for how we build at Agoda, and we look forward to sharing more of what we learn with the engineering community. Thank you to our speakers, panelists, and everyone who joined us in Bangkok.

Explore opportunities in our Tech team today!

Key Takeaways from Tech at Agoda: From Assistants to AI Agents was originally published in Agoda Engineering & Design on Medium, where people are continuing the conversation by highlighting and responding to this story.

At Agoda, our post-booking systems support complex flows involving cancellations, amendments, refunds, and customer communications. These flows are coordinated through a write-oriented orchestration service that we will call “BAPI”.

As BAPI handled more traffic and attracted more contributions, its codebase became increasingly difficult to change. Responsibilities that should have remained separate grew tightly connected, so even a small update could require a broad understanding of the system and more effort to review safely.

Rewriting the service from scratch was not an option. Instead, we created the Platform framework, a modular architecture designed to introduce clear and enforceable boundaries into BAPI while allowing feature development to continue.

In this blog post, we share how we adopted the framework incrementally, enforced its rules in CI, and measured its relationship with review speed and AI-assisted development.

Why we built the platform framework

In 2019, we had to decide what to do with an aging core monolith. But as Agoda’s business grew rapidly, it became increasingly challenging to maintain the core of our business within a single monolith. Hence, we decided to split our domain into two parts: booking creation and post-booking. Thus, the application BAPI was born. To make BAPI a reality, we chose a deliberately pragmatic path: to extract and translate the post-booking business line by line into a new application, without attempting a broader redesign. The expectation was that a separate application would give our postbooking system plenty of headroom to grow.

The approach worked. In the language of John K. Ousterhout’s A Philosophy of Software Design, we had deliberately traded a strategic design approach for a tactical one to prioritize speed to market. This tactical mindset was exactly what the business needed at the time. The service ran smoothly, features shipped rapidly, and operations remained uninterrupted. For several years, it proved to be the right decision.

Over time, however, the limitations of the line-by-line migration became harder to ignore. After we deprecated the legacy service and rerouted its remaining traffic to BAPI, the service saw increased traffic and attracted more contributions. The number of merge requests more than quadrupled between 2022 and 2025.

Without strong architectural guardrails, the codebase expanded along the paths of least resistance. Over time, the legacy codebase grew substantially. Some files exceeded 4,000 lines, and changing one often required understanding a large web of connected behavior.

The original structure used a traditional package-by-layer design, organizing packages around separate technical concerns. This approach followed the classic Presentation-Domain-Data Layering pattern described by Martin Fowler. In practice, however, using technical layers as the top-level organization at our scale meant the boundaries were not strict enough. The server module consolidated business logic, repository access, controllers, mappers, and utilities, while the common module served as a destination for anything that seemed reusable.

This was not a failure of contributor judgment. The architecture did not provide enough guidance to make boundary decisions consistently.

Developers felt the impact in their day-to-day work. Changes took longer to review, tests were more likely to break, and pipelines became less reliable. Even small fixes started to feel riskier than they should have.

The decision: platform framework

We faced two constraints. We could not rewrite the entire system because the business could not pause feature development for several years. We also could not leave the architecture unchanged while development velocity continued to decline.

The Platform framework gave us a third option. Rather than replacing the legacy system at once, we introduced a modular monolith architecture with strict isolation rules. This time, we deliberately chose a strategic design approach over a tactical one. We enforced new modules to use the framework by default, while legacy areas migrate gradually as substantial feature work requires changes.

The goal was to reverse the existing incentive structure. The old codebase made tight coupling the path of least resistance. The Platform framework needed to make the well-structured path easier than taking a shortcut. By hiding structural complexity inside the framework itself, we wanted to lower the cognitive load on contributors so they could change code with confidence.

Our mental model was simple: grow a new skeleton beside the old system and allow it to carry more of the workload over time.

How the platform framework works

The core shift was moving away from traditional package-by-layer boundaries. Instead of organizing code horizontally by technical concerns, we adopted a vertical package-by-component pattern to build a modular monolith.

We organize our codebase into coarse-grained, isolated modules centered around specific business domains — such as `rebook`, `cancellation`, and `payment`. If business domains do not overlap conceptually, their code should not overlap either. For example, cancellation logic belongs strictly inside the cancellation component and cannot bypass boundaries to depend directly on communication or loyalty logic.

Each domain component acts as an encapsulated container that bundles its business logic and data access together behind a clean public interface. Internally, to ensure the code inside a component stays highly organized and uniform across the team, the framework applies a strict layered architecture:

• Application layer: Orchestrates the request. It knows what to do, but not how.
• Service layer: Owns the business logic for one domain. It knows how to act on the data.
• Repository layer: Owns data access and never exposes its internal models to other layers.
• Model layer: Contains the domain data structures shared across the other layers.

Determining these layers and how they interact with each other was not a trivial task. We intentionally set up strict boundary rules so that the day-to-day work within each module remains as predictable and brainless as possible. Again, we found our guiding rationale in John K. Ousterhout:

“Complexity is determined by the activities that are most common. If a system has a few parts that are very complicated, but those parts almost never need to be touched, then they don’t have much impact on the overall complexity of the system.”

By standardizing our module internals into an application, service, and repository skeleton, we isolated the structural complexity within the framework setup itself, a part of the codebase our contributors rarely need to modify. In return, the tasks our developers most commonly perform—adding features, changing business logic, or writing queries — occur in isolated, highly predictable places where cognitive load is minimized.

The most important rule is that services and repositories do not communicate directly, and neither do sibling services. Application-level components, including startup classes, middleware, handlers, and contexts, mediate the request flow.

At a simplified level, the framework looks like this:

This prevents a change on one side of a boundary from silently spreading across the module. The value lies less in the layer names than in the prohibition on shortcuts.

Two mechanisms are especially important: Data Mapper prevents repository internals from leaking upward, while typed builders make object creation explicit and searchable.

Data Mapper: translation at the boundary

The framework follows Martin Fowler’s Data Mapper pattern. Repository-specific models, including API payloads, database entities, and cache objects, never leave the repository layer. Mappers translate them into domain models before another layer can use them.

This keeps the domain layer independent of the underlying data source. A repository implementation can change without forcing corresponding changes throughout the business logic.

Builders: bounded and searchable

We use typed builders to keep object creation and transformation within clear boundaries. In the legacy code, an object could pass from a handler to a service, then to a repository, and back again, changing at several points along the way. A .copy() in one layer, a helper in another, and another transformation in a mapper meant we had to search for the type name and hope we had not missed anything.

In the Platform framework, objects can only be created through builders tied to a specific use case. We have separate builders for service inputs, domain models, and responses, and each one keeps the relevant transformations in one place.

If we want to see all the ways a Booking domain model can be created, we look in BookingDomainBuilder. To find where a service input is created, we search for its ServiceInputBuilder. We no longer have to trace a chain of changes across the entire request flow.

Encouraging incremental adoption

Designing the framework was only part of the work. We also needed a way to introduce it without pausing feature development.

Rather than migrate BAPI all at once, we made the Platform framework the default for new modules and submodules. We move existing areas when feature work gives us a practical reason to change them. Over time, several post-booking domains have begun building directly on the shared platform module.

The framework encourages adoption by making common work easier within its boundaries. It provides automatically injected request contexts and shared utilities for time handling, structured logging, measurement, and experimentation.

Reaching across domain boundaries is intentionally more difficult. This comes with an upfront cost, including additional boilerplate, a learning curve, and stricter review. We accepted that cost because a shortcut today can become a multi-day investigation later.

Enforcing it in CI, not just in docs

Making the right path easier helped us introduce the framework. Keeping its boundaries intact required more than documentation and code review. We treated these rules as what Neal Ford and Mark Richards call “architectural fitness functions” in Software Architecture: The Hard Parts. By codifying our layer boundaries into an ArchUnit test suite, our CI pipeline serves as an automated guardrail, catching structural drift on every pull request before a human reviewer ever sees it.

For example, model classes cannot access any other layer. A repository’s internal models, clients, and proxies contain the raw structures it uses to communicate with a database or an external API. Only that repository can access them. A service or handler cannot reach past the repository boundary and use them directly.

These are the same rules described in our design documentation, but instead of relying on reviewers to catch each violation manually, our CI pipeline runs them as executable tests on every build. One of those architecture checks looks like this:

s”$moduleName’s Repositories layer” should {
 “not expose internal model classes” in
 layers
 .optionalLayer(“RepositoryModels”)
 .definedBy(s”com.agoda.bapi.$modulePackage.repository.model..”)
 .whereLayer(“RepositoryModels”)
 .mayOnlyBeAccessedByLayers(“Repositories”)
 .check(importedClasses)
}

This changes the review contract. Reviewers no longer need to detect every forbidden dependency manually because CI rejects those violations before the merge request reaches them.

The checks are a hard gate. New violations fail the build rather than being absorbed into the baseline. Framework-backed domains are evaluated against these rules, while legacy areas that have not yet migrated remain outside their scope.

What the numbers actually say

Once the framework was in wider use, we wanted to know whether it was actually making development easier. We started by looking at merge-request data.

We focused our analysis on the first half of 2026, when the framework was mature, and adoption was broad enough for a meaningful comparison. We excluded the earlier rollout period because the framework was still evolving and the samples were small. We track three metrics:

• Review Time to First Feedback (RTTF): The time from opening an MR to receiving the first substantive review response, measured in hours.
• Code Review Finish Time (CTRF): The time required to complete review after it begins, measured in hours.
• Merge Lead Time Complete (MLTC): The total time from the first commit to merge, measured in days

Head-to-head: framework-backed vs. core legacy code

We compared two mutually exclusive cohorts:

• Platform at 90% or above: MRs with at least 90% of their changed lines in framework-backed domains.
• Legacy at 90% or above: MRs with at least 90% of their changed lines in the core legacy code, where most production changes outside the Platform framework are made.

We compared the Platform cohort with the busiest part of the legacy codebase rather than grouping all non-framework code together. This gave us a more meaningful view of how the two approaches performed under active development.

MRs that did not reach 90% in either direction were excluded from this comparison because they represent a different question: the cost of work that crosses architectural boundaries.

Within each monthly cohort, we also removed the smallest and largest 10% of MRs based on lines changed. We applied the size filter separately because Platform and Legacy MRs differed greatly in typical size. A single cutoff for both groups would have removed too many MRs from one cohort while having little effect on the other.

During the first quarter, neither cohort showed a consistent advantage. CTRF medians changed position from month to month, and RTTF was similar between the groups.

Then, in Q2, the lines separate. Legacy review times increased while the Platform results remained comparatively stable. Averaged over Q2 2026, framework-backed MRs completed code review 64% faster (CTRF median 1.08 vs 3.02 hrs) and received first feedback 34% sooner (RTTF median 10.34 vs 15.77 hrs).

This occurred even though the framework-backed MRs were generally larger.

MLTC varies for several reasons, including implementation time, scheduling, and release timing. Review is only one part of that measure, but it is one part that the codebase can directly influence.

When code is easier to place and review, those gains can help shorten the overall path to merge.

Why did review speed improve?

Our interpretation is that the first-to-second-quarter change reflects a familiarity curve. Before the framework, placement often depended on reviewer judgment. Reviewers had to ask where logic belonged and whether a change touched the correct files.

The framework reduced that ambiguity by giving every module a repeatable structure. As reviewers became familiar with it, conversations shifted away from code placement and toward business logic, edge cases, and failure modes.

The framework itself did not change between the two quarters. What changed was how familiar reviewers became with its conventions. We interpret this directionally, as the monthly sample sizes remain modest, with 4 to 21 Platform MRs and 14 to 32 Legacy MRs.

In a migrated module, a change stays within a small vertical slice. Instead of tracing the entire call graph, a reviewer can focus on that slice. Three design choices help make this possible:

• Explicit roles. Services decide, repositories translate, and models are built through builders. The file’s role signals to the reviewer what to expect, so placement questions become less common over time.
• Explicit wiring. Startup declares the request flow in one place instead of hiding it behind implicit resolution and cross-package indirection.
• Executable boundaries. ArchUnit turns invalid cross-layer shortcuts into build failures before a reviewer has to find them manually.
• Encapsulated complexity: Each layer and its interactions are engineered to reduce the cognitive load on the code readers rather than the writers. This directly mirrors John K. Ousterhout’s observation: “Complexity is more apparent to readers than writers”.

A migrated startup class looks like this:

lazy val acceptChangeSetupMiddlewareChain: MiddlewareChain[AcceptChangeSetupRequest, AcceptChangeSetupResponse] =
 MiddlewareBuilder(middlewareInitializer)
 .use(validateAcceptChangeSetupRequest)
 .use(fetchAcceptChangeBookingDetails)
 .use(orchestrateAcceptChangeSetup)
 .build()

That one chain answers the two questions that usually dominate review time in a legacy system: “where does this get called?” and “what else can this change touch?” Once both are cheap, cycle time tightens across the whole distribution.

The same boundedness changes the shape of risky migrations. When one framework-backed domain widened a numeric identifier from Int to Long, the expand-contract migration split into reviewable phases because the repository carried the transitional mapping logic while the domain model remained stable; in legacy code, the same change tends to collapse into a single cross-cutting PR. That’s the practical version of mesh programming: the architecture doesn’t just make code cleaner, it makes risky work decomposable.

Why did code contribution speed improve? (MLTC)

Review speed is only part of the equation; the rest is writing and landing the change. To probe whether the framework’s boundaries made code easier to reason about before review, we looked at one of the first platform modules to reach critical mass, `cancellation`, and ran static analysis on the place where its logic used to reside in the legacy codebase (`server`).

At first glance, comparing a global legacy module to a single domain component might not seem like a fair, like-for-like comparison. But from a contributor’s perspective, this is exactly where the value lies. Previously, a developer working on a cancellation feature had to navigate and commit to `server`, exposing them to the cognitive load and blast radius of an all-encompassing monolith. Today, that same developer’s cognitive footprint is restricted entirely to the isolated `cancellation` module.

The structural difference between these two working environments (without test files) is stark:

We also ran the cyclomatic-complexity check enforced by our CI, which uses a threshold of 10 branches per method.

AmendmentService.scala alone accounts for 7 of the server’s 136 cyclomatic complexity violations. The cancellation module had none across all 202 files. This provides a code-shaped explanation for the review results. Before the review begins, a framework-backed change generally asks the author and the reviewer to understand a smaller, flatter surface.

It’s not just humans anymore

When we designed the framework, we did not set out to measure its effect on AI-assisted development. But by 2026, AI coding agents were involved in a meaningful share of BAPI development, and the question became hard to ignore: Does an architecture built for human comprehension also help a machine?

The most striking signal is not cost or loops. It is adoption. From June through August 2026, 91% of Platform ≥90% MRs had recorded AI activity, against 69% for Legacy ≥90% MRs. We read this as engineers voluntarily reaching for agents more often on platform code — the place where an agent has the best chance of being useful.

Why would that be? An AI agent faces a challenge similar to that of a reviewer. It has to build a sufficient understanding of the codebase to decide what can be changed safely. If clear boundaries shrink a person's search space, they should also shrink the context and iteration an agent needs. When the payoff is higher, people use the tool more.

To explore this, we joined our filtered production-code MR dataset with our internal AI usage data, keyed to the GitLab MR identifier. We have reliably been tracking companion analysis from June through August 2026. That leaves us three months to work with, so the numbers below should be read directionally. We recorded per MR prompt activity and applied the same monthly, cohort-specific line-count trimming used in the review analysis.

Two things stand out once you look past adoption.

First, the per-MR prompt counts are nearly identical — 20 median prompts on platform code versus 22 on legacy. On a raw, per-MR basis, an agent iterates about as much in either world. That is worth stating plainly, because it means the platform advantage is not about the agent “getting it right the first time.”

Second, the advantage appears once you normalize by size. Platform MRs’ 15.54 total prompts per 100 lines of changed code against legacy MRs’ 21.80 — the platform cohort is roughly 29% cheaper per 100 LOC. The average-prompt-per-100-LOC figure points out that the platform cohort spends fewer iterations per unit of change. Though we treat these numbers as directional, a 29% reduction in AI prompts is not a coincidence.

Why are AI agents cheaper on platform code?

The mechanism we proposed for human reviewers seems to apply, largely unchanged, to agents. In each case, the win is a smaller search space.

• Predictable directory structure. In a layered module, an agent can locate the handler, service, repository, and model for a feature by convention rather than by exploration. Fewer files opened, fewer tokens spent orienting.
• The layer contract bounds the blast radius. Strict boundaries tell the agent and the CI that checks it where a change is allowed to reach. The agent does not have to reason about whether editing a repository quietly changes handler behavior three hops away, because the architecture forbids that path.
• Builders and generated scaffolding remove guesswork. Where the framework provides builders and code generation, the agent fills in a known shape rather than inventing one, which cuts the trial-and-error loops that dominate costs.
• Deterministic CI gives fast, unambiguous feedback. Arch-test guardrails and consistent test structure mean a wrong turn fails quickly and legibly, so the agent corrects on cheap signal rather than burning loops on a vague error.

None of this makes the agent iterate fewer times per MR, recall the near-identical median loops. It makes each unit of change carry less surrounding context, so the same iteration count buys more verified code. On legacy modules, where boundaries are softer and structure varies from module to module, the agent spends more per line establishing what is safe to touch, and it seems engineers notice and reach for it less.

We did not design the framework for machines. But the property that helps a reviewer, a codebase you can reason about locally, turns out to be exactly the property that makes an agent both cheaper and more attractive to use. The question we’ll keep watching, as our data window grows, is whether the gap between the two cohorts holds.

Conclusion

Clear architecture pays off by moving the question of “where does this code belong?” out of every individual change and into the framework itself. Handlers orchestrate, services decide, repositories translate. The structure tells humans and AI agents where the code lives before they start changing it.

That is what the Platform framework brought us. We did not rewrite the system; we built a new skeleton alongside it and let it take on more weight over time. The older code is still there, but the new work now has clearer boundaries to follow.

However, adopting a new framework isn’t a one-time victory; it requires continuous improvement and eternal vigilance. Codebases naturally drift toward entropy, and the moment we stop defending these boundaries, the old habits of tight coupling will slide back in. We must constantly audit, refine, and enforce these rules as the system evolves.

If you’re facing a similar coupling problem, our takeaway is this: intentional friction at the boundary is worth it, but don’t take it on faith. Instrument your pull requests. Prove it to yourself. We did, and while our early metrics are directional, they are highly encouraging. Our strategic decision to invest in long-term design is already starting to pay dividends at the six-month mark. And if John K. Ousterhout is right that the true payback period of good software design is 6–18 months, we expect to see this positive trend solidify and settle over the next year.

How We Cut the Coupling Tax in a Post-Booking API was originally published in Agoda Engineering & Design on Medium, where people are continuing the conversation by highlighting and responding to this story.

Download the Full Report: https://ago-da.co/4yXRa9T

A year ago, we launched our first AI Developer Report to understand how AI was reshaping engineering work across Southeast Asia and India. This year, agentic AI is the story. Developers are moving from using AI to assist individual tasks to delegating entire workflows to systems that can plan, act, and complete meaningful parts of the work on their own. We surveyed developers and engineering leaders across India, Indonesia, Malaysia, the Philippines, Singapore, Thailand, and Vietnam to see how far that shift has gone.

The report presents a grounded view of how AI is reshaping software development across the region. This article summarizes the key findings and their implications for engineering teams and leaders.

Performance strengthened while trust held steady

AI is doing more, and developers still check it just as closely. More than 55% now save at least 7 hours a week, up from 18% a year ago, and most say AI-generated code is usable with only minor changes. The caution held. 79% still worry about hallucinations or incorrect output, and 86% review AI output always or most of the time. Usage also spread across the lifecycle, with code review at 84%, architecture and design at 81%, and testing at 73%, each up sharply from 2025. AI now runs across the whole development process, not just code generation.

Agent adoption is outpacing readiness

Adoption has passed the halfway mark. 53% of developers report agents in production or broad use, and nearly all have started experimenting. Readiness has not kept up. Only 38% call their codebase mostly or fully ready for a fully autonomous agent. The findings show readiness is about more than clean code. It also depends on how agents connect to existing systems, what they can access, whether teams can control their cost, and whether engineers can catch incorrect output. The gap is widest at larger organizations, where agents reach broad use before the codebase is ready to support them.

Cost is the leading barrier, and it hides where few teams look

Developers name cost as the biggest barrier to wider agent adoption. 28% put it first, ahead of integration complexity and governance, and that rises to 33% among teams already running agents in production. The more revealing finding is where the cost sits. It rarely comes from the model call. Most of the spend goes into the design, review and validation work around it, and it stays hidden while teams track tokens and seats instead of the work that surrounds them. OpenAI is framing cost the same way. Derrick Choi, Head of Codex Applied AI, APAC, says the company is focused on “how much useful work each dollar of intelligence can deliver.” The report reaches the same conclusion. Measure cost per unit of finished, verified work, make usage visible by feature or team, and choose the model as a configuration decision from the start.

Autonomy expands while accountability stays human

Developers grant autonomy by risk. 43% allow full autonomy for documentation, but 79% require human approval before production deployment. When an agent causes a problem, 42% hold the individual developer responsible and only 2% blame the AI vendor. Governance is formalizing alongside this. Comprehensive AI guidelines rose from 23% of organizations in 2025 to 39% this year. As Harley Young, Head of High-Tech at Microsoft, puts it: “The winners won’t be whoever adopts agentic AI first, but whoever builds trust and accountability around it. The opportunity we all share is to treat AI adoption as an operating-model change, not just a productivity tool.”

Developer leverage is rising unevenly

The leverage AI gives each developer is not reaching everyone equally. 49% of junior developers believe agents will make their roles less secure, compared with 17% of CTOs and VPs of Engineering. At the same time, developers expect their value to move up the stack. They rank system architecture and design (65%), AI literacy (61%), and agent orchestration (51%) as the top future skills, while coding fundamentals appear in just 18%. Access to AI tools is no longer the differentiator. Judgment is, and junior engineers have had the least time to build it.

How companies across the region are implementing it

The survey patterns show up in how regional companies are putting agents to work, and each pairs a new capability with control.

Agoda extended its AI work from code review to development. Building on CodeBuddy, which reviews merge requests, we built CodeMaster this year, a platform that hosts coding agents on pre-warmed virtual machines running Agoda’s own environment. Instead of building its own agent, CodeMaster connects to the tools engineers already trust, aiming to write 10% of merge requests this year.

Omise moved from AI-assisted coding to a broader development capability, completing a major platform upgrade in weeks rather than months by using AI to assess impacts across its codebase and by building governed agent workflows that keep accountability human.

DataXet built an automated monitoring system where agents pull from internal and client data, flag anomalies and trends, and turn the findings into an executive summary with prioritized recommendations for leadership. It also treats cost control as an engineering discipline, on the view that the real risk is not AI being expensive but AI being used carelessly.

SCB 10X already runs services built almost entirely by agents, but scales trust carefully. After seeing agents make confident but incorrect claims, it now tests more autonomous behavior in shadow mode behind an operator-controlled gate.

Across all four, none removed human oversight. They built agents into their existing quality and cost controls rather than around them.

What this means for leaders

Rethink how AI cost is measured. Four in five developers already work under usage limits, yet cost is still the biggest barrier to adoption. Companies that measure cost by finished, verified work will scale further than those relying on restrictions alone.

Manage AI risk as it moves into production. Managing AI risk is now the top challenge for engineering leadership, named by 31% of respondents. As agents reach production, static policies fall behind. Leaders need repeatable ways to decide what agents can do, how their output is verified, and where human approval is required.

Invest in the next generation. Junior developers are far more likely to feel less secure. Access to AI tools is no longer the differentiator. Mentorship is when experienced engineers help juniors build the judgment that AI cannot teach.

Key Takeaways from Agoda’s AI Developer Report 2026 was originally published in Agoda Engineering & Design on Medium, where people are continuing the conversation by highlighting and responding to this story.

At Agoda, we use Slack to communicate and collaborate across teams. It is also one of the places engineers turn when they need help with an internal service or platform. The conversation may begin in Slack, but the information needed to resolve an issue often sits elsewhere.

Consider a message like this:

“My application is down. It cannot connect to the database. What should I check?”

Even a question that sounds straightforward can have several possible causes, from a recent deployment or cluster problem to an unavailable downstream service. To narrow them down, the engineer responding may need to inspect dashboards, traces, source code, tickets, documentation, and earlier Slack conversations.

Before the investigation can begin, the team often needs to gather basic context and determine where the request should go. We wanted to make that process easier while keeping it within the tools and workflows our engineers already used.

In this blog post, we share how we approached this challenge, how our engineering support workflow evolved, and what we learned along the way.

OpsBot: bringing service desk workflows into Slack

Our first step was OpsBot, a Slack-native service desk for engineering teams. It plays a role similar to Zendesk, but keeps the request, conversation, and handoff in Slack.

OpsBot gives teams a consistent way to receive requests, collect relevant details, route them, and create tickets when needed. Each team can configure the workflow for its own domain without asking engineers to move to another support tool.

The process starts when an engineer runs an OpsBot command in Slack. The channel’s YAML configuration selects the appropriate support flow and opens a form. After the engineer chooses a topic and provides the requested details, OpsBot routes the request to the responsible team and can run a Python automation when one is configured.

This made the process more consistent, but the forms still assumed that engineers knew which technical details mattered. During an outage, someone may have only an error message, a Grafana link, or a related Slack thread. They may not know which cluster, pod, dashboard, or downstream service is relevant.

The next challenge was to keep the structure OpsBot provided while allowing engineers to begin with the information they already had.

Evolving OpsBot from forms to conversations

Forms work well when requesters know what information to provide. When they do not, large language models give us another way to gather that context. With OpsBot AI, requesters can start by describing the problem in their own words and sharing whatever evidence they already have.

OpsBot AI uses the initial message, the surrounding Slack conversation, and shared links to understand the request. If any required information is missing, it asks a focused follow-up question. The forms, routing rules, ticketing, and escalation paths remain in place, but requesters no longer need to complete every field before asking for help.

The YAML configuration continues to define the support topic, required information, routing, ticket rules, and optional Python automations. A requester asks for help in natural language, and OpsBot AI interprets the request, gathers the required context, assigns the responsible team, and creates the ticket.

This changes where the work begins. Requesters can start with the problem they see, while the engineering team responding receives a request with more of the required context already in place.

Adding domain agents to the support workflow

Making it easier to ask for help solved only part of the problem. A request could now reach the right team with better information, but the engineer responding might still need to build the investigation from scratch.

This led us to add domain agents to the workflow. OpsBot AI sends the Slack conversation to the agent configured for that support flow. The agent follows the team’s investigation guidelines and uses approved tools to gather more context.

For a service-down question, the agent might inspect OpenTelemetry signals and traces, search for a related incident, consult documentation, or examine the relevant dependency path in source code. If it finds an ongoing incident affecting the service, it can point the requester to the relevant update. For a more complex case, it can give the on-call engineer a summary of what it checked, what it found, and where the investigation could continue.

The domain agent runs separately on Agoda’s Agents Platform. If the request still needs human support, OpsBot AI keeps the existing ticket and escalation paths available.

Letting domain teams build their own agents

Agoda already had an internal Agents Platform, which gave OpsBot AI a way to connect each support flow to a domain agent.

The platform allows teams to create and operate their own agents. They can choose a model, write domain instructions, add skills, connect approved tools and Model Context Protocol (MCP) servers, and expose the agent through an Open Responses-compatible API. The team that owns the domain also decides how its agent should work.

OpsBot AI connects to the Agents Platform, which hosts agents for areas such as BI, infrastructure, and DevOps. The platform combines their skills with approved MCP integrations.

A BI team understands its data models and recurring questions. An infrastructure team knows which operational signals matter and when to escalate. A platform team understands its dependencies and common failure modes. Each team can turn that experience into instructions and reusable investigation guidelines for its agent.

Depending on the domain, agents can use approved integrations with tools such as Sourcegraph, Atlassian, Grafana, Tempo, Pyroscope, Glean, and related-ticket search.

Skills do not contain the domain knowledge itself. They guide the agent through an investigation by explaining which sources to consult, which checks to run, and when to involve an engineer. Some teams have also built pipelines to keep these skills aligned with changes in their workflows.

The Agents Platform also traces each run. When OpsBot AI posts an answer in Slack, authorised users can follow the trace link and see which tools the agent used. This helps domain owners understand what the agent considered, what it missed, and what they could improve in the next version.

BI Product: an early success story

BI Product adopted OpsBot AI support agents across Marketing, Customer Engagement Group (CEG), and Supply. Each area has an agent configured with its own domain instructions, skills, tools, and support workflow.

Together, the three agents expose 134 domain skills: 50 for Marketing, 43 for CEG, and 41 for Supply. The skills guide recurring investigations and help the agents use sources such as Confluence, OpenMetadata, and data tools consistently.

Between 1 June and 24 August 2026, the agents participated in 402 of 556 actual support threads across BI Product’s four core support channels, giving them 72.3% coverage. Here, participation means that the agent posted at least one response in the thread, and coverage is the share of actual support threads in which this happened. Both requesters and supporters could rate a response. Of the 286 threads that received feedback, 241, or 84.3%, were rated very or partially useful.

In 87 support threads, the agent provided a direct answer, meaning it completed the investigation and resolved the request without human analysis. The supporter only needed to close the ticket.

Marketing also saw a change in the effort required for support. After the rollout, ticket volume increased by 19%, while average logged supporter time fell by 15%, from 45.3 minutes to 38.5 minutes. For longer investigations, p90 effort fell from 120 minutes to 60 minutes, while median effort remained at 20 minutes.

The improvement appeared in the long tail, where more involved investigations required less time from supporters. These are early observations from an ongoing rollout. They do not prove that the agents alone caused the change, and they do not mean that every request was automated.

Lessons from the rollout

Turn tribal knowledge into shared knowledge

Building support agents pushed teams to document knowledge that previously lived in people’s heads. This made many investigations easier to repeat and reduced the dependence on a few domain experts.

Skills guide the agent

A skill is a procedure, not a knowledge source. It tells the agent how to investigate and where to look. The knowledge remains in Confluence, GitLab, OpenMetadata, Agoda’s data platforms, and other sources.

Skills also guide routing. A policy question should use an approved document instead of searching through code and logs. Most teams maintain routing skills based on their domain.

The same BI Product team mentioned earlier also uses a pipeline that reviews recent negative feedback and proposes skill updates through an MR. A human reviews each change before it is merged.

Invest in error analysis

Understanding an agent requires time and good observability. Traces show the path it followed, the tools it called, and where the investigation failed. Without them, explaining a “not useful” response would be difficult.

Error analysis also exposed limitations around the agent. When Slack was unavailable, we introduced FeedMCP to monitor its status feed and those of similar SaaS services.

When a Tempo query returned a 12 MB response and exceeded the agent’s context window, we began experimenting with tool-side summarisation, code mode, and sub-agents.

Manual error analysis helps with individual failures, but it does not scale. A skill or model change can fix one investigation and break another. This led us to Agent Eval, where these lessons can become repeatable checks.

Closing the loop with Agent Eval

As teams began updating their agents, they faced another challenge: how could they tell whether a change improved the agent without causing a regression elsewhere? A new model, skill, or tool integration could improve one type of response while affecting another answer or investigation path.

To answer that question, we introduced Agent Eval, which is currently in beta. OpsBot AI sends successful support calls and their usefulness feedback to our data platform. Agent owners review those interactions and select real questions and answers for an evaluation dataset. The feedback helps with curation, but domain owners decide which cases belong in the dataset and what a good result looks like.

In the live workflow, OpsBot AI calls a domain agent through the Agents Platform and returns its answer to Slack. The interaction and its usefulness label are stored in the data platform. Agent owners can then use this production data to curate the evaluation dataset.

Agent Eval replays the saved cases through the Agents Platform and records the answers and tool calls. For each case, teams define the behaviour they want to check. They might assess the answer, require a particular tool call, or verify selected tool arguments. Running the same dataset against the current and proposed versions makes it easier to see how a change affects the agent.

We are also working towards shadow evaluation. This sends the same live request to two agent versions in parallel. The current version answers the requester, while the candidate version is evaluated without its response being shown. Teams can then assess the candidate using real traffic before releasing it.

Where we go next

Making engineering support better with AI has taken years, and the work is not finished. Have we replaced the entire support system with AI? No. Have we made the system better for requesters and supporters? The results so far suggest that we have.

The next stage is about helping the system improve continuously. Agent owners are building pipelines to evaluate agent behaviour and curate skills, with people reviewing the changes before they reach production. This is how we move towards self-improving agent harnesses while keeping domain owners in control.

Human support will remain essential when a problem requires judgment, ownership, or deeper investigation. The aim is to reduce the routine work that needs a person, giving engineers more time for the problems that genuinely need them.

How Agoda is evolving engineering support with AI agents was originally published in Agoda Engineering & Design on Medium, where people are continuing the conversation by highlighting and responding to this story.

Introduction

At Agoda, delivering real-time hotel rates to millions of customers relies on a responsive and scalable pricing architecture. At the center of this system is our Price Cache, which stores terabytes of volatile rate data fetched from our global network of suppliers. When customers search for hotels, our backend services retrieve rates from this cache. A cache hit provides an immediate response. A cache miss triggers an on-demand workflow that fetches the latest rates and integrates them back into the system.

For years, MSSQL served as the primary storage engine for this critical component. As Agoda and its supplier network grew, the cache had to support increasing data volumes and read/write throughput. This placed our existing SQL Server architecture near its operational limits and made further scaling increasingly complex and resource-intensive.

In this blog post, we share how we migrated our Price Cache from SQL Server to DragonflyDB. You will learn why we selected DragonflyDB, how we completed the migration with minimal disruption, and how we improved failover and maintenance after the rollout.

Challenges in scaling MSSQL for our use case

Our legacy MSSQL setup relied on 72 shards, spread across multiple physical servers. The routing logic is maintained at the application level, and a drawback of this setup was its rigidity. To distribute the load evenly, we could not add a single server whenever we needed more capacity. Instead, we had to scale in predefined steps, such as moving from 6 to 12 or from 12 to 18 servers.

Each scale-out operation required our database team to remap the shards manually and migrate data to match the new topology. In early 2024, surging read and write volumes led us to double our hardware capacity. Within a year, we were approaching the limits again.

It became clear that continuing to add resources to SQL Server was not a viable or cost-effective long-term strategy.

Searching for a successor: DragonflyDB

Replacing SQL Server required more than selecting a fast database. Before the migration, our Price Cache served approximately 300,000 reads per second and close to 1.5 million writes per second, with strict P99 latency requirements of under 50 milliseconds.

The price dataset occupied approximately 1.5 TB of memory and continued to grow. Because the Price Cache is a tier 1 production system, we also needed high availability and the ability to scale without disrupting customer traffic.

DragonflyDB, a modern in-memory data store, stood out as a strong fit for our needs, particularly in the following areas:

• Multi-threaded for high throughput: DragonflyDB uses a shared-nothing, multi-threaded architecture that partitions the dataset into smaller shards, each managed by a dedicated thread. This design enables parallel processing with minimal locking, helping DragonflyDB deliver high throughput at low latency.
• Redis compatibility and seamless scaling: DragonflyDB implements the Redis interface, allowing applications to use existing Redis clients. This suited our workload, which relies heavily on MGET and SET operations. DragonflyDB can also operate in Redis Cluster mode. When we need more cache capacity, we can add nodes to the cluster and redistribute slots to meet changing business requirements. This directly addresses the rigid scaling model we experienced with SQL Server.
• Built-in Expiry: Hotel rates are highly volatile and change frequently. With SQL Server, we had to build and maintain a cleanup job to remove stale supplier data because the storage engine did not provide the expiry behavior we needed. DragonflyDB handles expiry by allowing us to set a time-to-live when writing data, eliminating the need to maintain a separate cleanup process.
• Proven in-house expertise: Other engineering teams at Agoda were already using DragonflyDB for systems such as our Feature Store and Microcache DBaaS. This internal operational experience, combined with engineers who understood how to manage DragonflyDB clusters, gave us confidence to adopt it for a tier-1 production system.

Verifying DragonflyDB’s performance at Agoda’s scale

DragonflyDB published benchmarks showing up to 25× higher throughput than Redis for GET and SET operations at comparable tail latencies. For a tier 1 production system operating at Agoda’s scale, we needed to validate its performance against our own workload.

We created a load test using Redis’s memtier_benchmark. Based on production telemetry, we modeled a 1:6 read-to-write ratio, with an average of 10 keys per MGET operation.

To generate a realistic load of more than 100,000 requests per second, we provisioned 10 pods. Each pod ran 20 Redis clients, each client used 6 threads, and each thread handled approximately 140 requests per second.

Under the simulated production load, DragonflyDB delivered lower P99 latency for both read (MGET) and write (SET) operations than our SQL Server production baseline.

We also confirmed that DragonflyDB’s shared bandwidth usage would remain below the physical server limit of 25 gigabits per second.

Our migration strategy

Migrating a critical, high-throughput system required careful orchestration to protect customer experience and maintain business continuity. We divided the migration into four phases:

1. Planning and preparation: Provision the infrastructure, the application, and populate DragonflyDB with data.
2. Parity checking: Read from both data stores and compare DragonflyDB with the existing SQL Server source, while continuing to serve customer traffic from SQL Server.
3. Gradual rollout: Move read traffic from SQL Server to DragonflyDB through controlled experiments.
4. Maintenance and optimization: Improve maintenance workflows and introduce automatic failover behavior.

Existing architecture

Before the migration, our SQL Server Price Cache used an A/B setup for resilience. During data retrieval, the Price API performed round-robin reads across the SQL Server instances to distribute traffic between the two clusters.

Phase 1: Planning and preparation

We began the migration with a conservative design, then adjusted it as the dataset grew and our scaling requirements became clearer.

Our first deployment was a standalone DragonflyDB instance with 1 TB of memory. It stored only hot data, defined as itineraries with a length of stay between one and four days. We also introduced a new Price Writer and a gRPC gateway that provided an interface to the data layer.

Within a few months, organic growth pushed this hot dataset to approximately 900 GB. Reaching our 90% memory-safety threshold triggered cache evictions to prevent out-of-memory failures. These evictions risked preventing customers from seeing the best available prices for their hotel searches.

To create sufficient long-term headroom, we moved to a sharded design with three shards per cluster.

The additional capacity gave us an opportunity to store the complete price dataset in DragonflyDB. At that point, the dataset required approximately 1.5 TB of storage.

After deploying the cluster, we began writing all price data to DragonflyDB. It processed approximately 1.6 million writes per second with a P99 latency of around 10 milliseconds.

With data flowing reliably into DragonflyDB, the next step was to validate its accuracy.

Phase 2: Parity checking

Our Price API retrieves data using search criteria such as hotel, occupancy, check-in date, and check-out date. We implemented a dual-read mechanism in the Price API. SQL Server continued to serve customer requests, while the API asynchronously fetched the corresponding DragonflyDB data for comparison.

We evaluated parity across two dimensions:

• Supplier count: Did both data stores return the same number of suppliers for the search criteria?
• Price-data length: For each supplier, did both data stores return price data of the same length?

Comparing the full contents exactly would have been computationally expensive, so we used data length as a proxy.

We emitted the results as Prometheus metrics. Both dimensions achieved a match rate above 99.9%. Our investigation found that the remaining differences were primarily caused by benign variations such as timestamps and unique IDs.

This phase also allowed us to validate MGET performance under real production read traffic. The results aligned with our earlier benchmark. DragonflyDB delivered a P99 read latency of approximately 8 milliseconds while serving around 300,000 requests per second.

Phase 3: Execution and Zero-Downtime rollout

After validating data integrity and performance, we began serving customer requests from DragonflyDB. We used an A/B experiment to control the rollout. Customers in variant A continued to receive data from SQL Server, while customers in variant B received data from DragonflyDB.

Because DragonflyDB had already handled production-level read volumes during parity testing, we did not expect performance problems. The experiment focused on confirming that the transition would not negatively affect booking metrics. The results were positive. Within a few weeks, we moved 100% of customer traffic to variant B.

Once DragonflyDB was serving all price data, we deprecated the SQL Server read and write paths. This freed database resources for other use cases.

Phase 4: Maintenance support and optimizations

Our DragonflyDB architecture uses two clusters, A and B, for high availability. This allows our infrastructure team to perform routine maintenance, such as operating system patching and dependency upgrades, on one cluster without interrupting business operations.

The initial maintenance workflow involved several manual steps:

1. Traffic Rerouting: Mark cluster B as unreadable and send all read traffic to cluster A.
2. Process Shutdown: Stop DragonflyDB on cluster B. Its health status becomes unhealthy, which stops writes to that cluster.
3. Maintenance and Restart: Perform and complete the maintenance, then restart DragonflyDB. When the cluster becomes healthy, writes resume and begin repopulating the cache. Reads remain disabled because serving traffic from an empty cache could result in customers receiving no prices for their searches.
4. Cache Warming: Monitor the cache-hit ratio. When it approaches the level of cluster A, mark cluster B as readable and resume sending read traffic to it.

In the example above, cluster B required approximately three hours to reach the cache-hit ratio of cluster A, which was serving production traffic. The process required manual monitoring and consumed almost half a working day. We wanted to reduce both the manual effort and the recovery time.

The workflow also addressed only planned maintenance. During an unplanned outage, the absence of automatic failover meant that DragonflyDB could not continue serving prices until the issue was mitigated.

Teaching the system to make failover decisions

We wanted each application pod to detect an unhealthy cluster and automatically reroute traffic, without manual intervention.

The cache-hit ratio was the natural signal. A healthy cluster should have a hit ratio close to that of its counterpart. If one cluster’s ratio diverges significantly, it may indicate a problem.

The challenge is that cache-hit ratios fluctuate naturally as traffic patterns change. Rerouting traffic whenever the ratios differed would create frequent false alarms. Each pod, therefore, needed to distinguish a meaningful divergence from normal variation. Every five seconds, each pod compares the hit ratios of clusters A and B using the previous five minutes of traffic. It calculates how unlikely the observed gap would be if both clusters were healthy.

If the gap is too large to be explained by normal fluctuations, the pod treats it as a real problem and initiates failover. If the gap falls within the expected range, the pod continues serving traffic from both clusters.

Each pod performs this test independently using only the read and cache-hit counts it observes locally. Pods do not share their observations, and no coordinator or leader makes the decision for the fleet. This gives the failover mechanism a shared-nothing design in which every pod reaches its own conclusion about cluster health.

We used two different thresholds, depending on which direction we’re checking:

• Going unhealthy (healthy-threshold = 10%): if a cluster’s hit ratio falls at least 10 percentage points below the other’s, and the gap is statistically unlikely to be noise, the pod marks that cluster “unreadable” and stops sending reads to it.
• Recovering (recovery-threshold = 3%): When the difference returns to within 3 percentage points and is no longer statistically significant, the pod marks the cluster as readable and resumes sending traffic to it.

With this setup, we simulated an unplanned outage by intentionally bringing down one of the Dragonfly clusters. All application pods in the data center shown below entered the failover state within approximately two minutes. Around 40 pods were running in that data center at the time, and the process required no manual intervention.

The system is not perfect yet. We continue to tune the thresholds so that all pods reach the same conclusion more quickly. We are also exploring ways to shorten recovery by restoring an empty cluster from a snapshot of the healthy cluster instead of warming it from a cold cache.

Even with this remaining work, automated detection and rerouting represent a significant improvement over manually managed failover.

Conclusion

Since completing the migration from SQL Server to DragonflyDB, we have seen meaningful benefits. We improved P99 read latency by eight times while serving substantial traffic volumes. We also removed the rigid 72-shard SQL Server topology and retired the custom job that purged stale supplier data. Soon after the migration, growing business demand required us to increase storage capacity again. This time, we scaled without the operational complexity of the previous SQL Server architecture.

While our engineering teams continue to tune the failover thresholds and explore faster cluster-recovery methods, DragonflyDB has already proven to be a suitable foundation for the continued growth of Agoda’s pricing platform and global supply network.

Acknowledgements

This migration was a joint effort across several Agoda teams. We would especially like to recognize Charlene Chan, Yifan Huang, Dmitrii Sokolov, and Sachin Parthasarathy from the Supply Platform team. They led the migration and coordinated the work across the engineering teams involved.

We would also like to thank everyone who supported the migration:

• Initiative sponsors: Prashant Singh, Ziv Ben Zuk, Preutsaji Lertsathienchai, and Tsahi Hakikat
• Database team: Rinson John, Wichai Atchariyabunyong, Avika Sangpenprow, and Thanarat Narapiromsuk
• Supply Tech team: Kanadech Jirapongtanavech
• Price API team: Johan Tiesinga

How Agoda Scaled Its Price Cache with DragonflyDB was originally published in Agoda Engineering & Design on Medium, where people are continuing the conversation by highlighting and responding to this story.

Last year, we launched the first edition of the Agoda AI Developer Report, a research publication exploring how software developers across Southeast Asia and India are working with AI. The report surfaced insights from developers and tech leaders across the region on the tools they use, the challenges they face, and where they see AI heading. You can read the key takeaways here: Agodas AI developer report

We’re back with the second edition, and this year we’re going deeper.

This year’s focus: Agentic AI

AI is no longer just answering questions or suggesting lines of code. It is planning, building, and completing multi-step work autonomously. But how are developers actually experiencing this shift? What is working, what is not, and what does the future look like from where they sit?

These are the questions driving the 2026 Agoda AI Developer Report.

Who we’re surveying

We’re reaching out to software engineers, engineering managers, and tech leaders across Southeast Asia and India. Whether you’re already working with AI agents day-to-day or just starting to explore what agentic AI means for your team, your perspective matters.

How to participate

The survey takes around 15 to 20 minutes to complete. All responses are anonymous, and the findings will be published as a free external report later this year.

Take the survey here: AI Developer Report Survey 2026

From AI Assistant to Autonomous AI: The Next Chapter of AI in Software Engineering was originally published in Agoda Engineering & Design on Medium, where people are continuing the conversation by highlighting and responding to this story.

At Agoda, configuration management is mission-critical. We have thousands of microservices powering our travel platform across four global data centers, and we need a robust way to manage service configurations, feature flags, and runtime parameters. For years, HashiCorp Consul has been our backbone for service discovery and distributed configuration management.

Consul’s key-value store became the central nervous system of our infrastructure, storing everything from database connection strings and API timeouts to feature-flag toggles and regional service endpoints. The scale was substantial: over 34,000 key-value pairs managed critical configurations, with approximately 100+ changes made daily across our production environment. Engineers quickly updated configurations through Consul’s intuitive web UI, and services picked up changes in real time via Consul’s watch mechanism.

It worked beautifully… until it didn’t.

The Problem: When Configuration Changes Became Our Biggest Risk

As we scaled to hundreds of services and thousands of KVs across our four data centers (Hong Kong, Singapore, Amsterdam, and Ashburn), the way we managed configuration needed to evolve alongside it.

Teams were making direct changes through the Consul UI: quick edits that worked well at a smaller scale, but introduced new challenges as we grew:

• Limited validation: Consul only provides regex pattern validation, which is difficult to configure and maintain. Complex validations, such as checking object structure, ranges, or dependencies between values, were never enforced.
• No review process: Critical configuration changes bypassed team reviews, increasing risk.
• Disaster recovery gaps: Configurations lived only in Consul. If something went wrong, we had no source of truth to restore from.

The breaking point came when a misconfigured KV caused a production incident that took hours to diagnose because we couldn’t trace when or who made the change.

We needed a better way, one that brought the discipline of code review to configuration management.

Understanding Our Consul Architecture

Before diving into the solution, it’s important to understand how we structure configurations in Consul across our global infrastructure.

Global vs Local Scopes

Consul’s KV store at Agoda is organized into two distinct scopes:

Global Scope (global/*):

• Configurations that should be consistent across all data centers
• Managed exclusively in the Hong Kong (HK) data center
• Automatically replicated to Singapore, Amsterdam, and Ashburn via consul-replicate
• Perfect for: feature flags, API endpoints, shared service configurations, company-wide settings
• Example: global/payments/api-endpoint → same value in all 4 data centers

Local Scope (local/*):

• Data center-specific configurations
• Each data center maintains its own independent values
• No replication between data centers
• Perfect for: database connection strings, region-specific rate limits, local service endpoints
• Example: local/payments/database-host → different value in HK vs SG

A note on scope: Not everything stored in Consul KV today necessarily belongs there; some configurations would be better served by other mechanisms (service discovery, application config files, etc.). Migrating those values to more appropriate systems is a separate effort. This post focuses on how we brought safety to the configurations that live in Consul.

The Risk of Configuration Changes

This architecture came with tradeoffs worth understanding for both scopes:

Global scope risks:

• consul-replicate doesn’t support gradual rollout, so a single change to a global/ key in HK would instantly propagate to all four production data centers simultaneously.
• A typo in a global timeout value or an incorrect feature flag could impact production services across all regions at once.
• Without validation or review, mistakes were not caught until services began failing across multiple data centers simultaneously.

Local scope risks:

• While limited to a single data center, mistakes could still cause regional outages.
• Teams had to manually update the same configuration across multiple data centers without automation.
• No validation meant typos and incorrect values could slip into any data center.

The combination of instant global replication, a lack of validation, and the absence of audit trails made configuration management increasingly risky as we scaled. The larger the blast radius, the higher the stakes and global changes carried the highest risk of all.

The Solution: GitOps for Consul

We built a GitOps-based solution consisting of two key components:

1. Consul KV sync Service

A dedicated service that acts as the synchronization engine between Git and Consul. It handles all the heavy lifting:

• Watch for changes in the Git repository
• Performs schema validation against JSON Schemas
• Calculates diffs between Git state and current Consul state
• Applies changes (PUT/DELETE operations) to Consul servers across all data centers

2. Consul KV Git Repository

A Git repository that mirrors Consul’s KV structure, providing a familiar interface similar to the Consul UI but with Git’s benefits:

• Directory structure matches Consul’s data center and scope organization (e.g., hk/global/, sg/local/).
• Each file represents a KV entry, just like viewing keys in the Consul UI
• Engineers edit configuration files directly, commit changes, and open merge requests
• All the power of Git: version history, blame, diff, revert, and branch workflows

Instead of clicking through the UI, teams now manage configurations through merge requests with validation, code reviews, and full audit trails. The repository becomes the single source of truth, while Consul KV Sync ensures Consul stays in sync.

Architecture Overview

The system orchestrates a complete GitOps workflow from code review to production deployment:

Workflow Steps:

• Engineer modifies the KV files in the Git repository and opens a merge request
• CI pipeline validates changes against JSON Schemas and posts a diff report comment
• CODEOWNERS review and approve the changes
• MR merges to the main branch
• GitLab webhook triggers the Consul KV Sync service
• Consul KV Sync applies changes (PUT/DELETE) to the appropriate Consul data centers
• consul-replicate propagates global/ changes from HK to other DCs automatically

Repository Structure

Key design decisions:

• Git as the single source of truth: The repository is the authoritative source for all configuration changes. All changes must go through Git. Consul UI writes are blocked for onboarded paths, preventing configuration drift.
• Repository structure mirrors Consul: The directory structure (hk/global/, sg/local/, etc.) directly maps to Consul’s data center and scope organization, making it intuitive for engineers.
• Validation enforcement: Every KV has a JSON Schema that validates values before they reach production, catching errors during review. Integrated with AI assistance to help detect potential issues as well.
• Approval enforcement: No configuration change reaches production without approval from designated owners. This ensures more eyes review changes before deployment, catching potential issues early.

The Migration Journey

Migrating over 34,000 KVs manually from Consul to Git would have been impractical. Copying values, writing schemas, and configuring permissions for each team would have taken months of tedious work and been highly error-prone. Automation was the only viable path forward.

How We Migrated

We built a fully automated onboarding process that eliminated manual work entirely:

• Auto-sync existing KVs from Consul to Git: All configurations were automatically pulled from Consul and committed to the Git repository, preserving the exact directory structure.
• Auto-generated validation schemas: JSON Schemas were automatically generated based on the existing KV values. Teams didn’t write any schemas manually.
• Auto-assigned team permissions: CODEOWNERS were automatically assigned based on previous path ownership, routing reviews to the right teams.
• Auto-revoked UI write access: Once a team was onboarded, their Consul UI write permissions were automatically revoked, ensuring all future edits went through the new Git-based workflow.

Teams received a pre-built MR with their KVs, schemas, and permissions already configured. They simply reviewed for accuracy, approved, and immediately started using Git as their single source of truth. Zero manual migration work required.

The Results:

Over 4 months, 3 engineers completed the migration from ideation to full implementation:

• Over 34,000 KVs migrated from Consul UI to Git repository
• 100+ teams onboarded across all engineering organizations
• 4 data centers now fully managed through Git
• 100% adoption. All Consul KV configurations now managed via Git

Real-World Impact

Beyond the migration numbers, the day-to-day operational impact has been just as significant:

• 100% audit trail: Every configuration change is traceable with author, timestamp, and reason.
• Zero configuration-related outages: After teams onboarded to the new process, schema validation catches errors before production. No configuration change has caused an outage.
• Faster incident resolution: Full version control history makes troubleshooting straightforward, and rollback is simple, just revert the MR to restore the previous working state.

What’s Next

While the current system has eliminated configuration-related outages, we’re continuously improving:

• Gradual rollout: Currently, global configuration changes propagate instantly to all four data centers. We’re introducing a phased deployment that allows changes to roll out datacenter by datacenter. This prevents cascading service instability: if an issue is detected in one data center, the rollout can be paused before it affects other data centers, minimizing outages and improving the customer experience.
• Distributed ownership: Currently, when teams need to update the CODEOWNERS file to change ownership of their KV paths, DevOps approval is required. We’re evolving this model to allow teams to manage their own CODEOWNERS entries directly. This removes DevOps as a bottleneck and gives teams full autonomy to manage their configuration ownership without waiting for external approvals.

And more! We’re actively exploring additional enhancements, including automated drift detection, pre-deployment testing, and enhanced AI-powered change impact analysis to make configuration management even more robust and developer-friendly.

Conclusion

Moving from UI-based configuration management to GitOps wasn’t just a technical migration; it was a cultural shift in how we think about configuration. By treating configuration as code, we applied the same rigor, safety, and auditability we expect of application code.

The result is a more reliable, transparent, and scalable configuration management system that supports Agoda’s mission to serve millions of travelers worldwide.

How We Eliminated Configuration-Related Outages and Scaled Consul Management at Agoda was originally published in Agoda Engineering & Design on Medium, where people are continuing the conversation by highlighting and responding to this story.

Last year, we shared a behind-the-scenes look at how Agoda runs its bug bounty program, the lessons learned, the challenges of striking the right balance, and where we were heading. Today, we’re announcing the next chapter: Agoda’s bug bounty program is now open to the public on HackerOne.

From Private to Public

Since 2016, Agoda has run a private bug bounty program, working with an invited community of ethical hackers to identify and address vulnerabilities. Going public means opening that collaboration to the broader global security research community, more perspectives, more expertise, and more opportunities to catch what internal teams and private programs might miss.

Why Bug Bounty Programs Matter

Every website has bugs, and some will have a security impact. While traditional security measures like code reviews, penetration testing, and automated tools are critical, they can’t account for everything. A bug bounty program helps channel the skills of ethical hackers toward finding and reporting vulnerabilities, rather than exploiting them.

The scale of the problem is worth keeping in mind. In mid-2024, IBM published its Cost of a Data Breach report, which put the average cost of a breach at $4.88 million USD. The 2024 Verizon Data Breach Investigations Report, drawing on data from 6,963 attacks, identified a 180% increase in cyberattacks originating from software vulnerability exploitations compared to 2023. For a platform like Agoda, handling the payment information, personal details, and booking data of millions of travelers, robust external collaboration is essential.

What’s in Scope

The public program covers Agoda’s core web services and APIs, including Agoda.com and our mobile application. Researchers can test for a range of vulnerabilities, with common findings and their likely severity ranges including:

• Authentication bypass (High to Critical)
• SQL Injection (High to Critical)
• Remote Code Execution (High to Critical)
• Insecure Direct Object Reference (Medium to Critical)
• Stored Cross-site Scripting (Low to High)
• Multi-factor Authentication Bypass (Low to High)

Rewards are structured across four severity tiers:

• Low: $150
• Medium: $500 to $700
• High: $2,000 to $3,000
• Critical: $5,000 to $6,000

Bounties are awarded based on demonstrated impact. Researchers should provide clear steps to reproduce, proof of concept, and a realistic assessment of the vulnerability’s potential impact on Agoda and its users.

How to Participate

Researchers should self-provision test accounts, include a unique HackerOne identifier in HTTP requests, and submit detailed reports that include steps to reproduce, a proof of concept, and demonstrated impact. We commit to a first response within 2 business days and triage within 2 business days, with bounty payment within 14 business days of triage.

Reports should include a vulnerability summary, prerequisites, steps to exploit it, proof of the exploit, and an impact statement. The more detail you provide upfront, the faster we can triage and respond.
Our response targets are:

• Time to first response: 2 business days
• Time to triage: 2 business days
• Time to bounty: 14 business days from triage

Full program details, rules of engagement, and scope are available at https://hackerone.com/agoda-public. We look forward to working with the global security research community.

Agoda Launches Bug Bounty Program on HackerOne to Connect with Global Research Community was originally published in Agoda Engineering & Design on Medium, where people are continuing the conversation by highlighting and responding to this story.

Flights Connectivity Background

At Agoda, our flight booking platform integrates with many external partners, including airlines, OTAs, and providers. Each partner speaks its own dialect, has different API contracts, constraints, and protocols. To manage this complexity, we built a connector layer that sits between Agoda’s platform and each supplier, translating communications, mapping requests and responses, and handling errors and retry logic.

Building and testing new supplier connectors has traditionally been a bottleneck. Our end-to-end testing approach required spinning up a full production-like stack. When bugs surfaced late in this process, fixing them meant rerunning the entire suite, adding significant time to delivery.

In this blog, we share how we built a Workflow Simulator that validates supplier connectors by simulating their surrounding ecosystem, enabling engineers to run realistic multi-step scenarios locally and catch integration issues earlier in the development cycle.

What is the connector?

The connector is a critical layer in the architecture that sits between Agoda and suppliers, translating context across all business operations, including search, pricing, ancillaries, and booking, as well as post-booking flows. Because each supplier has its own variations, the connector is responsible for normalization. This includes field mapping, error handling, coordinating multiple API calls, and business logic to transform supplier-specific responses into Agoda’s context.

Since the connector facilitates the translation of all business operations, it requires extensive testing to ensure these operations work as expected and are normalized correctly. Incorrect mapping logic in the connector can propagate across multiple domains or expose customers to incorrect or incomplete information, leading to a poor user experience.

It is not enough to validate only individual business operations. We must ensure that the entire end-to-end (E2E) flow works correctly. Whenever a new Connector is built, E2E testing is mandatory to give us confidence that Agoda can interact successfully with the new supplier through the newly implemented connector, from itinerary search to successful booking and PNR issuance, as well as post-booking operations such as cancellation.

The Challenge with Testing Supplier Connectors

As described above, as a supplier connector is business critical, it usually requires substantial testing effort–including unit, functional, and E2E tests–to ensure the quality and reliability of the supplier integration.

For E2E testing, we typically run these tests only after the connector implementation is complete. This involves spinning up the full stack, mirroring the production architecture, and connecting it to the supplier’s test environment. The stack itself includes multiple components. These range from frontend services to asynchronous messaging systems, caching layers, multiple microservices, background workers, etc.

Once the E2E stack is ready, we execute multiple test scenarios starting from the UI and validate both the functional outcomes and the supplier request-response logs. The logs are then collected and used for test assertions.

This methodology slows development, increases costs, and often surfaces defects late in the delivery cycle. The tests themselves are highly supplier-specific with limited reusability. When bugs are discovered during E2E testing, they introduce higher project risk and significantly greater remediation cost. Moreover, after a fix is applied, we typically need to rerun the entire E2E test suite, which further increases effort and slows delivery.

Additionally, E2E tests can be blocked by issues in unrelated dependencies, such as platform issues or unstable upstream or downstream services, which can slow down the supplier integrations.

To address these challenges, we asked a simple question: What if we could test connectors without spinning up the entire system around them?

Introducing Workflow Simulator

A more effective approach is to validate connectors by simulating their surrounding ecosystem: modeling the upstream callers, generating realistic multi-step requests for specific business scenarios, preserving state across business processes, and asserting behavior across the entire workflow.

For example, instead of relying on real upstream systems from the E2E environment, the simulator can execute a complete booking flow from searching to booking with minimal resources. This allows the test to run faster, automatically, without irrelevant dependencies or context.

Because this simulator is designed specifically for connector testing, it prioritizes simplicity and clarity of execution over scalability and availability. For instance, while production systems rely on caching and asynchronous communication to achieve scale, the simulator intentionally omits these optimizations. Instead, it performs synchronous calls and records all interactions in a straightforward and transparent manner, making execution easier to trace and failures easier to diagnose.

The simulator builds realistic scenarios that reflect business logic in production, such as searched itineraries, selected pricing, provided passenger information, selected ancillaries, etc. It then simulates the same sequence of requests that upstreams would make. Across these calls, it maintains state so that carry-over data, such as supplier-specific data, selected seats, paid bags, etc., remains consistent throughout the flow.

Once the execution completes, the simulator runs assertions on the recorded request-response pairs at both the individual endpoint level and across multiple endpoints to ensure end-to-end connector correctness.

Architecture

At a high level, the Workflow Simulator replaces upstream systems with a lightweight, controllable testing environment. It consists of four main components that work together to simulate, execute, and validate connector behavior.

1. Scenario Builder: defines test criteria and booking
2. Test Runner: runs a list of scenarios
3. Workflow Executor: simulates the complex architecture and traffic
4. Assertions: validates correctness

Scenario Builder

The Scenario Builder enables us to define target acceptance testing scenarios that reflect real business use cases.

For example:

• Domestic one-way flight: 2 adults, 1 child, 1 infant (economy class with paid baggage)
• International multi-segment round-trip: 2 adults (business class)
• Domestic one-way with technical stop: 1 adult (economy class with paid seat)

This Scenario Builder instantiates the Scenario Context, which contains key scenario information such as passenger composition, itinerary details, cabin class, and selected ancillaries. These attributes are later used for both request generation and assertions, ensuring consistency across the entire workflow.

Some fields that would normally originate from upstream systems or user input are generated by the Scenario Builder using a strategy pattern. Depending on the objective, we can use:

• A deterministic strategy to keep values fixed and repeatable.
• A randomization strategy to generate unique requests or identifiers for each execution.

The latter is particularly useful for suppliers with stateful test environments, where repeated identifiers may be rejected or cached across runs.

Workflow Executor

The Workflow Executor is responsible for simulating the upstream behaviors and orchestrating the sequence of calls to a connector. This executor is supplier-agnostic because it interacts with a connector via internal contracts, so it can be used with any supplier, including new suppliers under development.

The workflow is modeled as a directed acyclic graph (DAG) that represents the simulated business steps. A DAG is a way to represent tasks in which each node has a clear direction, and no path loops back to an earlier node, meaning there are no circular dependencies. The main benefit of this representation is that it helps determine the sequence of execution, improving clarity and making dependencies easier to understand.

Complex interactions such as asynchronous operations, cache polling, and message passing are simplified to nodes and edges on the DAG below.

Once the workflow DAG is initialized, the executor traverses the graph from the start node to the end node. As it moves through the workflow graph, it interacts with most nodes, then with the connector, and eventually with the suppliers. The executor records each request and response and maintains a shared state, carrying forward any required information from one step to the next.

After workflow execution terminates, either because an error occurs or because the final node is reached, the executor runs the subsequent assertions based on the recorded interactions and accumulated state.

Assertions

The simulator supports two types of assertions: endpoint assertions and workflow assertions.

Endpoint assertion

It validates a single connector endpoint’s request and response, including contract validation, required fields, field type, format, and length constraints, and logical consistency across fields.

For example, at the pricing endpoint, it validates that the flight number field is alphanumeric and of valid length, and that the total price matches the sum of per-passenger prices.

Workflow-Scenario assertion

It validates end-to-end consistency across multiple connector endpoints using the defined scenario context and recorded request–response logs. It ensures responses align with the intended scenario (e.g., passenger details, leg count for round-trip itineraries, segment counts for multi-segment itineraries), verifies cross-step data propagation, and compares supplier-side booking details against the recorded requests and the scenario, including itinerary attributes and passenger information.

For example, for paid baggage, the Ancillaries Pricing step returns available baggage options, the selected baggage is included in the booking request, and the same baggage selection is reflected when retrieving booking information from the supplier.

Test Runner

The test runner serves as the entry point for initializing the simulator and executing the connector under test. Even if the implementation is not yet complete, engineers can still use the simulator to automate interactions across multiple suppliers’ endpoints and validate behavior and mapping logic using endpoint-level assertions.

Benefits

This approach enables fast local testing by allowing engineers to run realistic flows on their machines without spinning up the entire E2E stack. It significantly shortens the feedback loop, making it easier to reproduce issues, iterate on fixes, and validate changes early in the development cycle.

It also supports scenario-aware assertions. Instead of only checking that a single endpoint returns a valid response, the framework validates that overall behavior matches the intended scenario.

Because the simulator communicates through the connector’s internal contract, it is supplier-agnostic and reusable. This simulator can be applied across multiple suppliers and connectors with minimal additional work.

This simulator enables shift-left testing, as engineers can run it on realistic flows early in implementation. It catches issues sooner and reduces the risk of late-cycle defects and downstream surprises.

Limitations and Boundaries

This approach is not a replacement for full end-to-end testing. The E2E test is required for final validation and integration to verify behavior against the production architecture, including asynchronous communication, messaging, and parallelism.

There are some complex failures that may not be applicable to the workflow, or we may need to support this in the future. For example, race conditions and production-only constraints, such as rate limiting, cannot be reproduced without a full staging environment.

Finally, if a supplier’s test environment is unstable, that instability will propagate to the workflow simulator, reducing its reliability and overall effectiveness.

What’s next?

Testing supplier connectors by simulating the world around them, driving realistic scenarios, maintaining state across calls, and asserting correctness at both endpoint and workflow levels gives teams faster feedback and earlier visibility into integration issues.

Looking ahead, we will evolve the simulator to support richer, more production-like scenarios and explore AI-assisted techniques to help generate scenarios and edge cases, improving test coverage and quality. We’ll also expose the simulator capability to suppliers so both sides can validate the integration earlier, with shorter feedback loops and lower communication overhead.

How Agoda Simulates Booking Flows to Test Flight Integrations was originally published in Agoda Engineering & Design on Medium, where people are continuing the conversation by highlighting and responding to this story.

Introduction

When your white-label partner asks for a small color change, and it takes much more effort than it should, you know your design system does not scale. At Agoda, we work with multiple white-label travel partners, each with its own brand and visual identity. Our platform must adapt its appearance to match each partner’s brand identity, while still being powered by a single design system. Doing this in a scalable, easy-to-operate way has been a key challenge, especially as the number of partners has grown. In this post, you will learn how we power multiple brands from one design system, and how you can apply the same principles in your own work.

What is white-labeling?

At its core, white-labeling is the practice of creating a product or service that can be rebranded with a partner’s own name, logo, and customized visuals, while sharing the same underlying technology. In the context of design systems, white-labeling means building a set of UI components and styles that can be easily adapted to different brand identities without requiring code rewriting or duplication.

Agoda Design System

Challenges of managing white-label styling

One of the goals of white-label styling is to make global UI changes easy for us, while still allowing partners to customize their own look in a controlled way. Without a shared system, we end up achieving neither.

Each white-label partner typically has its own styling implementation. There is no single place to define shared styling decisions or tokens, and no clear boundary between what is shared and what is customizable. As a result, even minor visual changes must be applied manually to every partner.

For example, imagine changing a core UI token such as the standard button border radius or spacing values used across components. The intention is the same everywhere, but without a shared mechanism, the change has to be applied separately in each partner’s code. With many partners, this quickly becomes time‑consuming and error‑prone. Some partners may be missed, while others may implement the change differently.

This setup also makes it harder for partners to update their own values. Styling is often scattered across copied code rather than expressed as clear, overridable tokens. Small adjustments become risky, and it is not always obvious which values are safe to change without breaking layout or consistency.

Design tokens in the Agoda design system

Design tokens play an important role in Agoda’s design system. They are named, reusable values that store the visual decisions of a design system. We use design tokens instead of hard‑coding values to represent colors, spacing, font sizes, border radius, shadows, and more. For example, the `color.primary` token stores a brand’s primary color; one partner might use blue while another uses purple.

Another example is a token like `radius.base`, which controls the component border radius. This allows one brand to have rounded buttons while another can use squared buttons.

A design token is an identifier that maps to a concrete value, such as a color or a spacing value. You can see in Figure 2, for example, base.background that maps to a certain color. Changing these mappings allows the same identifiers to be used in the code while returning different real values across different white-label themes.

Design tokens are the mechanism that enables this customization. The goal is to allow flexibility while preserving the brand and product's core identity and cohesion, since changing a color token affects multiple areas. Now that we know what design tokens are, we can look at themes that use them.

Themes in the Agoda design system

In the Agoda design system, themes define how design tokens are used and overridden for a specific brand or context. Design tokens represent the core design decisions, such as colors, spacing, and typography. Themes provide a structured way for partners to replace selected token values with their own.

Themes provide partners with a structured interface for overriding tokens without modifying component code. For example, as shown in Figure 3A and Figure 3B, a partner can override the token used for the primary button color so it matches the color used elsewhere in their brand.

Benefits of theming system

The main benefits of theming are:

• Scalability: Reuse the shared component logic across partners and swap token values to deliver a unique UI for each theme. New partner = new theme, not new code.
• Maintainability: Centralize all the design tokens in a single source of truth (e.g., colors, spacing, and typography). Developers can easily review and update all themes in one place.
• Efficiency: Onboard a new partner quickly by creating a new theme and overriding design tokens from the base theme.

White-labeling on a technical level

Drone-js is Agoda’s shared library for building user interfaces. Our components are designed in Figma, built with React, and styled using ADS CSS utilities. This setup helps designers and developers work together efficiently and ensures our products look and feel consistent.

Figure 4A describes the main architecture of the Agoda Design System. The designers design the components in Figma, and after rounds of reviews and feedback, the design is finalized, and the developers implement it.

The drone-js library is composed of multiple NPM packages, including:

Core and base theme

• @drone-js/core — provides the core components and other foundational parts.
• @drone-js/theme — contains the base theme used across all brands.

White-label themes

Brand-specific theme packages such as:

• @drone-js/theme-agoda
• (and other white-label themes in their own @drone-js/theme-* packages)

Each of these themes builds on @drone-js/theme by applying brand-specific overrides.*

Consumer apps import @drone-js/core and all required white-label themes. The correct white-label theme is applied based on the information received from the backend.

The anatomy of a white-label theme

Our white-label theme NPM packages, @drone-js/theme and brand-specific themes, such as @drone-js/theme-agoda, consist of two main parts: CSS files and JavaScript code. These packages are built using both the default theme JSON and a white-label theme JSON.

The default theme JSON defines the base values for all design tokens. The white-label theme JSON contains only the token overrides specific to a brand. During the package building process, the white-label theme JSON is deep-merged into the default theme JSON to ensure all tokens exist and that the override theme can only include the tokens it wants to override.

Figure 4B shows a default theme JSON that provides base values for all design tokens, while a white-label theme JSON defines some overrides; these two files are deep-merged into a complete Theme JSON, then used to generate the final main.css and index.js file and bundle into the NPM package (Figure 4C theme-agoda package) that has all the necessary variables and overrides for the white-label theme.

When the theme is bundled into an NPM package for use, as shown in Figure 4C, only the CSS and JS files are included. The theme JSON file is not needed since it is only used to generate the CSS that will be applied. Essentially, the JSON is the source of truth, and everything that we need is generated from it. When JSON updates are made, the files are regenerated, and a new version of the theme is released.

How theming works at runtime

The user interface of our application is built using the same UI framework, @drone-js/core, across all partners. Our UI framework renders components with the same structure and core logic that control component behavior, regardless of the partner. At the same time, the framework picks up styling from the partner’s theme package at runtime, such as @drone-js/theme-agoda, allowing visual differences to be expressed entirely through CSS variables.

Check out our primary button as an example, showing each button with a unique look, including different colors, spacing, and border radii. In the DOM, all of the buttons have the same structure and class names, which are rendered by our UI framework (@drone-js/core). The theming system, from the theme package, uses main.css to define the CSS properties and theme.css to provide the theme-specific CSS variable overrides, allowing visual differences for each button.

How does the application pick up the theme?

Our UI framework provides a context provider, the Design System Context Provider, which configures the framework's behavior, such as text direction (e.g., LTR or RTL). One of its responsibilities is the theming system: it receives the white-label theme and injects the corresponding CSS variable values. Each component then references these variables via design tokens, allowing its appearance to be rendered according to the selected theme.

The Design System Context Provider wraps the entire application and is configured with the white-label theme. It injects all CSS variables for the given theme, allowing UI components placed as its children to automatically render with the appropriate styles.

Theme testing tools

We adopt and build multiple tools to test our theming system and preview how our theme is applied across all components.

Storybook

We have customized our Storybook by adding a Theme selector in the toolbar. The switcher updates our Design System Context Provider at runtime, so you can toggle between white-label themes and see how the same component is rendered in different themes. Developers consuming the library can also view supported props and accessibility guidelines in Storybook.

Design token visualization

Design tokens play a major role in the theming system. They control how components look and give each partner its own visual identity. We built a dedicated Storybook page for browsing tokens like color, spacing, and typography. This page makes tokens easier to understand, verifies their correctness, and improves visibility for both designers and developers who work closely with design tokens.

This page also works with the theme switcher, so we can see how the design tokens are displayed in each white-label theme.

Design token live editor

Our themes are stored as JSON, a human-readable format that is easy to understand and edit. However, updating token values can be unintuitive for non-technical contributors, such as designers, and prone to human error. Our Design Token Live Editor provides a web interface for editing design tokens and instantly previewing changes.

This tool bridges the gap between designers and developers. Designers are the primary users. After completing the token update and reviewing the changes, they can export a theme as JSON and hand it off to developers, who can then integrate the file directly into the codebase. It can also be useful for other stakeholders, such as product managers.

Conclusion

The biggest thing we got right was separating shared component logic from brand-specific styling. Design tokens are the contract between the two. That separation is what makes the system scale; adding a new partner means adding a new theme file, not touching component code.

Figure 10 shows the full picture, from design in Figma through component development and theming in drone-js, to theme switching in consumer applications.

In summary, Agoda’s white-labeling strategy demonstrates how a thoughtfully architected design system can power a diverse ecosystem of brands and make it easy to onboard new partners and make edits to existing ones.

For more on how we build our design system, check out these Agoda blogs:

• Building and scaling different travel websites with one codebase
• Pixels to Product: Innovating with Design System

How Agoda Powers Multiple Travel Brands with One Design System was originally published in Agoda Engineering & Design on Medium, where people are continuing the conversation by highlighting and responding to this story.