<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Adam Miskiewicz on Medium]]></title>
        <description><![CDATA[Stories by Adam Miskiewicz on Medium]]></description>
        <link>https://medium.com/@skevy?source=rss-9a357753a36f------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*lAL-k5N6WdzqvH5ceZ6-QA.jpeg</url>
            <title>Stories by Adam Miskiewicz on Medium</title>
            <link>https://medium.com/@skevy?source=rss-9a357753a36f------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 22:47:31 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@skevy/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Viaduct 1.0 and the future of Airbnb’s data mesh]]></title>
            <link>https://medium.com/airbnb-engineering/viaduct-1-0-and-the-future-of-airbnbs-data-mesh-6bab4ec98b89?source=rss-9a357753a36f------2</link>
            <guid isPermaLink="false">https://medium.com/p/6bab4ec98b89</guid>
            <category><![CDATA[engineering]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[infrastructure]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[graphql]]></category>
            <dc:creator><![CDATA[Adam Miskiewicz]]></dc:creator>
            <pubDate>Wed, 13 May 2026 17:01:00 GMT</pubDate>
            <atom:updated>2026-08-18T17:45:04.369Z</atom:updated>
            <content:encoded><![CDATA[<h4>Moving from an internal tool to a community-driven, production-ready data mesh.</h4><figure><img alt="Stone arch bridge crossing a wide river at sunset, with people leisurely walking along the top, golden light illuminating the weathered masonry. A vintage black-metal streetlamp appears in the foreground, while historic buildings and leafless winter trees line the riverbank in the background." src="https://cdn-images-1.medium.com/max/1024/1*uxoiyQybKqmOHm_AdF1kVg.jpeg" /></figure><p><strong>By</strong>: <a href="https://www.linkedin.com/in/ryanetanner/">Ryan Tanner</a>, <a href="https://www.linkedin.com/in/rstata/">Raymie Stata</a>, <a href="https://www.linkedin.com/in/adammiskiewicz/">Adam Miskiewicz</a></p><h3>Introduction</h3><p>We’re excited to announce the 1.0 release of the Viaduct. This release marks a shift from Viaduct being an Airbnb-internal tool that happens to be open source to a true community-driven project with a stable public API. The 1.0 release includes substantial new features and enhancements which we describe in the <a href="https://viaduct.airbnb.tech/blog/2026/05/13/viaduct-10-whats-new/">Viaduct blog</a>.</p><p>Viaduct is for platform engineers building a company-wide data API, service owners who want to contribute to a shared graph without spinning up their own server, or engineering organizations that have outgrown a single GraphQL service.</p><h3>What is Viaduct?</h3><p>Viaduct is Airbnb’s data-oriented service mesh, a GraphQL-based system that provides a single interface for accessing and interacting with any data source. For years it has supported Airbnb’s data infrastructure, allowing product engineers to access data efficiently and safely, while enabling service owners to decouple implementation details from the API surface.</p><h3>What is a data-oriented service mesh?</h3><p>A Viaduct service mesh is defined in terms of a GraphQL schema consisting of:</p><ul><li><em>Types</em> (and interfaces) describing data managed within your service mesh</li><li><em>Queries</em> (and subscriptions) providing means to access that data, abstracted from the service entry points that provide the data</li><li><em>Mutations</em> providing ways to update data, again abstracted from service entry points</li></ul><h3>Why Viaduct?</h3><p>Viaduct was built to solve a specific problem faced by most organizations that adopt GraphQL strategically: <strong>decentralized development of a central schema</strong>.</p><h3>Why a central schema?</h3><p>A central schema provides a single, consistent interface to the full range of an organization’s data and capabilities. Instead of every client needing to know which backend service to call, they interact with one unified graph that connects all of an organization’s domains. This makes APIs easier to discover, enables richer cross-domain queries, and provides a consistent place to enforce policies, observability, and schema governance.</p><h3>Why decentralized development?</h3><p>A central schema only works if it can evolve quickly. The domain experts who understand each part of the business must be able to design and implement the parts of the schema they know best. A central team cannot own everything, and shouldn’t try to. The challenge is giving teams autonomy over their own domain contributions while preserving the coherence and stability of the shared schema.</p><p>Viaduct solves this through multi-tenancy. A shared multi-tenant runtime hosts independently developed and tested tenant modules, each owning a portion of the schema. A team wanting to contribute simply creates a directory for their module, defines their schema definition language (SDL) and resolvers, and they are ready to serve. There is no need to set up or operate a separate GraphQL service, manage router composition, or become experts in GraphQL infrastructure. Teams focus on domain logic; the platform handles execution, scaling, and integration.</p><h3>Viaduct and GraphQL Federation</h3><p>We’re frequently asked how Viaduct compares with GraphQL Federation. Both address the same problem — decentralized development of a central schema — but they take different approaches.</p><p>Federation distributes development through services. Each team owns and operates its own GraphQL subgraph server; those subgraphs are composed by a federation router into a single unified graph. Viaduct distributes development through modules. A shared multi-tenant runtime hosts tenant modules that define and implement portions of the schema.</p><p><strong>Federation distributes development by distributing servers. Viaduct distributes development by distributing modules.</strong></p><p>We don’t see Viaduct as an alternative to federation, but as a complement to it. Viaduct can participate as a subgraph within a federated architecture. In a large organization where hundreds of teams contribute to the overall graph, a federated approach requires running hundreds of independent subgraph servers. With Viaduct, organizations can instead run a smaller number of Viaduct instances, each hosting many closely related tenant modules. Federation can then compose those instances into a larger enterprise graph.</p><h3>Community</h3><p>The “1.0” designation for Viaduct is a commitment to stability. Until now, Viaduct has evolved rapidly to meet internal needs, often with breaking changes managed through our internal monorepo tooling. Public release required a different approach. We have applied @StableApi, @ExperimentalApi, and @InternalApi annotations across all public surfaces, and we run Kotlin’s binary compatibility validator in CI to catch breaking changes before they ship. Viaduct is now published to Maven Central with automated releases and Dokka-generated API documentation.</p><p>We are committed to developing Viaduct in the open. Our intent is to involve the community in major architectural decisions before code is written, not after. Our first public discussion is the <a href="https://github.com/airbnb/viaduct/discussions/271">Connections RFC on GitHub</a>, and we plan to continue in that direction. Our goal going forward is to be a genuine community project, not simply an internal project that happens to be open source.</p><p>Whether you are looking to unify your data layer, contribute to the core engine, or build on top of the graph, now is the time to get involved. <a href="https://viaduct.airbnb.tech/getting_started/">Begin with Getting Started →</a></p><p>If this type of work interests you, check out some of our <a href="https://careers.airbnb.com/">open roles</a>!</p><h3>GraphQLConf 2026</h3><p>Heading to <a href="https://graphql.org/conf/2026/">GraphQLConf</a> next week? Check out four Viaduct-powered conversations by Airbnb engineers on May 20th.</p><p>Speaker: <a href="https://graphql.org/conf/2026/schedule/0a67525d56d469af7df6fc4763e3f75e/?name=Brute%20Force%20Correctness%20-%20James%20Bellenger,%20Airbnb">James Bellenger</a></p><p>Time: 3:50–4:15 PM PST</p><p>This talk explains how probabilistic testing exposes hidden bugs in complex GraphQL systems — demonstrated by Airbnb’s launch of a new GraphQL engine — and shows how you can use the same approach to harden your own systems.</p><p>Speaker: <a href="https://graphql.org/conf/2026/schedule/d1bdf1a3eb90cb599c172cbdfa7fdd1c/?name=Observability%20for%20a%20Multi-Tenant%20GraphQL%20Gateway%20at%20Scale%20-%20Vickey%20Yeh,%20Airbnb">Vickey Yeh</a></p><p>Time: 1:55–2:20 PM PM PST</p><p>A look at how Airbnb’s Viaduct system lets each team easily monitor and debug its own code — using built-in ownership tags, automatic alerts/dashboards, and cost-aware tracing — so everyone can treat their part of the shared service as if it were their own.</p><p>Speakers: <a href="https://graphql.org/conf/2026/schedule/a2d6aff0874a12a86810f7ffac23d12d/?name=Sharding%20a%20GraphQL%20Gateway%20for%20Blast%20Radius%20Reduction%20-%20Linquan%20Zhang%20&amp;%20Cetin%20Sahin,%20Airbnb">Linquan Zhang and Cetlin Sahin</a></p><p>Time: 2:30–2:55 PM PST</p><p>We will cover how we architected our sharding solution and how it improved our operational abilities. You will gain a clear understanding of how our implementation tradeoffs have fared over time, key production insights gathered since rollout, and strategies to evolve a GraphQL gateway towards greater isolation without fragmenting the API surface.</p><p>Speaker: <a href="https://graphql.org/conf/2026/schedule/f95ae2cfb0b6a5e2e0dfa5971f531749/?name=GraphQL%20Data%20Mocking%20at%20Scale%20With%20LLMs%20and%20@generateMock%20-%20Michael%20Rebello,%20Airbnb">Michael Rebello</a></p><p>Time: 3:05–3:30 PM PST</p><p>Producing valid and realistic mock data for prototyping and testing has been an unsolved challenge for years. Mock data is tedious to write and maintain, but attempts to improve the process such as random value generation and field stubbing fall short as they lack essential domain context to make test data realistic and meaningful. In this talk, I’ll share how we’ve reimagined GraphQL mocking at Airbnb by combining existing GraphQL infrastructure, rich product and schema context, and LLMs to generate convincing, type-safe mock data simply by adding a directive (@generateMock) to a field or operation.</p><p>Whether you’re fine-tuning a single service or running a multi-tenant gateway, these sessions will equip you with practical strategies to build robust, observable, and developer-friendly GraphQL systems. See you on May 20th!</p><h3>Acknowledgments</h3><p>Thanks to the entire Viaduct team, and especially Aileen Chen and Raymie Stata, for the tireless work on Viaduct Modern.</p><p><em>All product names, logos, and brands are property of their respective owners. All company, product, and service names used in this website are for identification purposes only. Use of these names, logos, and brands does not imply endorsement.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6bab4ec98b89" width="1" height="1" alt=""><hr><p><a href="https://medium.com/airbnb-engineering/viaduct-1-0-and-the-future-of-airbnbs-data-mesh-6bab4ec98b89">Viaduct 1.0 and the future of Airbnb’s data mesh</a> was originally published in <a href="https://medium.com/airbnb-engineering">The Airbnb Tech Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Viaduct, five years on: Modernizing the data-oriented service mesh]]></title>
            <link>https://medium.com/airbnb-engineering/viaduct-five-years-on-modernizing-the-data-oriented-service-mesh-e66397c9e9a9?source=rss-9a357753a36f------2</link>
            <guid isPermaLink="false">https://medium.com/p/e66397c9e9a9</guid>
            <category><![CDATA[data]]></category>
            <category><![CDATA[graphql]]></category>
            <category><![CDATA[engineering]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[api]]></category>
            <dc:creator><![CDATA[Adam Miskiewicz]]></dc:creator>
            <pubDate>Mon, 15 Sep 2025 18:54:00 GMT</pubDate>
            <atom:updated>2026-08-18T17:45:55.950Z</atom:updated>
            <content:encoded><![CDATA[<h4>A more powerful engine and a simpler API of rour data-oriented mesh.</h4><figure><img alt="Ancient three-tier stone aqueduct with arches spanning a green river gorge under a clear blue sky." src="https://cdn-images-1.medium.com/max/1024/1*q_owFEOLfQlioFXHP7UqBA.avif" /></figure><p><strong>By</strong>: Adam Miskiewicz, Raymie Stata</p><p>In November 2020 we <a href="https://medium.com/airbnb-engineering/taming-service-oriented-architecture-using-a-data-oriented-service-mesh-da771a841344">published</a> a post about Viaduct, our data-oriented service mesh. Today, we’re excited to announce Viaduct is available as open-source software (OSS) at <a href="https://github.com/airbnb/viaduct">https://github.com/airbnb/viaduct</a>.</p><p>Before we talk about OSS, here’s a quick update on Viaduct’s adoption and evolution at Airbnb over the last five years. Since 2020, traffic through Viaduct has grown by a factor of eight. The number of teams hosting code in Viaduct has doubled to 130+ (with hundreds of weekly active developers). The codebase hosted by Viaduct has tripled to over <strong>1.5M</strong> lines (plus about the same in test code). We’ve achieved all this while keeping operational overhead constant, halving incident-minutes, and keeping costs growing linearly with QPS.</p><h3>What’s the same?</h3><p>Three principles have guided Viaduct since day one and still anchor the project: a <strong>central schema</strong> served by <strong>hosted business logic</strong> via a <strong>re-entrant</strong> API.</p><p><strong>Central schema <br></strong>Viaduct serves our central schema: a single, integrated schema connecting all of our domains across the company. While that schema is developed in a <em>decentralized</em> manner by many teams, it’s one, highly connected graph. Over 75% of Viaduct requests are internal because Viaduct has become a “one‑stop” data-oriented mesh connecting developers to all of our data and capabilities.</p><p><strong>Hosted business logic <br></strong>From the beginning, we’ve encouraged teams to host their business logic directly in Viaduct. This runs counter to what many consider to be best practices in GraphQL, which is that GraphQL servers should be a thin layer over microservices that host the real business logic. We’ve created a serverless platform for hosting business logic, allowing our developers to focus on writing business logic rather than on operational issues. As noted by Katie, an engineer on our Media team:</p><blockquote>“As we migrate our media APIs into Viaduct, we’re looking forward to retiring a handful of standalone services. Centralizing everything means less overhead, fewer moving parts, and a much smoother developer experience!”</blockquote><p><strong>Re-entrancy</strong></p><p>At the heart of our developer experience is what we call <em>re-entrancy</em>: Logic hosted on Viaduct composes with other logic hosted on Viaduct by issuing GraphQL fragments and queries. Re-entrancy has been crucial for maintaining modularity in a large codebase and avoiding classic monolith hazards.</p><h3>What’s changed?</h3><p>For most of Viaduct’s history, evolution has been bottom-up and reactive to immediate developer needs. We added capabilities incrementally, which helped us move fast, but also produced multiple ways to accomplish similar tasks (some well‑supported, others not) and created a confusing developer experience, especially for new teams. Another side-effect of this reactive approach has been a lack of architectural integrity. The interfaces between the layers of Viaduct, described in more detail below, are loose and often arbitrary, and the abstraction boundary between the Viaduct framework and the code that it hosts is weak. As a result, it has become increasingly difficult to make changes to Viaduct without disrupting our customer base.</p><p>To address these issues, over a year ago we launched a major initiative we call <strong>“Viaduct Modern”</strong>, a ground-up overhaul of both the developer-facing API and the execution engine.</p><h3>Tenant API</h3><p>One driving principle of Viaduct Modern has been to simplify and rationalize the API we provide to developers in Viaduct, which we call the <strong>“Tenant API”</strong>. The following diagram captures the decision tree one faced when deciding how to implement functionality in the old API:</p><figure><img alt="Flowchart for choosing a GraphQL wiring: Flow: Node → node helpers. Connection → connection helpers. Node field → @serviceBackedNodeKey. Need schema → @derivedField/@component. Singleton field → @singleton. Else → @customFromCodegen." src="https://cdn-images-1.medium.com/max/1024/1*0Gujn6vAspuKh3ZuPjlK-g.png" /><figcaption>Viaduct’s original complex programming model</figcaption></figure><p>Each oval in this diagram represents a different mechanism for writing code. In contrast, the new API offers just two mechanisms: node resolvers and field resolvers.</p><figure><img alt="A simple flowchart. Decision box “Wiring a Node?”: Yes → “Node resolver.” No → arrow right to “Everything else,” which points down to “Field resolver.”" src="https://cdn-images-1.medium.com/max/1024/1*xuL1H_ylQR5PU2eh2JuXEA.png" /><figcaption>Viaduct Modern’s simpler model</figcaption></figure><p>The choice between the two is driven by the schema itself, not ad‑hoc distinctions based on a feature’s behavior. We unified the APIs for both resolver types wherever possible, which simplifies dev experience. After four years evolving the API in a use‑case‑driven manner, we distilled the best ideas into a single simple surface (and left the mistakes behind).</p><h3>Tenant modularity</h3><p>Strong abstraction boundaries are essential in any large codebase. Microservices achieve this via service definitions and RPC API boundaries; Viaduct achieves it via <strong>modules</strong> plus <strong>re‑entrancy</strong>.</p><p>Modularity in the central schema and hosted code has evolved. Initially, all we had was a vague set of conventions for organizing code into team-owned directories. There was no formal concept of a module, and schema and code were kept in separate source directories with unenforced naming conventions to connect the two. Over time, we evolved that into a more formal abstraction we call a “tenant module.” A tenant module is a unit of schema together with the code that implements that schema, and crucially, is owned by a single team. While we encourage rich graph‑level connections across modules, we <strong>discourage direct code dependencies</strong> between modules. Instead, modules compose via GraphQL fragments and queries. Viaduct Modern extends and simplifies these re‑entrancy tools.</p><p>Let’s look at an example. Imagine two teams, a “Core User” team that owns and manages the basic profile data of users, and then a “Messaging” team that operates a messaging platform for users to interact with each other. In our example, the Messaging team would like to define a <em>displayName</em> field on a <em>User</em>, which is used in their user interface. This would look something like this:</p><p><strong>Core user team</strong></p><pre>type User implements Node {<br>  id: ID!<br>  firstName: String<br>  lastName: String<br>  … <br>}</pre><pre>class UserResolver : Nodes.User() {<br>    @Inject<br>    val userClient: UserServiceClient<br><br>    @Inject<br>    val userResponseMapper: UserResponseMapper<br><br>    override suspend fun resolve(ctx.Context): User {<br>        val r = userClient.fetch(ctx.id)<br>        return userResponseMapper(r)<br>    } <br>}</pre><p>This is the base definition of the <em>User</em> type that lives in the Core User team’s module. This base definition defines the first- and last-name fields (among many others), and it’s the Core User team’s responsibility to materialize those fields.</p><p><strong>Messaging team</strong></p><pre>extend type User {<br>  displayName: String @resolver<br>}</pre><pre>@Resolver(&quot;firstName lastName&quot;)<br>class DisplayNameResolver : UserResolvers.DisplayName() {<br>    override suspend fun resolve(ctx: Context): String {<br>        val f = ctx.objectValue.getFirstName()<br>        val l = ctx.objectValue.getLastName()<br>        return &quot;$f ${l.first()}.&quot;<br>    }<br>}</pre><p>The Messaging team can then extend the <em>User </em>type with the display name field, and also indicates that they intend to provide a resolver for it. The code has an <em>@Resolver </em>annotation that indicates which fields of the <em>User</em> object it needs to implement the <em>displayName</em> field. The Messaging team doesn’t need to understand which module these fields come from, and their code doesn’t depend on code from the Core User team. Instead, the Messaging team states their data needs in a declarative fashion.</p><h3>Framework modularity</h3><p>A major goal of Viaduct Modern has been to make the framework itself more modular. We want to enable faster improvements to Viaduct — especially regarding performance and reliability — without extensive changes to application code. Viaduct is composed of three main layers: the GraphQL execution engine, the tenant API, and the hosted application code. While these layers made sense, the interfaces between them were weak, making Viaduct difficult to update. The new design is focused on creating strong abstraction boundaries between these layers to improve flexibility and maintainability:</p><figure><img alt="Three light-blue boxes labeled “Tenant 1,” “Tenant 2,” and “Tenant 3” sit side-by-side at the top. Vertical lines from each box connect down to a horizontal, pill-shaped light-blue bar labeled “Viaduct Tenant API.” Below a dotted line, that bar connects to a large pink rectangle labeled “Viaduct Engine,” indicating that all tenants interact with the shared engine through the common Tenant API." src="https://cdn-images-1.medium.com/max/1024/1*qvOOmI-Hs7-PMa52kSACqQ.png" /><figcaption>Viaduct Modern’s modular design</figcaption></figure><p>The most significant change is the boundary between the <strong>engine</strong> and the developer-facing <strong>tenant API</strong>. In the previous system, that boundary hardly existed. Viaduct Modern defines a strong <strong>engine API</strong> whose core is a <strong>dynamically‑typed</strong> representation of GraphQL values (input and output objects as simple maps from field name → value). The tenant API, by contrast, is <strong>statically typed</strong>: we generate Kotlin classes for every GraphQL type in the central schema. In the new architecture, generated types are thin wrappers over the dynamic representation. The tenant API forms the bridge between the engine’s untyped world and tenants’ typed world.</p><p>This separation lets us evolve the engine in relative isolation (to improve latency, throughput, and reliability) and evolve the tenant API in relative isolation (to improve dev experience). Large changes will still cross the boundary, but as Viaduct Modern stabilizes, that should be rare.</p><h3>Migration without a “big bang”</h3><p>Viaduct Modern would be a non‑starter if it required a step‑function migration of a million+ lines of code. To enable gradual migration, we’re shipping <strong>two tenant APIs</strong> side‑by‑side — the new <strong>Modern</strong> API and the existing <strong>Classic</strong> API — both on top of the new engine. This lets teams realize performance/cost wins from the engine immediately, while adopting the ergonomic wins of the Modern API over time.</p><p>Shipping two APIs has also improved the engine API design: building two tenant runtimes simultaneously forced us to keep the engine’s concerns clean and general. Over time, we expect to build additional tenant APIs on the engine (e.g., <strong>TypeScript</strong>).</p><h3>Other improvements</h3><p>Viaduct has seen a lot of other improvements since 2020. The story is too long to be told in this post, but to list some of the highlights:</p><ul><li><strong>Observability.</strong> Hosting software from 100+ teams means our framework has to make ownership and attribution crystal clear. In the old system, there is no clear dividing line between tenant code and framework code, so our instrumentation includes a bit of guesswork in its attribution to parties. The new architecture draws a crisp boundary, which enables deeper, more accurate attribution.</li><li><strong>Build time.</strong> We’re schema‑first: Developers write schema as source, and Viaduct generates code to provide a strongly typed, ergonomic surface. At our scale, build time is a constant battle. Over the years we’ve made numerous investments to improve build time, including direct-to-bytecode code generation that bypasses lengthy compilation of generated code. We anticipate that the improved modularity in Viaduct Modern will keep build times in check as the codebase grows.</li><li><strong>Dispatcher.</strong> We run Viaduct as a horizontally-scaled Kubernetes app. To mitigate blast radius, we use a dispatcher that routes operations to deployment shards, applying <a href="https://aws.amazon.com/blogs/architecture/shuffle-sharding-massive-and-magical-fault-isolation/">shuffle sharding</a>. It also simplifies isolating offline vs. online traffic and hosting experimental framework builds. (We don’t currently plan to open‑source the dispatcher as it’s tightly tied to Airbnb’s serving framework, but we may talk more about our strategies here in the future!)</li></ul><h3>Open-sourcing Viaduct</h3><p>Our intent from the start of Viaduct Modern was to open‑source it. We believe that setting out to build software for the world will result in higher quality software for ourselves. Also, as a significant consumer of open-source software, we feel an obligation to give back. Last but not least, while we’ve learned a lot by working with Airbnb developers, we think Viaduct can be massively improved by incorporating ideas and contributions from the wider community.</p><p>We’re open‑sourcing at an interesting moment. Viaduct is <strong>mature and battle‑tested</strong> <em>and also</em> <strong>new and evolving</strong>. The new engine is now in full production, while the new tenant API is still in alpha. We’ve implemented a small but robust kernel of the new API and are using it in a few (demanding) use cases. We’re investing heavily in the Modern API and migrating our own workloads to it, but it’s early days. By open‑sourcing early, we hope to grow a community who will shape the API with us together.</p><h3>Is Viaduct for you?</h3><p>Although our use case proves Viaduct can scale to massive graphs, we think it’s also a great GraphQL server when you’re just starting. We’ve emphasized developer ergonomics from day one, and we believe Viaduct provides one of the best environments for building GraphQL solutions. Whether you’re operating a super-graph today or just kicking the tires, we’d love for you to try Viaduct Modern and tell us what works — and what doesn’t.</p><p>Thanks to the entire Viaduct team, and especially Aileen Chen and Raymie Stata, for the tireless work on Viaduct Modern.</p><p><em>All product names, logos, and brands are property of their respective owners. All company, product and service names used in this website are for identification purposes only. Use of these names, logos, and brands does not imply endorsement.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e66397c9e9a9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/airbnb-engineering/viaduct-five-years-on-modernizing-the-data-oriented-service-mesh-e66397c9e9a9">Viaduct, five years on: Modernizing the data-oriented service mesh</a> was originally published in <a href="https://medium.com/airbnb-engineering">The Airbnb Tech Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Taming Service-Oriented Architecture Using A Data-Oriented Service Mesh]]></title>
            <link>https://medium.com/airbnb-engineering/taming-service-oriented-architecture-using-a-data-oriented-service-mesh-da771a841344?source=rss-9a357753a36f------2</link>
            <guid isPermaLink="false">https://medium.com/p/da771a841344</guid>
            <category><![CDATA[java]]></category>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[infrastructure]]></category>
            <category><![CDATA[graphql]]></category>
            <dc:creator><![CDATA[Adam Miskiewicz]]></dc:creator>
            <pubDate>Tue, 10 Nov 2020 22:02:50 GMT</pubDate>
            <atom:updated>2025-09-16T18:37:35.139Z</atom:updated>
            <content:encoded><![CDATA[<p>Introducing Viaduct, Airbnb’s data-oriented service mesh</p><p>By: Raymie Stata, Arun Vijayvergiya, Adam Miskiewicz</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KUevf-1aGcLQyit3wzmh8w.jpeg" /></figure><p>At Hasura’s <a href="https://hasura.io/enterprisegraphql/">Enterprise GraphQL Conf</a> on October 22, we presented Viaduct, what we’re calling a <em>data-oriented service mesh </em>that we believe will bring a step function improvement in the modularity of our microservices-based Service-Oriented Architecture (SOA). In this blog post, we describe the philosophy behind Viaduct and provide a rough sketch of how it works. Please <a href="https://www.youtube.com/watch?v=xxk9MWCk7cM">watch the presentation</a> for a more detailed look.</p><h3>Massive SOA Dependency Graphs</h3><p>For a while, <strong>S</strong>ervice-<strong>O</strong>riented <strong>A</strong>rchitectures have been moving towards ever larger numbers of small microservices. Modern applications can consist of thousands to tens of thousands of microservices connected in unconstrained ways. As a result, it’s not uncommon to see dependency graphs like the following:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/896/1*y2E1EmsS3paNhoChYGnJIw.png" /></figure><p>This particular dependency graph happens to be from Airbnb, but it’s not uncommon. <a href="https://twitter.com/werner/status/741673514567143424">Amazon</a>, <a href="https://medium.com/refraction-tech-everything/how-netflix-works-the-hugely-simplified-complex-stuff-that-happens-every-time-you-hit-play-3a40c9be254b">Netflix</a>, and <a href="https://eng.uber.com/microservice-architecture/">Uber</a> are examples of those that shared similarly tangled dependency graphs.</p><p>These dependency graphs are reminiscent of <a href="https://en.wikipedia.org/wiki/Spaghetti_code">spaghetti code</a>, just at the microservices level. Similar to how spaghetti code becomes harder and harder to modify over time, so does spaghetti SOA. To help manage the larger number of services inherent in a microservices-based architecture, we need organizing principles as well as technical measures to implement those principles. At Airbnb, we undertook an effort to find such principles and measures. Our investigations led us to the concept of a <em>data-oriented service mesh</em>,<em> </em>which we believe brings a new level of modularity to SOA.</p><h3>Procedure- vs Data-Oriented Design</h3><p>Organizing large programs into modular units is not a new problem in software engineering. Up until the 1970s, the main paradigm of software organization focused on grouping code into procedures and procedures into modules. In this approach, modules publish a public API to be used by code outside of the module; behind this public API, modules hide their internal, helper procedures and other implementation details. Languages such as Pascal and C are based on this paradigm.</p><p>Starting in the ’80s, the paradigm shifted to organizing software primarily around data, not procedures. In this approach, modules define classes of objects that encapsulate an internal representation of an object accessed via a public API of methods<em> </em>on the object. Languages such as Simula and Clu pioneered this form of organization.</p><p>SOA is a step back to more procedure-oriented designs. Today’s microservice is a collection of procedural endpoints — a classic, 1970s-style module. We believe that SOA needs to evolve to support data-oriented design, and that this evolution can be enabled by transitioning our service mesh from a procedural orientation to a data orientation.</p><h3>Viaduct: A Data-Oriented Service Mesh</h3><p>Central to modern, scalable SOA applications is a <em>service mesh</em> (e.g., <a href="https://istio.io/">Istio</a>, <a href="https://linkerd.io/">Linkerd</a>), which routes service invocations to instances of microservices that can handle them. The current industry standard for service meshes is to organize exclusively around remote procedure invocations without knowing anything about the data that makes up the application architecture. Our vision is to replace these procedure-oriented service meshes with service meshes organized around <em>data.</em></p><p>At Airbnb, we are using <a href="https://graphql.org/">GraphQL</a>™️ to build a data-oriented service mesh called <em>Viaduct.</em> A Viaduct service mesh is defined in terms of a GraphQL schema consisting of:</p><ul><li><em>Types</em> (and <em>interfaces</em>) describing data managed within your service mesh</li><li><em>Queries</em> (and <em>subscriptions</em>) providing means to access that data, which is abstracted from the service entry points that provide the data</li><li><em>Mutations </em>providing ways to update data, again abstracted from service entry points</li></ul><p>The types (and interfaces) in the schema define a single graph across all of the data managed within the service mesh. For example, at an eCommerce company, a service mesh’s schema may define a field productById(id: ID) that returns results of type Product. From this starting point, a single query allows a data consumer to navigate to information about the product’s manufacturer, e.g., productById { manufacturer }; reviews of the product, e.g. productById { reviews }; and even the authors of those reviews, e.g., productById { reviews { author } }.</p><p>The data elements requested by such a query may come from many different microservices. In a procedure-oriented service mesh, the data consumer would need to take these services as explicit dependencies. In our data-oriented service mesh, it is the service mesh, i.e., Viaduct, not the data consumer, that knows which services provide which data element. Viaduct abstracts away the service dependencies from any single consumer.</p><h3>Putting Schema at the Center</h3><p>In our talk we discuss how, unlike other distributed GraphQL systems like <a href="https://graphql-modules.com/">GraphQL Modules</a> or <a href="https://www.apollographql.com/docs/federation/">Apollo Federation</a>, Viaduct deals with the schema as a single artifact and has implemented several primitives that allow us to keep a unified schema while still allowing for many teams to collaborate on that schema productively. As Viaduct replaces more and more of our underlying procedure-oriented service mesh, its schema captures the data managed by our application more and more completely. We have taken advantage of this “central schema,” as we call it, as a place to define the APIs of some of our microservices. In particular, we have started using GraphQL for the API of some microservices. For these microservices, their GraphQL schemas are defined as a subset of the central schema. In the future, we want to take this idea further, using the central schema to define the schema of data stored in our database.</p><p>Among other things, using the central schema to define our APIs and database schemas will solve one of the bigger challenges of large-scale SOA applications: data agility. In today’s SOA applications, a change to a database schema often needs to be manually reflected in the APIs of two, three, and sometimes even more layers of microservices before it can be exposed to client code. Such changes can require weeks of coordinating among multiple teams. By deriving service APIs and database schemas from a single, central schema, a database schema change like this can be propagated to client code with a single update.</p><h3>Going Serverless</h3><p>Often in large SOA applications, there are many stateless “derived-data” services and “backend-for-frontend” services that take raw data from lower-level services and transform it into data that’s more appropriate for presentation in clients. Stateless logic like this is a good fit for the serverless computing model, which eliminates the operational overhead of microservices altogether and instead hosts logic in a “cloud functions” fabric.</p><p>Viaduct has a mechanism for computing what we call “derived fields” using serverless cloud functions that operate on top of<em> </em>the graph without knowledge of the underlying services. These functions allow us to move transformational logic out of the service mesh and into stateless containers, keeping our graph clean and reducing the number and complexity of services we need.</p><h3>Conclusion</h3><p>Viaduct is built on <a href="https://www.graphql-java.com/">graphql-java</a> and supports fine-grained field selection via GraphQL selection sets. It uses modern data-loading techniques, employs reliability techniques such as short-circuiting and soft dependencies, and implements an intra-request cache. Viaduct provides <em>data observability, </em>allowing us to understand, down to the field level, what services consume what data. As a GraphQL interface, Viaduct allows us to take advantage of a large ecosystem of open source tooling, including live IDEs, mock servers, and schema visualizers.</p><p>Viaduct started powering production workflows at Airbnb over a year ago. We started from scratch with a clean schema consisting of a handful of entities and have grown it to include 80 core entities that are able to power 75% of our modern API traffic.</p><p>As mentioned in the introduction, more details on the motivation and technology behind Viaduct can be found in our <a href="https://www.youtube.com/watch?v=xxk9MWCk7cM">presentation</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=da771a841344" width="1" height="1" alt=""><hr><p><a href="https://medium.com/airbnb-engineering/taming-service-oriented-architecture-using-a-data-oriented-service-mesh-da771a841344">Taming Service-Oriented Architecture Using A Data-Oriented Service Mesh</a> was originally published in <a href="https://medium.com/airbnb-engineering">The Airbnb Tech Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using GraphQL + Apollo at Expo]]></title>
            <link>https://medium.com/apollo-stack/using-graphql-apollo-at-expo-4c1f21f0f115?source=rss-9a357753a36f------2</link>
            <guid isPermaLink="false">https://medium.com/p/4c1f21f0f115</guid>
            <category><![CDATA[react-native]]></category>
            <category><![CDATA[expo]]></category>
            <category><![CDATA[api]]></category>
            <category><![CDATA[graphql]]></category>
            <category><![CDATA[javascript]]></category>
            <dc:creator><![CDATA[Adam Miskiewicz]]></dc:creator>
            <pubDate>Thu, 11 May 2017 17:36:06 GMT</pubDate>
            <atom:updated>2017-05-12T15:24:03.971Z</atom:updated>
            <content:encoded><![CDATA[<h4>Taking Expo Home from prototype to production in weeks</h4><p>It’s not often the best engineering decision to use the “coolest new thing”. It’s almost universally agreed upon that the smart thing to do is to rely on time-tested and stable technologies. With regards to a web or mobile backend, the sage advice is to use a relational database, build RESTful APIs, and in general just use things that everybody knows and understands; “Keep it simple” is the agreed upon mantra.</p><p>At <a href="https://expo.io/">Expo</a>, we diligently follow the “keep it simple” rule. Being a small startup, we have a lot of projects and not many people, and it’s important that we focus on shipping features. That being said, we strongly believe in evaluating technologies, both new and old, by their merits and flaws, over adhering a particular dogma.</p><p><a href="http://graphql.org/">GraphQL</a> and the <a href="http://dev.apollodata.com/">Apollo</a> toolchain were choices that we evaluated not by their GitHub star count or release dates, but by how effective we felt they would be in helping our small team build and iterate on our products: They allow for quick, reliable iteration, and an intuitive way to bridge the gap between back-end and front-end development. There’s a multitude of examples of large companies <a href="http://graphql.org/users/">using GraphQL</a> and Apollo in production, but this stack is just as powerful for a company the size of Expo. The tools continue to prove their worth, and are definitely examples of a time when choosing the cool new thing paid off.</p><h3>B.A.: Before Apollo</h3><p>At the end of 2016, one of Expo’s main internal goals was revamping our <a href="https://expo.io/tools#mobile">client</a> app, a project we affectionately call “Expo Home”.</p><p>The previous version of the app was basically <em>just</em> an Expo project viewer. You could view your recent projects but couldn’t view someone’s profile with the projects they’d worked on.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rxd-2VYAixk8SeylKCJOLg.png" /><figcaption>Expo (Exponent) Client — EOY 2016</figcaption></figure><h4>The new app</h4><p>Besides a nice visual facelift (pictured below), our goal with the new version of the client was to make it much more community-oriented. We have sign in/sign up, user profiles, a way to explore “featured”, “new”, and “top” projects, and project/user search. We have much more planned for Expo Home in the future, but these features are what we agreed would be our MVP, for launch in February 2017.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rzRJzfxJp3k5XX7XhalBjA.png" /><figcaption>Expo Client running “Expo Home” — Present Day</figcaption></figure><p>Prior to starting the project, we had some, but not all, of the necessary data available through a myriad of RESTful-ish endpoints exposed by our backend, which we had used in other ancillary parts of our products. In addition to not exposing the data we needed and not being well documented, this part of our codebase also wasn’t very well tested or maintained. We didn’t want to build the future of the Expo client on these shaky foundations, so we had two options —<strong> rearchitect and build out a more maintainable RESTful API, or take a different approach altogether.</strong></p><p>Prior to working at Expo, I had the opportunity to work on a few apps that used GraphQL and Relay, and was very impressed with the outcome, and was particularly enthusiastic about GraphQL. As the backend engineer on this project, I thought it would be a great idea to try GraphQL for Expo Home, and use Apollo on the frontend.</p><p>We had a ton of new features to build, and I had wanted to introduce GraphQL into the Expo tech stack for a while. This project seemed like the perfect fit, but I needed to sell it to my team.</p><h3>Why GraphQL</h3><p><a href="https://medium.freecodecamp.com/so-whats-this-graphql-thing-i-keep-hearing-about-baf4d36c20cf">Lots</a> <a href="https://medium.freecodecamp.com/give-it-a-rest-use-graphql-for-your-apis-40a2761e6336">has</a> <a href="https://0x2a.sh/from-rest-to-graphql-b4e95e94c26b">been</a> <a href="https://dev-blog.apollodata.com/the-business-case-for-graphql-cc7a2b93148d">written</a> about GraphQL vs REST, as well as about the problems REST APIs often face. I presented some of these arguments to the team while pitching them on using GraphQL to power Expo Home.</p><p>Building a performant, maintainable, RESTful application is absolutely possible. Doing it quickly, on the other hand, is another story. We had less than two months to ship the first version of the new Expo mobile app, and thus working quickly was important.</p><p>Engineering, as with most other things in life, is about tradeoffs. Some considerations I posed to the team:</p><ul><li>User experience — If we spend a ton of time making our data fetching logic in the app really robust, are we sacrificing the user experience of our app and polish around UI?</li><li>Performance and stability — If we spend a ton of time on UI polish, are we making a slow app, or one that doesn’t gracefully handle failure?</li><li>Over-fetching — On the backend, we could spend our time focusing on API coverage, and making sure that every last thing that a front-end engineer could ever think of doing is covered by the APIs. If we did that, however, are we building slow endpoints (by providing too much data) or are we causing a request waterfall from our clients because they have to request many API resources to build the UI for a single screen?</li><li>Tech debt— What if we spend time building individual, bespoke APIs that provide perfect coverage for a given screen with no over or under-fetching? Are we backing ourselves into an implementation corner, making everything difficult to manage when we want to add a feature?</li></ul><p>With GraphQL and Apollo, we didn’t have to make these same tradeoffs and could build a fast and easy-to-maintain front-end and back-end from the outset of the project.</p><h3>Developer Workflow &amp; Production-Ready Results</h3><p>In web and mobile development, it’s common practice to build some UI with mock data before integrating with any APIs/services. This workflow allows both back-end and front-end developers to work mostly in parallel.</p><p>But eventually it’s time to bring in real data. This process will typically be a carefully coordinated dance between front-end and back-end engineers, as each person (or team) finds places where they misjudged the other. Even if a careful spec is put in place beforehand, there will almost certainly be constant back-and-forth.</p><p>In our case, it was imperative that with the short timeline <a href="https://medium.com/u/d5a707474bd2">Brent Vatne</a> and I could work in parallel as much as possible. As I look back on building the new version of the Expo app, the part that sticks out to me the most is how smooth the workflow was as <a href="https://medium.com/u/d5a707474bd2">Brent Vatne</a> and I worked on the app.</p><p>Brent and I had almost none of this forced collaboration. This was due to two things:</p><ol><li>Apollo and its surrounding tooling makes it trivial to use mock data/a mock server, which we leveraged from the beginning of the project.</li><li><a href="https://github.com/graphql/graphiql">Graph<em>i</em>QL</a> the in-browser IDE for exploring GraphQL servers (pictured below) is simply an amazing tool for browsing an API and discovering its capabilities.</li></ol><p>Instead of spending time discussing API design or what data should be in which endpoint, Brent and I were able to talk about how to make the product better and provide a great user experience.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JCW8YMecimtxhvP__s3dFQ.png" /><figcaption>GraphiQL — the best part of GraphQL as a developer</figcaption></figure><p>Another thing that sticks out to me about the GraphQL/Apollo development experience is that at the end of the day, you’re not left with something that was easy to get started with but doesn’t fulfill production needs. It’s quite the opposite — everything we built, because we leveraged the Apollo toolchain and followed GraphQL best-practices, was production-ready from the get-go.</p><h3>The Future</h3><p>Expo Home is far from complete — we have tons of planned improvements for 2017.</p><p>An example of one such improvement is a feature that shows the current project you have open in <a href="https://github.com/expo/xde">XDE</a>, our desktop developer environment, on the main screen of Expo Home. There are a few moving pieces to this feature, but one of the parts I’m most excited about is leveraging GraphQL subscriptions to update the app front-end.</p><p>Apollo’s graphql-tools <a href="https://github.com/apollographql/graphql-tools">package</a> makes this pretty simple to do, and we can leverage <a href="https://www.pubnub.com/">PubNub</a> as the backend, alleviating us from implementing our own WebSocket-based solution.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qnhgSmEcAq9X-nSip0nMNA.png" /></figure><p>At Expo, we’re also working on bringing pieces of the Expo Home experience to web. All of Expo Home’s backend work, in addition to a reasonable amount of front-end work, can be reused in this effort. And as Expo grows, and we need to move pieces of our infrastructure out of our monolith and into some individual services, the web and mobile clients can stay untouched, allowing us to move quickly in all parts of our infrastructure without blocking any particular team or project.</p><p>It’s rare to find a technology that is valuable to both the enterprise and the ten person company. GraphQL can help the 5000 person organization wrangle their 1000’s of REST APIs and services into a coherent, discoverable schema and interface, but it can also help the small startup foster good communication patterns between their back-end and front-end developers, enable better parallel work, and provide a solid basis for future API and feature development.</p><p>GraphQL and the Apollo toolchain have already been instrumental in our product roadmap in 2017, and we’ve only scratched the surface of their overall potential.</p><p>If you want to know even more about how we use GraphQL and Apollo at Expo, watch my talk from Apollo Day 2017:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fvideoseries%3Flist%3DPLpi1lPB6opQzWZwqnG0YIrWcSS3kH5SR8&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DE398q4HGRBA&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FE398q4HGRBA%2Fhqdefault.jpg&amp;key=d04bfffea46d4aeda930ec88cc64b87c&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/5db2e44bbfe3e37b28736285a046e968/href">https://medium.com/media/5db2e44bbfe3e37b28736285a046e968/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4c1f21f0f115" width="1" height="1" alt=""><hr><p><a href="https://medium.com/apollo-stack/using-graphql-apollo-at-expo-4c1f21f0f115">Using GraphQL + Apollo at Expo</a> was originally published in <a href="https://medium.com/apollo-stack">Apollo GraphQL</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>