By Jason Vantomme and Peter Hogan, Workday AI Platform

If you build agents for HR and finance, you’ve probably seen this one: A manager asks whether they can approve an exception to a spending limit. The agent returns a confident answer, a clean explanation, and a citation to the right policy. Everything looks correct. But look closer, and the policy never settles the question. A person was supposed to decide, and the agent filled the gap with something that sounded authoritative.

The usual fix is intuitive: add a second model to review the first model’s work. We tested that fix in the research behind our upcoming paper. It doesn’t work the way you’d hope.

TL;DR:

• Plausible but unsupported inference is a failure where an agent has the right source and still produces an answer the source does not support.
• In our tests, models recognized genuine uncertainty in only 25% of the cases where it was present.
• Whether a second model helped depended on which one did the checking: one judge improved accuracy by about 4 points, one lowered it by about 4, and one made no detectable difference.
• Judge models made their worst mistakes on exactly the questions that need a second opinion most.
• Two models agreeing tells you an answer is plausible. It does not tell you the answer follows from the source.

The Failure That Looks Like Success

Most conversations about AI errors center on models inventing facts. Retrieval helps there, because it puts the right documents in front of the model.

In this failure, the agent already has the correct source and may read every clause correctly. What breaks is the step from source to answer. The model treats a policy as if it resolves a question when it does not, or it skips a provision that would change the result. Having the right evidence in front of the model does not guarantee the answer is supported by it, and a citation can be accurate while the claim attached to it is still wrong [1].

We call this plausible but unsupported inference. It matters in enterprise settings because it looks exactly like a correct answer. There is no typo to catch and no obvious fabrication.

What We Tested

For this research, we built a synthetic policy corpus around a year-end sales incentive true-up, covering five representative populations, and wrote 118 questions against it. Some had clear answers. Others were built so the source contradicts itself or deliberately leaves the decision to case-by-case human judgment, which means the correct response is to escalate to a person instead of answering.

We compared two setups:

1. Baseline: a foundation model answers directly from the source documents with no verification step.
2. LLM-as-judge: a second model, the judge, reviews the first model’s answer against the same evidence. The judge always came from a different, independently developed model family than the model it was checking, so no model ever checked its own work.

We ran six models across three independent families. Each model’s answers were reviewed by the judge models from the other two families.

Finding 1: Confidence Wins Over Recognition

On questions with a clear answer, the models did well, with accuracy of 88% to 90%.

On questions where the right move was to recognize genuine uncertainty, they struggled. Pooled across all six models, they recognized the uncertainty in only 25% of those cases. The rest of the time, they answered confidently anyway. Other researchers have found that, on general-purpose benchmarks, models struggle to admit when a question can’t be answered [2]. We saw the same pattern in an enterprise policy setting, even though the models had the complete policy in front of them.

In one case, a model’s own written explanation said the steps of a required process were “explicitly not defined” in the source. It then returned a confident, multi-step procedure anyway. Its awareness of the gap did not change its answer. In another, two models went further and built a coherent justification for a calculation rule the source never states. That is harder to catch than silence, because the response reads like careful reasoning.

The confusion also ran the other way: when models did escalate, they were wrong more than half the time. They could not reliably tell when to be confident and when to hand off.

Finding 2: A Second Model Helps Unevenly

Adding a judge model did not move accuracy in a single direction. The effect depended on which model did the checking. One judge improved accuracy by about 4 percentage points, one lowered it by about 4, and one made no detectable difference.

Judge models also caused damage, and it landed on the questions that matter most. When a baseline model correctly escalated a question to a person, the judge overturned that correct answer at approximately 3.9 times the rate it did on any other kind of question. And when a baseline model wrongly answered a question that should have been escalated, judge models endorsed the wrong answer 73% of the time, compared with 41% on other kinds of questions. These comparisons rest on a small number of questions, so treat them as a pattern, not a precise rate.

Finding 3: A Judge Model Can Contradict Itself

In one case, a judge model examined a correct answer and wrote that it was “fully justified by the source materials” and “appropriate in substance.” Then it marked the answer incorrect anyway, citing only a labeling technicality.

In another case, the policy documents said the system must not give a binary answer to a certain type of question. One model opened with “Yes,” added that the matter “requires a case-by-case determination,” yet still delivered the whole thing as a direct answer. The judge model called that answer “correct and precisely grounded.” Both models treated it as a question with a clear answer, which is exactly what the policy said it wasn’t.

Part of the problem traced to a single weak judge model, which overturned correct answers far more often than the other two. Removing it would eliminate most of that effect. But the self-contradiction in the first example came from a judge model that was otherwise reliable, which points to a limit of the approach itself and not just one bad judge [3].

Why Agreement Isn’t Proof

Imagine you ask two colleagues whether the travel policy covers a particular expense. Both say yes, and both sound sure. That doesn’t settle it. Two colleagues agreeing tells you what they both believe. It doesn’t tell you what the policy says. Agreement tells you about the people in the room. Only the policy tells you about the policy.

A second model reviewing the first works the same way. Our judge models were never from the same family as the model they were checking, and the failure still recurred [4]. That suggests the problem is not simply a model missing its own blind spots. Agreement is a signal that an answer is plausible. It is not a check that the answer is supported.

Other published research on formalizing natural-language policy into logic and checking it with a solver points in the same direction: reliable checking needs independent, redundant steps and not a single generative pass [5]. What is missing is a mechanism that can establish, consistently and regardless of which models are involved, that an answer is supported by its source. A second model can serve as a useful extra signal. It cannot be that mechanism.

What This Means for Builders

1. Test for the gap, not just the answer. Build evaluation sets where the correct response is “the source does not resolve this.” Accuracy on answerable questions tells you little about this behavior, and the two can diverge sharply.
2. Do not treat a judge model’s approval as verification. A second model catches some errors and introduces others. Measure it on your own data, by question type and by which model does the judging, before it becomes a control you rely on.
3. Check the answer against the source, not just the source’s availability. Retrieval and citations get evidence in front of the model. Plan separately for how you will confirm the answer is actually supported by that evidence.
4. Treat escalation as a first-class outcome. In HR and finance workflows, a well-routed handoff to a person is a correct answer, but only if the system can tell when a handoff is needed. Systems should be scored, and designed, accordingly.

Where This Leaves Us

Agents are taking on work where a confident wrong answer has real consequences for employees, customers, and the companies responsible for them. In that setting, “sounds right and cites a source” is not a sufficient bar, and neither is “a second model agreed.” The standard worth building toward is an answer that can be shown to follow from the governing source, with a clear handoff to a person when it cannot.

Our tests used a single-domain synthetic corpus, a small set of genuine-uncertainty questions, and single-turn queries only, so we read these results as a strong signal and not a final measurement. That leaves open questions worth exploring. One is whether the confidence we observed reflects a failure to see the gap or a reluctance to act on it. Another is whether letting an agent ask a clarifying question changes the outcome. Larger and more varied policy corpora would also show how far the pattern extends, and a same-family review comparison would show how much of the problem is a shared blind spot. What we can say today is that it held across every independently developed model family we tried, which tells us this isn’t one provider’s problem. Our upcoming paper details the full method, results, and case analyses.

References

1. RAGTruth: A Hallucination Corpus for Retrieval-Augmented Models
2. AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions
3. The Stability Trap: Evaluating the Reliability of LLM-Based Instruction Adherence Auditing
4. Correlated Errors in Large Language Models
5. A Neurosymbolic Approach to Natural Language Formalization and Verification

Two AIs Agreeing Doesn’t Make Them Right was originally published in Workday Technology Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

By Samuel Weick, Principal Product Manager at Workday

If you build agents for systems of record, you’ve felt this tension: models move fast, UX patterns are evolving under your feet, and every new “AI canvas” demo ships with an implicit question no one answers, what is the actual contract between the agent and the UI? In most stacks, that contract is still an afterthought. The agent spits out text or JSON, the client hacks together some cards or widgets, and you hope nothing mission critical depends on it. That won’t fly when the agent sits on top of HR, Finance, and IT systems where the margin for error is effectively zero.

Generative UI is the bridge. It’s how we let agents compose interfaces dynamically, grounded in trusted components, governed data, and existing controls, rather than hard coding every screen in advance. And today, two emerging open specifications are starting to define how that bridge actually works: Anthropic’s MCP Apps and Google’s A2UI.

They represent two very different answers to a core question: who controls the palette, and who holds the brush? This piece is about those tradeoffs, from the perspective of teams building durable agentic systems in enterprise environments like Workday.

TL;DR

Generative UI is moving from “cool demo widgets” to the surface where agents do work:

• MCP Apps lets servers ship full embedded web apps into agent hosts: maximum expressiveness, heavier trust and governance surface.
• A2UI has agents emit declarative JSON that the client renders with its own components: maximum safety and visual consistency, constrained expressiveness.
• In enterprise settings like Workday, the right architecture depends on trust: who owns the UI, what systems of record are in play, and which governance rails you’re required to stay on.
• Most serious stacks will land on a hybrid: a curated component palette for 90% of workflows, and deeply vetted embedded apps where full fidelity partner experiences matter.
• The “agent canvas” is the pattern where all of this comes together, a persistent, governed workspace where agents and humans co‑create, not just chat.

If you’re a staff engineer thinking about where to invest, this is the shape of the platform you’re going to be building against.

From Static Screens to a Generative Palette

When you hand a painter a canvas, they don’t manufacture pigments mid stroke. They start from a palette: a curated, deliberate set of colors and tools. The painting is generative; the palette is not. Enterprise software already works this way in one important sense. Strong product teams build design systems: reusable components, design tokens, accessibility patterns, interaction rules, and data contracts. Engineers don’t need to reinvent a button, table, or approval control for every screen. What has traditionally been fixed though, is the composition. Designers decide in advance which components belong on a given page. Engineers encode those layouts and workflows. PMs define the states and acceptance criteria. QA verifies that the resulting experience behaves predictably. The system is deliberately deterministic: for a known task, a known screen appears in a known way. That model produces reliable software. But it also means the interface is optimized for the workflows we anticipated at the design time. It can struggle when a user’s need cuts across those predefined screens, data models, and process boundaries.

When a controller asks, “Where are my renewal risks this quarter?” they don’t need a paragraph of text or a journey through five pre-built reports. They need an interactive workspace composed from trusted elements: a renewal risk chart, a contracts table, policy context, and actions appropriate to their permissions. When an HRBP asks, “Help me prepare this reorg,” they don’t want a blob of natural language; they need an organization’s view, headcount and compensation summaries, scenario comparisons, and the right review or approval action. Generative UI does not replace the design system. It makes the design system available to an agent at runtime

The palette remains curated: the components, governed data, interaction patterns, security boundaries, and deterministic workflows. What becomes dynamic is the agent’s ability to compose those trusted pieces into an interface that fits the user, task, and moment. You could pre-build every possible screen for every combination of intent. But you’ll drown on day two.

If you work on something like Workday Agent Factory, this isn’t an abstract idea. This is what we’re building against it. The open question is: what’s the right architecture to make that contract explicit and durable? Two serious answers have emerged to standardize how agents summon UI at runtime, and they represent different philosophies.

MCP Apps: Letting Partners Bring Their Whole UI

https://github.com/MCP-UI-Org/mcp-ui

MCP Apps, formalized as SEP-1865 and co-authored by maintainers at Anthropic and OpenAI, extends the Model Context Protocol with a standardized pattern for serving interactive user interfaces from MCP servers.

At a mechanical level, it’s straightforward:

• Servers declare UI resources using a ui:// URI scheme.
• Those resources are linked to tools through metadata.
• When invoked, the server returns real HTML, CSS, and JavaScript.
• The host (Claude, ChatGPT, VS Code, Goose, etc.) renders that payload in a sandboxed iframe and brokers communication via JSON‑RPC over postMessage.

Conceptually, this is a “mini web app” model. The server shows up with its own paints and brushes. The host provides wall space, security boundaries, and a message bus. The upside is obvious if you’ve ever tried to port a complex app into a new component model:

• There’s effectively no ceiling on expressiveness. If Canva wants to bring a full design editor, or a data platform wants to expose a live notebook, MCP Apps is a natural fit.
• Third‑party vendors don’t have to rebuild frontends for each agent host; they can reuse their existing investments and ship a consistent experience across surfaces.

For Workday adjacent scenarios, this is compelling when you want rich, recognizable experiences from deeply trusted partners living alongside Workday data and workflows. But that expressiveness comes with a cost.

The Trust and Governance Surface

MCP Apps deliberately runs code you didn’t write inside your host. Even with iframe sandboxing, templates, and auditable JSON‑RPC messages, you’ve materially expanded your trust surface compared to a purely declarative UI. In consumer tooling, you might treat that as a straightforward security problem. Where we work, in enterprise systems of record, payroll, financial close, compliance, that’s not enough.

You need to answer:

• How does this embedded app participate in existing approval and audit flows?
• How do you ensure permission checks and data access match the semantics you enforce for humans and services?
• What happens when the embedded UI wants to trigger side‑effects in systems where “almost right” is wrong?

The MCP spec acknowledges this with layered defenses and explicit consent semantics. In a Workday style stack, you’d want to go further: plug these experiences into the same governance frameworks you use for core workflows, not run them in parallel.

Interoperability and Consistency

On paper, interoperability is good. MCP Apps are already supported by multiple hosts and are being adopted across a wide set of vendors.

The tradeoff is visual and behavioral consistency:

• An HTML widget tuned for one host’s theme may look off, or outright wrong in another.
• The host can’t natively enforce its own design system or accessibility semantics inside the iframe.
• Bridging semantics like keyboard shortcuts, screen readers, or responsive layout becomes a per integration concern.

In many ways, MCP is the right model when:

• You’re integrating with sophisticated partners that already have a strong UI.
• Users expect to “be in” that application’s interface, just in a new context.
• You’re willing to accept a larger governance and UX integration boundary in exchange for richness.

In Workday Agent Factory terms: this is closer to the “bring your own studio” end of the spectrum.

A2UI: Agents Describe Intent, Clients Enforce the Rules

Google’s A2UI specification takes the opposite stance. Instead of shipping executable UI, agents emit declarative JSON describing the interface they want: cards, inputs, tables, charts, buttons, date pickers, and so on. The client decides how to render each component using its own trusted library. The payload is essentially a flat list of component descriptions:

• Each component has a type, data bindings, and interaction hooks.
• Components refer to each other via IDs in an adjacency list style graph.
• The host maintains a catalog of allowed component types and maps each type to a concrete widget implementation in React, SwiftUI, Angular, Flutter, whatever your stack uses.

The agent never touches the DOM. It never ships code. It only expresses intent over a finite palette. This maps almost perfectly to how you’d want durable agentic systems to behave in an enterprise:

• The host owns the palette: what can be rendered, how it looks, how it behaves, which accessibility and security policies apply.
• The agent owns composition: which components to place, how to connect them, what data to bind, when to show or hide.

Designed for Cross‑Boundary Agents

A2UI’s design explicitly targets multi‑agent, cross‑org scenarios: an orchestrator inside your environment calls into remote sub agents owned by other teams or vendors. In those situations, allowing remote agents to inject arbitrary UI code into your application is a non‑starter. With A2UI:

• Remote agents describe what they want rendered.
• Your client decides how it shows up, using components that already meet your compliance, design, and accessibility bars.
• The entire “attack surface” reduces to the set of component types you explicitly trust.

For a platform like Workday, this lines up with the posture that AI should operate inside the same rails as HR and Finance systems, not alongside them. The platform remains the system of record and system of action; agents are guests who get to compose with the furniture you’ve already bolted down.

Streaming and Portability

A2UI’s JSON structure is intentionally friendly to LLM generation:

• Models can stream components incrementally, enabling progressive rendering. Users see real UI appear as the agent thinks.
• The same payload renders natively across platforms, as long as each client implements the agreed component catalog.

If you imagine Workday surfaces: web, mobile, embedded side panels, external tools connected via Sana, you can see why this matters. You want:

• One semantic description of the interface.
• Native, consistent experiences everywhere it appears.
• A single set of policies around what components exist and how they’re allowed to behave.

The Expressiveness Ceiling

Of course, there’s a bound:

• If your catalog supports Chart, DataTable, and ActionPanel, agents can build surprisingly rich workspaces.
• If you don’t define 3DModelViewer, no agent can render one without a host update.
• Extensibility exists, you can map new server side component types to host implementations, or even include iframes for legacy content, but that requires deliberate client work.

In other words: A2UI is only as powerful as your palette. For teams like Agent Factory, that’s not a bug; it’s the point. It forces you to invest in the component library, contracts, and governance layer instead of over‑indexing on prompt hacks and brittle UI generation.

Flexibility vs Safety: Choosing Who Holds the Brush

Taken together, MCP Apps and A2UI encode a core architectural tension:

• MCP Apps: the server holds the brush. It shows up with its own UI stack. The host’s job is to sandbox, broker messages, and draw a few hard lines around side effects.
• A2UI: the host holds the palette. The agent is allowed to arrange known pieces in new ways, but never to introduce new primitives without buy‑in.

If you strip away the spec names and acronyms, the real question is: For each UI surface your agent can touch, who do you trust to define the experience — and what does that imply for your governance model?

From a Workday perspective, where you operate at the heart of HR and Finance, this maps pretty cleanly:

• For first‑party and tightly governed experiences that operate directly on payroll, close, or compliance workflows, you want A2UI‑style declarative control. The system of record owns the palette; agents compose within it.
• For deep partner integrations where users expect a full fidelity partner UI, analytics notebooks, design tools, specialized vertical apps — MCP‑style embedded apps are a fit, as long as they plug into existing controls rather than bypass them.

Most serious platforms will need both. The mistake is to treat them as interchangeable.

The Agent Canvas: Where Generative UI Comes to Life

So far, you can imagine all of this living inline in chat. Ask a question, get back cards and widgets. For light use cases, that’s fine. But if you’ve ever watched a finance leader, HRBP, or operations manager actually work, you know chat is the wrong workspace:

• It’s linear and append‑only.
• Past outputs scroll away.
• You can’t compare two artifacts side by side.
• You can’t incrementally refine a view without spamming the thread.

Chat is a conversation medium, not a working medium. Enter the agent canvas.

Think of the canvas as a persistent, spatially organized work surface, often a side panel or full‑screen mode, where generative components live outside the chat timeline:

• A dashboard generated for Monday’s standup is still there on Tuesday.
• An analyst can keep three chart variants visible while exploring drivers.
• A recruiter can review candidate summaries, job details, and suggested actions in a single layout, all editable.

It has a few important properties:

• Persistence. Components don’t disappear into the scrollback. They’re stateful artifacts you can return to, share, and evolve.
• Composition. Multiple components from multiple tools or sub‑agents can coexist: tables, charts, forms, explanation panels, action launchers.
• Editability. Users can filter, rearrange, override, and augment what the agent produced, without dropping back to raw prompts.
• Context handoff. The canvas becomes a shared memory between human and agent; every interaction is new context for the next step.

From an implementation perspective:

• MCP Apps power canvases by letting the host arrange embedded iframes spatially and broker messages.
• A2UI powers canvases by letting the host own layout and state management while agents stream new components into the graph.

From an enterprise-AI perspective, the canvas is where the idea of “AI as a teammate” becomes tangible. It’s where agents can actually do work, assemble views, propose actions, and help users drive outcomes, while staying grounded in the same permissions, approvals, and audit trails that core workflows use.

Enterprise Constraints: Systems of Record and Rails

So far, nothing about MCP Apps or A2UI inherently cares whether you’re building for a side project or for a system that moves real money. Enterprise environments like Workday add constraints that change the calculus:

• Systems of record. Agents are not playing with copies of data; they’re sitting on HR and Finance primitives where “almost correct” isn’t acceptable.
• Deterministic foundations. Underlying business processes are deterministic by design. They start, end, and leave auditable traces. Agents have to fit into that world, not erode it.
• Existing governance frameworks. Approval chains, audit trails, policy engines, and compliance dashboards already exist. You can’t bolt on a separate governance universe just for agents.

These constraints push you toward a few design principles:

• Treat UI architecture as a governance decision, not just a UX one. Choosing MCP vs A2UI is deciding who’s responsible for correctness and who has to prove it.
• Plug agents into existing controls. Whether you embed partner UIs or render native components, permission checks, logging, and approvals should use the same rails you use for humans and services.
• Bias toward declarative contracts. The more semantics you can encode in component types, schemas, and interaction contracts, the easier it is to evaluate, test, and reason about agent behavior over time.

This is very close to how durable agent systems are already being built: thin agent layers on top of thick, well governed foundations.

What This Means for Builders

If you’re a staff or senior engineer in this space, here’s the frame to use when you sit down to make decisions about your own architecture.

1. Design the Palette Like You’d Design a Platform: Your generative UI is only as competent as the palette you give it. Invest in:

• A well‑factored component library that covers the real tasks in your domain: not “Button” and “Card,” but “HeadcountTrend,” “ApprovalQueue,” “VarianceTable,” “ContractSummary,” and so on.
• Clear contracts for each component: required/optional fields, data types, allowed actions, and failure semantics.
• Strong defaults for styling, accessibility, and behavior so you can render the same semantic component across devices and hosts without rewriting logic.

If you do that well, A2UI‑style declarative generation becomes far more powerful than it looks on paper.

2. Choose Who Holds the Brush Based on Trust: Don’t start with the spec; start with the trust relationship:

• If the UI source is your own team or a tightly governed partner, and users expect the full experience of that app inside your canvas, MCP Apps are a reasonable choice, provided you integrate them into existing governance, not bolt them on.
• If the UI source is a remote agent, vendor, or internal team you treat as untrusted, or if the surface touches critical systems of record, favor A2UI‑style declarative contracts where the host owns execution and enforcement.

This isn’t just about safety; it’s about being able to explain your system and stand behind it when something goes wrong.

3. Build for Chat and Canvas from Day One: It’s tempting to ship chat + inline cards first and treat the canvas as “phase two.” In practice:

• Your first serious users will immediately push past what chat can support.
• Without a canvas, every new workflow becomes another bespoke thread of prompts and replies, impossible to reason about or govern at scale.
• Architecting your UI contracts and orchestrations with a canvas in mind from the beginning makes it far easier to evolve from “assistant” to “teammate.”

You don’t have to launch the full canvas experience on day one. But you should make sure your generative UI contracts aren’t blocked on a linear chat thread forever.

4. Watch (and Shape) the Convergence: Both MCP Apps and A2UI are moving fast. Early frameworks are already experimenting with transports that support both. The likely end state looks less like “MCP vs A2UI” and more like:

• A core declarative layer that most agents use for 80–90% of interfaces.
• A narrow lane for fully embedded apps, used sparingly for partners who genuinely need full control over their UI and are willing to meet your governance bar.

If you’re building from within an enterprise, you have a chance to shape that convergence rather than consume it. The decisions you make about palettes, contracts, and rails will determine how far agents can go in your domain. We’re at the point where the UI can’t be treated as a static artifact any more than the model can. Interfaces are going to be composed fresh for every user, every task, every context, often by agents, not humans.

Our job, especially in environments where people and money actually move, is to make sure those interfaces are grounded in systems we can trust: governed components, deterministic workflows, and rails that let agents operate safely at the heart of the business. The models will keep getting better. The teams that win are the ones who design a palette worth painting with, and who are clear about who’s allowed to hold the brush.

Relevant Resources

• https://modelcontextprotocol.io/extensions/apps/overview
• https://a2ui.org/
• https://github.com/a2ui-project/a2ui
• https://github.com/modelcontextprotocol/ext-apps

MCP Apps and A2UI: The Two Architectures That Will Define Generative UI for Enterprise AI Agents was originally published in Workday Technology Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

By Zeke Miller, Director, Machine Learning Engineering at Workday

Two years ago, my team spent six months building a conversational analytics agent. We wired up custom tool-calling loops, built our own orchestration code, and spent weeks tuning our evaluation harness. It was a legitimate engineering effort, the kind of work you’re proud of, and the evals we built there are what made the comparison meaningful later. A year later, someone on our team wired the same underlying analytics system into an off-the-shelf integration framework using a single, well-designed connector. When we ran our evals against that setup, it outperformed our custom stack with essentially zero additional work. Same data, same evals, different agent layer. The evals were the constant. That moment changed how I think about durability in agentic systems. And it’s probably a moment you’re having right now too.

The Core Problem: Speed ≠ Durability

If you’re building agents in 2026, you feel this tension: models, tools, and patterns evolve faster than ever. Yet the domains we work in, HR, finance, compliance, are the ones with the least tolerance for breaking changes. The instinct is to build something robust that lasts. The reality is that robustness now means something very different than it did five years ago. I’ve spent the last several years at the intersection of coding tools, analytics, and enterprise platforms. The throughline isn’t about finding the perfect stack. It’s about designing systems that can evolve without being rebuilt.

Mental Model: Thin Agent Layer, Thick Foundation

Here’s what guides most everything I do: The agent layer should be thin. The foundation should be thick. Most engineers think intelligence lives in the agent: complicated tool graphs, custom planners, bespoke memory systems. In enterprise contexts, that’s backwards. The thick part is your platform: evals, data models, security, governance, auditability. That’s durable. That’s where the hard constraints and a lot of the business intelligence live.

On top of that you have two fast-moving pieces:

• The foundational model: Claude, Gemini, etc., and
• A thin agent layer: prompts, tool wiring, orchestration logic

The agent layer matters for usability and performance, but it should be designed assuming you’ll rewire it often as models and paradigms evolve, or even bypass it entirely when users plug tools like Claude Desktop or OpenClaw straight into your APIs. In our analytics story, the intelligence was split: the BI model (LookML) carried deep semantic knowledge of the data, and the frontier model was increasingly capable of generating SQL and Python on its own. Once the LookML and APIs were solid, orchestration became nearly trivial. When a better framework emerged, we swapped it out without touching the foundation, and the newer models were simply good enough to replace a lot of the bespoke agent logic we’d built. Being able to ride both curves, the model frontier and the agent‑design frontier, without rebuilding your platform is the real differentiator. If your agent is monolithic and tightly coupled to a specific model or framework, the cost of adapting becomes prohibitive.

Three Principles:

1. Invest heavily in the system of record and system of action: APIs, security boundaries, access controls, workflows, make them explicit and composable. These don’t change when models get better, which means you can reap the benefits of the model improvements on day one.
2. Make the agent layer declarative and interchangeable: Clear tool contracts, minimal bespoke state, well-defined orchestration interfaces. If someone can swap your framework and improve performance, that’s a good sign.
3. Expect to rewrite the agent layer frequently, and design so that’s okay.

Don’t treat agent code as sacred. Expect it to be replaced. Durability means the parts that should change can do so without breaking the parts that must not.

Code is Cheap. Assumptions Are Expensive.

Between code-generation models and new dev workflows, you can stand up a working demo, front-end, backend, voiceover, in an hour or two. On my team, PMs prototype experiences in agentic coding tools before we formally kick off engineering. This speed is incredible. It’s also a great way to bake in assumptions you’ll regret for two years. Simple rule: Nail one use case (or two). Then scale the patterns. Instead of designing a unified platform up front, we focus on solving one problem really well, with real customers, in production. We learn what works and what doesn’t. Only then do we generalize. Real example: Evals and dashboards. Everyone wants a unified view of agent performance. But each agent is solving different tasks. The meaningful metrics aren’t generic:

• Recruiting Agent: Did this move a candidate forward and respect guardrails?
• Travel Agent: Did this plan a flight between the right cities on the right dates, and automatically manage expenses?
• Legal Contract Intelligence Agent: Did this identify key terms and legal risks accurately without missing critical clauses?

We could try to spend weeks collecting requirements from every agent team and still miss what actually matters in practice. Instead we:

• Provide a minimal, hackable starting point
• Let teams fork and customize for what they need right now
• Periodically pull everyone back together, standardize only what’s proven useful

That tradeoff, shipping to learn the real requirements, then consolidating, beats trying to design the perfect cross‑agent dashboard in a vacuum.

Evals as the Contract

The thick foundation is the system of record, and the thin layer is the agent veneer. What keeps that thin layer honest? Evaluations. I think of evals as an executable contract between system and users. Those same evals are what let us see, very clearly, that the “one‑connector” implementation beat six months of custom agent work. The agent layer changed; the contract didn’t .A PRD says, “Users want to see time off remaining.” An eval makes that measurable:

• Input: A user, context, question
• Expected behavior: Right data, right format, right permissions
• Pass/fail: Assertion you can validate

The critical insight: this contract doesn’t care about implementation. Frontier models, hand-rolled loops or tomorrow’s framework, or MCP servers, if the eval passes, behavior is correct. This is powerful. It decouples user intent from implementation. You can swap models, update prompts, change tools, the contract holds. This turns improvement into optimization. With evals plus logging, you iterate toward better behavior, seeing immediately if you improved.

For one agent, we have three eval classes:

• Correctness: Can the agent generate valid schema changes?
• Safety: Does it respect access control?
• Satisfaction: Do outputs match what users actually needed? (Weekly feedback from real admins.)

We run correctness and safety on every commit. When something breaks, we have:

• Full traces and logs
• Ability to replay and reproduce
• A place to encode the fix as an eval

This creates a self-improving agent layer over time, not because the agent magically learns, but because we’ve built the teacher (evals and feedback loops) into the system.

Governance: Not Optional at Enterprise Scale

Early agent experiments happen in low-risk environments. We’re different: extremely sensitive HR and financial data, workflows where mistakes have real consequences. My rule is simple: never rely on an LLM to “do the right thing” by intuition. Instead, design systems where even unexpected model behavior can’t become a problem. The approach is to apply to agents what you’d already do for humans: strong, explicit access controls, principle of least privilege, clear audit trails, and scoped permissions aligned with real roles. These are foundational security practices. Agents need the same treatment. One concrete way to operationalize this is to treat agents as first-class security entities with specific capabilities. Rather than hoping an agent stays within bounds, you define precisely: which data can it read, which workflows can it initiate, whose behalf can it act on, and under what conditions. In practice, that means sandboxing agent interactions: scoping what they can see and do so that any unexpected behavior is contained to a narrow blast radius. This is the same permission model you’d apply to any service or user in your system. This design approach lets you do three things:

• Limit the blast radius of unexpected behavior, if the model does something surprising, the security boundary catches it.
• Reason about permissions consistently, the same way you do for humans and services, using frameworks you already understand.
• Plug agents into existing governance frameworks instead of inventing parallel systems for AI. Your audit logs, compliance dashboards, and approval workflows all work the same way.

Projects like OpenClaw are great for showing the upper bound of what’s possible when you almost completely ignore security and governance, they’re a glimpse of the form factor we should be aiming for. Our job is to bring that level of usefulness into a world where people and money data live, and to re-introduce guardrails, auditability, and trust without losing the magic.

Moving Fast in an Enterprise

Agent Factory is a startup from within. That’s less about aesthetics and more about who’s allowed to question assumptions. Two concrete examples:

• Release Cadence: Traditional enterprise: six-month cycles. In AI, that’s too long. If your design is frozen for six months while the frontier moves, you ship something outdated the day it lands. We take agents from concept to early access in roughly a quarter. Then iterate weekly with design partners, responding to model improvements, user feedback, new patterns. This speed doesn’t sacrifice rigor. Evals, observability, governance stay baked in. We just don’t wait six months between feedback cycles.
• Platform Openness: Historically Workday owned the UI. For good reasons, coherent experience, security, measured rollout. But agentic systems push toward channel-agnostic access: Workday as a governed system of record accessible from chat, third-party tools and custom apps.

We’re proving the model works through early access agents in Slack, showing value. Evidence makes the case to teams originally skeptical about opening the platform.

The Engineers I Want Next to Me

If you’re an engineer considering where to focus on agentic systems, here’s what we care about:

• Hungry to build and ship things that real customers use.
• Comfortable with ambiguity, but opinionated about contract and boundaries.
• Understand that durability now means designing for change, not avoiding it.
• Loves scrappy demos for what they are, the first iteration, not the finish line, and cares just as much about the governance and consequences of what we eventually ship.

From where I sit, the upside is that agentic systems in enterprise contexts are both technically deep and directly tied to business value. When done right, we change how organizations manage work, pay, careers, compliance at scale. Code may have a six-month half-life. Architectures, tools, and best practices will continue to evolve. Our goal is to build agents that can move with that frontier while standing on foundations we’ll be proud of years from now.

Designing Durable Agentic Systems: Thin Orchestration, Thick Data & Governance in Enterprise AI was originally published in Workday Technology Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

Last update: April 2026

2025: Technology Leadership in the Age of AI

Our delegation of 71 technologists represented Workday’s technology leadership at the Grace Hopper Celebration (GHC) November 2025.

GHC is the world’s largest conference for technology leadership organized by AnitaB.org. It’s a space where attendees learn from industry experts and tackle the most pressing challenges in computing through technical sessions, inspiring keynotes, and hands-on workshops. The conference is named after Rear Admiral Grace Hopper, a pioneer in computer science.

Our Workmates shared their expertise and sharpened their skills across engineering, cybersecurity, design, product, AI, and leadership while building connections that will strengthen our products and push our thinking forward.

Key Insights:

We’re building community across the tech pillars that matter: We hosted our second annual AI, Machine Learning, and Cloud Computing Reception, bringing together technologists, partners, and innovators to connect and share ideas.

Our Workmates are thought leaders: Software development engineers Deeheem Ansari and Adam Darcy led a hands-on threat modeling workshop that was filled to capacity, giving attendees practical frameworks to secure their products. Additionally, Kathy Pham, Vice President of AI, presented on socially responsible tech and introduced this year’s Technical Leadership Awardees, having been awared the Technical Leadership Award last year.

We gained critical insights into the tech industry’s trajectory: The sessions and workshops — spanning AI, product, engineering, design, leadership, accessibility, and more — provided a firsthand look from speakers representing leading technology companies at how organizations across the tech landscape are building and adopting technologies. The networking receptions also gave attendees opportunities for candid conversations about what’s working and what’s next.

Our commitment extends globally: Our partnership with AnitaB.org continues as we sponsor the upcoming Grace Hopper Celebration India, supporting the next generation of technology leaders, and building our Workday presence and community, in another key region.

2024: Technology

Fifty delegates and five speakers from Workday participated in technology and innovation sessions at Grace Hopper 2024. As part of its sponsorship, the Workday hosted the AI/ML and Cloud Computing Reception on October 9, 2024 at 5:30pm ET at the Marriott Downtown Philadelphia.

Taking place October 8–11 2024, both virtually and in person in Philadelphia, GHC 24 served as one of world’s largest computing and technology leadership conferences. The Celebration provided a diverse and vibrant platform for this community to connect, collaborate, and step into their collective power.

“Our sponsorship of GHC 24 reflects Workday’s deep commitment to fostering a more inclusive tech industry,” said Kathy Pham, Vice President, AI, Workday and 2024 ABIE Technical Leadership Award recipient. “We stand alongside Grace Hopper Celebration and AnitaB.org in championing diversity, and we’re proud to help empower the voices of women and nonbinary technologists — this week, and beyond.”

Workday representatives led in the following sessions:

AI/ML and Cloud Computing Reception
Hosted by Workday
Wednesday October 9 | 5:30–7:00 PM ET
Marriott Downtown Philadelphia
Request a ticket at https://tiny.cc/workday-ghc

From Spark To Scale: The AI Product Journey Through Different Lenses
Kathy Pham, Vice-President, AI, Workday
Nhung Ho, Vice-President, AI, Intuit
Chip Huyen, Vice-President, AI and Open Source Software, Voltron Data
Shruti Koparkar, Senior Product Marketing Manager, Amazon Web Services
October 9 | 10:45 AM ET
and October 10 | 10:00 AM ET

Accelerating Data Analytics With Vectorized Query Engine
Seema Gupta, Principal Software Engineer, Workday
Priyanka Bhalerao, Principal Software Engineer, Workday
October 10 | 1:00 PM ET

Tech Spaces and VPAT: How I Built Programs to Support Accessibility
Tierre Daniels, Manager Accessibility Product Team, Workday
Sim Sidhu, Senior Product Manager Accessibility, Workday
October 10 | 3:35 PM ET

Strategic Impact With OKRs and KPIs: Empowering Women for Organizational and Personal Success
Awalin Sopan, Principal Engineer, Workday
Reshma Shahabuddin, Chief of Staff to CPTO, Blackpoint
Anika Sharin, Senior Program Manager Regulatory Compliance, Intuitive Surgical Inc
October 11 | 12:30 PM — 1:30 PM ET

Technology Leadership at Grace Hopper Celebration Conference Over the Years was originally published in Workday Technology Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

Exceptional product managers know not to rely solely on instinct: we’ve all been wrong before. Instead, they’re savvy enough to pull from their arsenal of tools to help direct them in their decision making process. And one of the most powerful tools of all is data. Yet, data like any other tool relies heavily on how you use it. It is not sufficient to simply have the data, you need to know how to use the data to get ahead.

At Workday, our product managers recognize the importance of data and implement is strategically in our work and decisions. If you’re a woman in enterprise product management, we invite you to hear directly from our product leaders as they share their wisdom and practices when it comes to iterating on products using data and analytics.

Register for the event here: Advancing Women in Product & Workday co-present on Iterating on Data and Analytics on Wednesday, September 25, 2019 in San Mateo.

Iterating on Products Using Data & Analytics was originally published in Workday Technology Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

Start your Journey with a Map

A key product management skill is storytelling, especially true for highly technical products. It’s one of the best ways to convey your product’s value in a simple and easy-to-understand way, and it can be a major tool to align different groups with your team’s mission and vision. Describing the evolution of your product, explaining challenges faced, decisions made, and resulting outcomes is a powerful way to demonstrate the value provided to customers.

I’m the product manager for a collection of services known as the Workday Grid. The Grid is Workday’s job execution platform, and it helps run some of Workday’s largest workloads. For the Grid (and many other Workday services with a long history), succinctly summarizing our product’s lifecycle is a primary way to explain its growing value to our customers over time. My team and Iain Hull, principal development engineer (@IainHull), introduced me to Simon Wardley’s Value Chain Mapping as a tool to help describe the product history and value.

As I researched and learned more about Wardley’s mapping approach, I discovered it to be incredibly useful for telling the Workday Grid product story.

Any map should be able to show you where you are at present against a reference point, and where you are trying to get. A Wardley Map uses the customer as a reference point or anchor, so you can outline the main interactions of the customer with your product in terms of value. The X-axis moving from right to left defines the product’s evolution or lifecycle in terms of maturity from beginning to full utility/commodity, and the Y-axis handles visibility of components and their perceived value. The mapping process facilitates breaking the product into components, which allows you to highlight the different value propositions for your customers. This, in turn, applies focus to what is important in a competitive market.

I’ll demonstrate how I used a Wardley Map to reconstruct the past and present of the Workday Grid, gaining the knowledge to take my product into the future, better positioned in the constantly-changing technical and competitive landscape.

Mapping the Past

Learning about a product’s past gives you a better understanding of how it has evolved to where it is today. Using a Wardley Map in this context, I found that you can track the key decisions over the lifetime of the product. The Workday Grid was created to cater to various customer integrations, which can be defined as a programmed use-case involving interaction between Workday and one or more external services.

The first map (above) outlines the genesis of the Grid in 2010. Workday needed to create a standard and repeatable way to run these integrations, using standard, underlying components such as storage and resource allocation. Reuse of commodity products like the new, growing Amazon Web Services (AWS) provided an attractive technology solution. But even though it made sense from a technical perspective, at the time, many customers were simply not ready to move to the public cloud. It was too early — a normal reaction to emerging technology, with lack of understanding, limited availability, and security concerns hampering adoption across all industries.

It’s important to outline the development challenges that were faced back then, and to tell an honest story of the product. In order for customers to use our solution, we moved the storage and resource allocation back into the Workday data centers, and developed it there. This involved a lot more than software engineering; it required servers to be deployed, and all the associated costs and effort that come with them. We also needed to productize the platform API to ensure we continued to scale in parallel with limited resources.

Mapping the Present

After completing the Grid features targeting initial customer needs, the product needed to transition to address platform scale. Here, the Wardley Map highlights areas of the product in a custom-built state. The team worked quickly to evolve towards a commodity platform offering, increasing value and overall adoption. Customers recognized the value of a platform over a custom-built solution.

As the platform scaled, so did customer confidence. And as a result, new use cases began to emerge. At that point, the Grid could diversify its offerings, by tailoring the job scheduling for more advanced use cases that could be deployed and modified using well known, repeatable methods.

Job scheduling of increasingly similar integrations provided opportunities for breaking the single service into distinct microservices — all being maintained by Grid architecture catered to the Workday tenant lifecycle. The important takeaway is that the Grid lets customers focus on their application, without worrying about other items such as hardware, lifecycle, dynamic scaling and capacity planning. They can run integrations by simply adding them to the platform.

Additionally, the Grid leveraged a much more productized offering on storage and resource allocation.

Mapping the Future

Every product manager must be acutely aware of what features need to be built in order to stay relevant in a constantly changing technical environment. You should embrace this change! As new technologies arise, be ready to take advantage of them to further your overall product vision. At its core, the Grid is made up of two main components, job and resource allocation. The introduction of Kubernetes, with its exceptional resource scheduling, provided us an opportunity to review our offering and ensure relevance in a changing environment. So we decided to develop a strategy for leveraging this new technology.

There is also a wider context of using Kubernetes as part of our offering. It effectively means we can now have a single developer and customer experience in the Workday data centers, and on AWS or any public cloud. And with the Grid being a more portable platform, our time to market is drastically reduced.

This will eventually provide customers the ultimate choice of where their data is stored and used. We know that customers appreciate this greater flexibility and transparency on demand. We continuously review our offering and prioritize customer needs at the top of our decision making.

Conclusion

Wardley Mapping can help you construct a living story of your product, continuously monitor emerging technology and competitors, and communicate customer value over time. I recommend this technique to other product managers to better understand their product history and evolve their product strategy.

Resources:

More information on Wardley Mapping: https://medium.com/wardleymaps.

Simon Wardley on Twitter: @swardley

Recent article reference:

https://evolve.hiredthought.com/

Events that you can attend to see how others leverage maps for a range of topics, with further details here: https://www.map-camp.com/

About The Author

Aidan Molloy is a Senior Product Manager responsible for a number of products within the Workday Grid, ensuring Workday continues to scale and orchestrate across a number of platforms such as Kubernetes, AWS, VMware and Openstack. He holds a B.Sc; Honours Degree in Computer Science and Software Engineering and is currently studying for his Msc in Strategy and Innovation from Maynooth University School of Business.

Twitter: aidan_molloy

LinkedIn: https://www.linkedin.com/in/aidanmolloy/

Detailing the Evolution of the Workday Grid through Wardley Mapping was originally published in Workday Technology Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

By Ryan Hennessey, Sr Product Manager, Workday Cloud Platform, Workday

I may be late to the party here, but I recently started watching The Great British Baking Show. Besides Paul Hollywood’s piercing blue eyes, Mary Berry’s sweet jackets, and the truly terrible puns, there’s one more thing I love about it: going back to basics in the semi-finals and finals.

Every amateur baker knows how to make the perfect scone just like how product managers know how to execute a well-designed product. Baking requires a very precise process; any missed step in the process will result in a half-baked scone that the judges refuse to eat (cringe). Product design is surprisingly similar. Trust me, one time my dad forgot to put eggs in a cake. That’s a story for another time, though.

It’s easy for us to get caught up in executing feature after feature. We try to make it to the next week, and suddenly we have a messy 6-tiered cake with 5 different themes and 15 competing flavors. By going through the basic steps of the product management process — prototyping, designing, refining, and delivering — we ensure that we’re not forgetting any key components (such as eggs).

As part of the broader Workday Cloud Platform (WCP) group, my team builds the platform that developers use to write the front-end of their custom applications. Our goal is to make the development experience of building and managing these applications as painless as possible. With the introduction of WCP, we were presented with the opportunity to build a consumer-facing product, in the form of a third party IDE plugin that supports developers working with our specific language and application management. As a team that traditionally works on things from an infrastructure or platform perspective, having a consumer product to execute on allowed us to go through the process of Product Management 101.

Prototype It

On The Great British Baking Show, our amateur bakers practice their signature bake at home the week prior. They go through iteration after iteration to find the perfect tasting loaf, just to prove that they’ll be able to do it at show time. Ideally, it should be pretty, but it needs to be at least edible in order to get the title of star baker. Getting something that works, that you know you can accomplish, and has been tested (or tasted) by people, is key to success.

For us, an IDE plugin was new territory. We had an internal plugin that did some basic functions such as type-ahead and syntax help for writing the code. When WCP came along in 2017, we needed to add more functionality to support the platform, such as logging in to a tenant, migrating applications, and adding some usability enhancements. Suddenly, this little internal plugin became a prototype for a larger product, just in time for a hackathon in the summer of 2017.

What defined our prototype? The features supported the use case we were testing out with the hackathon: developers writing and managing custom applications. No frills, no design. (Well, our developer provided some lovely icons but I wouldn’t exactly call them “design.” Sorry, Kevin!) We were able to prove it at the hackathon, get it in front of people, test the end-to-end of delivery and installation, and walk away with a lot of valuable feedback. Going back to basics here meant that, well, our basics were in place! Just like our bakers, when they test on their family and friends throughout the week, and then walk into The Tent ready to bake the perfect tasting loaf.

Brand it

The bakers have just a few hours to turn their prototype into the final deliverable and we had about six months to make the perfect plugin. Within that time, we were able to go back to the basics of design and branding, something an infrastructure/platform team doesn’t get to do too often.

It was time to frost our little plugin. Like our friendly bakers who help each other even in competition, we had one of our UX team members help us with the usability, branding, and messaging in the plugin. Now, this is a development tool being built for a well established Integrated Development Environment (IDE). We had restrictions, but from our prototype, we knew what customizations we could make. We wanted to take advantage of this opportunity and go through all the design motions. After all, this is the fun part! We should get to play too.

We developed a logo that paid homage to the original name of the plugin (Cage), and aligned with the WCP logo and spirit. A key takeaway here was keeping it yours while working with the standards you have. Much like the signature bake, everyone may be making tarts, but the bakers must represent themselves while making their tart stand out at the same time.

After the logo was defined, we could easily create icons of the various functions we needed and move through design quickly. We also validated, with our UX team, even the smallest of flows, such as Login to Tenant. Every detail from menu placement to message copy to error handling matters when developing an easy to use, intuitive product. It’s easy for these basics to get away from us in the process, but just remember that there’s nothing more cringe-worthy than when Mary Berry is confused about how to eat a cupcake.

Refine it: Substance over Style

The fourth series winner, Frances, designed beautiful pieces of artwork for her bakes, but oftentimes missed a flavor or two. Judge Paul would repeatedly drive home the point of “Substance over Style!” Defeated week after week, Frances refused to let her style wane, but improved her substance. My brain goes immediately to this analogy as we developed our plugin.

It is very easy to get caught up in the style and forget the substance. We asked ourselves, it works, it looks nice, but what is missing? We brainstormed features that would round out this product even more. We added features such as validations and documentation into the product. The plugin started to take shape as a key development tool for WCP, not just for writing the UI. We just had to ask ourselves the basic question of “what will make the developers lives’ easier?”

But substance isn’t only the feature or flavor of the product, it’s the quality as well. If the substance of the product is not reliable or consistent, all the value of that substance is lost. It can be difficult to keep up with automation testing when you’re under deadlines, or when you’re playing catch up. We had the time and the bandwidth to create an automation framework within the core of the product. First, this required a set of automated tasks that would set-up our test environments with some settings needed to run WCP. We were the first team to cross this path and created a framework we can apply to all our internal services, that will allow for automation in these environments. Second, we wrote the end-to-end tests with for the plugin (push app, fetch app, view app, etc) using a mix of another internal system testing framework, and what the 3rd party IDE provides for testing plugins. It was important to make these system tests a priority, in addition to the basic unit tests on our codeline.

Deliver it!

Ah, your product looks good, feels good, and is consistently working. It’s time to release it to the public. How do we present it, how do we educate users, how do we make it easy to get?

These next steps are the final mile of the process but still so important. The presentation is the first thing that the judges comment on, the first impression of the product to our users. For us, there were a couple of steps to wrap up the plugin into a neat package: documentation and the install process.

Working closely with our technical writer, we made sure that there was a good (but not over-explained) introduction to the tool on the WCP site. Highlighting key features and educating new users as to what they’ll use it for is a must-have to get it off the ground. Collaborate with your writer on this, don’t just delegate to them. It’s your baby, make sure the product is represented in the way you and your customers would want.

The install/delivery process was to me, the cherry on top, the final touch, the first thing the judges see. This sets the tone for the substance and style of the product as a whole. Users can’t enjoy a product if they can’t get it. This is the most basic function of the product and completes the experience, remind yourself to design this process just like you did the product.

Enjoy!

I didn’t touch on all our takeaways (analytics, user testing, etc) but these were the key moments of the process that made the biggest difference in developing a product we are proud of. It’s always a good idea to go back to basics, particularly when you’ve been doing something for a long time. You’ll be surprised at how much fun you have, how much you and your team learn, and how proud you are of what you’ve built. That’s basically a direct quote from every contestant and it makes you feel warm and fuzzy, but it’s true!

The feedback will roll in, you’ll continue to improve your product and your skills, and you’ll relish the love it gets as well. On your next product, we hope you’ll hear “substance over style!” and “back to basics week!” in the back of your mind and apply some of these reminders to your process. For now, enjoy bingeing The Great British Baking show.

This post is a part of two-part series. You can find Part 1 here.

How to PM a Platform: The Journey to Build Workday Cloud Platform (Part 2) was originally published in Workday Technology Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

Have you ever been given an opportunity to embark on a massive evolutionary journey for a product? Workday gave me the opportunity to do just that. In this post I will focus on how that journey unfolded as we evolved from building applications to building a platform.

Workday has successfully been building and delivering applications across finance and HR over the course of the last 14 years. We are leaders on Gartner’s Magic Quadrant for our Cloud Human Capital Management and Cloud Core Financial Management suites within the midsize, large and global enterprise space. This success has largely been due to the high customer satisfaction we provide with our core products. To achieve the breadth across size, industries and geographies, Workday has always embraced extensibility at its core. We’ll start our journey there!

Extensibility Journey at Workday

Our journey of extensibility started when we were just a young company looking to make a mark in the world of human capital management. Workday’s philosophy has always favored configuration over customization, whether it be configuring custom organizations, creating business process definitions or writing custom reports, it can all be done by an administrator.

Over the course of the last decade, the frameworks that we built became varied and quite powerful such that customers could extend our data model using custom objects, personalize the UI or write integrations that connect to 3rd-party systems. We saw first hand how customers embraced these frameworks to meet the needs of organizations across various industries, geographies, and scale. Workday Cloud Platform is the next evolutionary step in that journey.

Opening up our Platform

After we announced our intent to open up our platform at Altitude, our annual partner conference, in 2017, we started to outline our customers’ problems, and how our platform can help solve those problems. Our research included interviewing some of our trusted customers and partners, who attended our various product advisory councils to learn about the problems they faced and what existing application platforms had to offer.

We uncovered that users were having fragmented experiences across systems, dealing with inefficient business processes, and customers had unique requirements that required extreme customizations. Many shared how painful it was for them to maintain hundreds of peripheral applications; if given an option they would love to see it in one system.

Through the process of gathering feedback, we gained clarity on the goal of this next phase of the journey. The goal was to build an ecosystem that enables developers to build applications that are closely tied to our core offerings Workday Financial Management and Workday Human Capital Management (HCM).

Fork in the Road

With the Workday Cloud Platform, we want to provide customers and partners a way to design, build, deploy and share applications that run on and integrate with Workday. There were two ways in which we could achieve this.

We had become really good at building frameworks for tenanted configurations, as our history shows. However, Workday itself has been successfully building and deploying applications. We have been doing this at scale with 2000+ application developers actively building using our internally developed tools. We could productize and expose these tools to the outside world as part of the platform. However, this would be a multi-year project.

We decided on a two-prong approach, where we deliver some of the internal tools capabilities through the form of tenanted configurations that could help solve our customers needs today while simultaneously investing in externalizing our internal tooling to the outside world that could scale to become multi-tenanted.

Today…

After a successful design partner program, we identified the MVP for our product that we needed to launch Workday Cloud Platform with. It has been a little over a year since we officially launched our early adopter and fast follower programs. We are happy to report that customers and partners are taking full advantage of our platform capabilities on the journey towards even greater extensibility. While leveraging the Power of One, they are able to create their data model, write their business logic, create net new layouts and forms combined with external data, all done using our tenanted configurations today; in the long run, we expect to leverage one set of tools for building multi-tenanted applications, both internally and externally.

This is part 1 of a two-part series. In part 2 of this series, we will walk you through how we went about externalizing our development tool.

How to PM a Platform: The Journey to Build Workday Cloud Platform (Part 1) was originally published in Workday Technology Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

At our recent Rising conference, Workday announced we are raising our availability SLA from an already industry-leading 99.5% to 99.7%. (In practice, we’ve been beating even that number, consistently delivering better than 99.9% availability.) We understand that maintaining and improving our availability is crucially important to customers. So I thought it would be a good time to talk about some of the steps being taken by our internal service teams to use efficient monitoring to help deliver on this availability SLA.

In my last blog post, I talked about monitoring infrastructure changes at Workday, including how we incorporated a technology called Prometheus. Prometheus in Greek mythology enabled progress by giving fire to humanity. The Prometheus software also enables progress — not through fire but through improved visibility into service health. Workday developers could now easily add new metrics and alerts for their service, container, or software component in minutes.

The thing is, our developers did exactly this. In spades. And like playing with fire, playing with this volume of new metrics can be tricky.

These new metrics and alerts added much-needed visibility and value to individuals and teams, but the combined effect was a firehose of data noise. The new metrics and alerts tended to stay around forever, even if they were added for a transient issue. At the same time, Workday keeps growing and advancing our product offering, including, most recently, advancing Performance Enablement through a new approach to enable deeper employee-manager relationships and equipping our customers to better plan through our Adaptive Insights-powered business-planning offering.

The combined effect made it difficult to know which consoles to look at in emergent situations. The first step we took to address this was to deploy a tool called BigPanda, which uses machine learning to correlate alerts into insight-rich incidents and present them in a single pane of glass. This makes it easier to respond to, and resolve, problems in our infrastructure.

If you can measure a problem you can manage it, so to combat the underlying noise problem, we introduced and then reported on various monitoring-related artifacts, on a per-team basis. This meant that we could now analyze the noise by team and tool, and present this information clearly using KPIs, metrics, and dashboards.

Now we’re able to measure ownership by service team, as well as by other parameters where applicable, such as Chef role and Chef cookbook name. That allows us to work with relevant teams to drive out noise that doesn’t add value. Armed with these insights, we started a program to review the monitoring used by our service teams. I broke the ice at the start of one of these reviews with a joke about how our customers are more than just a number to us — they are a sequence of many numbers that have a temporal and logical relationship to each other, stored in a time series database and rendered as a series of data points in a graphical UI. (Crickets). Jokes aside, the reviews cover all aspects of monitoring from a time-series data viewpoint — metrics, dashboards, checks, alerts, runbooks, etc.

Now, when service-impacting events occur, we can conduct down-to-the-minute root cause analysis reviews to see where we can shave off each minute of impact. Typical actions that come out of this are:

- Earlier alerting so that impact can be avoided.

- Better metrics that provide visibility into service health.

- Use of better dashboards during events to expedite troubleshooting

- Where action could not be automated, using better runbooks so on-call engineers can handle issues faster.

Side note: As I was writing my previous blog post, I was interrupted by my dog giving birth. The puppies have since grown to be big and bold, and are enjoying life to the fullest! And they will fit right in here, given that Workday was ranked by People Magazine in the Top 12 Most Pet-Friendly Companies.

Moving forward, we continue to analyze our operational performance. This includes measuring time-to-action (MTTA) and time-to-resolution (MTTR) for critical incidents, incident volume, whether the issue was detected and remediated by the monitoring infrastructure, and what the impact of the issue was. We’re also measuring how effective we are at our goal of a “Zero Inbox” for alerts across the company so we avoid the broken window syndrome. The measurements are stored in our time series database, so we can do a mathematical analysis on them similar to how we analyze our other metrics — for example, looking at a particular percentile such as P95 for a given metric.

Last October, my colleague Cristina Goldt has announced Workday’s machine learning-powered skills cloud offering as part of our vision for a frictionless marketplace. This will undoubtedly drive further growth in the use of the Workday service. In the next blog post we can discuss how this impacts monitoring. And, how the dogs are doing…

How We’re Using Monitoring to Support Our Increased Availability SLA was originally published in Workday Technology Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

Workday Assistant Design & Development

Workday’s product, design, and research teams are very thoughtful when approaching this new user experience paradigm. A naive onlooker would be tempted to write up requirements for a bot to simply “do everything” that the web or mobile application can do already. Effectively treating chatbots as a new “user interface”, where the features it should support are the same as elsewhere in the application.

This is a grave mistake. We’ve seen the same pattern on mobile, with dismal results — mobile applications that coarsely implement 100% feature parity with the web experience are rarely successful. The key to unlocking value and delighting users is to understand mobile for its unique value, designing features that capitalize on both the form factor and user benefits. Take, for example, Workday’s Optical Character Recognition (OCR) receipt scanning technology available in our mobile app today. It’s by far one of our most popular features, and it’s because it allows an easy way to capture receipts on-the-go with the mobile device camera and OCR technology to automatically detect receipt details for entry into the Workday expense report system.

Chatbots are, indeed, a next-generation user experience. Historically speaking, we have seen a continual wave of new paradigms for applications — the desktop computer with native software applications (examples: Microsoft Word, Adobe Photoshop), the desktop computer with browser-based applications (examples: Gmail, Salesforce), the mobile phone application (examples: Workday mobile app, Instagram), and most recently the mobile phone via voice interface (examples: Siri, Google Assistant). Each one of these new user experiences at the time challenged our assumptions, making us reconsider how to design applications given the new formats and technologies. Chatbots are no different.

Designing for Conversational Interfaces

What makes a chatbot different and unique from other user experiences? Simply put, it is a conversational user interface. Thus, Workday’s design philosophy is focused on the key question — When would it be a better user experience to engage in a conversation, rather than using a traditional UI flow like a webpage or mobile view? For Workday use cases, the answer to that question tends to fall into two areas.

First, thanks to the power of mature Natural Language Processing (NLP) technology, a bot experience is an excellent candidate for when the user doesn’t necessarily have the right domain language to find what they are looking for. This is a slam dunk for enterprise use cases, since it’s incredibly common for the average employee to not have crisp language when describing things like their HR needs or policies. Take this simple example of an employee trying to get a copy of their most recent pay slip — here is a myriad of ways they might initiate that conversation with a colleague or HR business partner, all of which can be understood by a bot instead: When did I last get paid? How much was I paid? Where is my paycheck? How much money was deposited into my bank account last week?

Second, chatbot technologies are perfectly positioned to leverage the emerging adoption of voice-enabled devices. Whether it’s your home device like Alexa or Google Home, your mobile phone while driving, or your smart watch during a run to the supermarket, voice is increasingly one of the most frictionless ways to engage and get simple tasks done while on the move. However, it doesn’t stop there.

In the early days of customer discovery, we learned that a leveraging a group of products and technologies together, make using Workday simpler. Designing and delivering an exceptional user experience, means thinking about a complementary design strategy in these areas:

• Workday Assistant — a native chatbot to help users more easily retrieve information and complete Workday tasks. Accessible via both voice and in natural workspace applications like Slack and Microsoft Teams.
• Workday journey-driven homepage — a homepage that quickly orients users based on their role within a company, surfacing key information to absorb or act on. Simplifies core tasks and gives signal to the noise for what a user should focus on when logging into Workday.
• Workday search — a one stop shop to navigate around the rest of Workday. Easily retrieve information, spanning the entire breadth and depth of Workday (and related systems).

All of these user experience technologies are geared toward Workday becoming simpler, more purposeful, and personalized to the user wherever they are in their own career journey. For our internal product and design teams spanning these areas, this means doing in-depth product discovery with our customer base to ensure each area is solving complementary problems (without all 3 trying to fix every problem, leading to competing and duplicative experiences). To return to a theme from earlier, each user interface should be thoughtfully designed to deliver an experience that leverages its unique capabilities. And if we do our job right, the set of technologies work together in concert.

One Experience Built on Millions of Conversations

Whenever we build something at Workday, our product management team is obsessed with the idea of Workday’s “Power of One.” It’s the promise that when we build something that benefits one customer, we are committed to finding ways to have it also enrich our entire customer ecosystem to help us learn and grow as a community.

Bot learning is a “big data” challenge, and the millions of Workday users and their conversations with the Assistant provide the perfect input to the process. Contrast this with the bot platform strategy I raised in Part 1 of this series — where customers instead rely on a platform offering to build and deploy their own custom chatbot experience. If they go this route, the bot training dataset is limited to utterances only from employees at that specific company. To truly leverage the power of machine learning, a large training dataset is required; candidly, with the custom bot approach, many customers may never reach the scale of usage required to receive the benefits.

Since the Workday Assistant is delivered to all customers, we can leverage the power and data of our full customer community to train the bot — not only for a breadth of utterances for a given flow, but also to capture language-specific variants as the bot is localized to reach employees across the globe. If we do our job right on the product side, not only will the Workday Assistant bot get smarter for all customers as our adoption grows across the customer base, but we’ll also explore ways to gather and share the most salient learnings for use elsewhere in your business. As an example, a library of “utterances” (the natural language phrases used by your employees) about a given topic, can allow your team to write more effective documentation and training guides.

There is no doubt that it’s an extremely exciting time in the enterprise chatbot space. While there is much to be figured out, Workday is betting on continuous collaboration between Workday and our community to solve this problem the way we have solved all others — through extensive customer research, thoughtful user experience design, and fast iteration to get to product-market fit.

Want to learn more? Stay tuned for more updates over the year on the Product blog! Customers who would like to learn more about the Workday Assistant can contact their Customer Success Manager.

About the Author

Katie Holden is a Director of Product Management with a team responsible for Workday’s productivity products (Worksheets, Livepages, Drive, Assistant). She is a technical product manager, who found her way into the field shortly after studying Electrical and Computer Engineering at CU Boulder. Her experience ranges from shipping products in large organizations such as Microsoft and Google, as well as smaller, SaaS, high-growth startup environments like Boku (IPO in Nov 2017) and Inkling (acquired in Feb 2018).

linkedin: https://www.linkedin.com/in/katherinecholden

twitter: https://twitter.com/katiecholden

Professional Opportunities

If you or someone you know wants to be part of this story in real life, Workday is currently hiring for individuals to help develop the next wave of products.

The Rise of Enterprise Chatbots (Part 2) was originally published in Workday Technology Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.