Retell AI guide: How to build and test a reliable voice agent

What Retell AI is

Retell positions itself as a voice-AI orchestration layer: it connects speech-to-text, an LLM, and text-to-speech (or a combined speech-to-speech model) and adds the production glue that raw building blocks don't handle well on their own, automatic model fallback, background-noise filtering, echo cancellation, turn-taking, interruption handling, voicemail detection, and call transfer. The docs frame the whole product around one lifecycle: Build, Test, Deploy, Monitor.

To build, you pick between two agent paradigms: a Conversation Flow agent, a drag-and-drop, node-based builder for structured, deterministic call logic, or a Single or Multi-Prompt agent, which is LLM-prompt-driven and better suited to looser, more conversational calls. Retell is also model-agnostic on both sides of the stack: you choose the LLM (GPT, Claude, or Gemini families) and the voice provider (Retell's own platform voices, ElevenLabs, Cartesia, OpenAI, MiniMax, or Fish) independently, and Retell automatically falls back between providers if one has an outage.

It's built for developers, with an API, SDKs, an MCP server, webhooks, and custom telephony over SIP, but the self-serve tier ships full platform access: templates, call analytics, transcripts, simulation testing, and API access are all included, not gated behind a higher plan. For a feature-by-feature rundown against the rest of the AI voice agent category, see the full Retell AI review.

What you need before building

Before wiring up an agent, get the pieces in place:

• A Retell account with a phone number, either Retell-provided or your own via custom telephony.
• A connected Cal.com account, if the agent is booking appointments. Retell's calendar tools work against Cal.com directly; your actual calendar (Google, Outlook, etc.) connects underneath Cal.com, so Retell never needs to sync a separate copy of your schedule.
• A clear scope for the agent. What it should handle, what it shouldn't, and when it hands off to a person. This is a planning step, not a Retell feature, but skipping it is the single biggest reason agents feel unreliable once they're live.
• A support channel for the cases the agent can't close, whether that's a live phone transfer, a ticket in Zendesk or Freshdesk, or a Slack alert to a human team.

How to build an appointment-booking voice agent

Here's the workflow I'd follow, based on how Retell's own docs describe the calendar tools.

1. Define the conversation

Map the call before you write a prompt. What does a clean booking conversation look like start to finish? What happens if the requested time is taken, if the caller wants to reschedule, or if they ask something outside the agent's scope? Decide your handoff triggers now, such as a caller asking for a person directly, or a set number of failed attempts, rather than discovering them live. This is the step that turns a vague "book appointments" instruction into an agent you can actually test against, and it's worth spending real time on before touching the builder.

2. Supply current context with dynamic variables

Retell's dynamic variables inject live, call-specific data into the agent's prompt using {{variable_name}} syntax. For temporal context specifically, Retell provides {{current_time}} (Pacific time by default), {{current_time_[timezone]}} for a named timezone like {{current_time_Australia/Sydney}}, {{current_hour}}, and {{current_calendar}}, which generates a 14-day calendar view. Reference the timezone-specific variable that matches your callers, not the bare default, and the "what does 'next Wednesday' mean today" problem is solved with configuration rather than a custom function written to calculate it yourself.

3. Check calendar availability

Retell's built-in Check Availability tool lists open slots for a Cal.com event type in a given window, pulled live rather than from a synced copy. Set it up by grabbing the numeric event type ID from your Cal.com dashboard URL (the 1234567 in app.cal.com/event-types/1234567) and configuring one tool per event type your agent handles.

4. Book or offer another slot

Chain the Book Appointment tool after a confirmed slot to finalize the reservation, Cal.com sends the confirmation email automatically. For changes, the recommended pattern is to use List Bookings to find the existing booking by the caller's email, Check Availability to confirm a new slot is open, then Reschedule Booking to move it, mapping the booking UID through a dynamic variable so later tools can reference it. If a slot isn't available, the agent offers alternatives from the same availability check instead of failing the call.

5. Test normal and edge cases

Before any of this touches a real caller, test it. The next section covers Retell's testing tools in detail, the short version is: use the Playground for live debugging, then Simulation Testing to grade the agent against written scenarios, with every calendar tool and custom function mocked so a test run doesn't create real bookings.

6. Roll out and monitor

Once testing looks solid, activate the agent for a slice of real traffic before switching it on for everyone, a single phone number, one call type, or a percentage of inbound volume, and watch the results before expanding it. The section below on rolling out and monitoring covers what to actually watch.

Configure knowledge, prompts, and actions

A prompt sets the agent's personality and immediate task, but three separate pieces do the real work, and keeping them separate is what keeps an agent maintainable as it grows:

• Knowledge base. Retell's Knowledge Base add-on grounds answers in your own content via website crawling or document uploads, so you're not trying to teach the agent everything through prompt engineering. It's billed per minute of use plus a monthly fee per knowledge base after the first ten are free, worth factoring into cost if an agent leans on it heavily.
• Custom Functions. A custom function calls your own API mid-call and feeds the response back into the conversation, useful for order lookups, ticket creation, or anything that needs authenticated access to your own systems. You configure the endpoint, method, headers (including dynamic variables for auth), and how the agent should behave while it waits for a response.
• Code Tool. For logic that doesn't need a backend, Retell runs JavaScript directly in a sandboxed environment, good for calculations, formatting, or a low-risk public HTTP lookup via fetch(). If you're already running your own MCP server, Retell's MCP tools are the simpler path instead of writing a new custom function; the docs are explicit that a custom function is the right call only when you don't already have one.

The common failure mode here isn't a Retell limitation, it's cramming everything into one giant prompt instead of using these three surfaces separately. A prompt that tries to hold knowledge, instructions, and API logic all at once is fragile and painful to update; the fix is using the knowledge base for information, the prompt for personality and flow, and functions or tools for anything that touches an external system.

Test the agent with Retell's testing tools

Retell's self-serve tier includes real testing infrastructure, not just a live-call gamble:

• Playground. An interactive environment to debug the agent live, over text or audio, before anything is exposed to real callers.
• Simulation Testing. An AI-simulated caller runs through written scenarios, each graded pass or fail against success criteria you define, with dynamic variables to customize conditions. This is where you catch a broken date calculation or a bad handoff before it reaches a customer, not after.
• A/B Testing. Compare two agent versions against live traffic instead of guessing which prompt or flow performs better.

One caveat worth planning around: Retell's docs are explicit that unmocked custom functions, code tools, and calendar tools call their real endpoints during a simulation, which means an untested run can create actual Cal.com bookings or hit billed API calls. Mock every external action before running scenarios at scale. Simulation Testing also doesn't currently support agents built on a custom LLM, and it grades each scenario as a single pass/fail verdict rather than a per-criterion score, so treat it as a strong first gate rather than the only QA step.

Roll out and monitor it

Resist switching an agent on for everyone the moment simulation results look clean. Start with a small slice, a single number, a specific call type, or a percentage of total volume, and expand from there as real-world performance holds up.

Once live, Retell's monitoring tools cover the operational side: Live Call Monitoring for real-time transcripts and listening, AI QA for automated quality scoring on production calls, Custom Analytics dashboards, and webhooks for real-time call events feeding into your own systems or a CRM (Salesforce, HubSpot, and others sync natively). The metric worth tracking closest is the share of calls the agent closes without a transfer, the phone equivalent of an automation rate in text-based support.

Worth watching alongside performance is the actual cost per minute, since Retell's pricing is composable rather than a flat number. The $0.055/min voice infrastructure charge applies to every call regardless of configuration, and Retell's own pricing calculator lands its default example, a standard voice, a mid-tier LLM, and managed telephony, at $0.115/min, well above the advertised $0.07 floor. That's not a hidden fee; every component is published on the pricing page, but it's worth modeling your actual stack rather than quoting the headline number when estimating cost at volume.

Common implementation mistakes

Going through Retell's docs and setup flow, I found most of the frustration people run into traces back to a handful of avoidable setup choices, not the platform itself:

1. Stuffing everything into one prompt. Instructions, business knowledge, and API logic crammed into a single prompt is fragile and hard to maintain. Split them across the knowledge base, the prompt, and actions instead.
2. Skipping function mocks before simulation. Running Simulation Testing against live calendar tools or custom functions can create real bookings or hit billed endpoints. Mock everything external first.
3. Ignoring timezone on date variables. {{current_time}} defaults to Pacific time. If your callers are elsewhere, reference {{current_time_[timezone]}} explicitly, or the agent will reason from the wrong local date.
4. No defined handoff trigger. Without a clear rule for when to transfer to a human, agents tend to loop on requests they can't resolve instead of escalating cleanly. Decide this during planning, not after a bad call.
5. Estimating cost from the headline rate alone. The $0.07/min floor assumes the cheapest LLM and voice combination. Model your actual stack, LLM, TTS provider, telephony, and any add-ons, before budgeting for volume.

Retell AI versus eesel: different jobs

It's worth being direct about this since the two get compared: Retell and eesel solve different problems. Retell is a platform for building and running phone-call voice agents, the orchestration layer for STT, LLM, and TTS on live calls. eesel is an AI for the helpdesk, trained on your existing knowledge base and past support tickets, working inside tools like Zendesk, Freshdesk, or Front to answer tickets and chats.

They're not substitutes for each other. A support team might run a Retell agent on the phone line for appointment booking or order lookups, and separately run eesel on their ticket queue to handle the email and chat volume, both trained on the same underlying knowledge but built for different channels. If a customer's real need is testing AI against their own ticket history before launch rather than building a phone workflow, that's the difference worth knowing before picking either one.

Final practical checklist

Here's what I'd check before turning a voice agent on for real callers:

• The conversation is mapped, including edge cases and what happens when a slot isn't available
• Handoff triggers to a human are explicitly defined, not left implicit
• Date/time dynamic variables reference the correct timezone for your callers, not the Pacific default
• Cal.com is connected and the event type ID is configured in the Check Availability and Book Appointment tools
• The knowledge base is scoped to what this agent should actually answer
• Every custom function, code tool, and calendar tool is mocked before running Simulation Testing
• Simulation scenarios cover both the happy path and realistic edge cases, not just the ideal call
• Rollout starts on a slice of traffic, with Live Call Monitoring or AI QA watching the results
• The real per-minute cost has been modeled against your actual LLM, voice, and telephony choices, not the headline rate