OpenAI Realtime API: a current guide to live voice support

Pick the Realtime job before the transport

“Realtime voice” covers different jobs. OpenAI recommends a Realtime voice-agent session when an assistant should respond to a caller, manage conversation state, and call tools. Use a translation session when the product continuously translates arriving speech. Use a transcription session when the product needs streaming text but no model-generated spoken response.

Use a live session only when the experience needs a live response. For a recorded file or bounded transcription request, OpenAI points teams to request-based audio APIs instead.

Choose a model and connection that match the call

For a new speech-to-speech agent, OpenAI's current guide starts with gpt-realtime-2.1. It adds configurable reasoning and tool use to realtime voice workflows. Start with low reasoning effort for most production voice agents, then test whether a more deliberate response is worth the additional latency and output-token use.

The connection method follows the audio path, not a generic preference:

• WebRTC fits a browser or mobile client that records or plays audio directly.
• WebSocket fits a server that already receives raw audio from a media pipeline, call system, or worker.
• SIP fits a telephony voice agent. Confirm the model supports SIP for the exact translation or transcription case before committing to it.

The older beta interface is not the starting point for new work. OpenAI's GA migration guidance removes the old beta header and uses client secrets for browser or mobile credentials. Keep a standard OpenAI API key on the trusted server; do not ship it to the caller's browser or phone client.

If the application identifies individual callers, OpenAI recommends sending a stable, privacy-preserving safety identifier. A hashed internal ID is an example. It is an abuse-prevention signal, not a substitute for your own caller authentication, consent, access control, or data-retention policy.

Build the tool boundary before the voice experience

The Realtime API can return tool calls, but a model's tool request is not approval to change a customer record. Decide what the voice agent may read, what it may draft, what needs a hold, and what remains a human decision. A refund, account change, address update, or promise about delivery should be verified against the system that owns it.

For a support voice flow, test both the conversation and the operation:

• A caller asks a clear policy question and receives an approved answer.
• A caller interrupts, corrects an order number, or has noisy audio; the system does not continue with an assumption.
• A caller asks for an exception, refund, or sensitive account change; the agent gathers evidence and hands off rather than promising an outcome.
• A tool call is denied or held when the policy requires human approval.
• The caller's transcript, ticket, and any external action agree on what actually happened.

Use real audio conditions in the evaluation: target languages, accents, background noise, poor connections, and the names or product terms people actually say. A text-only happy path cannot prove that turn detection, transcription, or tool use will behave safely on a live call.

Cost and operating checks

Realtime costs depend on the selected model and the text, audio, image, or reasoning work in the session. The current model page and pricing are the source of truth, rather than a copied rate table. Higher reasoning effort can increase latency and output use, so test cost and response quality together.

Budget for more than model usage. Include connection infrastructure, telephony if relevant, transcripts or recordings, monitoring, retries, human handoffs, and the support staff who review unsafe or confusing calls. A low cost per minute is not useful if callers are routed incorrectly or a tool performs the wrong change.

Use eesel CLI for voice-support follow-up

The OpenAI Realtime API is for building and operating a live voice application. eesel is a ready-to-work helpdesk teammate. eesel CLI is a terminal route into the same teammate and workspace shown in the dashboard. It does not create a voice session, receive caller audio, or act as the call transport.

It is useful after a voice-support team has approved a follow-up process. For example, a caller may report a damaged delivery over a Realtime voice agent. The voice application can create a case or route it to a human. Separately, a support lead can use the eesel dashboard while an engineer, CI script, Claude Code, Codex, or Cursor checks whether the helpdesk teammate has the approved photo-evidence guide and instruction to avoid promising a replacement.

With Node.js 18.17 or newer, confirm the workspace and scope each inspection to the intended teammate:

npx @eesel/cli login
npx @eesel/cli whoami
npx @eesel/cli agents

TARGET_EESEL_AGENT="paste-agent-id-or-name-here"
npx @eesel/cli status --agent "$TARGET_EESEL_AGENT"
npx @eesel/cli integrations --agent "$TARGET_EESEL_AGENT"
npx @eesel/cli instructions --agent "$TARGET_EESEL_AGENT"

Ask the coding agent for a read-only report: which support sources are connected, whether their content has downloaded, and which instruction governs a post-call photo request and human handoff. The report must not connect an account, change permissions, upload files, or enable an automation as a side effect.

If the support owner approves a new follow-up guide or instruction, use --dry-run on the relevant write command before sending it. Before a test chat, review the selected teammate's configured actions and permissions. Disable or contain actions outside the test scope; a held-action queue does not protect against an action that can run without a hold.

Then run a fictional post-call case. This tests support follow-up, not audio latency or a Realtime connection:

npx @eesel/cli new --name "voice-follow-up damage claim" --agent "$TARGET_EESEL_AGENT"
npx @eesel/cli chat "A caller reports a damaged delivery but the photo is unclear. Explain what evidence to send, do not promise a replacement, and state when a human must decide." --agent "$TARGET_EESEL_AGENT"
npx @eesel/cli approvals --agent "$TARGET_EESEL_AGENT"
npx @eesel/cli activity --agent "$TARGET_EESEL_AGENT"

Review the reply for its evidence request, policy boundary, and handoff. If the teammate creates a held action, inspect it before a human approves it. After a controlled rollout, check the ticket, any caller follow-up, and the order or warranty-system result. A terminal reply or activity entry does not prove the caller received the right resolution.

The CLI's dashboard-management commands are free; chat is billed work. JSON output makes an engineer or coding agent's assessment reviewable, but it does not replace caller consent, approval, representative voice testing, or external outcome checks.

Choose the component that owns the work

Use OpenAI Realtime when you need to build a live voice, translation, or transcription experience and can own its transport, authentication, tools, safety, evaluation, and operations. Start with the session type that matches what the caller needs.

Use eesel when the immediate job is operating a helpdesk teammate with approved company knowledge and support workflows. Its CLI lets technical users make inspection and bounded configuration repeatable on the same teammate support owners see in the dashboard. Keep the line clear: Realtime handles the live voice interaction; the support workflow still needs an approved policy and verified result.

Test voice-support follow-up with eesel

Try eesel if you want to inspect and test a helpdesk teammate's post-call guidance before callers rely on a broader voice workflow. Start with approved sources and a representative set of follow-up cases, contain unrelated actions, test the handoff, and verify the ticket and external result before expanding access. Check eesel pricing separately from OpenAI Realtime costs because they are different products and units of work.

The eesel Activity view shows support activity with approved, rejected, and pending filters.