Delegation and tools in GPT-Live

Configure Responses delegation

Add this delegation configuration when creating your Live session. Choose the Responses model independently of the voice model:

1
2
3
4
5
6
7
8
9
10
11/** @type {import("openai/resources/live/live").SessionConfig} */
export const session = {
  model: "gpt-live-1",
  delegation: {
    type: "responses",
    responses: {
      model: "gpt-5.6-terra",
      instructions: "[Your backend prompt]",
    },
  },
};

1
2
3
4
5
6
7
8
9
10
11
12from openai.types.live.session_config_param import SessionConfigParam

session: SessionConfigParam = {
    "model": "gpt-live-1",
    "delegation": {
        "type": "responses",
        "responses": {
            "model": "gpt-5.6-terra",
            "instructions": "[Your backend prompt]",
        },
    },
}

Start with GPT-5.6 Terra, or try GPT-5.6 Luna for cost-sensitive workloads. Compare answer quality and latency on your tasks before choosing a backend model.

Register supported tools in delegation.responses.tools. Use delegation.responses.tool_choice to control which tools the backend can use: "auto" lets it choose, "required" requires a tool call, and "none" prevents one. You can also select a named function. Set delegation.responses.parallel_tool_calls to true to allow independent lookups together, or false when calls must run sequentially. Your application still executes its custom functions and enforces dependencies and approvals. These settings do not force the live model to delegate.

The Responses configuration requires a backend model at creation. It supports function definitions and web_search entries in tools. It also exposes max_output_tokens (at least 16 when set), service_tier, and the reasoning and text settings supported by the selected backend model. See Reduce backend latency for settings you can tune.

If Fast mode is available for your model and project, consider it for latency-sensitive calls. For GPT-Live, select it with delegation.responses.service_tier: "priority".

As the conversation changes, send session.update with changes in session.delegation.responses to update the backend model, instructions, available tools, tool_choice, or other supported settings without starting a new Live session. Omitted settings retain their values. Setting delegation to null selects client mode and cannot reset a running Responses session; switching modes fails with immutable_field_update.

These settings use familiar Responses concepts, but Live supports a subset of the standalone Responses API. Live supplies conversation context and initiates delegated work. Configure the backend through the session; the Live response.create command uses that configuration and does not accept a standalone Responses request body.

Steer the live conversation from your application

Responses delegation manages the backend workflow, but your application can still send context directly to the GPT-Live model. If you monitor the call through a sideband WebSocket or the main event connection, you can use session.instructions.append, session.thinking.append, or session.commentary.append with delegation_id: null. For example, a transcript-based guardrail can append an instruction to redirect the conversation. This steers the live model; it does not change the Responses backend prompt or cancel work already in progress.

Handle Responses delegation

For Responses-backed work, session.delegation.created has target: "responses" and a response_id. Subsequent Responses events arrive inside a response.event envelope:

1234567891011121314{
  "type": "response.event",
  "event_id": "event_response_1",
  "delegation_id": "item_9tA2cB6n2V8c4X1z7Q5r9",
  "event": {
    "type": "response.output_text.delta",
    "sequence_number": 4,
    "item_id": "msg_123",
    "output_index": 0,
    "content_index": 0,
    "delta": "The forecast is",
    "logprobs": []
  }
}

Dispatch on envelope.event.type and preserve the outer delegation_id. Do not handle every top-level response.* value as an unwrapped Responses event. Tolerate additional nested Responses lifecycle events.

Live speech and delegated work continue independently. A completed backend response does not itself mean the user heard the answer. Use the Live output transcript and audio for the spoken part of the interaction.

Complete a client-actionable function call

Read completed function calls from nested response.output_item.done events. The finished function item contains call_id, name, and arguments; an arguments-done event alone is not sufficient to identify the call.

Track the response ID from nested response.created alongside the outer delegation_id, and collect that response’s function calls from response.output_item.done. Forwarded lifecycle snapshots deliberately contain response.output: [], including at response.completed; their tools array is empty, instructions is null, and input is omitted. An empty terminal output list does not mean there are no pending function calls. Use the collected calls to determine which results must be submitted before continuing.

After executing the authorized operation, append the result as a Responses item:

1
2
3
4
5
6
7
8
9
10
11
12
13
14/**
 * @param {import("openai/resources/live/ws").LiveWS | import("openai/resources/live/sideband/ws").SidebandWS} connection
 */
export function sendUpdate(connection) {
  connection.send({
    type: "response.item.create",
    event_id: "tool_result_1",
    item: {
      type: "function_call_output",
      call_id: "call_123",
      output: '{"status":"confirmed","order_id":"order_123"}',
    },
  });
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17from openai.resources.live.live import AsyncLiveConnection
from openai.resources.live.sideband import AsyncSidebandConnection
from openai.types.responses.response_input_item_param import ResponseInputItemParam


async def send_update(
    connection: AsyncLiveConnection | AsyncSidebandConnection,
) -> None:
    item: ResponseInputItemParam = {
        "type": "function_call_output",
        "call_id": "call_123",
        "output": '{"status":"confirmed","order_id":"order_123"}',
    }
    await connection.response.item.create(
        event_id="tool_result_1",
        item=item,
    )

Then explicitly continue the response:

1
2
3
4
5
6
7
8
9/**
 * @param {import("openai/resources/live/ws").LiveWS | import("openai/resources/live/sideband/ws").SidebandWS} connection
 */
export function sendUpdate(connection) {
  connection.send({
    type: "response.create",
    event_id: "continue_1",
  });
}

1
2
3
4
5
6
7
8
9
10from openai.resources.live.live import AsyncLiveConnection
from openai.resources.live.sideband import AsyncSidebandConnection


async def send_update(
    connection: AsyncLiveConnection | AsyncSidebandConnection,
) -> None:
    await connection.response.create(
        event_id="continue_1",
    )

Submit every required result for the pending tool calls before continuing. Appending a function result does not automatically continue the response. response.item.create has no standalone success acknowledgment; keep processing errors and the subsequent nested response lifecycle.

response.create is a Live command for creating or continuing delegated Responses work, using the session’s configured backend. Do not attach a Responses API creation body, backend model override, or delegation_id to this event. Both commands require Responses delegation.

Configure client delegation

Set delegation when creating your Live session:

1
2
3
4
5
6
7/** @type {import("openai/resources/live/live").SessionConfig} */
export const session = {
  model: "gpt-live-1",
  delegation: {
    type: "client",
  },
};

1
2
3from openai.types.live.session_config_param import SessionConfigParam

session: SessionConfigParam = {"model": "gpt-live-1", "delegation": {"type": "client"}}

This selects client delegation for the session. Configure the backend separately: your application chooses its model or service, instructions, tools, and how to route work. If you use the Responses API for that backend, set its model and tools in your own Responses requests. The Live session does not configure or run those backend tools.

When GPT-Live requests help, your application builds the backend request from conversation and application context, runs the work, and decides which results to send back. Enforce permissions and required confirmations before executing your tools. Retain the full conversation history in your application so you can provide the relevant context for each backend request.

Keep the conversation context in your application

For client delegation, collect transcripts and keep the current task state yourself.

Listen for session.input_transcript.delta and session.output_transcript.delta. These events contain transcript text in delta, along with start_ms and end_ms timestamps. Keep enough history to understand short replies such as “yes,” corrections such as “Thursday, not Friday,” and details supplied earlier. A transcript fragment is not a complete user turn, and transcripts may contain mistakes.

The separate session.delegation.created event contains an offset_ms timestamp and delegation metadata, including delegation.id and delegation.target. It does not contain the user’s utterance or task text. Use the transcript events and application state to work out what the user wants. Save delegation.id so you can match updates to that request.

Keep long records and full tool output in the backend. If you create a replacement session, restore the relevant context from your application and check which actions already ran before repeating any work.

Receive a client delegation

session.delegation.created identifies a delegation:

12345678910{
  "type": "session.delegation.created",
  "event_id": "event_delegation",
  "offset_ms": 1000,
  "delegation": {
    "id": "item_9tA2bF3h7K9m2P5q8R1s4",
    "type": "delegation",
    "target": "client"
  }
}

Read event.delegation.id. The delegation object contains metadata, not task text. Maintain the transcript and application context needed by your own delegated-work handler. Current IDs have an item_ prefix, as illustrated here; treat the full ID as opaque and return it unchanged rather than constructing or parsing one.

Return a result using that ID:

1
2
3
4
5
6
7
8
9
10
11/**
 * @param {import("openai/resources/live/ws").LiveWS | import("openai/resources/live/sideband/ws").SidebandWS} connection
 */
export function sendUpdate(connection) {
  connection.send({
    type: "session.commentary.append",
    event_id: "result_123",
    delegation_id: "item_9tA2bF3h7K9m2P5q8R1s4",
    content: "The order shipped today and should arrive tomorrow.",
  });
}

1
2
3
4
5
6
7
8
9
10
11
12from openai.resources.live.live import AsyncLiveConnection
from openai.resources.live.sideband import AsyncSidebandConnection


async def send_update(
    connection: AsyncLiveConnection | AsyncSidebandConnection,
) -> None:
    await connection.session.commentary.append(
        event_id="result_123",
        delegation_id="item_9tA2bF3h7K9m2P5q8R1s4",
        content="The order shipped today and should arrive tomorrow.",
    )

Use session.thinking.append to add information to the model’s internal reasoning without speaking it aloud when appended. Use session.commentary.append for a result the model should speak aloud; the model is trained to paraphrase the appended text. All appends contain a plain string and require delegation_id, including when its value is null. A non-null ID must name a known client delegation.

Repeated result appends can continue the same client delegation. An appended acknowledgment arrives after estimated context injection; it is not proof that the model has consumed or spoken the result, or that an external action succeeded.

With Responses delegation, queue a user message for the backend:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19/**
 * @param {import("openai/resources/live/ws").LiveWS | import("openai/resources/live/sideband/ws").SidebandWS} connection
 */
export function sendUpdate(connection) {
  connection.send({
    type: "response.item.create",
    event_id: "typed_order_number",
    item: {
      type: "message",
      role: "user",
      content: [
        {
          type: "input_text",
          text: "My order number is A0042.",
        },
      ],
    },
  });
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17from openai.resources.live.live import AsyncLiveConnection
from openai.resources.live.sideband import AsyncSidebandConnection
from openai.types.responses.response_input_item_param import ResponseInputItemParam


async def send_update(
    connection: AsyncLiveConnection | AsyncSidebandConnection,
) -> None:
    item: ResponseInputItemParam = {
        "type": "message",
        "role": "user",
        "content": [{"type": "input_text", "text": "My order number is A0042."}],
    }
    await connection.response.item.create(
        event_id="typed_order_number",
        item=item,
    )

Send response.create when ready to run or continue the backend. If it is waiting for function results, return all required results first. Queuing text does not itself cancel work already running.

With client delegation, send the typed value directly to the backend that handles the conversation. If it corrects a running task, update that task instead of starting the same work again. You can mirror a short factual summary into the live session with session.thinking.append, or use session.commentary.append for a result the user should hear.

With Responses delegation, configure a vision-capable backend model. Queue a supported Responses image input item with response.item.create, then send response.create to run or resume backend work. Return all required pending function results before continuing. See Handle Responses delegation.

With client delegation, send visual input to the backend that handles delegated requests, alongside the relevant conversation and application state. Return concise findings using the client result flow.

Responses delegation

Live manages persistent WebSocket connections to Responses, prepares the connection and known request configuration in advance, and reuses prior response state when available. You do not need to implement those steps for the hosted backend. Reuse depends on the active connection and compatible state; it does not guarantee a cache hit or a specific latency.

Tune the backend through delegation.responses:

• model: choose the model that handles reasoning and tool selection independently of the voice model.
• reasoning.effort: balance reasoning time and task quality using values supported by that model.
• service_tier: use auto, default, flex, or priority, subject to model support and project access. auto follows the project’s configuration. Evaluate the performance and cost of the tier you choose.

Update supported settings during the session with session.update. Your custom tools still run in your application, so slow service calls, queues, and tool-result buffering can delay the answer even when Live manages the Responses connection. Return each required tool result promptly and continue the backend response.

Client delegation

Your application owns the path from delegation receipt to returning a result. Prepare that path while the voice session runs:

• Reuse backend connections. Keep the API client and its connection pool alive across delegations. For repeated Responses calls, consider a persistent Responses WebSocket.
• Prepare known configuration. Initialize instructions, tools, and connections before the first request needs them. Responses WebSocket mode also supports warming up known request state before generation; follow its setup guidance.
• Stream useful results. Return coherent, verified chunks with session.commentary.append. Use session.thinking.append for quiet progress. Preserve the client delegation ID and the 500-token limit per append. Keep private reasoning in the backend and confirm actions before announcing success.
• Keep reusable input stable. Preserve instructions, tool definitions and ordering, and unchanged history prefixes. Append new information after reusable content when your backend supports caching and continuation.
• Avoid unnecessary buffering. Forward a useful result as soon as it is ready. Buffer only enough to classify the output and form a coherent chunk. Prefer structured phase metadata; if you use text prefixes to distinguish progress from results, wait for the complete prefix before forwarding text.

Measure the first useful spoken answer when comparing this path with Responses delegation.