Loading

Using different models in Elastic Agent Builder

Serverless Elasticsearch Preview Serverless Observability Unavailable Serverless Security Unavailable Stack Preview 9.2.0

Elastic Agent Builder uses large language models (LLMs) to power agent reasoning and decision-making. By default, agents use the Elastic Managed LLM, but you can configure other models through Kibana connectors.

By default, Elastic Agent Builder uses the Elastic Managed LLM connector running on the Elastic Inference Service Serverless Preview ECH Preview 9.2.0 .

This managed service requires zero setup and no additional API key management.

Learn more about the Elastic Managed LLM connector and pricing.

By default, Elastic Agent Builder uses the Elastic Managed LLM. To use a different model, select a configured connector and set it as the default.

  1. Search for GenAI Settings in the global search field
  2. Select your preferred connector from the Default AI Connector dropdown
  3. Save your changes
  1. Find connectors under Alerts and Insights / Connectors in the global search bar
  2. Select Create Connector and select your model provider
  3. Configure the connector with your API credentials and preferred model
  4. Search for GenAI Settings in the global search field
  5. Select your new connector from the Default AI Connector dropdown under Custom connectors
  6. Save your changes

For detailed instructions on creating connectors, refer to Connectors.

Learn more about preconfigured connectors.

You can connect a locally hosted LLM to Elastic using the OpenAI connector. This requires your local LLM to be compatible with the OpenAI API format.

Refer to the OpenAI connector documentation for detailed setup instructions.

For programmatic access to connector management, refer to the Connectors API documentation.

Elastic Agent Builder requires models with strong reasoning and tool-calling capabilities. State-of-the-art models perform significantly better than smaller or older models.

The following models are known to work well with Elastic Agent Builder:

  • OpenAI: GPT-4.1, GPT-4o
  • Anthropic: Claude Sonnet 4.5, Claude Sonnet 4, Claude Sonnet 3.7
  • Google: Gemini 2.5 Pro

Agent Builder relies on advanced LLM capabilities including:

  • Function calling: Models must accurately select appropriate tools and construct valid parameters from natural language requests
  • Multi-step reasoning: Agents need to plan, execute, and adapt based on tool results across multiple iterations
  • Structured output: Models must produce properly formatted responses that the agent framework can parse

Smaller or less capable models may produce errors like:

Error: Invalid function call syntax
		
Error executing agent: No tool calls found in the response.

While any chat-completion-compatible connector can technically be configured, we strongly recommend using state-of-the-art models for reliable agent performance.

Note

GPT-4o-mini and similar smaller models are not recommended for Elastic Agent Builder as they lack the necessary capabilities for reliable agent workflows.