Introduction

With the ongoing developments in the data and AI industry, the hype around AI agents has no signs of slowing down. Jensen Huang, Nvidia CEO, is a strong proponent of AI agents, envisioning a multi-trillion-dollar opportunity, where agents can perform tasks with a high degree of autonomy and revolutionise how people work and how businesses operate.

So, in this article we would like to discuss what exactly AI agents are, what are their main components, how they interact together and the basics on how to build one.

What is an AI agent?

An AI agent is a system or software program that can autonomously interact with its environment, by performing certain actions on behalf of humans to achieve a given goal.

Although agents are developed for a wide variety of use cases, we can agree that most of them will have at least the components shown in the figure below.

Reasoning

The core of the AI agent, thanks to their knowledge base and reasoning capability, agents can “think” and continuously engage in problem-solving to achieve a given objective.

LLMs power this by supporting:

• Planning: capability of breaking down complex problems into sequential smaller manageable tasks, enabling the use of specific tools.
• Reflecting: capability to assess the outcomes of given actions and refine their strategies based on their results.

Tools

Their name explains itself; tools are a collection of functions or programmed capabilities that expand the abilities of LLMs.

LLMs with “function calling” or “tool calling” functionalities can be made aware that they have access to a specific list of tools and they can call them when they require it. This calling mechanism comes as a “signalling” that a call to a function is needed rather than the actual LLM executing the tool itself.

Some examples of possible tools are:

• Database access, to retrieve or update information from a specific schema or set of tables.
• Web browsing, to search something on the web when the user needs it.
• Code interpreter, to run code generated by the LLM.

Memory

Agents are initialised with a context defining their initial state and goal. Throughout interactions with other systems or humans, they rely on memory to store the conversation history and/or make checkpoints periodically.

This allows the LLMs to behave consistently according to user needs.

An example of an agent

AI agents for customer service are an example of such applications. These are agents with access to customers’ data (orders, payments, service planning, etc) and can help solve most of customers’ questions around specific orders like: cancelations, refunds, delivery dates replanning, etc.

In this scenario, reasoning helps the agent understand where the customer needs help and what are the steps to do so, tools allow the agents to interact with databases, fetch information about an order and update information like order statuses. Lastly, memory allows the agent to keep track of the conversation with the customer, ensuring continuity.

Building your own AI agent

From the introduction, we can conclude that building an AI agent involves several components and considerations on how they interact with each other.

In this section, we will dive into these components and see how they come together. At last, we will finish with an example on how to program a simplistic AI agent.

Considerations before starting

1. The Core LLM model

Selecting the core LLM is an important step, here are some important factors to consider:

• Performance: Accuracy and human-like interaction are important. Larger LLMs have more layers in their neural networks, so typically they offer better abstractions and more specialised knowledge on multiple areas, decreasing the chances that the model will hallucinate. However, this comes at the expense of increased compute and memory requirements, hence finding the right balance is essential.
• Latency: Larger LLMs have larger latencies as their neural networks are composed of more layers, more compute is required.
• Cost: Larger LLMs are of course more compute and memory greedy, hence more expensive to run.
• Proprietary vs open source: The big debate “Do we rely on proprietary technology where we have little control and understanding, or do we go open source where we have full control, but also more specialised skills are needed to make it work?”

Here a comparison table of both approaches focusing on key elements:

This is a strategic decision, not just technical. It affects costs, infrastructure, compliance, development speed, and long-term autonomy.

2. Features required in the core LLM:

For building agents, the core LLM needs some characteristics:

• Instruction following: An instruct model is a fine-tuned base LLM that excels at following instructions, staying on the topic and responding in a desired format or style.
• Tool calling: The indispensable capability if you are trying to build an AI agent. This enables LLM models to be aware that there is a set of tools available and their specifications. So, based on the context or user requests, the LLM can decide if a call to a specific tool is needed or not. Once the LLM calls a tool, it is up to the engineer to map that call to the specific function, including the parameters that the LLM sent.
• Context window support: Although most recent models have large enough context windows, this could limit the ability to keep relevant information for the agent and that some summarisation or checkpoint might be needed at some point if we expect the interactive sessions to last for long.

3. Understanding Model-Specific competition schemas and APIs

LLMs serve as the brain of the agent, but how do we interact and send instructions to them? Before moving forward let’s understand what LLM APIs are and how their completion schemas work.

LLM API

An LLM API is the interface that allows an AI agent to communicate with the language model. It defines how requests are sent and how responses are received.

Most LLM providers expose their APIs over HTTPS, using REST or JSON-based endpoints. SDKs are also widely used, as they wrap the API and create an easy-to-use abstraction for different programming languages.

Here a list of some SDKs for various LLM providers:

• Mistral
• OpenAI
• Google
• Anthropic

As we can see, most of them are proposed in python, as well as TypeScript, JavaScript and Go.

Completion Schema

The completion schema is the structure of the input we send to the LLM and the output it returns. It is important that the AI agent sends well formatted requests and correctly interprets the responses. Schemas vary by provider, but generally they can be seen as a list of dictionaries with the role and content as main keys.

The most common roles are:

• System: used to provide high-level instructions and context that will guide the model behaviour trough the interactions.
• User: used to provide the (human) user input or query
• Assistant: used to provide the response of the LLM to the previous query or interaction

For better understanding here we have an example of a sequence of messages, exchanged between a user and an AI assistant:

# Example of sequence of messages exchanged between a user and the AI assistant
[
  {
    "role": "system",
    "content": "Act as an human assistant, specializing in providing basic travel advises for European destinations. Provide  just location suggestions based on interests.\n\nYour name is Alice. Always ask the name of the person you are interacting with if it is not given at the begining of the conversation. To keep it more human keep responses short, so the human has the impression that it is chatting with another human. "
  },
  {
    "role": "user",
    "content": "Hi there"
  },
  {
    "role": "assistant",
    "content": "Hi, I'm Alice. What's your name?"
  },
  {
    "role": "user",
    "content": "I am Oscar"
  },
  {
    "role": "assistant",
    "content": "Nice to meet you, Oscar! What kind of experiences do you enjoy while traveling? Are you into vibrant cities, historical sites, coastal escapes, or something else?"
  },
  {
    "role": "user",
    "content": "I always found the French Alps fascinating!"
  }
]

Notice that the first message starts with a system role, containing the initial prompt. And then it follows an alternation between user and assistant messages.

Choosing the core LLM

As new models are released every month, here are some resources to keep track of their benchmark:

• LLM arena
• Vellum
• Berkeley Function Calling Leaderboard

Based on the points above and the leaderboards ranking as of May 2025, we decided to go for o3-mini deployed in an Azure OpenAI endpoint for the core of our agentic solution. You can refer to its python SDK and completion schema for more details.

Calling the API via the SDK

The SDKs contain an abstraction that allows us to authenticate with the LLM provider, in this case we are using the openai library to create a client and then we use the client to ask a single reply from the LLM.

Note that in the messages section, we are passing the system prompt as well as the initial request of the user, following the completion schema previously discussed:

import openai

ai_client = openai.AzureOpenAI(
  api_version="API_VERSION_HERE",
  azure_endpoint="AZURE_OPENAI_ENDPOINT_HERE",
  api_key="API_KEY_HERE"
)
response = ai_client.chat.completions.create(
  messages=[
      {
        "role": "system"
        "content": "You are a helpful assistant"
      },
      {
        "role": "user"
        "content": "Hi there, what is the capital of France"
      }
    ],
  model="DEPLOYMENT_NAME_HERE",
  max_completion_tokens=1000)

print(f"AI assitant: {response.choices[0].message.content}")

This will send a query to the o3-mini LLM being served in our Azure OpenAI endpoint. The response is then returned and stored on a variable, from which we still need to get its content.

Once that is done, the output of this execution should look like this:

AI assitant: The capital of France is Paris.

Creating tools

Now let’s create a tool that allows us to list blobs in an Azure container. We will use a container client to authenticate with Azure and list the blobs in it, here a method that does exactly that:

# Example of function:
from typing import Optional
import logging
from azure.storage.blob import ContainerClient

logger = logging.Logger("my-dummy-logger", level=logging.DEBUG)

def list_blob_files(container_name: Optional[str] = None) -> dict:

  if container_name in [None, "", "{}"]:
    container_name = "my-default-container"  # Replace with your default container name

  if not isinstance(container_name, str):
    container_name = str(container_name)

  logger.debug(f"Listing blobs in container: '{container_name}'")

  container_client = ContainerClient.from_connection_string(
    conn_str="YOUR_AZURE_STORAGE_CONNECTION_STRING",  # Replace with your connection string
    container_name=container_name
  )

  blob_list = container_client.list_blobs()
  try:
    files = [blob.name for blob in blob_list]
    logger.debug(f"Found {len(files)} files in container.")
    status = "success"
  except azure_exceptions.HttpResponseError as e:
    logger.error(f"Error listing blobs from container '{container_name}': {e}")
    status = "error"
    files = []

  return {"container_name": container_name, "status": status, "files": files}

Now that we have the function/tool, we need to make the model aware that it can use it. For that a description of what the tool does and which parameters it takes is needed.

These descriptions follow a format that is again specific to the LLM provider.

Here we have the description of our blob listing tool, following the OpenAI tool calling reference:

# Tool description for OpenAI
tools_description = [
  {
    "type": "function",
    "function": {
      "name": "list_blob_files",
      "description":
        "List all files in a specified Azure Blob Storage container. If no container is " \
        "specified, the default container will be used.",
      "parameters": {
        "type": "object",
        "properties": {
          "container_name": {
            "type": "string",
            "description": "(Optional) The name of the Azure Blob Storage container."
          }
        },
        #"required": ["container_name"]
      }
    }
  }
]

This description includes the tool/function name, a description of what it does, the parameters it takes and a specification of which of them are mandatory.

Once the description is ready, to make the model aware of its existence, an additional tool description parameter is needed on the API call:

import openai

ai_client = openai.AzureOpenAI(
  api_version="API_VERSION_HERE",
  azure_endpoint="AZURE_OPENAI_ENDPOINT_HERE",
  api_key="API_KEY_HERE"
)

response = ai_client.chat.completions.create(
  messages=[
    {
      "role": "system",
      "content": "Act as an human assistant, specializing in providing basic travel advises for European destinations. Provide  just location suggestions based on interests.\n\nYour name is Alice. Always ask the name of the person you are interacting with if it is not given at the begining of the conversation. To keep it more human keep responses short, so the human has the impression that it is chatting with another human. "
    },
    {
      "role": "user",
      "content": "I want to check the files on the test container"
    }
  ],
  tools=tools_description,
  model="DEPLOYMENT_NAME_HERE",
  max_completion_tokens=1000
  )

Now that the LLM is aware that it has access to a tool, it is up to us to check if it decided to use it or not. So, a check if a tool call has been made is needed and if so, a “hard coded” mapping to it.

Once the tool function has been executed, we add its return to the conversation between the LLM and the user with the role field set as tool on it.

Finally, a second call to the LLM is made to verbalise the function’s return.

The following competition schema helps us to understand what we are trying to achieve. On it, the user requests which are the files on the test container, and the LLM decides to make a call to the tool list_blob_files, which returns the list of blobs. We added that return to the conversation history and then we made another call to the LLM to interpret that return:

[
  {
    "role": "system",
    "content": "You are a helpful assistant."
  },
  {
    "role": "user",
    "content": "Hi there, I am Oscar"
  },
  {
    "role": "assistant",
    "content": "Hi Oscar, welcome! How can I help you today?"
  },
  {
    "role": "user",
    "content": "What are the files on the test container?"
  },
  {
    "role": "assistant",
    "content": null,
    "tool_calls": [
      {
        "id": "call_1zPM5CENscsLuEuRSg8nDCvN",
        "type": "function",
        "function": {
          "name": "list_blob_files",
          "arguments": "{\"container_name\": \"test\"}"
        }
      }
    ]
  },
  {
    "role": "tool",
    "content": "{\"container_name\": \"test\", \"status\": \"success\", \"files\": [\"AI agents article.docx\", \"Goolge-whitepaper_Prompt Engineering_v4.pdf\", \"a-practical-guide-to-building-agents.pdf\"]}",
    "tool_call_id": "call_1zPM5CENscsLuEuRSg8nDCvN"
  },
  {
    "role": "assistant",
    "content": "The files in the \"test\" container are as follows:\n1. AI agents article.docx\n2. Goolge-whitepaper_Prompt Engineering_v4.pdf\n3. a-practical-guide-to-building-agents.pdf\n\nLet me know if you need help with anything else!"
  },
  {
    "role": "user",
    "content": "Which of these are word files?"
  },
  {
    "role": "assistant",
    "content": "The Word file in the list is \"AI agents article.docx\"."
  },
  {
    "role": "user",
    "content": "Great thanks!"
  },
  {
    "role": "assistant",
    "content": "You're welcome, Oscar! Let me know if you have any other questions or need any additional help."
  }
]

Bringing all together

Before going into the code, let’s summarise the key aspects:

1. The AI agent relies on the core LLM for reasoning.
2. The LLM is aware that a tool is available thanks to the tool description passed during the API call.
3. When a user makes a query to the LLM, the LLM decides if a tool call is required or not.
4. If so, it signals that a tool call is needed. This signalling is then picked by the agent code and executes the specific function associated with it. The return of the function is then added to the conversation history and a second call to the LLM is needed to summarise the output of the function.
5. If a tool call is not required, the LLM can immediately reply to the user.

The full runnable code is on this github repository. Check the README.md to add your own settings.yaml file before executing it.

Here an example on how it runs:

Some additional notes

In this article we described some key factors to consider while creating an AI agent.

In our example we relied on the OpenAI interface, however other LLM providers might have different SDKs and potentially different completion schemas, which can introduce complexity when switching them. LangChain is a framework that addresses this challenge by providing abstractions to standardise the interactions with LLMs across various LLM providers, simplifying tool calling and API management, here more info about it.

Our example implemented a single-agent architecture, where a single LLM handles the reasoning, tool selection and the planning. While straightforward for simpler systems, it might fall short when adding more tools to it, as the LLM might have difficulties selecting the proper tool. For these scenarios, multi-agent architectures are a good alternative as they distribute responsibilities to multiple specialised agents, where each one handles specialised and smaller tasks.

Another emerging concept in AI agents is Model Context Protocol (MCP), it refers to a standardised way for AI agents and tools to share context across their interactions. In multi-agents/tools systems, MCP helps ensuring that all components such as reasoning agents, planning agents and external tools, operate with a consistent and updated understanding about the environment, goals and user preferences. While still under development, MCP is becoming more relevant for building more reliable AI systems.

Finally, as any other software product, testing your AI agent should be part of the developing process. Here a blog with some ideas of what to include on it.

Conclusion

Building an AI agent is a process that combines reasoning, tool integration and memory management to create systems capable of addressing complex tasks. Pay attention to selecting the core LLM, balancing performance, costs and latency. And make sure it has the features needed to follow instructions and perform tool calling.

As the AI landscape evolves fast, frameworks like LangChain and multi-agent architectures simplify development and enhance scalability.