Simple SQL Custom Agent

This is my second post on Medium, and I wanted to make it more practical rather than theoretical. In this blog, I will guide you through creating a simple SQL custom agent that performs all CRUD operations using an LLM. For the implementation part, I will be using the AzureAIOpenAIApiChatModel Chat Model from Azure.

Overview

SQL Custom Agent is a conversational AI assistant that can read and write a SQLite database using nothing but plain English. You ask it “Add a user named Alice, age 30, email alice@example.com” and it figures out which tool to call, calls it, and confirms the result — no hand-written SQL required.

Get Sujaykarumuri’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

The key design choice is that the LLM never directly touches the database. Instead, it decides what to do, and a dedicated MCP server does the actual work. This keeps the AI reasoning layer and the data layer completely separate.

How It Works — Step by Step

1. User sends a message via the CLI (e.g. _”List all users older than 25"_).
2. SQLAgent appends the message to the conversation history and calls `_run_tool_loop()`.
3. LLMClient receives the full conversation plus the tool schemas discovered from the MCP server. It calls `bind_tools()` on the Azure OpenAI model via LangChain and invokes the model asynchronously (`ainvoke`).
4. The model returns either: A tool call (e.g. `read_records(table=”users”, filters={“age”: 25})`) Or a final text answer (no more tools needed)
5. If it’s a tool call, MCPClient forwards it to the MCP server over a stdio pipe using JSON-RPC.
6. The MCP server delegates to the Database layer, which runs the parameterized SQL against the SQLite file and returns the result.
7. The result is appended to the conversation as a `tool` message and the loop repeats from step 3.
8. Once the model produces a plain text response, it is returned to the user.

SetUp and Running

1. Install Dependencies

python -m venv .venv
# Windows
.venv\Scripts\activate
# macOS/Linux
source .venv/bin/activate
pip install -r requirements.txt

2. Configure Environment

OPENAI_API_KEY=your-azure-openai-key
endpoint=https://your-resource.openai.azure.com/
model=gpt-4o
DATABASE_PATH=data/users.db

3. Run the agent (The agent automatically spawns the MCP server as a subprocess — no need to start it separately.)

python -m agent.agent

NOTE: This is a very basic agent that doesn’t have any Guardrails, RBAC to the DB, etc. For the next phase of releases, these will be included along with the next possible features, which can provide more power to the agent.

You can find the source code in this GitHub Repository : https://github.com/karumuri-sujay/SQLCustomAgent.

Thank you for checking on this blog.