The Model Context Protocol (MCP) has become an important piece of infrastructure in the AI ecosystem. It is often described as “USB-C for AI”.

That’s a useful analogy, but it doesn’t answer the question most engineers actually have:

Why do we need MCP when we already have APIs, SDKs, and function calling?

The short answer is:

MCP standardizes the integration boundary between AI applications and external tools and services.

MCP becomes valuable when the same tools and services need to be consumed by multiple AI applications.

The Problem: Connecting AI to the Real World

An LLM is good at reasoning and generating text, but useful AI applications need access to systems outside the model. For example:

• Databases
• GitHub
• Slack
• Jira
• CRMs
• Internal APIs
• Documentation
• Business Workflows

The Need for an MCP Server

A traditional AI application might look like this:

And there’s nothing wrong with this architecture. If you’re building one application and you control all of these systems, direct integrations may be the simplest solution.

The problem appears when multiple AI applications need access to the same capabilities.

Without a common protocol, every application needs its own integration, making the overall system harder to manage.

MCP addresses this by introducing a standardized protocol between the AI application and the capability provider.

The key value is therefore interoperability.

What Is MCP?

MCP is a protocol that standardizes how AI applications interact with external tools and services.

An MCP server can expose things such as:

• Tools — actions the AI can invoke
• Resources — contextual information the AI can retrieve
• Prompts — reusable prompt templates or workflows

For example, a GitHub MCP server might expose capabilities such as:

search_repositories()
get_issue()
search_pull_requests()
create_issue()

A database-oriented MCP server might expose:

search_customer()
get_customer_orders()
get_customer_profile()

The important part is that the AI application doesn’t need to know how these operations are implemented internally. The server could be backed by:

REST API
GraphQL
gRPC
SQL
SDK
Internal service

MCP vs APIs

A common misconception is that MCP is intended to replace REST, gRPC, GraphQL, or other application APIs, but it isn’t.

Suppose your company already has:

GET  /customers/{id}
POST /customers/search
POST /tickets

Those APIs can continue to exist and can be put behind an MCP server. So the MCP server doesn’t replace the APIs, it utilizes them.

Do I Need MCP for My Own AI Agent?

Well, it depends. If you have only one AI agent that you developed and there are no other agents involved, you don’t necessarily need an MCP server. You can integrate your internal systems and services directly with the agent.

However, if you have multiple AI agents, including your own agents as well as third-party agents like Claude, ChatGPT, Gemini, etc., and all of them need access to your tools and services, then an MCP server would make sense. It provides a standardized way for different agents to interact with your tools and services.

The Real Value of MCP

The biggest value of MCP isn’t that it allows an LLM to call a function. We could already do that. The bigger change is where the integration boundary lives.

Without MCP, integrations often live inside individual AI applications:

With MCP, capabilities can be exposed independently:

That enables a capability provider to build an integration once and make it consumable by multiple compatible AI applications.

Conclusion

MCP is not a replacement for APIs. And you don’t automatically need it just because you’re building an AI application.

If you’re building a single AI application that talks to your own backend, direct APIs and function calling may be all you need.

But if you’re building a tool or service that should be consumed by multiple AI applications, in that case MCP becomes significantly more valuable.

My first instinct when I started designing a Text-to-SQL chatbot was fairly straightforward: take a user’s question, give the LLM enough database context, ask it to generate SQL, execute the query, and return the result.

That sounds reasonable. It is also nowhere near enough for a production-oriented system.

As I worked through the design, I realized that SQL generation was actually only one part of the problem. The harder questions were around Context, Schema selection, Follow-Up Questions, Failed SQL, Safe Execution, Model Providers, Observability, and deciding which parts of the workflow actually needed to run for every request.

I also deliberately decided not to use LangChain, LlamaIndex, or another orchestration framework. I wanted to understand and control the individual pieces myself.

The result is a custom architecture built around Django REST Framework, PostgreSQL, Angular, and pluggable LLM providers.

This is the architecture I am building and the reasoning behind it.

Note: This article focuses mainly on New and Follow-Up user queries that require SQL generation or updates. Also the components I developed for the chatbot. It does not cover how the chatbot handles greetings, out-of-domain questions, capability-related questions, or clarification requests.

Why I Built a Text-to-SQL Chatbot

The goal wasn’t to build a chatbot that knows how to write SQL. The goal was to build an internal business analytics assistant.

The users are non-technical management team members. They shouldn’t need to know SQL, PostgreSQL, Table Names, Column Names, Joins, Database Schemas, etc.

They should be able to ask a question naturally:

How many email issue tickets were created in the last 7 days?

Then continue:

And in the last 1 month?

Then:

And how many are still pending?

And then:

Break them down by priority.

The technical implementation behind those questions should be invisible to the user. That distinction influenced almost every architectural decision I made afterward. I wasn’t building a SQL interface. I was building a business analytics interface that uses Text-to-SQL internally.

The Problem I Wanted to Solve

A basic Text-to-SQL pipeline looks like this:

This is attractive because it’s simple, but consider what happens when the user asks:

How many tickets were created for email issues in the last 7 days?

Then:

And in the last 1 month?

The second question isn’t a complete SQL request by itself. The model needs to understand that:

• we’re still talking about tickets
• we’re still talking about email issues
• the metric is still ticket count
• only the time period changed

Then the user asks:

And how many are still pending?

Now the system needs to preserve the previous analytical context and modify the analysis again.

Then the user says:

Give me the count of rides scheduled yesterday.

That’s a completely different analysis.

The conversation hasn’t ended, but the active analytical context has changed. This is where I realized that a production Text-to-SQL system needs considerably more structure than: Prompt -> SQL -> Database.

Technology Stack

The current stack is:

• Angular for the frontend
• Django REST Framework for the backend API
• PostgreSQL as the database being queried
• Qwen2.5-Coder 7B through Ollama for local development and testing
• OpenAI’s API for production

I also deliberately kept the LLM provider behind an abstraction. The application shouldn’t care whether the request is being sent to Ollama locally or to OpenAI APIs in production.

Conceptually:

This gives me the ability to test the workflow locally without making the entire architecture dependent on one provider.

Why I Didn’t Use LangChain or LlamaIndex

I considered the usual framework-based approach, but I decided to build the orchestration myself.

This isn’t because I think LangChain or LlamaIndex are inherently bad choices, but frameworks make debugging very difficult. Also, with a custom orchestration layer, you can track every single prompt.

Instead of hiding orchestration inside a framework, I wanted those decisions to be visible in my own Python code.

The trade-off is obvious: I now have to build and maintain that orchestration myself. But for this internal application, that trade-off is acceptable because understanding and controlling the workflow is more important to me than minimizing the amount of code.

High-Level Architecture

The architecture evolved into something closer to this:

There are several specialized components behind this diagram.

Note: Other Workflows are to handle Greetings, Out-of-Domain questions, Capability related questions, Clarification requests, and other types of user queries.

The Major Components

1. Intent Detector

The chatbot didn’t have an Intent Detector initially. I introduced one later as I added more workflows, because not every user message needs to go through the SQL Generation workflow.

The detector is intentionally business-oriented. It doesn’t need to know that the system generates SQL; its job is simply to determine what the user is trying to accomplish and route the request to the appropriate workflow.

For example:

• Hello, how are you? — is a greeting.
• What can you do? — is a capability request.
• Tell me a joke. / What is the weather today? / What is the match score? — All these questions are out of domain.

None of these requests require the SQL Generation workflow. The Intent Detector allows the chatbot to identify such requests early and avoid invoking an unnecessary workflow.

2. Request Router

The Intent Detector answers: “What kind of request is this?”

The Request Router answers: “Which workflow should handle it?”

I deliberately separated these two responsibilities because they solve different problems. The Intent Detector classifies the user’s request, while the Request Router decides where that request should go.

This separation gives me a cleaner architecture and keeps the responsibilities well defined.

It may seem like extra abstraction for a small application, but I prefer this approach because the routing logic will likely grow as the system evolves.

Keeping these responsibilities separate makes that growth easier to manage without making the Intent Detector responsible for workflow-specific decisions.

3. Workflows

The workflow layer is where the major orchestration happens. I initially had a single SQL Generation workflow, but as the system became more sophisticated, I moved toward separate workflows.

4. Schema Metadata

One of the most important decisions was not to send the entire database schema to the model for every request.

I maintain business-oriented metadata such as:

TABLES = {
    "tickets": {
        "description": "..."
    },
    # ...
}

and additional information in a separate dictionary, including Columns, Relationships, and Business Concepts.

This lets the application progressively provide context to the model. The first step is deliberately broad. I send the available table names and their descriptions to the LLM and ask:

“Which tables are relevant to this question?”

Only after I have the relevant tables do I retrieve their detailed schema information. That leads to a two-stage process:

User Question > All Table Descriptions > Table Selection LLM > Relevant Tables > Columns + Relationships + Business Concepts > SQL Generation LLM

This is much more controlled than blindly putting the entire schema into the SQL-generation prompt.

5. LLM Provider

I created an abstraction around the model provider. The rest of the application shouldn’t need to know whether it is talking to Ollama or OpenAI.

The provider layer is responsible for the model-specific communication, while the workflows decide what the model should be asked to do.

That separation also makes local development practical. I can use Qwen2.5-Coder 7B through Ollama locally and switch to OpenAI APIs without redesigning the workflow.

6. Conversation State

This was one of the areas where my thinking changed considerably. Initially, it is tempting to think: “I’ll just pass the conversation history to the LLM.”

But conversation history and analytical state aren’t the same thing. Conversation history tells me what people said. State tells me what we currently know about the active analysis.

For example, after:

“How many email issue tickets were created in the last 7 days?”

the active state can preserve information such as:

{
    "active_analysis": {
        "primary_tables": [
            "ticket",
            "ticket_detail" // Contains the ticket origin and related details.
        ],
        "last_sql": "...",
        "last_result_summary": "...",
        "filters": {
            "issue_type": "email",
            "time_range": "last_7_days"
        }
    }
}

The exact state will evolve, but the important concept is that the state represents the current analytical context. This becomes especially useful when the user asks:

“And in the last month?”

The application doesn’t have to rediscover everything from scratch.

7. Table Selector

The table selector has one job: Identify which business entities are relevant to the user’s question.

For example, “How many email issue tickets were created last week?” might identify the ticket-related tables rather than exposing every table in the database to the SQL generator. This also helps reduce prompt size and keeps the SQL-generation context focused.

8. SQL Generator

Once the relevant schema has been selected, the SQL generator receives:

• the user’s question
• relevant table information
• relevant columns
• relationships
• business concepts
• necessary conversational context

Its job is to produce read-only SQL, which is simply a SELECT query.

9. SQL Validator

I don’t trust generated SQL simply because it was produced by an LLM. AI-generated queries may contain harmful or destructive clauses, such as DROP, DELETE, or other data-modification statements. They may also generate expensive or inefficient queries that could negatively impact database performance.

This is why I want a clear boundary between:

LLM-generated SQL > SQL Validation > Database

That boundary is particularly important because this is a read-only internal analytics system. The model should never be given unrestricted access to modify the database.

10. SQL Repair

SQL generation can fail. A query can have:

• syntax errors
• incorrect assumptions
• invalid references
• other execution problems

Rather than immediately returning an error to the user, the workflow can send the failure information back to the LLM and ask it to repair the query. Refer to the High Level Architecture above to understand the flow.

I also log these attempts because a production system needs to make failures observable rather than invisible.

11. Result Summarizer

The database result isn’t necessarily the final response. A user doesn’t want to see a raw database result like:

[
    {"priority": "High", "count": 125},
    {"priority": "Medium", "count": 312}
]

The final layer converts the result into a concise business-oriented response. The workflow therefore ends with:

SQL Result > Result Summarizer > Business Response

This also keeps technical details away from the management user.

Final Thoughts

Building a Text-to-SQL chatbot turned out to be much less about “getting an LLM to write SQL” than I initially expected. The interesting engineering problem is everything surrounding the SQL.

I needed a way to select only relevant Schema Context. I needed structured conversation state so follow-up questions could work reliably. I needed intent detection and routing so every message wouldn’t trigger the expensive analytical workflow. I needed separate workflows for all the Intent Types as well as new and continuing analysis. I needed validation and repair because generated SQL isn’t automatically trustworthy.

The biggest lesson for me is that a production-oriented AI application shouldn’t treat the LLM as the application. The LLM is one component inside the application. The application still needs architecture, state, validation, routing, observability, security, and clear boundaries around what the model is allowed to do.

And that’s ultimately why I chose to build this system myself instead of handing the entire orchestration problem to a framework.

The interesting part isn’t just making an LLM generate SQL.

It’s engineering everything around it so that the system can actually be trusted to have a conversation.