The problem wasn’t a lack of AI tools.
It was that we were starting to build too many of them.
As MCP started becoming part of our AI infrastructure, different people in the organization began creating their own dedicated tools and integrations for specific use cases. Each solution worked, but over time this introduced more applications, more dependencies, and more things to maintain.
I started looking for a way to solve this at the infrastructure level instead of continuing to build another tool for every new requirement.
That led me to MCP Gateway.
The Initial Challenge
As more teams started experimenting with MCP, the pattern was becoming familiar.
Someone needed a new capability, so they would build a dedicated tool or service around it.
Then came the dependencies:
- Another application to deploy
- Another API to maintain
- Another authentication mechanism
- Another integration to manage
- Another service for the AI application to communicate with
Individually, these solutions made sense.
But at an organizational level, it started creating fragmentation.
I wanted to move towards a model where we could deploy MCP servers once and make their capabilities available to different AI applications through a common gateway.
Finding a Different Approach
Instead of building another custom integration, I came across an open-source MCP Gateway that could provide this centralized layer.
The idea was simple:
Don’t make every AI application integrate with every MCP server.
Put a gateway in between.
The architecture became:
AI Applications → MCP Gateway → MCP Servers
This gave us a central place to connect and manage MCP tools while keeping the individual MCP servers independent.
Deploying the MCP Gateway
I deployed the MCP Gateway as a self-hosted stack.
The deployment included:
- MCP Gateway data plane
- MCP Gateway admin plane
- Web UI
- PostgreSQL
- Database migration service
- Traefik for routing and HTTPS
- Tenant and token bootstrap
The gateway itself runs with separate data and administrative interfaces, while the UI provides a centralized way to manage the gateway.
Traefik handles external routing, allowing the gateway and its administration interface to be exposed through HTTPS.
Making Existing APIs MCP-Accessible
Another part that made this approach interesting was working with APIs that already existed inside our infrastructure.
Instead of building a completely new application around an existing service, we could use its OpenAPI specification as the basis for exposing its capabilities.
A lot of our internal services are Python REST APIs and already provide Swagger/OpenAPI documentation.
Get Shaikhkaysan’s stories in your inbox
Join Medium for free to get updates from this writer.
That meant we could reuse something we already had instead of creating another layer of custom integration.
This was one of the things that made the gateway approach much more practical for our environment.
The Part Most People Don’t See
After deploying it, the MCP Gateway became part of the infrastructure behind our AI applications.
One of those applications is our internal AI platform, Secure Chat.
Today, users interacting with Secure Chat may use MCP-powered capabilities without even knowing that an MCP Gateway is sitting behind the application.
The flow is essentially:
User -> Secure Chat ->Agent Service -> MCP Gateway ->MCP Server -> Tool / Service
Secure Chat can fetch and use the MCP servers connected through the gateway without having to maintain a separate direct integration with every individual server.
That was the real value of the deployment.
The gateway became infrastructure rather than another application that users needed to think about.
What Changed?
Before introducing this approach, adding another AI capability could mean creating another dedicated integration.
With the gateway, the model became much more centralized:
Deploy the MCP server → Connect it to the gateway → Make the capability available to AI applications.
This doesn’t eliminate the need to build MCP servers or APIs.
Instead, it gives us a common layer for connecting those capabilities to the applications that need them.
A Real Example: Presenton
One of the first examples that made this architecture particularly interesting was Presenton.
I deployed Presenton as a self-hosted MCP server and connected it through the MCP Gateway.
Once integrated with Secure Chat, the AI could use Presenton’s capabilities instead of simply describing how to create a presentation.
The result was a much more useful interaction:
Secure Chat ->MCP Gateway -> Presenton ->Presentation
This became a good example of why I wanted the gateway architecture in the first place.
The AI application didn’t need to become a presentation-generation application.
It simply gained access to a specialized capability.
Lessons Learned
The biggest lesson for me was that AI infrastructure needs to scale beyond the model itself.
As more people start building AI applications, the number of tools and integrations can grow very quickly.
Without some common infrastructure, every new capability can become another standalone application with its own dependencies and maintenance requirements.
An MCP Gateway provides a layer between those two worlds.
AI applications don’t need to know everything about every tool.
They can communicate with a common gateway, while the gateway handles the connection to the MCP ecosystem behind it.
Final Thoughts
What started as a problem of too many dedicated tools turned into an infrastructure problem.
Instead of continuing to build integrations one application at a time, I wanted to create a common layer that could connect MCP servers to the applications that need them.
Today, the MCP Gateway runs behind the scenes and is already being used by Secure Chat.
Users don’t necessarily see it.
And that’s probably the most interesting part.
Good infrastructure doesn’t always need to be visible.
Sometimes its job is simply to make everything else easier to build.
#AI #MCP #AIDevOps #DevOps #AIInfrastructure #PlatformEngineering #OpenSource #SecureChat #MCPGateway #ArtificialIntelligence
