Deploying Your Own Ai agent with Amazon Bedrock
what is amazon Bedrock?
If you haven’t used Amazon Bedrock yet, it’s basically a massive shortcut for building AI apps on AWS. It gives you access to a bunch of top-tier foundation models through a single API, and because it’s completely serverless, you don’t have to scramble for GPUs, manage infrastructure, or build messy training pipelines. You just connect to it and start building.
You get instant access to the heavy hitters: Anthropic’s Claude, Stability AI, Cohere, AI21 Labs, and Amazon’s own Titan models. Whether you’re generating text, whipping up images, writing code, or wiring together complex RAG workflows, Bedrock handles it.
It also comes baked with the stuff you actually need to push an app to production. We’re talking fine-tuning, AI agents, native knowledge base integrations, and enterprise-grade security (like IAM, VPC support, encryption, and Guardrails). Basically, it’s powerful and ready for the real world.
But here’s the question: How do we actually use it from a browser?
Hitting the Bedrock API directly is fine, but let’s be honest nobody wants to write cURL commands in a terminal just to chat with an LLM.
I wanted a clean, out-of-the-box web interface that actually looks good and lets me talk to the models naturally. So, doing what any efficient developer would do, I went looking for an existing solution so I wouldn’t have to build a UI from scratch.
That’s when I stumbled onto an absolute gem: OpenWebUI.
OpenWebUI has quickly become a go-to frontend for interacting with Large Language Models. It is a highly polished, self-hosted web application that runs entirely within a Docker container, making the initial deployment remarkably straightforward. Beyond offering a clean, responsive chat interface, it brings enterprise-grade features to the table. You get robust multi-user role management and SSO integration — supporting OAuth providers like Google and Okta — which is crucial if you are deploying this for a wider team. It also goes beyond standard text generation by supporting multimodal workflows, including hands-free voice and video interactions, native image generation, and a powerful local Retrieval-Augmented Generation (RAG) system. This means you can upload your own documents directly into the chat or hook the system up to web search providers to get highly contextual, grounded answers.
By design, OpenWebUI is built to interface seamlessly with local models via Ollama or any remote service that adheres to the standard OpenAI API specification. This creates a temporary roadblock for AWS developers: OpenWebUI does not natively understand Amazon Bedrock’s proprietary API structure, nor does it support the AWS Signature V4 authentication mechanism required to talk to AWS services.
This is where a highly practical open-source tool from AWS, the Bedrock Access Gateway (BAG), comes into play. You can think of the gateway as a seamless reverse proxy or a universal API translator. It sits directly between OpenWebUI and your AWS environment. When OpenWebUI sends out a standard OpenAI-formatted API request, the gateway intercepts it. It then dynamically maps the requested model to the corresponding Bedrock model ID, securely signs the request using your environment’s AWS IAM credentials, and forwards it to the Bedrock API. When Bedrock responds, the gateway reformats that data back into the OpenAI structure before handing it back to the user interface. The primary advantage of this architecture is that it requires absolutely zero source code modifications to OpenWebUI.
With this architecture in mind, the execution plan is highly methodical. We will deploy this as a two-container stack on AWS, which can be hosted on a service like Amazon ECS or a standard EC2 instance. First, we will spin up the Bedrock Access Gateway container and attach the necessary IAM roles so it has the authorization to invoke Bedrock models. Next, we will deploy the OpenWebUI container and configure its backend API endpoint to route traffic directly to our new gateway. Once the containers are networked together, we will have a fully functional, secure, browser-based environment to interact with Amazon’s most powerful foundation models. Let us begin the deployment process.
The Architectural Infrastructure
Let’s break down the actual infrastructure we are going to build. While the architectural diagram might look a bit involved at first glance, every component is chosen specifically to ensure the deployment is secure, scalable, and resilient.
Here is a detailed look at how the environment is structured:
Compute and Internal Networking
For the core hosting environment, we are utilizing Amazon Elastic Container Service (ECS) powered by AWS Fargate. Fargate is a serverless compute engine for containers, which means we can run both OpenWebUI and the Bedrock Access Gateway without having to provision, manage, or patch any underlying EC2 servers.
To maintain strict security boundaries, both of these services will be deployed into private subnets. Because they are in separate containers and their internal IP addresses can change upon restarts, we will use AWS Cloud Map. Cloud Map provides internal service discovery, acting as a private DNS so the OpenWebUI container can always reliably locate and communicate with the Bedrock Access Gateway using a consistent internal hostname.
Container Registry Strategy
Instead of pulling the Docker images directly from public registries during deployment, we will create dedicated Amazon Elastic Container Registry (ECR) repositories for each service. Even though OpenWebUI provides readily available public images, hosting them in our own ECR is a much better practice. It protects the deployment from third-party repository downtime, avoids Docker Hub rate limits, and gives us complete control over versioning and the software supply chain.
Data Persistence and Secrets Management
Get Joshua Kimathi’s stories in your inbox
Join Medium for free to get updates from this writer.
Because container storage is ephemeral, any data saved locally would be instantly wiped out if the OpenWebUI container restarts or scales. To solve this, we will attach an Amazon Elastic File System (EFS) volume. EFS provides a persistent, shared file system where OpenWebUI can store its internal SQLite database, user profiles, system prompts, and chat histories, ensuring zero data loss between container lifecycles.
For security, the Bedrock Access Gateway requires an API key to authenticate incoming requests. Rather than hardcoding this in our environment variables or configuration files, we will generate a secure key and store it in AWS Secrets Manager. ECS will then securely fetch and inject this secret into the containers strictly at runtime.
External Traffic and Routing
Since our containers are isolated in private subnets, we need a secure way to access the web interface from a browser. We will place an Application Load Balancer (ALB) in the public subnets to serve as the main entry point. The ALB will receive your HTTP/HTTPS requests and safely route that traffic down to the OpenWebUI container.
Conversely, when those private containers need to reach out to the public internet whether to download an update, fetch external data, or utilize OpenWebUI’s web-search features that outbound traffic will be safely routed through a NAT Gateway. This ensures the containers can speak to the outside world without ever being directly exposed to it.
Executing the Deployment
Now that the architecture is mapped out, we can move into the actual deployment phase.
To streamline this process and eliminate manual provisioning, I have consolidated the entire setup into a ready-to-use repository:
https://github.com/hakimizz-tech/aws-bedrock-chatbot-agent
This repository contains all the necessary Terraform configuration files required to spin up the architecture, alongside an automated execution script that handles the container image builds and infrastructure deployment from start to finish. To run this in your own environment, you only need two prerequisites configured on your local machine: your target AWS Account ID and an authenticated AWS CLI profile with the appropriate provisioning permissions.
Configuring Amazon Bedrock Model Access
Before initiating the deployment, there is an essential configuration step required within your AWS environment. By default, AWS accounts do not have immediate, blanket access to Amazon Bedrock’s foundation models. You must navigate to the Bedrock console and explicitly request access to the models you intend to use.
The approval process varies depending on the underlying AI provider. For third-party models such as Anthropic’s Claude, Meta’s Llama, or Mistral AWS requires you to submit a brief use-case justification form, and access is subject to a manual approval process which can sometimes be denied depending on the criteria. If you want to bypass this and get the deployment running immediately, I recommend starting with the Amazon Titan models. They offer robust performance for general conversational tasks and are typically accessible by default without requiring additional vetting.
Troubleshooting the Frontend Build
As the deployment script builds the Docker images to push to your Elastic Container Registry, you may encounter a known bottleneck. The OpenWebUI frontend compilation is resource-intensive, and the build process will occasionally crash due to Node.js memory exhaustion limits.
If your build fails with an out-of-memory error, you can resolve it by applying a minor patch to the OpenWebUI Dockerfile. Locate the step responsible for building the frontend assets, which typically reads RUN npm run build. You will need to explicitly allocate more memory to the Node process by modifying that line to read: RUN NODE_OPTIONS="--max-old-space-size=4096" npm run build. This allocates 4GB of RAM specifically for the build process and ensures the image compiles successfully.
Accessing Your Private Chat Interface
Once Terraform completes the infrastructure provisioning, it will output the public DNS address assigned to your Application Load Balancer.
Navigate to this URL in your web browser to access your OpenWebUI instance. Upon your first visit, you will be prompted to register an account. It is worth noting that this is a completely isolated environment; your account details, configurations, and chat logs are stored securely on the private Amazon EFS volume within your VPC and are never transmitted to any external servers.
Once your account is created, you will have a fully functional, enterprise-grade conversational interface running entirely on your own AWS infrastructure, ready to securely interface with Amazon Bedrock’s language models.
While the baseline deployment provides a highly functional environment, transitioning this architecture to a production-ready state requires a stricter approach to network security and access management. Here are the primary architectural upgrades to consider for hardening the stack.
Encrypting Internal Gateway Traffic The current setup relies on secure internal VPC routing, but implementing end-to-end encryption is a standard best practice. By placing an internal Application Load Balancer provisioned with SSL/TLS certificates directly in front of the Bedrock Access Gateway, you ensure that all traffic between the frontend and the backend API is fully encrypted in transit. This is relatively straightforward to implement and becomes mandatory if you plan to expose the gateway to other clients outside this specific VPC or if your organization enforces strict zero-trust network policies.
Implementing Strict Network Isolation with VPC Endpoints For maximum security, you can eliminate the NAT Gateway entirely and transition the Bedrock Access Gateway into a fully isolated subnet with zero outbound internet access. To ensure the gateway can still reach Amazon Bedrock, you deploy AWS PrivateLink via interface VPC Endpoints. This forces all model invocation traffic to stay strictly within the AWS network backbone, ensuring that only the OpenWebUI container can communicate with the gateway.
Architecting this level of isolation does introduce a specific dependency challenge. The gateway relies on the Python tiktoken library for prompt tokenization, which dynamically fetches its encoding cache from the internet at runtime. In a fully offline subnet, this request will crash the service. The engineering workaround is to modify your Dockerfile to pre-download and bundle the tiktoken cache directly into the container image during the CI/CD build phase. While this guarantees offline functionality, it introduces a maintenance tradeoff: whenever the tokenizer cache requires an update, you must trigger a full container rebuild. It is an added operational step, but highly manageable given the security payoff.
Hardening the Frontend Perimeter Securing the OpenWebUI entry point is critical since it acts as the public face of your deployment. Instead of relying on the default load balancer DNS, you should map a custom domain using Amazon Route 53 and provision a valid SSL/TLS certificate via AWS Certificate Manager (ACM) to enforce strict HTTPS. Furthermore, attaching an AWS Web Application Firewall (WAF) to the public-facing ALB provides a robust defense layer. A properly configured Web ACL allows you to implement geographic restrictions, rate limiting, and strict IP allow-lists, ensuring that only authorized users or corporate VPN traffic can even reach the login screen.
This architecture provisions a fully private, scalable, and browser-based Generative AI environment powered entirely by Amazon Bedrock and defined as code with Terraform. What originated as an exploratory concept following an AWS GameDay translates into an enterprise-ready foundation where you retain complete control over your data and infrastructure. Whether you are building internal corporate tooling, prototyping customer support agents, or simply mapping out how these AI components fit together, this stack provides a secure starting point. The repository is entirely open-source, so feel free to adapt the infrastructure to your specific requirements and submit pull requests with any improvements.
[Github Repo]
