Building a Batch Processing System with AWS Bedrock: A Practical Guide

Building a Batch Processing System with AWS Bedrock: A Practical Guide

As an Graduate Student of Computer Science and researcher, I’ve been exploring various AI/ML services across cloud providers. While working with Azure OpenAI’s batch processing and Google’s Gemini API, I became curious about AWS Bedrock’s batch inference capabilities. This article chronicles my journey building a batch processing system for AWS Bedrock, including the challenges, solutions, and infrastructure automation.

The Big Picture: Why Batch Processing?

Batch processing for Large Language Models (LLMs) offers significant advantages:

• Cost Savings: Up to 50% reduction compared to real-time API calls
• Efficiency: Process thousands of requests without rate limiting concerns
• Asynchronous Processing: Submit jobs and retrieve results when ready
• Resource Optimization: Better for non-time-sensitive workloads

This pattern is common across major AI providers:

• Azure OpenAI: Batch API with Global-Batch SKU
• Google Gemini: Batch prediction API
• AWS Bedrock: Model invocation jobs

The Initial Goal

I wanted to build a Python-based batch processor that could:

1. Accept multiple prompts/questions
2. Submit them as a batch job to AWS Bedrock
3. Monitor the job progress for up to 24 hours
4. Download and format results in JSON

The use case? Processing large volumes of questions for documentation, research, or data analysis tasks where immediate responses aren’t critical.

Architecture Overview

The system consists of three main components:

• Prompt.py — to ask to LLM using processor batch script
• S3 bucket — to store json input and output of batch jobs
• Bedrock Batch Job

Infrastructure Setup with CloudFormation

Rather than manually configuring IAM roles, S3 buckets, and permissions, I created a CloudFormation template to automate the entire infrastructure setup.

What the CloudFormation Template Creates

Resources:
  1. S3 Bucket (with versioning and lifecycle policies)
  2. IAM Role for Bedrock (allows Bedrock to access S3)
  3. IAM User for Script Execution (with access keys)
  4. Policies and Permissions (least privilege access)

Key Components Explained

1. S3 Bucket Configuration

BatchProcessingBucket:
  Type: AWS::S3::Bucket
  Properties:
    VersioningConfiguration:
      Status: Enabled
    LifecycleConfiguration:
      Rules:
        - ExpirationInDays: 30  # Auto-cleanup old files

The bucket stores:

• Input: JSONL files with batch requests
• Output: JSONL files with model responses

2. IAM Role for Bedrock

BedrockBatchRole:
  Type: AWS::IAM::Role
  Properties:
    AssumeRolePolicyDocument:
      Principal:
        Service: bedrock.amazonaws.com
    Policies:
      - S3 Read/Write access to batch bucket

This role allows Bedrock service to:

• Read input files from S3
• Write output files to S3
• Nothing more (security best practice)

3. IAM User for Script

BatchProcessingUser:
  Type: AWS::IAM::User
  Policies:
    - Bedrock API access
    - S3 bucket access
    - PassRole permission

Deploying the Infrastructure

The deployment is straightforward:

# Deploy CloudFormation stack
aws cloudformation create-stack \
  --stack-name bedrock-batch-processing \
  --template-body file://cloudformation-bedrock-batch.yaml \
  --capabilities CAPABILITY_NAMED_IAM \
  --region us-east-1

# Wait for completion
aws cloudformation wait stack-create-complete \
  --stack-name bedrock-batch-processing \
  --region us-east-1

# Get outputs (credentials, bucket name, role ARN)
aws cloudformation describe-stacks \
  --stack-name bedrock-batch-processing \
  --query 'Stacks[0].Outputs'

The CloudFormation outputs provide everything needed for the .env file:

• AWS Access Key ID
• AWS Secret Access Key
• S3 Bucket Name
• Bedrock Role ARN
• AWS Region

The Code: Batch Processing Implementation

Project Structure

batch/
├── bedrock_batch_processor.py        # Batch processing (requires approval)
├── bedrock_standard_processor.py     # Standard API (works immediately)
├── prompts.py                        # Questions to process
├── cloudformation-bedrock-batch.yaml # Infrastructure as Code
├── .env                              # Configuration (not in git)
├── .env.bedrock.example              # Template
├── requirements.txt                  # Dependencies
└── README.md                         # Documentation

Understanding the Batch Processor Code

Step 1: Setup and Configuration

import boto3
from dotenv import load_dotenv

load_dotenv()

# Initialize Bedrock client
bedrock = boto3.client(
    service_name='bedrock',
    region_name=os.getenv("AWS_REGION"),
    aws_access_key_id=os.getenv("AWS_ACCESS_KEY_ID"),
    aws_secret_access_key=os.getenv("AWS_SECRET_ACCESS_KEY")
)

# Initialize S3 client
s3 = boto3.client(
    service_name='s3',
    region_name=os.getenv("AWS_REGION"),
    aws_access_key_id=os.getenv("AWS_ACCESS_KEY_ID"),
    aws_secret_access_key=os.getenv("AWS_SECRET_ACCESS_KEY")
)

Step 2: Creating Batch Input File

Get Dibya Darshan Khanal’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

The input format follows Bedrock’s batch inference specification:

from prompts import questions

batch_requests = []
for i, question in enumerate(questions, 1):
    batch_requests.append({
        "recordId": f"request-{i}",
        "modelInput": {
            "anthropic_version": "bedrock-2023-05-31",
            "max_tokens": 1024,
            "messages": [
                {
                    "role": "user",
                    "content": question
                }
            ]
        }
    })

# Write to JSONL (one JSON object per line)
with open("bedrock_batch_input.jsonl", 'w') as f:
    for request in batch_requests:
        f.write(json.dumps(request) + '\n')

Each line in the JSONL file is a separate request with:

• recordId: Unique identifier for tracking
• modelInput: Model-specific request format (Claude in this case)

Step 3: Uploading to S3

bucket_name = os.getenv("S3_BUCKET_NAME")
input_key = f"batch-input/bedrock_batch_input.jsonl"
output_prefix = f"batch-output/{datetime.now().strftime('%Y%m%d-%H%M%S')}"

# Upload input file
s3.upload_file("bedrock_batch_input.jsonl", bucket_name, input_key)

Step 4: Creating the Batch Job

response = bedrock.create_model_invocation_job(
    roleArn=os.getenv("BEDROCK_ROLE_ARN"),
    modelId=os.getenv("BEDROCK_MODEL_ID"),
    jobName=f"batch-job-{datetime.now().strftime('%Y%m%d-%H%M%S')}",
    inputDataConfig={
        "s3InputDataConfig": {
            "s3Uri": f"s3://{bucket_name}/{input_key}"
        }
    },
    outputDataConfig={
        "s3OutputDataConfig": {
            "s3Uri": f"s3://{bucket_name}/{output_prefix}"
        }
    }
)

job_arn = response['jobArn']

Key parameters:

• roleArn: IAM role allowing Bedrock to access S3
• modelId: Which Bedrock model to use (Claude, Llama, Titan, etc.)
• inputDataConfig: S3 location of input JSONL
• outputDataConfig: S3 location for output files

Step 5: Monitoring Job Progress

start_time = time.time()
max_wait_time = 24 * 60 * 60  # 24 hours
check_interval = 60  # Check every 60 seconds

status = "InProgress"
while status in ["InProgress", "Submitted"]:
    elapsed_time = time.time() - start_time
    
    if elapsed_time >= max_wait_time:
        print("24 hours elapsed. Job still in progress.")
        break
    
    time.sleep(check_interval)
    job_details = bedrock.get_model_invocation_job(jobIdentifier=job_arn)
    status = job_details['status']
    
    hours = int(elapsed_time // 3600)
    minutes = int((elapsed_time % 3600) // 60)
    print(f"Status: {status} (Elapsed: {hours}h {minutes}m)")

The script polls the job status every 60 seconds, displaying:

• Current status (Submitted, InProgress, Completed, Failed)
• Elapsed time
• Progress information (if available)

Step 6: Retrieving Results

if status == "Completed":
    # List output files in S3
    response = s3.list_objects_v2(
        Bucket=bucket_name,
        Prefix=output_prefix
    )
    
    # Download and parse results
    for obj in response['Contents']:
        if obj['Key'].endswith('.jsonl.out'):
            s3.download_file(bucket_name, obj['Key'], local_file)
            
            with open(local_file, 'r') as f:
                for line in f:
                    result = json.loads(line)
                    # Extract answer from modelOutput
                    content = result['modelOutput']['content'][0]['text']
                    results.append({
                        "request_id": result['recordId'],
                        "status": "success",
                        "answer": content,
                        "usage": result['modelOutput']['usage']
                    })
    
    # Save as formatted JSON
    with open(f"bedrock_results_{timestamp}.json", 'w') as f:
        json.dump({
            "job_arn": job_arn,
            "total_requests": len(results),
            "results": results
        }, f, indent=2)

The Challenge: Batch Inference Requires Approval

When I first ran the batch processor, I encountered this error:

ValidationException: Your account is not authorized to perform this action. 
Please create a support case with details about your use case.

What I learned:

• AWS Bedrock batch inference is not enabled by default
• It requires explicit approval from AWS Support
• This is different from standard Bedrock API access
• Even AWS Community Builder credits (I was a CB for previous year) don’t automatically grant batch access

This mirrors how Azure OpenAI requires specific SKU types (Global-Batch or Data Zone Batch) for batch processing, not just any deployment.

The Solution: Standard Processing Alternative

While waiting for batch approval, I created a standard processing version that works immediately:

# bedrock_standard_processor.py
bedrock_runtime = boto3.client(service_name='bedrock-runtime', ...)

for question in questions:
    request_body = {
        "anthropic_version": "bedrock-2023-05-31",
        "max_tokens": 1024,
        "messages": [{"role": "user", "content": question}]
    }
    
    response = bedrock_runtime.invoke_model(
        modelId=os.getenv("BEDROCK_MODEL_ID"),
        body=json.dumps(request_body)
    )
    
    answer = json.loads(response['body'].read())['content'][0]['text']

Key differences:

• Uses bedrock-runtime client instead of bedrock
• Calls invoke_model() instead of create_model_invocation_job()
• No S3 upload/download needed
• Processes requests sequentially
• Works without special approval
• No cost savings (standard pricing)

Getting Started: Step-by-Step Guide

Prerequisites

1. AWS Account with Bedrock access
2. Python 3.9+ installed
3. AWS CLI configured
4. Boto3 library

Installation

# Clone the repository
git clone https://github.com/yourusername/bedrock-batch-processing.git
cd bedrock-batch-processing

# Create virtual environment
python3 -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

Configuration

1. Deploy Infrastructure

aws cloudformation create-stack \
  --stack-name bedrock-batch-processing \
  --template-body file://cloudformation-bedrock-batch.yaml \
  --capabilities CAPABILITY_NAMED_IAM \
  --region us-east-1

2. Get Configuration Values

aws cloudformation describe-stacks \
  --stack-name bedrock-batch-processing \
  --query 'Stacks[0].Outputs'

3. Create .env File

Copy the outputs to .env:

AWS_ACCESS_KEY_ID=AKIA...
AWS_SECRET_ACCESS_KEY=...
AWS_REGION=us-east-1
BEDROCK_MODEL_ID=anthropic.claude-3-sonnet-20240229-v1:0
BEDROCK_ROLE_ARN=arn:aws:iam::123456789:role/bedrock-role
S3_BUCKET_NAME=bedrock-batch-1234567890

4. Add Your Questions

Edit prompts.py:

questions = [
    "Explain quantum computing in simple terms",
    "What are the benefits of serverless architecture?",
    "How does machine learning differ from deep learning?",
    # Add more questions...
]

Running the Processor

Option 1: Standard Processing (Works Immediately)

python bedrock_standard_processor.py

Output:

======================================================================
AWS Bedrock Standard Processing Demo
======================================================================
Region: us-east-1
Model: anthropic.claude-3-sonnet-20240229-v1:0
======================================================================

Processing 10 prompts...

[1/10] Processing: Explain quantum computing in simple terms...
Completed

[2/10] Processing: What are the benefits of serverless...
Completed

...

Results saved to bedrock_results_20251125123456.json
Completed: 10/10 requests successful

Option 2: Batch Processing (Requires AWS Approval)

python bedrock_batch_processor.py

If not approved, you’ll see the authorization error and need to:

1. Go to AWS Support Console
2. Create a case requesting Bedrock batch inference access
3. Explain your use case
4. Wait for approval (typically 1–3 business days)

Cost Comparison

Based on AWS Bedrock pricing (as of Nov 2025):

Standard Processing:

• Claude 3 Sonnet: $3.00 per 1M input tokens, $15.00 per 1M output tokens

Batch Processing:

• Claude 3 Sonnet: $1.50 per 1M input tokens, $7.50 per 1M output tokens
• 50% cost savings

For 10,000 requests averaging 500 input tokens and 200 output tokens each:

• Standard: $45 (5M input + 2M output)
• Batch: $22.50 (50% savings)

Lessons Learned

1. Service Availability ≠ Feature Availability

Having AWS and Bedrock access doesn’t automatically grant batch inference permissions. Always check feature-specific requirements.

2. Infrastructure as Code is Essential

The CloudFormation template saved hours of manual IAM/S3 configuration and ensures reproducible deployments.

3. Build Fallback Options

Having both batch and standard processors allows flexibility based on account permissions and use case urgency.

4. Cross-Cloud Patterns

The batch processing pattern is remarkably similar across providers:

• Azure OpenAI: Upload JSONL → Create batch → Poll status → Download results
• AWS Bedrock: Upload to S3 → Create job → Poll status → Download from S3
• Google Gemini: Upload to Cloud Storage → Create batch → Poll status → Download

Practical Use Cases

This batch processing system is ideal for:

1. Documentation Generation: Process hundreds of code snippets into explanations
2. Data Analysis: Analyze survey responses or customer feedback at scale
3. Content Creation: Generate variations of marketing copy or product descriptions
4. Research: Process academic papers for summarization or key point extraction
5. Translation: Translate large volumes of text across languages
6. Code Review: Analyze codebases for improvements or bug detection

Conclusion

Building this AWS Bedrock batch processing system taught me valuable lessons about cloud AI services, infrastructure automation, and the importance of understanding service-level permissions. While I’m still awaiting batch inference approval, the standard processing alternative provides a working solution.

The CloudFormation template makes deployment straightforward, and the modular code structure allows easy customization for different use cases. Whether you’re processing 10 or 10,000 prompts, this system provides a solid foundation.

I’m excited to see how batch inference performs once approved and will update this project with performance benchmarks and cost comparisons.

Resources

• GitHub Repository: https://github.com/Dibae101/AWS-Bedrock-Batch-Processing
• AWS Bedrock Docs: https://docs.aws.amazon.com/bedrock/latest/userguide/batch-inference.html