<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Nils Rhode on Medium]]></title>
        <description><![CDATA[Stories by Nils Rhode on Medium]]></description>
        <link>https://medium.com/@nils.rhode?source=rss-f69b61e45ffb------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*o_lO7H_WayiBHovPF20M9Q.jpeg</url>
            <title>Stories by Nils Rhode on Medium</title>
            <link>https://medium.com/@nils.rhode?source=rss-f69b61e45ffb------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 23:19:25 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@nils.rhode/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[What It Actually Costs to Run an AI SaaS on AWS — A Real Monthly Breakdown]]></title>
            <link>https://medium.com/@nils.rhode/what-it-actually-costs-to-run-an-ai-saas-on-aws-a-real-monthly-breakdown-49fd225c42c3?source=rss-f69b61e45ffb------2</link>
            <guid isPermaLink="false">https://medium.com/p/49fd225c42c3</guid>
            <dc:creator><![CDATA[Nils Rhode]]></dc:creator>
            <pubDate>Fri, 27 Feb 2026 07:01:01 GMT</pubDate>
            <atom:updated>2026-02-27T07:01:01.655Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*l02oEhT0dcGwAQIlBH_Vcw.jpeg" /></figure><p><strong><em>Everyone talks about serverless being cheap. Nobody shows the bill. Here’s every line item from running a production AI-powered SaaS on AWS — Lambda, Bedrock, DynamoDB, CloudFront, Cognito, the whole stack — with real numbers.</em></strong></p><p>I’ve been running <a href="https://www.meinerechtshilfe.de/">meinerechtshilfe.de</a> in production for a few months now. It’s a legal tech platform that provides AI-powered legal assessments for German law — users submit a legal question, optionally upload documents, and get an AI-generated analysis with cited statutes and court decisions.</p><p>The entire backend is serverless on AWS. No EC2 instances. No containers. No Kubernetes. Just Lambda functions, DynamoDB tables, S3 buckets, and Bedrock for AI. The infrastructure is defined in 9 CDK stacks with about 30 resources.</p><p><strong>People keep asking me: “What does it actually cost?” Here’s the honest answer, broken down by service.</strong></p><h3>The Architecture (Quick Overview)</h3><p>Before the numbers make sense, here’s what’s running:</p><ul><li><strong>CloudFront</strong> — CDN for the React frontend + reverse proxy for /api/* to API Gateway</li><li><strong>S3</strong> — Frontend static files, user document uploads, legal knowledge base documents, analytics data</li><li><strong>API Gateway</strong> — REST API with Cognito authorizer</li><li><strong>Lambda</strong> — 9 functions (auth, AI, documents, payments, history, user, export, MFA, contact)</li><li><strong>DynamoDB</strong> — 4 tables (users, assessments, documents, payments), all on-demand billing</li><li><strong>Cognito</strong> — User authentication with Google social login</li><li><strong>Bedrock</strong> — AI models (Nova Lite, Nova 2 Lite, Claude Sonnet 4.5, Claude Opus 4.6) + Knowledge Bases with Cohere embeddings</li><li><strong>Secrets Manager</strong> — Stripe API keys</li><li><strong>CloudWatch</strong> — Logs, metrics, alarms, X-Ray tracing</li><li><strong>Route53</strong> — DNS</li><li><strong>ACM</strong> — SSL certificates</li></ul><h3>The Monthly Bill: Service by Service</h3><p>These numbers are based on an early-stage SaaS with moderate traffic — not thousands of daily users, but real production usage with real AI calls. I’ll note where costs scale linearly and where they stay flat.</p><h3>Amazon Bedrock — The Big One</h3><p>This is where most of the money goes. Bedrock charges per token, and the cost varies wildly by model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eTQh_gDeKEjYcbrDf06Zvw.png" /></figure><p>Free tier assessments cost essentially nothing. You could serve 10,000 free assessments for about $2. The premium and unlimited tiers are where costs add up, but they’re also where revenue comes in (€4.49 and €8.49 per assessment respectively, after launch discount).</p><p><strong>Knowledge Base costs (RAG):</strong></p><ul><li>Cohere Multilingual v3 embeddings: ~$0.0001 per query (5 retrievals × ~500 tokens each)</li><li>Knowledge Base storage: negligible (S3 pricing for the vector index)</li><li>Document parsing with Nova 2 Lite during ingestion: one-time cost per document, ~$0.001 each</li></ul><p><strong>Estimated monthly Bedrock total: $5–50</strong> (scales linearly with paid assessments)</p><p>At low volume, Bedrock is cheap. At 500 premium assessments/month, you’re looking at ~$25. The free tier barely registers.</p><h3>AWS Lambda</h3><p>9 functions, all Python 3.11. The AI handler is the heaviest (1024 MB, 5-minute timeout for async processing). Most others are 256 MB with 30-second timeouts.</p><ul><li><strong>Free tier:</strong> 1M requests + 400,000 GB-seconds/month (this covers a lot)</li><li><strong>AI handler:</strong> ~$0.0000167 per second at 1024 MB. A 30-second Bedrock call costs ~$0.0005</li><li><strong>All other handlers:</strong> Typically complete in &lt;1 second. Negligible cost.</li></ul><p><strong>Estimated monthly Lambda total: $0.50–5</strong></p><p>Lambda is absurdly cheap for this workload. Even the async AI processing (which runs a second Lambda invocation per assessment) stays well within free tier for months. You’d need thousands of daily assessments before Lambda costs become noticeable.</p><h3>Amazon DynamoDB</h3><p>4 tables, all on-demand (pay-per-request) billing:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SNLLVG8wTnU3obfisn5P4g.png" /></figure><ul><li><strong>On-demand pricing:</strong> $1.25 per million write units, $0.25 per million read units</li><li><strong>Storage:</strong> $0.25 per GB/month</li></ul><p>With GSIs (Global Secondary Indexes) on each table for email lookups and user-based queries, read costs are slightly higher than a single-table design. But at this scale, it doesn’t matter.</p><p><strong>Estimated monthly DynamoDB total: $1–3</strong></p><p>DynamoDB on-demand is perfect for early-stage SaaS. You pay nothing when nobody’s using it, and it scales automatically when they do. The break-even point where provisioned capacity becomes cheaper is around 10,000+ daily active users.</p><h3>Amazon CloudFront</h3><p>Serving the React frontend (SPA) + proxying API calls to API Gateway.</p><ul><li><strong>Data transfer:</strong> $0.085/GB for first 10 TB (Europe)</li><li><strong>Requests:</strong> $0.0075 per 10,000 HTTPS requests</li><li><strong>CloudFront Functions:</strong> $0.10 per million invocations (the /api prefix stripper)</li><li><strong>Free tier:</strong> 1 TB transfer + 10M requests/month for the first year</li></ul><p>The frontend bundle is ~500 KB gzipped. With aggressive caching (CACHING_OPTIMIZED for static assets, 6-hour TTL for SEO files), repeat visitors cost almost nothing.</p><p><strong>Estimated monthly CloudFront total: $0.50–2</strong> (mostly free tier in year one)</p><h3>Amazon S3</h3><p>Multiple buckets:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eKa1Q0vCvSTVDEKbJD2DRQ.png" /></figure><ul><li><strong>Storage:</strong> $0.023/GB/month (Standard)</li><li><strong>Requests:</strong> $0.0004 per 1,000 GET, $0.005 per 1,000 PUT</li></ul><p><strong>Estimated monthly S3 total: $0.50–1</strong></p><p>S3 is the cheapest service in the stack. Even with versioning enabled on all buckets, storage costs are trivial at this scale.</p><h3>Amazon Cognito</h3><p>User authentication with email/password + Google social login + TOTP MFA.</p><ul><li><strong>Free tier:</strong> 50,000 MAUs (monthly active users) — this is generous</li><li><strong>Beyond free tier:</strong> $0.0055 per MAU</li></ul><p><strong>Estimated monthly Cognito total: $0</strong> (free tier covers early-stage easily)</p><p>You’d need 50,000 monthly active users before Cognito costs a single cent. For a startup, this is effectively free forever during the growth phase.</p><h3>API Gateway</h3><p>REST API with Cognito authorizer, request validators, usage plans.</p><ul><li><strong>Free tier:</strong> 1M API calls/month for 12 months</li><li><strong>Beyond free tier:</strong> $3.50 per million calls (EU region)</li></ul><p><strong>Estimated monthly API Gateway total: $0–3.50</strong></p><p>Each page load triggers a few API calls (auth check, load data). Each assessment triggers 3–5 calls (submit, poll status, get result). At moderate traffic, you stay in free tier.</p><h3>Amazon Bedrock Knowledge Bases</h3><p>6 Knowledge Bases (5 legal categories + 1 general), each with its own vector store.</p><ul><li><strong>Vector storage:</strong> Included in Bedrock pricing (no separate OpenSearch cluster needed — this is huge)</li><li><strong>Retrieval queries:</strong> Charged per embedding + retrieval call</li><li><strong>Data ingestion:</strong> One-time cost per sync (embedding all documents)</li></ul><p>The decision to use Bedrock’s built-in vector store instead of OpenSearch Serverless saved significant money. OpenSearch Serverless has a minimum cost of ~$350/month for 2 OCUs. Bedrock Knowledge Bases with S3 vector stores: effectively $0/month for storage, pay only per query.</p><p><strong>Estimated monthly Knowledge Base total: $1–5</strong> (mostly embedding queries)</p><h3>Other Services</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6SDpk0gwnV19vyGkWoWXBQ.png" /></figure><p><strong>Estimated monthly “other” total: $2–4</strong></p><h3>The Total</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/948/1*FH_wWq6ljWYi0OU9C-Lkkw.png" /></figure><p>At low traffic (launch phase, a handful of daily users): <strong>about $10/month.</strong></p><p>At moderate traffic (50–100 assessments/day, mix of free and paid): <strong>about $40/month.</strong></p><p>The dominant cost is always Bedrock. Everything else is noise until you hit serious scale.</p><h3>What’s NOT in This Bill</h3><p>A few things that cost money but aren’t AWS:</p><ul><li><strong>Stripe fees:</strong> 1.5% + €0.25 per transaction (European cards). On a €4.49 payment, Stripe takes ~€0.32. That’s 7% — more than the entire AWS cost of serving that assessment.</li><li><strong>Domain registration:</strong> ~€10/year for .de</li><li><strong>Development tools:</strong> IDE, GitHub, etc.</li></ul><p>Stripe is actually the biggest “infrastructure” cost at low volume. A single Stripe transaction fee exceeds what AWS charges for the entire assessment pipeline.</p><h3>Where the Money Goes at Scale</h3><p>If this product grows to 1,000 paid assessments per day:</p><ul><li><strong>Bedrock</strong> becomes $300–500/month (still manageable — revenue would be $4,000–8,000/month)</li><li><strong>DynamoDB</strong> might warrant switching to provisioned capacity (~$50/month vs. on-demand)</li><li><strong>CloudFront</strong> transfer costs become real (~$20–50/month)</li><li><strong>Lambda</strong> concurrent executions might need reserved concurrency ($10–20/month)</li></ul><p>The serverless model means costs scale linearly with revenue. There’s no cliff where you suddenly need a $500/month RDS instance or a $200/month ECS cluster. Every dollar of AWS cost is directly tied to a user doing something.</p><h3>Key Takeaways</h3><ol><li><strong>Bedrock is 60–80% of the bill.</strong> Optimize here first. The tiered model approach (cheap models for free users, expensive models for paying users) is essential. Giving everyone Claude Opus would 100x the AI cost.</li><li><strong>Serverless free tiers are incredibly generous.</strong> Lambda, DynamoDB, CloudFront, Cognito, API Gateway, SES — all have free tiers that cover early-stage usage completely. Your first 1,000 users might cost $10/month total.</li><li><strong>Avoid OpenSearch Serverless.</strong> For RAG, Bedrock Knowledge Bases with S3 vector stores are dramatically cheaper than OpenSearch Serverless ($0 vs. $350/month minimum). Unless you need sub-millisecond vector search, don’t pay for it.</li><li><strong>Stripe costs more than AWS at low volume.</strong> This surprised me. A single payment processing fee exceeds the AWS cost of generating the entire AI assessment. Factor this into pricing.</li><li><strong>On-demand everything.</strong> DynamoDB on-demand, Lambda pay-per-invocation, no reserved capacity. At early stage, the flexibility is worth more than the ~20% savings from commitments. Optimize later when you have predictable traffic patterns.</li><li><strong>The $10/month SaaS is real.</strong> A fully functional AI-powered SaaS with authentication, payments, document processing, RAG, and CDN — running in production for the cost of two coffees. Serverless on AWS makes this possible in a way that wasn’t feasible five years ago.</li></ol><p><em>These numbers are from </em><a href="https://www.meinerechtshilfe.de/"><em>meinerechtshilfe.de</em></a><em>, a legal tech platform built entirely on AWS serverless. The stack is CDK-defined, fully automated, and runs without any servers to manage. If you’re planning a similar architecture, I hope this gives you realistic expectations for what the bill looks like — it’s probably cheaper than you think.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=49fd225c42c3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I Ditched My API Subdomain for CloudFront Path-Based Routing — Here’s What Broke (and How I Fixed…]]></title>
            <link>https://medium.com/@nils.rhode/i-ditched-my-api-subdomain-for-cloudfront-path-based-routing-heres-what-broke-and-how-i-fixed-5a82de62d98a?source=rss-f69b61e45ffb------2</link>
            <guid isPermaLink="false">https://medium.com/p/5a82de62d98a</guid>
            <dc:creator><![CDATA[Nils Rhode]]></dc:creator>
            <pubDate>Wed, 25 Feb 2026 06:21:00 GMT</pubDate>
            <atom:updated>2026-02-25T06:21:00.860Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>I Ditched My API Subdomain for CloudFront Path-Based Routing — Here’s What Broke (and How I Fixed It)</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*esGMS-immQr9l7c_34y4cw.png" /></figure><p><strong><em>When I moved my API from `api.meinerechtshilfe.de` to `meinerechtshilfe.de/api/`, I expected a 20-minute CDK change. Instead, I got a crash course in CloudFront’s most obscure restrictions. Here’s the full story so you don’t have to learn it the hard way.</em></strong></p><p>— -</p><p>If you’re running a serverless SaaS on AWS — React frontend on S3/CloudFront, API Gateway behind a custom domain — you’ve probably set it up the “standard” way: frontend on `www.example.com`, API on `api.example.com`. Two origins, two certificates, CORS headers everywhere.</p><p>It works. But it’s more infrastructure than you need. And especially more costs as you might pay in many cases for 2 requests (OPTION + GET)</p><p>I’m building <a href="https://www.meinerechtshilfe.de/">meinerechtshilfe.de</a>, a legal tech platform that uses AI to provide instant legal assessments in Germany. The stack is straightforward: React + Vite on S3, CloudFront in front, API Gateway with Lambda behind it, all wired up with CDK. When I decided to consolidate everything under one domain — routing `/api/*` to API Gateway and everything else to S3 — I thought it would be a clean afternoon task.</p><p>It wasn’t.</p><h3><strong>The Goal</strong></h3><p>Before:</p><blockquote><a href="http://www.meinerechtshilfe.de/">www.meinerechtshilfe.de</a> → CloudFront → S3 (frontend)</blockquote><blockquote>api.meinerechtshilfe.de → API Gateway → Lambda</blockquote><p>After:</p><blockquote><a href="http://www.meinerechtshilfe.de/">www.meinerechtshilfe.de</a> → CloudFront → S3 (frontend)</blockquote><blockquote><a href="http://www.meinerechtshilfe.de/api/*">www.meinerechtshilfe.de/api/*</a> → CloudFront → API Gateway → Lambda</blockquote><p>One domain. No CORS. No extra certificate. No subdomain DNS record. Simpler CSP headers.</p><p><strong>Step 1: Add API Gateway as a Second CloudFront Origin</strong></p><p>The CDK code is clean enough. Parse the API Gateway URL to extract the domain and stage, create an `HttpOrigin`, and add a behavior for `/api/*`:</p><pre>import re<br><br>match = re.match(r&#39;https://([^/]+)/([^/]+)/?&#39;, api_gateway_url)<br>api_domain = match.group(1)  # e.g. abc123.execute-api.eu-central-1.amazonaws.com<br>api_stage = match.group(2)   # e.g. prod<br><br>api_origin = origins.HttpOrigin(<br>    api_domain,<br>    origin_path=f&quot;/{api_stage}&quot;,<br>    protocol_policy=cloudfront.OriginProtocolPolicy.HTTPS_ONLY,<br>)</pre><p>The `origin_path=”/prod”` means CloudFront prepends `/prod` to every request before forwarding it. So a request to `/api/payments/checkout` becomes `/prod/api/payments/checkout` at the origin — which is wrong. API Gateway expects `/prod/payments/checkout`.</p><p><strong>Step 2: Strip the `/api` Prefix with a CloudFront Function</strong></p><p>CloudFront Functions run at the edge, before the request hits the origin. A tiny JavaScript function strips the prefix:</p><pre>strip_api_prefix_fn = cloudfront.Function(<br>    self,<br>    &quot;StripApiPrefixFunction&quot;,<br>    code=cloudfront.FunctionCode.from_inline(<br>        &quot;function handler(event) {\n&quot;<br>        &quot;  var request = event.request;\n&quot;<br>        &quot;  request.uri = request.uri.replace(/^\\/api/, &#39;&#39;);\n&quot;<br>        &quot;  if (request.uri === &#39;&#39;) request.uri = &#39;/&#39;;\n&quot;<br>        &quot;  return request;\n&quot;<br>        &quot;}&quot;<br>    ),<br>)</pre><p>Now `/api/payments/checkout` → URI becomes `/payments/checkout` → CloudFront prepends `origin_path=/prod` → API Gateway receives `/prod/payments/checkout`. Correct.</p><p><strong>Step 3: The Cache Policy Disaster</strong></p><p>API calls shouldn’t be cached. My first instinct was to create a custom `CachePolicy` with TTL=0 and forward all headers:</p><pre># DON&#39;T DO THIS<br>api_cache_policy = cloudfront.CachePolicy(<br>    self, &quot;ApiCachePolicy&quot;,<br>    default_ttl=Duration.seconds(0),<br>    min_ttl=Duration.seconds(0),<br>    max_ttl=Duration.seconds(0),<br>    header_behavior=cloudfront.CacheHeaderBehavior.allow_list(<br>        &quot;Authorization&quot;, &quot;Content-Type&quot;, &quot;Accept&quot;<br>    ),<br>)</pre><p>CloudFormation rejected it:</p><blockquote><em>”The parameter HeaderBehavior is invalid for policy with caching disabled.”</em></blockquote><p>When caching is disabled (all TTLs = 0), CloudFront doesn’t let you specify custom header behavior in the CachePolicy. It’s all-or-nothing.</p><p>The fix: use the built-in managed policy.</p><pre>api_cache_policy = cloudfront.CachePolicy.CACHING_DISABLED</pre><p><strong>Step 4: The Forbidden Headers</strong></p><p>Next, I needed an `OriginRequestPolicy` to forward headers like `Authorization` and `Content-Type` to API Gateway. My first attempt:</p><pre># DON&#39;T DO THIS EITHER<br>origin_request_policy = cloudfront.OriginRequestPolicy(<br>    self, &quot;ApiOriginRequestPolicy&quot;,<br>    header_behavior=cloudfront.OriginRequestHeaderBehavior.allow_list(<br>        &quot;Authorization&quot;, &quot;Content-Type&quot;, &quot;Accept&quot;,<br>        &quot;X-Amz-Date&quot;, &quot;X-Api-Key&quot;, &quot;X-Amz-Security-Token&quot;<br>    ),<br>)</pre><p>CloudFormation:</p><blockquote><em>”The parameter Headers contains X-Amz-Date that is not allowed.”</em></blockquote><p>CloudFront has a hidden blocklist. These headers cannot appear in an OriginRequestPolicy:</p><p>- `X-Amz-Date`</p><p>- `X-Api-Key`</p><p>- `X-Amz-Security-Token`</p><p>And `Authorization` can’t be in an OriginRequestPolicy either — it can only be forwarded via CachePolicy. But we’re using `CACHING_DISABLED`, which already forwards everything.</p><p>The real fix: use another managed policy.</p><pre>origin_request_policy = cloudfront.OriginRequestPolicy.ALL_VIEWER_EXCEPT_HOST_HEADER</pre><p><strong>Step 5: The Host Header Trap</strong></p><p>This was the sneakiest one. After deploying, I tested:</p><pre>curl -X POST https://www.meinerechtshilfe.de/api/payments/checkout \<br>  -H &quot;Content-Type: application/json&quot; \<br>  -d &#39;{&quot;test&quot;: true}&#39;</pre><p>Response: HTML. The S3 frontend’s `index.html`. Not JSON from API Gateway.</p><p>CloudFront returned `x-cache: Error from cloudfront`. The origin was correct (I verified with `aws cloudfront get-distribution-config`), the path pattern matched, but API Gateway was getting the wrong `Host` header.</p><p>When you use `ALL_VIEWER_AND_CLOUDFRONT_2022` as the origin request policy, it forwards the `Host` header from the viewer — which is `www.meinerechtshilfe.de`. API Gateway receives that, doesn’t recognize it as its own domain, and returns an error. CloudFront then falls back to the default behavior (S3), which serves `index.html`.</p><p>The fix: `ALL_VIEWER_EXCEPT_HOST_HEADER`. This forwards everything except `Host`, letting CloudFront set the correct origin host (`abc123.execute-api.eu-central-1.amazonaws.com`).</p><p><strong>The Final Working Configuration</strong></p><pre>api_behavior = {<br>    &quot;/api/*&quot;: cloudfront.BehaviorOptions(<br>        origin=api_origin,<br>        viewer_protocol_policy=cloudfront.ViewerProtocolPolicy.REDIRECT_TO_HTTPS,<br>        allowed_methods=cloudfront.AllowedMethods.ALLOW_ALL,<br>        cached_methods=cloudfront.CachedMethods.CACHE_GET_HEAD_OPTIONS,<br>        cache_policy=cloudfront.CachePolicy.CACHING_DISABLED,<br>        compress=True,<br>        origin_request_policy=cloudfront.OriginRequestPolicy.ALL_VIEWER_EXCEPT_HOST_HEADER,<br>        function_associations=[<br>            cloudfront.FunctionAssociation(<br>                function=strip_api_prefix_fn,<br>                event_type=cloudfront.FunctionEventType.VIEWER_REQUEST,<br>            )<br>        ],<br>    ),<br>}</pre><p>Three managed policies. One CloudFront Function. Zero custom policies.</p><h3><strong>What I Removed After the Migration</strong></h3><p>Once `/api/` goes through CloudFront on the same domain:</p><p>- <strong>CORS configuration</strong> on API Gateway — same-origin requests don’t need it</p><p>- <strong>API subdomain</strong> (`api.meinerechtshilfe.de`) — Route53 record, ACM certificate, API Gateway custom domain mapping</p><p>- <strong>CSP connect-src entries</strong> — no more `api.meinerechtshilfe.de` or `*.execute-api` in the Content-Security-Policy header</p><p>- <strong>Frontend environment variable</strong> — changed from `VITE_API_BASE_URL=https://api.meinerechtshilfe.de/` to just `VITE_API_BASE_URL=/api/`</p><p>The Lambda handlers still return `Access-Control-Allow-Origin: *` in their responses. It’s harmless and I left it for backward compatibility, but it’s technically unnecessary now.</p><h3><strong>Lessons Learned</strong></h3><p>1. <strong>Use managed policies first.</strong> CloudFront’s custom policy restrictions are poorly documented and will waste your time. `CACHING_DISABLED` + `ALL_VIEWER_EXCEPT_HOST_HEADER` covers 90% of API proxy use cases.</p><p>2. <strong>The Host header is the silent killer.</strong> If your API origin returns HTML or errors, check whether the viewer’s `Host` header is being forwarded. It almost certainly shouldn’t be.</p><p>3. <strong>CloudFront Functions &gt; Lambda@Edge for simple transforms.</strong> The `/api` prefix strip runs in ~1ms at the edge. No Lambda cold starts, no extra cost worth mentioning.</p><p>4. <strong>Test with curl, not the browser.</strong> Browsers cache aggressively and add headers. `curl -v` shows you exactly what CloudFront returns, including `x-cache` and `x-amz-*` headers that tell you which origin responded.</p><p>5. <strong>Invalidate after deploying.</strong> CloudFront caches aggressively. After changing behaviors: `aws cloudfront create-invalidation — distribution-id $DIST_ID — paths “/api/*”`</p><p>The whole migration took about 4 hours of debugging across multiple failed deployments. The final diff was maybe 30 lines of CDK code. Classic AWS.</p><p>— -</p><p><em>*This is part of the infrastructure behind [meinerechtshilfe.de](https://www.meinerechtshilfe.de), a serverless legal tech platform built with CDK, Lambda, Bedrock, and CloudFront. If you’re building something similar, I hope this saves you a few deployment cycles.*</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5a82de62d98a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a RAG Pipeline for German Legal Texts — What Generic Chunking Gets Wrong]]></title>
            <link>https://medium.com/@nils.rhode/building-a-rag-pipeline-for-german-legal-texts-what-generic-chunking-gets-wrong-d766323d0cc8?source=rss-f69b61e45ffb------2</link>
            <guid isPermaLink="false">https://medium.com/p/d766323d0cc8</guid>
            <dc:creator><![CDATA[Nils Rhode]]></dc:creator>
            <pubDate>Mon, 23 Feb 2026 14:41:00 GMT</pubDate>
            <atom:updated>2026-02-23T14:41:00.851Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*98FG5kMOZDXMs0ygXi_zpA.jpeg" /></figure><p><strong><em>Legal language is not like blog posts or documentation. Paragraphs reference other paragraphs. Definitions span multiple sections. A 300-token chunk can strip away the context that makes a legal provision meaningful. Here’s how I built a domain-specific RAG pipeline for German law using Amazon Bedrock Knowledge Bases.</em></strong></p><p>Retrieval-Augmented Generation sounds simple in theory: embed your documents, store the vectors, retrieve relevant chunks at query time, stuff them into the prompt. In practice, the quality of your RAG pipeline depends almost entirely on what you’re retrieving and how you chunked it.</p><p>I learned this the hard way while building <a href="https://www.meinerechtshilfe.de/">meinerechtshilfe.de</a>, a platform that provides AI-powered legal assessments for five areas of German law. The AI needs to reference actual statutes (BGB, StGB, ArbGG, etc.) and court decisions when answering user questions. Generic RAG with default settings produced answers that cited the right law but got the details wrong — because the retrieved chunks were missing critical context.</p><p>This article covers the architecture, the mistakes, and what actually worked.</p><h3>The Problem with Legal Text</h3><p>German legal texts have specific structural properties that break naive chunking:</p><p><strong>Cross-references everywhere.</strong> § 280 BGB (damages for breach of duty) references § 276 BGB (responsibility of the obligor) which references § 278 BGB (responsibility for third parties). A chunk containing § 280 without § 276 is incomplete.</p><p><strong>Definitions are separated from usage.</strong> The BGB defines terms in early sections and uses them hundreds of paragraphs later. If your chunk contains “Schuldverhältnis” (obligation) but the definition chunk wasn’t retrieved, the model might hallucinate the meaning.</p><p><strong>Numbered sub-paragraphs matter.</strong> § 573 BGB has paragraphs (1) through (4), each describing a different ground for lease termination. Retrieving paragraph (1) when the user’s question is about paragraph (3) produces a confident but wrong answer.</p><p><strong>Court decisions are long and structured.</strong> A BGH (Federal Court of Justice) decision can be 20+ pages. The holding (Leitsatz) is at the top, the reasoning (Entscheidungsgründe) is in the middle, and the specific legal interpretation might be buried in paragraph 47. Fixed-size chunking splits this arbitrarily.</p><h3>Architecture: One Knowledge Base Per Legal Category</h3><p>Instead of one giant knowledge base, I created six:</p><pre>CATEGORIES = [<br>    {&quot;id&quot;: &quot;privatrecht&quot;,    &quot;name&quot;: &quot;Privatrecht&quot;},     # Private/Contract Law<br>    {&quot;id&quot;: &quot;verkehrsrecht&quot;,  &quot;name&quot;: &quot;Verkehrsrecht&quot;},   # Traffic Law<br>    {&quot;id&quot;: &quot;arbeitsrecht&quot;,   &quot;name&quot;: &quot;Arbeitsrecht&quot;},    # Employment Law<br>    {&quot;id&quot;: &quot;mietrecht&quot;,      &quot;name&quot;: &quot;Mietrecht&quot;},       # Rental Law<br>    {&quot;id&quot;: &quot;sozialrecht&quot;,    &quot;name&quot;: &quot;Sozialrecht&quot;},     # Social Law<br>]<br># Plus one &quot;general&quot; KB for cross-cutting legal principles</pre><p>Each Knowledge Base has its own vector store and is backed by two shared S3 buckets — one for statutes (Gesetze), one for court decisions (Rechtsprechung). The bucket structure uses category prefixes:</p><pre>meinerechtshilfe-kb-gesetze/<br>  ├── arbeitsrecht/<br>  │   ├── ArbGG.pdf<br>  │   ├── KSchG.pdf<br>  │   └── ...<br>  ├── mietrecht/<br>  │   ├── BGB_Mietrecht.pdf<br>  │   └── ...<br>  └── ...<br><br>meinerechtshilfe-kb-rechtsprechung/<br>  ├── arbeitsrecht/<br>  │   ├── BAG_2024_001.pdf<br>  │   └── ...<br>  └── ...</pre><p>Why separate KBs instead of one with metadata filtering? Two reasons:</p><ol><li><strong>Precision.</strong> When a user asks about their landlord raising rent, I query only the mietrecht KB. No risk of retrieving employment law provisions that happen to share similar vocabulary (both use &quot;Kündigung&quot; — termination — but in completely different contexts).</li><li><strong>Independent data sources.</strong> Each KB has its own data source configuration. I can re-sync the employment law KB after adding new court decisions without touching the other four.</li></ol><h3>Embedding Model: Why Multilingual Matters</h3><pre>EMBEDDING_MODEL_ARN = &quot;arn:aws:bedrock:eu-central-1::foundation-model/cohere.embed-multilingual-v3&quot;</pre><p>German legal text is not English. It contains compound nouns (Mietminderungsanspruch = rent reduction claim), Latin phrases (pacta sunt servanda), and domain-specific abbreviations (BGB, StGB, ArbGG, BAG). Cohere’s multilingual embedding model handles German significantly better than English-only alternatives.</p><p>I tested with Amazon Titan Embeddings v2 initially. It worked for simple queries but struggled with compound legal terms. A query about “Kündigungsschutzklage” (unfair dismissal lawsuit) would retrieve chunks about “Kündigung” (termination) in general rather than the specific employment protection context. Cohere Multilingual v3 resolved this — it understands that the compound word is a specific legal concept, not just “termination” + “protection” + “lawsuit.”</p><h3>Chunking: The Tradeoff I Settled On</h3><pre>chunking_configuration = {<br>    &quot;chunking_strategy&quot;: &quot;FIXED_SIZE&quot;,<br>    &quot;fixed_size_chunking_configuration&quot;: {<br>        &quot;max_tokens&quot;: 300,<br>        &quot;overlap_percentage&quot;: 10<br>    }<br>}</pre><p>300 tokens with 10% overlap. This is a compromise.</p><p><strong>Why not semantic chunking?</strong> Bedrock Knowledge Bases support it, and it sounds ideal for legal text — chunk by meaning, not by token count. In practice, the semantic chunker struggled with German legal formatting. Statutes use a rigid numbering system (§, Abs., Nr., S.) that the chunker didn’t always respect. It would split a paragraph mid-sentence or merge unrelated sub-paragraphs.</p><p><strong>Why not larger chunks?</strong> I tested 500 and 1000 tokens. Larger chunks improved context completeness but hurt retrieval precision. A 1000-token chunk from the BGB might contain three different paragraphs. When the model cites “§ 573 BGB” from that chunk, it might be pulling from the wrong paragraph within the chunk. Smaller chunks with overlap gave better source attribution.</p><p><strong>Why 10% overlap?</strong> Legal paragraphs often start with references to the previous paragraph (“Abweichend von Absatz 1…” — “Deviating from paragraph 1…”). The overlap ensures these back-references aren’t orphaned.</p><h3>Parsing: Using Nova for Document Understanding</h3><pre>PARSING_MODEL_ARN = &quot;arn:aws:bedrock:eu-central-1:...:inference-profile/eu.amazon.nova-2-lite-v1:0&quot;</pre><p>Bedrock Knowledge Bases can use a foundation model to parse documents before chunking. This is critical for PDFs of German statutes, which often have complex formatting — multi-column layouts, footnotes, margin notes with paragraph numbers.</p><p>Nova 2 Lite handles the parsing step. It’s cheap enough to run on every document ingestion and smart enough to preserve the logical structure of legal text. Without it, the raw text extraction would lose paragraph numbering and merge footnotes into body text.</p><h3>Retrieval: Category Routing and Score Thresholds</h3><p>At query time, the Knowledge Base service determines which category to query based on the user’s question:</p><pre>class KnowledgeBaseService:<br>    def retrieve_relevant_context(<br>        self,<br>        question: str,<br>        max_results: int = 5,<br>        min_score: float = 0.7,<br>        specific_category: str = None<br>    ):<br>        # If user already selected a category, use it directly<br>        if specific_category:<br>            kb_id = self.knowledge_base_ids.get(specific_category)<br>        else:<br>            # Determine category from question content<br>            kb_id = self._route_to_category(question)<br><br>        # Retrieve from Bedrock Knowledge Base<br>        response = self.bedrock_agent_client.retrieve(<br>            knowledgeBaseId=kb_id,<br>            retrievalQuery={&#39;text&#39;: question},<br>            retrievalConfiguration={<br>                &#39;vectorSearchConfiguration&#39;: {<br>                    &#39;numberOfResults&#39;: max_results<br>                }<br>            }<br>        )<br><br>        # Filter by minimum relevance score<br>        results = [<br>            r for r in response[&#39;retrievalResults&#39;]<br>            if r[&#39;score&#39;] &gt;= min_score<br>        ]<br>        return results</pre><p>The min_score=0.7 threshold is important. Without it, the model receives low-relevance chunks that add noise. A chunk about traffic law fines scoring 0.4 against a rental law question does more harm than good — the model might try to incorporate it and produce a confused answer.</p><p>When no chunks pass the threshold, the system falls back to the model’s parametric knowledge. This is intentional. A confident answer without sources is better than a hedged answer citing irrelevant statutes.</p><h3>Prompt Construction: Context Window Management</h3><p>The retrieved chunks go into a structured prompt section:</p><pre>def build_legal_assessment_prompt(user_question, document_content, retrieved_context):<br>    prompt_parts = []<br><br>    # User&#39;s uploaded documents (if any)<br>    if document_content:<br>        prompt_parts.append(f&quot;UPLOADED DOCUMENTS:\n{document_content}&quot;)<br><br>    # RAG-retrieved legal context<br>    if retrieved_context:<br>        context_text = &quot;\n\n&quot;.join([<br>            f&quot;[Source: {doc[&#39;source&#39;]}]\n{doc[&#39;text&#39;]}&quot;<br>            for doc in retrieved_context<br>        ])<br>        prompt_parts.append(f&quot;RELEVANT LEGAL CONTEXT:\n{context_text}&quot;)<br><br>    prompt_parts.append(f&quot;USER QUESTION:\n{user_question}&quot;)<br><br>    return &quot;\n\n---\n\n&quot;.join(prompt_parts)</pre><p>The order matters. User documents come first (highest relevance — the user uploaded them for a reason), then RAG context (supporting legal references), then the question. The system prompt instructs the model to prioritize user documents over retrieved context and to cite specific paragraph numbers when referencing statutes.</p><h3>Source Extraction: Giving Users Verifiable References</h3><p>After the model generates its response, I extract legal source citations:</p><pre>sources = extract_legal_sources(<br>    response_text=response[&#39;completion&#39;],<br>    retrieved_docs=retrieved_docs<br>)</pre><p>This cross-references citations in the model’s response (like “§ 573 Abs. 2 Nr. 1 BGB”) with the retrieved chunks. If a cited statute was in the retrieved context, it gets a high confidence score. If the model cited something that wasn’t retrieved, it’s flagged as parametric knowledge — still shown to the user, but with lower confidence.</p><p>Users see these sources in the UI alongside the assessment. For a legal product, verifiability isn’t optional. Users need to be able to look up the cited paragraphs themselves or show them to a lawyer.</p><h3>What Didn’t Work</h3><p><strong>Single knowledge base with metadata filtering.</strong> Bedrock supports filtering by metadata attributes during retrieval. I tried tagging chunks with category: &quot;mietrecht&quot; and filtering at query time. The problem: embedding similarity doesn&#39;t respect category boundaries. A chunk about &quot;Kündigung&quot; in employment law would score high for a rental law question about lease termination, even with the filter. Separate KBs with separate vector stores eliminated this entirely.</p><p><strong>Large overlap percentages.</strong> I tried 20% and 30% overlap. It improved context continuity but created near-duplicate chunks in the vector store. The retriever would return 3 out of 5 results that were essentially the same text with slightly different boundaries, wasting the context window.</p><p><strong>Automatic category detection without user input.</strong> Early versions tried to classify the legal category purely from the question text. “My employer didn’t pay me” is clearly employment law. But “I signed a contract and the other party didn’t deliver” could be private law, commercial law, or consumer protection. Now the user selects a category upfront, and premium tiers get an AI-powered pre-classification that suggests the right category before submission.</p><h3>Results</h3><p>After tuning the pipeline:</p><ul><li><strong>Source accuracy</strong> improved from ~60% (generic RAG) to ~85% (category-specific KBs with score filtering). Measured by manually checking whether cited statutes were relevant to the question.</li><li><strong>Hallucinated citations</strong> dropped significantly. With generic RAG, the model would sometimes cite non-existent paragraphs. With category-specific retrieval and source cross-referencing, this became rare.</li><li><strong>Response quality</strong> (user ratings) correlated strongly with whether RAG context was available. Questions in well-covered legal areas (rental law, employment law) consistently scored higher than niche topics where the KB had sparse coverage.</li></ul><h3>What’s Next</h3><p><strong>Hierarchical chunking.</strong> Instead of flat 300-token chunks, I want to experiment with parent-child chunks — a large chunk (1000 tokens) for context, with smaller sub-chunks (200 tokens) for precise retrieval. Retrieve the small chunk, but inject the parent chunk into the prompt. Bedrock doesn’t support this natively yet, but it can be built with a custom retrieval layer.</p><p><strong>Incremental sync.</strong> Currently, re-syncing a knowledge base after adding new documents reprocesses everything. For a growing corpus of court decisions, this gets slow. A delta-sync approach that only processes new or modified documents would cut sync time significantly.</p><p><strong>Cross-category retrieval for complex cases.</strong> Some legal questions span multiple categories — a workplace injury might involve employment law, social law, and insurance law. The current architecture queries one KB at a time. A fan-out retrieval that queries multiple KBs in parallel and merges results would handle these cases better.</p><p><em>The RAG pipeline described here powers the legal assessments on </em><a href="https://www.meinerechtshilfe.de/"><em>meinerechtshilfe.de</em></a><em>. If you’re building domain-specific RAG — especially for structured, cross-referencing content like legal texts, medical guidelines, or regulatory documents — I hope the lessons here save you some iteration cycles. The generic defaults rarely work. The domain knowledge you bring to chunking, routing, and filtering is what makes RAG actually useful.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d766323d0cc8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How I Built a Multi-Tier AI Product on Amazon Bedrock — From Free to Premium in One Lambda]]></title>
            <link>https://medium.com/@nils.rhode/how-i-built-a-multi-tier-ai-product-on-amazon-bedrock-from-free-to-premium-in-one-lambda-c32a8edb87f1?source=rss-f69b61e45ffb------2</link>
            <guid isPermaLink="false">https://medium.com/p/c32a8edb87f1</guid>
            <dc:creator><![CDATA[Nils Rhode]]></dc:creator>
            <pubDate>Sat, 21 Feb 2026 21:31:07 GMT</pubDate>
            <atom:updated>2026-02-21T21:31:07.555Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Jzh6FmHucFHEzoUEkVcZGw.jpeg" /></figure><p><strong><em>Most AI SaaS products offer one model at one price. I needed four tiers with different models, token limits, and pricing — all running through the same Lambda function. Here’s how I structured it with Amazon Bedrock and a single config file.</em></strong></p><p>When you’re building an AI-powered product, the model is your biggest cost center. Giving every user Claude Opus is a fast way to burn through your AWS bill. But giving everyone the cheapest model means your premium tier feels no different from the free one.</p><p>I needed a middle ground: a tiered system where the AI model, token limits, and features scale with what the user pays. And I needed it to be dead simple to change — one config file, not scattered if-statements across the codebase.</p><p>This is how I built it for <a href="https://www.meinerechtshilfe.de/">meinerechtshilfe.de</a>, a platform that provides AI-powered legal assessments for German law across five specialized areas (employment, rental, traffic, private, and social law).</p><h3>The Tier Structure</h3><p>Four tiers, four Bedrock models, four price points:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QbL6Dzy2TioEvdpvxUYfFA.png" /></figure><p>The cost difference between Nova Lite and Claude Opus is roughly 80x per token. That margin is what makes the tiered model work financially.</p><h3>Single Source of Truth: One Config File</h3><p>Everything lives in packages.py. Model mapping, pricing, character limits, follow-up limits, Stripe prices, feature lists — all in one place:</p><pre># Model tier → Bedrock model ID mapping (eu-central-1 cross-region inference)<br>MODEL_TIERS = {<br>    &#39;free&#39;:      &#39;eu.amazon.nova-lite-v1:0&#39;,<br>    &#39;basic&#39;:     &#39;eu.amazon.nova-2-lite-v1:0&#39;,<br>    &#39;premium&#39;:   &#39;eu.anthropic.claude-sonnet-4-5-20250929-v1:0&#39;,<br>    &#39;unlimited&#39;: &#39;eu.anthropic.claude-opus-4-6-v1&#39;,<br>}<br><br>PACKAGES = {<br>    &#39;free&#39;: {<br>        &#39;char_limit_total&#39;: 200,<br>        &#39;page_limit&#39;: 0,<br>        &#39;follow_up_limit&#39;: 0,<br>        &#39;model_tier&#39;: &#39;free&#39;,<br>        # ...<br>    },<br>    &#39;one_time_standard&#39;: {<br>        &#39;char_limit_total&#39;: 500,<br>        &#39;page_limit&#39;: 2,<br>        &#39;follow_up_limit&#39;: 1,<br>        &#39;model_tier&#39;: &#39;basic&#39;,<br>        &#39;price_cents&#39;: 399,<br>        # ...<br>    },<br>    &#39;one_time_premium&#39;: {<br>        &#39;char_limit_total&#39;: 5000,<br>        &#39;page_limit&#39;: 15,<br>        &#39;follow_up_limit&#39;: 5,<br>        &#39;model_tier&#39;: &#39;premium&#39;,<br>        &#39;price_cents&#39;: 999,<br>        # ...<br>    },<br>    # ... and so on<br>}</pre><p>When the AI handler needs to know which model to use, it calls get_model_tier(purchase_type) and passes the result to the Bedrock client. No switch statements. No hardcoded model IDs in Lambda code.</p><h3>The Bedrock Client: One Class, Multiple Models</h3><p>The BedrockClient class resolves model tiers at init time and accepts a model_tier parameter on every call:</p><pre>class BedrockClient:<br>    NOVA_LITE_MODEL_ID = MODEL_TIERS[&#39;free&#39;]<br>    NOVA_2_LITE_MODEL_ID = MODEL_TIERS[&#39;basic&#39;]<br>    CLAUDE_SONNET_MODEL_ID = MODEL_TIERS[&#39;premium&#39;]<br>    CLAUDE_OPUS_MODEL_ID = MODEL_TIERS[&#39;unlimited&#39;]<br><br>    def invoke_claude(self, prompt, model_tier=None, **kwargs):<br>        # Resolve model ID from tier<br>        if model_tier == &#39;unlimited&#39;:<br>            model_id = self.CLAUDE_OPUS_MODEL_ID<br>        elif model_tier == &#39;premium&#39;:<br>            model_id = self.CLAUDE_SONNET_MODEL_ID<br>        elif model_tier == &#39;basic&#39;:<br>            model_id = self.NOVA_2_LITE_MODEL_ID<br>        else:<br>            model_id = self.NOVA_LITE_MODEL_ID<br><br>        # Same invoke path for all models<br>        response = self.client.invoke_model(<br>            modelId=model_id,<br>            body=json.dumps({...}),<br>        )<br>        return response</pre><p>The method is called invoke_claude for historical reasons (it started as Claude-only), but it handles all Bedrock models through the same Converse-style API. Nova and Claude both support the Messages API format, so the invocation code is identical — only the model ID changes.</p><h3>The Async Pattern: Why It Matters for Multi-Tier</h3><p>API Gateway has a hard 29-second timeout. Nova Lite responds in 3–5 seconds. Claude Opus can take 30–60 seconds for complex legal analysis with RAG context. If you call Bedrock synchronously, premium users hit timeouts.</p><p>The solution: async self-invocation.</p><pre>User submits question<br>    → Lambda validates, creates DynamoDB record (status: &quot;processing&quot;)<br>    → Lambda invokes ITSELF asynchronously (InvocationType: &#39;Event&#39;)<br>    → Returns assessmentId immediately (&lt; 2 seconds)<br><br>Async Lambda invocation<br>    → Retrieves documents, calls Bedrock (takes 10-60 seconds)<br>    → Updates DynamoDB record (status: &quot;completed&quot;)<br><br>Frontend polls GET /questions/{id} until status = &quot;completed&quot;</pre><pre># Fast path: return immediately<br>lambda_client = boto3.client(&#39;lambda&#39;)<br>lambda_client.invoke(<br>    FunctionName=os.environ.get(&#39;AWS_LAMBDA_FUNCTION_NAME&#39;),<br>    InvocationType=&#39;Event&#39;,  # Fire-and-forget<br>    Payload=json.dumps({<br>        &#39;action&#39;: &#39;process_assessment&#39;,<br>        &#39;assessmentId&#39;: assessment_id,<br>        &#39;purchaseType&#39;: purchase_type,<br>        # ... other params<br>    })<br>)<br><br>return {&#39;assessmentId&#39;: assessment_id, &#39;status&#39;: &#39;processing&#39;}</pre><p>The async handler picks up the action: &#39;process_assessment&#39; event, resolves the model tier from the purchase type, and calls Bedrock with the appropriate model. Free users get Nova Lite (fast, cheap). Premium users get Claude Sonnet (slower, much better). The frontend doesn&#39;t care — it just polls until the status flips.</p><h3>Resilience: Circuit Breaker for Bedrock</h3><p>Bedrock can throttle you, especially with Claude models during peak hours. The client wraps every call in a circuit breaker:</p><pre>class CircuitBreaker:<br>    def __init__(self, failure_threshold=5, timeout_seconds=60):<br>        self.failure_count = 0<br>        self.failure_threshold = failure_threshold<br>        self.state = &#39;closed&#39;  # closed = healthy<br>        self.last_failure_time = None<br>        self.timeout_seconds = timeout_seconds<br><br>    def call(self, func, *args, **kwargs):<br>        if self.state == &#39;open&#39;:<br>            if self._should_attempt_reset():<br>                self.state = &#39;half-open&#39;<br>            else:<br>                raise CircuitBreakerError(&quot;Circuit breaker is open&quot;)<br><br>        try:<br>            result = func(*args, **kwargs)<br>            self._on_success()<br>            return result<br>        except Exception as e:<br>            self._on_failure()<br>            raise</pre><p>After 5 consecutive failures, the circuit opens and all calls fail fast for 60 seconds. This prevents a cascade where one overloaded model takes down the entire Lambda function. Combined with exponential backoff retries (3 attempts, 1s → 2s → 4s), it handles transient Bedrock throttling gracefully.</p><h3>What I’d Do Differently</h3><p><strong>Model fallback chains.</strong> If Claude Opus is throttled, fall back to Claude Sonnet instead of failing. The user gets a slightly less powerful model, but they get a response. I haven’t implemented this yet, but the tier structure makes it straightforward — just walk down the MODEL_TIERS dict.</p><p><strong>Per-tier token budgets.</strong> Right now all tiers use max_tokens=4096 for the response. Premium and Unlimited users would benefit from higher limits (8K-16K), especially for complex multi-issue legal questions. The config structure supports it — I just need to add a max_response_tokens field per package.</p><p><strong>Cost tracking per tier.</strong> Bedrock returns token usage in every response. I store it in DynamoDB but don’t aggregate it per tier yet. Knowing that “Premium tier costs us €0.12 per assessment on average” would help with pricing decisions.</p><h3>The Economics</h3><p>A rough breakdown of Bedrock costs per assessment (including RAG retrieval):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3PEdLl_gsCzasJVcg9BHNg.png" /></figure><p>Free tier costs essentially nothing. Even Unlimited at €0.14 per assessment leaves healthy margin on an €8.49 price point. The real cost is in the RAG retrieval (Cohere embeddings + Knowledge Base queries), but that’s shared across all tiers and adds maybe €0.01 per request.</p><h3>Key Takeaways</h3><ol><li><strong>Centralize your config.</strong> One file that maps tiers to models, limits, prices, and features. Everything else reads from it. When you want to swap Nova Lite for a newer model, it’s a one-line change.</li><li><strong>Use async invocation for expensive models.</strong> API Gateway’s 29-second timeout is non-negotiable. Self-invoking Lambda with InvocationType=&#39;Event&#39; is the simplest pattern that works.</li><li><strong>The Bedrock Messages API is model-agnostic.</strong> Nova and Claude use the same request format. Your invocation code doesn’t need to know which model it’s calling.</li><li><strong>Tier your models by cost, not just capability.</strong> Nova Lite is “good enough” for a 200-character free preview. Don’t waste Claude tokens on it.</li><li><strong>Build resilience from day one.</strong> Circuit breakers and retries aren’t premature optimization when your product depends on a third-party AI service.</li></ol><p><em>This architecture powers </em><a href="https://www.meinerechtshilfe.de/"><em>meinerechtshilfe.de</em></a><em>, where users can get AI-powered legal assessments across five areas of German law. The multi-tier approach lets us offer a free preview that costs fractions of a cent while delivering premium Claude-powered analysis for paying users — all from the same Lambda function.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c32a8edb87f1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Unleashing the Power of Data: 10 Challenges on the Way towards a Data-Driven Organization]]></title>
            <link>https://medium.com/@nils.rhode/unleashing-the-power-of-data-10-challenges-on-the-way-towards-a-data-driven-organization-aadb5e6e56ec?source=rss-f69b61e45ffb------2</link>
            <guid isPermaLink="false">https://medium.com/p/aadb5e6e56ec</guid>
            <category><![CDATA[transformation]]></category>
            <category><![CDATA[organizational-culture]]></category>
            <category><![CDATA[data-driven]]></category>
            <category><![CDATA[cloud]]></category>
            <category><![CDATA[data]]></category>
            <dc:creator><![CDATA[Nils Rhode]]></dc:creator>
            <pubDate>Tue, 30 May 2023 04:01:42 GMT</pubDate>
            <atom:updated>2023-05-30T04:01:42.321Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>This article addresses 10 challenges when moving towards a data-driven organization and gives some advice on how to overcome them.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LY1u3c8bztE4eB5096bc4w.jpeg" /></figure><p>In today’s digital era, data has become the lifeblood of organizations, providing valuable insights and opportunities for growth. As businesses strive to become more data-driven, IT professionals play a crucial role in enabling this transformation. However, numerous challenges can impede progress towards building a truly data-driven organization. In this article, we will address these challenges and provide valuable advice and strategies for IT professionals to navigate the path to a data-driven organization successfully.</p><ol><li><strong>Mindset Shift<br></strong>Shifting employees’ mindset to prioritize data-driven decision making can be challenging. Many employees may be accustomed to relying on intuition or past experiences rather than data and evidence.<br>Foster a data-driven approach for decision-making, encourage employees to always question if relevant data exists and, if not, how to obtain it through research, data collection, or external sources.</li><li><strong>Leadership Example<br></strong>Creating a data-driven culture starts at the top. Leaders need to set an example by making decisions based on data and showing that it is the norm rather than the exception.<br>Establish key performance indicators (KPIs) to measure the success of decisions and use them as a feedback loop for continuous improvement and optimization.</li><li><strong>Metric Selection<br></strong>Choosing the right metrics and ensuring they align with business objectives is crucial. Leaders need to carefully select metrics that drive desired behavior and align with the organization’s goals.<br>When selecting metrics, prioritize those that directly contribute to the organization’s strategic goals and objectives. Ensure that the chosen metrics are actionable, measurable, and provide meaningful insights to drive behavior and inform decision-making. Regularly evaluate and update the metrics to adapt to changing business needs and priorities.</li><li><strong>Data Access<br></strong>Providing easy and timely access to data is essential for creating a data-driven culture. When employees struggle to access the data they need, it hinders their ability to make informed decisions based on data.<br>Establish a centralized data repository that ensures data availability and accessibility for employees across the organization. Implement robust data governance practices to ensure data quality, security, and privacy. Additionally, invest in user-friendly data visualization tools and training programs to empower employees to explore and analyze data independently.</li><li><strong>Collaboration<br></strong>Collaboration between data scientists and business leaders is crucial. Breaking down the barriers between these groups and fostering collaboration can help integrate data analysis into the decision-making process. Establish cross-functional teams where both groups work together on projects and initiatives.</li><li><strong>Training and Education<br></strong>Offering specialized training to employees in analytical concepts and tools just before they are needed can be more effective than generic, one-time training efforts. Training should be practical and focused on the specific needs of employees.</li><li><strong>Explaining Analytical Choices<br></strong>Encouraging teams to explain their analytical choices, consider alternatives, and evaluate tradeoffs helps foster a deeper understanding of the decision-making process and encourages critical thinking.</li><li><strong>Flexibility and Consistency<br></strong>Balancing flexibility and consistency in data analysis approaches can be a challenge. Standardizing metrics and programming languages across the organization can help reduce inconsistencies and improve collaboration.</li><li><strong>Value of Data<br></strong>Recognizing the business value of data is essential. Data has the potential to drive business results and increase profits, but its value is often overlooked or undervalued.</li><li><strong>Agile Data Practices<br></strong>Embracing agile practices and leveraging data to support business agility is crucial in today’s rapidly changing digital world. Organizations need to be able to respond quickly and flexibly to changing circumstances.</li></ol><p><strong>In conclusion</strong>, building a data-driven organization requires a multifaceted approach that starts at the top and permeates throughout the entire company. Leaders should set an example by making data-driven decisions and integrating data into decision-making processes. They should strategically choose metrics that align with business objectives and drive desired behaviors. It is crucial to break down the barriers between data scientists and business leaders, fostering collaboration and ensuring both groups understand each other’s language. Additionally, organizations should address data access issues promptly and provide employees with the necessary tools and training to analyze data effectively. Emphasizing the quantification of uncertainty, simplifying proofs of concept, offering just-in-time specialized training, empowering employees to handle data, promoting consistency over flexibility in data analysis approaches, and encouraging teams to explain their analytical choices all contribute to building a robust data-driven culture. By embracing these practices, organizations can harness the power of data to drive innovation, improve decision-making, and achieve sustainable success in the digital age.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=aadb5e6e56ec" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Navigating the Cloudscape: Unveiling Dynamic Cloud Migration Strategies for the Modern IT Landscape]]></title>
            <link>https://medium.com/@nils.rhode/navigating-the-cloudscape-unveiling-dynamic-cloud-migration-strategies-for-the-modern-it-landscape-12bd04e43b6d?source=rss-f69b61e45ffb------2</link>
            <guid isPermaLink="false">https://medium.com/p/12bd04e43b6d</guid>
            <category><![CDATA[cloud]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[cloud-migration]]></category>
            <category><![CDATA[amazon-web-services]]></category>
            <dc:creator><![CDATA[Nils Rhode]]></dc:creator>
            <pubDate>Sun, 21 May 2023 17:19:34 GMT</pubDate>
            <atom:updated>2023-05-21T17:19:34.698Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*a4KYIC-tzY5BfRfvGDtHMQ.jpeg" /></figure><p>In today’s digital landscape, cloud migration has become a crucial step for organizations seeking to leverage the benefits of cloud computing. However, migrating to the cloud is not a one-size-fits-all endeavor. Each organization has unique requirements, priorities, and existing IT infrastructure that influence the selection of a suitable migration strategy. In this chapter, we will delve into various cloud migration strategies, exploring their advantages, challenges, and best practices.</p><p>Before we embark on our exploration of cloud migration strategies, it’s essential to acknowledge the common tendency to limit the discussion to just two approaches: rebuilding in the cloud, following a cloud-native approach, or executing a “Lift and Shift.” This narrow focus is not surprising, as the former approach offers long-term cloud benefits but requires a significant upfront investment, while the latter provides a faster migration with lower upfront costs but may not deliver the expected long-term advantages. This shows the whole range from one extrem to another. However, it’s crucial to recognize that many factors contribute to the decision-making process, including the application’s importance, age, technical foundation, budgetary constraints, and organizational mindset. In order to provide a comprehensive overview, we will now delve into four additional cloud migration strategies, discussing each one in detail and offering advice on when to consider choosing a particular strategy.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/144/1*XlEkw1L9ay3C89vKROY4VQ.png" /></figure><p><strong>Lift and Shift Strategy</strong>: The lift and shift strategy, also known as rehosting, involves moving applications and data from an on-premises environment to the cloud with minimal changes. This strategy aims to quickly achieve the benefits of cloud infrastructure without significant redesign or modification. It is often suitable for legacy systems or scenarios where time is a critical factor. While this strategy offers a fast migration path, it may not fully leverage cloud-native capabilities and cost optimizations.<br><strong><em>Advice</em></strong>: Choose the lift and shift strategy when time is a critical factor, and there is a need for rapid migration to the cloud. It is suitable for legacy systems or scenarios where minimal changes are acceptable. However, ensure to conduct a thorough assessment of the target cloud environment and evaluate potential compatibility issues to avoid unforeseen challenges during the migration process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/144/1*uX4aJysFQtHt4FmR_Em5eQ.png" /></figure><p><strong>Replatforming Strategy</strong>: The replatforming strategy involves making minor adjustments to existing applications to take advantage of cloud-native features and services. This approach provides a middle ground between a lift and shift strategy and a full application redesign. By optimizing applications for the cloud environment, organizations can achieve improved scalability, performance, and cost efficiency. Replatforming typically involves modifying the application architecture, adopting managed services, or using platform as a service (PaaS) offerings.<br><strong><em>Advice</em></strong>: Opt for the replatforming strategy when you want to strike a balance between quick migration and leveraging cloud-native features. This strategy is ideal for organizations that want to optimize their applications for the cloud environment without significant redesign efforts. While making adjustments to the existing architecture, consider adopting managed services and PaaS offerings to enhance scalability, performance, and cost efficiency.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/144/1*odCnqoTOhsJhKDgs_XPsUw.png" /></figure><p><strong>Refactoring (Re-architecting) Strategy</strong>The refactoring strategy, also known as re-architecting, focuses on redesigning applications from the ground up to fully leverage cloud-native capabilities. This approach involves breaking down monolithic applications into microservices, adopting containerization, and using serverless computing. While refactoring requires more effort and resources, it enables organizations to fully exploit the benefits of scalability, agility, and resilience offered by the cloud. It also allows for more efficient cost management through resource optimization.<br><strong><em>Advice</em></strong>: Consider the refactoring strategy (re-architecting) when you aim to fully exploit cloud-native capabilities and achieve maximum scalability, agility, and resilience. Choose this strategy for applications that would benefit from breaking down monolithic structures into microservices, adopting containerization, and utilizing serverless computing. However, be prepared for substantial effort and resource investment, including comprehensive testing and validation of the redesigned architecture.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/144/1*BSEggx-rA3V2MSBXRQuSLQ.png" /></figure><p><strong>Repurchasing Strategy</strong>: The repurchasing strategy involves replacing existing applications with commercially available software-as-a-service (SaaS) solutions. Instead of migrating an application to the cloud, organizations opt to adopt a cloud-native SaaS application that fulfills their requirements. This approach offers several advantages, including reduced maintenance overhead, faster deployment, and access to the latest features. However, it may require data migration and integration efforts, as well as potential customization to align with specific business needs.<br><strong><em>Advice</em></strong>: Embrace the repurchasing strategy when existing applications can be replaced with cloud-native SaaS solutions that align with your organization’s needs. Prioritize SaaS solutions for non-core functionalities or when specialized expertise and continuous updates are crucial. Ensure thorough evaluation of available options, including integration capabilities, customization possibilities, and data migration requirements.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/144/1*fb7HxcmbsOLFtf1nFHJ8Fw.png" /></figure><p><strong>Retiring Strategy</strong>: The retiring strategy involves identifying and decommissioning applications or systems that are no longer needed or have become obsolete. By retiring these assets, organizations can streamline their cloud migration efforts, reduce complexity, and optimize costs. This strategy requires careful analysis to ensure that essential functionality is not compromised and that relevant data is safely archived or migrated to alternative solutions.<br><strong><em>Advice</em></strong>: Employ the retiring strategy to identify and decommission applications or systems that are no longer needed or have become obsolete. Conduct a comprehensive inventory assessment to identify redundant or underutilized assets. Prioritize retiring applications that no longer align with business goals or cannot be effectively migrated to the cloud. Safely archive or migrate relevant data to alternative solutions, considering data security and compliance requirements.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/144/1*zmnO1YSPUVRBtUHLWBvYbQ.png" /></figure><p><strong>Retaining Strategy</strong>: The retaining strategy, also known as the re-evaluating strategy, involves deciding to keep certain applications or systems in their current state, without migrating them to the cloud. This approach is typically chosen when an application has significant dependencies or when the cost and effort of migration outweigh the potential benefits. However, organizations must periodically re-evaluate such decisions, as evolving cloud technologies and changing business needs may warrant reconsideration.<br><strong><em>Advice</em></strong>: Consider the retaining strategy when migrating certain applications or systems to the cloud would introduce unnecessary complexity or when the cost and effort outweigh the potential benefits. Regularly reassess the decision to retain applications, as evolving cloud technologies and changing business needs may warrant reconsideration. Keep a close eye on advancements in cloud capabilities and evaluate future migration opportunities.</p><h3><strong>Conclusion</strong></h3><p>Cloud migration strategies play a pivotal role in enabling organizations to unlock the benefits of cloud computing. The choice of strategy depends on factors such as the organization’s goals, existing infrastructure, application complexity, and timeline. Whether opting for a lift and shift approach, replatforming, refactoring, repurchasing, retiring, or retaining, it is crucial to plan meticulously, engage stakeholders, and consider the long-term implications. By selecting an appropriate strategy and executing it effectively, organizations can embark on a successful cloud migration journey that enhances their agility, scalability, and overall competitiveness in the digital era.</p><p><em>Hint</em>: All public cloud providers have similiar information and even tools to help evaluate and analyze your current landscape. The decision of the migration strategy is always up to the business as it depends on the business impact and business strategy. It can be and should be influenced by the IT but it should be not a pure tech decision. The cloud providers support you in the migration, once you defined the overall strategy :)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=12bd04e43b6d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cloud migration — more than just a technology shift]]></title>
            <link>https://medium.com/@nils.rhode/cloud-migration-more-than-just-a-technology-shift-dd9362b6fa59?source=rss-f69b61e45ffb------2</link>
            <guid isPermaLink="false">https://medium.com/p/dd9362b6fa59</guid>
            <category><![CDATA[cloud-migration]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[cloud]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <dc:creator><![CDATA[Nils Rhode]]></dc:creator>
            <pubDate>Sun, 12 Jun 2022 20:08:44 GMT</pubDate>
            <atom:updated>2022-06-12T20:08:44.043Z</atom:updated>
            <content:encoded><![CDATA[<h3>Cloud migration — more than just a technology shift</h3><p><strong>There will be few companies in 2022 that have not already considered the cloud as a huge opportunity and chance. Cloud adoption is often a silent transformation until a certain point — and then boom.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7Eef8SJNbGTG8BxR1MH7Pw.jpeg" /></figure><p>Powered by a product team, IT department, or another tech-related business unit, the transformation typically begins quietly. An example would be the newly hired developer who joined the product team with previous cloud experience or the article in the press explaining the benefits of cloud computing.</p><p>It is common for silent transformations to increase on voice and after reaching a certain level, companies consider a cloud initiative to start a broader migration plan. Most often, the initiative focuses on how to transform certain technologies, products, or assets to the cloud. Cloud migration plans are about to be released. Typically, one component crucial to a successful transformation is ignored: the people and the organisation behind them. The foundation of any successful transformation and even more important for a cloud transformation.</p><blockquote>Cloud is not just a technology transformation, rather a mindset and behavioural change.</blockquote><p>The shift into the cloud is mostly founded on goals like:</p><ul><li>faster product development</li><li>more innovation and easier access to innovate</li><li>better pricing opportunities through pay-per-use</li><li>less, but better operations (reduce of business risks)</li><li>serving globally and many more…</li></ul><p><strong>But </strong>considering a public cloud like another, classical hosting provider will quickly turn into a painful transformation where wrong expectations will face reality. The benefits mentioned above will kick-in once the understanding of cloud-native is understood. It requires the acceptance of a certain vendor-lockin but brings all the benefits by leveraging cloud services. Using those allows the development team a new abstraction layer where they can focus more on the development of business features and less on operational tasks.</p><p>A short example to understand the various abstraction layer of cloud better:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Nz5dlrXAphhj4c9J6txddw.png" /></figure><p>Based on the goals of a cloud journey above, serverless would make the biggest impact. Consequently, it requires a total rethinking of how software developers build products. A change in programming language? A change in architecture e.g. towards event-driven architecture? A change in deployments and operations? Introducing new interfaces and integrating serverless components at an entirely new level…</p><p>A new mindset is definitely needed, more cloud-native, more agile, more DevOps… without the organization’s ability to grasp the shift in mindset, the cloud journey will end in frustration — a long time and significant financial investment for lift&amp;shift of the current assets into the cloud, with similar pain points as before and a high probability of higher “hosting” costs.</p><p>As already Albert Einstein said:</p><blockquote>“We cannot solve our problems with the same thinking we used when we created them.”</blockquote><p>How do you plan a strategic cloud migration in a complex socio-technical system? In the next chapters we will cover them one by one — first to understand the challenges ahead and how to plan for a cloud migration — what should anyone be aware of. We will talk about the various cloud migration strategies and when and how to decide for which. How to involve the organisation to have an exiting cloud journey.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dd9362b6fa59" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>