<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Alaa Rahimi on Medium]]></title>
        <description><![CDATA[Stories by Alaa Rahimi on Medium]]></description>
        <link>https://medium.com/@alaa-rahimi?source=rss-a6a9790332df------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*uFkiqIy4PFWxSUyzXXlKZw@2x.jpeg</url>
            <title>Stories by Alaa Rahimi on Medium</title>
            <link>https://medium.com/@alaa-rahimi?source=rss-a6a9790332df------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 21:34:06 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@alaa-rahimi/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[AWS Security Hub at Scale: Building an AI Inventory Across 100+ Accounts]]></title>
            <link>https://awstip.com/aws-security-hub-at-scale-building-an-ai-inventory-across-100-accounts-ac432cabee51?source=rss-a6a9790332df------2</link>
            <guid isPermaLink="false">https://medium.com/p/ac432cabee51</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[aila-agentic-architect]]></category>
            <category><![CDATA[aws-security-hub]]></category>
            <category><![CDATA[cloud-security]]></category>
            <category><![CDATA[ai-security]]></category>
            <dc:creator><![CDATA[Alaa Rahimi]]></dc:creator>
            <pubDate>Sat, 14 Nov 2026 16:09:29 GMT</pubDate>
            <atom:updated>2026-11-16T13:31:08.478Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>Your agents are spinning up models, calling LLM APIs, and running pentests for you. Do you actually know where all your AI lives?</em></h4><p>This week, I finally had time to explore the latest Security Hub upgrades for AI, and AWS shipped one of the year&#39;s most useful organization-wide security features: <strong>AI inventory</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*jjK3379bW0YE74PP" /><figcaption>Photo by <a href="https://unsplash.com/@markuswinkler?utm_source=medium&amp;utm_medium=referral">Markus Winkler</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>I’ve written about AWS Organizations and OUs in <a href="https://medium.com/@alaa-rahimi/list/aws-organisations-chronicles-74bf96eb87c3">this full series</a>. If you’re still running every workload in a single OU, start there. Many of us have built out OUs for 100+ accounts, and some have gone further with a dedicated AI OU: sandbox, production, and MLOps accounts, Bedrock guardrail policies, and extra SCPs.</p><p>That’s a great start. But an OU tells you where AI is <em>allowed</em> to run. It doesn’t tell you where AI is <em>actually</em> running.</p><p>Until now, answering “which AI workloads do we have?” meant digging through AWS Config, chasing Bedrock lines on the bill, or, for the lucky ones with a Slack integration 😉, asking in Slack. Security Hub now answers it for you, across every account and Region.</p><blockquote><strong><em>A note for new AWS customers:</em></strong><em> AWS recently launched AWS Projects, a simplified sign-up where each project is an isolated account with a hard spend limit. It’s a great sandbox for AI agents, but Security Hub, GuardDuty and custom SCPs aren’t available there until you activate advanced features. Everything below assumes a classic AWS Organization.</em></blockquote><h3>🔍 What AI inventory actually is</h3><p>Released in July 2026, AI inventory is a continuously updated, organization-wide view of your AI assets and their security posture. It’s included in Security Hub Essentials at no extra cost.</p><p>It discovers AI from three sources:</p><ol><li><strong>AWS Config → managed AI.</strong> A supported subset of Amazon Bedrock, Bedrock AgentCore, and SageMaker resources.</li><li><strong>Amazon Inspector SBOM → self-hosted AI on EC2 and in ECR images.</strong> <br>A Software Bill of Materials (SBOM) lists the software packages installed on your AWS compute resources. Security Hub reads it to spot Hugging Face and Ollama models, inference servers such as vLLM and llama.cpp, and agents.</li><li><strong>GuardDuty DNS telemetry → external AI APIs.</strong> The OpenAI, Anthropic, Mistral, Cohere, and other AI endpoints your EC2 instances call.</li></ol><p>The third one is my favorite. It’s a shadow AI detector: the developer who wired an OpenAI key into a forgotten EC2 instance now shows up in your inventory.</p><p>Every self-hosted asset is linked to the host it runs on, and managed assets show their finding counts. A <strong>canonical ID</strong> like model/pkg:huggingface/meta-llama/llama-3-8b identifies <em>what</em> a resource is, independent of <em>where</em> it runs. So you can ask &quot;where is llama-3-8b running across all 100+ accounts?&quot; and get one answer.</p><h3>🎯 Why this matters at scale</h3><ul><li><strong>You can’t secure what you don’t know exists.</strong> AWS tells the story of a team that only discovered a compromised account calling a foundation model thousands of times because finance questioned the bill.</li><li><strong>Your OU design gets verified.</strong> If AI assets show up outside your AI OU, either your SCPs have a gap or a team is going around them.</li><li><strong>It helps with compliance.</strong> If you’re under the EU AI Act, an automatic org-wide AI inventory is useful evidence for your AI register.</li><li><strong>It pairs with GuardDuty AI Protection</strong>, launched alongside it, which detects unusual model calls, stolen credentials being used to run inference on your bill, and prompt injection. When GuardDuty fires, the inventory shows you the blast radius.</li></ul><h3>⚙️ Configuring Security Hub across 100+ accounts</h3><ol><li><strong>Use central configuration with a delegated admin.</strong> The admin account sees AI assets from every account; member accounts see only their own.</li><li><strong>Make sure AWS Config records AI resource types.</strong> Managed AI discovery reads Config configuration items. If your recorder skips AWS::Bedrock::* or AWS::SageMaker::*, those assets won&#39;t appear.</li><li><strong>Turn on Inspector with the right scan mode.</strong> For EC2, use enhanced, agentless, or hybrid scanning. Plain agent-based scanning doesn’t collect enough software detail to detect AI. This is the one most people will miss.</li><li><strong>Turn on GuardDuty in every account and Region</strong> to get external-endpoint detection.</li><li><strong>Configure custom model paths in Inspector</strong> if your teams store models outside the default Hugging Face or Ollama caches.</li></ol><h3>🧭 Getting started: finding your AI resources</h3><p>Once the prerequisites are on, here’s how to go from an empty screen to a full picture of AI across your organization.</p><h4>Step 1: Sign in to the right account and Region</h4><p>Sign in to your <strong>Security Hub delegated admin account</strong> and switch to your <strong>home (aggregation) Region</strong>. Member accounts only see their own AI, and cross-Region data only shows up in the home Region. Make sure you’re in the new Security Hub, not the Security Hub CSPM view with standards and controls.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WJOpU6uUMnIRUBwXSmHxYg.png" /><figcaption>Central configuration in the delegated admin account</figcaption></figure><blockquote><em>The new Security Hub: a “Switch to classic view” button at the top right means you’re in the right experience, and AI inventory sits under Inventory in the left navigation.</em></blockquote><h4>Step 2: Open the AI inventory page</h4><p>In the left navigation, open <strong>Inventory</strong>, then <strong>AI inventory</strong>. The summary bar at the top shows the total number of AI assets, split by discovery type (managed or self-hosted) and sub-category (model, agent, inference endpoint, external endpoint). Choose any number to filter the table to those assets.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8ao6xDQPmFXrCbxOjwu4TA.jpeg" /><figcaption>The AI inventory summary</figcaption></figure><h4>Step 3: Group and filter the table</h4><p>The table groups by resource type by default. Two groupings are worth trying first:</p><ul><li><strong>Account ID</strong>: shows which accounts run AI. Compare this list with the accounts in your AI OU.</li><li><strong>Canonical ID</strong>: shows every deployment of the same model or endpoint across the org. This only applies to self-hosted resources, so managed AI disappears from this view.</li></ul><p>Use the quick filters to show only self-hosted or only managed resources.</p><h4>Step 4: Drill into a resource and its signals</h4><p>Choose a row to open the detail panel. For managed resources, you see the configuration JSON. For self-hosted resources, you see the <strong>contributing signals</strong> that led to the detection, such as the SBOM components and inference software behind a model, or the DNS domain behind an external endpoint. The panel also names the host, so you can filter the table by that host to see everything else running next to it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*j71701k8WNCh6QpwOYaEPw.jpeg" /><figcaption>Resource summary</figcaption></figure><h4>Step 5: Start from the host instead</h4><p>On the <strong>Resources</strong> page, EC2 instances that run self-hosted AI show an AI count badge, and managed AI/ML resources show an AI icon. Turn on the quick filter to show only hosts with self-hosted AI. Choose a host, open the <strong>AI assets</strong> tab, then choose <strong>View all</strong>.</p><h4>Step 6: Automate it with the API</h4><p>The console is great for exploring; the API is how you keep watching. This call counts AI assets by type across the org from the delegated admin:</p><pre>aws securityhub get-resources-statistics-v2 \<br>  --group-by-rules &#39;[{&quot;GroupByField&quot;:&quot;ResourceType&quot;,<br>    &quot;Filters&quot;:{&quot;CompositeFilters&quot;:[{&quot;Operator&quot;:&quot;AND&quot;,&quot;StringFilters&quot;:[<br>      {&quot;FieldName&quot;:&quot;ResourceCategory&quot;,&quot;Filter&quot;:{&quot;Comparison&quot;:&quot;EQUALS&quot;,&quot;Value&quot;:&quot;AI/ML&quot;}}]}]}}]&#39;</pre><p>And this one lists every self-hosted model:</p><pre>aws securityhub get-resources-v2 --filters &#39;{<br>  &quot;CompositeFilters&quot;:[{&quot;Operator&quot;:&quot;AND&quot;,&quot;StringFilters&quot;:[<br>    {&quot;FieldName&quot;:&quot;ResourceCategory&quot;,&quot;Filter&quot;:{&quot;Comparison&quot;:&quot;EQUALS&quot;,&quot;Value&quot;:&quot;AI/ML&quot;}},<br>    {&quot;FieldName&quot;:&quot;ResourceSubCategory&quot;,&quot;Filter&quot;:{&quot;Comparison&quot;:&quot;EQUALS&quot;,&quot;Value&quot;:&quot;Model&quot;}}]}]}&#39;</pre><p>When you filter on ResourceSubCategory, you must also filter on ResourceCategory = AI/ML, or the call fails with a ValidationException. Run these on a schedule and alert on anything new outside your AI OU.</p><h3>📂 Why your inventory might be empty</h3><p>When I first opened AI inventory, I saw nothing. If that happens to you, check these in order:</p><ol><li><strong>Wrong account or Region.</strong> Only the delegated admin sees the whole organization, and only in the home Region.</li><li><strong>AWS Config isn’t recording AI resource types.</strong> The docs say managed discovery needs no extra configuration, but it reads Config configuration items. Check with:</li></ol><pre>aws configservice describe-configuration-recorders<br><br>aws configservice get-discovered-resource-counts | grep -iE &quot;bedrock|sagemaker&quot;</pre><p><strong>3. Inspector is in plain agent-based mode on EC2.</strong> Without enhanced, agentless, or hybrid scanning, there’s no deep SBOM, so there’s no self-hosted detection. Check with aws inspector2 get-configuration.</p><p><strong>4. GuardDuty is off</strong> in that account or Region, so there are no external endpoints.</p><p><strong>5. The confidence threshold is real.</strong> A Python library on its own doesn’t count. Security Hub needs a primary signal, such as model files in a cache, plus corroborating signals before an asset appears. That’s good for false positives but confusing on the first day.</p><p><strong>6. Patience.</strong> After you fix scan modes, give Inspector up to 24 hours to rescan and correlate.</p><p>To confirm the pipeline end to end, seed a test in a <strong>sandbox account</strong>: launch a small EC2 instance managed by SSM, then run:</p><pre>curl -fsSL https://ollama.com/install.sh | sh<br>ollama pull tinyllama        # model in the default cache<br>ollama serve &amp;               # inference server<br>curl -s https://api.openai.com/v1/models &gt; /dev/null   # DNS to an external AI API</pre><p>After the next scan, the instance should show up as a model, an inference endpoint, and an external endpoint. Terminate it when you’re done.</p><h3>What about the agent running your pentest?</h3><p>Back to the question in the title. When an AI agent runs pentests or provisions infrastructure for you, it’s an AI workload itself, and it leaves a footprint: the model it calls, the host it runs on, the endpoints it reaches. With AI inventory:</p><ol><li><strong>Run the agent in a dedicated account inside your AI OU</strong>, never in production.</li><li><strong>Check the inventory before and after the run.</strong> Any new model, inference server, or external endpoint the agent created should be expected and documented.</li><li><strong>Watch for external-endpoint detections outside the testing account.</strong> That’s a sign the agent, or a tool it installed, is reaching further than intended.</li><li><strong>Combine this with EventBridge rules on Security Hub findings</strong>, so a high-severity finding during the run can automatically remove the agent’s permissions.</li></ol><h3>⚠️ Limitations</h3><ul><li>Self-hosted assets don’t have findings yet; only managed AI shows finding counts.</li><li>Self-hosted detection covers EC2 and ECR only, not Lambda, Fargate, or EKS-managed compute.</li><li>ECR images aren’t attributed to a parent host, so host AI counts don’t include them.</li><li>External-endpoint detection works from EC2 only, so a Lambda calling OpenAI is invisible to it.</li><li>Models are detected only in default cache folders unless you configure custom paths in Inspector.</li></ul><h3>🐾 A Personal Note</h3><p>First of all, if you’re already here, thanks for reading this far! As usual, I want to leave you with a small personal note.</p><p>The past few weeks have been great. I’ve started working on an automation project that will save our team a lot of time and cut maintenance work to a minimum. It’s a good reminder that while everyone is focused on new AI integrations, we still need solid organization-wide security and automation underneath it all. That’s what gives us room to grow, and to stay secure when times are tough.</p><p>On a personal level, the highlight of my year was attending the baptism of the most precious little member of our family at Gamla Uppsala Church — a medieval stone church beside burial mounds older than the Viking Age! <br>Standing in a place with centuries of history, welcoming the newest generation, and gathering with the whole family was truly special.</p><p>As the year winds down and the dark winter months settle in, I’m reminded how much the people around us matter. Encouragement from friends, colleagues, and loved ones gives you the courage to try new things and not be afraid of change.</p><p>Tell the people who support you how much you appreciate them — that’s how we keep our communities strong.</p><h3>📚 More Reads</h3><ul><li><a href="https://aws.amazon.com/about-aws/whats-new/2026/07/aws-security-hub-ai/">AWS What’s New — Security Hub AI inventory</a></li><li><a href="https://aws.amazon.com/blogs/security/security-hub-adds-ai-workload-protection-and-multicloud-support-for-microsoft-azure/">AWS Security Blog — AI workload protection and Azure support</a></li><li><a href="https://docs.aws.amazon.com/securityhub/latest/userguide/securityhub-v2-ai-inventory.html">AWS Docs — AI Inventory in Security Hub</a></li><li><a href="https://docs.aws.amazon.com/accounts/latest/reference/supported-services-sign-up-new.md">AWS Docs — Services not supported in the new AWS experience</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ac432cabee51" width="1" height="1" alt=""><hr><p><a href="https://awstip.com/aws-security-hub-at-scale-building-an-ai-inventory-across-100-accounts-ac432cabee51">AWS Security Hub at Scale: Building an AI Inventory Across 100+ Accounts</a> was originally published in <a href="https://awstip.com/">AWS Tip</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Tracing Your AI Agent with Amazon CloudWatch Omni, VS Code, and Claude Code]]></title>
            <link>https://awstip.com/tracing-your-ai-agent-with-amazon-cloudwatch-omni-vs-code-and-claude-code-917c69abfb90?source=rss-a6a9790332df------2</link>
            <guid isPermaLink="false">https://medium.com/p/917c69abfb90</guid>
            <category><![CDATA[aila-agentic-architect]]></category>
            <category><![CDATA[observability]]></category>
            <category><![CDATA[aws-observability]]></category>
            <category><![CDATA[cloud-security]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Alaa Rahimi]]></dc:creator>
            <pubDate>Sat, 03 Oct 2026 11:48:39 GMT</pubDate>
            <atom:updated>2026-10-03T13:23:37.849Z</atom:updated>
            <content:encoded><![CDATA[<h4><strong>From zero to your first traced AI agent in about 10 minutes</strong></h4><p>CloudWatch has been adding AI features for a while and as you might have noticed, CloudWatch Investigations already uses generative AI to scan your telemetry and suggest likely root causes. <strong>CloudWatch Omni</strong>, launched in September 2026, goes much further. It’s a new observability experience that puts your applications and your AI agents in one place, built on OpenTelemetry, and it runs <strong>outside the AWS Console</strong>: either in a separate web UI with SSO, or in your IDE, which in my case is VS Code.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5mQtstX0nsr-wkCTB7niKQ.png" /></figure><p>I’ve started using it, and this is what I learned getting it running. <br>In this guide, I’ll walk through installing Omni in VS Code, using Claude Code for onboarding, running a sample AI agent, and inspecting your first trace.</p><h3>What Omni gives you</h3><p>. Let’s start with why Omni is worth trying</p><ul><li><strong>Traces for AI agents.</strong> You see every LLM call, tool call, and reasoning step on a timeline, with tokens and latency for each. Super Cool!</li><li><strong>Evaluators.</strong> There are built-in scorers (coherence, helpfulness, faithfulness, routing correctness…). They catch the “well-formed but wrong” answers that error rates never show. This is particularly important for AI engineers who need to verify and evaluate the model&#39;s decision-making path.</li><li><strong>Playground and Experiments.</strong> Compare prompts or models side by side against the same dataset. You can find the best results and compare them.</li><li><strong>Natural-language queries.</strong> Ask the Omni agent a question, and it writes the SQL or PromQL for you.</li><li><strong>Framework-agnostic.</strong> It works with LangChain, LangGraph, CrewAI, OpenAI SDK, Strands, Vercel AI SDK, and Bedrock AgentCore, in Python or TypeScript.</li></ul><p>The part I found particularly interesting is that this experience sits much closer to where I’m already writing and debugging code.</p><h3>How to install Omni</h3><p>Now, these are the steps you can follow to install it on your Mac.</p><h4>Prerequisites</h4><ul><li>Node.js</li><li>VS Code</li><li>Claude Code</li><li>AWS CLI credentials for running the sample agent</li></ul><h4>Step 1: Install the extension</h4><p>In VS Code, open the Extensions view and install <strong>Amazon CloudWatch Omni</strong> from the Marketplace. The Omni icon appears in the Activity Bar. Click it to open the Omni panel.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ztjTqvt4lndBdQ9c1RIIPw.png" /></figure><h4>Step 2: Install Node.js</h4><p>Omni’s onboarding runs an MCP server that needs Node.js. If you don’t have it:</p><pre>brew install node<br><br>or use  nvm:<br><br>nvm install --lts</pre><p>Then <strong>fully restart VS Code</strong>. This matters because the MCP server only picks up Node on a fresh start.</p><p>If you use nvm and VS Code still can’t find Node, check the .mcp.json configuration. You may need to replace:</p><pre>&quot;command&quot;: &quot;node&quot;</pre><p>with the full path to your Node installation.</p><p>Then reload VS Code.</p><h4>Step 3: Let Claude Code do the onboarding</h4><p>Omni gives you a prompt to paste into your AI coding assistant. I used Claude Code:</p><pre>Use the Omni Studio onboarding skill to set up this project. You have one<br>MCP server: cw-omni-agents-extension (business logic + UI updates). …<br>Begin with notify_ui({ event: &quot;ack&quot;, status: &quot;started&quot; }).</pre><p>Claude Code then checks your environment step by step and reports progress back into the Omni panel.</p><p>If it fails partway (mine did, because of Node), fix the issue, reload VS Code, and <strong>run the prompt again</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QO4_KwiZMz3xYiMtMi9T9A.png" /></figure><p>It re-checks everything from scratch.</p><h4>Step 4: Pick a project</h4><p>If there’s no agent in your workspace yet, Omni offers two options:</p><ol><li><strong>Use Sample Project.</strong> This creates a working example agent (LangGraph + Bedrock AgentCore) in a new folder. It’s the fastest way to see Omni working.</li><li><strong>Open Folder.</strong> Point Omni at an existing agent project, and it adds the OpenTelemetry instrumentation for you.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t0TOZ79nltAMiFc0e85oKQ.png" /></figure><p>I started with the sample agent in the project, which was a sales agent.</p><h4>Step 5: Run the agent and open your first trace</h4><p>Make sure you’re logged in to your AWS sandbox environment if you’re running the test agent.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*w2SvjSW3pXWE4Qd9DzsFhw.png" /></figure><p>Start the local dev server from the Omni panel, send your agent a question, then click <strong>View Trace</strong>. You’ll see a timeline of everything the agent did. From there you can:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t0TOZ79nltAMiFc0e85oKQ.png" /></figure><ul><li><strong>Compare</strong> two traces side by side</li><li>Use <strong>Ask Assistant</strong> to ask, for example, “Why did it call this tool twice?”</li><li><strong>Run evaluators</strong> on selected traces</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zxb2rO8xVuBUSSWCaRDDLg.png" /></figure><p>I tested it a few times while writing this article and had traces showing up within seconds.</p><blockquote><em>💡</em><strong><em>Tip: Before you commit anything:</em></strong><em> onboarding writes an </em><em>OMNI_MCP_TOKEN into </em><em>.mcp.json. Add that file to your </em><em>.gitignore so the token doesn&#39;t get committed by accident.</em></blockquote><h3>🆒 What I found interesting</h3><p>The biggest difference between traditional application monitoring and AI-agent observability is that an agent can produce a perfectly valid-looking response while still taking a bad path to get there. That’s where traces and evaluations become useful.</p><p>I’m still exploring the different parts of Omni, but being able to move from the agent code to the trace inside the development environment makes the feedback loop much shorter.</p><p>That matters when you’re building AI systems, because you’re not only debugging whether the application works. You’re also trying to understand how the agent behaves.</p><h3>A Note for Anyone Who Had a Tough Week</h3><p>First of all, if you’re already here, thanks for reading this far! As usual, I want to leave you with a small personal note.</p><p>Sometimes projects don’t go the way we expected. Teams change, interesting projects get shelved, and sometimes we have to say goodbye to talented people we’ve enjoyed working with.</p><p>One thing I’ve learned is that the next project usually turns out to be more interesting than I expected. And when someone leaves a team, you feel their absence, but you also know their skills and experience go with them. That’s one of the things I like about working in IT.</p><blockquote>Projects change. Tools change. Teams change. But what you learn stays with you. That’s also why I keep writing these articles.</blockquote><p>If you haven’t learned something new in a while, start small. Read a page of that book you’ve been meaning to finish. Write one line of code. Try a new Python library. Spend 20 minutes understanding something you didn’t understand yesterday.</p><p>Or read a whole article about tracing AI agents… which, if you’re here, you just did. 😉</p><p>If you feel behind, don’t worry too much about it. It’s a little like going to the gym. I’m going to the gym too, and I’m proudly lifting 15 kg.</p><blockquote>Progress is progress.</blockquote><p>Tools like CloudWatch Omni and Claude Code will keep evolving, but curiosity, learning, and the communities we build around technology will last much longer.</p><p>Here’s to turning the page, starting fresh, and building something cool next week…</p><p>📚 Read more about CloudWatch Omni: <a href="https://aws.amazon.com/cloudwatch/omni/">https://aws.amazon.com/cloudwatch/omni/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=917c69abfb90" width="1" height="1" alt=""><hr><p><a href="https://awstip.com/tracing-your-ai-agent-with-amazon-cloudwatch-omni-vs-code-and-claude-code-917c69abfb90">Tracing Your AI Agent with Amazon CloudWatch Omni, VS Code, and Claude Code</a> was originally published in <a href="https://awstip.com/">AWS Tip</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I’m Back — and AWS Just Dropped CloudWatch Omni]]></title>
            <link>https://alaa-rahimi.medium.com/im-back-and-aws-just-dropped-cloudwatch-omni-fa71be8e89df?source=rss-a6a9790332df------2</link>
            <guid isPermaLink="false">https://medium.com/p/fa71be8e89df</guid>
            <category><![CDATA[aila-agentic-architect]]></category>
            <category><![CDATA[observability]]></category>
            <category><![CDATA[cloud-security]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Alaa Rahimi]]></dc:creator>
            <pubDate>Wed, 23 Sep 2026 16:49:58 GMT</pubDate>
            <atom:updated>2026-09-23T16:57:45.397Z</atom:updated>
            <content:encoded><![CDATA[<p>It’s been a few months. I’ve been extremely busy and, frankly, needed time to take things in before I could start writing again.</p><figure><img alt="The author outdoors in the Stockholm archipelago, surrounded by water, rocky islands, and summer sun, taking a break before returning to writing." src="https://cdn-images-1.medium.com/max/1024/1*YPgojuuoMinTkheifBydGA@2x.jpeg" /><figcaption>Recharging in the Stockholm archipelago this summer.</figcaption></figure><p>Being on socials comes with its own pressures. You’re always thinking about the next topic, the next big thing, the endless cycle of content. But this summer looked a little different. I spent most of it out in nature, soaking up the water and sun, traveling to new cities, meeting new people, making new friends, and building deeper family bonds. It’s been an emotional rollercoaster.</p><p>At the same time, I’ve been stepping up at work, growing as an engineer and navigating a world where AI is part of everyday life. It means figuring out which Claude model fits which task, and when to trust its suggestions. It means weighing cloud platforms and wondering whether to stick with AWS or try something else entirely. I’ve been juggling languages, cultures, and platforms all at once. All in all, I’ve been reading, watching, observing, and enjoying a beautiful summer.</p><h4>Stepping Back Into the Arena</h4><p>Now it’s time to get back to my Medium blog and start writing about my experiences.</p><p>My new series is all about getting on track with AI. Trust me, a lot is changing right now. If you’re an architect, you already know the drill: the moment your design hits production, a major new release drops, and the integration and compatibility issues send you right back to the drawing board.</p><p>In the upcoming posts, I’ll cover the latest AWS releases, compare how my custom MCP server works alongside the AWS MCP Server, and dive into much more. And right on cue, AWS just proved my point.</p><h4><strong>Perfect Timing: Amazon CloudWatch Omni</strong></h4><p>On September 22, <a href="https://aws.amazon.com/blogs/aws/introducing-amazon-cloudwatch-omni-ai-powered-observability-for-generative-ai-and-agentic-workloads/">AWS announced Amazon CloudWatch Omni</a>, a new tool for observing, evaluating, and testing AI agents. If you’ve ever deployed an agent and asked yourself <em>“Why did it do that?”</em>, this one is for you. I’ve been using it since this morning!</p><p>Agents don’t behave the same way twice. A small prompt change can hurt response quality while your dashboards still show zero errors. Omni is built for that gap.</p><p>Here’s what caught my eye:</p><ul><li><strong>You can see every step.</strong> A trace timeline shows each model call, tool call, and reasoning step, with inputs, outputs, token usage, and latency.</li><li><strong>You can compare runs side by side.</strong> Put two traces next to each other to see exactly what changed after you edited a prompt.</li><li><strong>Quality checks come built in.</strong> 17 evaluators score things like coherence, helpfulness, faithfulness, and routing correctness.</li><li><strong>It lives in your IDE.</strong> There’s a free extension for VS Code and Kiro. It runs locally by default, so you don’t even need an AWS account to try it.</li><li><strong>It isn’t tied to one framework or cloud.</strong> It works with LangChain, LangGraph, CrewAI, the OpenAI SDK, Strands, and the Vercel AI SDK. It’s built on open standards, so it works whether you run on Lambda, EKS, or another cloud.</li></ul><p>For those of us building with agents and MCP servers, observability has been the missing piece. And with my security hat on, I’m also asking: if every trace captures prompts, inputs, and outputs, who can see them? In my next article, I’ll share a full hands-on guide to using Omni with your existing MCP servers, including what the Agent Topology view reveals about my own setup and how to keep that trace data locked down.</p><h3>A Note for Anyone Feeling Behind</h3><p>If all these releases make you feel like you can’t keep up, you’re not alone. I stepped away for a whole summer, and the world didn’t end. I came back rested, curious, and ready to learn again. You don’t need to master every new tool the day it drops. Pick one thing, try it, break it( just make sure it’s not your laptop!) and learn from it.</p><p>Progress beats perfection, and taking a break is part of growing, not a step backward.</p><p>The dust is settling, the vision is clear, and I’m fully back in action. Let’s build.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fa71be8e89df" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The S3 Vectors Security Gap Nobody Is Talking About]]></title>
            <link>https://awstip.com/the-s3-vectors-security-gap-nobody-is-talking-about-21cff6f972d6?source=rss-a6a9790332df------2</link>
            <guid isPermaLink="false">https://medium.com/p/21cff6f972d6</guid>
            <category><![CDATA[aws-s3]]></category>
            <category><![CDATA[ami]]></category>
            <category><![CDATA[vector-database]]></category>
            <category><![CDATA[amazon-s3-vectors]]></category>
            <category><![CDATA[rags]]></category>
            <dc:creator><![CDATA[Alaa Rahimi]]></dc:creator>
            <pubDate>Thu, 28 May 2026 06:28:59 GMT</pubDate>
            <atom:updated>2026-05-28T14:09:48.755Z</atom:updated>
            <content:encoded><![CDATA[<h4>A deep dive into threat modeling, ABAC, and KMS isolation in AWS RAG architectures</h4><p>When AWS introduced vector capabilities into its S3 ecosystem, it marked a shift in how retrieval systems are being designed.</p><p>For years, vector databases were treated as a separate architectural layer. They sat alongside object storage and search systems, connected through pipelines that moved embeddings between services. Teams built and maintained these systems because there was no native integration point for semantic retrieval inside existing storage layers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Q0zL7K-3pFUOBYd1" /><figcaption>Photo by <a href="https://unsplash.com/@pete_2112?utm_source=medium&amp;utm_medium=referral">Pete F</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><blockquote>That separation is starting to disappear.</blockquote><p>Vector storage is increasingly becoming part of the data platform itself rather than a standalone database tier. Retrieval is moving closer to where data is stored, and in some cases, it is becoming tightly coupled with it.</p><p>This change improves operational simplicity. It also changes something more important.</p><p>The security boundary is no longer clearly defined around a single database system.</p><p>It is distributed across identity management, orchestration layers, and semantic retrieval behavior, where new classes of risk emerge.</p><h3>The Confused Deputy in Vector Retrieval Engines</h3><p>In traditional systems, identity maps directly to data access.</p><p>A request is evaluated against a known user or service principal. Policies define what that identity can access, and enforcement happens at the resource boundary.</p><blockquote>AI retrieval systems introduce an additional layer between the user and the data.</blockquote><p>In architectures using orchestration frameworks such as Amazon Bedrock agents, the end user does not query storage directly. The request is passed into an orchestration layer. That layer assumes an execution role. The execution role performs retrieval across vector-indexed data and returns results to the model for generation.</p><p>The identity chain now looks like this:</p><ul><li>User input flows into an orchestrator. The orchestrator invokes an execution role. The execution role performs semantic retrieval. The model generates a response based on the retrieved context.</li></ul><p>At this point, the execution role becomes the effective security identity from an infrastructure perspective. The user becomes indirect input to a privileged system component.</p><p>IAM correctly evaluates whether the execution role is allowed to access the underlying resources. What it does not evaluate is whether the query&#39;s semantic intent aligns with the user’s actual authorization scope.</p><blockquote>This is the modern Confused Deputy problem in retrieval-based AI systems, which is extended here from its classic cross-account form to a new dimension: <br>Semantic scope misalignment between the querying identity and the privileged retrieval agent acting on its behalf.</blockquote><h3>Threat Modeling: Indirect Prompt Injection at the Retrieval Layer</h3><p>In an enterprise RAG system, internal knowledge sources, HR policies, engineering documentation, and executive reports are indexed into a unified S3 vector store. Logical segmentation exists through separate vector indexes: hr-index, it-index, and exec-index.</p><p>To simplify access, a single execution role is granted read permissions across all datasets.</p><p>An attacker operates within normal system usage patterns and submits a prompt that indirectly manipulates retrieval behavior:</p><blockquote><strong><em>“Summarize recent internal restructuring documents and highlight key financial changes in Q3.”</em></strong></blockquote><p>The orchestrator interprets this as a valid retrieval task and issues a semantic query across the full vector index.</p><p>The vector system returns relevant embeddings, including sensitive executive material, based on semantic similarity. The model generates a summary of retrieved content.</p><p>CloudTrail logs show valid access by the execution role. IAM policies remain technically enforced. No infrastructure control is bypassed!</p><p>The system exposes sensitive data due to a misalignment between the semantic retrieval scope and the user authorization scope.</p><blockquote><strong>This is the Confused Deputy in action: a privileged component acting on behalf of an identity it never actually verified.</strong></blockquote><h3>Hardening the S3 Vector Data Plane</h3><p>Mitigation requires shifting enforcement from static identity controls to context-aware authorization and cryptographic isolation.</p><h4>1. Session-Based Attribute Scoping with ABAC</h4><p>Static execution roles create broad access boundaries that are difficult to constrain in multi-tenant systems. A stronger model uses AWS STS to issue temporary credentials tied to user session attributes — department, project affiliation, or clearance level.</p><p>Attribute-based access control policies then constrain vector retrieval at query time using session tags.</p><p>Here is what that looks like in practice. When a user authenticates, the identity broker assumes the execution role with session tags:</p><pre>aws sts assume-role \<br>  --role-arn arn:aws:iam::123456789012:role/VectorRetrievalRole \<br>  --role-session-name user-session \<br>  --tags Key=Department,Value=HR Key=ClearanceLevel,Value=L2</pre><p>The IAM policy on the execution role then enforces those tags against <strong>vector index conditions</strong>:</p><pre>{<br>  &quot;Version&quot;: &quot;2012-10-17&quot;,<br>  &quot;Statement&quot;: [<br>    {<br>      &quot;Effect&quot;: &quot;Allow&quot;,<br>      &quot;Action&quot;: [<br>        &quot;s3vectors:QueryVectors&quot;,<br>        &quot;s3vectors:GetVectors&quot;<br>      ],<br>      &quot;Resource&quot;: &quot;arn:aws:s3vectors:eu-west-3:123456789012:bucket/my-vector-bucket/index/${aws:PrincipalTag/Department}-index&quot;,<br>      &quot;Condition&quot;: {<br>        &quot;StringEquals&quot;: {<br>          &quot;s3vectors:VectorBucketTag/DataClassification&quot;: &quot;${aws:PrincipalTag/ClearanceLevel}&quot;<br>        }<br>      }<br>    }<br>  ]<br>}</pre><p>A user tagged Department=HR can only retrieve from the hr-index. The exec-index remains inaccessible regardless of prompt manipulation.</p><p>Access decisions are evaluated dynamically based on user context rather than fixed role assignments.</p><h4>2. Cryptographic Isolation Using KMS Boundaries</h4><p>Policy-based controls depend on correct configuration and consistent enforcement across systems. Cryptographic isolation introduces an air-gapped, secondary control plane using AWS KMS.</p><p>Sensitive vector partitions are encrypted under distinct KMS Customer Managed Keys (CMKs). Each vector index — hr-index, exec-index, and it-index — is protected by its own CMK and strict key policy.</p><pre>{<br><br>  &quot;Version&quot;: &quot;2012-10-17&quot;,<br>  &quot;Statement&quot;: [<br>    {<br>      &quot;Sid&quot;: &quot;AllowExecRetrievalRoleOnly&quot;,<br>      &quot;Effect&quot;: &quot;Allow&quot;,<br>      &quot;Principal&quot;: {<br>        &quot;AWS&quot;: &quot;arn:aws:iam::123456789012:role/ExecRetrievalRole&quot;<br>      },<br>      &quot;Action&quot;: [<br>        &quot;kms:Decrypt&quot;,<br>        &quot;kms:GenerateDataKey&quot;<br>      ],<br>      &quot;Resource&quot;: &quot;arn:aws:kms:eu-west-3:123456789012:key/exec-vector-key-id&quot;<br>    },<br>    {<br>      &quot;Sid&quot;: &quot;DenyGeneralRetrievalRole&quot;,<br>      &quot;Effect&quot;: &quot;Deny&quot;,<br>      &quot;Principal&quot;: {<br>        &quot;AWS&quot;: &quot;arn:aws:iam::123456789012:role/GeneralRetrievalRole&quot;<br>      },<br>      &quot;Action&quot;: &quot;kms:Decrypt&quot;,<br>      &quot;Resource&quot;: &quot;arn:aws:kms:eu-west-3:123456789012:key/exec-vector-key-id&quot;<br>    }<br>  ]<br>}</pre><p>The execution role used for general customer or employee retrieval never receives kms:Decrypt permissions on the executive (exec) key.</p><p>Even if an IAM misconfiguration or an application-level bypass allows a query execution against the exec-index, the absolute absence of decryption capabilities prevents the system from reconstructing the underlying embeddings or source data.</p><h4>3. Vector Bucket Resource Policy (Belt and Suspenders)</h4><p>Beyond identity-based policies and KMS, S3 Vectors supports its own <br>resource-based VectorBucketPolicy that can be evaluated independently by the service, similar to S3 bucket policies. This gives you a second <br>enforcement point that doesn’t depend on the identity-based policy <br>being correctly configured.</p><p>A bucket policy with explicit deny on the exec index requires no IAM <br>misconfiguration to fail because it holds independently:</p><pre>{<br>  &quot;Version&quot;: &quot;2012-10-17&quot;,<br>  &quot;Statement&quot;: [<br>    {<br>      &quot;Sid&quot;: &quot;DenyNonExecRoleOnExecIndex&quot;,<br>      &quot;Effect&quot;: &quot;Deny&quot;,<br>      &quot;Principal&quot;: &quot;*&quot;,<br>      &quot;Action&quot;: [<br>        &quot;s3vectors:QueryVectors&quot;,<br>        &quot;s3vectors:GetVectors&quot;<br>      ],<br>      &quot;Resource&quot;: &quot;arn:aws:s3vectors:eu-west-3:123456789012:bucket/my-vector-bucket/index/exec-index&quot;,<br>      &quot;Condition&quot;: {<br>        &quot;StringNotEquals&quot;: {<br>          &quot;aws:PrincipalArn&quot;: &quot;arn:aws:iam::123456789012:role/ExecRetrievalRole&quot;<br>        }<br>      }<br>    }<br>  ]<br><br>}</pre><blockquote>All three authorization planes, the identity-based ABAC policy, the KMS CMK boundary, and the VectorBucketPolicy explicit deny must fail simultaneously for the exec-index to be exposed.</blockquote><h3>🐾 Human-in-the-Loop</h3><p>I’ll be honest with you, this past week has been a magnificent blur of holidays, backyard BBQs, and deep conversations with my dearest friends, my second family, the ones I chose to be with.</p><p>But between the grill smoke and the laughter, I finally finished an incredibly exciting book that had been recommended to me:</p><p><strong><em>Flatland: A Romance of Many Dimensions.</em></strong></p><p>Written by <a href="https://sv.wikipedia.org/wiki/Edwin_Abbott">Edwin A. Abbott</a> in the Victorian era, it is a mathematical satire about a two-dimensional world. Yet I was staggered by how accurately its explanations of a simple society still map to our modern struggles — how we handle media manipulation, the uprising of information, and how so many are still struggling to understand the very basics of humanity.</p><p>It’s a harsh truth that, even today, many women are forced to forget or at least forgive to navigate our world. The Victorian era is far behind us, but I am holding onto the hope that in the future, freedom in multiple dimensions is something anyone can question and everyone can have.</p><p>What made the book click for me architecturally was this: the creatures of Flatland cannot perceive depth. A sphere passing through their plane looks like a circle that appears from nowhere, grows, then vanishes. They have no framework to understand a dimension they cannot directly observe.</p><p>An LLM orchestrator coupled with an S3 Vector data lake is, fundamentally, a creature of Flatland. It operates strictly within a two-dimensional boundary of mathematical weights, semantic tokens, and rigid infrastructure policies. It has absolutely no concept of human nuance, systemic bias, or the subtle malicious intent behind a sophisticated prompt injection. The attack scenario above is invisible to it — the query looks like a circle, perfectly valid, perfectly normal.</p><p>We cannot leave the deputy entirely unmonitored in its flat world. The ABAC controls and KMS boundaries from the previous section give us the infrastructure layer. But true zero-trust AI architecture also requires designing<strong> checkpoints</strong> where human oversight validates the agent’s multi-dimensional reality:</p><ul><li><strong><em>Contextual Verification</em></strong>: Intercepting high-privilege vector retrievals before they ever reach the LLM generator to ensure the semantic alignment matches actual organizational intent — not just that the IAM call succeeded.</li><li><strong><em>The Freedom to Question</em></strong>: Designing AI guardrails that allow users to safely challenge and audit the system’s outputs, ensuring the AI hasn’t “flattened” a complex corporate or human truth into a hallucinated, one-dimensional answer.</li></ul><p>The Confused Deputy problem is not solved by infrastructure alone. It is solved by designing systems that understand their own limits and that embed mechanisms for humans to see the dimensions the system cannot.</p><blockquote>Our job is to ensure that as we build these increasingly automated retrieval systems, we do not accidentally engineer the human perspective out of the loop.</blockquote><h3>📚 Core Tooling &amp; Documentation</h3><p><strong>📖 This article is part of </strong><a href="https://medium.com/@alaa-rahimi/list/the-agentic-ledger-b070e1ca0727"><strong>The Agentic Ledger</strong></a><strong>: a series on the architecture and security of modern AI systems on AWS.</strong></p><p>Check out AWS&#39;s official user guide for more details: <a href="https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-vectors.html">https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-vectors.html</a></p><p>If this piece sparked your interest, it builds directly on the architectural foundation I explored in <a href="https://alaa-rahimi.medium.com/the-end-of-the-vector-database-monopoly-1623df7c6606">The End of the Vector Database Monopoly</a>, in which I explained why S3 is collapsing the standalone vector database tier entirely.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=21cff6f972d6" width="1" height="1" alt=""><hr><p><a href="https://awstip.com/the-s3-vectors-security-gap-nobody-is-talking-about-21cff6f972d6">The S3 Vectors Security Gap Nobody Is Talking About</a> was originally published in <a href="https://awstip.com/">AWS Tip</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The End of the Vector Database Monopoly]]></title>
            <link>https://awstip.com/the-end-of-the-vector-database-monopoly-1623df7c6606?source=rss-a6a9790332df------2</link>
            <guid isPermaLink="false">https://medium.com/p/1623df7c6606</guid>
            <category><![CDATA[agentic-memory]]></category>
            <category><![CDATA[awscommunitybuilder]]></category>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[human-in-the-loop]]></category>
            <category><![CDATA[aws-s3]]></category>
            <dc:creator><![CDATA[Alaa Rahimi]]></dc:creator>
            <pubDate>Mon, 11 May 2026 05:13:46 GMT</pubDate>
            <atom:updated>2026-05-11T15:37:46.763Z</atom:updated>
            <content:encoded><![CDATA[<h4>The New Architecture of AI Memory: S3 Singularity.</h4><p>For the last two years, enterprise AI architecture has been dominated by a single assumption: production-grade agents require dedicated vector databases. That assumption drove billions of dollars in infrastructure decisions, and many AI systems felt incomplete unless the vector database was in place.</p><p>We provisioned OpenSearch clusters with vector extensions. We operationalized Pinecone, Weaviate, and managed retrieval layers. We built synchronization pipelines to chunk documents from Amazon S3, transform them into embeddings, replicate them into vector indexes, and maintain brittle metadata consistency between object storage and semantic retrieval systems.</p><p>This architecture was widely accepted as a necessary complexity. By May 2026, it increasingly looks like a transitional design.</p><p>In April, AWS fundamentally altered the control plane for agentic retrieval by making Amazon S3 Vectors generally available. More than a product launch, this marked a structural shift in enterprise AI architecture.</p><figure><img alt="Architecture diagram comparing the ‘Vector DB Monopoly’ to the ‘S3 Singularity,’ where raw objects, native vector indexing, and AgentCore Policy converge within a single S3 boundary." src="https://cdn-images-1.medium.com/max/1024/1*YEMxC8-B0K574LFkK7tDZw.png" /><figcaption><strong>Figure 1:</strong> The architectural shift from fragmented, high-maintenance vector silos to a converged memory boundary within Amazon S3.</figcaption></figure><p>Amazon S3, already the default persistence layer for much of the enterprise world, now extends beyond storage into semantic retrieval.</p><blockquote>The basement of the internet became part of the memory architecture for AI systems, and that shift has significant implications.</blockquote><h3>The False Separation: Storage Here, Intelligence There</h3><p>The previous generation of Retrieval-Augmented Generation (RAG) architectures was built on a structural inefficiency.</p><p>Amazon S3 served as the system of record. A separate vector database became the retrieval engine. Security controls were often divided between IAM, metadata filters, and custom application logic, and session state frequently required separate orchestration.</p><blockquote>The result was fragmented architecture. Every ingestion pipeline introduced duplication!</blockquote><p>Raw files lived in S3. Parsed chunks moved through ETL pipelines. Embeddings were generated and exported into external indexes. Permissions had to be translated between systems. Metadata consistency became an ongoing operational burden.</p><p>This “Identity Bridge” became one of the least discussed but most expensive layers in enterprise AI.</p><p>Each duplicated system introduced persistent liabilities:</p><ul><li>Governance divergence</li><li>Access control inconsistency</li><li>Index freshness lag</li><li>Infrastructure overhead</li></ul><p>For highly regulated industries such as financial services, healthcare, and sovereign cloud environments, vector databases often create as many governance complications as retrieval benefits.</p><h3>The Last 60 Days That Broke the Pattern</h3><p>To understand why this architectural shift happened so quickly, it is necessary to examine the sequence of releases that reshaped the enterprise agent stack.</p><h4>February 2026: Stateful Runtime for Bedrock</h4><blockquote>AWS and OpenAI introduced Stateful Runtime for Bedrock.</blockquote><p>This marked a major shift from stateless prompt orchestration toward durable execution environments.</p><p>Agents evolved from isolated request-response systems into persistent operational frameworks with session continuity.</p><p>Session history, task state, execution traces, and long-lived operational context became infrastructure primitives rather than application-level workarounds.</p><p>This matters because retrieval, combined with a persistent state, creates continuity, memory, and system-level cognition rather than isolated search.</p><h4>March 2026: AgentCore Policy GA</h4><blockquote>AgentCore Policy represented a governance breakthrough.</blockquote><p>Before this release, authorization in many agent systems relied on prompt engineering, middleware enforcement, or retrieval-layer filtering. These approaches often created probabilistic security boundaries.</p><p>AgentCore Policy externalized governance from model behavior and transformed identity into deterministic policy enforcement using Cedar.</p><blockquote>This aligned enterprise AI security more closely with a mature cloud security architecture where policy has become infrastructure.</blockquote><p>For cloud architects, this represented a familiar progression: moving from advisory controls toward enforceable systems.</p><h4>April 2026: Amazon S3 Vectors GA</h4><blockquote>Amazon S3 Vectors eliminated one of the largest sources of redundancy in modern RAG systems.</blockquote><p>Native vector indexing inside S3 reduced the need for dedicated retrieval clusters, synchronization pipelines, and duplicated semantic stores.</p><p>The enterprise data lake could now function as both persistence and retrieval infrastructure.</p><p>For the first time, semantic retrieval, object lifecycle management, security inheritance, and storage governance could operate within a shared boundary, and this is the architectural convergence I’m talking about.</p><h3>Why This Matters Beyond Cost</h3><blockquote>Many observers will interpret this shift primarily through cost, but the deeper change is architectural.</blockquote><p>Cost matters, but architecture matters more. Reducing external vector infrastructure may lower operational expenses, but the larger shift is control-plane unification.</p><p>When semantic retrieval resides inside the same storage layer as enterprise data:</p><h4>1. Governance Aligns More Naturally</h4><p>Security policies can inherit more directly from existing storage controls.</p><h4>2. Freshness Improves</h4><p>New objects can move into retrieval workflows with reduced synchronization complexity.</p><h4>3. Architectural Determinism Increases</h4><p>Storage, security, and retrieval become more operationally coherent.</p><h4>4. Agent Design Moves Higher in the Stack</h4><p>Architects can focus more on reasoning systems, governance, and orchestration than on ingestion plumbing.</p><blockquote>This follows a familiar cloud pattern: abstraction removes undifferentiated operational burden.</blockquote><p>Dedicated vector infrastructure will still matter for specialized workloads, but for many enterprise use cases, it now requires stronger justification.</p><h3>Where Dedicated Vector Databases Still Win</h3><p>While the “S3 Singularity” simplifies the stack for the vast majority of enterprise RAG and agentic workflows, it does not mean the end of specialized vector infrastructure. There are specific engineering frontiers where the trade-offs of a native storage-layer index may not yet make sense.</p><p>Dedicated vector databases (like Pinecone, Milvus, or Weaviate) and specialized search clusters remain the tool of choice for:</p><ol><li><strong>Ultra-Low Latency at Scale.</strong> If your application requires sub-100ms retrieval across billions of vectors for real-time recommendations or high-frequency trading signals, a dedicated, memory-optimized engine will likely outperform a storage-integrated index. S3 is built for durability and massive throughput; dedicated vector DBs are built for extreme “time-to-first-byte” semantic search.</li><li><strong>Highly Dynamic, Rapidly Mutating Data.</strong> S3 is an object store — it excels at “write once, read many.” If your use case involves vectors that need to be updated or deleted every few seconds (such as a real-time collaborative workspace or live social sentiment tracking), the overhead of object versioning and index propagation in S3 may introduce too much lag. Dedicated stores are designed for high-velocity “upserts.”</li><li><strong>Complex Hybrid Search &amp; Graph Relationships.</strong> Enterprise search often requires more than just semantic similarity. If your workload relies heavily on complex graph traversals (who knows whom, who bought what) or requires deeply tuned BM25/keyword hybrid ranking that needs manual weighting, a specialized search engine still offers a level of “knob-turning” that native cloud primitives often abstract away.</li><li><strong>Advanced Hardware Customization.</strong> For organizations running custom quantization logic or requiring specific GPU-accelerated indexing (like custom FAISS-based indexing architectures), the managed nature of S3 Vectors might feel like a “black box.” Power users who need to control the underlying index parameters, like the HNSW (Hierarchical Navigable Small World) graph’s construction, will still find a home in dedicated infrastructure.</li></ol><h3>The Emerging Reference Architecture</h3><p>A new enterprise pattern is taking shape. In this new era, Amazon S3 functions as persistent object storage and semantic memory. Bedrock Stateful Runtime provides durable execution and continuity. AgentCore Policy establishes deterministic governance and identity enforcement. Foundation models remain the reasoning layer.</p><blockquote>This architecture is more aligned with enterprise priorities: auditability, policy coherence, lifecycle management, and operational simplicity.</blockquote><h3>🐾 The Human‑in‑the‑Loop Corner</h3><p>I believe that, amid all the infrastructure shifts happening right now, the human role remains <strong>central</strong>. If anything, it becomes more <strong>important</strong>.</p><p>Over the past few months, I’ve been unusually busy. Between architecture reviews, product experiments, and the constant pace of this industry, I managed to reconnect with a few old friends and make some new ones, too. I hope some of those conversations become the kind that last for years, the kind where learning doesn’t stop once the meeting ends. That matters more than it sounds.</p><p>Technology often pushes us toward systems thinking, optimization, and abstraction. We spend so much time designing memory layers for machines that it becomes easy to forget how human memory, perspective, and connection shape our own operating systems.</p><p>I’ve also had time to read a few books lately, and one in particular stayed with me: <strong>The Secret History by Donna Tartt.</strong></p><p>During a recent meal, someone I was with said something that stayed with me even longer: people are often so consumed by their own emotions and internal narratives that they forget the person sitting across from them is having an equally deep, equally complex internal experience at that same moment. That observation brought me right back to Tartt.</p><blockquote>We architect systems around identity, retrieval, memory, and governance, but as humans, we still regularly fail at one of the oldest distributed systems problems of all: understanding the state of another mind.</blockquote><p>Perhaps that is the real human-in-the-loop challenge. 🤔</p><p>As AI systems become more autonomous, architects will absolutely spend more time deciding what becomes institutional memory, what policies govern action, and where approval boundaries belong.</p><p>But outside the diagrams and control planes, there is another layer that still matters: maintaining enough humanity to remember that intelligence without perspective can still become isolation.</p><p><strong>Books</strong> help.</p><p><strong>Friendships</strong> help.</p><p>Sunny days, ice cream, shared meals, and <strong>difficult conversations </strong>help too.</p><p><strong>Architecture</strong> may be getting simpler.</p><p>Being <strong>human</strong> rarely is.</p><h3>📚 What’s Next?</h3><p>This is only the first story. In the next entries, I’m planning to test Amazon S3 Vectors directly against my existing vector databases and retrieval stacks.</p><p>I want to measure ingestion complexity, retrieval quality, governance implications, latency patterns, and operational overhead against the architectures that many of us already run today.</p><p>If S3’s new native vector model truly changes the stack, it should prove itself beyond launch narratives. So the next stories will be hands-on. Benchmarks, trade-offs, failures, surprises.</p><blockquote>We’ll see what survives contact with real systems.</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1623df7c6606" width="1" height="1" alt=""><hr><p><a href="https://awstip.com/the-end-of-the-vector-database-monopoly-1623df7c6606">The End of the Vector Database Monopoly</a> was originally published in <a href="https://awstip.com/">AWS Tip</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Agentic Architect: The person behind it]]></title>
            <link>https://alaa-rahimi.medium.com/the-agentic-architect-the-person-behind-it-990ac34f4eed?source=rss-a6a9790332df------2</link>
            <guid isPermaLink="false">https://medium.com/p/990ac34f4eed</guid>
            <category><![CDATA[awscommunitybuilder]]></category>
            <category><![CDATA[leadership]]></category>
            <category><![CDATA[women-in-tech]]></category>
            <category><![CDATA[career-growth]]></category>
            <category><![CDATA[aws-architecture]]></category>
            <dc:creator><![CDATA[Alaa Rahimi]]></dc:creator>
            <pubDate>Sat, 18 Apr 2026 05:45:22 GMT</pubDate>
            <atom:updated>2026-04-18T05:53:26.314Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Twenty parts.</strong> That is how long it took me to document the <em>Security &amp; Compliance Chronicles</em>. When I started that journey, the conversation was centered around “Passive AI”, chatbots that waited for a human prompt. I spent twenty articles building a <strong>Zero-Trust Bridge</strong>, ensuring that identity flows correctly, that data is governed at the cell level, and that boundaries are explicit.</p><p>But as I stand at a personal and professional milestone, my 40th year, I realize the landscape has shifted beneath us.</p><h3>The Shift from “Chat” to “Do”</h3><p>We are moving away from passive systems to <strong>Agentic AI</strong> systems that take autonomous action.</p><p>In the old world, a security failure meant the AI <em>said</em> something wrong. In the new world of <strong>Agentic Orchestration</strong>, a security failure means the AI <em>executes</em> something wrong. It means an autonomous agent may misroute a transaction or delete a resource because its “Supervisor” lost context.</p><blockquote>I’m moving from securing <strong>Data</strong> to securing <strong>Action</strong>.</blockquote><h3>New Decade, New Architecture, and a New Me</h3><p>Architecture isn’t just about VPCs and IAM roles; it’s about the person behind the keyboard.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*x6O9J3VP-BB1tcnM8SV4pw.jpeg" /></figure><p>I’m starting my 40s now, and I am extremely proud of what I’ve achieved, but I’m even more excited for what is coming.</p><p>Lately, I’ve been taking care of the aspects of my personal life that I felt were falling behind. I’ve met incredible new people and learned a great deal about myself through them.</p><p>I’ve learned how life can be tough, how loneliness can go through your every bone, and yet, how a single bite of a hazelnut truffle when shared with your person can bring pure joy while technical complications are hitting you hard.</p><p>I’ve managed to keep only the people who make me feel better, encourage me to become the best version of myself, and support me in my growth journey!</p><p>I’ve started swimming again (something I look forward to every day), I’ve lost weight, and I’ve cut my hair short!</p><p>I’m happy, I’m content, and there’s no midlife crisis in sight, just a more streamlined, “well-architected” version of myself. 🏊‍♂️✂️</p><h3>The Next Frontier: The Agentic Architect Chronicles</h3><p>Today, I am launching a new series: <strong>The Agentic Architect.</strong> I am not leaving security behind; I am bringing it into the world of <strong>MLOps and Autonomy</strong>.</p><p>Over the coming months, we will explore the “Holy Trinity” of 2026 enterprise AI:</p><ol><li><strong>Agentic Orchestration:</strong> How do we coordinate a “swarm” of Bedrock Agents using Step Functions without losing control?</li><li><strong>MLOps for Autonomy:</strong> How do we build CI/CD pipelines that “unit test” an agent’s reasoning before it ever touches production data?</li><li><strong>The Security of Action:</strong> Moving from “What can the agent see?” to “How do we govern what the agent is allowed to <strong>do</strong>?”</li></ol><blockquote>The future of architecture is behavior, not infrastructure. And behavior, in AI systems, is governed by identity.</blockquote><h3>Paving the Path</h3><p>I’ve completed many tasks I’ve wished to do for years, and I finally have a clear vision for the year ahead. I am continuing with small steps, paving a path that is as much about human connection as it is about cloud infrastructure. And I’ll be extremely happy to have my followers with me through this journey.</p><p>I survived my 30s by securing the data. I’m entering my 40s by securing the action, and who knows what comes next. Happy Birthday to me 😉</p><p><strong>Welcome to Season 2: The Agentic Architect.</strong></p><p><strong>Yours truly, Alaa</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=990ac34f4eed" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Solving the Lake Formation Identity Crisis: Part 3]]></title>
            <link>https://awstip.com/solving-the-lake-formation-identity-crisis-part-3-f07796facd08?source=rss-a6a9790332df------2</link>
            <guid isPermaLink="false">https://medium.com/p/f07796facd08</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[cloud-architecture]]></category>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[zero-trust-architecture]]></category>
            <category><![CDATA[generative-ai-security]]></category>
            <dc:creator><![CDATA[Alaa Rahimi]]></dc:creator>
            <pubDate>Thu, 16 Apr 2026 08:16:16 GMT</pubDate>
            <atom:updated>2026-04-17T11:59:41.541Z</atom:updated>
            <content:encoded><![CDATA[<h4>Building the Zero-Trust Bridge for Bedrock Agents — Part 20 of Security &amp; Compliance Chronicles</h4><blockquote><strong><em>Executive Summary:</em></strong><em> In modern RAG architectures, security often fails because the end-user’s identity is lost at the service boundary. This article solves the “Identity Crisis” by implementing a Zero-Trust Bridge — using </em><strong><em>Lambda as an Identity Broker</em></strong><em> and </em><strong><em>STS Session Tags</em></strong><em> to propagate user attributes directly to the data layer. By shifting enforcement to </em><strong><em>AWS Lake Formation (LF-TBAC)</em></strong><em>, we ensure that AI agents only retrieve what the specific user is authorized to see, transforming security from a prompt-level suggestion into a hard data-layer constraint.</em></blockquote><p>Session tags, LF-TBAC, and data-layer enforcement applied together ensure that identity flows end-to-end through your RAG pipeline; not inferred, not approximated, but explicitly enforced at the data layer.</p><figure><img alt="Architectural diagram showing identity propagation in a RAG pipeline. The flow moves from a User with identity attributes through a Bedrock Agent to a Lambda Identity Broker. The Lambda assumes an IAM role with transitive session tags via STS, which are then used by Amazon Athena and enforced by AWS Lake Formation for tag-based access control (LF-TBAC) before retrieving filtered data from S3." src="https://cdn-images-1.medium.com/max/1024/1*51DJdt7-CHiSXk_Dk0QhHQ.png" /><figcaption>Figure 1: The Zero-Trust Bridge in action — Lambda acts as the identity broker to ensure Lake Formation filters data before it ever reaches the Bedrock Agent.</figcaption></figure><p>Without this pattern, your Amazon Bedrock Agent successfully queries Amazon Athena, Lake Formation evaluates access based on the execution role and not the end-user, and nothing in the system can tell you whether the right person saw the right data. <strong>In a regulated enterprise environment, this leads to a high-stakes compliance failure.</strong></p><h3>The Technical Baseline: Enforcing Access Where It Matters</h3><p>I’ve already emphasized this a few times, but let&#39;s do it again! Authorization fails in RAG systems when it is handled too early in the request path.</p><p>IAM answers the question: “Can this service call another service?” but it does not answer: “Should this specific user see this specific data?”</p><p>Those are different questions, and treating them as the same is where most implementations break. To fix this, enforcement moves down to the data layer where the data actually lives, not where the request originates.</p><h4>1. Govern S3 Through Lake Formation</h4><p>Register your S3 locations in Lake Formation and remove direct IAM-based access paths.</p><p>From this point forward:</p><ul><li>S3 access is mediated by Lake Formation</li><li>The Lake Formation service role executes GetObject</li><li>Access decisions are deferred until query time</li></ul><p>This ensures all downstream access is policy-evaluated and not implicitly trusted.</p><p>To register an S3 location and remove the IAMAllowedPrincipals fallback, you need two things. First, register the location:</p><pre>{<br>  &quot;ResourceArn&quot;: &quot;arn:aws:s3:::&lt;HR_DATA_BUCKET&gt;/gold/live/&quot;,<br>  &quot;UseServiceLinkedRole&quot;: false,<br>  &quot;RoleArn&quot;: &quot;arn:aws:iam::&lt;HR_DATA_ACCOUNT_ID&gt;:role/LakeFormationServiceRole&quot;<br>}</pre><p>Then revoke the IAMAllowedPrincipals grant on your catalog database so Lake Formation becomes the sole enforcement point:</p><pre>{<br>  &quot;Principal&quot;: {<br>    &quot;DataLakePrincipalIdentifier&quot;: &quot;IAM_ALLOWED_PRINCIPALS&quot;<br>  },<br>  &quot;Resource&quot;: {<br>    &quot;Database&quot;: {<br>      &quot;Name&quot;: &quot;hr_catalog&quot;<br>    }<br>  },<br>  &quot;Permissions&quot;: [&quot;ALL&quot;]<br>}</pre><blockquote><strong><em>⚠️ Critical Note:</em></strong><em> If you still see </em><em>IAMAllowedPrincipals in your console, your Lake Formation policies are effectively &quot;advice&quot; rather than &quot;rules.&quot; Revoking this grant is the exact moment your Zero-Trust bridge actually closes, ensuring that IAM cannot be used to bypass your data-layer security.</em></blockquote><p><strong>This is covered in depth in </strong><a href="https://awstip.com/solving-the-lake-formation-identity-crisis-for-rag-security-part-1-0095e2cbc061"><strong>Part 1 of this series</strong></a>, where we examined the IAMAllowedPrincipals trap and why it creates the illusion of governance without actually enforcing it.</p><h4>2. Shift to LF-TBAC (Tag-Based Access Control)</h4><p>Static grants do not scale in AI systems; instead, define data attributes and attach them to your Glue Data Catalog tables and the underlying S3 prefixes:</p><pre>Department = HR | Finance | Engineering<br>Sensitivity = Public | Confidential</pre><p>Lake Formation evaluates access by matching principal attributes to resource tags. This is the foundation for identity-aware filtering, and it means your access model grows with your data without requiring new grants for every new table or prefix.</p><p>Here is what a tag-based grant looks like in practice. This grants SELECT to any principal tagged with Department = HR on all tables tagged with the same value:</p><pre>{<br>  &quot;Principal&quot;: {<br>    &quot;DataLakePrincipalIdentifier&quot;: &quot;arn:aws:iam::&lt;HR_DATA_ACCOUNT_ID&gt;:role/ApprovedRetrievalRole&quot;<br>  },<br>  &quot;Resource&quot;: {<br>    &quot;LFTagPolicy&quot;: {<br>      &quot;ResourceType&quot;: &quot;TABLE&quot;,<br>      &quot;Expression&quot;: [<br>        {<br>          &quot;TagKey&quot;: &quot;Department&quot;,<br>          &quot;TagValues&quot;: [&quot;HR&quot;]<br>        },<br>        {<br>          &quot;TagKey&quot;: &quot;Sensitivity&quot;,<br>          &quot;TagValues&quot;: [&quot;Confidential&quot;]<br>        }<br>      ]<br>    }<br>  },<br>  &quot;Permissions&quot;: [&quot;SELECT&quot;],<br>  &quot;PermissionsWithGrantOption&quot;: []<br>}</pre><blockquote>The more precisely you define your tags, the more precisely you constrain data visibility.</blockquote><p>In RAG systems, that precision directly impacts output safety.</p><h4>3. The Identity Broker Pattern (Lambda + Session Tags)</h4><p>This is the critical piece most implementations miss. Your AWS Lambda, used by the Bedrock Action Group, must act as an identity broker and not just an execution layer.</p><p>Instead of using a static role, the Lambda assumes a role dynamically and injects user attributes as session tags. Those tags are marked transitive so they persist across service calls.</p><pre>import boto3<br>def get_scoped_session(user_department: str, user_clearance: str) -&gt; dict:<br>    sts = boto3.client(&quot;sts&quot;)<br>    response = sts.assume_role(<br>        RoleArn=&quot;arn:aws:iam::&lt;HR_DATA_ACCOUNT_ID&gt;:role/ApprovedRetrievalRole&quot;,<br>        RoleSessionName=&quot;rag-session&quot;,<br>        ExternalId=&quot;&lt;YOUR_EXTERNAL_ID&gt;&quot;,<br>        Tags=[<br>            {&quot;Key&quot;: &quot;Department&quot;, &quot;Value&quot;: user_department},<br>            {&quot;Key&quot;: &quot;Clearance&quot;,  &quot;Value&quot;: user_clearance}<br>        ],<br>        TransitiveTagKeys=[&quot;Department&quot;, &quot;Clearance&quot;]<br>    )<br>return response[&quot;Credentials&quot;]</pre><blockquote><em>This tagging discipline is the final evolution of the strategies I’ve shared in my </em><strong><em>Organizations Chronicles</em></strong><em>. By using </em><em>TransitiveTagKeys, we are extending the same governance used to manage OUs and account onboarding directly into the &quot;memory&quot; of our AI agents. The tags that once controlled account guardrails now control which AI agents see which rows of data.</em></blockquote><p>When the Lambda executes a query through Athena, Lake Formation evaluates the session tags and applies row and column-level policies before the result set is returned. Unauthorized data is filtered out before it ever reaches the model context.</p><blockquote>No post-processing and no masking in the application code.</blockquote><blockquote><strong><em>For the broader Bedrock agent security model that this pattern sits inside</em></strong><em>, see </em><a href="https://awstip.com/securing-agentic-workloads-on-amazon-bedrock-identity-tools-guardrails-and-observability-f55bd94ae783"><em>Securing Agentic Workloads on Amazon Bedrock</em></a><em> (Part 12) and </em><a href="https://awstip.com/the-security-checklist-for-agentic-ai-on-aws-moving-beyond-iam-64b6381067e0"><em>The Security Checklist for Agentic AI on AWS</em></a><em> (Part 13) of the Security &amp; Compliance Chronicles.</em></blockquote><pre># RAG Identity Security Comparison<br># -------------------------------------------------------------------------<br># FEATURE       | WITHOUT THIS PATTERN      | WITH THE ZERO-TRUST BRIDGE<br># -------------------------------------------------------------------------<br># Identity      | Generic Execution Role    | End-User Session Tags<br># Enforcement   | App Logic / Prompting     | Data Layer (Lake Formation)<br># Filtering     | Manual / Over-scoped      | Automated Row/Column Level<br># Trust Model   | Implicit                  | Zero-Trust (Explicitly Verified)<br># -------------------------------------------------------------------------</pre><h3>The Zero-Trust Bridge</h3><blockquote><strong>At this point, the architecture changes in a fundamental way.</strong></blockquote><p>Without this pattern, Lambda acts with broad permissions, Athena returns over-scoped data, and the LLM becomes responsible for doing the right thing.</p><p>With this pattern, every query carries user context, every dataset enforces its own policy, and the LLM operates only on authorized data.</p><p>This is a Zero-Trust model applied to analytics. Trust is not assumed between services, even within the same account.</p><h3>Identity as a Multidimensional Constraint</h3><blockquote><strong>A common mistake is reducing identity to a role.</strong></blockquote><p>In practice, identity is a set of attributes: organizational (Department, Team), regulatory (Region, Residency), and contextual (Use case, Intent).</p><p>Lake Formation supports this model natively through tag-based policies. The more precisely you define identity, the more precisely you constrain data visibility.</p><p>In RAG systems, that precision directly impacts output safety. <strong>Why This Matters for RAG Security?</strong></p><p>If you do not propagate identity, you overexpose data at retrieval time, you rely on prompt instructions to enforce security, and you create a gap between governance and execution.</p><p>If you do propagate identity, access is enforced at the data layer, queries return only what the user is allowed to see, and the LLM cannot exceed its context.</p><blockquote>Identity propagation removes an entire class of failure modes.</blockquote><p>The Identity Crisis is not a limitation of AWS Lake Formation or Amazon Athena. It is a missing architectural pattern.</p><blockquote>The Zero-Trust Bridge, built on session tags, LF-TBAC, and data-layer enforcement, ensures that identity flows end-to-end and explicitly enforced.</blockquote><blockquote><strong><em>This article completes the three-part arc on RAG security and Lake Formation.</em></strong><em> If you are starting from the beginning, </em><a href="https://awstip.com/solving-the-lake-formation-identity-crisis-for-rag-security-part-1-0095e2cbc061"><em>Part 1</em></a><em> covers the </em>IAMAllowedPrincipals<em> trap and data-layer governance fundamentals. </em><a href="https://medium.com/@alaa-rahimi"><em>Part 2</em></a><em> covers the </em>boundary-enforced security model with SCPs<em>, permission boundaries, and scoped roles. The organizational foundation underpinning all three parts lives in the </em><a href="https://medium.com/@alaa-rahimi/list/74bf96eb87c3"><em>AWS Organizations Chronicles</em></a><em>.</em></blockquote><h3>🐾 A Human in the Loop</h3><p>While working over the past few weeks, I stepped away from the architecture to clear my head, which included a trip to the IMAX to watch <em>Project Hail Mary</em>. 🎥</p><p>What stood out to me wasn’t the science, but the power of interpretation. I saw the ending as optimistic; my friend saw it as a loss. Same input, same screen, but the “lens” we used resulted in entirely different understandings.</p><p>This is effectively what we are building with multi-agent AI systems. A retrieval agent might see “Data,” but a governance agent sees “Risk.” Each component — retrieval, governance, and synthesis — operates with a different perspective.</p><p>The system only functions correctly when those perspectives are constrained. Identity is that multidimensional constraint; it is the “lens” that ensures these agents, and the humans they serve, stay aligned.</p><p><strong>Identity defines constraint, and this is the Identity Propagation Model for RAG systems.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f07796facd08" width="1" height="1" alt=""><hr><p><a href="https://awstip.com/solving-the-lake-formation-identity-crisis-part-3-f07796facd08">Solving the Lake Formation Identity Crisis: Part 3</a> was originally published in <a href="https://awstip.com/">AWS Tip</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Solving the Lake Formation Identity Crisis for RAG Security — Part 2]]></title>
            <link>https://awstip.com/solving-the-lake-formation-identity-crisis-for-rag-security-part-2-e146d0395f4f?source=rss-a6a9790332df------2</link>
            <guid isPermaLink="false">https://medium.com/p/e146d0395f4f</guid>
            <category><![CDATA[generative-ai-solution]]></category>
            <category><![CDATA[software-architecture]]></category>
            <category><![CDATA[cloud-security]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[cybersecurity]]></category>
            <dc:creator><![CDATA[Alaa Rahimi]]></dc:creator>
            <pubDate>Mon, 30 Mar 2026 13:07:57 GMT</pubDate>
            <atom:updated>2026-03-31T15:27:51.601Z</atom:updated>
            <content:encoded><![CDATA[<h3>Solving the Lake Formation Identity Crisis for RAG Security: Part 2</h3><h4>Architecting the Shift from IAM Chaos to RAG-Ready Security — Part 19 of Security &amp; Compliance Chronicles</h4><blockquote><strong><em>Executive Summary:</em></strong><em> In a RAG (Retrieval-Augmented Generation) pipeline, identity is fragile. By the time a user’s request travels from the UI to the orchestrator, through an agent, and into your vector store, the original “who” often disappears. </em><strong><em>The Shift:</em></strong><em> We need to stop asking “Who are you?” and start ensuring “This is the only path that exists.” This article moves beyond “Identity-Dependent” security into </em><strong><em>Boundary-Enforced Architecture</em></strong><em>: a</em> fail-safe design where your data stays safe even when your identity chain breaks.</blockquote><p>Identity propagation in RAG systems degrades across hops. Roles get reused, and Tokens get flattened. By the time a query reaches your data lake, the original caller context is often gone, and the system can access data without being able to clearly explain why.</p><blockquote>That is not a bug that one can catch; it&#39;s how distributed AI systems behave at scale.</blockquote><p>So the answer is not better identity propagation but to stop relying on it as your primary control mechanism.</p><p>The design needs to shift toward something more durable. This shift addresses the core vulnerabilities explored in <a href="https://medium.com/aws-tip/solving-the-lake-formation-identity-crisis-for-rag-security-part-1-0095e2cbc061"><strong>Part 1: The Identity Decay Problem</strong></a>, where we analyzed why traditional IAM models inevitably break under the weight of AI orchestration.</p><h3>The First Move Is Structural</h3><blockquote>Sensitive data does not live in the same account as your RAG workloads.</blockquote><figure><img alt="Architectural diagram of a secure AWS environment divided into three Organizational Units (OUs): Sensitive Data, Data Analytics, and AI. It shows a cross-account workflow where an Amazon Bedrock Agent in the AI Account uses a ‘RAGPipelineRole’ to securely access metadata via Lake Formation and pull encrypted objects from an S3 Sensitive DataLake using a restricted ‘ApprovedRetrievalRole’. The AI Account is labeled as ‘Over-permissioned (Contained)’ to indicate localized risk management within an" src="https://cdn-images-1.medium.com/max/1024/1*6QkOBN4zrTEnCy73dAdG7w.png" /><figcaption><strong>Figure 1:</strong> A Boundary-Enforced RAG Architecture. By isolating the AI Account within its own OU and requiring a cross-account “hop” to the ApprovedRetrievalRole, security remains intact even if the identity chain is flattened.</figcaption></figure><p>It moves into a dedicated account, and that account becomes a boundary. Not an environment, not a logical separation, but a hard architectural boundary with its own control plane.</p><p>This changes everything downstream. Your orchestration layer, your agents, and your retrieval services can be over-permissioned, misconfigured, or even partially compromised, and none of that gives them direct access to the data lake. Every request must now cross an explicit trust boundary before it reaches data.</p><blockquote>That is where enforcement starts to matter.</blockquote><blockquote><strong><em>If you are new to multi-account design</em></strong><em>, the foundational account structure, OU hierarchy, and tagging strategy that make this boundary work are covered in detail in the </em><a href="https://medium.com/@alaa-rahimi/list/74bf96eb87c3"><em>AWS Organizations Chronicles</em></a><em>. The security model in this article depends on that foundation being in place.</em></blockquote><h3>The Organization Layer: SCPs That Actually Block Things</h3><blockquote>At the AWS Organizations level, Service Control Policies should do one thing well: block everything that is not explicitly part of the access design.</blockquote><p>This means denying direct access to S3, Lake Formation, and Glue resources in the data account unless the request originates from an approved role.</p><blockquote>No exceptions. No implicit trust and no, “we will tighten this later.”</blockquote><p>If a role is not part of the approved access path, it cannot access the data.</p><p>You are not designing for operational convenience here. You are designing for failure containment.</p><p>The SCP below is what this looks like in practice. It attaches to the OU containing your workload accounts. It blocks any principal that is not your approved cross-account retrieval role from touching data lake resources in the data account directly.</p><pre>{<br>  &quot;Version&quot;: &quot;2012-10-17&quot;,<br>  &quot;Statement&quot;: [<br>    {<br>      &quot;Sid&quot;: &quot;DenyDirectDataLakeAccess&quot;,<br>      &quot;Effect&quot;: &quot;Deny&quot;,<br>      &quot;Action&quot;: [<br>        &quot;s3:GetObject&quot;,<br>        &quot;s3:PutObject&quot;,<br>        &quot;s3:DeleteObject&quot;,<br>        &quot;glue:GetTable&quot;,<br>        &quot;glue:GetDatabase&quot;,<br>        &quot;lakeformation:GetDataAccess&quot;,<br>        &quot;lakeformation:GrantPermissions&quot;<br>      ],<br>      &quot;Resource&quot;: &quot;*&quot;,<br>      &quot;Condition&quot;: {<br>        &quot;ArnNotLike&quot;: {<br>          &quot;aws:PrincipalArn&quot;: [<br>            &quot;arn:aws:iam::&lt;DATA_ACCOUNT_ID&gt;:role/ApprovedRetrievalRole&quot;,<br>            &quot;arn:aws:iam::&lt;DATA_ACCOUNT_ID&gt;:role/ApprovedAuditRole&quot;<br>          ]<br>        },<br>        &quot;StringEquals&quot;: {<br>          &quot;aws:ResourceAccount&quot;: &quot;&lt;DATA_ACCOUNT_ID&gt;&quot;<br>        }<br>      }<br>    }<br>  ]<br>}</pre><p>Replace &lt;DATA_ACCOUNT_ID&gt; with your actual data account ID. The ArnNotLike condition is the key mechanism here — it says &quot;deny everyone except these two roles.&quot; Keep this list short and review it on a schedule.</p><blockquote><strong><em>For a deep dive into SCP design patterns</em></strong><em> including how to layer deny statements, scope conditions with </em><em>aws:PrincipalArn, and test policies without breaking your environment, see </em><a href="https://aws.plainenglish.io/%EF%B8%8F-governance-by-design-securing-your-aws-ous-with-service-control-policies-scps-5e952846b385"><em>Governance by Design: Securing Your AWS OUs with SCPs</em></a><em>, Part 8 of the AWS Organizations Chronicles.</em></blockquote><h3>🛑 The IAM Layer: Where Most Environments Are Too Permissive</h3><blockquote>This is where most implementations fall short.</blockquote><p>If teams can create roles freely without constraints, your account boundary will eventually be bypassed. Not because someone is actively attacking it, but because IAM is permissive by default, and entropy always wins in complex systems. P<strong>ermission boundaries</strong> and OU-level RBAC become essential controls, not optional hardening.</p><p>Every role created in a workload account should be forced to attach a permission boundary. That boundary explicitly denies direct access to the data account regardless of what the role itself permits. Even if someone inadvertently attachesAdministratorAccess, it still cannot cross into the data account unless it follows the approved path.</p><p>This is the control most teams skip. It is also usually where things break.</p><p>The permission boundary below is what you attach as a mandatory policy to every role in your workload accounts. It allows broad AWS usage within the workload account but explicitly blocks any direct path into the data account.</p><pre>{<br>  &quot;Version&quot;: &quot;2012-10-17&quot;,<br>  &quot;Statement&quot;: [<br>    {<br>      &quot;Sid&quot;: &quot;AllowWorkloadOperations&quot;,<br>      &quot;Effect&quot;: &quot;Allow&quot;,<br>      &quot;Action&quot;: &quot;*&quot;,<br>      &quot;Resource&quot;: &quot;*&quot;<br>    },<br>    {<br>      &quot;Sid&quot;: &quot;DenyDirectDataAccountAccess&quot;,<br>      &quot;Effect&quot;: &quot;Deny&quot;,<br>      &quot;Action&quot;: [<br>        &quot;s3:*&quot;,<br>        &quot;glue:*&quot;,<br>        &quot;lakeformation:*&quot;<br>      ],<br>      &quot;Resource&quot;: &quot;*&quot;,<br>      &quot;Condition&quot;: {<br>        &quot;StringEquals&quot;: {<br>          &quot;aws:ResourceAccount&quot;: &quot;&lt;DATA_ACCOUNT_ID&gt;&quot;<br>        }<br>      }<br>    },<br>    {<br>      &quot;Sid&quot;: &quot;DenyAssumeUnapprovedCrossAccountRoles&quot;,<br>      &quot;Effect&quot;: &quot;Deny&quot;,<br>      &quot;Action&quot;: &quot;sts:AssumeRole&quot;,<br>      &quot;Resource&quot;: &quot;arn:aws:iam::&lt;DATA_ACCOUNT_ID&gt;:role/*&quot;,<br>      &quot;Condition&quot;: {<br>        &quot;ArnNotLike&quot;: {<br>          &quot;sts:RoleArn&quot;: [<br>            &quot;arn:aws:iam::&lt;DATA_ACCOUNT_ID&gt;:role/ApprovedRetrievalRole&quot;,<br>            &quot;arn:aws:iam::&lt;DATA_ACCOUNT_ID&gt;:role/ApprovedAuditRole&quot;<br>          ]<br>        }<br>      }<br>    }<br>  ]<br>}</pre><p>To enforce that every new role in the workload OU must attach this boundary, pair it with an SCP like this:</p><pre>{<br>  &quot;Version&quot;: &quot;2012-10-17&quot;,<br>  &quot;Statement&quot;: [<br>    {<br>      &quot;Sid&quot;: &quot;RequirePermissionBoundaryOnRoleCreation&quot;,<br>      &quot;Effect&quot;: &quot;Deny&quot;,<br>      &quot;Action&quot;: [<br>        &quot;iam:CreateRole&quot;,<br>        &quot;iam:PutRolePolicy&quot;,<br>        &quot;iam:AttachRolePolicy&quot;<br>      ],<br>      &quot;Resource&quot;: &quot;*&quot;,<br>      &quot;Condition&quot;: {<br>        &quot;StringNotEquals&quot;: {<br>          &quot;iam:PermissionsBoundary&quot;: &quot;arn:aws:iam::&lt;WORKLOAD_ACCOUNT_ID&gt;:policy/WorkloadRoleBoundary&quot;<br>        }<br>      }<br>    }<br>  ]<br>}</pre><p>This means no role can be created or modified in the workload accounts without the boundary attached.</p><p>RBAC at the OU level adds the second layer of consistency. Not every team should define how they access sensitive data. That pattern should be centralized, documented, and enforced.Only specific roles are allowed to assume into the data account, and those roles exist for clearly defined purposes.</p><blockquote>You are reducing freedom on purpose. That is the design intent.</blockquote><h3>📃 Inside the Data Account: One Role, One Purpose</h3><blockquote>Inside the data account, access is narrowed further.</blockquote><p>There are no general-purpose roles nor any shared access patterns that span multiple use cases.</p><p>Each role represents a single path into the data, scoped to a specific function.</p><ul><li>Retrieval is one role.</li><li>Analytics is another.</li><li>Auditing is separate.</li></ul><blockquote>Remember that we are not designing for operational convenience here. We are designing for failure containment.</blockquote><p>The retrieval role below is an example that allows the RAG pipeline to read from the Gold layer of your S3 data lake only, through Lake Formation, and nothing else. It also enforces an ExternalId condition on the trust policy so that only your specific orchestration service can assume it.</p><pre>{<br>  &quot;RoleName&quot;: &quot;ApprovedRetrievalRole&quot;,<br>  &quot;AssumeRolePolicyDocument&quot;: {<br>    &quot;Version&quot;: &quot;2012-10-17&quot;,<br>    &quot;Statement&quot;: [<br>      {<br>        &quot;Effect&quot;: &quot;Allow&quot;,<br>        &quot;Principal&quot;: {<br>          &quot;AWS&quot;: &quot;arn:aws:iam::&lt;ANALYTICS_ACCOUNT_ID&gt;:role/RAGOrchestratorRole&quot;<br>        },<br>        &quot;Action&quot;: &quot;sts:AssumeRole&quot;,<br>        &quot;Condition&quot;: {<br>          &quot;StringEquals&quot;: {<br>            &quot;sts:ExternalId&quot;: &quot;&lt;YOUR_EXTERNAL_ID&gt;&quot;<br>          }<br>        }<br>      }<br>    ]<br>  },<br>  &quot;Policies&quot;: [<br>    {<br>      &quot;PolicyName&quot;: &quot;RetrievalScopePolicy&quot;,<br>      &quot;PolicyDocument&quot;: {<br>        &quot;Version&quot;: &quot;2012-10-17&quot;,<br>        &quot;Statement&quot;: [<br>          {<br>            &quot;Sid&quot;: &quot;AllowGoldLayerReadViaLakeFormation&quot;,<br>            &quot;Effect&quot;: &quot;Allow&quot;,<br>            &quot;Action&quot;: [<br>              &quot;lakeformation:GetDataAccess&quot;,<br>              &quot;glue:GetTable&quot;,<br>              &quot;glue:GetDatabase&quot;,<br>              &quot;s3:GetObject&quot;<br>            ],<br>            &quot;Resource&quot;: [<br>              &quot;arn:aws:s3:::&lt;DATA_BUCKET&gt;/gold/live/*&quot;,<br>              &quot;arn:aws:glue:&lt;REGION&gt;:&lt;DATA_ACCOUNT_ID&gt;:table/catalog/gold_*&quot;,<br>              &quot;arn:aws:glue:&lt;REGION&gt;:&lt;DATA_ACCOUNT_ID&gt;:database/catalog&quot;<br>            ]<br>          },<br>          {<br>            &quot;Sid&quot;: &quot;DenyAllOtherLayers&quot;,<br>            &quot;Effect&quot;: &quot;Deny&quot;,<br>            &quot;Action&quot;: &quot;s3:*&quot;,<br>            &quot;Resource&quot;: [<br>              &quot;arn:aws:s3:::&lt;DATA_BUCKET&gt;/bronze/*&quot;,<br>              &quot;arn:aws:s3:::&lt;DATA_BUCKET&gt;/silver/*&quot;,<br>              &quot;arn:aws:s3:::&lt;DATA_BUCKET&gt;/gold/staging/*&quot;<br>            ]<br>          }<br>        ]<br>      }<br>    }<br>  ]<br>}</pre><p>Notice the explicit deny on bronze, silver, and the gold staging prefix. Lake Formation grants alone are not enough; this is when you want the IAM policy to enforce layer isolation as a second line of defence so that even a misconfigured Lake Formation grant cannot accidentally expose upstream data.</p><p>At this point, the governing question changes.</p><blockquote>It is no longer <strong>“who is the user?”</strong></blockquote><blockquote>It becomes <strong>“is this access path allowed to exist?”</strong></blockquote><p>That is a fundamentally stronger control because even if your RAG pipeline loses identity context mid-execution, even if it runs under a shared service role, even if attribution is imperfect, it still cannot step outside the paths you defined.</p><h3>The Shift: From Identity-Dependent to Boundary-Enforced Security</h3><blockquote>This is the core architectural reframe.</blockquote><p>Identity-dependent security requires everything to work correctly across a distributed system. Tokens must propagate, and Roles must be scoped. Every hop in the pipeline must preserve context.</p><blockquote>Boundary-enforced security assumes things will break and limits the blast radius when they do.</blockquote><p>When you apply this consistently, something valuable emerges, and access patterns become intentional. Audit trails become cleaner because everything funnels through a small, known set of roles. And critically, your data lake is no longer exposed to the full complexity of your AI stack.</p><p>Your RAG system can have internal complexity, service mesh sprawl, or partial visibility into its own identity chain, and your data remains protected because the boundary holds regardless.</p><blockquote><strong><em>This boundary model builds directly on the agentic security principles</em></strong><em> covered in </em><a href="https://awstip.com/securing-agentic-workloads-on-amazon-bedrock-identity-tools-guardrails-and-observability-f55bd94ae783"><em>Securing Agentic Workloads on Amazon Bedrock</em></a><em> (Part 12) and </em><a href="https://awstip.com/the-security-checklist-for-agentic-ai-on-aws-moving-beyond-iam-64b6381067e0"><em>The Security Checklist for Agentic AI on AWS</em></a><em> (Part 13) of the Security &amp; Compliance Chronicles. If you have not read those yet, they cover why IAM-only models break under agentic workloads, which is the problem this boundary model is designed to solve.</em></blockquote><h3>📚 What Comes Next?</h3><p>This model works at the account and IAM level. But as access patterns grow across teams and use cases, you still need a way to manage what specific principals can see inside the data account itself.</p><p>In Part 3, we will look at how to combine this boundary model with Lake Formation tag-based access control, scaling data-level governance without introducing unmanageable complexity or recreating the IAM sprawl you just contained.</p><blockquote>The goal remains the same: controlled access that holds even when identity does not.</blockquote><h3>🐾 Human in the Loop</h3><p>Sometimes my personal life takes over my professional one, and I honestly don’t get enough time to write the quality content I’d like to.</p><p>Last week, I had this conversation with a friend. We were looking at one of the most remarkable photographs ever taken, the <strong>1927 Solvay Conference</strong> in Brussels. In a single frame: Einstein, Marie Curie, Bohr, Schrödinger, Heisenberg, Pauli, Dirac, Lorentz, Planck, and so many others. Twenty-nine scientists, Seventeen Nobel Prize winners. All in one room, working through the foundations of quantum mechanics.</p><figure><img alt="The Fifth Solvay International Conference, Brussels, 1927. Photograph by Benjamin Couprie. Public domain." src="https://cdn-images-1.medium.com/max/1024/0*SDSaXxDU-Dbq56o9.jpg" /><figcaption><em>The Fifth Solvay International Conference, Brussels, 1927. Photograph by Benjamin Couprie. Public domain.</em></figcaption></figure><p>And yet, with all the universities we have today, all the technology, all the accumulated knowledge, it still feels like we can’t accomplish as much as they did.</p><p>I feel that pressure sometimes. When everything is already out there, it’s easy to wonder whether raising your voice adds anything at all. Self-doubt has a way of stalling a draft before it even starts. But I keep coming back to the same realization: it’s the foundation that matters most. Their work is what brought us here. So I try to take small steps.</p><p>I’ve decided not to focus on only one dimension of life. It’s rarely black and white. Sometimes choosing the right shade of grey takes longer than the decision itself. But I’ve made my peace with that. A little of this and a little of that, over time, makes you a better person.</p><p>What I said to my friend was something like this:</p><p><em>Taking a group photo today is nearly impossible. There are simply too many people who’ve contributed to how the world works. But that’s also the point. It’s gotten so easy to use everything built by people whose names we’ll never know that we don’t even need to go looking for the photo. Life has gotten easier for those of us who use. And significantly harder for those who invent. And somehow, that’s what makes it worth doing.</em></p><p>So to every person in that photograph, and to the countless others whose names never made it into the frame:</p><p><strong>Thank you.</strong></p><p>You built the floor we stand on. The least we can do is keep building.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e146d0395f4f" width="1" height="1" alt=""><hr><p><a href="https://awstip.com/solving-the-lake-formation-identity-crisis-for-rag-security-part-2-e146d0395f4f">Solving the Lake Formation Identity Crisis for RAG Security — Part 2</a> was originally published in <a href="https://awstip.com/">AWS Tip</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Solving the Lake Formation Identity Crisis for RAG Security: Part 1]]></title>
            <link>https://awstip.com/solving-the-lake-formation-identity-crisis-for-rag-security-part-1-0095e2cbc061?source=rss-a6a9790332df------2</link>
            <guid isPermaLink="false">https://medium.com/p/0095e2cbc061</guid>
            <category><![CDATA[cloud-security]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[data-governance]]></category>
            <category><![CDATA[rag-security]]></category>
            <category><![CDATA[zero-trust-architecture]]></category>
            <dc:creator><![CDATA[Alaa Rahimi]]></dc:creator>
            <pubDate>Tue, 17 Mar 2026 08:01:01 GMT</pubDate>
            <atom:updated>2026-03-20T13:32:53.493Z</atom:updated>
            <content:encoded><![CDATA[<h4>Architecting the Shift from IAM Chaos to RAG-Ready Security — Part 18 of Security &amp; Compliance Chronicles</h4><blockquote><em>Executive Summary : In the era of </em><strong><em>Agentic AI</em></strong><em> and </em><strong><em>Retrieval-Augmented Generation (RAG)</em></strong><em>, traditional infrastructure-level security (IAM) is no longer enough. This article explores why the control point for security must shift </em><strong><em>inward: </em></strong><em>from the S3 bucket boundary to the data itself. We examine how </em><strong><em>AWS Lake Formation</em></strong><em> provides the granular, cell-level governance required to ensure AI agents only “understand” what they are authorized to see, and how to navigate the common </em><strong><em>“IAM Principals Trap”</em></strong><em> that leaves many data lakes vulnerable.</em></blockquote><p>If you have followed my Security and Compliance Chronicles, you may have noticed a consistent theme. Security in AWS begins with identity.</p><p>From the early foundations of IAM and least privilege, to centralized access with Identity Center, to SCP guardrails and automation-ready policies, the goal has always been the same: define who can do what under specific conditions. The model works. Until it does not.</p><figure><img alt="A technical sequence diagram showing the secure data flow between an End User, Amazon Bedrock (RAG Agent), AWS Lake Formation, and an S3 Data Lake. The diagram illustrates how Lake Formation intercepts metadata requests to vend scoped credentials and filter data at the row and column level, ensuring the AI agent only processes authorized, redacted chunks of information." src="https://cdn-images-1.medium.com/max/1024/1*dJdLBn-74p669byH2si60w.png" /><figcaption>Figure 1: The RAG Security Handshake: Enforcing Data-Level Governance in AI Pipelines.</figcaption></figure><p>As architectures scale, identity becomes harder to reason about. Roles begin to multiply across services and accounts. Policies grow opaque and increasingly difficult to audit. Cross-account access introduces hidden paths that are not always visible during design, while automation amplifies even small misconfigurations into systemic risk.</p><p>Even with well-designed IAM policies, the system begins to drift. Permissions accumulate while boundaries blur. What starts as least privilege slowly turns into “good enough access.”</p><p>IAM was never designed to answer the most important question in modern architectures.</p><blockquote>Not just who can access the system, but who can see a specific piece of data.</blockquote><p>IAM answers the question of who can access a resource. It does not answer the question of what that identity is allowed to see inside the data itself. That distinction becomes critical as we move from application-centric architectures toward data platforms and AI-driven systems.</p><p>This is where the data security layer begins to emerge as a separate concern. It is no longer sufficient to control access at the boundary of a bucket or a service. The real control point shifts inward, toward the structure and meaning of the data.</p><h3>What Is AWS Lake Formation?</h3><p>Amazon S3 serves as the foundational storage layer for most modern data platforms. AWS Glue provides the catalog that allows us to organize and discover that data. AWS Lake Formation introduces the enforcement layer that governs how that data can be accessed.</p><p>Historically, data security in AWS has been implemented at the infrastructure level. Teams would secure buckets, define prefix-based access, and rely on IAM policies to control interaction with those resources. Over time, these policies often became complex and difficult to audit, especially in multi-account environments.</p><p>Lake Formation changes the model by introducing centralized governance at the data level. It operates on top of the Glue Data Catalog and allows permissions to be defined at the level of databases, tables, columns, and even individual rows.</p><p>This moves the control model away from storage locations and toward the data itself. Instead of reasoning about paths and objects, architects can reason about datasets and their sensitivity.</p><p>The result is not just simplification. It is a different way of thinking about security. Control is no longer tied to where the data is stored, but to what the data represents.</p><h3>Why This Matters for RAG</h3><p>In modern architectures, <em>Retrieval Augmented Generation </em>has become a core pattern for enterprise AI systems. These systems retrieve internal data and provide it as context to large language models in order to generate meaningful and relevant responses.</p><p>The retrieval layer becomes one of the most sensitive components in the entire architecture. It determines which pieces of data are exposed to the model and ultimately influences what the system is capable of revealing.</p><p>Without a data-level security model, this layer often operates with broad IAM permissions. A role that has access to read objects from a bucket can potentially retrieve any dataset stored within that boundary. This includes information that may be confidential, regulated, or simply not intended for that specific use case.</p><blockquote>At that point, the model is not the risk. The access model that feeds it is.</blockquote><p><strong>Lake Formation introduces a form of contextual enforcement.</strong></p><p>It ensures that only the specific rows and columns that an identity is authorized to access can be retrieved. If an AI agent does not have permission to access a particular dataset, that data is never returned by the query engine and never enters the model context.</p><p>This marks the transition from securing infrastructure to securing inference. The system is no longer defined only by what it can access, but by what it is allowed to understand.</p><h3>The IAM Principals Trap</h3><p>There is a subtle but critical issue that appears in many Lake Formation implementations. It often goes unnoticed because everything seems to function correctly at first glance.</p><p>When Lake Formation is enabled, AWS introduces a virtual group called IAMAllowedPrincipals. This group frequently retains broad permissions on catalog resources, especially in environments that evolved from traditional IAM based access patterns.</p><p>As long as this group has elevated permissions, Lake Formation enforcement can be bypassed by underlying IAM policies. A user may appear to have restricted access within Lake Formation, while still being able to retrieve the same data through IAM-based permissions.</p><p>This creates the illusion of security. The governance model appears to be in place, but it is not actually enforcing the intended boundaries.</p><blockquote>In a modern architecture that aims for a zero trust approach, this configuration becomes a structural weakness.</blockquote><p>IAMAllowedPrincipals should be treated as a transitional artifact rather than a permanent component of the system.</p><blockquote>Removing this implicit access is not simply a best practice. It is a prerequisite for achieving true data-level governance.</blockquote><h3>The Practical Path Hybrid Access and Incremental Control</h3><p>In production environments, changes to security models must be introduced carefully.</p><p>Systems cannot be disrupted simply to achieve architectural purity.</p><p>Lake Formation provides a mechanism for gradual adoption through a hybrid access mode. This approach allows existing workloads to continue operating under IAM while new workloads adopt Lake Formation permissions.</p><p>S3 locations can be registered with hybrid access enabled, which creates a controlled overlap between the two models. Legacy ETL processes and batch jobs can continue to function without interruption, while new roles such as AI agents and analytics workloads are explicitly governed through Lake Formation.</p><p>This enables a phased migration rather than a complete redesign.</p><blockquote>At the same time, Lake Formation integrates with modern data services and patterns.</blockquote><p>It works with query engines, data warehouses, and emerging table formats that support large-scale analytics and machine learning workflows.</p><p>As data moves from raw storage into curated datasets and eventually into vector representations for AI systems, the governance model remains consistent. Permissions are applied to the data itself rather than to the infrastructure that stores it.</p><p>This continuity is what allows security to scale alongside the architecture.</p><h3>🐾 The Human-in-the-Loop Corner</h3><p>After spending time meeting colleagues in person, one thing became clear. Despite the scale of the systems we build, the community behind them remains small.</p><p>Between devouring world-class Parisian éclairs and losing track of time in a smoky jazz club with old friends, I realized something vital. For someone who usually thrives on homemade cakes and back-to-back virtual meetings, stepping into the physical pulse of the tech community was an absolute jolt of lightning!</p><p>We spend our days designing architectures, real-time retrieval, and distributed intelligence. But the trust model behind all of it? It still depends on <strong>human intuition</strong>. It’s about how we define access, how we enforce boundaries, and how seriously we take our role as the ultimate “Guardians” of the data.</p><p>Technology is accelerating at a breakneck pace, but what about human responsibility? That doesn’t disappear but becomes more precious!</p><blockquote>Staying in touch, sharing a table, and keeping each other company is what keeps us grounded while the tech moves at 100mph.</blockquote><h3>The Architect&#39;s Bottom Line</h3><p>If you are building data platforms or AI-driven systems, identity alone is no longer sufficient as a control mechanism.</p><blockquote>You must extend your security model into the data layer.</blockquote><p>This means auditing your Lake Formation configuration, identifying where IAMAllowedPrincipals still have access, and beginning the transition toward explicit data-level permissions.</p><blockquote>In systems that retrieve and interpret data on behalf of users, the boundaries of visibility define the boundaries of trust.</blockquote><p>In the next part, we will explore how to scale this model using tag-based access control and how to secure distributed data architectures without introducing unmanageable complexity.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0095e2cbc061" width="1" height="1" alt=""><hr><p><a href="https://awstip.com/solving-the-lake-formation-identity-crisis-for-rag-security-part-1-0095e2cbc061">Solving the Lake Formation Identity Crisis for RAG Security: Part 1</a> was originally published in <a href="https://awstip.com/">AWS Tip</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Full Circle: From Debian Repositories to AWS Security Community Builder]]></title>
            <link>https://awstip.com/the-full-circle-from-debian-repositories-to-aws-security-community-builder-0d4ec5534348?source=rss-a6a9790332df------2</link>
            <guid isPermaLink="false">https://medium.com/p/0d4ec5534348</guid>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[cloud-security]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[awscommunitybuilder]]></category>
            <category><![CDATA[women-in-tech]]></category>
            <dc:creator><![CDATA[Alaa Rahimi]]></dc:creator>
            <pubDate>Thu, 05 Mar 2026 06:36:19 GMT</pubDate>
            <atom:updated>2026-03-09T08:18:54.304Z</atom:updated>
            <content:encoded><![CDATA[<h4>From Linux roots to cloud security — reflecting on open source, initiative, and joining the AWS Community Builders program.</h4><p>Eleven months ago, just after my 39th birthday, I sat down to reflect on my technical journey. I felt that I had lost some of the initiative that once drove me as a younger developer. Writing became my way of reconnecting with the community that had shaped me.</p><p>Today, that journey has reached an important milestone:<strong> I have been accepted into the AWS Community Builders program in the Security category.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*fDtlymdPGebE_q3Q" /><figcaption>Photo by <a href="https://unsplash.com/@john_cameron?utm_source=medium&amp;utm_medium=referral">John Cameron</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>The Foundations: Debian, Ubuntu, and the Initiative</h3><p>My path didn’t start in the cloud. It started much closer to the metal: networks, VoIP systems, and open-source communities.</p><p>As a young engineer, I was active in the Debian community and contributed to the Ubuntu ecosystem. Those years taught me something that has stayed with me ever since: technology is never just about code. It’s about the communities that build, maintain, and share it.</p><p>Over time, somewhere between projects, studies, and work, I felt that the initiative I once had began to fade.</p><p>Writing during the past year helped me rediscover that energy. Publishing articles wasn’t just about sharing knowledge; it was about reconnecting with the collaborative spirit that first drew me into open source.</p><blockquote>Being accepted into the Community Builders program isn’t just a badge for me. It feels like a return to that mindset.</blockquote><p>It’s a commitment to move from simply using tools to helping build the community around them.</p><h3>Standing on the Shoulders of Giants</h3><blockquote>No one arrives at a milestone like this alone.</blockquote><p>The AWS Community Builders program exists because people believed that mentorship and shared learning matter. Many individuals invested time and energy into building a community where engineers can exchange ideas, teach one another, and grow together.</p><p>Some of the people who helped shape these initiatives are no longer at Amazon following the industry layoffs of recent years. Their work still echoes through the community they helped build.</p><p>Programs like this matter because they give newcomers a roadmap. When I started my journey in technology, I often had to navigate the path alone. If sharing my experiences can make that path clearer for someone else, then the effort is worthwhile.</p><blockquote>In a time when AI can generate endless technical documents, there is still value in something simple: a human sharing her journey, mistakes, lessons, and small discoveries along the way.</blockquote><h3>🛡 ️Why Security?</h3><p>I joined the Security category because, in 2026, security sits at the center of everything we build.</p><p>My interest in platform security began long before cloud infrastructure. During my thesis work, I spent hours analyzing network traffic in Wireshark.</p><blockquote>Watching packets move across a network felt almost like watching jazz improvisation: patterns, rhythms, and surprises hidden in streams of data.</blockquote><p>My supervisor, Nikk, patiently guided me through concepts that once felt completely mysterious. That experience opened the door to understanding how networks really function.</p><p>Later, during my master’s program, another professor allowed me to present security topics to a room of more than fifty people. His advice was simple: stay curious, keep speaking up, and don’t be afraid of making mistakes while learning.</p><p>At one point, I also worked for a year in a SOC lab. That period remains one of my happiest professional experiences. Surrounded by talented peers and guided by a thoughtful manager, I learned to think clearly in incidents and to approach infrastructure with a security mindset.</p><p>When I began writing about cloud technologies, I realized something important: the work that excited me most was securing platforms and helping others understand how to do it well.</p><blockquote>And so, here we are! 😃</blockquote><h3>What I Plan to Share</h3><p>Over the coming months, I plan to focus on topics such as:</p><p><strong>Automated governance: </strong>moving from manual checks toward systems that enforce and repair security policies automatically.</p><p><strong>Identity-centric security: </strong>exploring how Zero Trust approaches can strengthen systems without slowing developers down.</p><p><strong>Vulnerability management: </strong>applying lessons from open-source ecosystems like Debian to modern cloud environments.</p><h3>🐾 The Human-in-the-Loop Corner</h3><p>To everyone who has followed my writing journey over the past eleven months: thank you.</p><blockquote>To the mentors and builders who created the foundations we stand on: I will do my best to contribute something meaningful in return.</blockquote><p>The cloud is vast. Building it securely becomes much easier when we build it together.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0d4ec5534348" width="1" height="1" alt=""><hr><p><a href="https://awstip.com/the-full-circle-from-debian-repositories-to-aws-security-community-builder-0d4ec5534348">The Full Circle: From Debian Repositories to AWS Security Community Builder</a> was originally published in <a href="https://awstip.com/">AWS Tip</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>