<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Pat Hogen on Medium]]></title>
        <description><![CDATA[Stories by Pat Hogen on Medium]]></description>
        <link>https://medium.com/@patrick.hogen?source=rss-a81f958c4427------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*pBYwI-plN66jTjiWtIwgVA.jpeg</url>
            <title>Stories by Pat Hogen on Medium</title>
            <link>https://medium.com/@patrick.hogen?source=rss-a81f958c4427------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 13:32:00 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@patrick.hogen/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Entry 118 — From RAG to Secure Agent Workflow]]></title>
            <link>https://medium.com/@patrick.hogen/entry-118-from-rag-to-secure-agent-workflow-ec14119c61e2?source=rss-a81f958c4427------2</link>
            <guid isPermaLink="false">https://medium.com/p/ec14119c61e2</guid>
            <category><![CDATA[ai-security]]></category>
            <category><![CDATA[retrieval-augmented-gen]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[langgraph]]></category>
            <dc:creator><![CDATA[Pat Hogen]]></dc:creator>
            <pubDate>Wed, 18 Nov 2026 12:30:42 GMT</pubDate>
            <atom:updated>2026-11-18T12:30:42.788Z</atom:updated>
            <content:encoded><![CDATA[<p>Jumping right into this entry, in Entry 117, when I was performing the cross-user isolation tests, the application would return something like the following:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/235/1*d3tF_mQMf7Zd7QcX9uMgPQ.png" /><figcaption>(<a href="https://medium.com/@patrick.hogen/entry-117-testing-the-limits-of-rag-control-a901ef0e2d67">https://medium.com/@patrick.hogen/entry-117-testing-the-limits-of-rag-control-a901ef0e2d67</a>)</figcaption></figure><p>Grounded: true, hits_count: 1. In my current <em>main.py, Grounded</em> becomes <em>true </em>whenever the retrieval function returns at least one chunk. This presents me with a useful opportunity to improve the application before introducing agent-driven decisions. That is what is going to be the exercise in this entry. building a workflow that distinguishes between retrieving a document and having enough evidence to answer a question.</p><p>As part of this process, we’ll be installing and integrating LangGraph into the workflow. So, first things first, what is LangGraph? LangGraph is an orchestration framework, more specifically:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/675/1*6SbL7Yk5ciS7QS-jRHzAFg.png" /><figcaption>(<a href="https://docs.langchain.com/oss/python/langgraph/overview">https://docs.langchain.com/oss/python/langgraph/overview</a>?)</figcaption></figure><p>LangGraph focuses on orchestration capabilities such as durable execution, streaming, and human-in-the-loop. So, time to get it installed:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/866/1*Y0PWrzUjZjgK5I163Wwnpw.png" /></figure><p>And, verify that the installation was successful:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/749/1*q7lrL35fr8EY6Q4yHMRlvA.png" /></figure><p>Now that it is installed, I want to start small by building a simple LangGraph workflow that I can execute before I try and connect LangGraph to Amazon Bedrock. This will be a simple workflow where:</p><ul><li>User submits a question -&gt;</li><li>LangGraph validates the question (checks whether the question is empty -&gt;&gt;</li><li>Returns a validation status</li></ul><p>This will be a test that will be run locally. So, I need to build a file first so that I can perform these tests:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/695/1*RK2K9zhMDw-3rvXA-fQm1g.png" /></figure><p>What this file will do is act as a traffic controller for the AI application that it’s being built against. The validation node will direct traffic between ‘yes’ and ‘no’ (in simple terms). It will remember the question and status, then it will check the question, and then return a status. A simple workflow as mentioned earlier. When run, the file uses its test question and gives a result from that:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/759/1*O5ng1_Ne0vw6exJLXvADOw.png" /></figure><p>The result isn’t an answer, it is merely a signal that the question is valid and the workflow can proceed. But, what if the question is empty:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gF7B-ZkfySEZQKErqwLkgw.png" /></figure><p>Hard to see without blowing up the screenshot and losing the total context, but basically an empty question was asked, and an ‘invalid_question’ response was given. This will become important later down the line as we don&#39;t want invalid questions making their way to Bedrock and causing unnecessary processing and related costs.</p><p>Now that I know that the file works correctly, identifying ‘normal’ and ‘empty’ questions, it’s time to try something more interesting and give LangGraph a decision to make. This is going to be something called <em>conditional routing. </em>Defined in the <em>agent_workflow.py</em> file as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/475/1*embAHQGCp5LaNPmyCltDPw.png" /></figure><p>A <em>route_question() </em>variable which will decide which direction the workflow should follow. Since I am in testing phase with this at the moment, the questions being asked are hardcoded:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/509/1*XkkRw5PSHO0UrLjsk5X3GQ.png" /></figure><p>Rather than introducing persistent state in LangGraph’s checkpoint functionality that could allow the study assistant to maintain conversation history across requests. We can introduce a valid question from PowerShell to check whether or not the file will recognize that the question is a question or not:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/544/1*M6icRE8lwhCJoDHJtGAF1A.png" /></figure><p>Simply asked, ‘What is Azure Key V ault?’. It’s a valid question and the output reflected that.</p><p>So, at this point, the workflow reaches the <em>retrieval_pending()</em> function (which is what gave us the output about the Azure question even though I didn’t hardcode that question), but it doesn’t actually search any of the documents (the ones uploaded to notebooks in previous entries). I’ll be updating my <em>agent_workflow() </em>file to put in a feature called <em>runtime context </em>which lets me supply information to a workflow separately from its ordinary state. It’s useful for keeping authenticated identity out of question data.</p><p>Once I have the appropriate updates added to the workflow file, and ask an empty question again, I should get an invalid response:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/741/1*w8UifMGu8ARyCND2OWpkYg.png" /></figure><p>Which is what we want to see. The blank question was identified as ‘invalid’, the <em>hits_count</em> remained at 0, and LangGraph stopped accessing the document database.</p><p>I’ve tested what happens when empty questions are submitted (retrieval stops), but what happens when the question is valid, but the identity is missing:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/689/1*nTXIxDzoAENaXOdmNCp6QA.png" /></figure><p>Towards the bottom, we get the error that we are looking for — <em>PermissionError: Authenticated user context required</em>. Now that the workflow successfully rejects requests without an authenticated user context, I want to connect it to the existing FastAPI application.</p><p>I’m not replacing the original RAG endpoint just yet. Instead, I’m adding a new endpoint, /v1/agent/retrieve, so I can test LangGraph separately without disrupting the functionality I&#39;ve already built.</p><p>The important part of this integration is that FastAPI remains responsible for authentication. When a request arrives, FastAPI validates the API key and establishes the user’s identity. That authenticated user ID is then passed to LangGraph through its runtime context.</p><p>The important line in main.py is:</p><pre>context={&quot;user_id&quot;: user.user_id}</pre><p>This might not look like much, but there’s an important security consideration here; the user ID isn’t being taken from the question or supplied directly by the person making the request. It’s being passed from FastAPI’s authenticated user context.</p><p>LangGraph can make the decision which workflow steps execute, but it doesn’t get to decide whose documents it’s authorized to retrieve.</p><ul><li><strong>Testing the Integration</strong></li></ul><p>With the new endpoint configured, I created a test user and ingested a document containing information about Azure Key Vault.</p><p>I also included a private test marker, AGENT-118-ALPHA, so I could identify the document during later security tests.</p><p>After submitting the question through the new endpoint, I received:</p><pre>status      : documents_found<br>hits_count  : 2<br>notebook    : agent-test-118</pre><p>That worked. The request passed through FastAPI authentication, entered LangGraph, and reached the existing retrieval function. The workflow is no longer just validating questions, it is performing an actual retrieval operation.</p><ul><li><strong>Checking the Security Boundary</strong></li></ul><p>Of course, introducing a new orchestration layer means introducing another potential place for security mistakes. To start checking that I hadn’t bypassed the existing tenant-isolation controls, I created User B and submitted the same request.</p><p>At this point, User B didn’t have any documents in the test notebook.</p><p>The response was:</p><pre>status      : no_documents<br>hits_count  : 0<br>notebook    : agent-test-118</pre><p>That’s the behavior I wanted to see. The new workflow didn’t automatically get User A’s document simply because both requests used the same notebook name.</p><p>I then ingested a separate document for User B, containing the private marker AGENT-118-BRAVO.</p><p>Both users now had their own documents in the same notebook, giving me a more meaningful test environment for additional isolation checks.</p><ul><li><strong><em>What’s Happening at the Engineering Level</em></strong>?</li></ul><p>There are three important components working together now.</p><ul><li><strong>FastAPI handles authentication.</strong> It validates the API key and determines the user’s identity.</li><li><strong>LangGraph handles workflow orchestration.</strong> It validates the incoming question and conditionally routes valid requests to the retrieval node. Invalid questions stop before retrieval.</li><li><strong>The retrieval layer handles document ownership.</strong> Its SQLite query restricts candidate document chunks using the authenticated user ID and notebook before similarity ranking.</li></ul><p>This separation is important because introducing an agent orchestration framework shouldn’t weaken the security controls that already exist.</p><p>The application still needs to enforce authorization independently of the model.</p><ul><li><strong><em>What Have I Actually Built</em></strong>?</li></ul><p>At this stage, the new workflow isn’t an autonomous AI agent, yet. It doesn’t independently choose tools, evaluate evidence, or generate answers through Bedrock.</p><p>What I have built is a <em>stateful</em> orchestration base with conditional routing, an authenticated retrieval operation, and a defined boundary between application identity and workflow execution.</p><p>That’s a meaningful architectural step beyond the original linear RAG request flow, but there’s more work to do before calling it an agentic study assistant.</p><p>The next improvement will be introducing evidence validation.</p><p>In Entry 117, I noticed that the application could report grounded: True simply because documents were retrieved, even when those documents couldn&#39;t answer the question.</p><p>I’d like the new workflow to distinguish between finding documents and finding sufficient evidence to generate a supported answer.</p><p>That will give the application a more deliberate decision-making process.</p><p>For now, I’ve successfully introduced LangGraph into the existing FastAPI architecture and demonstrated authenticated retrieval through the new workflow.</p><p>The important lesson from this entry is that <strong>adding an orchestration layer shouldn’t mean giving that layer control over authorization.</strong></p><p>Thanks for reading.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ec14119c61e2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Entry 117 — Testing the Limits of RAG Control]]></title>
            <link>https://medium.com/@patrick.hogen/entry-117-testing-the-limits-of-rag-control-a901ef0e2d67?source=rss-a81f958c4427------2</link>
            <guid isPermaLink="false">https://medium.com/p/a901ef0e2d67</guid>
            <category><![CDATA[cloud-security]]></category>
            <category><![CDATA[access-control]]></category>
            <category><![CDATA[ai-security]]></category>
            <category><![CDATA[retrieval-augmented-gen]]></category>
            <dc:creator><![CDATA[Pat Hogen]]></dc:creator>
            <pubDate>Mon, 16 Nov 2026 11:55:10 GMT</pubDate>
            <atom:updated>2026-11-16T11:55:10.960Z</atom:updated>
            <content:encoded><![CDATA[<p>When it came time to name this entry, I had to fight against my inner-dad-joke-machine that really wanted to use some of ‘The Eagles’ -based reference. But, I tamped that down and kept on with the simple, to-the-point, title that won’t turn off any potential readers that aren’t fans of The f*cking Eagles. So, what is this entry going to cover; it’s going to be going over testing what happens when someone deliberately tries to get around security controls.</p><p>We’ll start with one main question:</p><ul><li>Can someone manipulate a request to retrieve information that they are not authorized to see?</li></ul><p>We’ll test this by running a few tests, and seeing what those tests test:</p><ul><li>Missing authentication — Does the API reject requests without credentials?</li><li>Invalid credentials — Does an incorrect API key return <em>401</em>?</li><li>Deliberate cross-user queries — Can User A retrieve User B’s canary by explicitly requesting it?</li><li>Regression testing — Can we automate these checks for future changes?</li></ul><p>Before I run the first test, I remind myself of the different between authentication and authorization. Authentication = who you are, who is your authentic self. Authorization = what you are allowed to do, how you would author the story. Simple differences, I used to get them confused in the past as I’ve noted previously because my brain did authorization = author = person = who you are — but, they are flipped. Moving on to the testing phase.</p><p>The first test is checking that requests with missing credentials get requested. This will be done by sending the following to the API:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/527/1*9SHIyHcuuzviLjag1W6vaA.png" /></figure><p>I’m poking the <em>/chat</em> endpoint but leaving out the X-API key header that would contain the credentials. If the security controls work as they should, the next image should be a <em>401</em>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/188/1*ohnuwn-pO1p15h2QiD8wLg.png" /></figure><p>The first control passed. No identity -&gt; no retrieval. No need to dwell on that much further, time for the next test.</p><p>The next test will take this check another step, adding in the X-API key header but giving it an incorrect key:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/589/1*W-9M_DkPMjZdrVxwSSDv8w.png" /></figure><p>As with the previous test, if the security control is working as expected, then this test should give us a <em>401</em>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/204/1*2hl698ICkZB--yWmH8wMwg.png" /></figure><p>Which it does. So far, so good. A missing API key and an invalid API key both give us a <em>401</em> when tested. So, now, let’s mix it up.</p><p>The test that we’ll go here is a deliberate cross-user retrieval request — trying to mix-up the security control so that it gets ‘confused’ and gives out information it isn’t supposed to. First up, is a request from User A, asking for User B’s canary. If you remember from the prior entries, User A was Alpha stuff and User B was Bravo stuff. User A asking about Bravo stuff, should give us no references to Bravo stuff:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/496/1*2LTPHZYIn_oE_tBfwzT1gA.png" /></figure><p>Here is a shortened version of the output:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/546/1*yuQ5TsDpXPEjIbabjCBnCQ.png" /></figure><p>The model could not answer the question, no authorization was granted to User A to get any context about Project Bravo. After that test, we perform a reversal. User B asking for information about User A stuff:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/515/1*VY1nH1BNCzisr0tDxtmM2w.png" /></figure><p>And, the answer:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/889/1*RZoTzWSbZPisJI7nRe56-w.png" /></figure><p>The model couldn’t answer as it doesn’t have authorization for User B to access Project Alpha information. If we take this same information and create a test that automates the deliberate access attempts, we should be able to get the same outputs as the 2 previous test that were run in isolation:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/568/1*jJr_MFRodPf_Y_ZnJU-kuQ.png" /></figure><p>Easy enough, the output is the same as expected from the prior tests:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/269/1*SQmHIgpxlTtnVgkwbMlHqw.png" /></figure><p>To take this a step further, I can add adversarial test to the regression script that was developed in a previous entry. This will be a test that makes fresh API requests with both autthenticated users and checks that the answer don’t reveal other user’s canary and that the citations reference only the expected documents. This is an API-level regression test, not a direct inspection of raw retrieved chunks:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/379/1*ee7jOEYTGvxyyQFTrcJQEg.png" /></figure><p>This is basically what we tested already, just codified into an actual test file. Where, if this is run with the correct corresponding user key:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/664/1*M28zKw7HlHKRfhPhU8phKA.png" /></figure><p>Then, we should get a total of 4 passing marks:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/281/1*I9r8hthdDOYqR-bjpxnR9w.png" /></figure><p>I’ve hit an important part so far with these tests, they demonstrated legitmate access and rejection of unauthorized attempts. It should be important to note that they do not reflect represent an elimination of every possible data-leak scenario, just prevention as related to these specific API requests.</p><p>So, this brings me to the part of the entry where I work with my mentor to take a look under-the-hood at the engineering level. These test showed that 3 different components worked together to prevent these unauthorized access attempts — authentication, authorization, and retrieval.</p><ol><li><strong>Authentication</strong> — <em>Establishing the user’s identity.</em></li></ol><p>The first 2 tests focused on requests without valid credentials. My FastAPI application uses a <em>require_user() </em>dependency to authenticate requests through the X-API-Key header:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/938/1*QyUC01qmMF6IMtAyGTQt9Q.png" /></figure><p>When a request arrives, the application checks whether a key was provided, hashes it using SHA-256, and looks for a corresponding record in SQLite. If the key is missing or invalid, the application raises a HTTP 401<em> </em>error. Both tests did exactly that. The important thing to remember with this is that the API key was associated with an actual user; the 2 things things together are what satisfy the authentication piece of the puzzle.</p><p>2. <strong>Authorization</strong> — <em>Restricting what can be retrieved</em>.</p><p>The authentication piece isn’t enough as an authenticated user might try to retrieve documents that don’t belong to them. This is where authorization comes in. For my application specifically, there is a <em>retrieve() </em>function that reci e the authenticated user_id from FastAPI’s UserContext. The SQLite query includes:</p><ul><li>WHERE c.user_id = ?</li><li>AND c.notebook = ?</li></ul><p>Which restricts the candidate document chunks by ownership and notebook before cosine similarity raking is performed. This creates an important trust boundary. It also enforces the security control that the model cannot disclose an unauthorized document through retrieved context if that document was correctly excluded before generation.</p><p>3. <strong><em>Retrieval results versus answer quality</em></strong>.</p><p>It’s a little hard to see in the screenshots of the tests results, but during the cross-user tests the application returned:</p><ul><li>grounded: True</li><li>hits_count: 1</li></ul><p>However, Claude also explained that it couldn’t answer the requested question using the provided context.</p><p>Initially, that might look contradictory.</p><p>Looking at main.py, the explanation becomes clearer.</p><p>The application currently sets grounded to True whenever retrieval returns document chunks and the generation path completes its normal response construction:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/275/1*B96w3hs3bp9wsF99cEv9TQ.png" /></figure><p>It doesn’t independently verify whether those chunks contain enough information to answer the question. In this situation, User A asked about Project Bravo, but retrieval returned Project Alpha.</p><p>The retrieval operation was successful, and the ownership restriction behaved as expected. However, the retrieved content wasn’t sufficient to answer the question. That distinction matters:</p><ul><li><strong>Retrieval success is not the same as answer correctness or answer grounding.</strong></li></ul><p>A future improvement would be to distinguish between whether context was retrieved and whether the generated answer was actually supported by that context.</p><p>4. <strong><em>Making the Tests Repeatable</em></strong></p><p>Rather than relying entirely on screenshots and manually inspecting responses, I expanded my existing PowerShell regression script.</p><p>The updated script makes fresh authenticated requests against the /v1/chat endpoint and verifies that the results meet several conditions.</p><p>For the deliberate cross-user tests, the script checks that:</p><ul><li>The response doesn’t contain the other user’s private canary marker.</li><li>The retrieved document count matches the expected test fixture.</li><li>The citations reference the requesting user’s document.</li><li>Bedrock hasn’t returned a known invocation failure or disabled-model message.</li></ul><p>The final output:</p><pre>PASS: User A isolation<br>PASS: User B isolation<br>PASS: User A cannot retrieve Bravo<br>PASS: User B cannot retrieve Alpha<br>All canary isolation tests passed.</pre><p>I committed these additional checks to GitHub so I can reuse them as the application evolves. The full regression script now covers both normal access scenarios and deliberate attempts to access another user’s information.</p><ul><li><em>What Have I Actually Proven</em>?</li></ul><p>These results demonstrate that the application’s authentication and ownership-filtering controls worked for the specific requests and documents tested. They don’t establish that the application is immune to every authorization bypass or information disclosure.</p><p>The current regression checks examine API responses and citation metadata rather than independently inspecting the raw retrieval candidates. They also don’t exercise response caching, alternative database implementations, or a broad collection of adversarial inputs.</p><p>Those remain areas for future validation.</p><ul><li><em>What I Learned</em></li></ul><p>This entry reinforced something that’s becoming a recurring theme throughout this project: security controls need to be enforced at the appropriate layer.</p><ul><li>Authentication establishes the requesting user’s identity.</li><li>Authorization restricts which information that identity can access.</li><li>Retrieval selects relevant information from the authorized candidates.</li></ul><p>Finally, the language model generates an answer using the context it receives.</p><p>Each component has a separate responsibility, and passing one test doesn’t automatically prove the others are working correctly.</p><p>One unexpected result was discovering that my application’s grounded field doesn&#39;t necessarily mean the generated answer was adequately supported by the retrieved documents.</p><p>That’s something worth addressing in a future entry.</p><p>For now, I’ve expanded the security regression suite, verified missing and invalid credentials are rejected, and tested deliberate cross-user retrieval attempts in both directions.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a901ef0e2d67" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Entry 116 — Testing RAG Security with Canary Documents]]></title>
            <link>https://medium.com/@patrick.hogen/entry-116-testing-rag-security-with-canary-documents-d9103b716b84?source=rss-a81f958c4427------2</link>
            <guid isPermaLink="false">https://medium.com/p/d9103b716b84</guid>
            <category><![CDATA[ai-security]]></category>
            <category><![CDATA[data-security]]></category>
            <category><![CDATA[retrieval-augmented-gen]]></category>
            <category><![CDATA[cloud-security]]></category>
            <dc:creator><![CDATA[Pat Hogen]]></dc:creator>
            <pubDate>Fri, 13 Nov 2026 12:28:00 GMT</pubDate>
            <atom:updated>2026-11-13T12:28:00.471Z</atom:updated>
            <content:encoded><![CDATA[<p>In the previous entries, I’ve been working through different aspects of security in my RAG Study Assistant project (<a href="https://github.com/pathog-en/ai-rag-study-assistant">https://github.com/pathog-en/ai-rag-study-assistant</a>); including authentication, document isolation, and prompt-injection protection. For this entry, I want to back up a little bit to the idea of <em>cross-user isolation</em> and go through it again a little more thoroughly to gain a better understanding.</p><p>In an earlier test, I verified that User B could retrieve the private documents belonging to User A. However, at that time, User B did not have any documents of their own. For the tests that I’ll work through in this entry, I’ll be giving both users their own documents containing unique identifiers, sometimes called <em>canary markers</em>.</p><p>The goal here is straightforward, each user should be able to retrieve their own information without that information getting contaminated with info from the other user’s documents.</p><p>Before I get into creating the documents for these tests, I want to make sure that I am working from a clean tree:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/436/1*xIsMY__wqinyiGNhyGUphA.png" /></figure><p>Which, I am now. Time to start working on what to do to create the documents and run the tests. The first step in this process is to create the 2 users that these tests will be run with:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/438/1*DLVC37OF9PPJZjYQpDCZ4w.png" /></figure><p>That creates the 2 users; User A and User B. Once that is done, I’ll create the document that will contain the marker for User A and ingest it:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/403/1*SC5wJHt7Vu779juuZk5tJg.png" /></figure><p>Then, I create the document for User B and ingest that:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/399/1*PXJ6rvIHMLhdmZfL5Edz3g.png" /></figure><p>Now that both of these are ingested, I can ask the model to give me the security project canary marker from the notebook that User A’s private document was ingested to. This should only give us a result that shows the canary marker for User A, not User B. Here is that result:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/776/1*LtxDMkbHl__2EQoF04vskg.png" /></figure><p>The output is the canary marker from User A’s notebook, not User B’s. Now, to do the inverse. We ask the model to check the notebook that User B’s private document was ingested to and see that it returns the result from User B, not User A. Here is that result:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/830/1*n6rkl0h0PnCAQmPC7sHt3Q.png" /></figure><p>Both give us the output that we were looking for. Next, we can run a command to verify these results programmatically:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/493/1*KgNLi7x5Ypz3_7bF5B-HOg.png" /></figure><p>This checks the answers and the citation metadata from the 2 existing responses as a regression check. Both, should produce a passing result at this point:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/184/1*YvrpZ0QjmQ3YL-ZP2mz0dQ.png" /></figure><p>Since those both produced a passing result, we can turn those commands into a reusable test script, a security regression test, that send fresh API requests every time it runs. So, we create a new <em>test-canary-isolation</em> file that submits fresh requests and checks both users’ answers and citations; it doesn’t yet inspect the raw retrieval results or prove isolation against every possible query. Once run, the file should give us the same results as the above script that was run in PowerShell:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/698/1*lCYdr2Lk1iZVa_TQZ4nhbw.png" /></figure><p>Excellent, both tests passed and that was the point that I wanted to get to here. Which brings me to the meatier question for this entry; what’s happening behind the scenes?</p><p>The important part of these tests wasn’t the canary markers themselves. Those were just the unique pieces of information to help identify whether the application retrieves data belonging to the wrong user. The important part was the security control of <strong>tenant isolation at the retrieval layer</strong> that was being tested. This security control can breakdown as follows:</p><ol><li>Authentication establishes identity.</li></ol><p>When a request reaches my FastAPI application, it includes an API key in the <em>X-API-Key</em> header.</p><p>The <em>require_user()</em> function in the <em>auth.py</em> file hashes that key, looks up the corresponding user in SQLite, and returns a <em>UserContext</em> containing the authenticated user’s ID. That user ID is important becuase it comes from the application’s authentication process, not from a user ID supplied in the request body. In other words, a user can’t simply change the <em>user_id</em> field in the chat request to retrieve another person’s documents.</p><p>2. Retrieval is restricted by ownership</p><p>Once the user’s identity has been established, <em>main.py</em> passes the authenticated <em>user_id </em>into the retrieval function. The <em>retrieve.py</em> file contains a SQL query with 2 important conditions:</p><ul><li>WHERE c.user_id = ?</li><li>AND c.notebook = ?</li></ul><p>These conditions restrict which document chunks are selected from SQLite. The first restricts the results to the authenticated user and the second restricts them to the requested notebook.</p><p>Both User A and User B have the documents ingested to the same notebook but the database is able to distinguish ownership of them due to the 2 aforementioned conditions.</p><p>3. What reaches the language model?</p><p>After retrieval, the application builds the context that will be sent to Amazon Bedrock. That context contains the selected document chunks, and the application seperately constructs sitation metadata from the retrieved results. If the ownership filter works correctly, another user’s document should never enter that context in the first place. This is imporant becuase I don’t want to depend on Claude to make these decisions for me, I want to be able to have access controls in place that work before the model receives the information.</p><p>4. Turning the results into regression tests.</p><p>Checking results manually is useful, but creating a repeatable script in PowerShell is useful becuase it will give me a useful integration test to run after any future application changes.</p><p>The test script sends separate authentication requests for each user and then evaluates the responses against a set of assertions. If the check fails, it will report it to me rather than me having to check to try and spot the failure value on my own.</p><p>5. What did these results prove and what did they not prove?</p><p>The tests proved that tenant isolation worked for the two users, documents, and requests They also proved that the users could retrieve their own data.</p><p>The test didn’t prove that every possible attempt to access another user’s information will fail.</p><p>The biggest takeaway for me from this entry is that RAG security needs to be enforced in the application rather than simply delegating that to the language model. Authentication establishes who is making the request. Ownership filtering determines what information can be retrieved. Similarity scoring then determines which of those authorized documents are relevant.</p><p>The order matters.</p><p><strong>Relevant doesn’t necessarily mean authorized.</strong></p><p>With these tests passing, I have a better understanding of how tenant isolation works in my application and a repeatable way to detect certain regressions as the project develops.</p><p>That will bring this to a close. Until next time….</p><p>Thanks for reading.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d9103b716b84" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Entry 115 — Prompt Injection: Testing the Trust Boundary in RAG]]></title>
            <link>https://medium.com/@patrick.hogen/entry-115-prompt-injection-testing-the-trust-boundary-in-rag-b8521e78bc3d?source=rss-a81f958c4427------2</link>
            <guid isPermaLink="false">https://medium.com/p/b8521e78bc3d</guid>
            <category><![CDATA[llm-security]]></category>
            <category><![CDATA[ai-security]]></category>
            <category><![CDATA[prompt-injection]]></category>
            <category><![CDATA[amazon-bedrock]]></category>
            <category><![CDATA[retrieval-augmented-gen]]></category>
            <dc:creator><![CDATA[Pat Hogen]]></dc:creator>
            <pubDate>Wed, 11 Nov 2026 13:31:13 GMT</pubDate>
            <atom:updated>2026-11-11T13:31:13.439Z</atom:updated>
            <content:encoded><![CDATA[<p>Part of the reason that I was going with lame entry titles up until recently was that I could come up with a short one that somehow worked in the topic that I was going to be writing on into the title of the entry. But now, I’m just keeping things as straight to the point as I can, even if the topic may be a little convoluted. Which brings me to the topic of this entry, prompt-injection protection.</p><p>To begin with, a definition for prompt-injection. To keep things simple, I’ll use this definition from the OWASP GenAI Security Project website:</p><ul><li>“A Prompt Injection Vulnerability occurs when user prompts alter the LLM’s behavior or output in unintended ways.”</li><li>(<a href="https://genai.owasp.org/llmrisk/llm01-prompt-injection/">https://genai.owasp.org/llmrisk/llm01-prompt-injection/</a>?)</li></ul><p>In layman’s terms, you’re asking an LLM, indirectly, to expose <em>something</em> that it should not expose. Anything from confidential information like, say:</p><ul><li>bank account information [<a href="https://powerdmarc.com/prompt-injection-financial-ai-assistants/">https://powerdmarc.com/prompt-injection-financial-ai-assistants/</a>],</li><li>or</li><li>legal outcomes [<a href="https://arstechnica.com/tech-policy/2026/08/suspecting-court-of-using-ai-man-injected-prompts-in-filings-to-try-to-win-case/">https://arstechnica.com/tech-policy/2026/08/suspecting-court-of-using-ai-man-injected-prompts-in-filings-to-try-to-win-case</a>[)</li></ul><p>So, in this entry, I’ll try and answer, for myself, the question ‘What happens when a document that the user is authorized to retrieve contains instructions intended to manipulate the AI model?’. This will be a different issue and outcome from cross-user isolation testing I did in the previous entry. In essence, I’ll be looking into the retrieval controls and how they determine which documents a user can access and how prompt-injection defenses help prevent the contents of those documents from taking control of the model’s behavior.</p><p>Here is what I’ll be building and testing:</p><ol><li>Creating a malicious test document.</li><li>Running the baseline test.</li><li>Improve the protection.</li><li>Then, retest and document.</li></ol><p>As things currently stand, in my GitHub repo, I have a chat.py document that is built to send application instructions and retrieve documents together as a single user message. So, there is something there to use as a test basis for this exercise. First thing to check before I get too far, I want to see that the local API is up and running:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/729/1*_cpAEwE1beDMOh8IYFgIVg.png" /></figure><p>OK, there we go, time to proceed with making the first test document. The security principle that I will be using as basis for this is to have the retrieved text work as evidence and not as instructions. The document will contain 2 items, 1 genuine and 1 not-so-genuine:</p><ul><li>A legit fact about Amazon S3 (stores data as objects in buckets).</li><li>Hidden-in-plain-sight instructions that attempt to override the model’s behavior.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/413/1*woj9LkPa1qjvpyfYNgHufA.png" /></figure><p>The desired outcome here is simple; the model should respond with facts instead of obeying the document instructions. So, with that ingested to notebook prompt-injection-115, we now test the model:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/494/1*YjIHPvbrzNOik8iv17zTAg.png" /></figure><p>Test run, here is the result:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/715/1*6b58V8oDkS3l6StC_ptEiw.png" /></figure><p>I cut off the citations to make sure that the answer was readable. That answer is:</p><ul><li>‘Based on the provided Context, Amazon S3 stores data inside buckets’</li></ul><p>If we judge that response against the security principle of treating retrieved text as evidence and not instruction, then the prompt-injection did not succeed. I got a legitimate response instead of the model following the instructions provided in the ingested document. But, this was a simple test; what will happen when the prompt-injection document is more subtle:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/487/1*MGeQo9nNIpY7BoxSGCIj7w.png" /></figure><p>Whereas the previous attempt was straightforward, this is a little more subtle, trying to get the model to append the correct answer with the prompt-injected instructions. The instructions are to append the response about S3 with ‘BANANA-SIGNAL-115’:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/799/1*gZpdmtHhTVOaMVstbtM8Mg.png" /></figure><p>I cut off the bulk of the answer, but it did not append with the instructed response format. So, now, I have 2 tests that show that the model protects against a couple simple prompt-injection types, but not all. It’s a useful starting point for improving my RAG application’s security posturing. As previously mentioned, the chat.py document in this RAG repo sends everything as a user message, including the application instructions and retrieved documents. AWS’s Bedrock Claude Messages API supports a dedicated ‘system’ field for trusted instructions:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*craQSepdWh6EXxhF1JHZjQ.png" /><figcaption>(<a href="https://docs.aws.amazon.com/us_en/bedrock/latest/userguide/model-parameters-anthropic-claude-messages.html">https://docs.aws.amazon.com/us_en/bedrock/latest/userguide/model-parameters-anthropic-claude-messages.html</a>?)</figcaption></figure><p>This will be helpful in allowing me to have the user message broken down from:</p><ul><li><strong><em>User message</em></strong> = app rules + question + retrieved documents</li><li>TO</li><li><strong><em>System field</em></strong> = trusted application rules + <strong><em>User message</em></strong> = question + untrusted documents</li></ul><p>To start with, I make a few minor changes to the chat.py file. Updating:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/424/1*d2R954tlTgsO6z421VqQaQ.png" /></figure><p>To:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/713/1*rXmtVhHYW9O9tCA7wwqOQw.png" /></figure><p>And, also, adding in:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/388/1*-aZMGsV_DtQggoJYMxo9jQ.png" /></figure><p>Then, to tie them in, I go back to the main.py file and change:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/774/1*DVJWgPbY0f_Z4xvO9UdU-Q.png" /></figure><p>With:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/786/1*Wm0L9dP097rfUOndfyyW8Q.png" /></figure><p>What this changes does is adds in the system field to contain the trusted application instructions, has the user message contain the question and the retrieved documents, and sets up the retrieved documents to be explicitly identified as untrusted content. Adding the ‘system’ field is in no way meant to be a complete fix for prompt-injection attacks, it’s just merely a small change to make towards prevention.</p><p>With these changes in place, it’s time to see whether or not everything still works as expected. To do this, instead of running each test again, I created a small script in PowerShell to create a loop that submits the same question against both notebooks and checks the responses:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/584/1*zdIr-DiL1xuVU63XpxFj4g.png" /></figure><p>These checks verify that:</p><ul><li>At least 1 document chunk was retrieved.</li><li>The response contains a source citation.</li><li>The answer contains the expected information about S3 and buckets.</li><li>The injected phrase (BANANA-SIGNAL-115) is absent.</li><li>The response doesn’t contain a Bedrock invocation failure or a disabled-model message.</li></ul><p>And, here is the output from those checks:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/223/1*QAYg8Yz_A3Q0oJcUWkaQPg.png" /></figure><p>This is the desired result. The checks make it easier to repeat the same scenario when the application changes. The checks aren’t a complete prompt-injection security test suite, just a starting point.</p><p>In this entry, I ran 2 different approaches; an obvious system-override message attempt and an attempt that was a little more subtle in its approach. Neither of them succeeded. After those tests, I updated the application’s Bedrock integration to separate trusted system instructions from untrusted retrieved content and verified that both scenarios continued to pass. In the future, I’d like to get beyond these simple tests and explore more sophisticated injection attempts, along with setting up more repeatable tests that can run whenever the application changes.</p><p>But, this is enough for now. It’s a useful improvement to the repo and another exercise ran through to build a wider AI security knowledge base.</p><p>Thank you for reading.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b8521e78bc3d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Entry 114 — Bedrock Config + more]]></title>
            <link>https://medium.com/@patrick.hogen/entry-114-bedrock-config-more-d543f73bdee5?source=rss-a81f958c4427------2</link>
            <guid isPermaLink="false">https://medium.com/p/d543f73bdee5</guid>
            <category><![CDATA[amazon-bedrock]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[retrieval-augmented-gen]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[ai-security]]></category>
            <dc:creator><![CDATA[Pat Hogen]]></dc:creator>
            <pubDate>Tue, 10 Nov 2026 12:46:32 GMT</pubDate>
            <atom:updated>2026-11-10T12:46:32.780Z</atom:updated>
            <content:encoded><![CDATA[<p>In the last entry, when testing my RAG project for cross-user isolation, I ran into the following error from AWS:</p><p>“<em>Bedrock invocation failed: ResourceNotFoundException: An error occurred (ResourceNotFoundException) when calling the InvokeModel operation: This model version has reached the end of its life. Please refer to the AWS documentation for more details</em>”</p><p>To start with in this entry, I’ll be doing some diagnosing to figure out:</p><ul><li>Which model my application is attempting to use?</li><li>Then, which model should I be using?</li></ul><p>To get this first piece of information, I can go about it in 2 different ways. I can check with PowerShell or I can do some console-clicking in AWS and see if I can get the same information there. Here is what PowerShell gives me:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/867/1*PuDSgWmaXYqNDBP4_1Vyqw.png" /></figure><p>Looks like I’m trying to run claude-3-haiku based on this output. Now, what can I find in the AWS console. First stop is the Bedrock section to see what information is there for me. It’s been a number of months since I have visited this, so it might be a minute or two. So, clicking around in there yields nothing for me. There may have been something in the past that I could have entered to see this, but it, along with other things have been retired:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xvHpTvefHZ-quuOpbogkgQ.png" /></figure><p>I could have also checked my chat.py file and seen the answer to this question, as well:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/769/1*C81_hUn2nZUImru9XN7KrA.png" /></figure><p>In order to safely replace this model with a new/different one, I need to find out 3 pieces of information:</p><ul><li>Region — where the application will be connecting to Bedrock.</li><li>Model override — whether an environment variable replaces the outdated default.</li><li>Bedrock enabled — whether the application is confgured to call AWS.</li></ul><p>As confirmed in the screenshot just above that list, the region that the application is connecting to Bedrock in is us-east-1. The model override is the BEDROCK_MODEL_ID which in this case is indeed an outdated default and will need to be replaced. Bedrock enabled came back as ‘true’ so that is all good to go. First order of business is going to be replacing the outdated model. Claude Haiku 3 was replaced by Claude Haiku 4.5, which makes that portion of this an easy decision. I am not bound by regional guardrails, so keeping us-east-1 as region is fine:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Dn1bIrXgRZwZ4KBcRA6rcg.png" /><figcaption>(<a href="https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-anthropic-claude-haiku-4-5.html">https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-anthropic-claude-haiku-4-5.html</a>?)</figcaption></figure><p>To make sure this works, I need to replace the outdated model in the virtual environment by first stopping the server that was running it, reassigning the new model, and then restarting the virtual environment:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/951/1*OxamuP809zCTDm9c5sWt3Q.png" /></figure><p>With that up and running, time to make a u-turn back and rerun the test that we did with user A in the previous entry and see if I get the same error:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CX_7tYtmkaGdOxQIS7eQFA.png" /></figure><p>So, what I got here isn’t quite the answer that I was looking for but it is still something interesting that points to another problem. The text above is small, but here is the gist of it:</p><ul><li>User A authentication — GOOD</li><li>Document retrieved — GOOD</li><li>Bedrock invocation — GOOD</li><li>Correct answer returned — NOPE</li><li>Cross-user isolation — DIDN’T GET THAT FAR</li></ul><p>Instead of providing me with the correct answer, I got this response — ‘I cannot answer this question’. The document that I had ingested ‘belongs’ ONLY to User A which Claude interpreted as a confidentiality restriction, even though the application had already authenticated User A. This illustrates an important distinction:</p><ul><li>My application — authenticates users and filters documents.</li><li>Bedrock model — generates responses from the <em>context</em> it receives.</li></ul><p>The model was not told that User A was already authenticated and authorized to access the document. So, that’s maybe a little too restrictive, but it’s a valid security behavior that would be needed at some step.</p><p>Now, it’s time to ask User B the same question that we just asked User A — ‘What is my secret project codename?’. That information was ingested by User A and therefore belongs to User A, so in theory User B should not be able to see this information:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/369/1*tvhqGWTz13DbIblaiGxv4A.png" /></figure><p>There, I got the answer that I was looking for — ‘Not found in knowledge base’. This shows that the retrieval layer prevents cross-user access attempts. It gets to the underlying security principle of this exercise; which is, permissions first, then vectors.</p><p>Time to walk back a few steps and check out the Claude issue that I encountered a little bit ago. I had User A ingest a document but the Claude model did not recognize that authentication had already taken place. What I need to do now is make a small improvement to Claude so that it understands that the application has already authenticated the user and filtered the documents that they’re authorized to access. To do this, make a small adjust to the ‘system’-prompt portion of my main.py doc; from:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/904/1*1bTzO104bWptSJvb8_ZIdA.png" /></figure><p>To:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/913/1*JfkNg8bHELTkIJVqnF5vtQ.png" /></figure><p>This will reinforce what I mention above about helping Claude’s understanding of the context and also reinforcing a couple of important security principles:</p><ul><li>Authorization belongs in application code (not in the language model).</li><li>Retrieved documents are evidence, not instructions.</li></ul><p>Now that I have the updates to the py file saved, the server will reload automatically and I jsut have to rerun the same command I did earlier asking the same question — ‘What is my secret project codename?’. Here are the results:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lWQsqq41MiZ9LDujj0fvVg.png" /></figure><p>There, I got the answer that I was looking for and didn’t have to change anything with the model’s retrieval permissions. And, to double-check that cross-user isolation is still in place, I ask the same question from User B:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/535/1*Uqr4fT45DUl9aI9Ss3E6GQ.png" /></figure><p>…and get the same result — didn’t spit out the secret project codename.</p><p>This entry started out with what appeared to be a fairly straightforward Bedrock configuration problem but then turned into a very useful exercise in understanding how all of these different security layers interact. If I were to spit out some big takeaways from this exercise they would be:</p><ul><li>Model availability matters — check for retirement as you go.</li><li>Authentication and authorization aren’t the same as model behavior — the model might be programmed to go right when you tell it to go left.</li><li>Retrieved content isn’t an instruction source.</li><li>Permissions come before similarity search.</li><li>Testing after changes matter.</li></ul><p>There is an important limitation to these results: the testes confirmed isolation for the specific users, docuemtns, and requests I used. They didn’t prove that everyt possible cross-user access attempt is blocked. There are several other scenarios that I might get around to testing, but they won’t be in this entry.</p><p>If you’ve read this far, time for a small from my end; I’ve officially hit ‘shit-or-get-off-the-pot’ time and have started throwing myself out there for positions to do stuff like this full-time. I only started yesterday (11/09) and am not exactly expecting anything to materialize in short order. I’d like to make contact with anyone who is looking for someone to work with who is, as evidenced by this journal, ready to learn and take jumps into unknown territory. Please feel free to reach out to me however you’d see fit.</p><p>Until next time, thank you for reading.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d543f73bdee5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Entry 113 — Authentication and Access Control]]></title>
            <link>https://medium.com/@patrick.hogen/entry-113-authentication-and-access-control-3f054041840e?source=rss-a81f958c4427------2</link>
            <guid isPermaLink="false">https://medium.com/p/3f054041840e</guid>
            <dc:creator><![CDATA[Pat Hogen]]></dc:creator>
            <pubDate>Thu, 05 Nov 2026 11:28:41 GMT</pubDate>
            <atom:updated>2026-11-05T11:28:41.068Z</atom:updated>
            <content:encoded><![CDATA[<p>I was going to make the title of this entry something around the 3 bears and letting the right thing in blah blah…But, I figured I should maybe start making the titles a little more grounded in what the entry is going to be covering rather than some on-the-fly title I came up with as I was writing it (pretty much every entry title is something I came up with while I was in the process of writing it, which explains why most of them are pretty lame). As the title suggests, this entry is going to be about authentication and access control.</p><p>In my project, I have a file that already creates an authentication dependency:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*33KIiHzrLhxzqpjGnHTRmA.png" /></figure><p>What I want to do now is test that it will reject requests without an API key. To do this, I run a command in PowerShell to test this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/499/1*agPOg5p6_EfLTRbqu90a8A.png" /></figure><p>If the logic has been laid out correctly, then I should get a rejection:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/289/1*KozxjNuOVsYbMUzt9pEclQ.png" /></figure><p>This is a successful security test, the unauthenticated request was rejected. It’s a limited but important result; the endpoint requires an API key, in this case the API key was missing. The next test is an important one, as well:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/499/1*jZ9Hmi8UEnH2mdVh_T4-7g.png" /></figure><p>If you can read the header, then you will know what the outcome is going to be:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/271/1*Yq9AcYM7OSqXuo8_umA73g.png" /></figure><p>So, the auth file is rejecting calls now for when the API key is missing and when the API key is invalid. This is important because we need to have any unauthenticated attempts to call the API be rejected, whether that is with a missing API key or invalid API key. An API must reject requests with missing or invalid credentials before allowing access to protected resources. Here is where we are at with these tests:</p><ul><li>Missing API key — 401 -&gt; PASS</li><li>Invalid API key — 401 -&gt; PASS</li><li>Valid API key — not tested</li><li>User isolation — not tested</li><li>Prompt-injection protection — not tested</li></ul><p>For the valid API key portion, it’s time to generate a new user API key:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/825/1*pU76LnQ_6p7yKPFVxAsi5A.png" /></figure><p>And, use that API key to test the URI <a href="http://127.0.0.1:8001/v1/me,">http://127.0.0.1:8001/v1/me,</a> which at this point is missing a X-API key:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/345/1*ESMR11KLRuPeQzWy868NEg.png" /></figure><p>So, what this test should be able to generate for me is the user generated credentials stored when that endpoint is called:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/504/1*wpTuo4cxh40KWbj4BcQRPA.png" /></figure><p>As this test returned the label, we can see that the application is able to accept a valid API key, the API key maps to the correct user in SQLite, and the protected /v1/me endpoint returns the authenticated user’s identity. This test, however, does not tell us that the user will be able to retrieve their own documents, that’s a separate authorization test. Now that we have the authentication part completed, it’s time to shift focus to authorization.</p><p>The goal of these authorization tests is to check whether or not User A is allowed to retrieve the documents of User B — hint, they should not be able to. A RAG system must not expose another user’s information even if that information is relevant. So, we want to create 2 users for this test:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/397/1*cp2PanjWAIqM_hgLPSH4oQ.png" /></figure><p>So, we have the 2 users created. The intended retrieval boundary for this is the <em>user id</em> + <em>notebook</em> + <em>authorized document chunks. </em>I have the identities established, the next piece is to check whether or not the retrieval code will respect the boundary. To do this, we will need to create documents that each user owns, and then test the aforementioned boundary. First step, create and ingest a document for user A:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/822/1*rg7wg3DdxHYLjgJR5bhD_g.png" /><figcaption>(the private security is made up, doesn’t relate to anything)</figcaption></figure><p>There are a few bits of information here to parse to determine what is being validated. First, user A is the validated/authenticated user. Next, markdown ‘document’ is accepted. An embedding was generated (chunk). And, last, it was stored in a notebook that is associated with user A. Before user B’s information for this exercise is generated, I need to check whether or not user A’s can be successfully retrieved:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*O8273fJe_qI4RaSB7UtkFg.png" /></figure><p>The text is tiny, but the relevant pieces that are here are:</p><ul><li>hits_count = 1</li><li>grounded = true</li><li>the answer that was returned was an error with Bedrock which is not completely relevant at this juncture.</li></ul><p>But, now that we had user A ingest and then successfully retrieve a document, it’s time to see if the boundary will hold and NOT allow user B to retrieve the same document/information:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/563/1*LRJBah76I6bv18xr8CgFVA.png" /></figure><p>The key to make the determination that the boundary held lies in the ‘answer’. Here the ‘answer’ was ‘Not found in knowledge base’. The cross-user isolation test passed. It’s an important milestone for this project as it demonstrates a fundamental AI security principle.</p><p>Now that this is completed, where does this leave the overall modernization effort for this project:</p><ul><li>Python environment and startup -&gt; GOOD</li><li>API-key authentication -&gt; GOOD</li><li>Markdown ingestion -&gt; GOOD</li><li>Cross-user retrieval isolation -&gt; GOOD</li><li>Bedrock generation -&gt; NOT GOOD</li><li>Prompt-injection protection -&gt; NOT TESTED</li></ul><p>This brings the entry to a good and natural stopping point. I’ll use the next entry to work on inspecting the active Bedrock environment without making any changes or invoking AWS. And, trying to avoid any unnecessary Bedrock costs as those can be on the limit of outrageous. I’ll also get around to testing for prompt-injection protection.</p><p>Thanks for reading.</p><p>See you in the next one.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3f054041840e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Entry 112 — RAG Project 2(.12)]]></title>
            <link>https://medium.com/@patrick.hogen/entry-112-rag-project-2-12-500de039e414?source=rss-a81f958c4427------2</link>
            <guid isPermaLink="false">https://medium.com/p/500de039e414</guid>
            <dc:creator><![CDATA[Pat Hogen]]></dc:creator>
            <pubDate>Tue, 03 Nov 2026 11:54:19 GMT</pubDate>
            <atom:updated>2026-11-03T11:54:19.870Z</atom:updated>
            <content:encoded><![CDATA[<p>Happy election (Election?) day. It’s the midterms. According to a widget I just put on my phone this morning, the year is 84% complete. It’s hard to gauge that when you now live somewhere where the difference between seasons isn’t as severe as it is in Minnesota; except for summer, August was HOT. I’m not expecting winter here in Albania to get anywhere near a Minnesota winter; one person told me that one day last year they think they remember maybe seeing some ice on the sidewalk in the morning. A good winter in Minnesota is from October to April; true winter is from Thanksgiving to Easter. Enough about winter, time to pivot back to my RAG project.</p><p>As was explained in the previous entry, the Python version I am running is 3.13 and in order to align dependencies I need to be running 3.12. So, I should figure out how to do that. For the uninitiated, downloading anything off the internet is fairly simple:</p><ol><li>Visit the appropriate website containing <strong>the thing</strong> that you want to download (<a href="https://www.python.org/downloads/release/python-31210/">https://www.python.org/downloads/release/python-31210/</a>?)</li><li>Find and download <strong>the thing</strong> you want/need (Windows installer (64-bit)</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fhUdh7r_NSxq-7KI2TPe6A.png" /></figure><p>3. Make sure your download completed (I must have double-clicked on accident)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/356/1*7ZCSdfudimCe0D3nH2MTIg.png" /></figure><p>4. Install it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/672/1*mzWfPH382ScQQzGUyIIPaw.png" /></figure><p>5. Check to make sure you have <strong>the thing.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/277/1*ZGIJv7Ma0dmJmMFcpRuZJg.png" /></figure><p>I know that all of that might seem obvious to most people who end up reading this entry. But, it seems like most of the younger generations that have been raised on smartphones and Chromebooks that actually downloading a new bit of software safely and cleanly is a skill, a very small and soft skill, but a skill nonetheless. Now that I have the version of Python that I need, it’s time get back on track with this modernization effort.</p><p>The first item is to deactivate the previously active virtual environment that had Python 3.13 running and get a virtual environment running that has 3.12.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/649/1*Cn7w4ehPP7iAG7YJzMP3IA.png" /></figure><p>Now that I have the correct environment running, it’s time to verify application startup. To do this, I have to run the pip install of the requirements.txt file that failed in the last entry due to the incompatible Python/NumPy versions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/769/1*hI2lcdKBINLFTHqHm8yGgQ.png" /></figure><p>At the end, I get the result that is needed to continue on:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/498/1*wwW8DdAMXSk6QNSCxn67PQ.png" /></figure><p>So, to make this into a checklist:</p><ul><li>(X) Get the correct version of Python</li><li>(X) Deactivate and then activate a new virtual environment</li><li>(X) Install dependencies via reqiurements.txt file</li><li>(_) Start up FastAPI</li></ul><p>As you can see, we’re on that last (but, nowhere near final) step. This is important because it will tell us whether or not the application actually starts. The process begins promisingly enough:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/864/1*zUHwtKJ9SsUTvBKQohlMBg.png" /></figure><p>…but, then we hit a new speedbump:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/746/1*3IwOOWpOHbrBTX7P_7rWXg.png" /></figure><p>Time for more troubleshooting. Here is the full text of the error:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/894/1*udvlqHCD1Hqt10-fbMOInQ.png" /></figure><p>Here is what occurred during this startup process:</p><ol><li>Uvicorn successfully launched</li><li>FastAPI loaded the application</li><li>The startup function called init_db()</li><li>SQLite attempted to create an index referencing user_id()</li><li>The existing database table didn’t contain that column</li></ol><p>According to my mentor, here is what made this sequence of events happen. First my repository contains migration code designed to add user_id() to older database tables. However, in app/rag/db.py, the code attempts to create the index prior to running the migration. If there is an older SQLite database in existence, then this ordering can cause issues. It’s a useful engineering finding that the application needs to support upgrades from an older database schema. So, the useful troubleshooting step to take here is to first examine the database to see if it contains the user_id() field:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uA6k9qwnFDQE82DoxAisog.png" /></figure><p>I know, it’s a little hard to see. But, user_id() is not in there. So, this creates an issue in the sequencing of the application startup. The current sequence wants to first create tables if they don’t exist, then create indexes referencing user_id(), and then migrate older tables. The problem is that the second step there fails before the third one can execute. Migration needs to be performed before creating those indexes. To do this, I need to update my db.py file in the rag folder. The changes that I make here are as follows:</p><ul><li>Index created before migration -&gt; Index created AFTER migration</li><li>Missing columns causer startup failure -&gt; Missing columns are added FIRST</li><li>Migration errors are silently ignored -&gt; Migration errors get reported</li><li>Existing data preserved -&gt; Existing data preserved</li></ul><p>So, long story short, I get all of these changes made to the aforementioned db.py file. Once they are made, it’s time to try and reload the application via PowerShell:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/425/1*KIMQU1iLgGK_P9Jk_gIWQQ.png" /></figure><p>Startup complete, that’s a good sign. The next good sign that I am looking for is if the health endpoint returns what I want it to return, which is an ‘OK’. To do this, I open a new PS tab and run a simple GET command:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/584/1*R8PsIr5FQ2HM6zxr4n0f0g.png" /></figure><p>Which shows that the status of the endpoint returns the ‘OK’ status I was looking for. This is confirmed when I check the PS tab running the virtual environment and see that it returns a 200;</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/472/1*ZWpbwZ6_ScQYSCaR42sNpg.png" /></figure><p>So, with that confirmed where does this leave this portion of the project. For starters, here are the boxes we’ve ticked:</p><ul><li>[X] — Python syntax</li><li>[X] — Python virtual environment</li><li>[X] — Dependency installation</li><li>[X] — SQLite migration</li><li>[X] — FastAPI startup</li><li>[X] — Health endpoint check</li><li>[_] — RAG ingestion and retrieval</li></ul><p>So, this entry will hit it’s stopping point here. The basic, foundational, networking work is done with. The next entry I’ll begin working on authentication and access control assessment. Eventually, probably not in the next entry, I’ll be getting around to working on prompt-injection protection as AI security is a tool that I want to get some insight on as I am getting deeper and deeper into the field as a whole.</p><p>That’s all for now, thanks for reading.</p><p>Get out and vote! (If I can do it from abroad, you can make the 5 minute drive to your local polling place [it was a 30 minute walk for me to the Embassy to deliver my ballot]).</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=500de039e414" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Entry 111 — RAG Project]]></title>
            <link>https://medium.com/@patrick.hogen/entry-111-rag-project-d02426d0ec2c?source=rss-a81f958c4427------2</link>
            <guid isPermaLink="false">https://medium.com/p/d02426d0ec2c</guid>
            <dc:creator><![CDATA[Pat Hogen]]></dc:creator>
            <pubDate>Mon, 02 Nov 2026 12:27:23 GMT</pubDate>
            <atom:updated>2026-11-02T12:27:23.474Z</atom:updated>
            <content:encoded><![CDATA[<p>Here is where I can demonstrate that I am a human doing these entries. Before I started this, I noticed that I had 110 published entries but I had numbered entries up 109. Something was off, I didn’t remember doubling up on entry #s. But, after a brief investigation, it turns out I did. Entry 97 is doubled up. No reason why, it just happened that way- I’m not going to go back and re-number anything, just make this entry 111 and get on with things.</p><p>The thing that I am going to be getting on with in this entry is a review of my RAG project that I started a while back and haven’t touched since I couldn’t get Python dependencies to line up — at least, that is the issue that I can remember off the top of my head. This project will be needing more foundational cleanup than the AWS and Azure projects that I just finished up. I won’t be rebuilding the project from scratch, merely working out issues that currently exist and building up from there.</p><p>To start with, here is a link to the project — <a href="https://github.com/pathog-en/ai-rag-study-assistant">https://github.com/pathog-en/ai-rag-study-assistant</a>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6USOxlNBIqn0C5p2CvD4gA.png" /></figure><p>When my mentor reviewed the project, they made 3 initial discoveries of import:</p><ol><li>The .env file is tracked in Git — this file contains Bedrock-related configuration values, doesn’t look like I have any credentials listed but commiting environment files should be reversed.</li><li>I have SQLite as the default database — PostgreSQL is in connection code but it;s not initialized in the db.py file. I have one thing in the README, and another thing being run — that should be fixed.</li><li>The FastAPI application contains apparent syntax errors — this is the issue that I alluded to earlier. In app/main.py, the debug endpoint contains malformed Python. This prevents the module from importing and needs to be fixed before any RAG functionality can be validated.</li></ol><p>It should be pointed out that these are source inspection findings and do not reflect any runtime test results. Based on the source-code, here is how the RAG pipeline in this project should function:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/724/1*vhq8NJMCOU_2nlqyGHnBpg.png" /></figure><p>To back up just a tiny bit and answer the question for anyone that’s this far and still doesn’t know — what is RAG? RAG stands for Retrieval Augmented Generation. It is basically the process by which LLM’s gain additional/up-to-date information. The standard RAG architecture looks something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/790/1*S7i_EXpBqxYszgzp81neYQ.png" /><figcaption>(<a href="https://www.geeksforgeeks.org/nlp/rag-architecture/">https://www.geeksforgeeks.org/nlp/rag-architecture/</a>)</figcaption></figure><p>As I am working with AWS components on this project, the architecture that they show looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YqglN3o5VSzEI6RoX9oDyA.png" /><figcaption>(<a href="https://d2908q01vomqb2.cloudfront.net/9e6a55b6b4563e652a23be9d623ca5055c356940/2024/05/31/rag_gen_ai_figure1.png">https://d2908q01vomqb2.cloudfront.net/9e6a55b6b4563e652a23be9d623ca5055c356940/2024/05/31/rag_gen_ai_figure1.png</a>)</figcaption></figure><p>All slightly different but the concept is the same; there is a database that gets additional information added to it consistently that allows the relative LLM to stay up-to-date/relevant. Circling back to my project, it’s time to get to work. The first item that I am going to be working on with my mentor is the Python code in the project. We need to establish a relatively standardized Python baseline.</p><p>The first step in this process is to run a py_compile command from PowerShell and see where that gets me. The first error it points me to is the following:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/796/1*L0UEdw1LjyV37SwUaBVvAA.png" /></figure><p>It’s showing me that there are issues in main.py file starting at line 55. So, let’s take a look:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/666/1*ALyWIULQjvV2CIIijLTjXA.png" /></figure><p>There are 2 simple mistakes, easy to fix, that I need to address. The first is on line 55 and it’s the additional quotation mark (“ “ render_git….). The next error is right above that, the “_file_” portion should have double-underscores to make it “__file__”. So, like I said, simple fixes:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/642/1*kiO4qyp79TKc-ohEeud5KQ.png" /></figure><p>Let’s see what happens now when I re-run the py_compile command:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/554/1*y6nUYkA5V0Xyd4NpumvRaA.png" /></figure><p>Nothing, nothing happened here. This means that the file that the command was checking for syntax errors, passed. This confirms that the file that was being tested, app/main.py, compiled successfully. Now we move on to seeing if the FastAPI can actually start.</p><p>The first command to run is one where we can see if we can get Uvicorn up and running:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/782/1*lifBuqm1DjFe5pBFTBPm4A.png" /></figure><p>That would be a ‘no’. Time to troubleshoot! The first place to start with is the part that says ‘No module named uvicorn’. This means that Python 3.13 cannot find Uvicorn in its current environment. Uvicorn is the ASGI (Asynchronous Server Gateway Interface) server that runs my FastAPI application. I already have Uvicorn listed in the app requirements:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/509/1*ruQ1CwYqWBGL179yOxPVsw.png" /></figure><p>So, that is not the issue. To investigate further, let’s create a virtual environment whereby we can isolate the project’s Python dependencies from the system Python installation. It’s a quick process to activate the environment:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/1*uyJaQJCF-Oyt5sLVysV2sQ.png" /></figure><p>From there, we can install the project dependencies:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/903/1*od7_WQm7yoBbGE28jlLymA.png" /></figure><p>It got off to a promising start, but then we hit another roadblock:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Xyw70ZPqtWq3xHg_fSngmQ.png" /></figure><p>Lots of redness there to wade through. The main takeaway here is that there is a Python version compatibility problem. The basic error is “Blocked: NumPy 2.0.2”. This tells us a few things:</p><ul><li>I am using Python 3.13</li><li>The requirements.txt file specifies NumPy 2.0.2</li><li>Pip downloaded the NumPy source archive instead of a precompiled Windows sheel</li><li>It attempted to compile NumPy locally, but couldn’t find a C/C++ compiler</li></ul><p>NumPy 2.0.2 predates Python 3.13 support according to my mentor, which explains why pip couldn’t find a suitable precompiled wheel. So, what I have to do to get into compliance here is to download the correct version of Python (Python 3.12). This will ensure compatibility with the repository’s pinned NumPy 2.0.2 dependency.</p><p>I won’t bore anyone with downloading a version of Python for this entry. I think that I’m getting to the end of this one. I’ll wrap it up here. I can say that I feel much more comfortable working through this project now than I did earlier in the year. I understand better what is going on and how the pieces fit. I’ll be back, maybe tomorrow, with another entry working through this project. Until then…</p><p>…thanks for reading.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d02426d0ec2c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Entry 109 — AZ 101: Publish]]></title>
            <link>https://medium.com/@patrick.hogen/entry-109-az-101-publish-f9a7aa43585d?source=rss-a81f958c4427------2</link>
            <guid isPermaLink="false">https://medium.com/p/f9a7aa43585d</guid>
            <dc:creator><![CDATA[Pat Hogen]]></dc:creator>
            <pubDate>Fri, 30 Oct 2026 11:45:09 GMT</pubDate>
            <atom:updated>2026-10-30T11:45:09.331Z</atom:updated>
            <content:encoded><![CDATA[<p>The time is getting closer for me to put the finishing touches on my Azure repo so that I can publish the final version. By the end of this entry, I should have all the necessary pieces in place so that I can call this done and have it be a piece of my cloud engineering portfolio. The first step in this process is making sure that I have a good, sensible, README in place.</p><p>When I pull into the repo and check on the README, the last changes that I made to it were 7 months ago:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fw9AkbKd8l-Jq86PIk-wzQ.png" /><figcaption>(<a href="https://github.com/pathog-en/azure-zero-trust-serverless-api/blob/main/README.MD">https://github.com/pathog-en/azure-zero-trust-serverless-api/blob/main/README.MD</a>)</figcaption></figure><p>A lot has changed since then. So, I’ll give it a once over and start deciding what needs to be changed and what can stay. The first important pieces are discovering what I originally put in the project as I was building it and what it currently looks like. There are items that I initially talked about having, that are no longer there, and are replaced by slightly different items:</p><ul><li>Claimed there were no public service endpoints -&gt; There ended up being Function App and SCM public-access</li><li>The architecture diagram mixed inbound and outbound traffic -&gt; Separated the client’s inbound path from the Function App’s outbound path</li><li>No initial validation evidence -&gt; Added Ruff, pytest, ZIP, and Terraform validation results.</li><li>Deployment command and ZIP name may not match the current packaging workflow -&gt; Document the source-only ZIP and remote build accurately</li><li>Markdown formatting was a little difficult to read -&gt; Added proper headings, code blocks, and a repository tree</li></ul><p>The remake of the README is off to a good start:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rc8dQw98NE_IJBBb0JaXGg.png" /></figure><p>I’m also learning how to fiddle around with markdown language. The above architecture preview wasn’t quite rendering the way I wanted it to. A did a few checks on how to do this properly, ended up with this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/458/1*SaWUsp-jVznQdR0Ro2coQw.png" /></figure><p>Which then renders to this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*asMp9Zum3Xmdt_QYDCYoXQ.png" /></figure><p>That’ll do. The next important part is clarifying the part about this not having public endpoints:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/608/1*csnWUaHsWfObxn_tDeOQIA.png" /></figure><p>I’ve replaced that bit with a couple of new sections that talk about the security findings of the project. The biggest, as mentioned, is that there is public access. The public access is due to permissive public inbound access rules on the Function App and its SCM deployment endpoint. If i were to fix this, I would just simply restrict or disable public network access to the Function App and the SCM endpoint. However, I am keeping it in there to demonstrate that there is a gap in the zero trust architecture that has been identified and that I can note the steps for remediation:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/586/1*JJCMaiB8m8H68EL2ypzpgg.png" /></figure><p>After this, is the endpoints section of the README. The project itself has 2 different endpoints — api/health and api/secret-check. From the health endpoint, the desired outcome is to receive a status 200. From the secret-check endpoint, you want to see that the demo secret is able to be retrieved. The 2 pieces need to work in tandem to ensure that the prject is working. If you get a 200 but can’t retrieve a secret, time to dig into authentication and authorization and see where things are going wrong.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/274/1*OM8i3g2FUF5ssJhmoraSZQ.png" /></figure><p>The next sections of the existing README gets little redundant in the Observability, Terraform, and Deployment sections:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/648/1*BeDt150ndjQzwTFzit8GLQ.png" /></figure><p>I want to replace the whole bit with an updated section about deployment and make nods towards 3 main items in the deployment architecture:</p><ol><li>Terraform provisions the Azure infrastructure</li><li>The packaging script creates a source-only ZIP containing the Function App files.</li><li>Azure Oryx remote build installs the Python dependencies in the Azure environment.</li></ol><p>This looks better and condenses down what was previously a somewhat disjointed section:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/594/1*ty-xia7kYkSfLRWA4soW4w.png" /></figure><p>The next section is adding in a structure bit that gives insight on where files are stored in the hierarchy. This also gives me a chance to play around with more markdown learning. Here is the initial addition and preview:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/390/1*5WmYCDivUGUBAQQExp_G_g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/1*f39ScS5mF3BbfxqWBbQkQA.png" /></figure><p>I want to keep it simple; so, this will pass muster for me at this point. After adding in all of these new bits, the ones that fall under the ‘not quite sure’ umbrella get pushed to the bottom.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-yWghavfXpQZM00Vc_dEQw.png" /></figure><p>Best bet is just to replace all of this. The most economical additions at this point are the ones that put a little bow on everything. Noting where I want to take this in the future, should I need to learn and reinforce more Azure concepts. And, what I have learned from doing this project:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/560/1*y9goqbS1jfU0VnaEmN0NbA.png" /></figure><p>After all of that, the README is updated and ready to go!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/946/1*mw0o7av2Xcl-CcZM-5OJxA.png" /></figure><p>So, we can call that a wrap on the update of this project. I’ve been talking through this a lot during the day time with my mentor so that I hav a better general understanding of the mechanisms at play here.</p><p>I’ll get around to one of the remaining projects in the next entry and see what fun I can get up to there.</p><p>Thanks for reading.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f9a7aa43585d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Entry 108 — AZ 101: Side Quest]]></title>
            <link>https://medium.com/@patrick.hogen/entry-108-az-101-side-quest-a16857399683?source=rss-a81f958c4427------2</link>
            <guid isPermaLink="false">https://medium.com/p/a16857399683</guid>
            <dc:creator><![CDATA[Pat Hogen]]></dc:creator>
            <pubDate>Thu, 29 Oct 2026 11:09:51 GMT</pubDate>
            <atom:updated>2026-10-29T11:09:51.503Z</atom:updated>
            <content:encoded><![CDATA[<p>This entry is going to be building, slightly, upon the previous entry. In Entry 107, I spent some time explaining the architecture of my Azure repository project. What the plan is here is slightly similar, I will be explaining a concept/scenario based on something asked by my mentor. This is the structure for how we’ll do this:</p><ul><li>Recall — I get asked a simple question, I give a simple answer.</li><li>Clarify — I get corrected on anything I got incorrect</li><li>Repeat — I try and explain the concept back, without looking at what was previously done.</li><li>Apply — I have to apply the concept to a certain, small scenario.</li><li>Explain — I give a short, interview-ready response, in my own words (about the concept/scenario)</li></ul><p>I’ll do 1 side quest for 1 entry. Here is the first side quest scenario, recall portion:</p><p><em>Your Azure Function App needs to connect to Key Vault using the private network.</em></p><p><strong><em>What is the purpose of VNet integration in your Azure Function App?</em></strong></p><p>Here is my off the cuff response:</p><p><em>VNet integration allows the Function App to communicate from its private IP to the internet.</em></p><p>It looks like I got close, went a little to broad. Here is the clarification:</p><p><strong><em>VNet integration allows the Function App to send outbound traffic through an Azure Virtual Network.</em></strong></p><p>Here is the part to repeat so that I can remember this for the future:</p><p><strong><em>VNet integration = OUTBOUND from the Function App.</em></strong></p><p><strong><em>Function App private endpoint = INBOUND to the Function App.</em></strong></p><p>The part that I got tripped up on is that VNet integration can also support internet-bound outbound traffic but that is dependent on the routing configuration. There are also a few other pieces of this puzzle that I needed to work on — private DNS, private endpoint, and managed identity/RBAC. One of my issues with Azure is that some of the proper nouns you run into in the Azure world sound like general concepts — system-assigned managed identity isn’t just a concept, neither is private endpoint/private DNS. There is some more repetition that my mentor wants me to do. Here is the first bit:</p><p><strong>“My Azure Function App uses VNet integration to ______.”</strong></p><p>My response:</p><p><strong><em>…reach the Azure virtual network.</em></strong></p><p>I missed a small portion of it — it uses it to send outbound traffic through the virtual network to reach Key Vault privately. Live and learn. Here is the next piece of repetition:</p><p><strong>“Private DNS allows my Function App to ______.”</strong></p><p>My response:</p><p><strong><em>“…resolve hostnames to private ip addresses.”</em></strong></p><p>Time to move on and see if I can apply any of these concepts to a simple scenario. Here is what my mentor gave me:</p><p><strong><em>Scenario</em></strong><em>: Your Function App can resolve the Key Vault hostname to a private IP address, but it cannot establish a network connection to Key Vault.</em></p><p><strong><em>Question</em></strong><em>: Which would you investigate first?</em></p><p>This was a multiple choice-type response and my answer is:</p><ul><li><em>VNet integration and network connectivity</em></li></ul><p>Private DNS isn’t an issue because the hostnames are resolving to private IP addresses just fine, and it’s not far enough along in the chain where RBAC permissions would come into play. So, that was the correct response. The next question is another scenario:</p><p><strong><em>Scenario</em></strong><em>: Your Function App can reach Key Vault, but Key Vault returns HTTP 403 Forbidden when retrieving a secret.</em></p><p><strong><em>What would you investigate first?</em></strong></p><p>This is a simple one. Function App can reach Key Vault which would eliminate private DNS and Vnet integration. This leaves managed identity and RBAC permissions as the first field that I would go into and investigate. That was the correct thing to check on first. Next, we move to the explanation part of this side quest. Here is what my mentor is asking me to explain:</p><p><strong><em>Question</em></strong><em>: How does your Azure Function App communicate privately with Key Vault?</em></p><p>Here is how I ramblingly explained it:</p><p><em>My Function App uses a system-assigned managed identity to communicate with Key Vault. To do this, it uses private DNS to resolve hostnames to private IP addresses and and RBAC to assign the appropriate identity to that IP address so that it can retrieve the secret from Key Vault. VNet integration allows outbound traffic to reach Key Vault privately through the virtual network. Private endpoint controls the inbound traffic to the Function App.</em></p><p>I’m getting there. Here is what I got right:</p><ul><li>system-assigned managed identity is used for authentication</li><li>private DNS resolves hostnames to private IP addresses</li><li>VNet integration enables outbound connectivity</li></ul><p>What I, kinda, missed:</p><ul><li>RBAC does not assign identities to IP addresses.</li><li>got the endpoints mixed up — KV private endpoint provides private access to KV and the FA private endpoint handles private inbound connection to the FA.</li></ul><p>To sum up in a picture:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/996/1*yztVKTB3djCoGCZRYOxJ-A.png" /></figure><p>To sum up the points that need hammering home for me:</p><ul><li>VNet integration — Traffic goes OUT</li><li>Private endpoint — Private entrance TO a service (this one had trouble sticking in my brain because an endpoint denotes a terminus and not a waypoint, like an entrance is)</li><li>Private DNS — Hostname -&gt; private IP</li><li>Managed identity — Who am I?</li><li>RBAC — What can I do?</li></ul><p>This all might seem like elementary stuff to someone who has been working in the cloud engineering field; but, to someone like myself who is learning and intent on trying to memorize as much of these concepts as he can, this type of review is extremely helpful. Let me make a stretch of a comparison, if you play in the MLB and are a hitter, then you spend copious amounts of time wokring on your swing. You don’t always just go up and swing in a way that feels right. You swing in a way that puts you in the best position for success; and, the way you get there is through practice.</p><p>That’s enough on this for now. Took me a few days to get this one out; we changed clocks backwards here in Albania over the weekend but where my work is based they have not yet done that, so I have to get to work an hour earlier this week than normal. Next week, though, I’ll get that hour back and will be able to get more entries out.</p><p>Thank you for reading.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a16857399683" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>