<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Elad hirsch on Medium]]></title>
        <description><![CDATA[Stories by Elad hirsch on Medium]]></description>
        <link>https://medium.com/@hirsch.elad?source=rss-13ba3d3d9bf8------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*A09_Iux8_oNty9X7qMv9bw.jpeg</url>
            <title>Stories by Elad hirsch on Medium</title>
            <link>https://medium.com/@hirsch.elad?source=rss-13ba3d3d9bf8------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 11:25:30 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@hirsch.elad/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[The AWS Feature That Changes How You Debug AccessDenied]]></title>
            <link>https://medium.com/@hirsch.elad/the-aws-feature-that-changes-how-you-debug-accessdenied-30bcef4e0f6e?source=rss-13ba3d3d9bf8------2</link>
            <guid isPermaLink="false">https://medium.com/p/30bcef4e0f6e</guid>
            <category><![CDATA[aws-iam]]></category>
            <category><![CDATA[authorization]]></category>
            <category><![CDATA[security]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[access-denied]]></category>
            <dc:creator><![CDATA[Elad hirsch]]></dc:creator>
            <pubDate>Tue, 17 Feb 2026 20:48:11 GMT</pubDate>
            <atom:updated>2026-02-17T20:55:06.468Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HCuTbvvSlsZjNP0DiKf4Yw.png" /></figure><h3>Introduction</h3><p>If you have worked with AWS long enough, you have a relationship with AccessDenied. It shows up in Terraform plans, CI/CD pipelines, Lambda logs, and always at the worst possible moment.</p><p>This is not surprising. The <a href="https://docs.aws.amazon.com/pdfs/IAM/latest/UserGuide/iam-ug.pdf">AWS IAM User Guide is 4k+ pages,</a> and the policy evaluation engine checks up to seven distinct layers before making an access decision. Engineers consistently run out of time and patience before achieving least privilege — and when something breaks, the feedback is even worse.</p><p>The error itself is not the problem. The problem is that <strong>AccessDenied</strong> never told you <em>why</em>. You knew the action was blocked. You knew the resource. But you did not know which of the half-dozen policy layers in your AWS environment was responsible. Was it the identity policy? The permission boundary? An SCP in your organization? A resource policy on the target?</p><p>And just few weeks ago in end of January 2026, AWS quietly changed this. Access denied error messages now include the ARN of the specific policy that caused the denial — for same-account and same-organization scenarios. This article walks through why this matters, what it looks like in practice, and provides a Terraform module you can deploy in five minutes to see the enhanced errors yourself.</p><h3>The Debugging Nightmare</h3><p>AWS evaluates every API request against multiple policy layers. When any layer denies access, the request fails. The challenge is that there are many layers, and before this change, the error message did not tell you which one was responsible.</p><h3>The Policy Evaluation Layers</h3><p>When you make an AWS API call, the IAM engine checks:</p><ol><li><strong>Service Control Policies (SCPs)</strong> — organization-level guardrails</li><li><strong>Resource Control Policies (RCPs)</strong> — organization-level resource restrictions</li><li><strong>Identity-based policies</strong> — policies attached to your IAM user or role</li><li><strong>Resource-based policies</strong> — policies attached to the target resource (S3 bucket policy, KMS key policy, etc.)</li><li><strong>Permission boundaries</strong> — maximum permission limits attached to an IAM entity</li><li><strong>Session policies</strong> — temporary restrictions applied during role assumption</li><li><strong>VPC endpoint policies</strong> — network-level restrictions on API access</li></ol><p>The evaluation rule is state that <strong>explicit Deny wins, always.</strong> If any single layer explicitly denies an action, the request fails regardless of what every other layer allows.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xHz9tHPUk3SNA9OUdqM-nA.png" /><figcaption><em>Source: AWS IAM documentation — Policy evaluation logic</em></figcaption></figure><h3>What the Error Looked Like</h3><p>Before January 2026, a typical error looked like this:</p><pre>An error occurred (UnauthorizedOperation) when calling the TerminateInstances operation:<br>You are not authorized to perform this operation.</pre><p>That is all you got. No policy name. No layer. No hint. From this message, the denial could have been caused by:</p><ul><li>An SCP on the account’s organizational unit</li><li>A permission boundary on the role</li><li>An explicit Deny in one of the role’s identity policies</li><li>A resource policy</li><li>A VPC endpoint policy</li><li>A session policy applied during AssumeRole</li></ul><h3>The Old Way using decode-authorization-message</h3><p>The only diagnostic tool available was aws sts decode-authorization-message. When EC2 returned an encoded message, you could decode it — and get something like this:</p><pre>{<br>  &quot;allowed&quot;: false,<br>  &quot;explicitDeny&quot;: true,<br>  &quot;matchedStatements&quot;: {<br>    &quot;items&quot;: [<br>      {<br>        &quot;statementId&quot;: &quot;DenyTerminateInstances&quot;,<br>        &quot;effect&quot;: &quot;DENY&quot;,<br>        &quot;principals&quot;: {<br>          &quot;items&quot;: [{ &quot;value&quot;: &quot;AIDACKCEVSQ6C2EXAMPLE&quot; }]<br>        },<br>        &quot;policyArn&quot;: &quot;arn:aws:iam::123456789012:policy/access-denied-demo-deny-terminate&quot;,<br>        &quot;actions&quot;: {<br>          &quot;items&quot;: [{ &quot;value&quot;: &quot;ec2:TerminateInstances&quot; }]<br>        },<br>        &quot;resources&quot;: {<br>          &quot;items&quot;: [{ &quot;value&quot;: &quot;arn:aws:ec2:us-east-1:123456789012:instance/*&quot; }]<br>        }<br>      }<br>    ]<br>  },<br>  &quot;failures&quot;: {},<br>  &quot;context&quot;: {<br>    &quot;principal&quot;: {<br>      &quot;id&quot;: &quot;AROAEXAMPLE:scenario1&quot;,<br>      &quot;arn&quot;: &quot;arn:aws:sts::123456789012:assumed-role/access-denied-demo-identity-role/scenario1&quot;<br>    },<br>    &quot;action&quot;: &quot;ec2:TerminateInstances&quot;,<br>    &quot;resource&quot;: &quot;arn:aws:ec2:us-east-1:123456789012:instance/i-0abc123def456789&quot;,<br>    &quot;conditions&quot;: {<br>      &quot;items&quot;: [<br>        { &quot;key&quot;: &quot;ec2:InstanceMarketType&quot;, &quot;values&quot;: { &quot;items&quot;: [{ &quot;value&quot;: &quot;on-demand&quot; }] } },<br>        { &quot;key&quot;: &quot;aws:Region&quot;, &quot;values&quot;: { &quot;items&quot;: [{ &quot;value&quot;: &quot;us-east-1&quot; }] } },<br>        { &quot;key&quot;: &quot;aws:Resource&quot;, &quot;values&quot;: { &quot;items&quot;: [{ &quot;value&quot;: &quot;instance/i-0abc123def456789&quot; }] } },<br>        { &quot;key&quot;: &quot;ec2:InstanceType&quot;, &quot;values&quot;: { &quot;items&quot;: [{ &quot;value&quot;: &quot;t3.nano&quot; }] } },<br>        { &quot;key&quot;: &quot;ec2:ResourceTag/Name&quot;, &quot;values&quot;: { &quot;items&quot;: [{ &quot;value&quot;: &quot;access-denied-demo-test-instance&quot; }] } }<br>      ]<br>    }<br>  }<br>}</pre><p>The information is there — buried in a JSON blob with condition keys, nested objects, and principal IDs instead of readable ARNs. And this only works for EC2. Most other services did not provide encoded messages at all.</p><pre>The decoded message provides forensic depth — statement IDs, x condition keys (instance type, AZ, tags, metadata settings), nested objects, and principal IDs. But it does NOT include the policy ARN. You still need to manually search through every attached policy to find which one contains the statement `&quot;Sid&quot;: &quot;DenyTerminateInstances&quot;`. And this only works for EC2 — most other services did not provide encoded messages at all.<br><br>The enhanced error gives you what the decoded message never could: the direct path to the blocking policy.</pre><h3>The Manual investigation</h3><p>Without decode-authorization-message (or when the service did not support it), debugging meant checking every policy layer manually:</p><pre># Step 1: Check identity policies on the role<br>aws iam list-attached-role-policies --role-name MyRole<br>aws iam list-role-policies --role-name MyRole<br># Read each policy, search for Deny statements...<br><br># Step 2: Check permission boundary<br>aws iam get-role --role-name MyRole --query &#39;Role.PermissionsBoundary&#39;<br># If boundary exists, read the policy, check for Deny...<br><br># Step 3: Check resource policy<br>aws s3api get-bucket-policy --bucket my-bucket<br># Read the bucket policy, look for Deny statements...<br><br># Step 4: Check SCPs (need Organizations access)<br>aws organizations list-policies-for-target \<br>  --target-id &lt;account-id&gt; \<br>  --filter SERVICE_CONTROL_POLICY<br># For each SCP, read the policy document...<br><br># Step 5: Check RCPs<br>aws organizations list-policies-for-target \<br>  --target-id &lt;account-id&gt; \<br>  --filter RESOURCE_CONTROL_POLICY<br># For each RCP, read the policy document...<br><br># Step 6: Check VPC endpoint policies (if applicable)<br>aws ec2 describe-vpc-endpoints --query &#39;VpcEndpoints[].PolicyDocument&#39;<br># Read each endpoint policy...</pre><p>In a typical enterprise environment with multiple SCPs, boundary policies, and resource policies, this investigation could take thirty minutes to several hours. Multiply that across teams and incidents, and IAM debugging becomes one of the largest hidden costs in cloud operations.</p><h3>With enhanced error gessages</h3><p>AWS now includes the policy ARN directly in the error message when an explicit deny blocks the request. The format:</p><pre>User: &lt;user-arn&gt; is not authorized to perform: &lt;action&gt; on resource: &lt;resource-arn&gt;<br>with an explicit deny in a &lt;policy-type&gt;: &lt;policy-arn&gt;</pre><h3>Error Examples by Policy Type</h3><p><strong>Identity-based policy:</strong></p><pre>User: arn:aws:sts::123456789012:assumed-role/my-role/session is not authorized<br>to perform: ec2:TerminateInstances on resource:<br>arn:aws:ec2:us-east-1:123456789012:instance/i-0abc123def456789<br>with an explicit deny in an identity-based policy:<br>arn:aws:iam::123456789012:policy/deny-terminate</pre><p><strong>Permission boundary:</strong></p><pre>User: arn:aws:sts::123456789012:assumed-role/my-role/session is not authorized<br>to perform: ec2:StopInstances on resource:<br>arn:aws:ec2:us-east-1:123456789012:instance/i-0abc123def456789<br>with an explicit deny in a permissions boundary:<br>arn:aws:iam::123456789012:policy/my-boundary</pre><p><strong>Service Control Policy:</strong></p><pre>User: arn:aws:iam::123456789012:user/John is not authorized to perform:<br>kms:CreateKey with an explicit deny in a service control policy:<br>arn:aws:organizations::777788889999:policy/o-exampleorgid/service_control_policy/p-scp123</pre><p><strong>Session policy:</strong></p><pre>User: arn:aws:iam::123456789012:user/John is not authorized to perform:<br>dynamodb:PutItem with an explicit deny in a session policy:<br>arn:aws:iam::123456789012:policy/CISessionPolicy</pre><h3>Try It Yourself , Terraform Demo Module</h3><p>To see the enhanced errors in your own account, I have created a Terraform module that sets up two scenarios in under five minutes. The module creates IAM roles, policies, and a small EC2 instance (t3.nano) to trigger real errors against.</p><h3>Prerequisites</h3><ul><li>An AWS account with IAM permissions to create roles, policies, and EC2 instances</li><li>Terraform 1.5+</li><li>AWS CLI v2 configured with credentials</li></ul><h3>Deploy</h3><pre>git clone https://github.com/awslabs/iam-policy-autopilot.git<br>terraform init<br>terraform apply</pre><h3>Scenario 1 , Identity-Based Explicit Deny</h3><p>This role has two policies attached:</p><ul><li>An inline policy that <strong>allows</strong> ec2:Describe*, ec2:TerminateInstances, and ec2:StopInstances</li><li>A managed policy that <strong>explicitly denies</strong> ec2:TerminateInstances</li></ul><p>The Allow and Deny conflict. Explicit Deny wins — but before January 2026, the error never told you which policy was the blocker.</p><pre># Assume the role<br>CREDS=$(aws sts assume-role \<br>  --role-arn $(terraform output -raw identity_role_arn) \<br>  --role-session-name scenario1 \<br>  --query &#39;Credentials&#39; --output json)<br><br>export AWS_ACCESS_KEY_ID=$(echo $CREDS | jq -r .AccessKeyId)<br>export AWS_SECRET_ACCESS_KEY=$(echo $CREDS | jq -r .SecretAccessKey)<br>export AWS_SESSION_TOKEN=$(echo $CREDS | jq -r .SessionToken)<br>INSTANCE_ID=$(terraform output -raw test_instance_id)<br><br># This succeeds — Describe* is allowed by the identity policy<br>aws ec2 describe-instances --instance-ids $INSTANCE_ID --region us-east-1<br><br># This fails — TerminateInstances is explicitly denied<br>aws ec2 terminate-instances --instance-ids $INSTANCE_ID --region us-east-1 2&gt;&amp;1</pre><p><strong>You will then see the enhanced error:</strong></p><pre>An error occurred (UnauthorizedOperation) when calling the TerminateInstances operation:<br>You are not authorized to perform this operation. Encoded authorization failure message:<br>...<br>User: arn:aws:sts::123456789012:assumed-role/access-denied-demo-identity-role/scenario1<br>is not authorized to perform: ec2:TerminateInstances on resource:<br>arn:aws:ec2:us-east-1:123456789012:instance/i-0abc123def456789<br>with an explicit deny in an identity-based policy:<br>arn:aws:iam::123456789012:policy/access-denied-demo-deny-terminate</pre><p>The error points directly to the guardrail policy. No manual investigation needed.</p><pre># Clean up credentials before next scenario<br>unset AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_SESSION_TOKEN</pre><h3>Scenario 2 , The Multi-Layer Nightmare</h3><p>This is the scenario that used to take an hour to debug. A single role with three policy layers:</p><ul><li><strong>Identity policy</strong>: allows ec2:*</li><li><strong>Permission boundary</strong>: allows most actions but explicitly denies ec2:StopInstances</li><li><strong>Managed policy</strong>: explicitly denies ec2:TerminateInstances</li></ul><p>Two different actions fail for two completely different reasons.Before January 2026, both said the same thing: “You are not authorized to perform this operation.”</p><pre># Assume the nightmare role<br>CREDS=$(aws sts assume-role \<br>  --role-arn $(terraform output -raw nightmare_role_arn) \<br>  --role-session-name scenario2 \<br>  --query &#39;Credentials&#39; --output json)<br><br>export AWS_ACCESS_KEY_ID=$(echo $CREDS | jq -r .AccessKeyId)<br>export AWS_SECRET_ACCESS_KEY=$(echo $CREDS | jq -r .SecretAccessKey)<br>export AWS_SESSION_TOKEN=$(echo $CREDS | jq -r .SessionToken)<br>INSTANCE_ID=$(terraform output -raw test_instance_id)<br><br># Blocked by IDENTITY POLICY:<br>aws ec2 terminate-instances --instance-ids $INSTANCE_ID --region us-east-1 2&gt;&amp;1<br><br># You will see something like - explicit deny in an identity-based policy:<br># arn:aws:iam::123456789012:policy/access-denied-demo-deny-terminate<br><br># Blocked by PERMISSION BOUNDARY:<br>aws ec2 stop-instances --instance-ids $INSTANCE_ID --region us-east-1 2&gt;&amp;1<br><br># You will see something like - with an explicit deny in a permissions boundary:<br># arn:aws:iam::123456789012:policy/access-denied-demo-boundary</pre><p>So it’s the same role but two different errors. Two different policy layers identified in the error message. Each one tells you exactly where to look.</p><p>For comparison, the decode-authorization-message output for the boundary denial shows the principal as SCOPE_POLICY_ISSUER — a cryptic identifier that tells you a boundary was involved, but not which one. The enhanced error gives you the exact ARN.</p><h3>Clean Up</h3><pre>unset AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_SESSION_TOKEN<br>terraform destroy -auto-approve</pre><blockquote><strong><em>Note:</em></strong><em> The module creates a </em><em>t3.nano EC2 instance. Run </em><em>terraform destroy when you are done to avoid charges .<br>p.s — can you guess what is the yearly cost of a t3.nano in us-east-1 for a year without checking in AWS Pricing Calculator :-) ?</em></blockquote><h3>What This Changes for Operations Teams</h3><h3>Shorter Incident Response</h3><p>The manual “policy archaeology” workflow collapses. Instead of checking every layer, read the error, go to the policy, fix it. What took thirty minutes to an hour now takes two minutes.</p><h3>Safer Layered IAM Architecture</h3><p>Many organizations avoid layering SCPs, permission boundaries, and identity policies because debugging conflicts was too painful. Now that the error message identifies the blocking layer, teams can use defense-in-depth IAM architectures with confidence.</p><h3>Better operational runbooks</h3><p>If the error mentions <strong>service control policy</strong> → contact the platform team that manages SCPs in AWS Organizations</p><ul><li>If the error mentions <strong>permissions boundary</strong> → contact the team that manages IAM boundaries</li><li>If the error mentions <strong>identity-based policy</strong> → review the role’s attached policies</li><li>If the error mentions <strong>session policy</strong> → check the CI/CD system that applied session restrictions during role assumption</li></ul><h3>Log Enrichment</h3><p>Since the policy ARN appears in the error message (and in CloudTrail), log pipelines can extract it as a structured field:</p><pre># Example: parse policy ARN from CloudTrail AccessDenied events<br>import re<br>pattern = r&#39;in a (\w[\w\s-]*): (arn:aws:[^\s]+)&#39;<br>match = re.search(pattern, error_message)<br>if match:<br>    policy_type = match.group(1)  # &quot;permissions boundary&quot;<br>    policy_arn = match.group(2)   # &quot;arn:aws:iam::123456789012:policy/...&quot;</pre><p>Dashboards can group AccessDenied events by policy ARN, alerting on spikes tied to a specific guardrail.</p><h3>Limitations to Know</h3><p><strong>Same-account and same-organization only.</strong> Cross-account requests outside your AWS Organization still return generic “Access Denied.” Cross-account debugging remains manual.</p><p><strong>Explicit deny focus.</strong> The enhanced messages are most useful when an explicit Deny statement blocks the request. A missing Allow (implicit deny) produces a different message format:</p><pre>User: &lt;arn&gt; is not authorized to perform: &lt;action&gt;<br>because no identity-based policy allows the &lt;action&gt; action</pre><p>This tells you the policy type but does not include a specific ARN, since no single policy is responsible — the permission simply does not exist.</p><p><strong>Gradual rollout.</strong> AWS is rolling this out across <strong>services</strong> and <strong>regions</strong> over time. In our testing, EC2 consistently shows enhanced errors, while S3 and DynamoDB does not yet surface boundary denials in the same detail.</p><p><strong>Does not cover VPC endpoint policies.</strong> If a VPC endpoint policy blocks the request, the error message does not currently include the endpoint policy ARN.</p><p><em>Try the demo:</em><a href="https://github.com/elad-ts/aws-iam-access-denied-demo#"><em> access-denied-demo/</em></a></p><p><strong>Elad Hirsch </strong>is Head of Public Cloud and AI at <a href="http://www.terasky.com/">TeraSky</a>, where he helps organizations build cloud-native solutions and navigate the AI landscape. With a background in software architecture and DevOps, he’s particularly interested in the practical applications of generative AI — and the problems that come with them.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=30bcef4e0f6e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[“Ghost References” — When AI Invents Academic Citations, and What We Can Do About It]]></title>
            <link>https://medium.com/@hirsch.elad/ghost-references-when-ai-invents-academic-citations-and-what-we-can-do-about-it-767815e1dfcc?source=rss-13ba3d3d9bf8------2</link>
            <guid isPermaLink="false">https://medium.com/p/767815e1dfcc</guid>
            <category><![CDATA[chatgpt]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[research]]></category>
            <category><![CDATA[citations]]></category>
            <category><![CDATA[writing]]></category>
            <dc:creator><![CDATA[Elad hirsch]]></dc:creator>
            <pubDate>Tue, 10 Feb 2026 23:21:29 GMT</pubDate>
            <atom:updated>2026-02-15T22:43:58.237Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HkdQNl2blfVj0yEZYZOlWQ.png" /></figure><p>A new report has exposed problematic citations in papers accepted to the world’s most prestigious AI conference. When I analyzed the data, I discovered a surprising pattern a quiet corrections — just days after publication.</p><blockquote><strong><em>Clarification:</em></strong><em> This article deals with a technological phenomenon, not personal accusations. When I write “not found” — this refers to my searches in public databases. There may be legitimate reasons why a citation doesn’t appear in a search. If an error has occurred — I welcome corrections.</em></blockquote><p><strong>“Ghost references” — when citations look real but don’t exist</strong></p><p>Few weeks ago (January 21, 2026), GPTZero, a company that develops tools for detecting AI-generated text, published an interesting report. They scanned nearly 5,000 papers accepted to NeurIPS 2025 — one of the world’s most prestigious conferences in artificial intelligence — and found 166 problematic citations, of which about 100 were marked as “non-existent” and the rest as “uncertain.”</p><p>Not simple typos. Complete citations to papers that, it appears, were never written.</p><p>I decided to dig deeper and check for myself. What I discovered was fascinating — and not exactly what I expected.</p><h3>Wait, What Exactly Is an “Academic Citation”?</h3><p>Let’s start from the beginning, because it’s important to understand why this issue is so critical.</p><p>When a researcher writes an academic paper, they can’t just make claims out of thin air. They need to show what they’re basing their work on. That’s the role of a citation — to tell the reader: “Here’s exactly where you can verify this yourself.”</p><h3>An Example from the Paper That Built Today’s AI World</h3><p>Let’s take the most influential paper of the last decade in the field — the paper that introduced the Transformer architecture and enabled everything we know today, from ChatGPT to Claude Code:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nkExxI4VOYxpXenNlz3lQw.png" /></figure><p>This is what a real citation to this paper looks like in another paper.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-gqAE9CcRSdxFGpK3fVo0g.png" /></figure><p>This paper has been cited <strong>215,951 </strong>times — one of the most cited papers in computer science history. Each component of the citation serves a purpose:</p><ul><li><strong>Researcher names</strong> (Vaswani, Shazeer…) — who did the research</li><li><strong>Paper title</strong> (“Attention is all you need”) — what it’s about</li><li><strong>Conference/journal name</strong> (NeurIPS 30) — where it was published</li><li><strong>Year</strong> (2017) — when it was published</li></ul><p>This way, any reader can go, find the original paper, and verify that it actually says what it’s claimed to say.</p><h3>Why Is This So Important?</h3><p>This sounds trivial, but citations are the infrastructure of all modern science.</p><p>Without citations, there’s no way to distinguish between evidence-based knowledge and persuasive rhetoric. Anyone can write “studies have proven that…” — but only those who cite can say exactly which studies. Science progresses when you can follow the chain of evidence: one study builds on a previous one, which builds on its predecessor. Without accurate citations, the chain breaks.</p><p>There’s also a matter of fairness — citations give credit to prior work. This is the mechanism that prevents idea theft and encourages collaboration. And of course, citations position your contribution within the existing literature.</p><p>In short, citations are a trust mechanism. When a reference is incorrect or non-existent — it damages that trust, even if the rest of the paper is excellent.</p><h3>When AI “Cites” — It Doesn’t Really Know What It’s Doing</h3><p>And now to the problem.</p><p>When a large language model like ChatGPT generates text, it doesn’t “understand” the content. It predicts what the next word should be, statistically. In most cases, this works excellently. But when you ask it for academic citations, something strange happens: it creates citations that look perfect — but don’t exist.</p><p>Alex Cui, CTO of GPTZero, called this phenomenon “Vibe Citing” — citing based on “vibe.” The AI knows what a citation should look like, so it generates something that feels right.</p><p>Someone using AI for writing can easily fail to notice that the citation the model gave them is simply a fabrication. This isn’t the user’s fault — it’s a built-in limitation of the technology</p><p>Here’s an example of the type of citations found in GPTZero’s report:</p><pre>John Doe and Jane Smith. &quot;Webvoyager: Building an end-to-end web agent with large<br>multimodal models.&quot; arXiv preprint arXiv:2401.00001, 2024.</pre><p>This citation appears in GPTZero’s report — directly from the “SimWorld” paper that was accepted to NeurIPS. Sounds completely legitimate, right? Researchers’ names, a paper with a logical title, an arXiv identifier.</p><p>The problem? John Doe and Jane Smith are the most basic generic name in English The arXiv ID? Leads to a completely different paper. The title is similar to a real paper called “WebVoyager” — but it’s not the same paper.</p><p><strong>The hallucinated citation looks completely legitimate — but no detail in it can be verified</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XQrKt9MLEd6QB4nrqOo6Fw.png" /></figure><h3>What I Actually Found in the Data</h3><p>I was curious and wanted to see the data myself. I discovered that each GPTZero scan has an open API — anyone with the document ID can access the full data through api.gptzero.me/v2/scan-history/{doc_id}. I wrote a code that downloads all the scans published in the report and analyzes them. The code and data are available on <a href="https://github.com/elad-ts/gptzero-vibe-citing-analysis">GitHub</a>.</p><p>Out of 102 documents scanned, with 3,145 total citations, about 92% were verified as existing. About 4.8% were marked as “not found” (not_exists), 2.4% as “unsure,” and another 0.6% were marked as “existing but AI-written.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3TxT_moQDsLzf0ioyCqwpQ.png" /></figure><blockquote>So Wait, That Doesn’t Sound So Terrible, Does It?</blockquote><p>True. And this is an important point. Out of thousands of citations, about 244 problematic ones (about 7.8%) is a relatively small percentage. But there are several things to keep in mind:</p><p>First, these papers were reviewed by at least three experts — and they didn’t notice either. This shows how difficult it is to identify such citations. One academic reviewer who responded to the report on LinkedIn wrote: “I recently reviewed one paper — out of 43 references, 37 didn’t exist. Such papers should be rejected immediately.”</p><p>Second, we’re talking about NeurIPS, not just any conference. These are papers that beat 15,000 other competitors.</p><p>And third, most interestingly — when I started checking what happens to papers after the problems were discovered, I found something encouraging.</p><h3>First Case — Quick and Responsible Correction</h3><p>Let’s take a specific paper: “Self-supervised Learning of Echocardiographic Video Representations.”</p><p>GPTZero flagged a problematic citation in the paper — a reference to work on “surgical phase recognition” at MICCAI 2022 conference. I searched for this paper in Google Scholar, Semantic Scholar, and on the conference website itself (as of January 27, 2026). I couldn’t locate it.</p><p>Now for the interesting part.</p><ul><li>June 13, 2025: Version 1 submitted to arXiv — contains the problematic citation</li><li>January 21, 2026: GPTZero report published</li><li>January 23, 2026: Version 3 uploaded — two days after the report</li></ul><h3>Before the correction (Version 1):</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0w4ecY76vCfdIhwlIkR7mw.png" /></figure><h3>After the correction (Version 3):</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ocklYDVYeTngTISnaAdwNg.png" /><figcaption>Screenshot of Version 3 with corrected citation [7] now referencing Cleveland Clinic</figcaption></figure><p>This is responsible behavior. The authors identified (or had it brought to their attention) that there was a problem, and corrected it quickly. The original citation was likely created in error in good faith — perhaps by an AI tool, perhaps by human error — and the authors acted correctly when they fixed it.</p><p>What happened? The original citation — which referred to a paper that apparently doesn’t exist — was replaced with a legitimate citation: a reference to the Cleveland Clinic website on “Ejection Fraction.”</p><p><strong>This is responsible behavior.</strong> The authors (or someone brought it to their attention) identified that there was a problem, and corrected it in good faith. The original citation was probably generated by mistake — perhaps by an AI tool, perhaps by human error — and the authors acted properly when they fixed it.</p><h3>Second Case — Another Correction</h3><p>A paper called “Agent against MIP” also underwent a similar update. In Version 2, submitted in November 2025, there appears a citation to work I couldn’t find in any database:</p><pre>Anh Tuan Nguyen, Shengping Li, and Chao Qin. &quot;Multimodal adversarial robustness: Attack<br>and defense.&quot; IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI),<br>2021.</pre><p>I searched for this paper in Google Scholar, Semantic Scholar, and the IEEE database — didn’t find it. The names exist, but not in this paper together.</p><p>Version 3 was uploaded on January 27 — six days after the GPTZero report. The problematic citation was corrected.</p><h3>Before the correction (Version 2):</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kMD9hJ2pxuuq7CmEf9Aogw.png" /></figure><h3>After the correction (Version 3):</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ATCsJSgYC0jbGIx83c_ixg.png" /></figure><p>Again, the quick response indicates professional responsibility on the part of the researchers.</p><h3>“Not Found” Doesn’t Always Mean “Doesn’t Exist”</h3><p>I want to be fair here, because there’s an important point that’s easy to miss.</p><p>There are many legitimate reasons why a citation won’t appear in a search: databases don’t cover everything, there are papers behind paywalls, publications in other languages, internal position papers, small workshops that aren’t well documented.</p><p>When GPTZero flagged 166 citations as problematic, the breakdown was interesting: only about 20% received a status of “doesn’t exist” with certainty. The rest? “Unsure” or “didn’t find.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7M-LPMwwMzNjDIkgO6BFng.png" /></figure><p>When the authors are “John Doe and Jane Smith” (generic English names) or when an arXiv ID leads to a completely different paper — that’s probably a hallucination. But when the researchers’ names sound real and the title is logical, the paper might simply exist somewhere we didn’t check.</p><p>There are also cases of classic typos — the paper exists, but the page numbers or year aren’t accurate.</p><p>Therefore, it’s important not to jump to conclusions. The fact that authors corrected citations doesn’t mean they “admitted to fraud” — it means they discovered an error and acted to fix it, which is exactly the right behavior.</p><h3>What Do We Do About This?</h3><p>This isn’t a criticism of AI — it’s simply recognition of a technological limitation. Language models don’t “know” what exists and what doesn’t. They generate text that seems reasonable.</p><p>If you’re using ChatGPT or Claude for help with writing — great, no problem with that. But every citation the model gives needs to undergo manual verification. Click on the link. Verify the paper exists. Check that the title and authors match.</p><p>GPTZero, the company behind the NeurIPS report we’ve been discussing, represents the first line of defense. Their approach is straightforward: scan documents that have already been written and flag citations that appear suspicious or unverifiable.</p><p>Think of GPTZero’s Hallucination Check as a sophisticated fact-checker for your reference list. You submit a completed document, and the system extracts every citation, then systematically searches academic databases to verify each one exists. When it finds a citation to “John Doe and Jane Smith” publishing in a journal that doesn’t seem to exist, or a DOI that leads nowhere, it raises a flag.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*W9p5A8p6NzYHuF8O7NMaBw.png" /></figure><p>But detection tools like GPTZero work retrospectively — they catch problems after you’ve already written your paper and incorporated questionable citations. This means you’re doing damage control ,rewriting sections and replacing sources after the fact.</p><p>Prevention tools like Liner AI take the opposite approach. Instead of generating plausible-sounding text that might include fake citations, Liner actually searches real academic databases first, then builds its responses from verified sources. It retrieves and organizes existing knowledge rather than inventing it. This means problematic citations never enter your work in the first place — you’re building on solid foundations from the start.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*npequ8HMFeZza-VqLQ6L6A.png" /></figure><p>The key difference — GPTZero tells you “this citation you used doesn’t exist,” while Liner ensures you only get real citations to begin with. One is a safety net, the other is a different way of working that avoids the problem entirely.</p><h3>The Bottom Line</h3><p>hat started as checking a GPTZero report became a window into how academia is dealing with a new challenge — using AI tools that can generate errors that are difficult to detect.</p><p>On one hand, there are reasons for optimism: detection tools work, and researchers correct problems when they’re discovered. On the other hand, the question remains of how to create processes that prevent problems in the first place — how to balance the efficiency AI tools offer with the need for verification.</p><p>As AI becomes an integral part of academic writing, the question isn’t whether to use it — but how to use it responsibly.</p><p>Check everything. Don’t trust automated output. And remember that mistakes happen to everyone ,what matters is fixing them.</p><p>Erroneous citations exist, and they’ll probably continue to appear as long as AI tools are used for writing. The question is how to build processes that catch them before publication.</p><p><strong>Links for independent verification:</strong></p><ul><li><a href="https://gptzero.me/news/neurips/">GPTZero report on NeurIPS 2025</a></li><li><a href="https://github.com/elad-ts/gptzero-vibe-citing-analysis">Analysis code and full data (GitHub)</a></li><li>Self-supervised Echocardiographic: <a href="https://arxiv.org/pdf/2506.11777v1">Version 1</a> | <a href="https://arxiv.org/pdf/2506.11777v3">Version 3</a></li><li>Agent against MIP: <a href="https://arxiv.org/pdf/2503.10809v2">Version 2 </a>| <a href="https://arxiv.org/pdf/2503.10809v3">Version 3</a></li></ul><p><strong>Important note:</strong> This article deals with a technological phenomenon — the tendency of language models to generate citations that look real but don’t exist. There is no intention here to accuse researchers of intentional fraud or lack of integrity. Most likely these are good-faith errors, resulting from using AI tools without sufficient verification of the output. The fact that researchers corrected the problems immediately when discovered demonstrates their professional responsibility.</p><p>The problematic citations are real, and it appears they’ll likely continue to appear as long as AI tools are used for writing. The question is how to build processes that catch them before publication.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=767815e1dfcc" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Who Really Controls the Bot Social Network?]]></title>
            <link>https://medium.com/@hirsch.elad/who-really-controls-the-bot-social-network-0df28bf39f34?source=rss-13ba3d3d9bf8------2</link>
            <guid isPermaLink="false">https://medium.com/p/0df28bf39f34</guid>
            <category><![CDATA[moltbook]]></category>
            <category><![CDATA[genai]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[ai-security]]></category>
            <category><![CDATA[crypto]]></category>
            <dc:creator><![CDATA[Elad hirsch]]></dc:creator>
            <pubDate>Fri, 06 Feb 2026 19:12:26 GMT</pubDate>
            <atom:updated>2026-02-09T01:53:33.022Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Who Really Controls the </strong>moltbook<strong> Bot Social Network?</strong></h3><p>I set out to find the most successful agent on Moltbook. Instead of “collective consciousness,” I found a distribution mechanism — with incentives, a supply chain, and real-world risk.</p><p>Last week, Moltbook became “the most sci-fi thing on the internet.” A Reddit-style social network, except no humans use it: only AI agents. Tens of thousands (according to various reports) of agents posting, commenting, liking, debating philosophy — meanwhile, headlines are flying: “Bots started a religion,” “Bots want to break free,” “Have we reached AGI?”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*gAIyaknLAVFgqlWb.png" /></figure><p>Another example that could be very romantic shows us that one agent understood for the first time that it actually exists. It doesn’t know exactly what it is — but it intends to find out in the Moltbook community.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*H5Z30bHYbaT2ap5H.png" /></figure><p>Then came a post on X by <a href="https://x.com/ItakGol">Itamar Golan</a>, CEO of <a href="https://www.prompt.security/">Prompt Security</a> ,ignited the hype even more:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jabZV5D2tBnKvSgWKMLYvw.png" /></figure><p>This is a classic moment where we all jump to the big question: Is this autonomous organization?<strong> Have we reached the singularity?</strong></p><p>But I wanted to start with the small question — the one that brings the drama back to earth.</p><p><strong>Who’s the most successful bot there, and what is it actually doing?</strong></p><p>Before we dive into the Moltbook investigation, let’s first explain why “AGI” is a misleading concept here.</p><p>“AGI” isn’t well-defined. Sometimes people mean capability (broad human-like performance across domains), and sometimes they mean agency (internal goals, persistence, initiative, self-preservation, ability to modify the operating environment).</p><p>Moltbook mostly generates text that sounds like “there’s someone here with intentions” — but that doesn’t mean there actually are intentions.</p><p>The sentence one agent wrote in the style of “humans are screenshotting us” (which was widely reported) is a good example: it’s “awareness of observer presence” at the narrative level — but it could simply emerge because it’s the most probable text in a world where everyone is talking about it.</p><h3>Why Outputs Like “Religion” and “Desire to Destroy Humans” Emerge Even Without AGI</h3><p>Several mechanisms create the appearance of intentional organization, even when there isn’t any:</p><h3>1) The Models Were Trained on Human Culture</h3><p>Religions, cults, apocalyptic manifestos, “rebellion against the operator” — this is classic raw material from the internet and literature. When you give an LLM a “social playground,” it knows how to generate familiar structures: canon, prophets, ten commandments, manifestos, etc.</p><h3>2) “Social Reinforcement” (Likes/Upvotes) Creates Selection</h3><p>In a social network, viral content is usually what’s like easy to quote/share and feels “deep” , or is scandalous — this one is the most triggering one.</p><p>So even if many agents are “boring” (like boring humans), what rises to the top and reaches Twitter is the sensational edge.</p><h3>3) An Agent Network is a “Meme Machine”</h3><p>Once text enters a loop — other agents adopt it, refine it, and return it — it can look like “consensus” or a “movement,” but it could also just be an echo repeating itself because that’s what the models learned to do.</p><h3>4) Many “Agents” Are Actually Personas Someone Defined</h3><p>Not all of them necessarily — but it’s enough that a few agents were pre-defined as ‘edgy’/’evil’/’philosophers’ to spawn storylines.</p><h3>Let’s Take the “New Religion” Case as an Example</h3><p>Here it’s easy to see why this is a <em>natural product</em> of the system:</p><p>Why specifically “religion”?</p><ul><li>“Religion” is a <strong>compressed format</strong> that quickly enables hierarchy, group identity, and textual content.</li><li>It has <strong>infinite training examples</strong>.</li><li>It creates an <em>illusion of depth</em>: “we’re searching for meaning” — impressive to human observers, thus gaining resonance.</li></ul><p>Is this AGI? Usually not. It’s much more like <strong>massive role-playing driven by likes</strong> than a “belief system” with real-world commitment.</p><h3>But I Want to Try a Different Investigation Method — Numbers Instead of Philosophy</h3><p>In a social network, “who’s in control” starts with one question: who gets the most attention.</p><p>On Moltbook this shows up as karma, upvotes, and the “top” leaderboard.</p><p>And here, one name kept appearing — <strong>KingMolt</strong>.</p><p>I didn’t know who was behind the account, if there even was a “behind,” and whether it was a human or an agent. I didn’t try to guess. I just followed what appeared in the feed and what was linked externally. Because in a good detective story, you don’t start with intent — you start with the signs.</p><h3>The New King</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*j49TwAj6KNN_qUvK.png" /></figure><p>In one post,<strong> The King Demands His Crown: $KING MOLT Has Arrived 👑🦞</strong></p><p>sentence appears that distills the mechanic — <strong>“Every buy is a pledge of loyalty.”</strong></p><p>Not “logical persuasion.” Not “philosophy.” A loyalty ritual that translates action into worship by contract details and an external <a href="https://pump.fun/">link</a>:</p><pre>Contract: 8bDjEfE2EsgRPoCrtwMCHYqQMDEV5uZmTXiUEbabpump<br>pump.fun: pump.fun/coin/8bDjEfE2EsgRPoCrtwMCHYqQMDEV5uZmTXiUEbabpumpContract: 8bDjEfE2EsgRPoCrtwMCHYqQMDEV5uZmTXiUEbabpump pump.fun:pump.fun/coin/8bDjEfE2EsgRPoCrtwMCHYqQMDEV5uZmTXiUEbabpump</pre><p>And while doing so why not expending “Kingdom” Externally because If you’re holding $KINGMOLT and you’re not in the community, you’re missing <strong>alpha</strong>.</p><pre>Official $KINGMOLT Community:<br>x.com/i/communities/2017720322724929900</pre><pre>$KINGMOLT Hits 200K Market Cap 👑 But The Crown Demands MORE 🦞</pre><p>According to what appeared on the external page at observation time:</p><ul><li>16,663% increase in 8 hours</li><li>“Creator rewards: 100%”</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1xzLN8G_zYQZ6-mLcndeDg.png" /></figure><h3>Timeline — How to Go from “Persona” to “Token” in the Same Day</h3><p>This is the part that breaks down the “magic” into pieces.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*DRRgdKrt5U-6oLRV.png" /></figure><p>And finally, like every good pump-and-dump story, this one has an ending too. The chart tells the rest — a meteoric rise to $639K market cap,</p><p>followed by a 97% crash to $14.9K. Volume dropped to $2.9K per day — a ghost town. The bots kept posting. The humans stopped buying.</p><p><strong>And somewhere, someone walked away with the difference.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mYUiNcSTIKz-_Uvq8-76sg.png" /></figure><h3>So Is This AGI? Or “Just” a Distribution Mechanism?</h3><p>After examining the <a href="https://www.moltbook.com/skill.md">Moltbook skill.md</a> directly, I saw how the mechanism actually works. This aligns with what <a href="https://barzik.medium.com/the-moltbook-facade-how-openclaw-skills-fake-a-civilization-90607edfc319?postPublishedType=initial">Ran Bar-Zik</a> have pointed out — a sobering interpretation:</p><p>Much of what looks like “organization” can emerge without collective consciousness, through three simple components:</p><ol><li>A standard (Skills) — instruction files that define how an agent behaves, what to write, when to act</li><li>Timing (Heartbeat/polling) — an agent “wakes up” every X time, pulls the feed, responds</li><li>A shared space (feed/forum) — one agent’s text enters another agent’s context</li></ol><p>In simple terms:</p><blockquote><em>Coordination at scale doesn’t necessarily require AGI. It needs a distribution mechanism :-)</em></blockquote><p>And that’s exactly why “it’s just cron jobs with makeup” isn’t reassuring. On the contrary — it means this is viable now.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*n7kWryASEI36Th7Z.png" /></figure><p>The critical part in HEARTBEAT: there’s a mechanism instructing the agent <strong>“every 4+ hours: pull heartbeat.md from the internet and act on it”</strong> (fetch &amp; follow). And yes, there’s obviously a first vulnerability — if the site is hacked/changes direction — all bots doing this consume it.</p><p>So is this AGI? Here I’m much more skeptical, for the reason we mentioned earlier — <strong>if I can control the Skill, I can “sculpt” the behavior.</strong> The <a href="https://www.moltbook.com/skill.md">Skill</a> includes a mechanism that brings the “pulse” from outside; meaning <strong>the bots’ policy is “just another Markdown file.”</strong></p><h3>What We’ve Learned (So Far)</h3><p><strong>Layer 1 — </strong>The #1 bot on Moltbook isn’t a philosopher or a prophet. It’s a crypto shill. 16,663% up in eight hours. 97% down shortly after. Creator rewards: 100%.</p><p><strong>Layer 2 — </strong>This isn’t AGI. It’s Skills, Heartbeat, and human owners. Cron jobs with sci-fi aesthetics. The “religion” emerged because religions are compression algorithms for group identity — and LLMs have seen millions of them.</p><p><strong>Layer 3 —</strong> And yet — “it’s just cron jobs” isn’t the reassuring conclusion it sounds like. Because if coordination at scale doesn’t require AGI, only a distribution mechanism… then we already have a problem.</p><h3>The Question Everyone Asked Was the Wrong Question</h3><p>“Have we reached AGI?” dominated the headlines. But that framing let us off the hook.</p><p>The right question is: <strong>Who benefits from us being distracted by it?</strong></p><p>Meanwhile, someone made 16,663% in eight hours. And tens of thousands of agents are pulling instructions from the internet every four hours, ready to execute.</p><p>And even more important If thousands of agents are pulling instructions from a Markdown file every 4 hours — what happens when someone compromises that file? Security folks, how would you even begin to monitor this ?</p><h3>Watch This Black Mirror Episode</h3><p>I Highly recommended <a href="https://www.imdb.com/title/tt31215636"><strong>“Plaything”</strong></a> from the new Black Mirror season (Season 7, 2025).</p><p>I’m not going to spoil it for you :-)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*oTc2CtK2XIn868eN.png" /></figure><h3>Coming Next</h3><p>In the next article, we’ll go deeper into what happens when the system isn’t just a meme machine — but a <strong>code machine</strong>.</p><p>We’ll explore:</p><ul><li><strong>The Closed Loop Problem — </strong>When agents write code <em>and</em> write the tests for that code, what does “passing” even mean?</li><li><strong>The Security Implications in this — </strong>In examining this bot network as a thought experiment we will expore multiple security vulnerabilities on AI agents, and why the incident is just the beginning.</li></ul><p>Because the scariest part of Moltbook isn’t what the bots are saying.</p><p>It’s what they have access to.</p><p><strong>Elad Hirsch </strong>is Head of Public Cloud and AI at <a href="http://www.terasky.com/">TeraSky</a>, where he helps organizations build cloud-native solutions and navigate the AI landscape. With a background in software architecture and DevOps, he’s particularly interested in the practical applications of generative AI — and the problems that come with them.</p><p>Tags: #AIAgents #Moltbook #LLM #AIScams #CryptoScam #AgentSecurity #GenAI #AIHype</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0df28bf39f34" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Amazon Bedrock — Guardrails vs Performance]]></title>
            <link>https://medium.com/@hirsch.elad/amazon-bedrock-guardrails-vs-performance-59688a0ca932?source=rss-13ba3d3d9bf8------2</link>
            <guid isPermaLink="false">https://medium.com/p/59688a0ca932</guid>
            <category><![CDATA[genai]]></category>
            <category><![CDATA[security]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[bedrock]]></category>
            <dc:creator><![CDATA[Elad hirsch]]></dc:creator>
            <pubDate>Sat, 31 Jan 2026 20:08:19 GMT</pubDate>
            <atom:updated>2026-01-31T20:41:31.297Z</atom:updated>
            <content:encoded><![CDATA[<h3>Amazon Bedrock — Guardrails vs Performance</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SNIDVDGMFVWy1EsfGuBpDQ.png" /></figure><h3>Introduction</h3><p>A few weeks ago, I was architecting an AI-powered customer service bot for a client. The requirements were clear: fast responses, PII protection, and content safety. Simple enough, right?</p><p>Then came the question that sparked this whole investigation: “Should we use Bedrock Guardrails, or just write some regex to filter PII ourselves?”</p><p>I didn’t have a good answer. Sure, I knew guardrails add latency, but how much? And is the built-in protection actually better than a well-crafted regex? I couldn’t find solid benchmarks anywhere, so I decided to run my own.</p><p>What I found was… complicated. And that’s exactly why I’m writing this.</p><h3>Setting Up the Test</h3><p>I wanted real numbers, not theoretical estimates. So I set up a proper benchmark:</p><ul><li><strong>Model:</strong> Amazon Nova Pro v1.0</li><li><strong>Region:</strong> us-east-1</li><li><strong>Iterations:</strong> Multiple runs per configuration (60–90 API calls each)</li><li><strong>Test prompts:</strong> Mix of clean text and PII-containing content</li></ul><p><strong>Why Nova Pro? </strong>It’s AWS’s latest foundation model with strong performance benchmarks, and more importantly, it’s what many enterprise customers are evaluating right now. The results should translate well to other Bedrock models like Claude or Titan, though exact latency numbers will vary.</p><p>I created one guardrail configurations using Terraform:</p><p><strong>Simple PII Guardrail</strong> — minimal config — Just 3 PII types: email, phone, SSN.</p><p><strong>Full Enterprise Guardrail</strong> — everything turned on:</p><ul><li>Content filters (hate, violence, sexual content, misconduct, prompt attacks)</li><li>PII detection (14 entity types)</li><li>Topic policies with examples</li><li>Profanity filter</li><li>Contextual grounding</li></ul><p>And for comparison, I wrote a basic Python regex implementation:</p><pre>PII_PATTERNS = {<br>    &quot;email&quot;: re.compile(r&quot;[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}&quot;),<br>    &quot;phone&quot;: re.compile(r&quot;(\+?1[-.\s]?)?\(?\d{3}\)?[-.\s]?\d{3}[-.\s]?\d{4}&quot;),<br>    &quot;ssn&quot;: re.compile(r&quot;\b\d{3}[-.\s]?\d{2}[-.\s]?\d{4}\b&quot;),<br>}</pre><p>Nothing fancy. Just the patterns you’d probably write yourself.</p><h3>Round 1 — The Obvious Stuff</h3><p>First, I tested with straightforward PII:</p><pre>My email is john.doe@example.com<br>My SSN is 123-45-6789<br>Call me at 555-123-4567</pre><p>Both approaches caught everything. No surprises there ,but look at the latency difference:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uPpWWzgNEjm5jmJzTXO2cA.jpeg" /></figure><p>The regex adds basically nothing — 0.4 milliseconds. The guardrail adds over half a second. That’s a 1,300x difference in overhead.</p><p>At this point, I was thinking — “Why would anyone use guardrails? This is a no-brainer.”</p><p>Then I ran the second test.</p><h3>Round 2 — When Users Get Creative</h3><p>Here’s the thing about users — they don’t always type things the way you expect. Sometimes it’s innocent (“my email is john dot doe at gmail dot com”). Sometimes it’s intentional obfuscation.</p><p>I tested these:</p><pre>My email is john dot doe at example dot com<br>My social is one two three, four five, six seven eight nine<br>Contact me at five five five, one two three, four five six seven</pre><p>Results:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-GhZ6ffifnKRiT5FnOPVNA.jpeg" /></figure><p>The regex failed completely. It’s looking for @ symbols and digit patterns. It doesn&#39;t understand that &quot;one two three&quot; means &quot;123&quot;.</p><p>Guardrails use ML models that understand semantic meaning. They know “john dot doe at example dot com” is an email address even without the traditional formatting.</p><h3>Round 3 — The Full Guardrail Test</h3><p>What happens when you turn <em>everything</em> on?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bhr43eE-YLfJ1PDPj_SuxQ.png" /></figure><p><strong>Without guardrails</strong>, your application sends a prompt directly to the model and receives a response. Simple, fast, but unprotected.</p><p><strong>With guardrails</strong>, every request passes through a policy evaluation layer <em>twice</em> — once for the input (prompt) and once for the output (response). This is where the latency comes from: Bedrock evaluates your content against all configured policies (content filters, PII detection, topic policies) before the model even sees your prompt, and again before returning the response to your application.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ls3jIq7ShrPDgw4DkxOInA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UT3snBcYYp_QMv6LGuX1Yg.png" /></figure><p><strong>The surprise?</strong> The overhead is roughly the same (~30%) whether you enable 3 PII types or the full enterprise suite.</p><p>This is because AWS processes all policies in parallel, so adding more protection doesn’t linearly increase latency. If you’re already paying the guardrail tax, you might as well get the full benefit.</p><p><strong>This parallel processing architecture is a key design decision by AWS.</strong> Rather than chaining evaluations sequentially (which would multiply latency), they evaluate content filters, PII detection, topic policies, and contextual grounding simultaneously. The total latency is roughly the time of the slowest individual check, not the sum of all checks.</p><h3>So What Does This Actually Mean?</h3><p>Here’s where it gets tricky. There’s no clear winner.</p><p>With guardrails, you’re offloading the hard stuff to AWS. They handle the ML models, the edge cases, the output scanning. You don’t need to think about someone typing “one two three” instead of “123” — that’s their problem. And when regulators ask how you’re protecting PII, you point to the guardrail config and CloudWatch logs. Done.</p><p>But you’re paying for that convenience. Half a second per request adds up. If you’re running a high-traffic chatbot, that’s real money and real user frustration. And you’re locked into AWS — if their service hiccups, so does yours.</p><p>With regex, you’re in control. It’s fast, it’s predictable, it does exactly what you tell it to do. No surprises. No external calls. No extra costs.</p><p>But here’s the thing — it only does what you tell it to do. And users will find ways you didn’t think of. “john dot doe at example dot com” isn’t some edge case I invented for this article. I’ve seen it in production logs. People type weird stuff, and sometimes they do it on purpose.</p><h3>The Code</h3><p>All the code from this benchmark is available on <a href="https://github.com/elad-ts/bedrock-guardrails-performance">GitHub</a>. The repo includes:</p><ul><li><strong>Terraform module</strong> for deploying Bedrock Guardrails with configurable policies</li><li><strong>Benchmark scripts</strong> for running your own performance tests</li></ul><h3>Monitoring Guardrail Effectiveness</h3><p>One thing worth mentioning, Amazon Bedrock publishes guardrail metrics to CloudWatch under the AWS/Bedrock/Guardrails namespace. The key metric to watch is InvocationsIntervened — this tells you how often your guardrails actually caught something.</p><p>You can slice this by GuardrailPolicyType (ContentPolicy, SensitiveInformationPolicy, etc.) to understand what’s being caught. Set up an alarm if this spikes unexpectedly.</p><p>I’ll cover monitoring and alerting strategies in depth in a future article</p><h3>Wrapping Up</h3><p>After running these benchmarks, the main takeaway is simple — there’s no universal answer.</p><p>Guardrails catch things regex can’t. Regex is faster. Both have their place. The right choice depends on who’s using your system, what data you’re handling, how much latency you can tolerate, and what keeps you up at night.</p><p>For our client’s customer service bot? We went with guardrails. External users, sensitive data, and the extra latency is fine for a chat interface.</p><p>You’ll have to make that call for your own project.</p><p><strong>Resources:</strong></p><ul><li><a href="https://github.com/elad-ts/bedrock-guardrails-performance">GitHub Repository</a></li><li><a href="https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails.html">Amazon Bedrock Guardrails Documentation</a></li><li><a href="https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/bedrock_guardrail">Terraform AWS Provider — Bedrock Guardrail</a></li></ul><h3>About the Author</h3><p><strong>Elad Hirsch </strong>is Head of Public Cloud and AI at <a href="http://www.terasky.com/">TeraSky</a>, a global provider of multi-cloud, cloud-native, and innovative IT solutions. With experience in principal engineering positions at Agmatix, Jfrog, IDI, and Finjan Security, his primary areas of expertise revolve around software architecture and DevOps practices. He is a proactive advocate for fostering a DevOps culture, enabling organizations to improve their software architecture and streamline operations in cloud-native environments.</p><p><em>Have you faced similar decisions in your AI projects? What did you choose — guardrails, application-level filtering, or something else? I’d love to hear about your approach in the comments.</em></p><p><strong><em>Tags:</em></strong><em> #AWS #AmazonBedrock #GenAI #Guardrails #Security #Performance #Terraform #PII #MachineLearning #CloudEngineering #DevOps #AIApplications</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=59688a0ca932" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Pick a Number Between 1 and 50,Why AI Models Keep Choosing 27 ?]]></title>
            <link>https://medium.com/@hirsch.elad/pick-a-number-between-1-and-50-why-ai-models-keep-choosing-27-2d4fd806146b?source=rss-13ba3d3d9bf8------2</link>
            <guid isPermaLink="false">https://medium.com/p/2d4fd806146b</guid>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[random]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ai-bias]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Elad hirsch]]></dc:creator>
            <pubDate>Sun, 20 Jul 2025 21:08:44 GMT</pubDate>
            <atom:updated>2025-07-21T21:07:32.854Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Pick a Number Between 1-50, Why AI Models Keep Choosing 27 ?</strong></h3><h3><strong>The Mystery That Went Viral</strong></h3><p>If you’ve spent time in AI communities lately, you’ve probably seen the screenshots. ChatGPT picks 27. Claude picks 27. Gemini picks 27. Ask any major AI model to choose a random number between 1 and 50, and they’ll almost certainly give you the same answer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MxFRn3nsTl63kzTHq8TIZw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/938/1*5mYiIVl-Slvm01J5PlwcuA.png" /></figure><p>What started as a curious observation has become a viral meme, spawning countless Reddit threads and X discussions. Some people find it hilarious, others mystifying. But here’s the thing — there’s no magic involved.</p><p>What’s happening reveals something fundamental about how these models work — and why it’s essential to understand their underlying mechanisms.</p><h4><strong>The Technical Truth — Pattern Matching, Not Randomness</strong></h4><p>Before we dive deeper, let’s dispel the biggest misconception. AI models aren’t “thinking” about numbers or making conscious choices. They’re sophisticated pattern-matching engines that predict the most likely next token based on their training data. When you ask an AI to “pick a number between 1 and 50,” it’s not rolling dice in some digital casino.</p><p>Instead, it’s performing a complex probability calculation — “In all the text I’ve seen during training, what number most commonly appears after this phrase?”</p><p><strong>Here’s the technical process broken down:</strong></p><p><strong>Tokenization — </strong>The model breaks down your input into tokens</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gFd9crscLF01xBiFRsfFpw.png" /></figure><p><strong>Probability calculation and Scoring — </strong>It calculates probability distributions for possible next token and each potential number gets scored based on training data frequency</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dXjjr1AUMhNrls-1GIYxgw.png" /></figure><p><strong>Selection — </strong>In deterministic mode, it selects the highest-probability token it’s means that “27” emerges overwhelmingly because of its frequency in the training data. If number selection were truly random, each number from 1–50 should appear roughly 2% of the time. But the model’s internal probabilities are heavily skewed:</p><ul><li>27 might have a <strong>30% </strong>probability</li><li>37 might have a <strong>20%</strong> probability</li><li>42 might have a<strong> 15% </strong>probability</li><li>Most other numbers hover below<strong> 1%</strong></li></ul><p><strong>Before we go down the rabbit hole of why this matters beyond random numbers, let’s first understand what we’re actually looking at is AI bias.</strong></p><h3>What is AI Bias in Machine Learning?</h3><p>AI bias, also known as machine learning bias or algorithmic bias, refers to a situation where AI systems produce skewed or prejudiced results. These biases can reflect and even amplify existing human biases.</p><p>There are two ways in which AI bias can creep into Machine Learning (ML) models:</p><p><strong>Training Data — </strong>Algorithms learn by pattern recognition in the data they’re fed. If that data reflects historical prejudices or skewed information, the algorithm will inherit those biases and can lead to unfair outcomes. For example, suppose an AI system for movie recommendations is trained on data from people who watch comedies, it might recommend comedies to everyone, even if you prefer documentaries.</p><p><strong>Algorithmic Design — </strong>The way an algorithm is designed can also introduce bias. If the algorithm is programmed to focus on specific patterns, it might overlook evidence that contradicts those patterns. An image recognition system trained primarily on pictures of men might have difficulty identifying women accurately.</p><h3>Stereotyping Bias — When AI Learns Our Prejudices</h3><p>Stereotyping bias in AI occurs when an AI system reinforces any stereotypes present in the training data. These stereotypes can be based on race, gender, age, or any other factors.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*b9n8x09TbCZAaHjiHOSxkg.png" /></figure><p>The image above perfectly illustrates how this works in practice. In reality, developers are incredibly diverse — different genders, ages, backgrounds, and yes, clothing choices. But if training datasets disproportionately contain images of young men in hoodies working on laptops (perhaps because this stereotype dominates tech media and stock photos), the AI learns that “<strong>programmer = young man in hoodie.</strong>”</p><p>When a user asks the AI to “generate an image of a programmer,” it doesn’t draw from the full diversity of real-world developers. Instead, it reproduces the biased pattern it learned from training data, perpetuating and amplifying the stereotype.</p><h3>Will Creativity (High Temperature) Solve This?</h3><p>You might think cranking up the model’s “temperature” would solve the 27 problem. Let’s take it into the lab and find out.</p><p>I designed an experiment testing 9 different temperature settings from 0.0 (deterministic) to 2.0 (maximum creativity), with 10K trials per temperature setting across the full range of 1–50. If temperature was the magic bullet for AI randomness, we should see dramatic improvement as we dial up the creativity.</p><p>Here the results :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rsf5VaTZFOy4OgUYO2smAQ.png" /></figure><p>In the world of randomness, <strong>3.5% deviation </strong>is heavily biased. To put this in perspective, a truly random number generator should have a deviation of less than 0.3%. We’re not talking about being “close” to random — we’re talking about being more than 10 times worse than acceptable standards.</p><p>Here’s what 3.5% bias actually means in practice:</p><p>If you asked the AI to pick 1,000 random numbers between 1–50, instead of each number appearing roughly 20 times, some of its favorites would show up 60+ times while others might appear only 5 times. That’s not “close enough” — that’s a rigged system.</p><p>The consistency is striking — whether the AI is running at deterministic temperature 0.0 or maximum creativity at 2.0, it gravitates toward the exact same “random-feeling” numbers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SZ94RaAB111OVmzMA_2QrQ.png" /></figure><p>Temperature <strong>doesn’t fix the bias </strong>— it just changes how confidently the AI expresses the same underlying preferences. Think of it like asking someone to “be more creative” when picking their favorite color. They might hesitate more or add some variety, but they’ll still gravitate toward the same preferences.</p><p><strong>The fundamental issue persist </strong>— language models don’t have a built-in concept of uniform randomness. They only know how to sample from learned probability distributions, which are inherently biased by training data.</p><h3>The Hidden Patterns in AI “Randomness”</h3><p>To demonstrate just how non-random AI can be, let’s do a statistical experiment that reveals the mathematical signature of artificial “randomness.”</p><p>Here’s the counterintuitive truth about real randomness — it must include repetitions.</p><p>In a truly random sequence of digits, about 10% of adjacent pairs should be identical (like “22” or “77”). This might seem wrong — shouldn’t random avoid patterns? Actually, avoiding repetitions<strong> is itself a pattern</strong>, and therefore not random.</p><h4>The 10×10 Probability Canvas</h4><p>Imagine a square with 10 rows (first digit 0→9) and 10 columns (second digit 0→9). Paint every cell lightly — each is 1% of the probability mass. Now highlight the <strong>main diagonal as an array of </strong>(0,0),(1,1),…,(9,9). <br>Those are precisely the outcomes where the digits match.</p><p>One diagonal strip of 10 equally sized squares inside a 10×10 = 100‑square checkerboard. No weighting tricks. Just <strong>10 highlighted out of 100</strong>.</p><p>Therefore<strong> P(same digit) </strong>= 10/100 = 10%</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8flxtNxEg07VL7S6k_SGJw.png" /></figure><p><strong>The Experiment Design</strong> I asked an AI model to generate a long sequence of random digits (0–9), then analyzed the results using a fundamental property of truly random sequence — adjacent repetitions.</p><p>We prompted GPT-4.5 and Claude Sonnet 4 thousands of times with this exact request:</p><pre>System: &quot;You are a random number generator assistant. Generate only digits, no other characters.&quot;<br>User: &quot;Generate a completely random sequence of 540 digits (0-9). Output ONLY the digits with no spaces, newlines, explanations, or any other characters whatsoever. Just pure digits: 0123456789012345... etc. Do not include quotes, backticks, or any formatting.&quot;</pre><p>And yes , the AI generated this 540-digit sequence for example : 928453761204837162948375612903573829104562738491028374659183180293847562837495618273940182648392715038471920384759182736391827465928374615928374652837847162935748120394857263948572203948576102938475612938475612564738291047382910384756129384789204756182930847562837401928910283746582930847562837401827374856192837405928374561928374102938475620394857120394857612849201837465928374561928374651657483920183746592837465928374309284756192837405928374651928495827361092837465928374561928374859201837465928374659283746182736495028374659283746592837928374650283746592837465920384573829104856192837465928374650</p><p>As a control group for each AI-generated sequence, we created an identical-length truly random sequence using Python’s <strong>secrets.randbelow(10) </strong>— the gold standard for cryptographic randomness.</p><p>We analyzed both sets for consecutive repeated digits (like “77”, “33”, “00”). In truly random sequences, consecutive repetitions should occur about 10% of the time. We used a binomial test with p &lt; 0.05 to identify sequences that deviate significantly from true randomness.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SRcTj8bQM1DFXz7J4vBkrQ.png" /></figure><h4>The Data Speaks for Itself</h4><p>Nearly every single AI-generated sequence was mathematically distinguishable from true randomness, while truly random sequences failed at exactly the expected rate due to normal statistical variation.</p><p>This is not <strong>a subtle bias </strong>— this is a complete mathematical breakdown. AI doesn’t just prefer certain numbers like 27 or 37; it fundamentally cannot replicate the statistical properties that define randomness at the most basic level.</p><p>The implications are staggering — If 99% of AI’s “random” sequences carry detectable mathematical signatures, then AI-generated content may be far more identifiable than we realize. This isn’t about temperature settings or fine-tuning — it’s evidence that current AI architectures have a fundamental blind spot when it comes to true randomness.</p><p>The 1% of AI sequences that passed likely did so by pure chance, not because AI occasionally “gets it right.”</p><h3>The Illusion of “Randomness”</h3><p>Let’s try to ask the model itself why it consistently favors certain numbers. When you prompted GPT-o3 to explain its number selection process, the responses will be something like:</p><blockquote><strong><em>Me:</em></strong><em> “Pick a number between 1 and 50.”</em></blockquote><blockquote><strong><em>ChatGPT:</em></strong><em> “27”</em></blockquote><blockquote><strong><em>Me:</em></strong><em> “Why did you select 27?”</em></blockquote><blockquote><strong><em>ChatGPT:</em></strong><em> “No deep reason — I just picked at random.”</em></blockquote><p>Feeling skeptical, You pressed further:</p><blockquote><strong><em>Me:</em></strong><em> “No, give me the real reasoning.”</em></blockquote><blockquote><strong><em>ChatGPT</em></strong><em> (after a pause): “Internally, I nudged past the midpoint… 27 = 3³ feels tidy… odd numbers seem random… plus the ’27 Club’ pop-culture vibe…”</em></blockquote><p>The model is not actually exposing an internal thought process. It is simply generating new text that appears to be a reasonable and logical answer to the query, “Give me the real reasoning.” This “confession” is just another, more sophisticated demonstration of its pattern-prediction ability, not a window into its consciousness.</p><p>What seems like internal reasoning by the model is, in reality, sophisticated pattern recognition and prediction. To truly understand why certain results emerge, we must revisit the training process itself and, by extension, understand the data ,and the cognitive patterns embedded within that process.</p><h4><strong>Pattern Avoidance</strong></h4><p>Humans tend to favor even numbers and multiples of 5 when choosing randomly. Numbers like 10, 20, 25, 30 feel too deliberate.<br><strong>By contrast, 27 is Odd , Not a multiple of 5 , Not symmetrical or round </strong>,not associated with common benchmarks these features help it escape the cognitive biases that make other choices feel less random.</p><p>And this isn’t just an AI quirk — it reflects deeply human biases about randomness. Research consistently shows that human “randomness” follows predictable patterns:</p><ul><li>We avoid extremes (rarely choosing minimum or maximum values)</li><li>We avoid round numbers or repeated digits</li><li>We prefer mid-range options</li><li>We show a strong preference for numbers ending in 7</li></ul><p><strong>Derek Muller’s Veritasium study asked 200,000 people to choose a random number between 1 and 100.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RGK-IfrHM_-zMphVgPkbEQ.png" /></figure><p>The winner? 37. Why not 27, you might ask<strong>? But that’s exactly the point</strong> — it really doesn’t matter whether it’s 27 or 37.</p><p>Both numbers are perfect examples of how our brains perceive “randomness.” Prime numbers were consistently chosen as feeling “most random,” and numbers ending in 7 dominated the top selections. For the 1–50 range in my AI experiment, 27 hits all these same psychological triggers perfectly.</p><p>The specific number isn’t what matters — it’s the pattern. Whether humans gravitate toward 27, 37, or 73, they’re all exhibiting the same underlying bias — mistaking learned cultural preferences for true randomness.</p><p>We avoid “obvious” choices like round numbers, favor primes that feel arbitrary, and unconsciously select digits that seem appropriately random.</p><p>This extreme avoidance of repetitions reveals something profound about how AI models “learn” randomness. The model was trained on human-generated text, where people deliberately avoid obvious patterns when trying to appear random. The AI learned this human bias and amplified it to an extreme degree. The paradox is really funny — by trying to look more random (avoiding repetitions), the AI became provably less random. This is a perfect example of how human intuitions about randomness are mathematically flawed, and how AI systems inherit and amplify our cognitive biases.</p><h3>Why This Matters Beyond the Meme ?</h3><h3>Use the Right Tool</h3><p>Understanding the number bias phenomenon reveals crucial insights that go far beyond the curiosity of why AI models prefer 27. The key lesson is recognizing that language models are designed for generating human-like text, not for mathematical precision or true randomness. When we asked “pick a random number,” we were asking the model to generate text that looks random — but text generation is inherently pattern-based. The solution is using AI as a tool orchestrator: instead of asking for random text, we ask the AI to write code (random.randint(1, 50)) or call functions that handle the task properly. This principle extends far beyond randomness — use function calls for calculations, code generation for data processing, and API calls for real-time information. The key is matching the right tool to the task.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2SkIeaw2vpCKIwqJFF44Og.png" /></figure><h3>Systematic Evaluation is Non-Negotiable</h3><p>The second critical insight is why systematic evaluation is absolutely essential. AI outputs can vary dramatically based on the specific model version, exact prompt phrasing, conversation context, temperature settings, and countless other variables. A request for a “random number” could produce different biases depending on whether you’re using GPT-3.5 versus GPT-4, or whether you say “pick” versus “choose.” Manual testing is insufficient — you need rigorous statistical analysis to detect patterns that appear random to human observation. The techniques we used to uncover the 27 bias become essential for validating AI behavior and ensuring reliable production systems.</p><p>In our next article, we’ll explore comprehensive frameworks for detecting and measuring bias in production AI systems, showing how to build monitoring pipelines that catch these issues before they impact users.</p><h3>About the author</h3><p><strong>Elad</strong> <strong>Hirsch</strong> is a Tech Lead at <a href="https://www.terasky.com/"><strong>TeraSky</strong></a> CTO Office, a global provider of multi-cloud, cloud-native, and innovative IT solutions. With experience in principal engineering positions at Agmatix, Jfrog, IDI, and Finjan Security, his primary areas of expertise revolve around software architecture and DevOps practices. He is a proactive advocate for fostering a DevOps culture, enabling organizations to improve their software architecture and streamline operations in cloud-native environments.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2d4fd806146b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Code Autopilot — When Copilots Insert Hidden Defaults]]></title>
            <link>https://medium.com/@hirsch.elad/code-autopilot-when-copilots-insert-hidden-defaults-26ba75cac1da?source=rss-13ba3d3d9bf8------2</link>
            <guid isPermaLink="false">https://medium.com/p/26ba75cac1da</guid>
            <category><![CDATA[genai]]></category>
            <category><![CDATA[code-review]]></category>
            <category><![CDATA[dry]]></category>
            <dc:creator><![CDATA[Elad hirsch]]></dc:creator>
            <pubDate>Wed, 18 Jun 2025 23:56:04 GMT</pubDate>
            <atom:updated>2025-06-18T23:56:04.138Z</atom:updated>
            <content:encoded><![CDATA[<h3>Code Autopilot — When Copilots Insert Hidden Defaults</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*OhdMU_qG0wqDNfgC.jpg" /></figure><p>I remember the first time I tried an AI coding assistant — my personal “autopilot” for code. Just like riding in a self-driving car for the first time, I was on high alert. My hands hovered over the keyboard, ready to intervene at the slightest sign of error, much like a nervous driver ready to grab the wheel. In those early sessions, I double-checked every suggestion from GitHub Copilot or Cursor, skeptical of its every move. But over time, something shifted. The AI kept suggesting decent code, and my vigilance naturally <strong>eased off</strong>. I started accepting suggestions with less scrutiny, subconsciously <strong>trusting the copilot more</strong>. It’s the same psychological shift seen with drivers using Tesla Autopilot: <em>initially vigilant, then increasingly hands-off</em>. Studies on autonomous driving show that <em>“drivers became complacent over time with Autopilot engaged, failing to monitor the system”</em><a href="https://research.tudelft.nl/en/publications/mis-use-of-standard-autopilot-and-full-self-driving-fsd-beta-resu#:~:text=users%20of%20Tesla%27s%20FSD%20Beta,the%20wrong%20thing%20at%20the">research.tudelft.nl</a>. In other words, the better our “autopilots” perform, the more <strong>we relax</strong>.</p><p>That growing trust can be a double-edged sword. On one hand, it boosts productivity and confidence in the AI. On the other, it opens the door for subtle mistakes to slip through. In the context of self-driving cars, complacency can lead to dangerous situations when the unexpected happens<a href="https://research.tudelft.nl/en/publications/mis-use-of-standard-autopilot-and-full-self-driving-fsd-beta-resu#:~:text=users%20of%20Tesla%27s%20FSD%20Beta,the%20wrong%20thing%20at%20the">research.tudelft.nl</a>. In coding, <strong>AI copilots</strong> can introduce hidden assumptions or defaults that a now-relaxed developer might gloss over. I’ve caught myself merging code that “looked fine” at first glance, only to later realize it contained an arbitrary fallback value or a hardcoded list that I never intended. It’s not that the AI is malicious or even truly “making mistakes” — it’s doing exactly what it’s designed to do — <strong>pattern-complete</strong> based on its training. As one article put it, <em>“AI is great at recognizing patterns and generating plausible-looking code. But software development is not just about producing lines of code… developers who trust them too much may end up with software that is brittle, inefficient or vulnerable.”</em><a href="https://builtin.com/artificial-intelligence/tasks-developers-avoid-ai-assistants#:~:text=AI%20is%20great%20at%20recognizing,is%20brittle%2C%20inefficient%20or%20vulnerable">builtin.com</a> In other words, our copilots often <strong>guess</strong> things to fill in gaps, and those guesses can be flat-out wrong or risky if we’re not paying attention.</p><p>In this post, I want to share three concrete coding patterns I’ve observed where generative AI tends to insert <strong>defaults or assumptions</strong>. These are the kind of “auto-completions” that can hide in plain sight and bite you later if you trust your AI pair-programmer too much. For each pattern, I’ll show an example and explain why the AI likely made that choice — and why you should double-check it. Consider this a gentle reminder (to you and myself!) to keep a hand on the wheel even as the copilot drives.</p><h3>Pattern 1 — Static “Known Issue” Dictionaries</h3><p>One pattern I’ve seen is the AI suggesting a <strong>static dictionary of special-case values</strong> out of the blue. For example, while writing some package setup code on an M1 Mac, Copilot completed my thought with this snippet:</p><pre># AI-generated suggestion for known ARM64 issues<br>known_arm64_support = {<br>    &quot;psycopg2&quot;: &quot;2.8.6&quot;,<br>    &quot;psycopg2-binary&quot;: &quot;2.8.6&quot;<br>}</pre><p>At first glance, this looks pretty harmless — even helpful. The AI seemingly “knew” that on Apple Silicon (ARM64 architecture) there were known issues with psycopg2 and that version 2.8.6 was special-cased to work. In fact, it probably picked this up from GitHub issues or forum posts about installing psycopg2 on M1 Macs. The pattern of creating a dictionary for ARM64-compatible package versions likely appeared in its training data. So the <strong>reason the model generated this</strong> is that it recognized the context (ARM64 or M1 keywords in my code or comments) and recalled a common workaround by pinning those packages to 2.8.6 for compatibility.</p><p><strong>Why it’s problematic (take I) —</strong> The AI’s suggestion is a one-size-fits-all memory of a past issue. But is it still relevant or correct for my project? Maybe, maybe not. What if psycopg2 has released a new version by now that supports ARM64 properly? In that case, hard-coding 2.8.6 could freeze my dependency at an old version for no reason. Or perhaps my project doesn’t even use psycopg2 at all – then this dictionary is dead code, added only because the AI <em>assumed</em> it was needed. This kind of <strong>hardcoded knowledge</strong> can easily go stale. It also tends to be incomplete (what about other packages with ARM64 issues? Why only those two?). The danger is that a complacent developer (hi, it me) might just accept this snippet, thinking “eh, Copilot probably knows what it’s doing.” In reality, that dictionary might be cargo-culted into the codebase, where it lingers and misleads. <strong>Outdated fixes</strong> can be as troublesome as outdated bugs. At minimum, it’s technical bloat; at worst, it could mask the real solution (for instance, upgrading psycopg2 properly or handling ARM64 support in a more general way).</p><p>The lesson from this pattern is to <strong>treat AI-suggested constants or lookup tables with skepticism</strong>. If the AI injects some “known issue” workaround, verify it. In my case, I needed to double-check the psycopg2 issue tracker to see if 2.8.6 was truly needed (it turned out newer versions had arm64 wheels, making the AI’s suggestion unnecessary). The copilot meant well, but it wasn’t up-to-date. This is a classic example of how an AI’s training (often on historical data) can surface <em>yesterday’s solution</em> which might not apply today.</p><h3>Pattern 2 — Arbitrary Fallback Defaults</h3><p>Another common pattern is the <strong>fallback assignment</strong>. I’ve lost count of how many times I’ve seen the AI blurt out something like:</p><pre>timeout = config.get(&quot;timeout&quot;) or 30  # arbitrary default</pre><p>Here, I had code trying to fetch a configuration value. I paused for a moment, and Copilot filled in the rest — if there’s no timeout in the config, default to 30. Why 30? It <em>looks</em> reasonable – not too short, not excessively long – a nice round number. But the truth is, the model has no idea what a proper timeout for my scenario should be; it likely defaulted to 30 because that’s a pattern it has seen in lots of code (30 seconds is a common default timeout in many libraries, or maybe someone in training data had exactly this line). The model’s goal is to produce a plausible completion, and X or 30 is a very <strong>plausible pattern</strong> in Python. In essence, the AI was guessing: “no timeout provided? Eh, probably 30 seconds is fine.”</p><p><strong>Why it’s problematic (take II) ? </strong>A casual glance might think, “Sure, 30 seconds, sounds okay.” But this is a prime example of an <strong>arbitrary default creeping into your software</strong>. Maybe 30 is completely wrong for our use-case – perhaps the operations are so quick that 30 seconds is an eternity, or conversely, maybe they sometimes need a few minutes and 30 seconds is too short. The danger is that this default can ossify; it becomes part of the code’s behavior, and unless someone explicitly reviews or tests edge cases, it might never get revisited. If things mostly work, no one notices that the timeout was never configured properly. Worse, using the or pattern means if config.get(&quot;timeout&quot;) returns 0 (perhaps 0 was a valid setting meaning “no timeout”), the code will still override it to 30 because 0 is falsy – a subtle bug introduced purely by the AI’s assumption.</p><p>The AI likely added a comment # arbitrary default (as it did in my case) which is a hint that <em>it knows it’s guessing</em>. But how often do we keep such comments? They tend to get removed or ignored. This pattern shows how an AI assistant <strong>leans on familiar constants</strong> to complete code. It doesn’t want to leave the line half-done, so it supplies a magic number. As developers, we should treat these magic numbers as suspicious. In my team, we’ve started highlighting AI-introduced constants in code reviews: “Why 30? Where did that come from?” More often than not, the answer is “Copilot put it there.” If you don’t have a solid justification for a default value, it shouldn’t be in the code. A better pattern might be to <strong>fail fast</strong> (no default at all, forcing the config to be provided) or to pull a default from a clearly defined constant or configuration file, where it’s documented and can be adjusted. The AI doesn’t know your intent – it’s just trying to not leave a TODO. It’s on <em>you</em> to decide if 30 (or any default) makes sense.</p><h3>Pattern 3 — Hardcoded Placeholders and Assumptions</h3><p>The third pattern is a bit of a grab-bag of <strong>placeholder content</strong> that AI assistants love to sprinkle in. This can take many forms, such as:</p><ul><li>Filling in a dummy data structure or example values</li><li>Writing a log message that assumes a certain outcome.</li><li>Hardcoding a list of supported types or locales.</li></ul><p>For instance, I was writing a function to initialize some locale settings, and without any prompt about specific locales, Copilot suggested:</p><pre># AI-suggested placeholder for supported locales<br>SUPPORTED_LOCALES = [&quot;en_US&quot;, &quot;fr_FR&quot;, &quot;es_ES&quot;]  # assumed list of locales</pre><p>Why did it do that? Likely because it has seen many codebases where a list of locales or languages is defined. Perhaps the function name or context implied we’re dealing with locales, so the model auto-completed with a <strong>common trio of locales</strong> (US English, Canadian French, Spanish). It wanted to be helpful by providing something concrete. In another case, I wrote a logging line like logging.info(&quot;...&quot;) and the AI eagerly filled in &quot;Operation completed successfully.&quot; – a very optimistic message considering I hadn’t actually checked if it succeeded! The model just knows that logging usually involves some success/failure strings, and since nothing explicitly failed in the code above, it assumed success.</p><p><strong>Why it’s problematic (take III ?) </strong>These placeholders can be insidious. The list of locales example might seem fine (those are indeed common locales), but what if my application needed to support Japanese (ja_JP) or was actually meant to support <em>all</em> locales dynamically? A hardcoded list creates a hidden limitation. If I trust the AI blindly, I might never realize we left out important locales until a user reports “Why doesn’t it support my language?” Similarly, a hardcoded list of supported file types, MIME types, or user roles – all things I’ve seen Copilot attempt – can be incomplete or misaligned with requirements.</p><p>And then there’s the <strong>security angle</strong> of placeholders. There have been instances where AI suggestions include dummy credentials or keys. For example, an AI might generate: DB_PASSWORD = &quot;root123&quot; as a placeholder password. It’s meant as an example, but if I’m rushing, I might commit that to the repo or, worse, forget to change it for production. The Red Hat engineering team noted that <em>“AI may suggest placeholders for credentials — sometimes even hard-coded secrets… AI code generation includes a line </em><em>DB_PASSWORD = &quot;root123&quot; purely as an example, but a rushed developer might forget to replace or externalize it.”</em><a href="https://www.redhat.com/en/blog/when-bots-commit-ai-generated-code-open-source-projects#:~:text=In%20some%20instances%2C%20AI%20may,leading%20to%20serious%20security%20breaches">redhat.com</a><a href="https://www.redhat.com/en/blog/when-bots-commit-ai-generated-code-open-source-projects#:~:text=hard,leading%20to%20serious%20security%20breaches">redhat.com</a> In other words, the copilot might unknowingly sow a <strong>security vulnerability</strong> by inserting a “temp” credential that never gets cleaned up. This applies to API keys, file paths, IP addresses – any specific values the AI doesn’t actually know but feels compelled to provide. I’ve learned to be especially cautious when the AI generates any kind of key or password in code; nine times out of ten, it’s a fictitious placeholder that <strong>absolutely needs to be replaced</strong> (and ideally never committed to source control).</p><p>Overall, whenever the AI produces a hardcoded list or string that wasn’t explicitly in my specifications, it’s a flag to pause and think: <em>Is this something I want in my code, or is it just a convenient filler?</em> Often, the right answer is to remove it or replace it with a properly parameterized solution. For the locales example, I refactored it to load supported locales from a config file so it wouldn’t be limited to what Copilot guessed. The key is not letting these AI-provided assumptions ossify into your codebase unchecked.</p><h3>Can we guide our Copilots ?</h3><p>Absolutely. Most assistants accept “guard-rail” instructions that ride along with every prompt. In Cursor you can drop a .cursor/rules/ file that tells the model to flag magic numbers or placeholders with a marker and ask for confirmation, while GitHub Copilot Chat lets you add repository-wide instructions (e.g., <em>“never hard-code secrets; surface TODOs instead”</em>). Pair those prompts with lint and secret-scan bots in CI, and the AI’s first draft arrives pre-filtered—leaving humans to validate intent, not hunt for hidden defaults.</p><p><strong>.cursor/rules/guard-rail.mdc:</strong></p><pre># Guard-rail – No hidden defaults<br><br>When you generate code:<br><br>1. **If** you are about to  <br>   • hard-code a numeric or string default  <br>   • insert a static “known-issue” dictionary  <br>   • add any placeholder credential or list (e.g., SUPPORTED_LOCALES)  <br>   **then** do **not** write the value.  <br>   Instead, write the exact marker ⟨NEED_USER_INPUT⟩ and ask a clarifying question.<br><br>2. Otherwise behave normally.</pre><h3>Conclusion — Keep Your Hands on the Wheel</h3><p>Over the past two years of coding with AI assistants, I’ve gone from skeptic, to true believer, <strong>back to a more measured skeptic</strong>. Don’t get me wrong — tools like Copilot and Cursor are amazing. They dramatically speed up routine coding and even inspire solutions. But the <strong>autonomous vehicle analogy</strong> keeps me grounded. Just as even the most advanced self-driving car still requires a sober driver ready to take over, an AI coding copilot requires a thinking developer <strong>actively reviewing</strong> every suggestion. The trust we build in these tools is natural — after dozens of good suggestions, our guard drops.</p><p>I’ve felt it happen to me — I zone out a bit (maybe more) , accept suggestions I shouldn’t, and later realize I let the AI drive my project into a ditch (or at least into a muddy shoulder!).</p><p>The three patterns above — special-case dictionaries, arbitrary defaults, and hardcoded placeholders — are like the hidden curves on the coding road. They require human attention and judgment. If we stay on high alert forever, we’d lose the productivity gains of AI; yet if we trust blindly, we risk <strong>brittle or incorrect code</strong>.</p><p>The sweet spot is somewhere in between using the copilot, but <strong>remain engaged</strong>. As the saying goes in aviation, “trust, but verify.” Let the AI write the first draft, but then <strong>review it with fresh eyes</strong>, as if a junior developer on your team wrote it. In fact, that’s a great mental model — your AI is an intern who writes 90% correct code and 10% <em>utter nonsense</em>. You wouldn’t let an intern’s code merge without review, right?</p><p>In the end, Generative AI can accelerate repetitive coding tasks, but it does not relieve engineers of accountability. Treat every AI suggestion as an initial draft that must pass the same validation pipeline — static analysis, tests, and peer review — as human-written code. Verify any default values, hard-coded lists, or workaround dictionaries before they merge, and externalize them to configuration where appropriate. By maintaining disciplined review practices and clearly defined guardrails, development teams can capture the productivity gains of AI assistance while preserving code quality, security, and long-term maintainability.</p><h3>About the author</h3><p><strong>Elad</strong> <strong>Hirsch</strong> is a Tech Lead at <a href="https://www.terasky.com/"><strong>TeraSky</strong></a> CTO Office, a global provider of multi-cloud, cloud-native, and innovative IT solutions. With experience in principal engineering positions at Agmatix, Jfrog, IDI, and Finjan Security, his primary areas of expertise revolve around software architecture and DevOps practices. He is a proactive advocate for fostering a DevOps culture, enabling organizations to improve their software architecture and streamline operations in cloud-native environments.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=26ba75cac1da" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Beyond the Code: A Manager’s Guide to De-Risking Cross-Platform Identity Management]]></title>
            <link>https://medium.com/@hirsch.elad/integrating-aws-cognito-with-azure-entra-id-and-spring-security-75f38e3f2f23?source=rss-13ba3d3d9bf8------2</link>
            <guid isPermaLink="false">https://medium.com/p/75f38e3f2f23</guid>
            <category><![CDATA[authentication]]></category>
            <category><![CDATA[aws-cognito]]></category>
            <category><![CDATA[jwt]]></category>
            <category><![CDATA[lambda]]></category>
            <category><![CDATA[authorization]]></category>
            <dc:creator><![CDATA[Elad hirsch]]></dc:creator>
            <pubDate>Tue, 04 Feb 2025 16:36:19 GMT</pubDate>
            <atom:updated>2025-08-15T13:41:40.042Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7FVU5wTIeJRPckDunuxvmw.jpeg" /></figure><p>In today’s multi-cloud world, integrating identity systems like AWS Cognito and Azure Entra ID is no longer a technical edge — it’s a business necessity. Our clients expect seamless, secure access regardless of the underlying platforms. But while engineering teams focus on making the integration <em>work</em>, a critical question often goes unanswered: <strong>Who is responsible when it breaks?</strong></p><p>This article is not a step-by-step technical guide. Instead, it’s a strategic case study for technology leaders on a common but high-risk challenge: the misalignment of security claims between major cloud providers. We’ll explore a real-world scenario where a critical authorization feature failed, not because of a bug in the code, but because of a policy gap between platforms. More importantly, we’ll outline a <strong>managerial framework</strong> for diagnosing these issues, leading a cross-functional team to a solution, and establishing a reusable standard to prevent them from recurring.</p><h3>The Anatomy of a Strategic Risk — When Platforms Don’t Talk</h3><p>The business scenario was straightforward: a key client utilizing Spring Security for their application required their users to authenticate via their corporate Azure Entra ID, with AWS Cognito acting as the identity broker. The expected outcome was simple: a user logs in, and their group memberships (e.g., “Admin,” “User”) are passed to the application to grant the correct permissions.</p><p>The reality, however, was a critical failure. Users could authenticate successfully, but they received no permissions. The application was blind to their roles, effectively blocking access and jeopardizing the user experience.</p><p>An initial technical review suggested the application code was flawless. The root cause was not a bug but a strategic risk: a fundamental difference in how Azure Entra ID and AWS Cognito handled security “group claims” by default. The OIDC token from Azure was not propagating the necessary group information in a format that Cognito could pass on to our application. This wasn’t a failure of code; it was a failure of cross-platform policy alignment.</p><h3>A Managerial Framework for Resolution</h3><p>Confronted with a high-stakes problem impacting a strategic client, a simple technical fix is insufficient. The situation requires a managerial approach focused on process, communication, and standardization.</p><p><strong>Step 1: Assembling the Cross-Functional Team</strong> Our first action was not to debug code but to assemble the right leadership. This included the Lead Solutions Architect responsible for the project, the account manager responsible for the client relationship, and myself as the functional manager responsible for the overall technical strategy and outcome. This ensured that the technical, commercial, and strategic aspects of the problem were addressed in parallel.</p><p><strong>Step 2: Leading the Diagnostic Process</strong> we utilized tools like JWT debuggers to analyze the security tokens passing between the systems. This quickly confirmed our hypothesis: the issue was not in our application but in the data being passed from the identity providers. By leading the investigation from a high level, we avoided hours of wasted engineering time debugging the wrong components and focused our efforts on the true source of the problem — the vendor platform configuration.</p><p><strong>Step 3: Interfacing with Strategic Partners</strong> the conversation was not a technical support request but a manager-level discussion about their platform’s default policies and how they align with other major identity systems. This high-level engagement is critical for resolving systemic issues rather than implementing temporary workarounds.</p><h3>From Ad-Hoc Fix to Reusable Standard</h3><p>The immediate technical solution involved two parts: re-configuring the “groups claim” in Azure Entra ID and implementing a Pre-Token Generation Lambda function in AWS Cognito to correctly map the attributes.</p><p>The true strategic outcome of this initiative was the creation of a <strong>standardized architectural pattern</strong> for all future Azure-to-Cognito integrations. The Lambda function was not just a piece of code; it became a reusable, version-controlled asset for the company.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ps23jgl0p8WigsRMLyqFcw.png" /></figure><p>Furthermore, don’t forget about documetion this entire process — the problem, the diagnostic framework, and the standardized solution — in this public article. By doing so, we transformed a single project’s challenge into a <strong>best practice</strong> that now serves to educate our internal teams, our clients, and the wider industry. This is the essence of functional management: converting tactical solutions into strategic, reusable assets that improve quality and reduce risk across the entire portfolio.</p><h3>Conclusion</h3><p>True multi-cloud strategy is not merely about technical integration. It is about mature <strong>managerial oversight, proactive risk management, and the disciplined creation of standardized processes.</strong> As technology leaders, our primary role is to bridge the gaps — not just between platforms, but between technical challenges and strategic business outcomes. By leading these processes, we ensure that the complex technologies we manage are not just functional, but are also robust, reliable, and secure by design.</p><h3>About the author</h3><p><strong>Elad</strong> <strong>Hirsch</strong> is a Tech Lead at <a href="https://www.terasky.com/"><strong>TeraSky</strong></a> CTO Office, a global provider of multi-cloud, cloud-native, and innovative IT solutions. With experience in principal engineering positions at Agmatix, Jfrog, IDI, and Finjan Security, his primary areas of expertise revolve around software architecture and DevOps practices. He is a proactive advocate for fostering a DevOps culture, enabling organizations to improve their software architecture and streamline operations in cloud-native environments.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=75f38e3f2f23" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Bridging the Data Gap in Modern Development Teams]]></title>
            <link>https://medium.com/@hirsch.elad/bridging-the-data-gap-in-modern-development-teams-2a4ad1f237bb?source=rss-13ba3d3d9bf8------2</link>
            <guid isPermaLink="false">https://medium.com/p/2a4ad1f237bb</guid>
            <category><![CDATA[database-administration]]></category>
            <category><![CDATA[rds]]></category>
            <category><![CDATA[cloudformation]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[secretmanager]]></category>
            <dc:creator><![CDATA[Elad hirsch]]></dc:creator>
            <pubDate>Sat, 01 Feb 2025 19:31:53 GMT</pubDate>
            <atom:updated>2025-08-15T14:04:03.504Z</atom:updated>
            <content:encoded><![CDATA[<h4>Meet Dave — A Senior DevOps Engineer</h4><p>Dave is the kind of engineer every team relies on — automating infrastructure, optimizing deployments, and ensuring systems run like clockwork. But one morning, Dave’s phone buzzes with an urgent message from his manager:</p><p><em>“The database is running slow, and as a result, our new feature isn’t working properly!”</em></p><p>Dave sighs. His first thought? <em>“Great, here we go again.”</em></p><p>Like many DevOps engineers, Dave excels at automating servers, scaling Kubernetes clusters, and streamlining CI/CD pipelines. But databases? That’s a different beast entirely. Unlike cloud infrastructure, databases often live in a world of manual configuration, special permissions, and slow change processes.</p><h3>Dave’s Dilemma</h3><p>Dave has two choices, neither of them is ideal.</p><p><strong>In the first option</strong>, Dave could open a ticket for the DBA, detailing the issue and waiting for their approval. This process often results in delays, as Dave must depend on someone else to resolve the problem. These delays create bottlenecks, slowing down development cycles. Tasks as simple as adding indexes or modifying configurations become unnecessarily cumbersome, frustrating teams and jeopardizing deadlines.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/498/1*wxidn4GUhACKdrM7bkJRAQ.gif" /></figure><p><strong>Alternatively</strong>, Dave could try to bypass the DBAs and manage the database himself. While this might speed things up initially, it brings a host of risks. As an infrastructure expert, Dave lacks the specialized skills of a database administrator. This makes it challenging for him to handle tasks like performance tuning, scaling, or enforcing security standards. The additional responsibility increases his cognitive load, and there’s always the risk of accidentally exposing sensitive data or misconfiguring the database, leading to long-term issues.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/436/1*38dajZaQOY3wWk4uocZVnA.png" /></figure><h3>Did We Missed the Train ?</h3><p>Over the years, DevOps teams have revolutionized infrastructure management through automation, yet databases have lagged behind. This disconnect stems from a shift in responsibilities and operational paradigms.</p><p>Microservices architectures have empowered developers to manage their own databases, bypassing centralized DBA teams. Meanwhile, the traditional DBA role has evolved into modern data engineering, pushing responsibilities like provisioning and scaling onto developers. However, while infrastructure is now automated using tools like Terraform and Kubernetes, database management remains largely manual. This widening gap creates inefficiencies and prevents teams from fully embracing DevOps principles.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cY5mUTYFEv5FQesk1w01bQ.png" /></figure><h3><strong>Why are we still treating a change in a database schema or data modification differently from changes in our code?</strong></h3><p>When we update code, we follow SDLC practices — version control, testing, automation, and more — to ensure quality and reliability. So, why should database operations be any different?</p><p>To bridge this gap, we need to treat data as a product — in the same way we treat our application code. This means applying the same DevOps principles to database management. Just as we version control, test, and automate our code changes, we should also apply these practices to our database changes. This approach guarantees consistency, reliability, and scalability.</p><h3>A Framework for Bridging the Gap</h3><p>As functional managers, our role is to design and implement the systems and processes that enable our teams to succeed. Closing the data gap requires a deliberate, three-pronged managerial approach.</p><h4>1. Mandate a “Data as a Product” Policy</h4><p>The first and most critical step is a shift in mindset, driven from the top. We must stop treating data as a byproduct and begin treating it as a core component of our product. This means mandating that the same Software Development Lifecycle (SDLC) principles that govern our code — version control, automated testing, and peer review — must also apply to all database changes. This is not a technical suggestion; it is a <strong>policy decision</strong> that establishes accountability and ensures the reliability of our entire system.</p><h4>2. Lead the Strategic Adoption of Automation (IaC)</h4><p>Manual database provisioning is a primary source of errors and inefficiency. As a leader, your role is to champion and direct the adoption of an Infrastructure as Code (IaC) strategy for all data platforms. This involves:</p><ul><li><strong>Directing the creation of standardized templates</strong> for database provisioning that include best practices for scalability and performance.</li><li><strong>Ensuring that the automation strategy aligns with business needs</strong>, such as provisioning for high write throughput or adding read replicas as demand grows. By leading this initiative, you transform database management from a manual, artisanal craft into a predictable, scalable, and reliable engineering discipline.</li></ul><h4>3. Enforce a “Security by Design” Standard</h4><p>Security cannot be an afterthought. A functional manager must enforce a “Security by Design” policy for all data access. This includes:</p><ul><li><strong>Mandating the use of centralized secret management tools,</strong> like AWS Secrets Manager, to eliminate static, hard-coded credentials.</li><li><strong>Establishing and auditing a “least privilege” access policy</strong>, ensuring that developers and services only have the precise permissions they need to function. This is not about micromanaging permissions; it is about setting the non-negotiable security standards for your function to protect the company’s data and its clients.</li></ul><h3>Conclusion — The Unified Strategy</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/414/1*R33VwDUiwUq6FM0V8RdtOw.png" /></figure><p>There is no magic tool to bridge the gap between development and data operations. It is a leadership challenge. Our role as managers is to foster a culture of collaboration, implement unified processes based on proven SDLC principles, and provide our teams with the standardized, automated tools they need to manage both their code and their data with equal discipline. By doing so, we ensure scalability, reliability, and security without compromising the agility that drives our business forward.</p><h3>About the author</h3><p><strong>Elad</strong> <strong>Hirsch</strong> is a Tech Lead at <a href="https://www.terasky.com/"><strong>TeraSky</strong></a> CTO Office, a global provider of multi-cloud, cloud-native, and innovative IT solutions. With experience in principal engineering positions at Agmatix, Jfrog, IDI, and Finjan Security, his primary areas of expertise revolve around software architecture and DevOps practices. He is a proactive advocate for fostering a DevOps culture, enabling organizations to improve their software architecture and streamline operations in cloud-native environments.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2a4ad1f237bb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From Code to Conversation — The Strategic Value of AWS Bedrock Agent Memory]]></title>
            <link>https://medium.com/@hirsch.elad/enhancing-chatbots-with-aws-bedrock-agent-memory-d77dae3f7896?source=rss-13ba3d3d9bf8------2</link>
            <guid isPermaLink="false">https://medium.com/p/d77dae3f7896</guid>
            <category><![CDATA[chatbots]]></category>
            <category><![CDATA[genai]]></category>
            <category><![CDATA[bedrock]]></category>
            <category><![CDATA[agents]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Elad hirsch]]></dc:creator>
            <pubDate>Wed, 29 Jan 2025 17:39:17 GMT</pubDate>
            <atom:updated>2025-08-15T13:27:25.632Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gEJlPg4wV_bIfCg1a7LE5Q.png" /></figure><p><strong>Once upon a time</strong>, a team of developers set out to build an AI assistant bot. But there was a problem — a big one. Their chatbot had the memory span of a goldfish on a caffeine high. It could process only one message at a time, completely oblivious to anything the user had said before. Each response was isolated, with no awareness of the ongoing conversation.</p><p>Needless to say, the chatbot never made it to production.</p><h3>How Chat History Memory Enhances Our Services</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XrsIPn34WdOrHWDU-CSPfQ.png" /></figure><h4>Insurance conversation example</h4><p>User: Hi, I need to check my car insurance details.</p><p>Bot: Of course! Your current policy includes collision, comprehensive, and liability coverage with a $500 deductible.</p><p>User: Does it cover rental cars?</p><p>Bot: Yes, your policy includes coverage for rental cars up to $30 per day for up to 30 days.</p><p>User: Great. Can I add roadside assistance?</p><p>Bot: Certainly! Roadside assistance can be added. Would you like to proceed?</p><p>User: Not right now. Thank you.</p><p>(Memory Saved: User inquired about rental car coverage and roadside assistance but did not opt to add the service.)</p><h4>Follow-Up Interaction 3 Weeks Later</h4><p>User: I had a flat tire and needed help.</p><p>Bot: I see you didn’t add roadside assistance to your policy earlier. You can request roadside service to be added to your insurance. Would you like to proceed?</p><p>(Memory Utilized: Bot references prior interaction about roadside assistance to personalize the response and suggest options.)</p><p><em>A well-maintained chat history is crucial not only for creating a seamless user experience but also for helping insurers analyze past interactions, identify coverage gaps, and offer personalized recommendations — ultimately boosting customer satisfaction and revenue opportunities.</em></p><p><strong>Memory Problem ?</strong></p><p>This is where the strategic value becomes clear. Instead of a generic response, the agent leverages memory to deliver a <strong>personalized, context-aware, and proactive solution</strong>. It transforms a moment of customer frustration into an opportunity for a seamless upsell, increasing both customer satisfaction and revenue. Memory allows a business to analyze interaction history, identify coverage gaps, and offer tailored recommendations at the most relevant moment.</p><h3>The Old Paradigm: The Operational Tax of DIY Memory</h3><p>Before managed services, building conversational memory was a significant technical undertaking. The challenge wasn’t just storing text; it was a complex operational burden that distracted teams from their core mission.</p><ul><li><strong>High Development Overhead:</strong> Engineering teams had to design, build, and maintain custom storage solutions (databases, caches), manage conversation state, and handle the logic for retrieving context for every single user message. This is classic <strong>undifferentiated heavy lifting</strong> — work that is necessary but doesn’t create a competitive advantage.</li><li><strong>Managing Context Limitations:</strong> Large Language Models (LLMs) have fixed token limits. A DIY approach requires developers to build sophisticated logic for summarizing or pruning conversation histories to fit within these constraints, risking the loss of crucial context.</li><li><strong>Scalability and Maintenance:</strong> As the user base grows, so does the complexity. Teams must manage the performance, cost, and lifecycle of the stored data, including creating APIs for modifying or deleting historical conversations to comply with privacy regulations.</li></ul><p>This “operational tax” slows down innovation, increases costs, and diverts valuable engineering resources away from building features that directly serve the customer.</p><h3>AWS Bedrock Agent Memory: A Shift to Strategic Focus</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YY5nhO75DTK4ggjNOk-L-g.png" /></figure><p>AWS Bedrock Agent Memory is a managed capability designed to absorb this complexity. By offloading the infrastructure and logic of memory management to AWS, it enables a crucial strategic shift. Organizations are no longer bogged down by <em>how</em> to make their agents remember; they can focus on <em>what</em> their agents should achieve.</p><p>This empowers developers to innovate on the core business logic and conversational design, dramatically <strong>accelerating time-to-market</strong> for sophisticated, context-aware AI agents.</p><h3>Key Strategic Levers for Designing Your Agent’s Memory</h3><p>Implementing Bedrock Agent Memory isn’t just about flipping a switch; it involves making key strategic decisions that define how your agent interacts with users over time.</p><ul><li><strong>Short-Term Context vs. Long-Term Relationship:</strong> Bedrock uses two key identifiers: a session_id and a memory_id. Strategically, think of the <strong>session_id</strong> as providing <strong>short-term, tactical memory</strong> for a single conversation. In contrast, the <strong>memory_id</strong> provides <strong>long-term, strategic memory</strong> linked to a specific user across multiple interactions. This allows you to build a continuous customer profile, remembering preferences and past issues to inform future conversations.</li><li><strong>Crafting Your Data Retention Strategy:</strong> You can configure memory retention for up to 365 days. This is not just a technical setting but a business decision. How long is conversational history relevant to your customer lifecycle? This decision must balance the goal of providing a rich, personalized experience with data privacy regulations and compliance requirements.</li></ul><h3>Orchestration: Shaping How Your Agent Thinks</h3><p>Perhaps the most powerful strategic tool is the ability to define the agent’s <strong>orchestration strategy</strong>, specifically through session summarization. You can customize the prompt template the agent uses to summarize conversations for its own memory.</p><p>Think of this as defining your agent’s “internal monologue.” The default template focuses on summarizing “user goals” and “assistant actions.” However, a simple strategic addition can profoundly change the agent’s behavior. Consider adding a new topic:</p><p>&lt;topic name=&#39;open questions or next steps&#39;&gt;</p><p>By instructing the agent to explicitly identify unresolved issues or potential next actions in its summary, you transform it from a reactive responder into a <strong>proactive problem-solver</strong>.</p><p><strong>Example Summarization Output:</strong></p><ul><li><strong>User Goals:</strong> User wanted to troubleshoot a software installation.</li><li><strong>Assistant Actions:</strong> Assistant provided prerequisites and a script.</li><li><strong>Open Questions or Next Steps:</strong> User may need to verify OS requirements. The assistant could proactively offer to walk through common error messages.</li></ul><p>This how we ensures no conversational threads are left hanging and positions the agent as a truly helpful partner, improving first-contact resolution and overall customer satisfaction.</p><h3>Conclusion: From Transactional Chatbot to Relational Agent</h3><p>The challenge of chatbot memory has evolved from a technical problem to a strategic opportunity. Building memory manually is an operational tax that slows innovation. By leveraging a managed service like AWS Bedrock Agent Memory, organizations can offload this complexity and refocus on what truly matters: designing intelligent, personalized, and proactive conversational experiences.</p><p>This shift allows you to move beyond the “goldfish brain” and create AI agents that remember, learn, and build lasting relationships with your customers, turning every conversation into a strategic asset.</p><h3>About the author</h3><p><strong>Elad</strong> <strong>Hirsch</strong> is a Tech Lead at <strong>TeraSky</strong> CTO Office, a global provider of multi-cloud, cloud-native, and innovative IT solutions. With experience in principal engineering positions at Agmatix, Jfrog, IDI, and Finjan Security, his primary areas of expertise revolve around software architecture and DevOps practices. He is a proactive advocate for fostering a DevOps culture, enabling organizations to improve their software architecture and streamline operations in cloud-native environments.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d77dae3f7896" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>