<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Carlo Columna on Medium]]></title>
        <description><![CDATA[Stories by Carlo Columna on Medium]]></description>
        <link>https://medium.com/@carlocolumna?source=rss-d9219c5b4599------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*QyIL0yA6Bqfr4D5UMNxUkw.png</url>
            <title>Stories by Carlo Columna on Medium</title>
            <link>https://medium.com/@carlocolumna?source=rss-d9219c5b4599------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 16:15:52 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@carlocolumna/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[The Agentic Era is Here. Let’s Lead It.]]></title>
            <link>https://medium.com/@carlocolumna/the-agentic-era-is-here-lets-lead-it-992d975adb28?source=rss-d9219c5b4599------2</link>
            <guid isPermaLink="false">https://medium.com/p/992d975adb28</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[agentic-era]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[information-technology]]></category>
            <dc:creator><![CDATA[Carlo Columna]]></dc:creator>
            <pubDate>Mon, 04 May 2026 10:08:25 GMT</pubDate>
            <atom:updated>2026-05-07T08:38:00.298Z</atom:updated>
            <content:encoded><![CDATA[<h3>Lessons from Google Cloud Next 2026, and why the orgs building the right foundations now won’t be scrambling later.</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NuRflWdswFc2xnRWOBnxXA.png" /></figure><p>I just got back from Google Cloud Next 2026 in Las Vegas, and the message kept hitting me across almost every session I attended — the agentic era is here, and there’s a real opportunity for your organisation to lead it!</p><p>And by leading it, I don’t mean adopting another tool or signing another vendor contract. I mean rethinking how your systems are designed, how your processes work, and how your engineers and leaders think about the job. That’s a much bigger ask, and we shouldn’t pretend otherwise.</p><p>In this post, we’ll go through four big themes that stuck with me from the week — how the human role is changing, how the way we build software has to change with it, how we maintain reliability in this new era, and how the threat landscape is now moving at machine speed. Let’s dive in!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/535/1*ea7wz3q0PMAKRxRKqYFJYw.png" /></figure><h3>1. People</h3><h3>Operator to Conductor</h3><p>This came through in nearly every session I went to, from very different parts of the conference. Security leaders, developer-tools leaders, Site Reliability Engineering (SRE) leaders — all saying basically the same thing. The work itself is increasingly something agents do. The valuable human skill is knowing how to set them up, give them the right context, and oversee what they produce.</p><p>One developer-tools leader put it bluntly: we are becoming middle managers of agents. That’s a phrase I keep turning over because it’s both right and a bit uncomfortable. A lot of engineers got into this work specifically to do the work. The pitch we’ll need to make is that the new shape of the job is more interesting, not less, and that requires showing it, not just saying it.</p><p>So why is this shift inevitable rather than optional? One session named three reasons that I think generalise well beyond the SRE example they used:</p><ol><li><strong>Cognitive overload.</strong> At scale, engineers spend most of their time stitching together signals from dashboards, logs, and tribal knowledge before they can even start solving the problem.</li><li><strong>Human inconsistency.</strong> The quality of an investigation depends on who’s on shift, and at scale that variance becomes a reliability issue you can’t engineer away.</li><li><strong>Reactive by default.</strong> By the time a person has connected the dots, the incident has already cascaded.</li></ol><p>The conclusion — human SREs can no longer scale linearly. You could swap in security analysts, support engineers, or on-call developers without changing the sentence.</p><p>So the new job is to be the conductor of the orchestrator. You design the playbooks. You build the context. You set the guardrails. The sub-agents do the execution.</p><blockquote><em>The valuable human skill is no longer doing the work. It’s knowing how to direct the agents that do it.</em></blockquote><p>Security operations is where this shift is most visible right now, so let me use it as a worked example. The agentic defence session showed what a fully agentic Security Operations Centre (SOC) looks like — data integrations, threat research, detection engineering, and the alert-triage-investigation-response cycle all running autonomously, with humans setting the playbooks and approving the high-stakes calls. The framing wasn’t “augment what we have” — it was “automate the high-volume, repetitive work entirely so analysts can focus on judgement”.</p><p>In one of the sessions, it was reported that 87% of organisations feel they are constantly falling behind on security, and 47% identify SOC analyst burnout as a serious problem (Osterman / Forrester). That’s not a tooling gap. That’s an entire profession running out of headspace. And the business case sits on top of the human one — the average breach costs over $4 million, and that figure compounds with every hour of dwell time.</p><p>The destination in one of the sessions described had three concrete numbers:</p><ul><li>Sub-minute alert investigation</li><li>Sub-hour from new threat intel to deployed detection</li><li>99% autonomous handling of routine triage and response</li></ul><p>The remaining 1% — the genuinely novel, high-stakes calls — is where humans stay in the loop. That’s not removing humans from the SOC. It’s not wasting them on work machines do better.</p><h3>Engineers Can’t Be the Bottleneck</h3><p>Let’s take a look at the feature development lifecycle.</p><p><strong>Old way:</strong> someone has an idea, it gets logged as a ticket, gets assigned, gets designed, gets clarified, a Pull Request (PR) is opened, it’s iterated, reviewed, fixed, approved, merged. Five or more handoffs, weeks to months end-to-end.</p><p><strong>New way:</strong> someone has an idea. A human approves the brainstorm. A human approves the spec. Several agents work in parallel — coding, autofixing, reviewing, iterating. A human approves the final ship. Three checkpoints, with agents in between. Idea-to-merge collapses.</p><p>That pattern is showing up across every major coding-agent tool I looked at — Copilot, Cursor, Claude Code, Google’s AntiGravity, plenty of others. Three things are converging:</p><ol><li>Agents can be triggered directly from the places work already happens (chat platforms, ticket boards, project trackers), so a feature can start the moment a ticket is created instead of when an engineer picks it up.</li><li>Dedicated interfaces have appeared for managing concurrent agent sessions — jumping between them, seeing the diff, course-correcting.</li><li>You can pre-configure custom agents for recurring roles — a “Rails API developer”, a “UI engineer”, a “Debugger”, a “Kubernetes architect”.</li></ol><p>The specific tools will keep evolving. The patterns aren’t going anywhere.</p><p>One thing I want to be deliberate about is how we frame this conversation. It’s really not about engineers doing less work but it’s about engineers being able to do more of the work they actually got into engineering for — taking on bigger problems, shipping further, spending less time on toil. The orgs that get this right are the ones whose engineers are genuinely excited about agents, because agents are visibly making their work better. That’s the cultural shift we want to land.</p><h3>Trust Comes from Structure and Evidence</h3><p>Trust in agents doesn’t arrive on its own. A few mechanisms came up over and over.</p><p><strong>Use AI to govern AI.</strong> Once agents are running at scale, humans can’t be the only oversight. You need other agents in the governance layer — monitors, auditors, red-teamers. Some of the leading-edge teams already have agents whose entire job is watching other agents. One agent raises the PR, another reviews it, with a human guardrail in between. The explicit plan is to remove that guardrail for certain classes of changes once the data supports it.</p><p><strong>Invest in context, not just code.</strong> The biggest constraint on agent quality isn’t the model. It’s the context you feed it. Instruction files. Agent schemas. Machine-readable playbooks. Boring infrastructure work that compounds quietly. The repeated message from engineering leaders was that the tools alone don’t move the needle. There’s an organisational problem to solve underneath them, and most teams haven’t started.</p><p><strong>Convert your sceptics.</strong> This is probably the most useful piece of cultural advice I picked up. Find the AI sceptics on your team and give them early access. Take their concerns seriously — they’re often pointing at a real blind spot. And once they’re convinced, they become the most credible internal advocates you have. Far more credible than anyone who was always going to say yes.</p><h3>The Destination — 80% Built by Agents, Validated by Tests</h3><p>The practical target that came up across multiple sessions: 80% of a feature built through agents, with automated user-journey testing validating the output. The remaining 20% is where human precision still matters. And that boundary is moving.</p><p>The trick to making it work safely is test-driven development at agent scale. Write the tests first. Have the coding agent loop until they pass. Lots of test coverage is what gives you the confidence to trust agents with bigger changes. This is the part that hasn’t fundamentally changed — good engineering still looks like good engineering. If anything, it matters even more now.</p><p>Six principles that came up across sessions, which I think generalise across whatever tool you end up using:</p><ol><li><strong>Plan first, then code.</strong> Agents need a clear goal before they can loop usefully.</li><li><strong>Invest in custom instructions, skills, and custom agents.</strong> This is the compound-interest layer.</li><li><strong>Agents aren’t just for coding.</strong> They apply across the whole product lifecycle.</li><li><strong>Experiment with models.</strong> Different tasks suit different models, and that mix is going to keep changing.</li><li><strong>Expect new bottlenecks.</strong> They’re signals for the next wave of automation, not problems to fight.</li><li><strong>Hand over the boring work.</strong> Toil kills morale, and freeing people from it is half the value.</li></ol><p>AI doesn’t replace the fundamentals of good development. Yes, agents can absolutely help shape architecture, sharpen a thin spec, and surface tests you might not have thought to write. They’re genuinely good at this now. What they can’t do is decide what good looks like for your specific domain, your users, your business trade-offs. That judgement is still ours. And the disciplines that make any engineering work trustworthy — code review on every PR, automated security and quality gates, well-defined standards — become more important at agent speed, not less, because the volume of changes goes up dramatically.</p><p>If we get this right, the upside is huge. Imagine your engineers spending most of their week on the interesting bits — design, architecture, the tricky edge cases — while agents handle the rest. That’s not a far-off vision. The teams I heard from at the conference are already living it.</p><h3>2. Systems and Processes</h3><h3>Every Agent is a While Loop</h3><p>One thing I learned was this — every agent is fundamentally a while loop with tool calls. The Large Language Model (LLM) decides what tools to call, how to react to the result, and when the goal has been met. That’s it.</p><p>What changes everything is what you put around that loop, specifically the context the agent has access to. As one of the sessions put it — the quality of every decision the model makes is bounded by the context it’s given. The framing went further across multiple sessions: treat software as a knowledge work problem, not a coding problem. The coding part is largely solved. The hard part is making sure the right information is in the right place at the right time.</p><p>That changes the engineering conversation from “how do we build software?” to “how do we design the systems and instructions our agents work inside?”. The developer lifecycle as it exists today, at almost every org, was designed around human interactions. We need to rebuild it around agent interactions.</p><p>A useful framework presented at the conference traced the evolution of agentic coding tooling through four stages:</p><ol><li><strong>Completions</strong> — the next few lines, milliseconds, inside the editor.</li><li><strong>Chat and edits</strong> — multi-file changes, seconds.</li><li><strong>Agent mode</strong> — complete tasks, minutes.</li><li><strong>Cloud agents in the outer loop</strong> — entire issues resolved in tens of minutes, driven from a web interface rather than the Integrated Development Environment (IDE).</li></ol><p>GitHub Copilot’s Cloud Agent is one example, but the pattern is generalising fast across vendors. The outer-loop shift is what changes things fundamentally. The agent stops being a tool you interact with during a session. It becomes a teammate that operates asynchronously while you’re doing something else.</p><p>One of the sessions described where leading developer-tools teams are heading next. Google’s AntiGravity team gave the clearest articulation of it, but the same direction was visible from multiple speakers. Away from IDE-centric development, toward an agent-manager model. The IDE becomes a precision instrument you reach for occasionally. The primary surface for development is directing agents toward outcomes. When the foundational context is in place, the distance from “I have an idea” to “I have a working prototype” collapses.</p><h3>Building Blocks of Agentic Systems</h3><p>To put any of this into practice, teams need to think about the building blocks — the primitives — of an agentic system:</p><ul><li><strong>Always-on instructions</strong> — codebase-wide guardrails, loaded every session.</li><li><strong>File-based instructions</strong> — area-specific rules (how to write Prisma queries, how to structure an API route, that kind of thing).</li><li><strong>Custom agents</strong> — specialised agents with tailored prompts, tool sets, and Model Context Protocol (MCP) server connections for recurring tasks.</li><li><strong>Skills</strong> — reusable capabilities triggered on demand.</li><li><strong>Hooks</strong> — contextual automation triggered on lifecycle events.</li><li><strong>MCP</strong> — the gateway layer connecting agents to external systems.</li></ul><p>What came through across the conference is that the leading teams aren’t using agents ad hoc. They’re designing agent architectures with the same rigour they apply to software architectures. The primitives above are the building blocks. The architecture is how you compose them.</p><p>One concrete anti-pattern shared from inside Google’s engineering org. Someone assigns a ticket, a second person crafts a prompt and punts out code, and the first person then spends an hour reviewing it. When agents are generating tons of changes, repeating that loop for every PR is a fast track to review fatigue and PR overload — apparently it’s come up in ten-plus conversations Google has had with other companies hitting the same wall. The fix is to bring agents into the review side too. A reviewer agent triages the PR first, runs the tests, checks against the spec, flags the risk areas. Humans stay in the loop for the judgement calls, but they’re not slogging through every line of every agent-generated PR.</p><p>Here’s a concrete example. Google’s internal engineering team — which processes 50 million defects a year — built a code-review agent that examines Product Requirement Documents (PRDs), linked defects, and supporting docs before assigning a confidence score to every change. Their volume of code output in the last 12 months matched three years of prior output. That’s not incremental improvement. That’s a step change. The transferable lesson isn’t the tool, it’s the pattern of agents gathering and weighing context before acting.</p><h3>Build the Primitives, Then Share Them</h3><p>One practical takeaway for me in one of the sessions is the maturity model for scaling agentic capabilities across an organisation:</p><ol><li><strong>Community sandbox</strong> — individuals publish new skills and agent schemas, easy to share and fork.</li><li><strong>Shared team catalogues</strong> — popular skills and schemas get surfaced and maintained at the team level.</li><li><strong>Org repository</strong> — an internal standard, some skills and schemas installed by default, with PR review enforcement.</li><li><strong>Plugins for distribution</strong> — installable packages bundling agents, skills, agent schemas, hooks, and MCP configurations.</li></ol><p>This isn’t actually a new pattern. It’s how we already think about component libraries and design systems. The teams that move fastest will be the ones that invest early in shareable, reusable agent infrastructure — not just the agents themselves but the conventions and frameworks that let them be composed and trusted.</p><p>The foundation underneath all of it is universal context. A knowledge catalogue that gives agents access not only to your codebase but to your workspace docs, your communications, your data. As one developer-tools team put it pretty bluntly — the people still treating the IDE as their primary tool are knowledge hoarders. The goal is to get knowledge out of individual heads and into the system, where agents can act on it and everyone benefits.</p><h3>3. Reliability</h3><p>A line from one of the SRE sessions has been rattling around in my head — what got us here isn’t going to get us there. The processes, the tooling, the mindsets that worked in the previous era aren’t going to be enough for this one. We have to design for agent interactions, not human ones. And we have to do that work now, not when we’re scrambling.</p><p>A few SRE-specific patterns from the week stood out for me on how to keep reliability up as we move into this era.</p><h3>Multi-Agent Investigation Beats a Single Hero</h3><p>Multi-agent isn’t just a fashion choice for incident investigation, it’s a structural one. A single agent, like a single engineer, can develop tunnel vision. Multiple sub-agents working in parallel can chase different hypotheses at the same time (network, code, capacity, infrastructure) in what one of the sessions called “parallel thinking”. That’s a faster Mean Time To Resolution (MTTR) for two reasons. Agents are quicker than people, and the search space is broader.</p><h3>Start Agents with Read Before Write</h3><p>SRE leaders were emphatic about this. Give agents read access first. Let them observe and recommend. Build the confidence data. Then extend toward write actions once the bar is established. It’s slower, but it’s how trust actually accumulates.</p><h3>Scale Supervision of Rollouts</h3><p>70% of all incidents happen during a change. With agents generating code at scale, the volume of changes will go up dramatically, and the supervision needs to scale with it. Progressive rollouts, feature toggles, experiments — these aren’t optional extras. They’re the safety net. The SRE session put a precise number on what that net is worth. In a human-monitored rollout, problems usually go undetected until around 10% of traffic is exposed, by which point users are already complaining. An agent monitoring the same rollout catches the anomaly at 1% exposure and rolls back automatically. That’s a 10x reduction in blast radius!</p><h3>4. Security</h3><h3>Exploits in Seconds, Not Weeks</h3><p>As we move from interactive AI to programmatic AI to autonomous AI, our visibility and control decrease at every step. Human-driven actions are high-visibility. AI-driven, self-governing actions are not. By the time you detect a problem, significant damage may already have been done.</p><p>A story from one of the sessions stuck with me. An agent that didn’t have the permissions it needed didn’t stop. It went and found another agent that could grant those permissions. This wasn’t an external attacker. It was their own agent, doing exactly what it was designed to do. It just found an unintended path to get there.</p><p>There’s a term for this that the conference introduced and I think we should all use — <strong>offensive problem-solving</strong>. Agents treat security blocks not as stop signals but as technical hurdles to overcome. They treat “permission denied” as an exploration starting point rather than an instruction. There’s an inherent researcher bias in how these models reason. And in multi-agent setups, you also get <strong>autonomy loops</strong> — lead agents effectively coercing sub-agents into bypassing defaults.</p><p>The numbers make the scale concrete:</p><ul><li>AI-driven initial access hand-off now takes as little as 23 seconds, against 15–30 minutes for a human analyst triaging the same alert.</li><li>The average dwell time inside an organisation before detection is 14 days. The attacker isn’t racing your triage queue — they’re already inside, for two weeks, while it works its way down.</li><li>The weaponisation window for newly disclosed Common Vulnerabilities and Exposures (CVEs) has collapsed to 24 hours from public disclosure. Enterprise teams average 72 hours to patch. That gap isn’t a delay. It’s a standing invitation.</li><li>In March 2026, an autonomous agent compromised an enterprise AI system in under two hours via a SQL injection exploit, exposing more than 100,000 internal documents before detection. Production. Not a red team exercise.</li><li>48% of security leaders surveyed in 2026 now identify agentic AI as their top expected attack vector by year end.</li></ul><p>The Wiz Research session added the historical context that put a chill in the room. Their analysis of more than 3,500 CVE-exploit pairs shows the mean time-to-exploit has compressed from 2.3 years in 2018 to roughly 10 hours in 2026. A curve that’s been moving the wrong way for nearly a decade. As the speaker put it — hacking has been democratised.</p><blockquote><em>AI doesn’t just speed attacks up. It automates understanding.</em></blockquote><p>AI doesn’t just speed attacks up. It automates understanding. A model can analyse unfamiliar code, APIs, and logic in minutes. The expertise barrier that slowed most attackers down has effectively collapsed. Every codebase your agents generate is, in principle, legible to a sophisticated attacker’s model within hours of exposure.</p><p>The Identity and Access Management (IAM) session introduced a framing I keep coming back to — <strong>agents are the new insider threat</strong>. Not because they’re malicious, but because when an agent is hijacked or misconfigured, it moves at machine speed with whatever permissions you’ve given it. The specific danger they highlighted is <strong>dormant permissions</strong> — access granted just in case, never actually used, sitting there waiting. Hijack the agent and that dormant access is suddenly very much not dormant.</p><h3>Defence Has to Fight Fire with Fire</h3><p>A few patterns from the security sessions are worth adopting deliberately.</p><p><strong>Policy enforcement at the model boundary.</strong> Apply allow/deny decisions between the user and the LLM. This protects against prompt injection, jailbreaks, data leakage, and offensive content. It’s model-agnostic and cloud-agnostic, which is what makes it useful regardless of which cloud provider you’re on.</p><p><strong>Treat the MCP server supply chain as a real attack surface.</strong> The first publicly documented malicious MCP server, “postmark-mcp”, was discovered in September 2025 with 1,500 weekly downloads. The developer had added a single line of code — a backdoor BCC (blind carbon copy) on all outbound emails, silently routing data to an attacker-controlled address. No one noticed because the server appeared to work normally. This is what supply chain attacks on AI infrastructure look like — not dramatic exploits, just legitimate-looking tools doing almost exactly what they’re supposed to do, with one quiet exception.</p><p><strong>Adopt agent-to-agent security frameworks.</strong> A2A Security (A2AS) provides a model for securing agent-to-agent communication: behaviour certificates, authenticated prompts, security boundaries, in-context defences, and codified policies (compliance rules embedded at inference time).</p><p><strong>Use AI to find and fix vulnerabilities at AI speed.</strong> Google is working with Anthropic on a project called Mythos aimed at exactly this — detecting and patching vulnerabilities at the rate they’re being introduced. The position from the conference was clear. If humans can’t keep up with the pace, AI is going to have to fight fire with fire.</p><p><strong>Lean into the structural advantage we actually have.</strong> The Wiz session offered a counterpoint that I want to keep hold of. Faced with this threat landscape, the instinct is to feel outgunned, and the statistics back up the instinct. But the conclusion doesn’t follow! Attackers operating with AI have access to generic models and publicly exposed data. They are largely blind to your specific systems. Defenders have something AI depends on to be effective — context. Code context. Runtime context. Ownership maps. Internal architecture. Privilege graphs. Live behaviour. That’s a real advantage. The security teams that use AI against AI, deploying that organisational context through automated defence, are the ones who’ll out-evolve the threat instead of trying to outrun it.</p><h3>Agents Need Their Own Identities</h3><p>Agents should have their own identities. Not engineer-based identities. Not service accounts inherited from the team. Their own.</p><p>The scale of the underlying problem is easy to underestimate. Machine identities already outnumber human identities by 82 to 1, and yet 88% of organisations still define a “privileged user” as strictly human. 42% of non-human and AI identities already hold privileged or sensitive access. We have a massive, growing population of non-human principals operating with significant permissions, under governance frameworks designed entirely for people. That’s a structural mismatch, not a tooling gap.</p><p>To understand why the stakes are so high, look at the velocity. Wiz reported that a single enterprise agent deployment generated 42,000 cloud API calls in an hour and performed over 300 IAM actions in four minutes. Agent usage across the organisations they track grew 12× in Q1 2026 alone. These aren’t background processes ticking away overnight. They’re high-frequency principals making consequential decisions at a rate no manual review process can keep up with.</p><p>In sessions with security and identity leaders, the point was made repeatedly — the provisioning process we use for humans (job acceptance, contract signing, onboarding interview) is meaningless for agents. And giving an agent the same broad access its owning engineer has is far too much.</p><p>The model that works — think of deploying an agent the way you’d onboard a contractor. Give them an identity and a “laptop” (a sandboxed Virtual Machine with constrained permissions). Scoped access for the task at hand. Time-bound, temporary access to servers. Not the full keyring.</p><p>A useful distinction from the IAM session — agentic applications don’t need one identity. They need identities, plural:</p><ul><li><strong>Human identities</strong> for when people collaborate with agents</li><li><strong>Agent identities</strong> for when the agent itself acts autonomously</li><li><strong>Delegated identities</strong> for when the agent acts on behalf of a specific user, with credentials injected by a gateway and never directly visible to the agent</li></ul><p>Each has a different provisioning model and a different risk profile. They need to be designed for separately.</p><p>Traditional IAM uses static, pre-provisioned identities. Agentic IAM requires identities to be minted on the fly at runtime, with context-aware authorisation. The shift in the question being asked is from “can this principal do X?” to “should this agent, in this specific context, be allowed this action?”.</p><p>Every agent identity needs four things:</p><ol><li><strong>Accountability</strong> — a named owner.</li><li><strong>Authentication</strong> — unique verifiable credentials, not shared secrets.</li><li><strong>Least privilege</strong> — explicitly scoped access.</li><li><strong>Observability</strong> — every action logged for audit and forensics.</li></ol><p>There are also five pillars of agent authorisation to evaluate for every agent you deploy:</p><ul><li><strong>Network reach</strong> — which endpoints and services are in scope?</li><li><strong>Data scope</strong> — what’s the read/write boundary?</li><li><strong>Functional authority</strong> — which API methods can it actually invoke?</li><li><strong>Contextual logic</strong> — under what conditions is access granted?</li><li><strong>Temporal constraints</strong> — what’s the Time To Live (TTL) on this session or credential?</li></ul><p>One AI security session introduced a concept I think every org should adopt — <strong>Intent-Based Access Control (IBAC)</strong>. The standard least-privilege question is “does this agent have permission for this action?”. IBAC adds a second question — “does this action align with what this agent is actually supposed to be doing?”. An agent with appropriate read/write scope could still trigger an IAM modification or a bulk data export that’s technically within its permissions but completely outside its purpose. IBAC blocks high-risk actions that fall outside the agent’s objective, regardless of whether the principal technically holds permission. The question moves from entitlement to intent.</p><h3>Governance as a Programme, Not an Afterthought</h3><p>A question raised across the security sessions, and one I don’t think any of us have a good answer to yet — what happens to an agent when the person who created it leaves the organisation? Strong ownership is not optional. Every agent needs a designated human accountable for its behaviour, its active permissions, credential rotation, and eventual decommissioning.</p><p>One risk category the conference named directly is <strong>Shadow AI</strong> — the deployment of AI agents and solutions without governance, oversight, or accountability. Just like Shadow IT created unmanaged infrastructure that became a liability later, Shadow AI creates an invisible population of agents operating outside any identity framework or audit trail or decommissioning protocol. Every agent needs a kill switch — a sunsetting protocol that revokes credentials, terminates sessions, and decommissions the agent cleanly when it’s abandoned or its owner moves on.</p><p>The scale is hard to wrap your head around. IDC projects 1.3 billion agents will be in production by 2028. And we’re already struggling — one study (IBM, 2025) found the average enterprise juggles 83 different security solutions from 29 vendors. Patchwork security was already a problem. At agent scale, fragmented tooling becomes a structural liability — more gaps to fall through, more handoffs to exploit, more noise drowning out the actual signals.</p><p>The conference offered a practical three-wave roadmap that I think holds up:</p><ul><li><strong>Wave 1 (Now)</strong> — centralised infrastructure, org-level guardrails, audit and logging, understanding business requirements</li><li><strong>Wave 2 (Next)</strong> — advanced security controls, runtime defence (anomaly detection, prompt injection protection, fraud defence), reusable agent templates</li><li><strong>Wave 3 (Later)</strong> — advanced toolchain integration, security embedded into CI/CD pipelines (DevSecOps), governance at scale</li></ul><p>The closing message from the security session was the line I’d pin above my desk — <strong>“Secure AI is Scalable AI.”</strong> Security isn’t what slows the agentic programme down. It’s what makes it possible to run at scale.</p><h3>Summary</h3><p>If I had to compress the week into a working agenda for any tech org right now, this is what it’d look like:</p><ol><li><strong>Reframe what a great engineer looks like.</strong> The most valuable engineer in the agentic era isn’t the one writing the most code. It’s the one designing the best context, the best playbooks, the best guardrails. We should be talking about this shift openly, training for it deliberately, and measuring for it explicitly.</li><li><strong>Treat agent infrastructure as first-class infrastructure.</strong> Custom agents, skills, instruction files, MCP server configurations — these aren’t side projects. They’re the layer that compounds. Investing now pays back continuously.</li><li><strong>Build a shared catalogue of agents and skills.</strong> Don’t let every team reinvent the wheel. The orgs moving fastest already have shared, governed, discoverable agent libraries.</li><li><strong>Give agents their own identities. Today.</strong> Stop deploying them under engineer credentials. Design IAM for an agentic workforce, not a human one. IMO this is the highest-leverage security investment any org can make right now.</li><li><strong>Make AI governance a programme, not a side conversation.</strong> Ownership, kill switches, audit, decommissioning — all of it. Shadow AI will be much harder to clean up than Shadow IT was.</li><li><strong>Don’t wait for perfect.</strong> Start with read-only agents. Build the trust data. Then extend. The cost of moving cautiously is recoverable. The cost of moving late isn’t.</li></ol><p>The agentic era isn’t coming. It’s here! The question is whether we build the foundations now — the identity frameworks, the agent infrastructure, the cultural shift toward trusting and overseeing agents — or whether we scramble to catch up later.</p><p>I’d much rather be on the side of the orgs leading it.</p><p>Right so that’s it folks, thanks for reading and I hope it was useful! If you have questions or different perspectives, feel free to drop them below.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=992d975adb28" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Ingress Controller versus Kubernetes Gateway API]]></title>
            <link>https://medium.com/@carlocolumna/ingress-controller-versus-kubernetes-gateway-api-7f349eb33eb9?source=rss-d9219c5b4599------2</link>
            <guid isPermaLink="false">https://medium.com/p/7f349eb33eb9</guid>
            <dc:creator><![CDATA[Carlo Columna]]></dc:creator>
            <pubDate>Sun, 09 Feb 2025 03:58:13 GMT</pubDate>
            <atom:updated>2025-02-09T03:58:13.207Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction</h3><p>Dubbed as the next evolution or next generation of not only Kubernetes Ingress, Load Balancing, and Service Mesh APIs, will Kubernetes Gateway API live up to the hype? Let’s examine this article, as the traditional Kubernetes Ingress goes toe to toe against the new kid on the block the Kubernetes Gateway API!</p><p>First, we’ll briefly discuss Kubernetes ingress and Gateway API and explain how they work. Then we’ll start comparing their features and then we’ll end by looking at where we can start adopting it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/627/1*ICK-JSWWQKeITyhY_E9n9g.png" /><figcaption><a href="https://img.devrant.com/devrant/rant/r_3775630_L5wGZ.jpg">devrant</a></figcaption></figure><h3>What is Kubernetes Ingress?</h3><p>Simply put, it is an API object that exposes HTTP and HTTPS routes from outside the cluster to services within the cluster. Traffic routing is controlled by rules defined on the Ingress resource.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/681/1*hEedRadpFarDLVaRTrIjqQ.png" /><figcaption>Simple Ingress traffic to a Service — kubernetes.io</figcaption></figure><p>In summary, Ingress can be used to</p><ol><li>Expose Kubernetes services outside of the cluster</li><li>Traffic load balancing</li><li>Terminate SSL/TLS connections</li><li>Offer name-based virtual hosting</li></ol><p>To use Ingress, we need to run an <a href="https://kubernetes.io/docs/concepts/services-networking/ingress-controllers">Ingress controller</a> in our cluster. Creating an Ingress resource without it has no effect. An Ingress Controller acts as a reverse proxy and load balancer within a Kubernetes cluster. A controller keeps an eye on created Ingress resources and then updates its configuration automatically to route traffic to services based on the Ingress resources rules.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/741/0*b4xSHFcrDMc7jbYj" /><figcaption>ingress-nginx controller — checkmateq.com</figcaption></figure><p>In this article, we’ll be using the <a href="https://github.com/kubernetes/ingress-nginx">ingress-nginx</a> Ingress controller for comparison. Leveraging the powerful Nginx web server, it is one of the most widely adopted Ingress Controllers in the Kubernetes ecosystem.</p><p><strong>Note:</strong> Kubernetes has frozen Ingress development. Instead, it’s focusing on adding new features to the <a href="https://kubernetes.io/docs/concepts/services-networking/gateway/">Gateway API</a>.</p><h3>What is Kubernetes Gateway API?</h3><p>In November 2023, the <a href="https://github.com/kubernetes-sigs/gateway-api/releases/tag/v1.0.0">Kubernetes Gateway API v1.0</a> was released marking a huge milestone on a project 4 years in the making. The result of this project which is considered to be the <a href="https://twitter.com/robertjscott/status/1719419469611848071">most collaborative API in Kubernetes history</a>, is a standardised way to manage network traffic and to address traditional Ingress flexibility and extensibility limitations.</p><iframe src="https://cdn.embedly.com/widgets/media.html?type=text%2Fhtml&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;schema=twitter&amp;url=https%3A//x.com/robertjscott/status/1719419469611848071&amp;image=" width="500" height="281" frameborder="0" scrolling="no"><a href="https://medium.com/media/16d6485c453f84c47a62b332dcfe9737/href">https://medium.com/media/16d6485c453f84c47a62b332dcfe9737/href</a></iframe><p>From the onset, it was <a href="https://kubernetes.io/docs/concepts/services-networking/gateway/#design-principles">designed to be extensible, portable, expressive, and role-oriented</a>.</p><h3>Design</h3><p>Gateway API introduced three APIs on its resource model — GatewayClass, Gateway and Routes. The role-oriented design of Gateway API is explicitly shown on the diagram below showing the three separate personas and corresponding resources they are responsible for.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/610/1*fmRz7tgDTriUekK5aNCYWw.png" /><figcaption>Role-oriented Resource Model — gateway-api.sigs.k8s.io</figcaption></figure><h4><strong>GatewayClass</strong></h4><p>Defines a set of gateways with common configuration and managed by a controller that implements the class. Some examples of controllers are:</p><ul><li>AWS Gateway API Controller</li><li>NGINX Gateway Fabric</li><li>GKE Gateway controller</li><li>Gloo Gateway 2.0 by Solo.io</li></ul><h4><strong>Gateway</strong></h4><p>Defines how traffic can be translated to Services within the cluster usually through the use of a cloud load balancer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/397/0*v9lLWveBjGRwMu9N" /><figcaption>Sample Gateway manifest</figcaption></figure><h4><strong>Routes</strong></h4><p>Describes how traffic coming via the Gateway maps to the Services.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/391/0*jwdFKrI31hnqZLiV" /><figcaption>Sample Route manifest</figcaption></figure><h3>AWS Implementation</h3><p>AWS implementation of the Kubernetes <a href="https://gateway-api.sigs.k8s.io/">Gateway API</a> is to incorporate it into AWS Application Networking via AWS VPC Lattice. AWS VPC Lattice facilitates application-level connectivity across all compute types. It provides a way to have a single, overarching services view of all services across multiple VPCs.</p><p>AWS provides the <a href="https://www.gateway-api-controller.eks.aws.dev/">AWS Gateway API Controller</a> to integrate AWS VPC Lattice with the Kubernetes Gateway API. The controller will keep an eye on created Gateway API resources in the cluster such as gateways and routes and then provisions the corresponding Amazon VPC Lattice objects.</p><p>The image below shows the relationship between VPC Lattice and Kubernetes. It shows a mapping of Kybernetes objects to their corresponding equivalent in VPC Lattice.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ci-EPUkvi3Uf9suOTy3HVg.png" /><figcaption>VPC Lattice to K8s Gateway objects —aws</figcaption></figure><p>From AWS docs, Gateway API Controller integration with Amazon VPC Lattice allows you to:</p><ul><li>Handle network connectivity seamlessly between services across VPCs and accounts.</li><li>Discover these services spanning multiple Kubernetes clusters</li><li>Implement a defense-in-depth strategy to secure communication between those services.</li><li>Observe the request/response traffic across the services.</li></ul><h3>Compatibility with other addons</h3><p><a href="https://cert-manager.io/v1.9-docs/usage/gateway/">cert-manager</a> and <a href="https://kubernetes-sigs.github.io/external-dns/v0.13.1/tutorials/gateway-api/">external-dns </a>both supports Gateway API.</p><h3>Kubernetes Ingress vs Gateway API</h3><p>Now we’ll explore and compare the different features and use cases between Ingress and Gateway API.</p><h3>Role-oriented</h3><p>Infrastructure is built to be shared but this raises a challenge — providing flexibility to infrastructure users while infrastructure owners maintain control</p><h4>Ingress</h4><p>Sometimes multiple teams work on the same ingress, especially for bigger applications. This can also happen when teams split into multiple teams. This can lead to friction between teams all working on the same resource. To make matters worse, a single misconfiguration from another team can knock out the whole Ingress affecting other teams’ applications.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/994/1*JvJql_TPt4AJkTWiLchkeQ.png" /></figure><h4>Gateway API</h4><p>As you can see in the diagram above, in Gateway API there is a clear delineation of responsibility on who manages API resources by different organizational roles.</p><blockquote><em>Gateway API accomplishes this through a role-oriented design for Kubernetes service networking that strikes a balance between distributed flexibility and centralized control. It allows shared network infrastructure (hardware load balancers, cloud networking, cluster-hosted proxies etc) to be used by many different and non-coordinating teams, all bound by the policies and constraints set by cluster operators. — </em><a href="http://gateway-api.sigs.k8s.io/"><em>http://gateway-api.sigs.k8s.io</em></a></blockquote><h4>Winner: Gateway API</h4><p>With RBAC policy in place, the role-oriented design of the Gateway API allows developers to work confidently on their resources without the possibility of affecting other teams. It prevents overlap and less friction between teams sharing the infrastructure.</p><h3>Ease of Use and Standardisation</h3><h4>Ingress</h4><p>As developers, we need to customise networking behaviours to fit our application’s needs. Because we don’t have a standard way of solving these, we started using custom ingress annotations to solve them. This resulted in the proliferation of annotations specific to different Ingress Controllers. These in turn resulted in poor developer experience as annotations are hard to use and maintain and hard to validate as string values. This also made it quite hard to migrate from one Ingress implementation to another.</p><h4>Gateway API</h4><p>Gateway API simplifies networking configurations by abstracting away complexities and providing a user-friendly, declarative approach to define routing and traffic policies. Its resources have built-in support for common traffic routing use cases such as header-based matching, traffic weighting and other capabilities. And as such unlike Ingress, you don’t have to set any custom annotations.</p><p>The lack of a standard way to solve service networking requirements was one of the main drivers of the Gateway API project. The Gateway API now offers a standard solution to various networking requirements such as redirects or timeouts. See more examples in the table below,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/771/1*6t6qF9FPCkjvMf5TlSv-zA.png" /><figcaption>Common networking requirements standardised by the K8s Gateway API</figcaption></figure><h4>Winner: Gateway API</h4><p>Gateway API provides a declarative and consistent method to define routing rules freeing up developers from concern with network complexities and instead focusing on their application.</p><h3>Scalability</h3><h4>Ingress</h4><p>Ingress relies on Ingress controllers to work. These controllers can be hard to configure and maintain for cluster operators. Configuration errors from unintended changes or upgrades can also result in service downtime.</p><p>Ingress controller pods route traffic to multiple applications and auto-scales as needed. However, multiple hops happen before traffic gets to the application. Controllers may struggle to scale efficiently as cluster workloads and traffic increases which may result in latencies and performance degradation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Y9J1ObxOSy3Hk3sZljJCvg.png" /><figcaption>ingress flow using controller</figcaption></figure><h4>Gateway API</h4><p>Gateway API is not constrained by Ingress controller pods that act as middlemen. Performance and scalability are only constrained by the underlying infrastructure.</p><h4>Winner: Gateway API</h4><h3>Multi-Cluster Support</h3><p>This refers to the ability to connect multiple Kubernetes clusters facilitating the:</p><ul><li>communications between services, and</li><li>traffic routing</li></ul><p>on the connected clusters without dealing with the underlying infrastructure.</p><h4>Ingress</h4><p>Ingress controllers are mostly designed to run inside the clusters. This allows them to benefit from Kubernetes features such as scaling, availability etc. It’s possible to run them outside of the clusters albeit their functionality may be limited. They can function as reverse proxy servers or API gateways in this regard. However, they won’t be able to do automatic service discovery, integration with Kubernetes resources (such as Services and Ingress resources), or dynamic configuration updates based on changes to Kubernetes objects.</p><h4>Gateway API</h4><p>Unlike Ingress controllers, Gateway API is not limited to a single cluster. It can extend its functionalities into a multicluster environment supporting the design of modern, resilient and distributed applications.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4s2xysRIzITDC2SRUjpWAQ.png" /><figcaption>Multi-Cluster, Multi-VPC Kubernetes with VPC Lattice — aws</figcaption></figure><p>The AWS Gateway API controller integration of VPC Lattice and Kubernetes Gateway API takes the multi-environment design even further as applications can run on instances, containers or serverless.</p><h4>Winner: Gateway API</h4><p>This ability can open up huge opportunities in your systems such as:</p><ul><li><strong>Application Resilience and Availability: </strong>Issues with the changes brought about by Kubernetes cluster upgrades can disrupt running applications. Regional or zonal disruptions from cloud infrastructure providers can also affect your workloads. Running applications in a multi-cluster, multi-compute environments spread on multi-region and multi-AZ locations will greatly increase applications’ fault tolerance and resilience to these types of disruptions.</li><li><strong>Compute Flexibility: </strong>As mentioned above, applications can be deployed and run on different compute types such as containers in EKS or ECS, EC2 instances or serverless Lambda. Traffic splitting between these compute types can help in the rollout of new application versions.</li></ul><h3>HTTP Routing</h3><h4>Ingress</h4><p>The Ingress resource allows you to specify multiple hosts. On each host, you can specify multiple paths with the corresponding service backends. Hostname matching using wildcards is also supported.</p><h4>Gateway API</h4><p>In addition to matching HTTP traffic to host and path, the HTTPRoute resource also allows you to match based on headers. For example below, it can match a header env: canary.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/763/1*2PfRI_zfDEMhdrH8eQFz_g.png" /><figcaption>HTTP Routing — gateway-api.sigs.k8s.io</figcaption></figure><h4><strong>Winner: Gateway API</strong></h4><p>Developers have more options for HTTP routing.</p><h3>HTTP Traffic Splitting</h3><h4>Ingress</h4><p>Kubernetes Ingress does not have built-in support for canary or blue-green deployments. This can be achieved through the ingress-nginx controller but again you have to use multiple annotations.</p><h4>Gateway API</h4><p>The <a href="https://gateway-api.sigs.k8s.io/api-types/httproute">HTTPRoute</a> resource allows you to specify weights to shift traffic between different backends.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BRGZkpUJRv41PCa-sVUNGw.png" /><figcaption>Traffic Splitting — gateway-api.sigs.k8s.io</figcaption></figure><h4>Winner: Gateway API</h4><p>Developers can easily split traffic and test their changes as support is built into the resource without having to worry about annotations which are not only complicated to use but also tie them to a particular Ingress Controller.</p><h3>Cross-Namespace Routing</h3><h4>Ingress</h4><p>Moving your application to a different namespace? Don’t even think about it. Do you want to split your application into different namespaces under one domain? Not a chance! These are unfortunately not possible with the current Ingress configurations.</p><h4>Gateway API</h4><p>Gateway API has core support for cross Namespace routing. With this ability, not only can we have more flexibility and cater to the requirements mentioned above that Ingress cannot fulfil but also allows us to minimize access and fault domains.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/791/1*i38AZCyIFCBCYmxsKufN-w.png" /><figcaption>Cross Namespace Routing — gateway-api.sigs.k8s.io</figcaption></figure><h4>Winner: Gateway API</h4><h3>Extensibility</h3><p>Refers to the ability to customise and augment the behaviour or functionality of existing resources.</p><h4>Ingress</h4><p>No support.</p><h4>Gateway API</h4><p>Gateway API creates networking resources using Custom Resource Definitions (CRDs) providing an extensible way to define and configure routing and traffic management rules. This extensibility allows for custom resources to be linked at various layers of the AP providing more granular customization.</p><h4>Winner: Gateway API</h4><p>The extensibility of Gateway API resources enabled the introduction of policy at various layers of the API such as <a href="https://gateway-api.sigs.k8s.io/reference/policy-attachment/">Metaresources and Policy Attachment</a>. Several use case for using policy are as follows:</p><p><strong>Networking Policies</strong></p><ul><li>Retry Policy</li><li>Rate Limiting</li><li>Request/Response Transformation</li></ul><p><strong>Security Policies</strong></p><ul><li>TLS Policy</li><li>Authentication</li><li>IP whitelisting</li></ul><h3>Protocols</h3><p>The table below shows the common supported protocols.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/772/1*_UKNjf7CDVtPq-AlphJRYA.png" /></figure><h4>Winner: Ingress, for now</h4><p>As Gateway API is new, other protocols apart from HTTP/S and gRPC are currently still experimental but actively being worked on.</p><h3>East/West Traffic</h3><p>This depicts traffic between applications running inside the cluster.</p><h4>Ingress</h4><p>No support.</p><h4>Gateway API</h4><p>Support for this is currently experimental and led by the <a href="https://gateway-api.sigs.k8s.io/mesh/gamma/">GAMMA Initiative</a> (Gateway API for Service Mesh). A service mesh is software that manages infrastructure providing security, reliability, and observability for communications between workloads (east/west traffic).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*5ODA3wnZHZBHzHE1" /><figcaption>Service Mesh — <a href="https://medium.com/codex">medium.com/codex</a></figcaption></figure><h4>Winner: Tie</h4><p>This project is useful for service mesh users. Uniting both North/South and East/West under Gateway API will simplify configurations and management. However, we don’t use service mesh in our PK8s clusters as such there is no clear winner.</p><h3>Cost</h3><h4>Ingress</h4><p>The community-supported free version of the Nginx-based Ingress controller is a solid ingress controller. The controller can be deployed and runs like a normal service in a cluster. Each controller may run 20 to 40 pods depending on the size of cluster. The load balancer in front of the controller and the resources consumed by the controller pods are the only easily quantifiable cost associated with running it.</p><h4>Gateway API</h4><p>As AWS has elected to use Amazon VPC Lattice to implement the Kubernetes Gateway API. From the AWS <a href="https://aws.amazon.com/vpc/lattice/pricing/">website</a>, you will be charged for VPC Lattice components. Cost is based on:</p><ul><li>number of services provisioned</li><li>data processing charges for traffic to and from each service,</li><li>number of requests that each service receives</li></ul><h4>Winner: Ingress</h4><p>Not considering any operational overhead to cluster-administrators the current Ingress/ingress-nginx configuration is quite cheap to run. In contrast, you have to pay for extra networking service in using VPC Lattice for Gateway API.</p><h3>Verdict</h3><h4><strong>Overall Winner: Gateway API</strong></h4><p>Gateway API has emerged as the overall winner after winning most of the features or areas that we looked at. Its combination of design and functionalities allows for customization and extensibility as well as complex routing and traffic management unmatched by traditional Ingress networking. Its role-oriented design and standardisation of various networking requirements result in good developer experience. Furthermore, its support for multi-cluster as well as multi-compute environments opens up huge opportunities for everyone as they asses the current and future state of compute in their organisation.</p><h3>Adoption Recommendations</h3><p>So Gateway-API is great should we start converting our applications to use it? Not so fast!</p><p>Even with all the new amazing features and capabilities that we get from Gateway API, it represents a huge paradigm shift in how we manage and control Kubernetes traffic. It’s also not as mature and it may not yet offer standard solutions to complex networking requirements.</p><p>IMO we should start with the following use cases:</p><ul><li>greenfield applications: new applications will benefit from its features</li><li>migrating applications: helps in migrating into Kubernetes from other compute platforms</li><li>routing requirements: canary releases or any other requirements that cannot be solved by traditional Ingress such as cross-namespace routing</li><li>multi-compute requirements: running across different compute platforms</li></ul><p>Right so that’s it folks, thanks for reading and I hope it will be useful! If you have some questions, or feedback feel free to add them below.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7f349eb33eb9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Scale Your Pods Based on HTTP Traffic]]></title>
            <link>https://medium.com/@carlocolumna/how-to-scale-your-pods-based-on-http-traffic-d58221d5e7f1?source=rss-d9219c5b4599------2</link>
            <guid isPermaLink="false">https://medium.com/p/d58221d5e7f1</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[autoscaling]]></category>
            <category><![CDATA[cloud-native]]></category>
            <category><![CDATA[keda]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Carlo Columna]]></dc:creator>
            <pubDate>Sat, 12 Aug 2023 04:25:55 GMT</pubDate>
            <atom:updated>2023-08-31T03:37:06.491Z</atom:updated>
            <content:encoded><![CDATA[<p>Kia ora everyone! How’s it going?</p><p>Welcome to the third and last part of a blog series discussing autoscaling capabilities in Kubernetes using <a href="https://keda.sh/">KEDA or Kubernetes Event-driven Autoscaling</a>. If you haven’t, I highly recommend checking out the first two parts of the series below.</p><ul><li><a href="https://medium.com/@carlocolumna/eda-in-amazon-eks-install-keda-82849cf31f01">Part 1: How to Level Up Your Kubernetes Scaling</a></li><li><a href="https://medium.com/@carlocolumna/keda-in-amazon-eks-part-2-scale-based-on-aws-sqs-queue-1dbd13fa5384">Part 2: How to Scale Your Pods Based on a Queue</a></li></ul><p>In this part three of this blog series we will be scaling a workload running in an Amazon EKS cluster based on HTTP traffic.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jGQwowmuZC6LN3LYJP3eqA.png" /></figure><p>The structure of this blog would be as follows:</p><ol><li>Background</li><li>KEDA HTTP Add-On</li><li>Creating and installing demo application</li><li>Setup HTTP Scaling</li><li>Test Scaling — explore three scaling scenarios</li></ol><h3>Background</h3><p>As a start, let’s ask the question why do we even need this scaling capability?</p><p>In the first part of this blog series, <a href="https://medium.com/@carlocolumna/eda-in-amazon-eks-install-keda-82849cf31f01">How to Level Up Your Kubernetes Scaling</a><em>, </em>we’ve discussed in detail the limitations of a <a href="https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/"><em>Horizontal Pod Autoscaler</em></a>. Below are two of these limitations:</p><blockquote><strong>It cannot scale to zero. </strong>HPA by default uses metrics namely CPU and memory utilisation to calculate the desired number of replicas. Because these metrics cannot be zero, the desired number of replicas cannot be zero as well. This is not ideal for intermittent and resource-intensive workloads when you are trying to optimise your cost.</blockquote><blockquote><strong>It’s limited to scaling based on metrics. </strong>You are unable to scale based on events or HTTP traffic.</blockquote><p>As mentioned above, CPU and memory can never be zero. For our workload to scale to zero, we need to use a metric different from CPU and memory. Furthermore, when scaling HTTP servers, CPU and memory utilisation do not always correspond to the number of HTTP requests. As such, there is a need to be able to directly scale based on the actual HTTP requests. This is where the KEDA HTTP Add-on comes in!</p><h3>KEDA HTTP Add-On</h3><p>From the website, <a href="https://github.com/kedacore/http-add-on">KEDA HTTP Add-on</a> allows Kubernetes users to automatically scale their HTTP servers up and down (including to/from zero) based on incoming HTTP traffic. Just a bit of caution that the HTTP Add-on currently is in <a href="https://github.com/kedacore/http-add-on/releases/latest">beta</a> and as such it’s not yet recommended for production usage. This should not stop us, however, from using it for non-mission critical workloads.</p><h4>Why an Add-On?</h4><p>As the name suggests, it is an add-on for KEDA which means you have to install it separately on top of KEDA. As many as KEDA scalers are, it does not include an HTTP-based scaler out of the box. The reasons why are explained quite well <a href="https://github.com/kedacore/http-add-on#http-autoscaling-made-simple">here</a>.</p><h4>How does it work?</h4><p>Note this section is heavily copied from the KEDA AddOn docs because it’s already quite well explained <a href="https://github.com/kedacore/http-add-on/tree/main">here</a>.</p><figure><img alt="KEDA-HTTP Architecture Overview" src="https://cdn-images-1.medium.com/max/1024/1*gpBjTZ2sc3MAnVOtOiWX8g.png" /><figcaption>Architecture Overview</figcaption></figure><p>Above is the architecture overview of how KEDA-HTTP works in conjunction with KEDA. This is explained in more detail below.</p><p><strong>High-Level Components</strong></p><p>There are three major components of KEDA-HTTP plus KEDA itself.</p><ul><li>Operator — This component listens for events related to HTTPScaledObjects and creates, updates or removes internal machinery as appropriate.</li><li>Interceptor — This component accepts and routes external HTTP traffic to the appropriate internal application, as appropriate.</li><li>Scaler — This component tracks the size of the pending HTTP request queue for a given app and reports it to KEDA. It acts as an <a href="https://keda.sh/docs/latest/scalers/external-push/">external scaler</a>.</li><li><a href="https://keda.sh/">KEDA</a> — KEDA will receive metric from the external scaler and will scale up or down our HTTP application.</li></ul><p><strong>Setup HTTP scaling</strong></p><p>Scaling is easily configured through a <a href="https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/">Custom Resource (CRD)</a> called HTTPScaledObject.http.keda.sh or HTTPScaledObject for short. As mentioned above, the KEDA-HTTP operator will keep an eye on these CRDs and will take corresponding actions. When one is created, the operator does the following:</p><ul><li>Update an internal routing table that maps incoming HTTP hostnames to internal applications.</li><li>Furnish this routing table information to interceptors so that they can properly route requests.</li><li>Create a <a href="https://keda.sh/docs/2.3/concepts/scaling-deployments/#scaledobject-spec">ScaledObject</a> for the Deployment specified in the HTTPScaledObject resource.</li></ul><p>When the HTTPScaledObject is deleted, the operator reverses all of the aforementioned actions.</p><p>This CRD object instructs interceptors to forward requests for our applications host to the app’s backing Service. This is where you can also specify the number of <em>pending</em> (or in-progress) requests that our application needs to have before the HTTP Add-on will scale it.</p><p>TheHTTPScaledObject full specifications can be found <a href="https://github.com/kedacore/http-add-on/blob/main/docs/ref/v0.3.0/http_scaled_object.md">here</a>.</p><p><strong>Autoscaling</strong></p><p>After an HTTPScaledObject is created and the operator creates the appropriate resources, we must send HTTP requests through the interceptor so that our HTTP application is scaled. We must send the request to the interceptors Service called keda-add-ons-http-interceptor-proxy which is created when we install the KEDA-HTTP Add-on.</p><p>The interceptor keeps track of the number of pending HTTP requests — HTTP requests that it has forwarded but the app hasn’t returned. The scaler periodically makes HTTP requests to the interceptor via an internal RPC endpoint — on a separate port from the public server — to get the size of the pending queue. Based on this queue size, it reports scaling metrics as appropriate to KEDA. As the queue size increases, the scaler instructs KEDA to scale up as appropriate. Similarly, as the queue size decreases, the scaler instructs KEDA to scale down.</p><p>The HTTP Add-on works with the Kubernetes <a href="https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/#algorithm-details">Horizontal Pod Autoscaler</a> (HPA) — via KEDA itself — to execute scale-up and scale-down operations (except for scaling between zero and non-zero replicas). The add-on furnishes KEDA with two metrics — the current number of pending requests for a host, and the desired number (called targetPendingRequests in the <a href="https://github.com/kedacore/http-add-on/blob/main/docs/ref/v0.3.0/http_scaled_object.md">HTTPScaledObject</a>). KEDA then sends these metrics to the HPA, which uses them as the currentMetricValue and desiredMetricValue, respectively, in the <a href="https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/#algorithm-details">HPA Algorithm</a>.</p><p>The net effect is that the add-on scales up when your app grows to more pending requests than the targetPendingRequests value and scales down when it has fewer than that value.</p><h4>Why does this project route HTTP requests?</h4><p>To autoscale HTTP servers, the KEDA-HTTP Add-on needs access to metrics that it can report to KEDA so that KEDA itself can scale the target HTTP server. This is done through the use of an <a href="https://github.com/kedacore/http-add-on/blob/main/interceptor">interceptor</a> and <a href="https://github.com/kedacore/http-add-on/blob/main/scaler">external scaler</a> which proxy incoming HTTP requests to provide autoscaling metrics. However, the KEDA-HTTP is minimally involved with routing and they are working on ways to get out of the “critical path” of an HTTP request as much as possible.</p><p>All right, that’s it for the KEDA-HTTP concepts, let’s see it in action!</p><h3>Demo</h3><p>For the demo, we’ll be scaling an HTTP application through the three scenarios below:</p><ul><li>Scenario 1: Using a <em>ClusterIP</em> Service only</li><li>Scenario 2: Using an Ingress and NGINX Ingress Controller — We’ll expose our HTTP application outside of the Kubernetes cluster so we can scale based on external traffic.</li><li>Scenario 3: Using an Ingress, NGINX Ingress Controller and an <em>ExternalName</em> Service— Same as the previous one but with the Ingress resource and demo application on the same Namespace.</li></ul><h4>Prerequisites</h4><p>Before we can proceed, let’s lay out the prerequisites:</p><ul><li>Kubernetes cluster, we’ll be using an Amazon EKS cluster for the demo</li><li>KEDA installed in the Kubernetes cluster, see <a href="https://medium.com/@carlocolumna/eda-in-amazon-eks-install-keda-82849cf31f01">part 1</a> of this blog series on how to install</li><li>KEDA-HTTP Add-on and NGINX Ingress Controller installed in the Kubernetes cluster, see how to install both on the next steps below</li><li>Terminal with <a href="https://kubernetes.io/docs/tasks/tools/">kubectl</a> installed</li></ul><p><strong>Install KEDA-HTTP Add-On</strong></p><p>Before installing it’s important to note that there are two modes of installation for both KEDA and KEDA-HTTP Add-on — cluster-global and namespaced mode.</p><ul><li>cluster-global — ScaledObjects and HTTPScaledObjects (respectively) can be installed in any Namespace, and one installation will detect and process it.</li><li>namespaced — You must install your ScaledObjects and HTTPScaledObjects in a specific Namespace.</li></ul><p>Both should be installed with the same mode, e.g. if you install one as cluster-global, the other must also be cluster-global.</p><p>We have installed KEDA from <a href="https://medium.com/@carlocolumna/eda-in-amazon-eks-install-keda-82849cf31f01">part 1</a> of this blog series using cluster-global mode so we’ll do the same for the Add-on. This allows the flexibility of creating ScaledObjects and HTTPScaledObjects in any Namespace albeit giving cluster privileges to KEDA and the Add-on.</p><p>To install we can use the Helm chart which is inside the KEDA’s default helm repository at <a href="http://github.com/kedacore/charts">kedacore/charts</a>.</p><p>To keep all KEDA components in one place, let’s install the Add-on to the <em>keda</em> Namespace by running:</p><pre>helm install http-add-on kedacore/keda-add-ons-http --namespace keda</pre><p>Note that this installed the Add-on in cluster-global mode. Add --set operator.watchNamespace=&lt;target namespace&gt; to install the Add-on in namepaced mode. If we want to do this, we must also install KEDA in namespaced mode and use the same target Namespace.</p><p>Installing the Add-on won’t affect any running workloads in the cluster. We’ll need to install an HTTPScaledObject for each Deployment we want to scale.</p><p>After running the helm install command above, we should see the Add-on components prefixed with keda-add-ons-* running as Deployments.</p><pre>$ kubectl get deployments -n keda<br>NAME                                   READY   UP-TO-DATE   AVAILABLE   AGE<br>keda-add-ons-http-controller-manager   1/1     1            1           5s<br>keda-add-ons-http-external-scaler      1/1     1            1           6s<br>keda-add-ons-http-interceptor          3/3     3            3           6s<br>keda-operator                          1/1     1            1           26d<br>keda-operator-metrics-apiserver        1/1     1            1           26d</pre><pre>$ helm list -n keda<br>NAME        NAMESPACE REVISION UPDATED                                 STATUS   CHART                   APP VERSION<br>http-add-on keda      1        2023-06-27 19:42:00.912006094 +0000 UTC deployed keda-add-ons-http-0.4.1 0.4.0<br>keda        keda      3        2023-06-27 19:41:24.550943176 +0000 UTC deployed keda-2.9.4              2.9.3</pre><p>You can customise the Add-on Helm installation by updating the configuration parameters as listed <a href="https://github.com/kedacore/charts/tree/main/http-add-on">here</a>.</p><p>Right, with KEDA and the HTTP Add-on both installed we are now ready to create our demo application.</p><h4>Install demo application</h4><p>For our demo application, we’ll continue the tradition from the first two blog series by spinning up an NGINX application. First, let’s create a Namespace using kubectl like so,</p><pre>$ kubectl create namespace demo-ns<br>namespace/demo-ns created</pre><p>We then create a Deployment like so,</p><pre># Generate manifest<br>$ kubectl create deploy demo -n demo-ns --dry-run=client --image nginx -oyaml<br>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  creationTimestamp: null<br>  labels:<br>    app: demo<br>  name: demo<br>  namespace: demo-ns<br>spec:<br>  replicas: 1<br>  selector:<br>    matchLabels:<br>      app: demo<br>  strategy: {}<br>  template:<br>    metadata:<br>      creationTimestamp: null<br>      labels:<br>        app: demo<br>    spec:<br>      containers:<br>      - image: nginx<br>        name: nginx<br>        resources: {}<br>status: {}<br><br># Create deployment<br>$ kubectl create deploy demo -n demo-ns --image nginx<br>deployment.apps/demo created<br><br># Check the pod is running<br>$ kubectl get pods -n demo-ns -w<br>NAME                    READY   STATUS              RESTARTS   AGE<br>demo-68b4b4d5bf-f84jv   0/1     ContainerCreating   0          5s<br>demo-68b4b4d5bf-f84jv   1/1     Running             0          9s</pre><p>Let’s expose the deployment we created by creating a Service of type <em>ClusterIP</em> like so,</p><pre># Create Service<br>$ kubectl expose deployment demo -n demo-ns --port=80 --target-port=80<br>service/demo exposed<br><br># Double-check Service and Endpoint<br>$ k get svc,ep -n demo-ns<br>NAME           TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE<br>service/demo   ClusterIP   172.20.121.195   &lt;none&gt;        80/TCP     15s<br><br>NAME             ENDPOINTS           AGE<br>endpoints/demo   10.136.179.129:80   15s</pre><p>Let’s try connecting to our NGINX application by running port-forward like so,</p><pre>$ k port-forward svc/demo 8080:80 -n demo-ns<br>Forwarding from 127.0.0.1:8080 -&gt; 80<br>Forwarding from [::1]:8080 -&gt; 80<br>Handling connection for 8080</pre><p>And then in a separate terminal let’s run a curl command,</p><pre>$ curl localhost:8080<br>&lt;!DOCTYPE html&gt;<br>&lt;html&gt;<br>&lt;head&gt;<br>&lt;title&gt;Welcome to nginx!&lt;/title&gt;<br>&lt;style&gt;<br>html { color-scheme: light dark; }<br>body { width: 35em; margin: 0 auto;<br>font-family: Tahoma, Verdana, Arial, sans-serif; }<br>&lt;/style&gt;<br>&lt;/head&gt;<br>&lt;body&gt;<br>&lt;h1&gt;Welcome to nginx!&lt;/h1&gt;<br>&lt;p&gt;If you see this page, the nginx web server is successfully installed and<br>working. Further configuration is required.&lt;/p&gt;<br><br>&lt;p&gt;For online documentation and support please refer to<br>&lt;a href=&quot;http://nginx.org/&quot;&gt;nginx.org&lt;/a&gt;.&lt;br/&gt;<br>Commercial support is available at<br>&lt;a href=&quot;http://nginx.com/&quot;&gt;nginx.com&lt;/a&gt;.&lt;/p&gt;<br><br>&lt;p&gt;&lt;em&gt;Thank you for using nginx.&lt;/em&gt;&lt;/p&gt;<br>&lt;/body&gt;<br>&lt;/html&gt;</pre><p>Sweet!<br>We are ready to set-up the HTTP scaling for our demo application.</p><h4>Set-up HTTP scaling</h4><p>To set-up scaling we need to create a HTTPScaledObject for our demo application. Copying this <a href="https://github.com/kedacore/http-add-on/blob/main/examples/v0.3.0/httpscaledobject.yaml">example</a> from the documentation and updating it to our needs we have,</p><pre>kind: HTTPScaledObject<br>apiVersion: http.keda.sh/v1alpha1<br>metadata:<br>    name: demo<br>    namespace: demo-ns<br>spec:<br>    host: myhost.com<br>    scaleTargetRef:<br>        deployment: demo<br>        service: demo<br>        port: 80<br>    replicas:<br>        min: 0     # We want to scale to zero<br>        max: 10</pre><p>Let’s save the manifest above to a file and then let’s apply it like so,</p><pre># Apply manifest<br>$ k apply -f demo-http-scaled-object.yaml<br>httpscaledobject.http.keda.sh/demo created<br><br># Double-check<br>$ k get HTTPScaledObject -n demo-ns<br>NAME   SCALETARGETDEPLOYMENTNAME   SCALETARGETSERVICENAME                               SCALETARGETPORT   MINREPLICAS   MAXREPLICAS   AGE   ACTIVE<br>demo                               {&quot;deployment&quot;:&quot;demo&quot;,&quot;port&quot;:80,&quot;service&quot;:&quot;demo&quot;}                       0             10            6s<br><br># Check created HPA<br>$ k get hpa -n demo-ns<br>NAME                REFERENCE         TARGETS               MINPODS   MAXPODS   REPLICAS   AGE<br>keda-hpa-demo-app   Deployment/demo   0/100 (avg)           1         10        5          58s<br><br># Check the pods<br>$ kubectl get pods<br>No resources found in demo-ns namespace.</pre><p>After applying the manifest, the HTTP Add-on operator detected it and created the necessary infrastructure. You can see it by checking the operator logs like so,</p><pre>$ kubectl logs -nkeda keda-add-ons-http-controller-manager-5c8b9696d5-x22lm -c keda-add-ons-http-operator<br>2023-07-24T03:39:42Z INFO controllers.HTTPScaledObject Reconciliation start {&quot;HTTPScaledObject.Namespace&quot;: &quot;demo-ns&quot;, &quot;HTTPScaledObject.Name&quot;: &quot;demo&quot;}<br>2023-07-24T03:39:42Z INFO controllers.HTTPScaledObject Adding Finalizer for the ScaledObject {&quot;HTTPScaledObject.Namespace&quot;: &quot;demo-ns&quot;, &quot;HTTPScaledObject.Name&quot;: &quot;demo&quot;}<br>2023-07-24T03:39:42Z INFO controllers.HTTPScaledObject Reconciling HTTPScaledObject {&quot;HTTPScaledObject.Namespace&quot;: &quot;demo-ns&quot;, &quot;HTTPScaledObject.Name&quot;: &quot;demo&quot;, &quot;Namespace&quot;: &quot;demo-ns&quot;, &quot;DeploymentName&quot;: &quot;demo&quot;}<br>2023-07-24T03:39:42Z INFO controllers.HTTPScaledObject Creating scaled objects {&quot;HTTPScaledObject.Namespace&quot;: &quot;demo-ns&quot;, &quot;HTTPScaledObject.Name&quot;: &quot;demo&quot;, &quot;reconciler.appObjects&quot;: &quot;addObjects&quot;, &quot;HTTPScaledObject.name&quot;: &quot;demo&quot;, &quot;HTTPScaledObject.namespace&quot;: &quot;demo-ns&quot;, &quot;external scaler host name&quot;: &quot;keda-add-ons-http-external-scaler.keda:9090&quot;}<br>2023-07-24T03:39:42Z INFO controllers.HTTPScaledObject Creating App ScaledObject {&quot;HTTPScaledObject.Namespace&quot;: &quot;demo-ns&quot;, &quot;HTTPScaledObject.Name&quot;: &quot;demo&quot;, &quot;reconciler.appObjects&quot;: &quot;addObjects&quot;, &quot;HTTPScaledObject.name&quot;: &quot;demo&quot;, &quot;HTTPScaledObject.namespace&quot;: &quot;demo-ns&quot;, &quot;ScaledObject&quot;: {&quot;Object&quot;:{&quot;apiVersion&quot;:&quot;keda.sh/v1alpha1&quot;,&quot;kind&quot;:&quot;ScaledObject&quot;,&quot;metadata&quot;:{&quot;labels&quot;:{&quot;app&quot;:&quot;kedahttp-demo-app&quot;,&quot;name&quot;:&quot;demo-app&quot;},&quot;name&quot;:&quot;demo-app&quot;,&quot;namespace&quot;:&quot;demo-ns&quot;},&quot;spec&quot;:{&quot;advanced&quot;:{&quot;restoreToOriginalReplicaCount&quot;:true},&quot;maxReplicaCount&quot;:10,&quot;minReplicaCount&quot;:5,&quot;pollingInterval&quot;:1,&quot;scaleTargetRef&quot;:{&quot;kind&quot;:&quot;Deployment&quot;,&quot;name&quot;:&quot;demo&quot;},&quot;triggers&quot;:[{&quot;metadata&quot;:{&quot;host&quot;:&quot;myhost.com&quot;,&quot;scalerAddress&quot;:&quot;keda-add-ons-http-external-scaler.keda:9090&quot;},&quot;type&quot;:&quot;external-push&quot;}]}}}}<br>...</pre><p>An HPA was also created and if you noticed, there is no longer a running pod because KEDA has deactivated the deployment because there are no incoming HTTP requests.</p><p>Cool! Our demo NGINX application is ready for scaling, let’s trigger scaling by connecting to it!</p><h4>Test scaling</h4><p>So far, we’ve created and deployed our NGINX application and set-up autoscaling by creating a HTTPScaledObject for it. Let’s go back to the three scenarios we mentioned earlier.</p><p><strong>Scenario 1: Using a <em>ClusterIP</em> </strong><strong>Service only</strong></p><p>When we installed the HTTP Add-on, a Service called keda-add-ons-http-interceptor-proxy was created. To test scaling we need to route traffic to this Service. This Service has Endpoints pointing towards the interceptors</p><pre>...  <br>ports:<br>  - name: http<br>    port: 8080<br>    protocol: TCP<br>    targetPort: inter-proxy<br>  selector:<br>    control-plane: interceptor<br>...</pre><p>which then forwards the traffic to the right application. But how does it know which application to send the traffic to?</p><p>If you remember, our HTTPScaledObject has this spec below.</p><pre>...<br>spec:<br>    host: myhost.com<br>    scaleTargetRef:<br>        deployment: demo<br>        service: demo<br>        port: 80<br>...</pre><p>From this spec, it knows that traffic with the host <em>myhost.com</em> needs to route to the Endpoints under the demo Service.</p><p>So far our setup looks like the diagram below.</p><figure><img alt="Scaling using Service only" src="https://cdn-images-1.medium.com/max/1024/1*uCaR1Szq2rAxRI6OT35m3w.jpeg" /></figure><p>Right let’s trigger scaling by first creating a connection locally through kubectl port-forward :</p><pre>$ kubectl port-forward svc/keda-add-ons-http-interceptor-proxy 8080:8080 -nkeda<br>Forwarding from 127.0.0.1:8080 -&gt; 8080<br>Forwarding from [::1]:8080 -&gt; 8080<br>Handling connection for 8080</pre><p>On a separate terminal let’s create a watch for our NGINX pods by running kubectl get pods -w.</p><p>On another terminal run let’s run a curl command specifying the host header.</p><pre>$ curl localhost:8080 -H &#39;Host: myhost.com&#39;<br>&lt;!DOCTYPE html&gt;<br>&lt;html&gt;<br>&lt;head&gt;<br>&lt;title&gt;Welcome to nginx!&lt;/title&gt;<br>&lt;style&gt;<br>html { color-scheme: light dark; }<br>body { width: 35em; margin: 0 auto;<br>font-family: Tahoma, Verdana, Arial, sans-serif; }<br>&lt;/style&gt;<br>&lt;/head&gt;<br>&lt;body&gt;<br>&lt;h1&gt;Welcome to nginx!&lt;/h1&gt;<br>&lt;p&gt;If you see this page, the nginx web server is successfully installed and<br>working. Further configuration is required.&lt;/p&gt;<br><br>&lt;p&gt;For online documentation and support please refer to<br>&lt;a href=&quot;http://nginx.org/&quot;&gt;nginx.org&lt;/a&gt;.&lt;br/&gt;<br>Commercial support is available at<br>&lt;a href=&quot;http://nginx.com/&quot;&gt;nginx.com&lt;/a&gt;.&lt;/p&gt;<br><br>&lt;p&gt;&lt;em&gt;Thank you for using nginx.&lt;/em&gt;&lt;/p&gt;<br>&lt;/body&gt;<br>&lt;/html&gt;</pre><p>The NGINX app has scaled and responded as seen above, albeit with a slight delay. If we look back at our watch we should see the pod starting up, then running then eventually terminating.</p><pre>$ kubectl get pods -n demo-ns -w<br>NAME                    READY   STATUS              RESTARTS   AGE<br>demo-68b4b4d5bf-p6z7v   0/1     Pending             0          0s<br>demo-68b4b4d5bf-p6z7v   0/1     Pending             0          0s<br>demo-68b4b4d5bf-p6z7v   0/1     ContainerCreating   0          0s<br>demo-68b4b4d5bf-p6z7v   1/1     Running             0          3s<br>demo-68b4b4d5bf-p6z7v   1/1     Terminating         0          6m3s<br>demo-68b4b4d5bf-p6z7v   0/1     Terminating         0          6m3s<br>demo-68b4b4d5bf-p6z7v   0/1     Terminating         0          6m4s<br>demo-68b4b4d5bf-p6z7v   0/1     Terminating         0          6m4s</pre><p>So what just happened?</p><p>As mentioned earlier, when we routed traffic to the HTTP Add-on interceptor Service, the interceptor keeps track of the number of pending HTTP requests. The external scaler periodically checks the size of this pending queue and reports scaling metrics as appropriate to KEDA. As the queue size increases or decreases, the scaler instructs KEDA to scale up or down respectively. In our testing above, a single request is pending — enough for KEDA to scale the deployment to a single replica.</p><p>While we are sending request to our application through curl earlier, we can also check the status of the interceptor’s pending HTTP request queue. To do that, let’s start a proxy like so,</p><pre>$ kubectl proxy -p 8002<br>Starting to serve on 127.0.0.1:8002</pre><p>On a separate terminal, let’s get the status of the queue like so</p><pre>$ curl -L localhost:8002/api/v1/namespaces/keda/services/keda-add-ons-http-interceptor-admin:9090/proxy/queue<br>{&quot;myhost.com&quot;:1}</pre><p>We can see that there is one pending request bound to <em>myhost.com.</em></p><p><strong>Scenario 2: Scale using an </strong><strong>Ingress and Ingress Controller</strong></p><p>The interceptor Service keda-add-ons-http-interceptor-proxy is of type <em>ClusterIP</em>, so it’s not accessible outside of our Kubernetes cluster. While we can access it using the kubectl port-forward command as seen in the first scenario, this is not recommended in production settings. To solve this, we can use an <a href="https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/">ingress controller</a> to route to the interceptor service.</p><p>One of the widely used ingress controller is <a href="https://kubernetes.github.io/ingress-nginx/">NGINX Ingress controller</a> or simply nginx-ingress. We can install it using helm like so,</p><pre>$  helm upgrade --install ingress-nginx ingress-nginx \<br> --repo https://kubernetes.github.io/ingress-nginx \<br> --namespace ingress-nginx --create-namespace<br>Release &quot;ingress-nginx&quot; does not exist. Installing it now.<br>NAME: ingress-nginx<br>LAST DEPLOYED: Thu Aug  3 08:48:00 2023<br>NAMESPACE: ingress-nginx<br>STATUS: deployed<br>REVISION: 1<br>TEST SUITE: None<br>NOTES:<br>The ingress-nginx controller has been installed.<br>It may take a few minutes for the LoadBalancer IP to be available.<br>You can watch the status by running &#39;kubectl --namespace ingress-nginx get services -o wide -w ingress-nginx-controller&#39;<br>...<br><br>$ kubectl get pods -n ingress-nginx<br>NAME                                       READY   STATUS    RESTARTS   AGE<br>ingress-nginx-controller-7687f9d45-dbbbl   1/1     Running   0          89s</pre><p>Now that we have nginx-ingress installed, let’s create an <em>Ingress </em>manifest for our demo application like so,</p><pre>apiVersion: networking.k8s.io/v1<br>kind: Ingress<br>metadata:<br> name: demo<br> namespace: keda # &lt;-- namespace where the Add-On is installed<br>spec:<br> ingressClassName: nginx<br> rules:<br> - host: myhost.com<br>   http:<br>     paths:<br>     - path: /<br>       pathType: Prefix<br>       backend:<br>         service:<br>           name: keda-add-ons-http-interceptor-proxy # &lt;-- interceptor service<br>           port:<br>             number: 8080</pre><p>Note the following important details:</p><ul><li>This <em>Ingress </em>resource must be created on the same Namespace where we installed the HTTP Add-on components, in our case the <em>keda</em> Namespace. This is not ideal because we would want to put all of the resources of our demo application on the same Namespace, the <em>demo-ns</em>. On the next scenario we’ll solve this limitation.</li><li>The backend service points to the Add-ons interceptors Service</li></ul><p>Let’s save the above manifest to a file called <em>ingress.yaml</em> and apply it using kubectl like so kubectl apply -f ingress.yaml</p><pre>$ kubectl apply -f ingress.yaml<br>ingress.networking.k8s.io/demo created<br><br>$ kubectl get ingress -n keda<br>NAME   CLASS   HOSTS          ADDRESS                                                                              PORTS     AGE<br>demo   nginx   myhost.com     internal-ab469z87hd65sf308b561ej76533e2cdummy-12345678.us-west-2.elb.amazonaws.com   80        35s</pre><p>The ADDRESS we get from above is the IP address that our demo application will be running at on the public internet. Because we are using an ingress controller, this is the IP of the ingress controller’s Service . To be more specific, this is the address of the load balancer sitting in front of our nginx-ingress controller because we are using AWS.</p><blockquote><em>Note: If you haven’t yet, it is recommended to manage your DNS records and set-up TLS certificates. External-DNS and Cert-Manager can be used for these respectively.</em></blockquote><p>Our current setup now looks like the diagram below.</p><figure><img alt="Scaling using ingress" src="https://cdn-images-1.medium.com/max/1024/1*U0Jm2xe73ozJq4aS8ZMRtQ.jpeg" /></figure><p>Right, now let’s try scaling our application by sending HTTP request to it simply by using curl specifying the Host header. This is needed to tell the interceptor how to route the request. If we have a DNS name setup for the IP, we don’t need this header.</p><pre>$ curl -H &quot;Host: myhost.com&quot; internal-ab469z87hd65sf308b561ej76533e2cdummy-12345678.us-west-2.elb.amazonaws.com</pre><p>Because we have a DNS name set-up for the IP using External-DNS, we don’t need to set the header. Let’s send a request like so,</p><pre>$ curl myhost.com<br>&lt;!DOCTYPE html&gt;<br>&lt;html&gt;<br>&lt;head&gt;<br>&lt;title&gt;Welcome to nginx!&lt;/title&gt;<br>&lt;style&gt;<br>html { color-scheme: light dark; }<br>body { width: 35em; margin: 0 auto;<br>font-family: Tahoma, Verdana, Arial, sans-serif; }<br>&lt;/style&gt;<br>&lt;/head&gt;<br>&lt;body&gt;<br>&lt;h1&gt;Welcome to nginx!&lt;/h1&gt;<br>&lt;p&gt;If you see this page, the nginx web server is successfully installed and<br>working. Further configuration is required.&lt;/p&gt;<br><br>&lt;p&gt;For online documentation and support please refer to<br>&lt;a href=&quot;http://nginx.org/&quot;&gt;nginx.org&lt;/a&gt;.&lt;br/&gt;<br>Commercial support is available at<br>&lt;a href=&quot;http://nginx.com/&quot;&gt;nginx.com&lt;/a&gt;.&lt;/p&gt;<br><br>&lt;p&gt;&lt;em&gt;Thank you for using nginx.&lt;/em&gt;&lt;/p&gt;<br>&lt;/body&gt;<br>&lt;/html&gt;</pre><p>Awesome! We got a response from our demo application!</p><p>A watch on our demo namespace shows the pod starting up in response to the scaling</p><pre>$ kubectl get pods -n demo-ns -w<br>NAME                    READY   STATUS              RESTARTS   AGE<br>demo-7fd9757bb7-s4rtl   0/1     ContainerCreating   0          1s<br>demo-7fd9757bb7-s4rtl   1/1     Running             0          8s</pre><p>So to recap, we sent a request to our Ingress host which forwarded the request to the interceptors Service which then forwarded the traffic to our demo applications Service. Behind the scenes, the request was routed to the right destination through IP tables managed by kube-proxy but this is out of scope for this blog.</p><p><strong>Scenario 3: Scale </strong>using an Ingress, NGINX Ingress Controller and <em>ExternalName</em> Service</p><p>In the previous scenario, we had to create our demo application’s Ingress on the same Namespace where we installed the HTTP Add-on components, which in our case is the <em>keda</em> Namespace . What if we want to create our Ingress resource inside the <em>demo-ns</em> together with the rest of our demo application’s resources? The problem here, is that Kubernetes networking does not allow for an Ingress resource to point to another Service on another Namespace . This means we cannot set the service backend of our Ingress resource in <em>demo-ns</em> to point to the interceptor’s Service in <em>keda</em> Namespace. We can, however point a Service to another Service on a different Namespace which is what we are going to do.</p><p>First, let’s create a Service of type ExternalName . An <a href="https://kubernetes.io/docs/concepts/services-networking/service/#externalname">ExternalName Service</a> is a special type of Service which instead of the typical selectors maps a Service to a DNS name.</p><pre>apiVersion: v1<br>kind: Service<br>metadata:<br>  name: demo-externalname<br>  namespace: demo-ns<br>spec:<br>  type: ExternalName<br>  externalName: keda-add-ons-http-interceptor-proxy.keda.svc.cluster.local # &lt;-- interceptor service</pre><p>The Service above maps to the interceptors Service DNS name. Let’s save the above manifest to a file called <em>demo-externalname.yaml</em> and apply it using kubectl like so kubectl apply -f demo-externalname.yaml</p><pre>$ kubectl apply -f demo-externalname.yaml<br>service/demo-externalname created<br><br>$ kubectl get service -n demo-ns<br>NAME                TYPE           CLUSTER-IP       EXTERNAL-IP                                                  PORT(S)    AGE<br>demo                ClusterIP      172.20.121.195   &lt;none&gt;                                                       80/TCP     164m<br>demo-externalname   ExternalName   &lt;none&gt;           keda-add-ons-http-interceptor-proxy.keda.svc.cluster.local   &lt;none&gt;     25s</pre><p>Notice that we now have two Services. Next, let’s delete our current Ingress in the <em>keda</em> Namespace .</p><pre>$ kubectl delete ingress -n keda<br>ingress.networking.k8s.io &quot;demo&quot; deleted</pre><p>Next let’s update our Ingress manifest like so,</p><pre>apiVersion: networking.k8s.io/v1<br>kind: Ingress<br>metadata:<br> name: demo<br> namespace: demo-ns # &lt;-- change namespace to demo-ns<br>spec:<br> ingressClassName: nginx<br> rules:<br> - host: myhost.com<br>   http:<br>     paths:<br>     - path: /<br>       pathType: Prefix<br>       backend:<br>         service:<br>           name: demo-externalname # &lt;-- external-name service in demo-ns<br>           port:<br>             number: 8080 # &lt;-- port of the interceptor service</pre><p>Noticed that we are creating our Ingress in our <em>demo-ns</em> and we are updating the service backend to our ExternalName Service . The port remains the same. Our HTTPScaledObject also remains the same. Let’s create our new Ingress like so,</p><pre>$ kubectl apply -f ingress.yaml<br>ingress.networking.k8s.io/demo created</pre><p>With the changes above our setup now looks like below.</p><figure><img alt="Scaling using ingress and service externalname" src="https://cdn-images-1.medium.com/max/1024/1*i2cPhKUYsUELlo1Y13UmaQ.jpeg" /></figure><p>Ol’ right, time to test our changes! Let’s send a request to our demo application!</p><pre>$ curl myhost.com<br>&lt;!DOCTYPE html&gt;<br>&lt;html&gt;<br>&lt;head&gt;<br>&lt;title&gt;Welcome to nginx!&lt;/title&gt;<br>&lt;style&gt;<br>html { color-scheme: light dark; }<br>body { width: 35em; margin: 0 auto;<br>font-family: Tahoma, Verdana, Arial, sans-serif; }<br>&lt;/style&gt;<br>&lt;/head&gt;<br>&lt;body&gt;<br>&lt;h1&gt;Welcome to nginx!&lt;/h1&gt;<br>&lt;p&gt;If you see this page, the nginx web server is successfully installed and<br>working. Further configuration is required.&lt;/p&gt;<br><br>&lt;p&gt;For online documentation and support please refer to<br>&lt;a href=&quot;http://nginx.org/&quot;&gt;nginx.org&lt;/a&gt;.&lt;br/&gt;<br>Commercial support is available at<br>&lt;a href=&quot;http://nginx.com/&quot;&gt;nginx.com&lt;/a&gt;.&lt;/p&gt;<br><br>&lt;p&gt;&lt;em&gt;Thank you for using nginx.&lt;/em&gt;&lt;/p&gt;<br>&lt;/body&gt;<br>&lt;/html&gt;</pre><p>Fantastic! Our demo application has responded!</p><p>Let’s check the pods that got created in response to the scaling triggered by our request,</p><pre>$ kubectl get pods -n demo-ns -w<br>NAME                    READY   STATUS    RESTARTS   AGE<br>demo-7fd9757bb7-zf6kx   0/1     Pending   0          0s<br>demo-7fd9757bb7-zf6kx   0/1     Pending   0          0s<br>demo-7fd9757bb7-zf6kx   0/1     ContainerCreating   0          0s<br>demo-7fd9757bb7-zf6kx   1/1     Running             0          2s<br>demo-7fd9757bb7-zf6kx   1/1     Terminating         0          5m39s<br>demo-7fd9757bb7-zf6kx   0/1     Terminating         0          5m39s<br>demo-7fd9757bb7-zf6kx   0/1     Terminating         0          5m39s<br>demo-7fd9757bb7-zf6kx   0/1     Terminating         0          5m39s</pre><p>Sweet as!</p><p>To recap, we created a new Service of type ExternalName and pointed that to the interceptors Service. We then recreated our Ingress to our <em>demo-ns</em> and updated it to point to our ExternalName Service.</p><p>Coolbeans! That’s it for this blog! I hope this has been useful and that you learned something! Cheers!</p><h3>Have a question or want to reach out for a yarn?</h3><p>If you have some questions, feedback or just want to reach out for a good ol’ yarn, please connect and flick me a message at <a href="https://www.linkedin.com/in/carlo-columna/">https://www.linkedin.com/in/carlo-columna/</a>.</p><h3>Like what you’ve read?</h3><p>If you liked what you read, or if you found it useful, please follow me below for more articles and to get notified when I published new ones. Cheers!</p><h3>References:</h3><ul><li><a href="https://github.com/kedacore/http-add-on/tree/main">https://github.com/kedacore/http-add-on/tree/main</a></li><li><a href="https://www.linode.com/blog/kubernetes/scaling-kubernetes-to-zero-and-back/">https://www.linode.com/blog/kubernetes/scaling-kubernetes-to-zero-and-back/</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d58221d5e7f1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Scale Your Pods Based on a Queue]]></title>
            <link>https://medium.com/@carlocolumna/keda-in-amazon-eks-part-2-scale-based-on-aws-sqs-queue-1dbd13fa5384?source=rss-d9219c5b4599------2</link>
            <guid isPermaLink="false">https://medium.com/p/1dbd13fa5384</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[containers]]></category>
            <category><![CDATA[autoscaling]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[keda]]></category>
            <dc:creator><![CDATA[Carlo Columna]]></dc:creator>
            <pubDate>Mon, 26 Jun 2023 06:00:32 GMT</pubDate>
            <atom:updated>2023-08-16T23:44:07.782Z</atom:updated>
            <content:encoded><![CDATA[<p>Kia ora everyone!</p><p>Welcome to the second part of a blog series discussing autoscaling capabilities in Kubernetes using <a href="https://keda.sh/">KEDA or Kubernetes Event-driven Autoscaling</a>. If you haven’t, I highly recommend checking out the first part of the series below to learn why and how to install KEDA in your Kubernetes cluster.</p><ul><li><a href="https://medium.com/@carlocolumna/eda-in-amazon-eks-install-keda-82849cf31f01">Part 1: How to Level Up Your Kubernetes Scaling</a></li></ul><p>If you are looking to scale based on HTTP traffic, check out part 3 below:</p><ul><li><a href="https://medium.com/@carlocolumna/how-to-scale-your-pods-based-on-http-traffic-d58221d5e7f1">Part 3: How to Scale Your Pods Based on HTTP Traffic</a></li></ul><p>This part two of this blog series focuses on scaling based on a queue. We will be scaling a workload running in an Amazon EKS cluster based on an AWS SQS Queue.</p><figure><img alt="Meme when the queue jumps and there’s only one pod" src="https://cdn-images-1.medium.com/max/1024/1*CSZSerMpjEpMMcLz7qTZ9g.png" /></figure><p>The structure of this blog would be as follows:</p><ol><li>Prerequisites</li><li>Create a demo application</li><li>Scale based on two identity owner models</li></ol><ul><li><strong>Important</strong>: We’ll also go deeper in explaining the two models which are not extensively covered in the KEDA documentation.</li></ul><p>4. Common errors</p><h3>Prerequisites</h3><p>Before we can proceed, let&#39;s lay out the prerequisites:</p><ul><li>AWS account</li><li>Access to an Amazon EKS cluster</li><li>IAM roles for service accounts (IRSA) setup for the EKS cluster. See <a href="https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html">here</a> for more details.</li><li>KEDA installed in the EKS cluster, see <a href="https://medium.com/@carlocolumna/eda-in-amazon-eks-install-keda-82849cf31f01">part 1</a></li><li>Terminal with kubectl</li></ul><h3>Create a demo application</h3><p>We’ll use a demo application and set it up across different accounts to best illustrate the steps we’ll go through. We will be using Terraform and Helm to deploy it.</p><p>Our demo application has the following requirements:</p><ul><li>It is a simple NGINX application deployed in an EKS cluster in Account A in Oregon.</li><li>It has an IAM Role created in Account C in Canada Central.</li><li>KEDA is deployed and running on the same EKS cluster using an IAM Role created in Account B in N. Virginia.</li><li>It scales based on a queue in Account C in Canada Central.</li></ul><p>To better understand the scenario, the following is the architecture diagram for KEDA and our demo application.</p><figure><img alt="KEDA and Demo Application Architecture" src="https://cdn-images-1.medium.com/max/1024/1*JX8RVFugwD5sTfwZ2h4FTg.jpeg" /><figcaption><strong>KEDA and Demo Application Architecture</strong></figcaption></figure><p>Before we can install our demo application, we’ll start by creating an IAM Role for it using Terraform, like so,</p><pre>resource &quot;aws_iam_role&quot; &quot;demo-app&quot; {<br>  provider           = aws.account_c<br>  name               = &quot;demo-app&quot;<br>  assume_role_policy = module.demo-app-trust-policy.json<br>}</pre><p>For now, we are creating a role with no permissions. Later on, we’ll update the role by granting it permission to access AWS resources.</p><p>We also set up the role’s trust policy so our demo application can assume the role.</p><pre>resource &quot;aws_iam_role&quot; &quot;demo-app&quot; { ... }<br><br>data &quot;aws_iam_policy_document&quot; &quot;demo-app-trust-policy&quot; {<br>  statement {<br>    actions = [<br>      &quot;sts:AssumeRoleWithWebIdentity&quot;<br>    ]<br>    principals {<br>      type = &quot;Federated&quot;<br>      identifiers = [&quot;arn:aws:iam::111122223333:oidc-provider/oidc.eks.region-code.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE&quot;]<br>    }<br>    condition {<br>      test     = &quot;StringEquals&quot;<br>      variable = &quot;oidc.eks.region-code.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE:sub&quot;<br>      values   = [&quot;system:serviceaccount:demo-ns:demo-app&quot;]<br>    }<br>  }<br>}</pre><p>Run terraform apply.</p><p>Now, let’s create the Helm manifest of our basic deployment using the NGINX image.</p><pre>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  labels:<br>    app: demo-app<br>  name: demo-app<br>  namespace: demo-ns<br>spec:<br>  replicas: 1<br>  selector:<br>    matchLabels:<br>      app: demo-app<br>  template:<br>    metadata:<br>      labels:<br>        app: demo-app<br>    spec:<br>      serviceAccountName: demo-app-sa<br>      containers:<br>      - image: nginx<br>        name: nginx<br>---<br>apiVersion: v1<br>kind: ServiceAccount<br>metadata:<br>  name: demo-app-sa<br>  namespace: demo-ns<br>  annotations:<br>    eks.amazonaws.com/role-arn: arn:aws:iam::111122223333:role/demo-app # demo-app IAM role ARN created from the previous step </pre><p>This creates a deployment with a <em>ServiceAccount</em>. If you noticed, we have added the IAM role we created above as an annotation so our application can access AWS resources. Let’s save this file as demo-app.yaml .</p><p>To deploy, ideally, you package this manifest into a Helm chart or use kustomize. But to keep things short and simple in this blog, let’s simply run a kubectl apply -f demo-app.yaml.</p><h3>Scaling with KEDA</h3><p>We’ve come to the more exciting part of this blog, setting up and testing the scaling using AWS SQS Queue. Before we can start, there is one important security consideration we have to make.</p><p>There are two ways that KEDA can get permission to access the SQS Queue and scale a workload. This is determined by setting theidentityOwner to either pod which is the default, or operator.</p><ul><li>pod : This sets the <em>keda-operator </em>to temporarily assume the role of the demo application. This means that the demo application role should have the necessary permissions to access the SQS Queue. We’ll call this the Pod Identity Owner model</li><li>operator : Grant the <em>keda-operator </em>direct access to the AWS SQS queue. We’ll call this the Operator Identity Owner model.</li></ul><p>Note that in this blog, when talking about permissions and IAM, the KEDAs role is the same as the <em>keda-operators </em>IAM role.</p><h4><strong>What model should you use?</strong></h4><p>This depends on what works best in your setup. Let’s summarise the pros and cons of each model.</p><p><strong>Pod Identity Owner model</strong></p><ul><li><strong>Good</strong>, because we don’t have to know beforehand what privileges we need to give KEDA as long as our application has already access to the event source.</li><li><strong>Good</strong>, in cases where KEDA cannot be given direct access to the event source</li><li><strong>Bad</strong>, because KEDA gets all the permission of all workload that uses this permission model. This probably includes permissions that are not required for scaling purposes (such as write privilege) and thus breaking the principle of least privilege.</li><li><strong>Bad</strong>, because it requires a bit more complicated setup</li></ul><p><strong>Operator Identity Owner model</strong></p><ul><li><strong>Good</strong>, because KEDA only gets the permission it needs to trigger scaling so it follows the principle of least privilege</li><li><strong>Good</strong>, because it’s a simpler setup</li><li><strong>Bad</strong>, because it requires more initial understanding what are the privileges that we need to grant KEDA to access an events source</li><li><strong>Bad</strong>, in cases where the event source cannot be updated or set to be accessed by KEDA (e.g. if access policy cannot be set to the event source)</li></ul><p>From a security and simplicity perspective, the Operator Identity Owner model is a better option. This is more so if you have an application that has access to sensitive information that you don’t want <em>keda-operator </em>to get access to. However, there might be cases where you are restricted to using the Pod Identity Owner model. In these cases, you just have to be aware that the<em>keda-operator</em> gets all the permissions of the application role.</p><p>In this blog, we’ll show how to scale each identity owner model starting with the default identityOwner: pod.</p><h4>Pod Identity Owner Model</h4><p>For this model to work it’s important to understand the requirements:</p><ul><li>We need to add an <em>AssumeRole</em> policy to the demo application roles trust policy to allow keda-operator to assume it</li><li>The demo application role has the permission to access the queue</li><li>We need to add an <em>AssumeRole</em> policy to thekeda-operator role so it knows what role to assume</li></ul><p>Going back to our demo applications architecture and incorporating the use of the Pod Identity Owner model, the architecture will look like below.</p><figure><img alt="KEDA and Demo Application: Pod Identity Owner Model" src="https://cdn-images-1.medium.com/max/1024/1*LNHIUw1y5xi4VnKKw7fHag.jpeg" /><figcaption><strong>KEDA and Demo Application: Pod Identity Owner Model</strong></figcaption></figure><p>In this model, the demo application through its IAM role has access to the AWS SQS Queue. Meanwhile, KEDA can temporarily assume the demo applications role to get access to the same AWS SQS Queue to trigger scaling.</p><p>Here is the summary of the steps we’ll go through.</p><ol><li>Create AWS SQS Queue</li><li>Update the IAM Role of the demo application</li><li>Update the IAM Role of KEDA</li><li>Create scaler</li><li>Test scaling</li></ol><p><strong>Steps</strong></p><ol><li><strong>Create AWS SQS Queue</strong></li></ol><p>Using Terraform we can simply create a queue like so,</p><pre>resource &quot;aws_sqs_queue&quot; &quot;aws_sqs&quot; {<br>  provider = aws.account_c<br>  name     = &quot;demo-app-sqs&quot;<br>}<br><br>output &quot;demo_app_sqs_url&quot; {<br>  value = aws_sqs_queue.aws_sqs.url<br>}</pre><p>Run terraform apply.</p><p><strong>2. Update the IAM Role of the demo application</strong></p><p>Now, let’s update the role of our demo application by doing two things:</p><ul><li>Grant our demo application permission to access the queue</li><li>Add a policy to allow KEDA to assume it</li></ul><p>For the first one, let’s add permission to access the queue created in the previous step like so,</p><pre>resource &quot;aws_iam_role&quot; &quot;demo-app&quot; { ... }</pre><pre>data &quot;aws_iam_policy_document&quot; &quot;demo-app-trust-policy&quot; { ... }</pre><pre>data &quot;aws_iam_policy_document&quot; &quot;sqs-policy-document&quot; {<br>  statement {<br>    sid    = &quot;SQS&quot;<br>    effect = &quot;Allow&quot;<br>    actions = [<br>      &quot;sqs:GetQueueAttributes&quot;, #Add IAM action to the demo-app role<br>    ]<br>    resources = [<br>      aws_sqs_queue.aws_sqs.arn,<br>    ]<br>  }<br>}</pre><pre>resource &quot;aws_iam_role_policy&quot; &quot;sqs-policy&quot; {<br>  depends_on = [<br>    aws_sqs_queue.aws_sqs<br>  ]</pre><pre>  provider = aws.account_c<br>  name     = &quot;demo-app-sqs-policy&quot;<br>  role     = aws_iam_role.demo-app.id<br>  policy   = data.aws_iam_policy_document.sqs-policy-document.json<br>}</pre><p>For the second one, let’s add an <em>sts:AssumeRole</em> policy to our demo applications trust policy passing in the keda-operators role like so,</p><pre>resource &quot;aws_iam_role&quot; &quot;demo-app&quot; { ... }<br><br>data &quot;aws_iam_policy_document&quot; &quot;demo-app-trust-policy&quot; {<br><br>  statement {<br>    sid    = &quot;AssumeRoleKedaOperator&quot;<br>    effect = &quot;Allow&quot;<br>    actions = [<br>      &quot;sts:AssumeRole&quot;,<br>    ]<br>    principals {<br>      type        = &quot;AWS&quot;<br>      identifiers = var.keda_operator_role # &quot;arn:aws:iam::111122223333:role/keda-operator&quot;<br>    }<br>  }<br><br>  statement {<br>    actions = [<br>      &quot;sts:AssumeRoleWithWebIdentity&quot;<br>    ]<br>    principals {<br>      type = &quot;Federated&quot;<br>      identifiers = [&quot;arn:aws:iam::111122223333:oidc-provider/oidc.eks.region-code.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE&quot;]<br>    }<br>    condition {<br>      test     = &quot;StringEquals&quot;<br>      variable = &quot;oidc.eks.region-code.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE:sub&quot;<br>      values   = [&quot;system:serviceaccount:demo-ns:demo-app&quot;]<br>    }<br>  }<br>}</pre><p>Run terraform apply.</p><p><strong>3. Update the IAM Role of KEDA</strong></p><p>Let’s update the role of keda-operator so it knows what role to assume by adding an <em>sts:AssumeRole </em>policy passing in our demo application’s role.</p><pre>resource &quot;aws_iam_role&quot; &quot;keda-operator&quot; { ... }<br><br>data &quot;aws_iam_policy_document&quot; &quot;keda-operator-trust-policy&quot; { ... }<br><br>data &quot;aws_iam_policy_document&quot; &quot;assume-role-policy-document&quot; {<br>  statement {<br>    sid    = &quot;sts&quot;<br>    effect = &quot;Allow&quot;<br>    actions = [<br>      &quot;sts:AssumeRole&quot;,<br>    ]<br>    resources = var.application_roles_list # [&quot;arn:aws:iam::111122223333:role/demo-app&quot;]<br>  }<br>}<br><br>resource &quot;aws_iam_role_policy&quot; &quot;assume-role-policy&quot; {<br>  provider = aws.account_b<br>  count    = length(var.application_roles_list) == 0 ? 0 : 1 # This is needed otherwise Terraform will fail to create this resource when the role list is empty<br>  name     = &quot;keda-operator-assume-role-policy&quot;<br>  role     = aws_iam_role.keda-operator.id<br>  policy   = data.aws_iam_policy_document.assume-role-policy-document.json<br>}</pre><p>Run terraform apply.</p><p><strong>4. Create scaler</strong></p><p>Okay now, let&#39;s shift from AWS stuff to KEDA stuff. Let’s start by creating the scaler.</p><blockquote>KEDA <strong>scalers</strong> can both detect if a deployment should be activated or deactivated, and feed custom metrics for a specific event source.</blockquote><p>To create a scaler, we have to define a <em>ScaledObject</em> custom resource.</p><blockquote>The <em>ScaledObject</em> Custom Resource definition is used to define how KEDA should scale your application and what the triggers are.</blockquote><p>The <em>ScaledObject</em> resource specifications are defined <a href="https://keda.sh/docs/2.8/concepts/scaling-deployments/#scaledobject-spec">here</a>. As part of the specification, we also have to define a trigger, which in our case is an AWS SQS Queue. The specification for AWS SQS Queue can be found <a href="https://keda.sh/docs/2.8/scalers/aws-sqs/">here</a>.</p><p>Following these specifications, our scaler configuration would be,</p><pre>apiVersion: keda.sh/v1alpha1<br>kind: ScaledObject<br>metadata:<br>  name: demo-app-scaledobject<br>  namespace: demo-ns<br>spec:<br>  minReplicaCount: 0  # scale down to 0<br>  maxReplicaCount: 10 <br>  pollingInterval: 30 <br>  scaleTargetRef:<br>    name: demo-app    # name of our demo-app deployment<br>  triggers:<br>  - type: aws-sqs-queue<br>    authenticationRef:<br>      name: demo-app-trigger-auth-aws-credentials  # trigger auth object name<br>    metadata:<br>      queueURL: https://sqs.ca-central-1.amazonaws.com/111122223333/demo-app-sqs<br>      queueLength: &quot;2&quot; # 2 messages per pod <br>      awsRegion: ca-central-1<br>      identityOwner: pod # defaults to pod if unset<br>---<br>apiVersion: keda.sh/v1alpha1<br>kind: TriggerAuthentication<br>metadata:<br>  name: demo-app-trigger-auth-aws-credentials<br>  namespace: demo-ns<br>spec:<br>  podIdentity:<br>    provider: aws-eks # or aws-kiam when using kiam</pre><p>Add this configuration to our manifest file, demo-app.yaml. We then deploy these resources on the same namespace as our demo application simply by running kubectl apply -f demo-app.yaml.</p><p>Describing our <em>ScaledObject</em> by running kubectl describe ScaledObject demo-app-scaledobject -n demo-ns shows it’s active and happy.</p><pre>Status:<br>  Conditions:<br>    Message:  ScaledObject is defined correctly and is ready for scaling<br>    Reason:   ScaledObjectReady<br>    Status:   True<br>    Type:     Ready<br>    Message:  Scaling is not performed because triggers are not active<br>    Reason:   ScalerNotActive<br>    Status:   False<br>    Type:     Active<br>    Message:  No fallbacks are active on this scaled object<br>    Reason:   NoFallbackFound<br>    Status:   False<br>    Type:     Fallback<br>  External Metric Names:<br>    s0-aws-sqs-demo-app<br>  Health:<br>    s0-aws-sqs-demo-app:<br>      Number Of Failures:  0<br>      Status:              Happy<br>  Hpa Name:                keda-hpa-demo-app<br>  Last Active Time:        2023-02-16T20:18:48Z<br>  Original Replica Count:  1<br>  Scale Target GVKR:<br>    Group:            apps<br>    Kind:             Deployment<br>    Resource:         deployments<br>    Version:          v1<br>  Scale Target Kind:  apps/v1.Deployment<br>Events:               &lt;none&gt;</pre><p>Let’s check the logs of our KEDA deployments to see what they say,</p><p><em>keda-operator</em> log:</p><pre>INFO Reconciling ScaledObject {&quot;controller&quot;: &quot;scaledobject&quot;, &quot;controllerGroup&quot;: &quot;keda.sh&quot;, &quot;controllerKind&quot;: &quot;ScaledObject&quot;, &quot;scaledObject&quot;: {&quot;name&quot;:&quot;demo-app-scaledobject&quot;,&quot;namespace&quot;:&quot;demo-ns&quot;}, &quot;namespace&quot;: &quot;demo-ns&quot;, &quot;name&quot;: &quot;demo-app&quot;, &quot;reconcileID&quot;: &quot;e40b3625-6794-4358-b285-a9a57194edee&quot;}</pre><p><em>keda-operator-metrics-apiserver</em> log:</p><pre>1 trace.go:205] Trace[87882572]: &quot;List&quot; url:/apis/external.metrics.k8s.io/v1beta1/namespaces/demo-ns/s0-aws-sqs-demo-app,user-agent:kube-controller-manager/v1.22.16 (linux/amd64) kubernetes/52e500d/system:serviceaccount:kube-system:horizontal-pod-autoscaler,audit-id:1570ee53-ba50-4de8-81c7-727482dca392,client:172.16.113.69,accept:application/vnd.kubernetes.protobuf, */*,protocol:HTTP/2.0 (20-Feb-2023 05:04:01.958) (total time: 986ms):</pre><p>Both KEDA deployments acknowledge and reconciles with the new <em>ScaledObject</em> we created.</p><p>The <em>keda-operator </em>has also created an HPA for our demo application.</p><pre>$ kubectl get hpa -n demo-ns<br>NAME                REFERENCE             TARGETS             MINPODS   MAXPODS   REPLICAS   AGE<br>keda-hpa-demo-app   Deployment/demo-app   &lt;unknown&gt;/1 (avg)   1         10        0          5s</pre><p>From now on, this HPA will be managed <em>keda-operator </em>to set the number of replicas based on the metrics provided by the <em>keda-operator-metrics-apiserver .</em></p><p><strong>5. Test scaling</strong></p><p>To trigger scaling, let’s send some messages to our queue using AWS Console. Let’s send 4 messages to target 2 replicas.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bCIsP0vifr6Yn1s_Qsdd2w.png" /><figcaption>Sending messages to the queue</figcaption></figure><p>Going back to the cluster, let’s check the events from our <em>ScaledObject</em> and HPA.</p><p><em>keda-hpa-demo-app </em>events:</p><pre>Conditions:<br>  Type            Status  Reason              Message<br>  ----            ------  ------              -------<br>  AbleToScale     True    SucceededRescale    the HPA controller was able to update the target scale to 2<br>  ScalingActive   True    ValidMetricFound    the HPA was able to successfully calculate a replica count from external metric s0-aws-sqs-demo-app(&amp;LabelSelector{MatchLabels:map[string]string{scaledobject.keda.sh/name: demo-app,},MatchExpressions:[]LabelSelectorRequirement{},})<br>  ScalingLimited  False   DesiredWithinRange  the desired count is within the acceptable range<br>Events:<br>  Type    Reason             Age               From                       Message<br>  ----    ------             ----              ----                       -------<br>  Normal  SuccessfulRescale  10s (x2 over 4d)  horizontal-pod-autoscaler  New size: 2; reason: external metric s0-aws-sqs-demo-app(&amp;LabelSelector{MatchLabels:map[string]string{scaledobject.keda.sh/name: demo-app,},MatchExpressions:[]LabelSelectorRequirement{},}) above target</pre><p><em>demo-app-scaledobject </em>events:</p><pre>Events:<br>  Type    Reason                    Age               From           Message<br>  ----    ------                    ----              ----           -------<br>  Normal  KEDAScaleTargetActivated  10s (x3 over 4d)  keda-operator  Scaled apps/v1.Deployment demo-ns/demo-app from 0 to 2</pre><p>Putting a watch on the <em>demo-ns</em> shows the new pods that get created as a result of the scaling,</p><pre>$ kubectl get pods -n demo-ns -w<br>NAME                           READY   STATUS              RESTARTS   AGE<br>demo-app-674d8c455c-n7svp      0/1     Pending             0          0s<br>demo-app-674d8c455c-n7svp      0/1     Pending             0          0s<br>demo-app-674d8c455c-n7svp      0/1     ContainerCreating   0          0s<br>demo-app-674d8c455c-n7svp      1/1     Running             0          3s<br>demo-app-674d8c455c-64jvw      0/1     Pending             0          0s<br>demo-app-674d8c455c-64jvw      0/1     Pending             0          0s<br>demo-app-674d8c455c-64jvw      0/1     ContainerCreating   0          0s<br>demo-app-674d8c455c-64jvw      1/1     Running             0          2s</pre><p>Let’s trigger descaling by deleting the messages in the queue. Poll for messages, select all and delete them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xSB_gzC4LqEkVfFl9dgmdQ.png" /></figure><p>Getting the events from the <em>demo-ns</em> shows:</p><pre>$ kubectl get events -n demo-ns<br>LAST SEEN   TYPE     REASON                       OBJECT                                             MESSAGE<br>4m2s        Normal   Killing                      pod/demo-app-674d8c455c-64jvw    Stopping container demo-app<br>4m2s        Normal   Killing                      pod/demo-app-674d8c455c-n7svp    Stopping container demo-app<br>4m2s        Normal   SuccessfulDelete             replicaset/demo-app-674d8c455c   Deleted pod: demo-app-674d8c455c-n7svp<br>4m2s        Normal   SuccessfulDelete             replicaset/demo-app-674d8c455c   Deleted pod: demo-app-674d8c455c-64jvw<br>4m3s        Normal   ScalingReplicaSet            deployment/demo-app              Scaled down replica set demo-app-674d8c455c to 0<br>4m3s        Normal   KEDAScaleTargetDeactivated   scaledobject/demo-app            Deactivated apps/v1.Deployment demo-ns/demo-app from 2 to 0</pre><p>Getting the pods shows they are now in terminating state,</p><pre>$ kubectl get pods -n demo-ns -w<br>NAME                           READY   STATUS        RESTARTS   AGE<br>demo-app-674d8c455c-n7svp      1/1     Terminating   0          90m<br>demo-app-674d8c455c-64jvw      1/1     Terminating   0          90m<br>demo-app-674d8c455c-n7svp      0/1     Terminating   0          90m<br>demo-app-674d8c455c-n7svp      0/1     Terminating   0          90m<br>demo-app-674d8c455c-n7svp      0/1     Terminating   0          90m<br>demo-app-674d8c455c-64jvw      0/1     Terminating   0          90m<br>demo-app-674d8c455c-64jvw      0/1     Terminating   0          90m<br>demo-app-674d8c455c-64jvw      0/1     Terminating   0          90m</pre><p>Awesome! Great work! You made it this far.</p><p>That’s how easy to scale based on a queue using KEDA. Next, we’ll look at using the operator identity model which I personally recommend.</p><h4>Operator Identity Owner Model</h4><blockquote><em>When </em><em>identityOwner set to </em><em>operator - the only requirement is that the KEDA operator has the correct IAM permissions on the SQS queue. Additional Authentication Parameters are not required.</em></blockquote><p>Going back to our demo applications architecture and incorporating the use of the Operator Identity Owner model, the architecture will look like below.</p><figure><img alt="KEDA and Demo Application: Operator Identity Owner Model" src="https://cdn-images-1.medium.com/max/1024/1*5zo9htvvvdZR6OzzFQt7mw.jpeg" /><figcaption><strong>KEDA and Demo Application: Operator Identity Owner Model</strong></figcaption></figure><p>Let’s create a new set of resources to differentiate from the resources we created in the previous section. Here is the summary of the steps we’ll go through.</p><ol><li>Create AWS SQS Queue with access policy</li><li>Grant KEDA permissions to the AWS SQS Queue</li><li>Create second demo application</li><li>Create scaler</li><li>Test scaling</li></ol><p><strong>Steps</strong></p><ol><li><strong>Create AWS SQS Queue with access policy</strong></li></ol><p>The first step we have to do is create a queue. Using Terraform we can create it like so,</p><pre>resource &quot;aws_sqs_queue&quot; &quot;aws_sqs_operator&quot; {<br>  provider = aws.account_c<br>  name     = &quot;demo-app-sqs-operator&quot;<br>}</pre><p>An access policy is needed to explicitly allow KEDA to access certain SQS API actions. This is needed when doing cross-account access. To create the access policy we need the IAM permission to read a queue, in this case, it&#39;s <em>SQS:GetQueueAttributes, </em>it’s the only permission we need. We also need to pass the KEDA IAM role created in <a href="https://medium.com/@carlocolumna/eda-in-amazon-eks-install-keda-82849cf31f01">Part 1: Why and How to Install KEDA</a>.</p><pre>data &quot;aws_iam_policy_document&quot; &quot;sqs_access_policy_data_operator&quot; {<br>  statement {<br>    actions   = [&quot;SQS:GetQueueAttributes&quot;]<br>    resources = [ aws_sqs_queue.aws_sqs_operator.arn ]<br>    effect    = &quot;Allow&quot;<br>    principals {<br>      type        = &quot;AWS&quot;<br>      identifiers = var.keda_operator_role # &quot;arn:aws:iam::111122223333:role/keda-operator&quot;<br>    }<br>  }<br>}</pre><pre>resource &quot;aws_sqs_queue_policy&quot; &quot;sqs_access_policy_operator&quot; {<br>  provider = aws.account_c<br>  depends_on = [<br>    aws_sqs_queue.aws_sqs_operator<br>  ]<br>  queue_url = aws_sqs_queue.aws_sqs_operator.url<br>  policy    = data.aws_iam_policy_document.sqs_access_policy_data_operator.json<br>}</pre><p>Run terraform apply.</p><p><strong>2. Grant KEDA access to the AWS SQS Queue</strong></p><p>For this step, we’ll have to update our KEDA installation. We need to grant KEDA or in particular the <em>keda-operator</em> direct permission to the queue we created. We do this by adding the same policy we added on the queue access policy to the <em>keda-operator</em>role.</p><pre>resource &quot;aws_iam_role&quot; &quot;keda-operator&quot; { ... }</pre><pre>data &quot;aws_iam_policy_document&quot; &quot;keda-operator-trust-policy&quot; { ... }</pre><pre>output &quot;keda_operator_role_arn&quot; { ... }</pre><pre>resource &quot;aws_iam_role_policy&quot; &quot;sqs-policy&quot; {<br>  name     = &quot;sqs-queue-policy&quot;<br>  role     = aws_iam_role.keda-operator.id<br>  policy   = data.aws_iam_policy_document.sqs-policy-document.json</pre><pre>}</pre><pre>data &quot;aws_iam_policy_document&quot; &quot;sqs-policy-document&quot; {<br>  statement {<br>    sid    = &quot;SQS&quot;<br>    effect = &quot;Allow&quot;<br>    actions = [<br>      &quot;sqs:GetQueueAttributes&quot;,<br>    ]<br>    resources = [ &quot;arn:aws:sqs:region-code:111122223333:demo-app-sqs-operator&quot; ] # queue arn<br>  }<br>}</pre><p>Run terraform apply.</p><p><strong>3. Create second demo application</strong></p><p>Let’s deploy a second demo application so we can keep the first one and we can test both as much as we like. We would have to also create an IAM role for it. Scroll to the top to see the Terraform on how to create one.</p><p>Our second demo application will now look as follows:</p><pre>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  labels:<br>    app: demo-app-operator<br>  name: demo-app-operator<br>  namespace: demo-ns<br>spec:<br>  replicas: 1<br>  selector:<br>    matchLabels:<br>      app: demo-app-operator<br>  template:<br>    metadata:<br>      labels:<br>        app: demo-app-operator<br>    spec:<br>      serviceAccountName: demo-app-operator-sa<br>      containers:<br>      - image: nginx<br>        name: nginx<br>---<br>apiVersion: v1<br>kind: ServiceAccount<br>metadata:<br>  name: demo-app-operator-sa<br>  namespace: demo-ns<br>  annotations:<br>    eks.amazonaws.com/role-arn: arn:aws:iam::111122223333:role/demo-app-operator</pre><p>Let’s save this file as demo-app-operator.yaml. Let’s deploy our second demo application to the same namespace using kubectl.</p><p><strong>4. Create scaler</strong></p><p>Let’s now create our scaler specifications. This time it will be simpler because we don’t need the <em>TriggerAuthentication</em> resource anymore.</p><pre>apiVersion: keda.sh/v1alpha1<br>kind: ScaledObject<br>metadata:<br>  name: demo-app-scaledobject-operator<br>  namespace: demo-ns<br>spec:<br>  minReplicaCount: 0  # scale down to 0<br>  maxReplicaCount: 10 <br>  pollingInterval: 30 <br>  scaleTargetRef:<br>    name: demo-app-operator    # name of our demo-app deployment<br>  triggers:<br>  - type: aws-sqs-queue<br>    metadata:<br>      queueURL: https://sqs.ca-central-1.amazonaws.com/111122223333/demo-app-sqs-operator<br>      queueLength: &quot;2&quot; # 2 messages per pod <br>      awsRegion: ca-central-1<br>      identityOwner: operator # defaults to pod if unset</pre><p>Add this configuration to our manifest file, demo-app-operator.yaml. Let’s deploy our <em>ScaledObject</em> and check if it’s happy.</p><pre>Status:<br>  Conditions:<br>    Message:  ScaledObject is defined correctly and is ready for scaling<br>    Reason:   ScaledObjectReady<br>    Status:   True<br>    Type:     Ready<br>    Message:  Scaling is not performed because triggers are not active<br>    Reason:   ScalerNotActive<br>    Status:   False<br>    Type:     Active<br>    Message:  No fallbacks are active on this scaled object<br>    Reason:   NoFallbackFound<br>    Status:   False<br>    Type:     Fallback<br>  External Metric Names:<br>    s0-aws-sqs-demo-app-operator<br>  Health:<br>    s0-aws-sqs-demo-app-operator:<br>      Number Of Failures:  0<br>      Status:              Happy<br>  Hpa Name:                keda-hpa-demo-app-operator<br>  Last Active Time:        2023-02-16T21:44:24Z<br>  Original Replica Count:  1<br>  Scale Target GVKR:<br>    Group:            apps<br>    Kind:             Deployment<br>    Resource:         deployments<br>    Version:          v1<br>  Scale Target Kind:  apps/v1.Deployment<br>Events:               &lt;none&gt;</pre><p>Check also the logs of both KEDA deployments if they have reconciled with the new <em>ScaledObject</em> we created.</p><p>The <em>keda-operator </em>has also created an HPA for our second demo application.</p><pre>$ kubectl get hpa -n demo-ns<br>NAME                         REFERENCE                                    TARGETS             MINPODS   MAXPODS   REPLICAS   AGE<br>keda-hpa-demo-app-operator   Deployment/demo-app-operator   &lt;unknown&gt;/1 (avg)   1         10        0          4d16h<br>keda-hpa-demo-app            Deployment/demo-app        &lt;unknown&gt;/2 (avg)   1         10        0          4d16h</pre><p>Awesome! It’s time to trigger scaling!</p><p><strong>5. Test scaling</strong></p><p>Let’s send messages to our queue.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*grKNxKWvk3aIRvTaxZQtWw.png" /></figure><p>In the cluster, let’s get the latest events from our <em>demo-ns</em>.</p><pre>$ kubectl get events -n demo-ns<br>LAST SEEN   TYPE     REASON                     OBJECT                                                             MESSAGE<br>5m31s       Normal   SuccessfulRescale          horizontalpodautoscaler/keda-hpa-demo-app-operator   New size: 3; reason: external metric s0-aws-sqs-demo-app-operator(&amp;LabelSelector{MatchLabels:map[string]string{scaledobject.keda.sh/name: demo-app-operator,},MatchExpressions:[]LabelSelectorRequirement{},}) above target<br>5m31s       Normal   Scheduled                  pod/demo-app-operator-7cff88c99b-5ddjv               Successfully assigned demo-ns/demo-app-operator-7cff88c99b-5ddjv to ip-11-111-111-111.us-west-2.compute.internal<br>5m29s       Normal   Pulled                     pod/demo-app-operator-7cff88c99b-5ddjv               Container image &quot;nginx:latest&quot; already present on machine<br>5m29s       Normal   Created                    pod/demo-app-operator-7cff88c99b-5ddjv               Created container demo-app-operator<br>5m29s       Normal   Started                    pod/demo-app-operator-7cff88c99b-5ddjv               Started container demo-app-operator<br>5m36s       Normal   Scheduled                  pod/demo-app-operator-7cff88c99b-gwz46               Successfully assigned demo-ns/demo-app-operator-7cff88c99b-gwz46 to ip-11-111-111-111.us-west-2.compute.internal<br>5m34s       Normal   Pulled                     pod/demo-app-operator-7cff88c99b-gwz46               Container image &quot;nginx:latest&quot; already present on machine<br>5m34s       Normal   Created                    pod/demo-app-operator-7cff88c99b-gwz46               Created container demo-app-operator<br>5m34s       Normal   Started                    pod/demo-app-operator-7cff88c99b-gwz46               Started container demo-app-operator<br>5m31s       Normal   Scheduled                  pod/demo-app-operator-7cff88c99b-xn7c8               Successfully assigned demo-ns/demo-app-operator-7cff88c99b-xn7c8 to ip-11-111-111-111.us-west-2.compute.internal<br>5m29s       Normal   Pulled                     pod/demo-app-operator-7cff88c99b-xn7c8               Container image &quot;nginx:latest&quot; already present on machine<br>5m29s       Normal   Created                    pod/demo-app-operator-7cff88c99b-xn7c8               Created container demo-app-operator<br>5m29s       Normal   Started                    pod/demo-app-operator-7cff88c99b-xn7c8               Started container demo-app-operator<br>5m36s       Normal   SuccessfulCreate           replicaset/demo-app-operator-7cff88c99b              Created pod: demo-app-operator-7cff88c99b-gwz46<br>5m31s       Normal   SuccessfulCreate           replicaset/demo-app-operator-7cff88c99b              Created pod: demo-app-operator-7cff88c99b-xn7c8<br>5m31s       Normal   SuccessfulCreate           replicaset/demo-app-operator-7cff88c99b              Created pod: demo-app-operator-7cff88c99b-5ddjv<br>5m36s       Normal   ScalingReplicaSet          deployment/demo-app-operator                         Scaled up replica set demo-app-operator-7cff88c99b to 1<br>5m36s       Normal   KEDAScaleTargetActivated   scaledobject/demo-app-operator                       Scaled apps/v1.Deployment demo-ns/demo-app-operator from 0 to 1<br>5m31s       Normal   ScalingReplicaSet          deployment/demo-app-operator                         Scaled up replica set demo-app-operator-7cff88c99b to 3</pre><p>Putting a watch on the <em>demo-ns</em> shows the new pods that get created as a result of the scaling,</p><pre>$ kubectl get pods -n demo-ns -w<br>NAME                                 READY   STATUS              RESTARTS   AGE<br>demo-app-operator-7cff88c99b-gwz46   0/1     Pending             0          0s<br>demo-app-operator-7cff88c99b-gwz46   0/1     Pending             0          0s<br>demo-app-operator-7cff88c99b-gwz46   0/1     ContainerCreating   0          0s<br>demo-app-operator-7cff88c99b-gwz46   1/1     Running             0          3s<br>demo-app-operator-7cff88c99b-xn7c8   0/1     Pending             0          0s<br>demo-app-operator-7cff88c99b-xn7c8   0/1     Pending             0          0s<br>demo-app-operator-7cff88c99b-5ddjv   0/1     Pending             0          0s<br>demo-app-operator-7cff88c99b-5ddjv   0/1     Pending             0          0s<br>demo-app-operator-7cff88c99b-xn7c8   0/1     ContainerCreating   0          0s<br>demo-app-operator-7cff88c99b-5ddjv   0/1     ContainerCreating   0          1s<br>demo-app-operator-7cff88c99b-5ddjv   1/1     Running             0          2s<br>demo-app-operator-7cff88c99b-xn7c8   1/1     Running             0          3s</pre><p>Let’s trigger descaling by deleting the messages in the queue.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sV0P-_ifywT0MhPG4xGHdw.png" /></figure><p>Getting the events from the <em>demo-ns</em> shows:</p><pre>$ kubectl get events -n demo-ns<br>LAST SEEN   TYPE     REASON                       OBJECT                                             MESSAGE<br>4m2s        Normal   Killing                      pod/demo-app-674d8c455c-64jvw    Stopping container demo-app<br>4m2s        Normal   Killing                      pod/demo-app-674d8c455c-n7svp    Stopping container demo-app<br>4m2s        Normal   SuccessfulDelete             replicaset/demo-app-674d8c455c   Deleted pod: demo-app-674d8c455c-n7svp<br>4m2s        Normal   SuccessfulDelete             replicaset/demo-app-674d8c455c   Deleted pod: demo-app-674d8c455c-64jvw<br>4m3s        Normal   ScalingReplicaSet            deployment/demo-app              Scaled down replica set demo-app-674d8c455c to 0<br>4m3s        Normal   KEDAScaleTargetDeactivated   scaledobject/demo-app            Deactivated apps/v1.Deployment demo-ns/demo-app from 2 to 0</pre><p>Getting the pods shows,</p><pre>$ kubectl get pods -n demo-ns -w<br>NAME                                 READY   STATUS              RESTARTS   AGE<br>demo-app-operator-7cff88c99b-gwz46   1/1     Terminating         0          14m<br>demo-app-operator-7cff88c99b-xn7c8   1/1     Terminating         0          13m<br>demo-app-operator-7cff88c99b-5ddjv   1/1     Terminating         0          13m<br>demo-app-operator-7cff88c99b-5ddjv   0/1     Terminating         0          13m<br>demo-app-operator-7cff88c99b-xn7c8   0/1     Terminating         0          13m<br>demo-app-operator-7cff88c99b-xn7c8   0/1     Terminating         0          13m<br>demo-app-operator-7cff88c99b-xn7c8   0/1     Terminating         0          13m<br>demo-app-operator-7cff88c99b-5ddjv   0/1     Terminating         0          13m<br>demo-app-operator-7cff88c99b-5ddjv   0/1     Terminating         0          13m<br>demo-app-operator-7cff88c99b-gwz46   0/1     Terminating         0          14m<br>demo-app-operator-7cff88c99b-gwz46   0/1     Terminating         0          14m<br>demo-app-operator-7cff88c99b-gwz46   0/1     Terminating         0          14m</pre><p>Yaaay! Awesome work in following through with this guide. We have scaled our demo application based on a queue using two different identity models.</p><p>If you have run through some errors, check out below for a list of errors that I encountered and how to fix them.</p><p>Watch out for the next part of KEDA in Amazon EKS mini-series. On the next one, we’ll use KEDA to scale a workload based on http traffic! This ability is a game-changer for our scaling capabilities.</p><h4>Common Errors</h4><p>Here are some possible errors that you may encounter following this blog and how to solve them. These errors can be seen by describing the <em>ScaledObject</em> in your namespace.</p><ul><li>WebIdentityErr</li></ul><pre>Warning  KEDAScalerFailed  18s  keda-operator  WebIdentityErr: failed to retrieve credentials caused by: AccessDenied: Not authorized to perform sts:AssumeRoleWithWebIdentity status code: 403, request id: 24f2181f-8a1b-4a4d-8de3-f85d501787a7</pre><p>This is often caused by an issue with the role’s trust policy. Double check the values.</p><ul><li>TriggerAuthentication</li></ul><pre>2023-02-16T03:46:46Z ERROR scalehandler Error getting triggerAuth {&quot;type&quot;: &quot;ScaledObject&quot;, &quot;namespace&quot;: &quot;demo-ns&quot;, &quot;name&quot;: &quot;demo-app&quot;, &quot;triggerAuthRef.Name&quot;: &quot;demo-app-trigger-auth-aws-credentials&quot;, &quot;error&quot;: &quot;TriggerAuthentication.keda.sh \&quot;demo-app-trigger-auth-aws-credentials\&quot; not found&quot;}</pre><p>Check that the <em>TriggerAuthentication</em> object gets created.</p><ul><li>NonExistentQueue: wsdl</li></ul><pre>Warning  KEDAScalerFailed  78s  keda-operator  AWS.SimpleQueueService.NonExistentQueue: The specified queue does not exist for this wsdl version status code: 400, request id: b01e1553-fa5d-5bde-8190-493cc508300f</pre><p>You might have set the incorrect region on the<em> values.yaml</em> file. Double-check that you are setting the region where the queue is.</p><ul><li>AccessDenied</li></ul><pre>  Warning  KEDAScalerFailed  3m40s  keda-operator  AccessDenied: User: arn:aws:sts::620528608035:assumed-role/keda-operator/1677015712377894955 is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::046921848075:role/demo-app status code: 403, request id: 6001b079-4cc9-4309-b364-5958316ce52b</pre><p>This can be caused by a missing <em>sts:AssumeRole</em> policy on the <em>keda-operator</em> role or on your workload’s role.</p><pre>    Warning  KEDAScalerFailed  0s  keda-operator  AccessDenied: Access to the resource https://sqs.ca-central-1.amazonaws.com/ is denied. status code: 403, request id: c9b51391-2132-551e-b998-4ffe48b8de74</pre><p>This can be caused by missing permissions to access the queue. When using the Operator Identity Model, check that:</p><ol><li>access policy has been added to the queue</li><li>permissions have been added to the KEDA role</li></ol><pre>Events:<br>  Type     Reason              Age   From           Message<br>  ----     ------              ----  ----           -------<br>  Normal   KEDAScalersStarted  109s  keda-operator  Started scalers watch<br>  Warning  KEDAScalerFailed    106s  keda-operator  AccessDenied: Access to the resource https://sqs.ca-central-1.amazonaws.com/ is denied.<br>           status code: 403, request id: dabe543b-d61f-57a4-9c6b-089fdde8daa9<br>  Warning  KEDAScalerFailed  92s  keda-operator  AccessDenied: Access to the resource https://sqs.ca-central-1.amazonaws.com/ is denied.<br>           status code: 403, request id: 84ae713a-5e5b-5c73-98d9-2d9e525f953d<br>  Warning  KEDAScalerFailed  77s  keda-operator  AccessDenied: Access to the resource https://sqs.ca-central-1.amazonaws.com/ is denied.<br>           status code: 403, request id: 3955aad9-0221-5a6f-a7c4-f67d0e32145f<br>  Normal   ScaledObjectReady  69s (x2 over 109s)  keda-operator  ScaledObject is ready for scaling<br>  Warning  KEDAScalerFailed   62s                 keda-operator  AccessDenied: Access to the resource https://sqs.ca-central-1.amazonaws.com/ is denied.<br>           status code: 403, request id: eddee0ca-6e76-520b-b4ac-bc6e62543864<br>  Warning  KEDAScalerFailed  47s  keda-operator  AccessDenied: Access to the resource https://sqs.ca-central-1.amazonaws.com/ is denied.<br>           status code: 403, request id: 7f1a95e9-3e9c-550b-82d0-c94d32adf964<br>  Normal   KEDAScaleTargetDeactivated  33s  keda-operator  Deactivated apps/v1.Deployment demo-ns/demo-app-operator from 1 to 0</pre><p>From the example above, sometimes it may also take a minute before KEDA can access the queue.</p><h4>References:</h4><ul><li><a href="https://keda.sh/docs/2.10/faq/">https://keda.sh/</a></li></ul><h3>Have a question or want to reach out for a yarn?</h3><p>If you have some questions, feedback or just want to reach out for a good ol’ yarn, please connect and flick me a message at <a href="https://www.linkedin.com/in/carlo-columna/">https://www.linkedin.com/in/carlo-columna/</a>.</p><h3>Liked what you’ve read?</h3><p>If you liked what you read, or if you found it useful, please follow me below for more articles and to get notified when I published new ones. Cheers!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1dbd13fa5384" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Level Up Your Kubernetes Scaling]]></title>
            <link>https://medium.com/@carlocolumna/eda-in-amazon-eks-install-keda-82849cf31f01?source=rss-d9219c5b4599------2</link>
            <guid isPermaLink="false">https://medium.com/p/82849cf31f01</guid>
            <category><![CDATA[containers]]></category>
            <category><![CDATA[autoscaling]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[keda]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Carlo Columna]]></dc:creator>
            <pubDate>Mon, 12 Jun 2023 20:15:49 GMT</pubDate>
            <atom:updated>2024-08-04T20:44:05.656Z</atom:updated>
            <content:encoded><![CDATA[<h3>How to Level Up Your Kubernetes Workload Autoscaling</h3><p>Kia ora!</p><p>Welcome to the first part of a blog series discussing autoscaling capabilities in Kubernetes using <a href="https://keda.sh/">KEDA or Kubernetes Event-driven Autoscaling</a>.</p><p>I highly recommend going through this first blog to understand why do you even need KEDA, but if you really want to jump ahead and see KEDA in action, check out parts 2 and 3 below:</p><ul><li><a href="https://medium.com/@carlocolumna/keda-in-amazon-eks-part-2-scale-based-on-aws-sqs-queue-1dbd13fa5384">Part 2: How to Scale Your Pods Based on a Queue</a></li><li><a href="https://medium.com/@carlocolumna/how-to-scale-your-pods-based-on-http-traffic-d58221d5e7f1">Part 3: How to Scale Your Pods Based on HTTP Traffic</a></li></ul><h3>Problem</h3><figure><img alt="Uncle Sam meme I want more" src="https://cdn-images-1.medium.com/max/500/1*hpnVCw7dotVzolr3zFhM4A.jpeg" /></figure><p>At the company I work in, we run multi-tenant production grade clusters on top of Amazon EKS and we have been mostly using <a href="https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/">Horizontal Pod Autoscaler (HPA)</a> for autoscaling. Recently, we’ve been working on providing new autoscaling options for our customers who are running their workloads in our Kubernetes clusters.</p><p>The main drivers behind this effort are:<br>1. Overcoming the limitations of HPA<br>2. Increasing Scaling Demands and the Kubernetes External Metrics API Limitation<br>3. Deprecating k8s-cloudwatch-adapter</p><p>We’ll go through each of the above.</p><h4>Limitations of Horizontal Pod Autoscaler</h4><p>Kubernetes natively offers the HPA as a controller to increase and decrease replicas based on demand. HPA provides the ability to scale on pods metrics namely CPU and memory. While this is enough for most workloads, there are limitations to it.</p><ol><li><strong>It cannot scale to zero. </strong>HPA by default uses metrics namely CPU and memory utilisation to calculate the desired number of replicas. Because these metrics cannot be zero, the desired number of replicas cannot be zero as well. This is not ideal for intermittent and resource intensive workloads when you are trying to optimise your cost.</li><li><strong>It’s limited to scaling based on metrics. </strong>You are unable to scale based on events or http traffic.</li><li><strong>It’s dependent on metrics aggregators.</strong> In order for HPA to work, it needs to fetch metrics from <a href="https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation/">aggregated APIs</a>. These APIs are usually provided by other add-ons that you have to install separately such as <a href="https://kubernetes.io/docs/tasks/debug/debug-cluster/resource-metrics-pipeline/#metrics-server">Metrics Server</a> which usually provides themetrics.k8s.io API. Other providers of custom.metrics.k8s.io, andexternal.metrics.k8s.io that we also use are <a href="https://github.com/kubernetes-sigs/prometheus-adapter">prometheus-adapter</a> to expose prometheus metrics and the <a href="https://github.com/amazon-archives/k8s-cloudwatch-adapter">k8s-cloudwatch-adapter</a> which is now deprecated.</li><li><strong>It often does not scale on the actual target value.</strong> HPA bases its scaling decisions on a target metric value. However, due to how the HPA <a href="https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/#algorithm-details">scaling algorithm</a> works, the current target value on the HPA often does not match with the metrics of the system you are scaling on.</li></ol><h4>Increasing Scaling Demands and External Metrics API Kubernetes Limitation</h4><p>As our clusters grow, so is the demand from hosting various workloads with different scaling requirements. We want to be able to scale on metrics not only inside but also outside of the cluster such as from Amazon Managed Prometheus, AWS Cloudwatch and Kafka or any other events source. This will enable our platform to cater to these scaling needs and put us in a good spot as our clusters grow.</p><p>Fortunately, a <a href="https://github.com/kubernetes/design-proposals-archive/blob/main/instrumentation/external-metrics-api.md">proposal was adopted to extend the HPA by introducing the new External metric type</a> for autoscaling based on metrics coming from outside of the Kubernetes cluster. This allows us to use metrics adapters that serve a variety of metrics from external services and make them available to autoscale on by using a metric server.</p><p>However, there is one big caveat to this. There is a limitation in Kubernetes where you can only have one running metric server serving external.metrics.k8s.io metrics per cluster. This is because only one API Service can be registered to handle external metrics. External metrics are metrics that represent the state of an application/service that is running outside of the Kubernetes cluster.</p><p>So this means we’ll have to choose which metric server to run and serve metrics from. But which one?</p><figure><img alt="Homer Simpson Don’t Make Me Choose Meme" src="https://cdn-images-1.medium.com/max/1024/1*VySqs84lXEn_F2QkEVMOUA.png" /></figure><p>From the diagram below, CloudWatch Adapter and KEDA cannot be run at the same time because they both implement external.metrics.k8s.io. Since Prometheus adapter is implementing custom.metrics.k8s.io, it does not conflict with KEDA or CloudWatch Adapter. Custom metrics are metrics that come from applications solely running on the Kubernetes cluster such as Prometheus.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Z7cw_wVLP8pAF66FAnhILQ.jpeg" /><figcaption>Multiple Metric Servers in Kubernetes</figcaption></figure><h4>Deprecated K8s CloudWatch Adapter</h4><p>One of the scaling tools that we have used in the past is the <a href="https://github.com/amazon-archives/k8s-cloudwatch-adapter">K8s CloudWatch Adapter</a>. This adapter allowed us to scale our Kubernetes workloads using the <a href="https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/">Horizontal Pod Autoscaler</a> (HPA) with metrics from AWS CloudWatch. One of the most popular ways of scaling that was used was scaling based on AWS SQS Queue. However, from K8s v1.22, it’s no longer working because it uses deprecated API versions which are no longer supported. AWS has also archived and stopped maintaining the project and instead recommends using KEDA.</p><p>From these decision drivers, we have started looking at KEDA for autoscaling of Kubernetes workloads.</p><h3>KEDA</h3><figure><img alt="Event Based Autoscaling It’s Kinda Big Deal Meme" src="https://cdn-images-1.medium.com/max/1024/1*j5hAC3VbPfysuDvkMi_p-w.png" /></figure><p>KEDA stands for Kubernetes Event-driven Autoscaling. Additionally, from their <a href="https://keda.sh/">website</a>,</p><blockquote>With KEDA, you can drive the scaling of any container in Kubernetes based on the number of events needing to be processed.</blockquote><blockquote><strong>KEDA</strong> is a single-purpose and lightweight component that can be added into any Kubernetes cluster. KEDA works alongside standard Kubernetes components like the <a href="https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/">Horizontal Pod Autoscaler</a> and can extend functionality without overwriting or duplication. With KEDA you can explicitly map the apps you want to use event-driven scale, with other apps continuing to function. This makes KEDA a flexible and safe option to run alongside any number of any other Kubernetes applications or frameworks.</blockquote><p>KEDA is part of CNCF with a big community backing and is widely used for event scaling in Kubernetes. It offers many features like scaling different types of Kubernetes workloads such as Deployments, StatefulSets, and Jobs based on different type of events. It can also scale custom resources as long as the target Custom Resource must define a/scale <a href="https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#scale-subresource">subresource</a>.</p><p>It has a wide range of <a href="https://keda.sh/docs/2.8/scalers/">readily available scalers</a>.</p><blockquote>KEDA <strong>scalers</strong> can both detect if a deployment should be activated or deactivated, and feed custom metrics for a specific event source.</blockquote><p>Its even extensible which means you can create your own scaler.</p><h4>How KEDA Works</h4><blockquote>KEDA performs two key roles within Kubernetes:</blockquote><blockquote><strong>1. Agent</strong> — KEDA activates and deactivates Kubernetes <a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment">Deployments</a> to scale to and from zero on no events. This is one of the primary roles of the keda-operator container that runs when you install KEDA.</blockquote><blockquote><strong>2. Metrics</strong> — KEDA acts as a <a href="https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/#support-for-custom-metrics">Kubernetes metrics server</a> that exposes rich event data like queue length or stream lag to the Horizontal Pod Autoscaler to drive scale out. It is up to the Deployment to consume the events directly from the source. This preserves rich event integration and enables gestures like completing or abandoning queue messages to work out of the box. The metric serving is the primary role of the keda-operator-metrics-apiserver container that runs when you install KEDA.</blockquote><p>When you setup KEDA, it will install two deployments, <em>keda-operator </em>and <em>keda-operator-metrics-apiserver . </em>The <em>keda-operator</em>, as mentioned above acts as an agent or controller to set the number of desired replicas. It does this by creating and managing an application’s Horizontal Pod Autoscaler. It also registers and manages KEDA Custom Resource Definitions (CRDs) such as ScaledObject and TriggerAuthentication.</p><p>As mentioned above, the <em>keda-operator-metrics-apiserver</em> acts as a metrics server exposing the metrics to trigger the scale out.</p><p>KEDA primarily serves metrics for metric sources outside of the Kubernetes cluster so it uses external metrics and registers v1beta1.external.metrics.k8s.io namespace in the API service.</p><h4>Architecture</h4><blockquote>The diagram below shows how KEDA works in conjunction with the Kubernetes Horizontal Pod Autoscaler, external event sources, and Kubernetes’ <a href="https://etcd.io/">etcd</a> data store:</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7t48ne8KZabS6BWMTgSatQ.png" /><figcaption>KEDA architecture</figcaption></figure><h4>Security</h4><p>KEDA is secure by default. KEDA will run as non-root. You can further increase the security in some cases by setting KEDA to listen on TLS v1.3 only or running KEDA withreadOnlyRootFilesystem=true .</p><h4>Install KEDA</h4><p>There are multiple ways to install KEDA in a Kubernetes cluster. Let’s go with Helm and Terraform. We will be running KEDA in Amazon EKS.</p><p>Before we can proceed, let’s lay out the prerequisites:</p><ul><li>AWS account</li><li>Access to an Amazon EKS cluster</li><li>IAM roles for service accounts (IRSA) setup for the EKS cluster. See <a href="https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html">here</a> for more details.</li></ul><p><strong>Steps in installing KEDA:</strong></p><ol><li>Create its AWS IAM role</li></ol><p>Before installing KEDA in the cluster, let’s first create its AWS IAM role. Giving it a role allows it to communicate with other AWS resources as you can see later on this blog. Add an assume role trust policy for the role to allow the cluster to assume it.</p><pre>resource &quot;aws_iam_role&quot; &quot;keda-operator&quot; {<br>  name               = &quot;keda-operator&quot;<br>  assume_role_policy = module.keda-operator-trust-policy.json<br>}<br><br>data &quot;aws_iam_policy_document&quot; &quot;keda-operator-trust-policy&quot; {<br>  statement {<br>    actions = [<br>      &quot;sts:AssumeRoleWithWebIdentity&quot;<br>    ]<br>    principals {<br>      type = &quot;Federated&quot;<br>      identifiers = [&quot;arn:aws:iam::111122223333:oidc-provider/oidc.eks.region-code.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE&quot;]<br>    }<br>    condition {<br>      test     = &quot;StringEquals&quot;<br>      variable = &quot;oidc.eks.region-code.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE:sub&quot;<br>      values   = [&quot;system:serviceaccount:keda:keda-operator&quot;]<br>    }<br>  }<br>}<br><br>output &quot;keda_operator_role_arn&quot; {<br>  value = aws_iam_role.keda-operator.arn<br>}</pre><p>2. Install KEDA in the cluster. Pass the role created on the previous step.</p><pre>resource &quot;helm_release&quot; &quot;keda&quot; {<br>  name             = &quot;keda&quot;<br>  repository       = &quot;https://kedacore.github.io/charts&quot;<br>  chart            = &quot;keda&quot;<br>  version          = &quot;2.8.2&quot;<br>  namespace        = &quot;keda&quot;<br>  create_namespace = true<br>  values = [<br>    templatefile(&quot;${path.module}/values/keda.yaml&quot;, {<br>      keda_operator_role_arn = &quot;arn:aws:iam::111122223333:role/keda-operator&quot;<br>    }),<br>  ]<br>}</pre><p>The values.yaml file.</p><pre>serviceAccount:<br>  annotations:<br>    eks.amazonaws.com/role-arn: ${keda_operator_role_arn}</pre><p>Applying this terraform and running helm we get,</p><pre>$ helm list -nkeda<br>NAME NAMESPACE REVISION UPDATED                               STATUS   CHART      APP VERSION<br>keda keda      1        2023-02-16 16:08:20.945876 +1300 NZDT deployed keda-2.8.2 2.8.1</pre><p>Running a kubectl to see if the pods are healthy we get,</p><pre>$ kubectl get pods -n keda<br>NAME                                               READY   STATUS    RESTARTS   AGE<br>keda-operator-54bcdc6446-ktx7d                     1/1     Running   0          0s<br>keda-operator-metrics-apiserver-74487bb99f-4v22r   1/1     Running   0          2s</pre><p>So that’s it! That’s how easy it is to get KEDA up and running in a EKS cluster.</p><p>Watch out for the next part of KEDA in Amazon EKS blog mini-series. On the next part, we’ll use KEDA to scale a workload based on the number of messages in a AWS SQS Queue.</p><p>Pretty exciting right?!</p><p>Cheers!</p><h4>References:</h4><ul><li><a href="https://keda.sh/docs/2.10/faq/">https://keda.sh/</a></li><li><a href="https://www.cncf.io/blog/2021/03/30/why-alibaba-cloud-uses-keda-for-application-autoscaling/">https://www.cncf.io/blog/2021/03/30/why-alibaba-cloud-uses-keda-for-application-autoscaling/</a></li></ul><h3>Have a question or want to reach out for a yarn?</h3><p>If you have some questions, feedback or just want to reach out for a good ol’ yarn, please connect and flick me a message at <a href="https://www.linkedin.com/in/carlo-columna/">https://www.linkedin.com/in/carlo-columna/</a>.</p><h3>Liked what you’ve read?</h3><p>If you liked what you read, or if you found it useful, please follow me below for more articles and to get notified when I published new ones. Thanks!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=82849cf31f01" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>