<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Sebastian Mincewicz on Medium]]></title>
        <description><![CDATA[Stories by Sebastian Mincewicz on Medium]]></description>
        <link>https://medium.com/@sebolabs?source=rss-3abdb2452418------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*wwv8siBGiHYGtalfLeReww.png</url>
            <title>Stories by Sebastian Mincewicz on Medium</title>
            <link>https://medium.com/@sebolabs?source=rss-3abdb2452418------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 20:08:52 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@sebolabs/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Reusing CloudFront, ALB, and API Gateway in a Serverless Platform]]></title>
            <link>https://faun.pub/reusing-cloudfront-alb-and-api-gateway-in-a-serverless-platform-5d21adb64478?source=rss-3abdb2452418------2</link>
            <guid isPermaLink="false">https://medium.com/p/5d21adb64478</guid>
            <category><![CDATA[api-gateway]]></category>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[application-load-balancer]]></category>
            <category><![CDATA[platform-engineering]]></category>
            <category><![CDATA[cloudfront]]></category>
            <dc:creator><![CDATA[Sebastian Mincewicz]]></dc:creator>
            <pubDate>Tue, 17 Mar 2026 17:49:09 GMT</pubDate>
            <atom:updated>2026-03-17T17:49:09.425Z</atom:updated>
            <cc:license>https://creativecommons.org/licenses/by-sa/4.0/</cc:license>
            <content:encoded><![CDATA[<p>Modern serverless platforms often need to support a mix of public-facing entry points — web applications and APIs — while keeping internal communication private and tightly controlled. Non-functional requirements around security, compliance, and operational isolation typically drive this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dF7MaqCUdD3XWiHQd4xOhQ.png" /></figure><p>At the same time, such platforms are increasingly built around multiple domain services, each representing a bounded context and owned by a small team. Independent deployments, limited blast radius, and clear ownership are usually explicit architectural goals.</p><p>These goals, however, often collide with another equally important requirement: speed. In fast-paced development environments, the ability to spin up feature environments on demand, test changes quickly, and tear everything down without friction is critical to maintaining delivery velocity.</p><p>This post explores how architectural choices at the edge — specifically around CloudFront, Application Load Balancers (ALB), and API Gateway — can either enable or severely constrain that speed. It looks at how careful reuse of these components can support isolated, per-service deployments and on-demand environments without driving up cost or operational complexity, and where the trade-offs start to appear.</p><h3>Architectural choices</h3><h4>Frontend</h4><p>In many serverless platforms, the frontend acts as the primary public entry point. A common pattern is to place CloudFront at the edge and route requests to different backend origins depending on their nature.</p><p>Static assets are typically served from S3, while dynamic requests are forwarded to compute workloads that render pages or handle frontend-driven API calls. These dynamic workloads may run as Lambda functions or containers, often inside a VPC to meet security and compliance requirements.</p><p>At this point, there is an architectural choice to make: whether dynamic frontend traffic should be handled by the <a href="https://docs.aws.amazon.com/prescriptive-guidance/latest/choosing-the-right-aws-service-for-your-microservice-endpoints/services-comparison.html">API Gateway or an Application Load Balancer (ALB)</a>.</p><p>From a non-functional requirements perspective, separating public access from private execution is key. While API Gateway is well-suited for exposing managed APIs, frontend traffic often benefits from being terminated at ALB, with requests forwarded to compute that remains fully private within a VPC. <strong>CloudFront supports private origins through VPC-origin integrations with internal ALB, whereas private API Gateways cannot be used directly as CloudFront origins.</strong> This makes ALB a more natural fit when frontend compute is intentionally kept off the public internet.</p><p>Placing <a href="https://docs.aws.amazon.com/elasticloadbalancing/latest/application/lambda-functions.html">Lambda functions behind an ALB</a> and inside a VPC also addresses increasingly common security requirements in regulated environments. When compute is fronted by an internal ALB, CloudFront becomes the only public ingress path by design. The ALB is not internet-facing and cannot be accessed directly, which ensures that all external traffic is consistently terminated, inspected, and controlled at the edge.</p><p>This pattern significantly reduces the risk of accidental or intentional bypass of edge-level controls. Combined with VPC-based execution, it enables tighter governance of inbound and outbound traffic through WAF, security groups, routing, and egress filtering. Together, these measures provide stronger guarantees than relying solely on managed service boundaries or publicly reachable endpoints.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t4ZXDrpvUd_9qa1F16COeA.png" /></figure><p>From an operational perspective, ALB offers lower latency for HTTP workloads, native support for both host-based and path-based routing, and seamless integration with both Lambda and container-based runtimes such as ECS. These characteristics make it particularly well-suited for frontend and backend-for-frontend (BFF) use cases, where routing flexibility, performance, and network controls often outweigh API-centric features such as request validation, usage plans, or request transformation.</p><p>This does not diminish the role of API Gateway in the overall architecture. API Gateway remains a strong choice for internal and external APIs, especially where API lifecycle management, authentication, throttling, and contract enforcement are primary concerns. Using ALB at the frontend edge simply allows each component to be applied where it fits best, rather than forcing a single service to satisfy fundamentally different requirements.</p><h4>Internal API</h4><p>Internally, platforms often expose functionality through APIs consumed by other services or internal clients. A key architectural decision is whether to operate a single internal API Gateway or multiple gateways aligned with services, domains, or teams.</p><p>There is no universal answer. Smaller platforms or early-stage systems often benefit from a single internal API, as it simplifies discovery and reduces operational overhead. As platforms grow, multiple APIs aligned with domains or “two-pizza teams” can become more appropriate, particularly when ownership boundaries, release cadence, or non-functional requirements diverge.</p><p>What matters more than the number of gateways, however, is recognising that <strong>API shape and deployment shape are not the same thing</strong>. A single logical API does not require a single infrastructure deployment unit, just as multiple services do not automatically justify multiple gateways. Treating these concerns separately allows teams to optimise for independent releases, blast-radius reduction, and clearer ownership, while keeping the external API surface coherent.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3Ua5dx4u3NrLb9BcOcriig.png" /></figure><p>This distinction becomes especially important in high-paced environments, where the ability to evolve services independently should not be constrained by how APIs are presented or grouped at the edge.</p><h4>Backend services and infrastructure-as-code</h4><p>From an infrastructure-as-code (IaC) perspective, backend services are often deployed as independent units to support isolation, targeted releases, and reduced blast radius. This aligns naturally with domain-driven design, where each service represents a bounded context and can evolve independently.</p><p>When multiple backend services are exposed through a <strong>single API Gateway</strong>, this model becomes more nuanced. API Gateway configuration changes — such as routes, integrations, or method settings — are only made effective through an explicit <a href="https://docs.aws.amazon.com/apigateway/latest/developerguide/how-to-deploy-api.html">deployment step</a>. As a result, while service-specific configuration can be defined independently, the API Gateway itself still has a shared deployment lifecycle.</p><p>This does not make per-service backend stacks incompatible with a single API Gateway, but it does require additional structure. In practice, this means decomposing API Gateway configuration into distinct concerns: a base or core API configuration, per-service or per-domain route and integration definitions, and a dedicated deployment component responsible for applying changes to the gateway. That deployment component must be notified when any service updates its portion of the API configuration, without introducing tight coupling between services or requiring knowledge of how many services exist.</p><p>Without this explicit separation, a shared API Gateway can easily become a point of implicit coupling, where independent service deployments are forced to coordinate around a central API deployment. With the right decomposition and signalling in place, however, a single logical API can still support independent service lifecycles and parallel provisioning.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gwLx45Nhn0gEFazB6e0xgA.png" /></figure><p>In contrast, using an <strong>API Gateway per backend service</strong> naturally scopes configuration and deployment to the service itself, avoiding the need for cross-service coordination at the gateway level. The trade-off is increased operational overhead and a more fragmented API surface. The choice between these approaches is therefore not about feasibility, but about how much complexity is absorbed by infrastructure design versus operational management, and how much flexibility is required as the platform continues to evolve.</p><h3>On-demand feature environments</h3><h4>Why they matter</h4><p>Fast feedback is essential. Feature environments allow teams to validate changes in realistic conditions, unblock reviews, and catch integration issues early. While tools like <a href="https://www.localstack.cloud/">LocalStack</a> can help with local development, they have limitations — especially for integration testing, CI pipelines, and workflows involving multiple services.</p><p>On-demand environments bridge that gap, but naïvely provisioning full stacks per feature quickly becomes expensive and slow, too slow.</p><h4>Reuse as an enabler</h4><p>Not all infrastructure needs to be duplicated. Components like CloudFront, ALB, and API Gateway are surprisingly well-suited to reuse, provided routing is designed with that goal in mind.</p><p>This is not just a cost consideration. Creating or modifying edge components such as CloudFront distributions or load balancers can easily take ten minutes or more, which makes them a poor fit for fast, iterative workflows. When feature environments depend on provisioning or reconfiguring these components, feedback loops slow down dramatically.</p><p>The key, therefore, is deciding what varies per environment and what stays shared. By reusing long-lived edge infrastructure and shifting environment-specific concerns into routing, configuration, and backend resources, platforms can keep costs predictable and provisioning times low. The trade-off is that this requires upfront discipline in routing design, naming conventions, and configuration boundaries, but that investment pays off quickly as the environment count and delivery pace increase.</p><h4>Ephemeral on-demand environments</h4><p>Ephemeral environments, the golden grail, aim for maximum speed. They rely on shared edge infrastructure and differentiate environments through routing.</p><p>This approach typically requires:</p><ul><li>Shared CloudFront configuration with a common domain name</li><li>Path-based routing at ALB</li><li>Frontend support for runtime basePath and assetPrefix changes</li><li>Multiple API Gateway stages mapped to environment identifiers</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WGeFZyRtJCOYJmxacE-5Jw.png" /><figcaption>Note: Internal API design is irrelevant here</figcaption></figure><p>When these conditions are met, spinning up a new environment becomes little more than adding routing rules and deploying service-specific resources. Cleanup is trivial, and costs remain largely constant regardless of the number of environments. Neither CloudFront nor the ALB needs to be aware of how many environments exist or may exist in the future.</p><h4>Semi-static on-demand environments</h4><p>Some frontends (e.g., Next.js) require environment-specific configuration at build time and cannot easily adapt to runtime path changes. In these cases, a semi-static approach is often necessary.</p><p>Here, each environment may have:</p><ul><li>Shared CloudFront configuration with alternate domain names and static Route 53 records</li><li>Shared ALB with host-based routing</li><li>Any API Gateway configuration implementation</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*f9MPXQbvQ-jtaL4HctL44w.png" /><figcaption>Note: Internal API design is irrelevant here</figcaption></figure><p>In contrast to the ephemeral environments option, this model requires CloudFront and Route 53 to be prepared upfront. Each environment must be explicitly represented through distribution configuration and DNS records, making environments longer-lived and less dynamic by design.</p><h4>Practical limits and quotas</h4><p>In practice, hard service quotas are rarely the first blocker. More often, provisioning time and operational friction become limiting factors as environments scale.</p><p>That said, quotas do influence how far reuse patterns can go and whether a limit can be raised or must be designed around matters. API Gateway REST APIs support a limited number of stages per API (10 by default, soft limit), which constrains stage-based environment reuse. Private API Gateway custom domain names are also subject to soft limits. On the load-balancing side, an ALB supports up to 100 listener rules (soft limit) and 100 target groups (hard limit), the latter requiring an architectural change.</p><p>For example, reusing a single ALB across ten environments can consume listener rules perhaps not so quickly, while a single API Gateway with one stage per environment approaches its stage limit fast. These constraints don’t prevent reuse, but they highlight the need to understand early which limits can be adjusted and which require rethinking the design.</p><h3>Conclusion</h3><p>The patterns described in this post are not a single fixed architecture, but a <strong>set of reuse-oriented edge and routing strategies</strong> for serverless platforms built on CloudFront, ALB, and API Gateway. Their value lies in being applied <strong>selectively</strong>.</p><p>For <strong>lower environments</strong> — feature, preview, and integration stages — reusing edge components and differentiating environments through routing enables fast provisioning, easy teardown, and predictable costs. These environments benefit most from shared CloudFront distributions, shared ALBs, and carefully structured API Gateway reuse, where speed and feedback outweigh the need for strict isolation.</p><p><strong>Upper environments are different</strong>. Staging, pre-production, and production typically require:</p><ul><li>Standalone, fully isolated infrastructure</li><li>Right-sized capacity and scaling characteristics</li><li>Stable DNS and edge configuration</li><li>The ability to run meaningful load, stress, and performance tests against an environment that truly behaves like production</li></ul><p>In those environments, reuse at the edge is usually inappropriate. Dedicated CloudFront distributions, ALBs, and API Gateways are not an optimisation failure — they are a requirement.</p><p>Supporting both approaches within the same platform does introduce additional complexity, particularly in infrastructure-as-code. Designing IaC that remains DRY while allowing some environments to share edge infrastructure and others to be fully isolated requires clear layering, strong conventions, and explicit ownership boundaries. This is more demanding than a uniform setup, but it is entirely achievable with deliberate design.</p><p>These patterns optimise for:</p><ul><li>Fast feedback and experimentation where environment churn is high</li><li>Independent service evolution with controlled blast radius</li><li>Predictable cost and provisioning time at scale</li><li>The ability to evolve toward stricter isolation when needed</li></ul><p>They are not optimised for:</p><ul><li>Maximum isolation across all environments</li><li>Treating edge infrastructure as immutable per environment</li><li>Eliminating all shared components</li></ul><p>Being explicit about where reuse applies — and where it must not — allows the platform to move quickly without compromising correctness. The goal is not to reuse everything, but to reuse the <em>right</em> things, in the <em>right</em> places, for the <em>right</em> reasons.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*6QbiWIxNcUuOSh7U.png" /></figure><h4>👋 If you find this helpful, please click the clap 👏 button below a few times to show your support for the author 👇</h4><h4>🚀<a href="http://from.faun.to/r/8zxxd">Join FAUN.dev() &amp; get similar stories in your inbox each week for free!</a></h4><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5d21adb64478" width="1" height="1" alt=""><hr><p><a href="https://faun.pub/reusing-cloudfront-alb-and-api-gateway-in-a-serverless-platform-5d21adb64478">Reusing CloudFront, ALB, and API Gateway in a Serverless Platform</a> was originally published in <a href="https://faun.pub/">FAUN.dev() 🐾</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AWS CodeBuild-powered GitHub Actions self-hosted runners — without webhooks]]></title>
            <link>https://faun.pub/aws-codebuild-powered-github-actions-self-hosted-runners-without-webhooks-95a8a488a9a1?source=rss-3abdb2452418------2</link>
            <guid isPermaLink="false">https://medium.com/p/95a8a488a9a1</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[webhooks]]></category>
            <category><![CDATA[self-hosted-runner]]></category>
            <category><![CDATA[github-actions]]></category>
            <category><![CDATA[codebuild]]></category>
            <dc:creator><![CDATA[Sebastian Mincewicz]]></dc:creator>
            <pubDate>Sun, 25 Jan 2026 21:13:09 GMT</pubDate>
            <atom:updated>2026-01-25T21:13:09.138Z</atom:updated>
            <cc:license>https://creativecommons.org/licenses/by-sa/4.0/</cc:license>
            <content:encoded><![CDATA[<h3>AWS CodeBuild-powered GitHub Actions self-hosted runners — without webhooks</h3><p>This topic may sound familiar, but this post intentionally goes beyond what you’ll find in AWS documentation or official blog posts.</p><p>The goal is to <strong>avoid webhooks</strong> and instead achieve maximum flexibility when using GitHub Actions for CI/CD with AWS CodeBuild-powered, ephemeral runners <strong>only when they are actually needed</strong>, while continuing to rely on GitHub-hosted runners for everything else.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*a4cLG0lRcNduHXKrY65zAQ.png" /></figure><h3>Webhooks — one-way door</h3><p>I’m not saying webhooks are wrong. If you have a clear reason to use them, they can be a good fit. However, in practice, they often become a one-way route that reduces flexibility — and I strongly prefer fit-for-purpose solutions.</p><p>Relevant AWS documentation:</p><ul><li><a href="https://docs.aws.amazon.com/codebuild/latest/userguide/action-runner-overview.html">https://docs.aws.amazon.com/codebuild/latest/userguide/action-runner-overview.html</a></li><li><a href="https://aws.amazon.com/blogs/devops/aws-codebuild-managed-self-hosted-github-action-runners/">https://aws.amazon.com/blogs/devops/aws-codebuild-managed-self-hosted-github-action-runners/</a></li></ul><p>The main issue with the webhook option is that you subscribe to specific GitHub event types, which effectively pushes you toward using CodeBuild for <em>all</em> jobs — unless you introduce increasingly complex filters and workflow logic.</p><p>At that point, your GitHub Actions configuration starts encoding infrastructure decisions, which is rarely ideal.</p><h3>On-demand &amp; ad-hoc runners — flexibility</h3><p>Instead, there’s a way to make CodeBuild-powered self-hosted runners available <strong>explicitly</strong> for the workflows that need them — and only when they’re actually required.</p><p>The idea is to start a CodeBuild project <strong>on demand</strong>, scoped to a specific GitHub Actions workflow run, and configured so it can only be used by that run. This avoids clashes, ghost runs, or unintended usage across repositories within your GitHub organisation.</p><p>Consider a setup where:</p><ul><li>some workflows only build artifacts,</li><li>others run unit tests or static analysis,</li><li>others perform deployments.</li></ul><p>Most of these can run perfectly fine on GitHub-hosted runners. However, there are cases where that breaks down:</p><ul><li>running tests against IP-allowlist-protected public endpoints,</li><li>running tests against private endpoints accessible only from within a custom VPC.</li></ul><p>In the first case, some teams attempt to add GitHub runners’ public IP ranges to allowlists. This creates a <strong>false sense of security</strong>, as those endpoints remain accessible to a large, shared address space.</p><p>In the second case — private endpoints — it’s simply a hard stop.</p><p>So how do you bring both worlds together and cover all of these use cases?</p><h4>Architecture overview</h4><p>In this architecture, a GitHub App is created with its associated private key, enabling secure authentication with one or more repositories. AWS CodeBuild leverages this identity to generate temporary tokens and register itself dynamically as an ephemeral GitHub Actions runner.</p><blockquote>This approach builds on the standard GitHub → AWS OIDC integration, as documented <a href="https://docs.github.com/en/actions/how-tos/secure-your-work/security-harden-deployments/oidc-in-aws">here</a>.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*D_8nYemzMrdhdclvsgf6rg.png" /><figcaption>HLD</figcaption></figure><h4>GitHub App</h4><p>A key part of making this pattern robust and production-ready is how the GitHub runner registration token is obtained. While many examples rely on a Personal Access Token (PAT), using a <strong>GitHub App</strong> is the more appropriate choice for team and organisation-level setups.</p><p>A GitHub App can be installed on <strong>specific repositories only</strong>, which aligns well with the idea of tightly scoped, purpose-built runners. This ensures that a CodeBuild-provisioned runner can only ever register against repositories you explicitly allow, reducing both blast radius and the risk of accidental reuse across the organisation.</p><p>The permission model is also much cleaner than with PATs. At a minimum, the GitHub App needs:</p><ul><li>read access to code and metadata</li><li>read and write access to administration</li></ul><p>No organisation-wide permissions are required, and the short-lived installation access tokens generated by the App naturally fit the ephemeral runner lifecycle.</p><blockquote>For <strong>personal projects</strong>, prototypes, or one-off experiments, a <strong>fine-grained or classic PAT</strong> can still be a perfectly acceptable and much simpler option. It avoids the additional setup overhead of a GitHub App and is often “good enough” when the scope and risk are limited.</blockquote><h4>AWS CodeBuild project &amp; buildspec</h4><p>From the CodeBuild project perspective, it really doesn’t get any simpler. The project does not need to be configured with a source repository at all — the buildspec can remain fully generic and work for any repository within your GitHub organisation.</p><p>The snippet below assumes the CodeBuild project is configured to use Amazon Linux running on Graviton-powered infrastructure.</p><pre># REQUIRED INPUT:<br># - GITHUB_ORG (from IaC)<br># - GITHUB_APP_ID (from IaC)<br># - GITHUB_APP_INSTALLATION_ID (from IaC)<br># - GITHUB_APP_PK_ASM_SECRET_ARN (from IaC)<br># - GITHUB_REPO (from GHA workflow)<br># - GITHUB_ACTIONS_RUNNER_NAME (from GHA workflow)<br><br>version: 0.2<br><br>env:<br>  variables:<br>    USER_HOME_DIR: &quot;/home/runneruser&quot;<br>  secrets-manager:<br>    GITHUB_APP_PK: &quot;${GITHUB_APP_PK_ASM_SECRET_ARN}&quot;<br><br>phases:<br>  install:<br>    commands:<br>      - |<br>        echo &quot;&gt; Creating runner user...&quot;<br>        useradd -m runneruser -d $USER_HOME_DIR<br>        echo &quot;runneruser ALL = NOPASSWD:/usr/bin/yum&quot; &gt;&gt; /etc/sudoers # &lt;- consider when need to install deps on the fly<br><br>        echo &quot;&gt; Downloading the lastest runner installation package...&quot;<br>        cd $USER_HOME_DIR<br>        RUNNER_VERSION=$(curl -s https://api.github.com/repos/actions/runner/tags | jq -r &#39;.[0].name&#39; | sed &#39;s/^v//&#39;)<br>        echo &quot;Latest tag: $RUNNER_VERSION&quot;<br>        curl -Ls https://github.com/actions/runner/releases/download/v${RUNNER_VERSION}/actions-runner-linux-arm64-${RUNNER_VERSION}.tar.gz -o actions-runner.tar.gz<br>        mkdir actions-runner &amp;&amp; tar xzf actions-runner.tar.gz -C actions-runner<br>        chown -R runneruser:runneruser .<br><br>  build:<br>    commands:<br>      - |<br>        echo &quot;&gt; Generating GitHub App JWT (header + payload)...&quot;<br>        cd $USER_HOME_DIR/actions-runner<br>        now=$(date +%s)<br>        iat=$((${now} - 60))  # Issues 1 miute in the past<br>        exp=$((${now} + 600)) # Expires 10 minutes in the future<br><br>        b64enc() { openssl base64 | tr -d &#39;=&#39; | tr &#39;/+&#39; &#39;_-&#39; | tr -d &#39;\n&#39;; }<br><br>        header_json=&#39;{<br>            &quot;typ&quot;:&quot;JWT&quot;,<br>            &quot;alg&quot;:&quot;RS256&quot;<br>        }&#39;<br>        # Header encode<br>        header=$(echo -n &quot;${header_json}&quot; | b64enc)<br><br>        payload_json=&quot;{<br>            \&quot;iat\&quot;:${iat},<br>            \&quot;exp\&quot;:${exp},<br>            \&quot;iss\&quot;:\&quot;${GITHUB_APP_ID}\&quot;<br>        }&quot;<br>        # Payload encode<br>        payload=$( echo -n &quot;${payload_json}&quot; | b64enc )<br><br>        # Signature<br>        header_payload=&quot;${header}&quot;.&quot;${payload}&quot;<br>        signature=$(<br>            openssl dgst -sha256 -sign &lt;(echo -n &quot;${GITHUB_APP_PK}&quot;) \<br>            &lt;(echo -n &quot;${header_payload}&quot;) | b64enc<br>        )<br>        # Generate an installation token for the app<br>        JWT=&quot;${header_payload}&quot;.&quot;${signature}&quot;<br><br>        echo &quot;&gt; Requesting GitHub App installation access token...&quot;<br>        INSTALLATION_TOKEN=$(curl --request POST \<br>            --url &quot;https://api.github.com/app/installations/${GITHUB_APP_INSTALLATION_ID}/access_tokens&quot; \<br>            --header &quot;Accept: application/vnd.github+json&quot; \<br>            --header &quot;Authorization: Bearer ${JWT}&quot; \<br>            --header &quot;X-GitHub-Api-Version: 2022-11-28&quot;  \<br>          | jq -r &#39;.token&#39;<br>        )<br><br>        echo &quot;&gt; Requesting ephemeral runner registration token...&quot;<br>        GITHUB_RUNNER_TOKEN=$(curl --request POST \<br>            --url &quot;https://api.github.com/repos/${GITHUB_ORG}/${GITHUB_REPO}/actions/runners/registration-token&quot; \<br>            --header &quot;Accept: application/vnd.github+json&quot; \<br>            --header &quot;Authorization: Bearer ${INSTALLATION_TOKEN}&quot; | jq -r &#39;.token&#39;<br>        )<br><br>        echo &quot;&gt; Configuring GitHub Actions runner for ${GITHUB_ORG}/${GITHUB_REPO} ...&quot;<br>        su runneruser -c &quot;./config.sh \<br>          --url https://github.com/${GITHUB_ORG}/${GITHUB_REPO} \<br>          --token ${GITHUB_RUNNER_TOKEN} \<br>          --unattended --ephemeral \<br>          --name ${GITHUB_ACTIONS_RUNNER_NAME} \<br>          --labels self-hosted,${GITHUB_ACTIONS_RUNNER_NAME}&quot;<br><br>        echo &quot;&gt; Starting runner...&quot;<br>        su runneruser -c &quot;./run.sh&quot;</pre><p>If you wish you can build a custom image and that way get rid of the install phase entirely.</p><h4>GitHub Actions workflow</h4><p>Below is a workflow snippet that completes the configuration and shows how everything fits together.</p><pre>jobs:<br>  start-cb-runner:<br>    runs-on: ubuntu-latest<br>    outputs:<br>      runner_name: ${{ steps.start-cb-project.outputs.runner_name }}<br>    steps:<br>      - name: Configure AWS Credentials<br>        uses: aws-actions/configure-aws-credentials@v5<br>        with:<br>          audience: sts.amazonaws.com<br>          aws-region: ${{ env.AWS_REGION }}<br>          role-to-assume: ${{ env.AWS_ROLE_ARN }}<br>          role-session-name: GithubActionsSession<br><br>      - name: Start CodeBuild project<br>        id: start-cb-project<br>        run: |<br>          RUNNER_NAME=&quot;cb-runner-${GITHUB_RUN_ID}-${GITHUB_SHA::8}&quot;<br>          CODEBUILD_PROJECT_NAME=&quot;${{ env.CODEBUILD_PROJECT_NAME }}&quot;<br>          echo &quot;runner_name=$RUNNER_NAME&quot; &gt;&gt; $GITHUB_OUTPUT<br>          PAYLOAD=$(jq -n \<br>            --arg project &quot;$CODEBUILD_PROJECT_NAME&quot; \<br>            --arg gh-repo &quot;${{ github.event.repository.name }}&quot; \<br>            --arg gha-runner-name &quot;$RUNNER_NAME&quot; \<br>            &#39;{<br>              projectName: $project,<br>              environmentVariablesOverride: [<br>                {name: &quot;GITHUB_REPO&quot;, value: $gh-repo, type: &quot;PLAINTEXT&quot;},<br>                {name: &quot;GITHUB_ACTIONS_RUNNER_NAME&quot;, value: $gha-runner-name, type: &quot;PLAINTEXT&quot;}<br>              ]<br>            }&#39;)<br>          aws codebuild start-build --cli-input-json &quot;$PAYLOAD&quot; &gt; /dev/null<br><br>  run-tests-on-cb-runner:<br>    needs: start-cb-runner<br>    runs-on: [self-hosted, &quot;${{ needs.start-cb-runner.outputs.runner_name }}&quot;]<br>    steps:<br>      - uses: actions/checkout@v6<br>      - name: Introduction message<br>        run: |<br>          echo &quot;Testing ...&quot;</pre><p>From a performance perspective, bringing a runner online should take <strong>no more than ~30 seconds</strong>, after which it is ready to pick up the queued job.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QTJugWqe0xTmqVk0AO3A5g.png" /><figcaption><em>GHA workflow view</em></figcaption></figure><p>One of the less obvious but critical elements is the RUNNER_NAME. In teams where multiple developers and testers run workflows in parallel, there is always a risk of workflows competing for runners. By generating a unique runner name per workflow run and passing it into the CodeBuild project, you guarantee that the runner you spin up is used exclusively for that specific execution and cannot be accidentally picked up by another job.</p><p>Finally, the CodeBuild project output should look similar to this:</p><pre>Configuring and starting runner for sebolabs/my-repo ...<br>--------------------------------------------------------------------------------<br>|        ____ _ _   _   _       _          _        _   _                      |<br>|       / ___(_) |_| | | |_   _| |__      / \   ___| |_(_) ___  _ __  ___      |<br>|      | |  _| | __| |_| | | | | &#39;_ \    / _ \ / __| __| |/ _ \| &#39;_ \/ __|     |<br>|      | |_| | | |_|  _  | |_| | |_) |  / ___ \ (__| |_| | (_) | | | \__ \     |<br>|       \____|_|\__|_| |_|\__,_|_.__/  /_/   \_\___|\__|_|\___/|_| |_|___/     |<br>|                                                                              |<br>|                       Self-hosted runner registration                        |<br>|                                                                              |<br>--------------------------------------------------------------------------------<br># Authentication<br>√ Connected to GitHub<br># Runner Registration<br>√ Runner successfully added<br># Runner settings<br>√ Settings Saved.<br>√ Connected to GitHub<br>Current runner version: &#39;2.331.0’<br>2026-01-19 20:34:07Z: Listening for Jobs<br>2026-01-19 20:34:09Z: Running job: run-tests-on-cb-runner<br>2026-01-19 20:34:17Z: Job run-tests-on-cb-runner completed with result: Succeeded<br>√ Removed .credentials<br>√ Removed .runner<br>Runner listener exit with 0 return code, stop the service, no retry needed.<br>Exiting runner...<br>[Container] 2026/01/19 20:34:17.683704 Phase complete: BUILD State: SUCCEEDED</pre><h3>Wrap-up</h3><p>Using AWS CodeBuild–powered, ephemeral GitHub Actions self-hosted runners without webhooks gives you precise, on-demand control over <em>when</em> and <em>why</em> you leave the GitHub-hosted runner pool. You retain full workflow flexibility, keep logs and execution visibility inside the GitHub Actions UI, and only incur additional infrastructure when a workflow genuinely requires network proximity to protected or private endpoints.</p><p>This model avoids over-coupling your CI/CD design to webhook-driven automation, reduces the risk of unintended runner usage, and scales cleanly even when many workflows are triggered in parallel. Treating CodeBuild runners as a specialised, opt-in execution environment — rather than a default — keeps both architecture and blast radius under control.</p><blockquote>Historically, self-hosted GitHub Actions runners were effectively <em>free from GitHub’s billing perspective</em> — you only paid for the infrastructure you ran them on. That changes on <strong>March 1, 2026</strong>, when GitHub will introduce a <strong>$0.002 per-minute GitHub Actions cloud platform charge</strong> for self-hosted runner usage, with those minutes counting toward your plan.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*fAgnozed-YG_aRAJ.png" /></figure><h4>👋 If you find this helpful, please click the clap 👏 button below a few times to show your support for the author 👇</h4><h4>🚀<a href="http://from.faun.to/r/8zxxd">Join FAUN.dev() &amp; get similar stories in your inbox each week for free!</a></h4><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=95a8a488a9a1" width="1" height="1" alt=""><hr><p><a href="https://faun.pub/aws-codebuild-powered-github-actions-self-hosted-runners-without-webhooks-95a8a488a9a1">AWS CodeBuild-powered GitHub Actions self-hosted runners — without webhooks</a> was originally published in <a href="https://faun.pub/">FAUN.dev() 🐾</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AWS Anywhere — a route to EKS Hybrid Nodes]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://faun.pub/aws-anywhere-a-route-to-eks-hybrid-nodes-ccc1ae703a41?source=rss-3abdb2452418------2"><img src="https://cdn-images-1.medium.com/max/1780/1*LCV_3T5HvyVHaVGom0zuPw.png" width="1780"></a></p><p class="medium-feed-snippet">Exploring the integration of on-prem systems with AWS for seamless hybrid Kubernetes environments, using AWS services and EKS Hybrid Nodes.</p><p class="medium-feed-link"><a href="https://faun.pub/aws-anywhere-a-route-to-eks-hybrid-nodes-ccc1ae703a41?source=rss-3abdb2452418------2">Continue reading on FAUN.dev()  »</a></p></div>]]></description>
            <link>https://faun.pub/aws-anywhere-a-route-to-eks-hybrid-nodes-ccc1ae703a41?source=rss-3abdb2452418------2</link>
            <guid isPermaLink="false">https://medium.com/p/ccc1ae703a41</guid>
            <category><![CDATA[eks-hybrid-nodes]]></category>
            <category><![CDATA[route-53-resolver]]></category>
            <category><![CDATA[iam-roles-anywhere]]></category>
            <category><![CDATA[aws-site-to-site-vpn]]></category>
            <category><![CDATA[ssm-agent]]></category>
            <dc:creator><![CDATA[Sebastian Mincewicz]]></dc:creator>
            <pubDate>Sun, 12 Jan 2025 22:00:54 GMT</pubDate>
            <atom:updated>2025-04-15T14:05:39.568Z</atom:updated>
            <cc:license>https://creativecommons.org/licenses/by-nc/4.0/</cc:license>
        </item>
        <item>
            <title><![CDATA[Databricks on AWS]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://faun.pub/databricks-on-aws-dffbc78477dd?source=rss-3abdb2452418------2"><img src="https://cdn-images-1.medium.com/max/747/1*SDksTc7kAyqx3WDRZCOOfQ.png" width="747"></a></p><p class="medium-feed-snippet">Platform-related lessons learned from building an enterprise data platform with Databricks on AWS.</p><p class="medium-feed-link"><a href="https://faun.pub/databricks-on-aws-dffbc78477dd?source=rss-3abdb2452418------2">Continue reading on FAUN.dev()  »</a></p></div>]]></description>
            <link>https://faun.pub/databricks-on-aws-dffbc78477dd?source=rss-3abdb2452418------2</link>
            <guid isPermaLink="false">https://medium.com/p/dffbc78477dd</guid>
            <category><![CDATA[data-platforms]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[databricks]]></category>
            <category><![CDATA[data-lakehouse]]></category>
            <category><![CDATA[aws-privatelink]]></category>
            <dc:creator><![CDATA[Sebastian Mincewicz]]></dc:creator>
            <pubDate>Tue, 30 Jul 2024 08:15:02 GMT</pubDate>
            <atom:updated>2024-08-02T09:58:16.096Z</atom:updated>
            <cc:license>https://creativecommons.org/licenses/by-nc/4.0/</cc:license>
        </item>
        <item>
            <title><![CDATA[Re-platforming to AWS Lambda container images]]></title>
            <link>https://faun.pub/re-platforming-to-aws-lambda-container-images-6c8a765e9428?source=rss-3abdb2452418------2</link>
            <guid isPermaLink="false">https://medium.com/p/6c8a765e9428</guid>
            <category><![CDATA[amazon-ecs]]></category>
            <category><![CDATA[migration]]></category>
            <category><![CDATA[replatforming]]></category>
            <category><![CDATA[aws-lambda]]></category>
            <category><![CDATA[amazon-eks]]></category>
            <dc:creator><![CDATA[Sebastian Mincewicz]]></dc:creator>
            <pubDate>Wed, 24 May 2023 21:19:26 GMT</pubDate>
            <atom:updated>2023-05-24T21:19:26.865Z</atom:updated>
            <cc:license>http://creativecommons.org/licenses/by/4.0/</cc:license>
            <content:encoded><![CDATA[<p>This is an attempt at how event-driven backend processing services can potentially be migrated with minimum changes from a container orchestration service to AWS Lambda powered by container images.<br>The driver for this story was one of my recent migration projects where a fleet of services running on an old EKS version had to be migrated to a brand-new environment running the latest versions of everything. During that journey, one of the weaknesses that got revealed in the architecture of the system in question was insufficient resistance to data loss due to the lack of loose coupling capabilities. That made me think… firstly, how a particular backend processing service can be redesigned accordingly, and secondly, whether we need EKS for this at all.<br>For the sake of this story let’s say the former has already been solved and so I’m going to focus on the latter as one of the exercises I did was checking how difficult <strong>re-platforming</strong> <strong>a containerized service</strong> <strong>from Amazon ECS or Amazon EKS to AWS Lambda with container images support</strong> is.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VbObf5C0JPM76jI2K3-iqw.png" /><figcaption><a href="https://docs.aws.amazon.com/lambda/latest/dg/images-create.html">Lambda container images</a></figcaption></figure><h3>EKS, ECS, and Lambda</h3><p>It’s important to emphasize again that I’m focusing on an event-driven backend processing service which is why this <em>comparison</em> is focused on a specific, but still common, use case rather than a holistic capabilities overview of these services. EKS, ECS, and Lambda are not mutually exclusive and each of these services is better at one thing while less optimal at the other. It’s usually a game of trade-offs anyway but depending on a scenario, workload type, number of services in an environment, etc., to come up with the best architecture those services from AWS can be combined or implemented interchangeably. My observations show that Lambda is barely considered whenever containerized workloads are in use, if at all. These days, containers usually equal Kubernetes.</p><h4>Event-driven architecture</h4><p>Conceptually, I’m a big fan of this type of architecture. Do only when there’s something to do and upon getting notified about it. Once done, turn off the lights and wait for another alarm to be woken up. Repeat forever. It’s efficient, sustainable, and cost-effective.<br>While Lambda along with other serverless services is designed to be at the heart of event-driven solutions, both EKS and ECS can be used to deal with such requirements, however, not always efficiently enough and not without additional overheads or complexity associated.</p><h4>Scaling</h4><p>One of the crucial capabilities of a service that is used to support event-driven architecture is dynamic and adaptive scaling. Lambda is known for its native scaling capabilities and handling of in-flight requests in parallel that can be controlled and supported with the use of the concurrency settings that were comprehensively described <a href="https://docs.aws.amazon.com/lambda/latest/dg/lambda-concurrency.html">here</a>. A Lambda function can be triggered by the majority of relevant AWS services, if not all of them, and therefore perfectly adapts to unpredictable traffic events.<br>In the world of Kubernetes, with EKS, there are add-ons (autoscalers) like <em>Keda</em> that enable similar capabilities with the use of supported <a href="https://keda.sh/docs/latest/scalers/">scalers</a> and allow for pod scaling based on external metrics.<br>ECS on the other hand supports so-called <a href="https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-autoscaling-stepscaling.html">step scaling</a> that leverages CloudWatch alarms and adjusts the number of service tasks in steps based on the size of the alarm breach. Interestingly, that type of scaling is <em>not recommended</em> as per the ECS Developer Guide which is why I decided to give it a try as a part of a PoC for this story. An alternative approach that I’ve seen people implementing is a Lambda function triggered periodically that watches a given external metric and based on its value manipulates the number of running ECS tasks. Even then, it’s not flexible and automatic as thresholds must be predefined.</p><h3>Migration, modernization, re-platforming</h3><p>While migrating more and more workloads to the cloud remains high on the list of initiatives organizations want to make progress on, according to <a href="https://www.flexera.com/about-us/press-center/flexera-2023-state-of-the-cloud-report"><em>Flexera 2023 State of the Cloud Report</em></a> cost savings is the top one for the seventh year in a row. Moreover, 71% of heavy cloud users want to optimize the existing use of the cloud.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wsI0YD-cmaGGzyNWPekA1Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GZKWnmT-1Q4V_9YRFfIteQ.png" /></figure><p>At the same time, cloud spend along with security and expertise are the top three cloud challenges recognized.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*n_RQl7yQTcppoGBXUku-dw.png" /></figure><p>Therefore, a potential path leading to cost optimization that simultaneously can elevate security and doesn’t necessarily require extensive expertise is modernizing with the use of AWS Lambda.<br>Depending on where you are with your cloud adoption journey and what the key business drivers are, you should choose the most appropriate migration strategy for your workloads. <strong>Modernization</strong> is something you can go for straight away or consider as the next step after re-hosting first.<br><strong>Re-platforming</strong> can turn out to be a golden mean as it allows for reshaping by leveraging cloud-native capabilities <strong>without modifying the application source code or its core architecture</strong>. That way you can benefit from increased flexibility and resilience with reduced time and financial investments. Obviously, it doesn’t mean it is <em>the way</em> and any decision on implementing any strategy should be supported by relevant analysis.</p><h3>PoC: ECS to Lambda</h3><p>Let’s imagine a scenario where the goal is to re-platform from ECS to Lambda with zero or minimal changes to:</p><ul><li>application code (<em>Python</em>)</li><li>container image build process (<em>Dockerfile</em>)</li></ul><p>to reduce risks associated with the platform change itself.<br>Why ECS and not EKS? Because for some reason I haven’t got much experience with it hence considered this a chance to see what it’s capable of in the given scenario and let myself expand my horizons.</p><h4>HLD</h4><p>It’s about re-platforming from this…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-eyHEKpDYmJjYgN6RoE-QA.png" /></figure><p>to this…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Sh8lsCzkeheKfiv7C2awcA.png" /></figure><h4>Application code</h4><p>What this sample code below does is it receives messages from a given SQS queue and deletes them immediately. That’s it, just enough for this PoC.</p><pre># app.py<br>import os<br>import boto3<br><br>QUEUE_URL = os.environ.get(&quot;SQS_QUEUE_URL&quot;, &quot;&quot;)<br>MAX_NUM_MSGS = int(os.environ.get(&quot;SQS_MAX_NUM_MSGS&quot;, 10))<br>VISIBILITY_TO = int(os.environ.get(&quot;SQS_VISIBILITY_TO&quot;, 15))<br>WAIT_SECONDS = int(os.environ.get(&quot;SQS_WAIT_SECONDS&quot;, 10))<br><br>sqs_client = boto3.client(&quot;sqs&quot;)<br><br><br>def handler(event, context):<br>    get_response = sqs_client.receive_message(<br>        QueueUrl=QUEUE_URL,<br>        MaxNumberOfMessages=MAX_NUM_MSGS,<br>        VisibilityTimeout=VISIBILITY_TO,<br>        WaitTimeSeconds=WAIT_SECONDS,<br>    )<br><br>    messages = get_response.get(&quot;Messages&quot;, [])<br><br>    messages_length = len(messages)<br>    print(f&quot;Number of messages received: {messages_length}&quot;)<br><br>    for msg in messages:<br>        print(f&quot;Message body: {msg[&#39;Body&#39;]}&quot;)<br><br>        del_response = sqs_client.delete_message(<br>            QueueUrl=QUEUE_URL, ReceiptHandle=msg[&quot;ReceiptHandle&quot;]<br>        )<br>        print(<br>            f&quot;Deletion status code: {del_response[&#39;ResponseMetadata&#39;][&#39;HTTPStatusCode&#39;]}&quot;<br>        )<br><br><br>if __name__ == &quot;__main__&quot;:<br>    handler(&quot;&quot;, &quot;&quot;)</pre><h4>Dockerfile &amp; RIC</h4><p>The only thing that had to be added to the existing Dockerfile, for Lambda to work with a container image, was installing the <strong>AWS runtime interface client</strong> (RIC) for Python — one additional line still keeping the image <em>generic</em>.</p><pre>FROM python:3.10<br><br>WORKDIR /app<br><br>COPY requirements.txt  .<br>RUN pip install -r requirements.txt<br><br><strong>RUN pip install awslambdaric</strong><br><br>COPY app.py .<br><br>CMD [&quot;python&quot;, &quot;app.py&quot;]</pre><p>RIC is an implementation of the Lambda runtime API that allows for extending a base image to become Lambda-compatible and is represented by an interface that enables receiving and sending requests from and to the AWS Lambda service. While it can be installed for Python, Node.js, Go, Java, .NET, and Ruby, AWS also delivers Amazon Linux-powered container images for those runtimes <a href="https://gallery.ecr.aws/lambda/">here</a>. Even though they might be bigger than standard (<em>slim</em>) base images they are pro-actively cached by the Lambda service meaning they don’t have to be pulled entirely every time.</p><h4>Lambda function configuration</h4><p>From the Lambda function configuration perspective, to instruct Lambda how to run the application code both <em>ENTRYPOINT</em> and <em>CWD</em> settings must be overridden as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/893/1*rJHFmPI4jGlMUSDrJ3Ilcw.png" /></figure><p>And that’s it. It’s ready to be invoked.</p><h4>Parallel testing</h4><p>For that purpose, I have used the SQS queue seeder Python code below and ran it from Lambda. Based on the settings provided, it generates random strings and sends them as messages to as many queues as you want, in this case, there were two, one for ECS and another one for Lambda.</p><pre># sqs_seeder.py<br>import os<br>import random<br>import time<br>import string<br>import boto3<br><br># SQS queues<br>SQS_QUEUE_URLS=[<br>    # queue for ECS,<br>    # queue for Lambda,<br>]<br><br># Settings<br>NUMBER_OF_MSG_BATCHES = 50<br>MAX_MSGS_PER_BATCH = 5<br>MIN_SECONDS_BETWEEN_MSG_BATCHES = 5<br>MAX_SECONDS_BETWEEN_MSG_BATCHES = 15<br><br>sqs_client = boto3.client(&quot;sqs&quot;)<br><br><br>def sqs_send_msg(msgs_count):<br>    MSGS_COUNT = msgs_count<br><br>    print(f&quot;[INFO] Number of messages in batch: {MSGS_COUNT}&quot;)<br><br>    while MSGS_COUNT &gt; 0:<br>        MESSAGE_CHAR_SIZE = random.randint(10, 50)<br>        MESSAGE = &quot;&quot;.join(random.choices(string.ascii_uppercase, k=MESSAGE_CHAR_SIZE))<br><br>        for queue_url in SQS_QUEUE_URLS:<br>            try:<br>                print(f&quot;[INFO] Sending message {MESSAGE} to {queue_url}&quot;)<br>                response = sqs_client.send_message(<br>                    QueueUrl=queue_url,<br>                    MessageBody=MESSAGE,<br>                )<br>                print(f&quot;[INFO] Message sent, ID: {response[&#39;MessageId&#39;]}&quot;)<br>            except Exception as error:<br>                print(f&quot;[ERROR] Queue URL: {error}&quot;)<br><br>        MSGS_COUNT -= 1<br><br><br>def handler(event, context):<br>    global NUMBER_OF_MSG_BATCHES<br><br>    print(f&quot;[INFO] Number of message batches: {NUMBER_OF_MSG_BATCHES}&quot;)<br><br>    BATCH_NUMBER = 1<br><br>    while NUMBER_OF_MSG_BATCHES &gt; 0:<br>        print(f&quot;[INFO] Batch number #{BATCH_NUMBER}&quot;)<br><br>        NUMBER_OF_MSGS = random.randint(1, MAX_MSGS_PER_BATCH)<br>        sqs_send_msg(NUMBER_OF_MSGS)<br><br>        BATCH_NUMBER += 1<br>        NUMBER_OF_MSG_BATCHES -= 1<br><br>        if NUMBER_OF_MSG_BATCHES &gt; 0:<br>            SLEEP_TIME = random.randint(MIN_SECONDS_BETWEEN_MSG_BATCHES, MAX_SECONDS_BETWEEN_MSG_BATCHES)<br>            print(f&quot;[INFO] Sleeping {SLEEP_TIME} seconds...&quot;)<br>            time.sleep(SLEEP_TIME)<br><br><br>if __name__ == &quot;__main__&quot;:<br>    handler(&quot;&quot;, &quot;&quot;)</pre><p>Here are the ECS service step scaling policies configured that didn’t do what I initially thought they would do but more on this later.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LIFDUNVk-9dv2CmcfF9rDQ.png" /></figure><p>The Lambda function was configured with reserved concurrency set to 1 to try to give ECS a head start.</p><p>The screenshot below shows relevant metrics illustrating how messages were being loaded, received, and deleted from the two individual queues.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*l95hVjjW1tph3P-PitKOPg.png" /><figcaption>ECS vs. Lambda</figcaption></figure><p>Clearly, something wasn’t right about ECS step scaling. It’s just not what I was hoping for. I thought it would be continuously adding 5 tasks as the number of messages in the queue grows but that wasn’t the case. According to the <strong><em>scale-out</em></strong> action, for the second run, I expected to get 15 tasks while the <em>DesiredTaskCount</em> remained at the level of 5, so it looks like a one-off scaling operation that is not adaptive over time. I admit I got misled by the <em>Set/Add</em> actions available there. I should have done better at reading and managing my expectations. Now I see why it wasn’t recommended.<br>Either way, it is also crystal clear when looking at the metrics that Lambda scaling is seamless and fast while ECS needs time before the CloudWatch alarm kicks in. That is because for AWS metrics the minimum period is 1 minute and so the reaction is not immediate.<br>Finally, not sure why but the <strong><em>scale-in</em></strong> action didn’t work, namely, it did not set the number of tasks to 0 even though the alarm history was showing that the action was successfully executed. I had to do it by hand.</p><p>When it comes to SQS itself, I learned one thing that seems important from the performance and efficiency point of view. Namely, having the <em>MaxNumberOfMessages</em> set to 10 (max) for the ReceiveMessage API call when the number of messages is not big enough, it will still return a single message most of the time. More on that <a href="https://github.com/boto/boto3/issues/324">here</a>.</p><h3>Wrap-up</h3><p>While I had no intention of going deep into ECS step scaling and demystifying things, even if it worked the way I expected it would still perform worse than Lambda. However, <strong>this is not about judging but about knowing your options and their constraints</strong>. It is also about realizing caveats relevant to the integration points and their limitations too. While one service may seem to be fit for purpose, there might be something else that will eventually have a decisive impact on the final design.<br>The takeaway is that thanks to containers&#39; portability such a comparison doesn’t necessarily have to be difficult to execute or requires much effort to draw some conclusions that will help in decision-making. A Lambda function powered by a container image is probably one of the easiest things to configure and so as long as there are no obvious obstacles, quite the opposite, when there are indications of potential improvement, why wouldn’t you give it a chance? The worst thing that can happen is that you’ll end up with additional arguments for your decision.<br>Again, know your options! Don’t hesitate to try things out by running PoCs to find the best solution and remember that modernization is a continuous improvement process that never ends. Technology, business requirements, functionalities, and KPIs keep on changing over time, therefore, try to continuously assess your solutions against the AWS Well-Architected Framework and strive for optimization.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*FiWXXMOKpb27EiJ8.png" /></figure><h4>👋 If you find this helpful, please click the clap 👏 button below a few times to show your support for the author 👇</h4><h4>🚀<a href="http://from.faun.to/r/8zxxd">Join FAUN Developer Community &amp; Get Similar Stories in your Inbox Each Week</a></h4><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6c8a765e9428" width="1" height="1" alt=""><hr><p><a href="https://faun.pub/re-platforming-to-aws-lambda-container-images-6c8a765e9428">Re-platforming to AWS Lambda container images</a> was originally published in <a href="https://faun.pub/">FAUN.dev() 🐾</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Amazon VPC Lattice — feasibility study]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://faun.pub/amazon-vpc-lattice-feasibility-study-c53d43c86a96?source=rss-3abdb2452418------2"><img src="https://cdn-images-1.medium.com/max/768/1*IA72IkGbmiv_17Vo-o4o1Q.png" width="768"></a></p><p class="medium-feed-snippet">A deep dive into Amazon VPC Lattice functional capabilities backed up by a PoC to validate expectations, reveal caveats and make an&#x2026;</p><p class="medium-feed-link"><a href="https://faun.pub/amazon-vpc-lattice-feasibility-study-c53d43c86a96?source=rss-3abdb2452418------2">Continue reading on FAUN.dev()  »</a></p></div>]]></description>
            <link>https://faun.pub/amazon-vpc-lattice-feasibility-study-c53d43c86a96?source=rss-3abdb2452418------2</link>
            <guid isPermaLink="false">https://medium.com/p/c53d43c86a96</guid>
            <category><![CDATA[application-load-balancer]]></category>
            <category><![CDATA[lambda]]></category>
            <category><![CDATA[vpc-lattice]]></category>
            <category><![CDATA[api-gateway]]></category>
            <category><![CDATA[service-mesh]]></category>
            <dc:creator><![CDATA[Sebastian Mincewicz]]></dc:creator>
            <pubDate>Wed, 26 Apr 2023 12:52:38 GMT</pubDate>
            <atom:updated>2023-04-26T12:52:38.163Z</atom:updated>
            <cc:license>https://creativecommons.org/licenses/by-sa/4.0/</cc:license>
        </item>
        <item>
            <title><![CDATA[AWS EKS with Terraform and GitOps in minutes]]></title>
            <link>https://faun.pub/aws-eks-with-terraform-and-gitops-in-minutes-b3ca33171209?source=rss-3abdb2452418------2</link>
            <guid isPermaLink="false">https://medium.com/p/b3ca33171209</guid>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[argo-cd]]></category>
            <category><![CDATA[gitops]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[amazon-eks]]></category>
            <dc:creator><![CDATA[Sebastian Mincewicz]]></dc:creator>
            <pubDate>Mon, 28 Nov 2022 07:17:10 GMT</pubDate>
            <atom:updated>2023-02-02T13:48:57.852Z</atom:updated>
            <cc:license>http://creativecommons.org/licenses/by/4.0/</cc:license>
            <content:encoded><![CDATA[<h3>Amazon EKS with Terraform and GitOps in minutes</h3><p>This one is simply a result of a need that I had and that was about getting a <strong><em>fully functional, flexible, and secure Amazon EKS cluster</em></strong> set up in under half an hour to be able to test anything asap. For that, I did not want to spend too much time developing IaC myself as there are so many great sources out there that are worth supporting rather than reinventing the wheel. The force is there in the community and as an AWS Community Builder I came across something that met my expectations hence I’m sharing my experience hoping you may find it helpful too.</p><blockquote>It is meant to get you your EKS cluster while you can go buy yourself a coffee ☕️</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fAd1UWVerpgoIxzUR9Fe9g.png" /></figure><p>This time I will start the other way around and go straight away to the solution while context and other details can be found down below.<br>The only thing to reveal at this stage is that I’m leveraging <a href="http://aws-ia.github.io/terraform-aws-eks-blueprints">Amazon EKS Blueprints for Terraform</a> 🚀</p><h3>MVP</h3><p>While one can use the flexibility of the EKS Blueprints solution to set things up in many different ways and depending on individual requirements, I’ve got the minimal/initial configuration I start with, and that consists of the following:</p><ul><li>the control plane with <em>whitelisted</em> public access,</li><li>the data plane (spot EC2 instances) communicating with the control plane privately,</li><li>all EKS-managed add-ons enabled and using the most recent versions,</li><li>ArgoCD publicly accessible (<em>whitelisted</em>) through an ALB configured with a Route53 domain and an ACM certificate,</li><li>a set of additional add-ons deployed with the use of ArgoCD and following the GitOps approach.</li></ul><p>The following extra add-ons are enabled by default:</p><ul><li>Cluster autoscaler</li><li>AWS load balancer controller</li><li>External DNS</li><li>FluentBit</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xb1Z8r3fHSrM9zeBkjmh7g.jpeg" /></figure><p>Here’s the code that sets everything up.</p><p><a href="https://github.com/sebolabs/eks-tf-gitops/tree/release-1-0-0">GitHub - sebolabs/eks-tf-gitops at release-1-0-0</a></p><p>It’s opinionated, however, I believe it’s a perfect starting point where you have a fully functional Kubernetes cluster with GitOps support and can immediately start deploying and testing anything you want.</p><p>The Terraform code (/terraform) in this repo consists of three components:</p><ul><li><strong>account</strong> <em>(optional)</em> — covers S3 bucket for storing logs, etc.</li><li><strong>core</strong> — covers networking</li><li><strong>k8s</strong> — covers EKS cluster configuration</li></ul><p>In addition, there’s a K8s configuration (/k8s) covering add-ons that is periodically read by ArgoCD to keep things set up as declared in Git — the GitOps way. If you decide to use that code simply look for <em>TODOs</em> and provide values that will be relevant to your set-up.</p><p>Finally, after running Terraform and then going to get your well-deserved coffee…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MWQowto9kjjPkej2y8s6mw.png" /></figure><p>… it’s there, up and running!</p><p>It needs a couple of minutes to deploy the add-ons automatically, including the <strong><em>AWS Load Balancer controller </em></strong>and <strong><em>External DNS</em></strong> responsible for exposing the ArgoCD UI publicly.<br>Then, you just have to retrieve ArgoCD&#39;s initial admin password…</p><pre>$ kubectl -n argocd get secret argocd-initial-admin-secret \<br>    -o jsonpath=”{.data.password}” | base64 -d <br><br>n20x3mwZoapDv9JC</pre><p>…and you can log in 😎</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tZcbJa9ADNqJQKe6hWcaLg.png" /></figure><h4>App of Apps</h4><p>Now there’s that first ArgoCD application called <strong>add-ons</strong> where the enabled core K8s controllers belong to.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eNR7ORsae52vLXD7GXW9cw.png" /></figure><p>To learn more about the App of Apps pattern in ArgoCD check this <a href="https://argo-cd.readthedocs.io/en/stable/operator-manual/cluster-bootstrapping/">link</a>.</p><h4>Logging</h4><p>Apart from cluster logs that were enabled also the logs from all the pods get nicely delivered to CloudWatch and can be easily queried with Logs Insights. See some examples below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*e35R9A35KmMr2Kj-4fIQ4A.png" /><figcaption>AWS Load Balancer controller logs</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ngY61GiCOJ1Q7tI27vv_eA.png" /><figcaption>Cluster Autoscaler logs</figcaption></figure><h3>EKS Blueprints</h3><p>Now, let’s get to the roots of the solution…</p><blockquote>EKS Blueprints helps you compose complete EKS clusters that are fully bootstrapped with the operational software that is needed to deploy and operate workloads. With EKS Blueprints, you describe the configuration for the desired state of your EKS environment, such as the control plane, worker nodes, and Kubernetes add-ons, as an IaC blueprint.</blockquote><p>Looks like a sponsored advertisement? Maybe, but it’s not!<br>What I was after personally is something that:</p><ul><li>follows best practices</li><li>is flexible and extensible</li><li>is being actively supported</li></ul><p>First of all, EKS Blueprints turned out not only to be an open-source project supported by some K8s and AWS enthusiasts only. It’s a result of cooperation between AWS representatives, their partners, and others as the answer to customers’ needs. That I believe has defined the direction and shaped the foundation of what that project represents. One of its pillars is that it follows AWS Well-Architected Framework best practices and therefore lets you focus more on the functional side of your set-up.<br>Secondly, it supports a wide and constantly growing range of Kubernetes <a href="http://aws-ia.github.io/terraform-aws-eks-blueprints/add-ons/">add-ons</a>. The EKS Blueprints allow it to be deployed either with Terraform or with AWS CDK, the two probably most popular tools out there.<br>Then, it implements the so-called <a href="https://aws-ia.github.io/terraform-aws-eks-blueprints/add-ons/#gitops-bridge">GitOps bridge</a> that takes care of configuring resources (e.g. IAM roles and service accounts) to satisfy add-on functionalities requirements.<br>Lastly, the already mentioned growing community — people using it for real, professionals testing it in battle, on real projects, in many different use cases, and for other purposes — made me realize it is not an ephemeral thing and enough quality is there.</p><h3>Caveats</h3><p>Things one should be aware of when using EKS Blueprints…</p><p>There are multiple module calls happening behind the scenes due to the fact the EKS Blueprints support quite a wide range of various controllers/add-ons which ultimately makes the Terraform initialization last a bit longer (~3 minutes).</p><p>When configuring private connectivity between the data plane and the API server endpoint sometimes things don’t work at the beginning. AWS recommends that if your endpoint does not resolve to a private IP address within the VPC you should enable public access and then disable it again.</p><p><a href="https://docs.aws.amazon.com/eks/latest/userguide/cluster-endpoint.html">Amazon EKS cluster endpoint access control</a></p><p>Deleting K8s namespaces created with Terraform is not that straightforward so check the link below to get yourself unblocked just in case.</p><p><a href="https://aws.amazon.com/premiumsupport/knowledge-center/eks-terminated-namespaces/">Troubleshoot terminated Amazon EKS namespaces</a></p><p>Beware! Terraform doesn’t know about AWS resources provisioned by K8s controllers running on the cluster. Make sure you tidy up after running terraform destroy or know what you should do to make the controllers delete relevant resources before destroying your infrastructure. The last thing you want is to have a couple of ALBs hanging there and costing you $16–$20 per month each.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*z2cEo-dHhQF648qW.png" /></figure><h4>👋 If you find this helpful, please click the clap 👏 button below a few times to show your support for the author 👇</h4><h4>🚀<a href="http://from.faun.to/r/8zxxd">Join FAUN Developer Community &amp; Get Similar Stories in your Inbox Each Week</a></h4><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b3ca33171209" width="1" height="1" alt=""><hr><p><a href="https://faun.pub/aws-eks-with-terraform-and-gitops-in-minutes-b3ca33171209">AWS EKS with Terraform and GitOps in minutes</a> was originally published in <a href="https://faun.pub/">FAUN.dev() 🐾</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AWS Landing Zone #3: Hybrid networking]]></title>
            <link>https://faun.pub/aws-landing-zone-3-hybrid-networking-3b4ce4e3178e?source=rss-3abdb2452418------2</link>
            <guid isPermaLink="false">https://medium.com/p/3b4ce4e3178e</guid>
            <category><![CDATA[route-53-resolver]]></category>
            <category><![CDATA[hybrid-dns]]></category>
            <category><![CDATA[hybrid-network]]></category>
            <category><![CDATA[transit-gateway]]></category>
            <category><![CDATA[aws-landing-zone]]></category>
            <dc:creator><![CDATA[Sebastian Mincewicz]]></dc:creator>
            <pubDate>Wed, 06 Jul 2022 13:11:00 GMT</pubDate>
            <atom:updated>2022-07-06T13:11:00.290Z</atom:updated>
            <cc:license>http://creativecommons.org/licenses/by/4.0/</cc:license>
            <content:encoded><![CDATA[<p>Previously, I fleshed out the core aspects of AWS Control Tower managed landing zone and brought closer how to approach accounts baselining to maintain consistency and elevate the security level across the estate.</p><p><a href="https://faun.pub/aws-landing-zone-1-expanding-control-tower-managed-estate-d6dd52e5b06c">AWS Landing Zone #1: Expanding Control Tower managed estate</a></p><p><a href="https://faun.pub/aws-landing-zone-2-control-tower-account-factory-and-baselining-debd72d2167">AWS Landing Zone #2: Control Tower Account Factory and baselining</a></p><p>This one is more of a try-out of how potentially hybrid networking could be set up, including hybrid DNS. Why so? It’s because it really depends on the individual requirements of an organisation. Such requirements can oscillate around various aspects like security, scalability, performance etc. and so the final architecture should be carefully considered to make sure the correct model is applied. Otherwise, one may end up with a configuration that in the long term won’t fit while modifications to such a fundamental matter can turn out to be very costly in many ways.<br>I decided to define objectives that could fit into a wide range of use cases there might be and set things up myself to gain even more experience with the recent AWS network services as well as share my thoughts, as usual.</p><h3>Hybrid networking</h3><p>Hybrid networking is nothing else than just connecting on-premises networks with the ones in the cloud in a secure and performant way.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/713/0*ipsPbEkRIcTq6vS3.png" /></figure><p>To establish hybrid network connectivity three elements are required:</p><ol><li>AWS hybrid connectivity service (Virtual Private Gateway, Transit Gateway, Direct Connect Gateway)</li><li>Hybrid network connection (AWS managed or software VPN, Direct Connect)</li><li>On-prem customer gateway</li></ol><h4>Objectives</h4><p>As for my on-prem network, I simply chose my home network there was only one set of building blocks I could use and these were respectively:</p><ol><li>AWS Transit Gateway (TGW)</li><li>AWS Managed Site-to-Site VPN (S2S VPN)</li><li>StrongSwan @ RaspberryPi</li></ol><p>The functional objectives were to get:</p><ul><li>centralised TGW in the Networking account shared across the Organisation with the use of AWS Resource Access Manager (RAM)</li><li>centralised egress routing via NAT Gateway (NGW) and Internet Gateway (IGW) living in the Networking account</li><li>hybrid DNS</li></ul><h4>Hybrid DNS</h4><p>DNS is a critical component of every network. I wanted to make sure that I can resolve my local/home DNS domain (<strong><em>sebolabs.home</em></strong>) hosted on my Synology NAS from AWS accounts across my Organization and at the same time be able to resolve Route53 (R53) Private Hosted Zones’ records configured in those accounts.<br>The challenge here was the decision itself that must have been made in order to get it set up in the least complicated way keeping in mind the associated costs coming from the fact the R53 resolvers are not the cheapest services out there. They cannot be avoided though as the VPC R53 native resolver (.2) is not reachable from outside of AWS. The concept of using the Inbound/Outbound resolvers is pretty straightforward per se, however, things get more complicated when considering a multi-account set-up.</p><p>One other objective that I set was to be able to provide flexibility and autonomy to manage the R53 Private Hosted Zones (PHZ) within individual accounts but under one condition. That condition was that those hosted zones must <em>overlap</em> with the <em>root</em> hosted zone living in the Networking account along with the resolvers, namely they must represent subdomains:</p><ul><li>Networking AWS account <em>root</em> R53 PHZ: <strong><em>sebolabs.aws</em></strong></li><li>Sandbox AWS account R53 PHZ: sandbox.<strong><em>sebolabs.aws</em></strong></li><li>another AWS account R53 PHZ: any.<strong><em>sebolabs.aws</em></strong></li></ul><p>Apart from the overlapping domain namespaces, one other requirement here is that all the R53 PHZ across the Organization accounts that want to benefit from the hybrid DNS must be associated with the VPC the <em>root</em> PHZ is associated with and where the R53 resolvers are located. At the same time, the Outbound R53 resolver must be shared through RAM to be associated with all other VPCs. The alternative of centralising multiple hosted zones in a shared AWS account didn’t feel appealing to me and so that was the idea I went with.</p><h3>The solution</h3><p>Just to make it crystal clear, the solution presented below is a sort of an MVP that in a real-world scenario would have to be expanded at least to introduce enough resiliency and performance. I will touch slightly upon that matter later in this section. Bear with me…</p><h4>High-level design</h4><p>For my PoC, I set everything up just like explained above leveraging my AWS Organization Networking account and a Sandbox one.<br>My on-prem network on the other hand is represented by a single <strong>Raspberry Pi 4</strong> running <strong>Ubuntu 20.04</strong> and <strong>StrongSwan 1.9.4</strong>, as well as <strong>Synology DS218+</strong>.</p><p><em>To keep the diagram below as clean as possible to highlight the concept the traffic lines were made to flow through the TGW while associated NICs are there to just indicate they physically exist as all that traffic is handled by them in fact. For the same reason, there are no availability zones visualised while this entire set-up is Multi AZ’d, as well as local routes in the routing tables were omitted.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/765/1*sFeJqtk24fjgqeD-GYI00Q.png" /></figure><h4>Transit Gateway</h4><p>As you can see the Transit Gateway routing has been simplified as the use case above is not complex. Normally, there would be multiple TGW routing tables assigned to attachments depending on the individual connectivity requirements of a particular VPC.</p><p>The centralised egress out to the Internet is apart from a way of reducing the costs of running NAT and Internet gateways in each VPC requiring them the ability to introduce security appliances (“bump-in-the-wire”) combined with AWS Gateway Load Balancer for traffic inspection or make use of the AWS Network Firewall.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*2BoQ1v0Xta7iUu3q.jpg" /><figcaption>Centralised north/south connectivity ~ AWS</figcaption></figure><h4>DNS resolution</h4><p>The support for overlapping domain names that is the core concept of the proposed set-up was introduced in late 2019 and made it easy to <em>distribute</em> permissions for managing private hosted zones across the organisation.<br>At the same time, it allows the R53 resolver route traffic based on the most specific match. If there is no hosted zone that exactly matches the domain name in the request, the R53 resolver checks for a hosted zone that has a name that is the parent of the domain name in the request.</p><p>As PHZs are global constructs and not regional they are also a perfect means to support DR scenarios leveraging a multi-region solution. Similar thing with R53 Inbound/Outbound resolvers, another pair of resolvers can be configured in another region to failover to in case of the primary region failure.</p><h4>Caveats</h4><p>The above is obviously just a fundament. Things complicate when you start considering how your workloads will run across your Organisation managed accounts and how services they host will be exposed.<br>Now, when there’s a centralised egress what about exposing your services to the Internet? Wait, wasn’t one of the main ideas behind centralising the egress to disallow the creation of Internet Gateways in managed accounts through SCPs? In such a case you probably either centralise your ingress or maybe disallow the creation of NAT Gateways and association of public IP addresses.<br>Hereby, I just wanted to emphasise how one decision can drive another one and eventually influence the shape of the target solution. In the end, every Organisation wants to end up with patterns and procedures for doing things, don’t they?</p><h3>Final thoughts</h3><p>Going back to my initial statement, there are multiple ways such a hybrid network can be designed and implemented depending on requirements. Each individual functionality must be carefully thought through.<br>Related aspects that I came across when working for companies in their cloud enablement phase were, among other things, considerations around:</p><ul><li>centralised VPC with subnets sharing with RAM along with centrally managed R53 PHZs per share</li><li>single, centralised ingress ALB/NLB with multiple rules passing traffic to internal ALBs/NLBs with a firewall in between</li></ul><p>Not all those ideas turned out to be good choices therefore thinking global and making your target solution as flexible as possible is the way to go. For that of course, there must be a lot of experience within the team.</p><p>Now when AWS offers an enormous range of services and options to deliver very complex solutions while organisations decide to migrate to the cloud we’re back to centralising things but in a different place. The reason for that is that with hundreds of AWS accounts running a big number of workloads organisations want to ensure some control and elevate the level of security which is probably the most important key factor for them when deciding on migrating to the cloud. The times when individual projects were treated independently seem to be over. We’re back doing the same work around networking but in someone else’s data centre :)</p><p>As all these things are not always easy to comprehend, especially when AWS services evolve I strongly suggest following the <a href="https://aws.amazon.com/blogs/networking-and-content-delivery/">Networking and Content Delivery Blog</a> from AWS where you can find many useful clues, and solutions or at least get your head around what’s going on in that world.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*hRMQAjOQBDGoKsDt.png" /></figure><h4>If this post was helpful, please click the clap 👏 button below a few times to show your support for the author 👇</h4><h4>🚀Developers: Learn and grow by keeping up with what matters, <a href="https://faun.to/8zxxd">JOIN FAUN.</a></h4><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3b4ce4e3178e" width="1" height="1" alt=""><hr><p><a href="https://faun.pub/aws-landing-zone-3-hybrid-networking-3b4ce4e3178e">AWS Landing Zone #3: Hybrid networking</a> was originally published in <a href="https://faun.pub/">FAUN.dev() 🐾</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Auth Portal powered by AWS/AzureAD and built with CDKs]]></title>
            <link>https://faun.pub/aws-aad-auth-portal-cdk-56c782e3a048?source=rss-3abdb2452418------2</link>
            <guid isPermaLink="false">https://medium.com/p/56c782e3a048</guid>
            <category><![CDATA[azure-ad]]></category>
            <category><![CDATA[cognito]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[cdktf]]></category>
            <category><![CDATA[cdk]]></category>
            <dc:creator><![CDATA[Sebastian Mincewicz]]></dc:creator>
            <pubDate>Wed, 08 Jun 2022 14:02:32 GMT</pubDate>
            <atom:updated>2022-07-08T13:06:42.105Z</atom:updated>
            <cc:license>https://creativecommons.org/licenses/by-sa/4.0/</cc:license>
            <content:encoded><![CDATA[<p>This one aims to bring together all the pieces required to build and deploy an authentication portal in AWS leveraging Azure AD as IdP. Something that has recently been more and more often used across AWS projects and this time I thought I would go about it a bit differently to try out new things and therefore gain more insights into some tech I haven’t had a chance to master yet. As usual, at the same time sharing some thoughts and experiences with whoever’s interested.</p><h3>Tech stack</h3><p>From the stack outlined below, I’ve already got a huge experience with AWS and Terraform which should not be a surprise if you read my previous publications. However, this time I wanted to shift my focus to using some other popular, open-source tooling I had limited knowledge of and put them into the mix.</p><ul><li><strong>AWS</strong> — to host the Portal</li><li><strong>AWS CDK</strong> (<em>v. 2.26.0</em>)— for developing the Portal (<em>type-script</em>)</li><li><strong>Azure AD</strong> — as an identity service provider (<em>federated authentication</em>)</li><li><strong>CDKtf </strong>(<em>v. 0.11.0</em>) — for configuring Azure AD (<em>type-script</em>)</li><li><strong>React/Amplify </strong>— for a bit of frontend</li></ul><p>As for my job, I must take decisions from time to time on what tooling should be used to deliver a solution, I badly wanted to give <strong><em>AWS CDK </em></strong>a full end-to-end go to witness whether that framework can be easily used for something more than just a PoC. And as there was <strong><em>CDKtf</em></strong> too I decided to give that one a chance as well.<br>Regarding the frontend bit, don’t expect too much ;) It’s not something I do and only set that up to cover up the entire infrastructure stack this story is mainly about and to show a tangible result.</p><h3>Design</h3><p>The high-level diagram below represents the spectrum of services composing the Auth Portal in AWS and its integration with Azure AD.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/851/1*PxX2osVEgqpyvXBFAggc6Q.png" /></figure><p>In case you want to deepen your understanding of the SAML user pool IdP authentication flow please navigate to this page: <a href="https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-user-pools-saml-idp-authentication.html">https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-user-pools-saml-idp-authentication.html</a></p><h4>Goals</h4><p>Apart from the fact that I wanted to get some hands-on experience with CDKs I also set several goals for myself in terms of what I wanted to try out. These were:</p><ul><li>Cognito hosted UI (with customisations)</li><li>Cognito custom domain</li><li>Pre token generation Lambda trigger</li><li>Cognito required claims</li><li>Azure AD additional/custom claims</li><li>Azure AD → Cognito claims mappings</li><li>Azure App access restrictions based on security group membership</li></ul><p>While I abandoned the last one as I wanted to keep the <em>FREE</em> Azure subscription plan that only allows access restrictions on the user account basis, the rest turned out not to be difficult to configure.</p><h4>Source code</h4><p>The source code for the entire stack can be referenced below.</p><p><a href="https://github.com/sebolabs/auth-portal">GitHub - sebolabs/auth-portal: Authentication Portal powered by AWS &amp; Azure AD and delivered with AWS CDK &amp; CDKtf</a></p><p>To use it, you’ll have to export the required environment variables.<br>Additionally, it expects you to have your own public domain and a valid ACM certificate in the N. Virginia region (<em>CloudFront and Cognito requirement</em>).</p><p>Because the frontend part is optional I decided to keep it separate from the AWS portal stack code. If you want to test the authentication flow in the simplest possible way just set theS3_DUMMY_PAGE_DEPLOYenvironment variable totrue , surf to the address below, sign in and then check things with Developers Tools in the browser and <a href="https://jwt.io/">https://jwt.io</a>.</p><p><em>https://</em><strong><em>&lt;cognito custom domain&gt;</em></strong><em>/login?response_type=code&amp;client_id=</em><strong><em>&lt;cognito client id&gt;</em></strong><em>&amp;redirect_uri=</em><strong><em>&lt;portal site url&gt;</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Jn775qPA0pxc9EMWEt11Xg.png" /></figure><h4>Caveats</h4><p>Cognito <strong><em>custom domain</em></strong> feature assumes that if you expose the Cognito authentication endpoint at auth.test.example.comthen your landing page is test.example.com . If that&#39;s not the case, like in my example, then Cognito expects you to have a resolvable A record configured for test.example.com to perform some verification. Moreover, on the Azure side, you must also let your root domain example.combe verified by configuring a TXT record with a provided value.</p><p>Cognito <strong><em>required claims</em></strong> can only be set up at the user pool provisioning stage and so cannot be modified at a later time. This means that if you change your mind you’ll have to delete your Cognito user pool and create it again. This also means your user pool ID changes along with the client application ID and so both the Azure AD and the frontend configuration must be updated with new values. What can turn out to be even more disturbing is the fact that you lose all user profiles previously created in Cognito. Luckily, users can now be imported from CSV.</p><p>The <strong><em>Pre token generation Lambda function</em></strong> hasn’t got any logic customising the claims and was set up only to see what useful information it can produce <em>out of the box</em>.</p><h3>AWS &amp; CDK</h3><p>With CDK the stack gets synthesized and translated into a CloudFormation template. Depending on <em>features</em> you decide to leverage across your code there could be additional, custom resources spun up for you to satisfy requirements.<br>A cool feature here is that any changes to the IAM resources require your attention and approval before getting deployed. This is useful from the security/audit perspective.<br>One thing to keep in mind when bootstrapping a CDK project is the qualifier option that helps you avoid resource name clashes when provisioning multiple bootstrap stacks in the same AWS account.</p><p>Must read: <a href="https://docs.aws.amazon.com/cdk/v2/guide/best-practices.html">https://docs.aws.amazon.com/cdk/v2/guide/best-practices.html</a></p><pre>$ cdk deploy<br>✨  Synthesis time: 3.59s</pre><pre>PortalStack: deploying...<br>[0%] start: Publishing 483ae06ed27ef8ca76e011264d772420593a6cfe8544759c306ef3b98c9e25be:XXXXXXXXXXXX-eu-central-1<br>[...]<br>[100%] success: Published ca2e471276d39c586eae61d73c2e253eb08b4a648a8676f2000f81271b73a405:XXXXXXXXXXXX-eu-central-1</pre><pre>PortalStack: creating CloudFormation changeset...<br>✅  PortalStack<br>✨  Deployment time: 393.82s</pre><pre>Outputs:<br>PortalStack.cognitoDomainName = <a href="https://auth.test.sebolabs.net/">https://auth.test.sebolabs.net</a><br>PortalStack.cognitoUserPoolClientId = 72661fo2r4bgob1ateqskfhicd<br>PortalStack.cognitoUserPoolId = eu-central-1_XXXXXXXXX<br>PortalStack.frontendS3BucketName = sebolabs-test-portal-XXXXXXXXXXXX-eu-central-1<br>PortalStack.portalSiteUrl = <a href="https://portal.test.sebolabs.net/">https://portal.test.sebolabs.net</a></pre><pre>Stack ARN:<br>arn:aws:cloudformation:eu-central-1:XXXXXXXXXXXX:stack/PortalStack/e9974180-e322-11ec-ab92-02e74f818fb0</pre><pre>✨  Total time: 397.4s</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rfEl2KSgKHw9vDt1WkXoSQ.png" /><figcaption>CloudFormation AWS resources view</figcaption></figure><h4>Testing</h4><p>One of the fundamentals and a huge advantage of using CDK is the fact that infrastructure code can be tested just like application code.</p><pre>$ npm test</pre><pre>&gt; portal@0.1.0 test<br>&gt; jest</pre><pre> PASS  test/cf.test.ts<br> PASS  test/cognito.test.ts<br> PASS  test/s3.test.ts<br> PASS  test/lambda.test.ts</pre><pre>Test Suites: 4 passed, 4 total<br>Tests:       9 passed, 9 total<br>Snapshots:   0 total<br>Time:        3.852 s, estimated 4 s<br>Ran all test suites.</pre><h3>Azure AD &amp; CDKtf</h3><p>With CDKtf the stack gets synthesized and translated into a Terraform plan (zipped) that is then executed either locally or remotely.<br>When using Terraform Cloud for your backend you can have it to store your state and optionally use it to run Terraform keeping a track of all your runs in one place. Another nice feature of using that remote backend is that it also versions states and highlights changes (<em>diff</em>) between consecutive Terraform runs.<br>Furthermore, CDKtf supports most of the Terraform well-known commands e.g. output that can become very useful for example to pass certain values between stages in a CI/CD pipeline, but also locals , remote states and other fundamental Terraform features.</p><p>Must read: <a href="https://www.terraform.io/cdktf">https://www.terraform.io/cdktf</a></p><pre>$ cdktf deploy<br>sebolabs-aad-auth-portal  Initializing the backend...<br>sebolabs-aad-auth-portal  Initializing provider plugins...<br>                          - Reusing previous version of hashicorp/azuread from the dependency lock file<br>sebolabs-aad-auth-portal  - Using previously-installed hashicorp/azuread v2.22.0</pre><pre>sebolabs-aad-auth-portal  Terraform has been successfully initialized!</pre><pre>azuread_claims_mapping_policy.portal_cmp (portal_cmp): Refreshing state... [id=a733375e-b67c-49de-9142-12af23de9afa]<br>azuread_group.portal_users (portal_users): Refreshing state... [id=e426314e-14fb-4415-9fd8-e170981b378e]<br>azuread_application.portal_app (portal_app): Refreshing state... [id=764cc3ae-2cf0-4cd5-9521-091b7bd3bece]<br>azuread_service_principal.portal_sp (portal_sp): Refreshing state... [id=a4d4926d-c5cc-402c-96c4-016dc396325f]<br>azuread_service_principal_claims_mapping_policy_assignment.portal_cmpa (portal_cmpa): Refreshing state... [id=a4d4926d-c5cc-402c-96c4-016dc396325f/claimsMappingPolicy/a733375e-b67c-49de-9142-12af23de9afa]</pre><pre>No changes. Your infrastructure matches the configuration.</pre><pre>Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.</pre><pre>sebolabs-aad-auth-portal<br>  app_id = 6e8b9c9d-715b-4371-810f-4avc07a27c2x</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*q9OtZHtdDW6v1l6yrNAdOg.png" /><figcaption>Terraform Cloud Azure AD resources view</figcaption></figure><h4>Testing</h4><p>Likewise, CDKtf also enables you to run unit tests against your code.</p><pre>$ npm test</pre><pre>&gt; portal-azure@1.0.0 test<br>&gt; jest</pre><pre>PASS  __tests__/main-test.ts<br>  Terraform<br>    ✓ check if the produced terraform configuration is valid<br>    ✎ todo check if this can be planned<br>  AzureAD configuration<br>    ✎ todo should contain an application<br>    ✎ todo should contain a service principal<br>    ✎ todo should contain a claims mapping policy<br>    ✎ todo should contain a service principal claims mapping policy assignment</pre><pre>Test Suites: 1 passed, 1 total<br>Tests:       5 todo, 1 passed, 6 total<br>Snapshots:   0 total<br>Time:        4.438 s, estimated 5 s<br>Ran all test suites.</pre><h3>Outcome</h3><p>And finally, here’s the result of putting all the things together…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EbCIncj_wDu8fNPVjcEBdA.png" /><figcaption>The Portal landing page</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9ZGc2DuyJoiS--Bn8fVJRQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yNSU_-CQmp1_a5qb8JXt3A.png" /><figcaption>Amazon Cognito User Pool Hosted UI integrated with Azure AD (IdP)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IBpLGs4cix3xHHSokKe92Q.png" /><figcaption>An authenticated user with claims retrieved</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_CqLMK4b32ZqGv8rLWoUWw.png" /><figcaption>Johny Bravo — AAD user profile</figcaption></figure><p><em>💡 In case some user information was not retrieved or mapped as expected, it’s worth comparing the SAML response (</em><strong><em>idpresponse</em></strong><em> payload) with the ID JWT token (</em><strong><em>token</em></strong><em> preview) using Developer Tools from within a browser.</em></p><h4>CloudWatch Logs Insights</h4><p>The aforementioned Lambda function that is meant to be used for customising ID token claims can also be used to just log certain information carried by tokens. Especially when Cognito is a black box not revealing anything, such data can become very useful e.g. if the authentication flow must be debugged or to generate some users activity statistics. Moreover, very often project teams working on the AWS side of things have no access to the Azure AD application sign-in logs and so that way they can gain some insights.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PbY4JqZmO58eW0aFDQJnaw.png" /></figure><h3>Wrap-up</h3><h4>AWS &amp; Azure AD</h4><p>First of all, setting up an end-to-end authentication flow using Amazon Cognito and Azure AD is fairly simple. Obviously, there are some caveats here and there + things one must be aware of as both services are <em>managed cloud services</em> working based on some assumptions etc. Therefore, it is worth spending some time reading relevant documentation in advance.</p><p>The authentication mechanism configured as a part of this story is just a beginning though. Going further, there’s logic one may want to implement with Lambda triggers and also the entire authorisation flow when integrating such a frontend solution with backend services. Regarding the latter, there are decisions to be made e.g. what type of an authoriser should be used when integrating with API Gateways and even more on access scopes. Either way, having a token carrying correct claims is a good start.</p><h4>AWS CDK &amp; CDKtf</h4><p>On the CDKs side of things, I must admit both have turned out to be very appealing and promising! During my try-out, I certainly enjoyed the fact things simply happen automatically without me having to worry about things I used to worry about when working with cloud infrastructure orchestrators natively. There were several issues I came across but they were rather related either to the underlying orchestrators or cloud providers themselves, nothing that would want me to ditch any of the CDKs.</p><p>In terms of AWS CDK, I’m not a big fan of some of the concepts CloudFormation is based on, e.g. the fact you can’t delete a resource manually in the console and then rerun cdk deploy to calculate what’s missing and reprovision that resource. Instead, you get a resource not found exception.</p><p>In terms of CDKtf, having configured only five resources is probably not representative enough to draw big conclusions, however, it just worked and literally took me minutes to have a working deployment mechanism.</p><p>Just like with any other frameworks, e.g. the serverless framework, CDKs can massively simplify developers&#39; lives. With AWS Security Speciality hat on, however, I want to emphasize the security aspect of the infrastructure being provisioned as a part of application development. I’ve seen many times already how insecure such infrastructure can become when it’s developed by individuals having not enough <em>security-in-the-cloud</em> awareness, especially when there’s networking involved. Therefore, getting compliant constructs or modules is definitely something that should be considered by organisations and project teams who care more about just application features.</p><p>Finally, both CDKs require you to figure out the best way you want to go about environments and how you’re going to provide environment-specific values when deploying stacks. It’s not as straightforward as with Terraform environment files but once you get it right you can’t go wrong with CDKs. Both frameworks can still be considered quite new and I strongly believe that with time they’ll become even more robust.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*L0KlZW-IWmyyqctz.png" /></figure><h4>If this post was helpful, please click the clap 👏 button below a few times to show your support for the author 👇</h4><h4>🚀Developers: Learn and grow by keeping up with what matters, <a href="https://faun.to/8zxxd">JOIN FAUN.</a></h4><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=56c782e3a048" width="1" height="1" alt=""><hr><p><a href="https://faun.pub/aws-aad-auth-portal-cdk-56c782e3a048">Auth Portal powered by AWS/AzureAD and built with CDKs</a> was originally published in <a href="https://faun.pub/">FAUN.dev() 🐾</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AWS Landing Zone #2: Control Tower Account Factory and baselining]]></title>
            <link>https://faun.pub/aws-landing-zone-2-control-tower-account-factory-and-baselining-debd72d2167?source=rss-3abdb2452418------2</link>
            <guid isPermaLink="false">https://medium.com/p/debd72d2167</guid>
            <category><![CDATA[aws-organizations]]></category>
            <category><![CDATA[aws-control-tower]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[aws-landing-zone]]></category>
            <category><![CDATA[aws-account-factory]]></category>
            <dc:creator><![CDATA[Sebastian Mincewicz]]></dc:creator>
            <pubDate>Fri, 26 Nov 2021 22:45:35 GMT</pubDate>
            <atom:updated>2021-11-26T22:51:45.668Z</atom:updated>
            <content:encoded><![CDATA[<p>Previously, I tried to bring closer general capabilities of the AWS Control Tower (CT) and identify the missing core elements of the landing zone it establishes and that I found crucial.</p><p><a href="https://faun.pub/aws-landing-zone-1-expanding-control-tower-managed-estate-d6dd52e5b06c">AWS Landing Zone #1: Expanding Control Tower managed estate</a></p><p>In this part, however, I want to focus more on how CT supports the provisioning of new AWS accounts becoming a part of the AWS Organizations and even more on how they can be further baselined automatically and kept up to date during their lifecycle.</p><h3><strong>Account Factory</strong></h3><p>The ability to easily provision new AWS accounts is very important. Usually, you’d create a new account per service, per environment or at least per service, per environment data type (like prod and non-prod) to ensure necessary isolation and separation of concerns that are aimed at increasing the overall security and reliability of your workloads.<br>AWS Control Tower gives us a form-based option for provisioning new accounts that is powered by the <strong><em>AWS Service Catalog</em></strong>. In simple words — all you have to do is provide account details, pick an OU and a new account gets created within ~15 minutes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Wd9h9Gy6_8MjPh3eABrDgQ.png" /></figure><p>If for some reason you’d want to automate the creation of multiple accounts there’s an existing CFN-powered solution developed by AWS. Whether you need that or not depends on the individual requirements of your organisation, however, I could risk a statement that in most cases you don’t.<br>Such custom automation, no matter if you create one or more accounts at the same time, gives you some options to do something extra with the data you provide and the outputs you get. However, there are other ways to achieve potentially the same thing by using APIs and CloudWatch events, therefore, I don’t perceive that feature as something indispensable.</p><h3><strong>Account baselining</strong></h3><p>Upon account creation with the use of the Account Factory, a CFN stack is executed to apply the baseline that consists of the following core elements:</p><ul><li><strong><em>IAM roles</em></strong> controlled by the AWS SSO in the master account</li><li><strong><em>AWS CloudTrail</em></strong> with CloudWatch logs stored locally, the same logs archived in S3 in the <em>Log archive</em> account, and log delivery notifications sent to a pre-configured SNS topic in the <em>Audit</em> account</li><li><strong><em>AWS Config</em></strong> recorder configured with the <em>Log archive</em> account for logs delivery and the <em>Audit</em> account as the one authorized for collecting compliance and configuration data as well as receiving configuration changes notifications via SNS.</li></ul><p>That’s not a lot. After expanding the landing zone like described in part #1 and by enabling some organization-wide AWS services the coverage looks a bit better, however, there are a couple of other things that still should be done in order to keep the AWS Security Hub happy. These are:</p><ul><li>EBS encryption enabled by default</li><li>S3 public access disabled by default</li><li>IAM password policy set</li><li>AWS Support dedicated IAM role created</li><li>default VPC deleted</li><li>CIS-related CloudWatch metrics and alarms set up</li></ul><p>Obviously, one can decide to put more stuff in the baseline depending on individual needs and requirements.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/629/1*LWX_i0BDX3XM5vTNG4Lwcg.png" /></figure><p>Initially, after a new account is provisioned its Security Hub security score calculated across both standards (FSBP and CIS) is quite low.<br>After I applied the extended, custom baseline the security score across four AWS accounts (Master, Audit, Log archive, and an additional one — Sandbox) went up to 99%.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kskch6Ls2c3FmhYxDyltRw.png" /></figure><p>Failed controls were:</p><ul><li>SNS topics should be encrypted at rest using AWS KMS</li><li>IAM principals should not have IAM inline policies that allow decryption actions on all KMS keys</li></ul><p>Surprisingly (or not), both relate to resources managed by the AWS Control Tower. Either way, it looks to me like a very good starting point!</p><h3>Custom baselining automation</h3><p>As I usually work with Terraform and already had a proven way for running it using AWS CodeBuild (<em>see link below</em>) I decided to set something up that would leverage that configuration.</p><p><a href="https://faun.pub/terraform-deployments-with-aws-codepipeline-342074248843">Terraform deployments with AWS CodePipeline</a></p><p>Let’s start with the requirements. I defined three scenarios when I would want to trigger accounts baselining. These are:</p><ul><li>on new AWS account creation with AWS Control Tower (<em>for a new account</em>)</li><li>on Terraform ‘baseline’ module change (<em>for all managed accounts</em>)</li><li>whenever I want, manually (<em>for 1-n managed accounts</em>)</li></ul><p>And here’s what I ended up with…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rcHJryTMDb5wLRkrdNBlVQ.png" /></figure><p>That Lambda function there can become a dispatcher and help you do even more custom things than just baselining accounts. Imagination is the only constraint here!</p><h4>How does it work?</h4><p>As you can see on the diagram when the Lambda function is triggered it then processes AWS accounts IDs and invokes the Step Functions (SFN) state machine with a JSON object as the input containing a <em>list</em> of account IDs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/984/1*Qw8UQqJID4FACOqKr_si0A.png" /><figcaption>Lambda output == Step Functions input</figcaption></figure><p>SFN state machine definition uses the Map state to iterate over the <em>list</em> of accounts and to trigger the AWS Code Build project that runs Terraform against those accounts simultaneously, a maximum of ten at a time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/455/1*zTRQCk4N7lGIhrgD6Za1gg.png" /><figcaption>Step Functions state machine execution graph</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*noSbLACvcBLJcistfu6MFw.png" /><figcaption>X-ray trace view</figcaption></figure><p>Unfortunately, Step Functions does not yet maintain the trace ID of the calling resource hence the Lambda function is not displayed in that X-Ray trace view.</p><h3>Wrap-up</h3><p>I think that what I came up with here is a pretty decent way of implementing an extended, fully automated AWS accounts baselining capability to what AWS Control Tower offers and that with a bit of imagination and depending on individual needs can be brought even further.</p><h3>What’s next?</h3><ul><li>More guardrails — SCPs to secure what the custom baseline manages</li><li>Security events management — in <em>Audit</em> account</li><li>Logging capabilities — in <em>Log archive</em> account</li></ul><figure><a href="https://faun.to/bP1m5"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BCiLLad3dvZLwBa-B5cAVQ.png" /></a></figure><p>Join FAUN: <a href="https://faun.to/i9Pt9"><strong>Website</strong></a><strong> </strong>💻<strong>|</strong><a href="https://faun.dev/podcast"><strong>Podcast</strong></a><strong> </strong>🎙️<strong>|</strong><a href="https://twitter.com/joinfaun"><strong>Twitter</strong></a><strong> </strong>🐦<strong>|</strong><a href="https://www.facebook.com/faun.dev/"><strong>Facebook</strong></a><strong> </strong>👥<strong>|</strong><a href="https://instagram.com/fauncommunity/"><strong>Instagram</strong></a><strong> </strong>📷|<a href="https://www.facebook.com/groups/364904580892967/"><strong>Facebook Group</strong></a><strong> </strong>🗣️<strong>|</strong><a href="https://www.linkedin.com/company/faundev"><strong>Linkedin Group</strong></a><strong> </strong>💬<strong>|</strong> <a href="https://faun.dev/chat"><strong>Slack</strong></a> 📱<strong>|</strong><a href="https://thechief.io/"><strong>Cloud Native</strong> <strong>News</strong></a><strong> </strong>📰<strong>|</strong><a href="https://linktr.ee/faun.dev/"><strong>More</strong></a><strong>.</strong></p><p><strong>If this post was helpful, please click the clap 👏 button below a few times to show your support for the author 👇</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=debd72d2167" width="1" height="1" alt=""><hr><p><a href="https://faun.pub/aws-landing-zone-2-control-tower-account-factory-and-baselining-debd72d2167">AWS Landing Zone #2: Control Tower Account Factory and baselining</a> was originally published in <a href="https://faun.pub/">FAUN.dev() 🐾</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>