<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Adrian Arba on Medium]]></title>
        <description><![CDATA[Stories by Adrian Arba on Medium]]></description>
        <link>https://medium.com/@adrianarba?source=rss-7f487fa71b14------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*aVQghwc6ZAwmOzbR.</url>
            <title>Stories by Adrian Arba on Medium</title>
            <link>https://medium.com/@adrianarba?source=rss-7f487fa71b14------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 23:15:54 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@adrianarba/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[GCP Cloud Run sidecars with shared volume (Part 2) — implementation]]></title>
            <link>https://medium.com/@adrianarba/gcp-cloud-run-sidecars-with-shared-volume-part-2-implementation-0f1e4e4dbb75?source=rss-7f487fa71b14------2</link>
            <guid isPermaLink="false">https://medium.com/p/0f1e4e4dbb75</guid>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[gcp]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[containers]]></category>
            <category><![CDATA[google-cloud-run]]></category>
            <dc:creator><![CDATA[Adrian Arba]]></dc:creator>
            <pubDate>Sat, 16 Mar 2024 22:14:41 GMT</pubDate>
            <atom:updated>2024-03-16T22:14:41.446Z</atom:updated>
            <content:encoded><![CDATA[<h3>GCP Cloud Run sidecars with shared volume (Part 2) — implementation</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kh8cTo3RT14vMRKA1JMUJg.png" /></figure><p>If you’ve read <a href="https://medium.com/@adrianarba/gcp-cloud-run-sidecars-with-shared-volume-part-1-lessons-learned-5dec1d6fa009">Part 1</a>, you have an understanding of why the solution I’m detailing below from an implementation standpoint looks the way it looks.</p><p>Let’s get cracking then!</p><h3>Scripts</h3><p>I chose to go with Python for writing my container scripts because I found it easier to do the needed work and also to keep a server up and running from the same main process.</p><p>Since Cloud Run doesn’t support Init containers (the functionality I needed), I have to run sidecar containers in parallel. And they both have to be alive, even though one can start after the other. To keep the container processes alive, I use the Flask framework to create hosts on the containers and keep small web applications alive.</p><h4>Render script</h4><p>My “init” sidecar will run the “Render script” which helps me read a Config file that has regular Shell Environment Variables templates ${MY_ENV_VAR}along with other static key-value pairs and checks the OS Environment Variables for matches and replaces them if they are found (and the expectation is they will be loaded inside the Container OS from Google Secret Manager).</p><p>It expects a specific Config file to be present in a specific location within the Container.</p><pre>from flask import Flask, jsonify<br>import os<br>import re<br><br>app = Flask(__name__)<br><br># source config file path<br>FILE_PATH = &#39;/var/www/config.yaml&#39;<br><br># output config file path where the processed content will be saved<br># and from where the main container will load it<br>NEW_FILE_PATH = &#39;/mnt/config.yaml&#39;<br><br># status message for the exposed Flask app endpoint &#39;/status&#39;<br>status_message = &quot;Processing not started&quot;<br><br>def process_and_save_file_content(original_filepath, new_filepath):<br>    global status_message<br><br>    try:<br>        with open(original_filepath, &#39;r&#39;) as file:<br>            content = file.read()<br><br>        # checking for ${...} type strings in the config file<br>        pattern = re.compile(r&#39;\$\{([a-zA-Z0-9_]+)\}&#39;)<br><br>        # found hit will be searched in the OS Environment Variables<br>        def replace_with_env(match):<br>            env_var_name = match.group(1)<br>            return os.getenv(env_var_name, match.group(0))<br><br>        templated_content = pattern.sub(replace_with_env, content)<br><br>        # output the rendered file in the output config file location<br>        with open(new_filepath, &#39;w&#39;) as new_file:<br>            new_file.write(templated_content)<br><br>        # update the Endpoint status message<br>        status_message = &quot;Processing done&quot;<br><br>    except Exception as e:<br>        status_message = f&quot;Error processing and saving the file: {str(e)}&quot;<br><br>@app.route(&#39;/status&#39;, methods=[&#39;GET&#39;])<br>def get_status():<br>    return jsonify({&#39;message&#39;: status_message})<br><br>if __name__ == &#39;__main__&#39;:<br>    # first, process and save the file content<br>    process_and_save_file_content(FILE_PATH, NEW_FILE_PATH)<br><br>    # then, run the Flask app<br>    # I don&#39;t really care about the port, I&#39;ll secure it in Google<br>    # but I need a port opened for the mandatory Cloud Run Startup Probe<br>    app.run(host=&#39;0.0.0.0&#39;, port=8181)</pre><h4>Mock “main App” script</h4><p>My “main App” script is a mock of the actual Application that will eventually be deployed to the Cloud Run instance.</p><p>For testing purposes, I want a web app that loads the final “rendered” Config file and displays it when I access the Cloud Run service revision URL. This allows me to check if my entire flow was successful, and see the process update when I make changes to the Config file or scripts.</p><p>It’s a placeholder for the Application that will later use this PoC.</p><pre>from flask import Flask, Response<br><br>app = Flask(__name__)<br><br># Path to the processed file<br>PROCESSED_FILE_PATH = &#39;/mnt/config.yaml&#39;<br><br>@app.route(&#39;/&#39;)<br>def serve_file():<br>    try:<br>        with open(PROCESSED_FILE_PATH, &#39;r&#39;) as file:<br>            content = file.read()<br>            # I can see my file&#39;s content in the browser at the Cloud Run<br>            # service URL<br>            return Response(content, mimetype=&#39;text/plain&#39;)<br>    except Exception as e:<br>        return Response(f&quot;Error serving the file: {str(e)}&quot;, mimetype=&#39;text/plain&#39;)<br><br>if __name__ == &#39;__main__&#39;:<br>    # port 8080 is where my final Application will live for this exercise<br>    app.run(host=&#39;0.0.0.0&#39;, port=8080)</pre><h3>Dockerfiles</h3><p>Each of my scripts will be packaged into an individual container image.</p><h4>Render Dockerfile</h4><pre># Use an official lightweight Python image<br>FROM python:3.9-slim<br><br># Set the working directory in the container<br>WORKDIR /app<br><br># Copy the dependencies file to the working directory<br># The requirements.txt file contains:<br># Flask==2.1.2<br># Werkzeug==2.2.2<br>COPY requirements.txt .<br><br># Install any dependencies<br>RUN pip install --no-cache-dir -r requirements.txt<br><br># Copy the content of the local src directory to the working directory<br># This is where my &#39;init&#39; script called &#39;main.py&#39; is<br>COPY src_init/ .<br><br># Specify the command to run on container start<br>CMD [ &quot;python&quot;, &quot;./main.py&quot; ]</pre><p>I build and push this image like so:</p><pre># make sure you are at Dockerfile level<br>cd /path/to/my/Dockerfiles<br>bash<br># authenticate to Google from the cli (I use Service Account impresonation)<br>gcloud auth login --impersonate-service-account=developer@&lt;project_id&gt;.iam.gserviceaccount.com<br><br># login to Artifact registry<br>gcloud auth configure-docker europe-west2-docker.pkg.dev --impersonate-service-account=developer@&lt;project_id&gt;.iam.gserviceaccount.com<br><br># build the Dockerfile (I call it Dockerfile_init)<br>docker build --platform linux/amd64 -t europe-west2-docker.pkg.dev/&lt;project_id&gt;/my-artifact-registry-instance-name/poc/cldrungcs:1 -f Dockerfile_init .<br><br># push the image to Google&#39;s Artifact Registry service<br>docker push europe-west2-docker.pkg.dev/&lt;project_id&gt;/my-artifact-registry-instance-name//poc/cldrungcs:1</pre><h4>Main App Dockerfile</h4><pre># The contents and the requirements.txt contents are identical with the other<br># Dockerfile <br>FROM python:3.9-slim<br><br>WORKDIR /app<br><br>COPY requirements.txt .<br><br>RUN pip install --no-cache-dir -r requirements.txt<br><br># the only difference is where I keep my python script which is also called<br># &#39;main&#39;<br>COPY src_main/ .<br><br>CMD [ &quot;python&quot;, &quot;./main.py&quot; ]</pre><p>And to push the image to Google, I use the same process as before, the only thing that changes is the image name cldrungcs:1 .</p><h3>Resources</h3><p>For provisioning the required resources in Google Cloud Run, I chose Terraform (+ Terragrunt) because the project already used these tools to provision Infrastructure.</p><p>I’m only going to go through the main.tf file so I won’t be using any variables in my code. You should adapt it to use Terraform variables if you need to use it.</p><pre># main.tf<br><br># to make this work, I need to use the v2 Terraform resource for <br># Cloud Run<br>resource &quot;google_cloud_run_v2_service&quot; &quot;default&quot; {<br><br>  # and because some features are still in Beta, like GCS volume mounting<br>  # or sidecar containers, I need to use the &#39;google-beta&#39; provider<br>  provider = google-beta<br>  name     = &quot;poc-gcs-volumes&quot;<br>  ingress  = &quot;INGRESS_TRAFFIC_INTERNAL_ONLY&quot;<br><br>  location     = &quot;europe-west2&quot;<br><br>  # and the BETA launch stage ties into the &#39;google-beta&#39; provider<br>  launch_stage = &quot;BETA&quot;<br><br>  template {<br>    <br>    # I also have ti use the Gen2 execution environment for Cloud Run<br>    execution_environment = &quot;EXECUTION_ENVIRONMENT_GEN2&quot;<br>    service_account       = &quot;my-cldrun-svc-account@&lt;project_id&gt;.iam.gserviceaccount.com&quot;<br><br>    # I want to have one instance always on for the purpose of this PoC<br>    scaling {<br>      max_instance_count = 1<br>      min_instance_count = 1<br>    }<br><br>    vpc_access {<br>      # don&#39;t really need egress traffic but meh<br>      egress    = &quot;ALL_TRAFFIC&quot;<br>      connector = &quot;my_vpc_connector&quot;<br>    }<br><br>    # my &quot;init&quot; sidecar container<br>    containers {<br>      image = &quot;europe-west2-docker.pkg.dev/&lt;project_id&gt;/my-artifact-registry-instance-name/poc/cldruninit:1&quot;<br>      name  = &quot;init&quot;<br>      <br>      # I mount the GCS bucket object - my config file<br>      volume_mounts {<br>        name       = &quot;bucket&quot;<br>        mount_path = &quot;/var/www&quot;<br>      }<br><br>      # and I also mount the in-memory disk which is shared between sidecars<br>      volume_mounts {<br>        name       = &quot;empty-dir-volume&quot;<br>        mount_path = &quot;/mnt&quot;<br>      }<br><br>      # I did not link the Secret Manager instance as it was pointless for<br>      # the PoC, but I needed to add an Env Var to the continer to be rendered<br>      # in my template config file<br>      env {<br>        name  = &quot;CLIENT_SECRET&quot;<br>        value = &quot;my_super_special_secret&quot;<br>      }<br><br>      # this is where it gets tricky, the sidecar container, althoug doing<br>      # a smal processing, needs to pass the Startup probe check and remain<br>      # up and running alongside my main container<br>      startup_probe {<br>        http_get {<br>          port = 8181<br>          path = &quot;/status&quot;<br>        }<br>      }<br>    }<br><br>    # my &quot;main&quot; Application container<br>    containers {<br>      image      = &quot;europe-west2-docker.pkg.dev/&lt;project_id&gt;/my-artifact-registry-instance-name/poc/cldrungcs:1&quot;<br>      <br>      # this depends_on parameter allows me to set a container startup order<br>      depends_on = [&quot;init&quot;]<br><br>      # also mounting the same in-memory volume which will hold my<br>      # templated config file<br>      volume_mounts {<br>        name       = &quot;empty-dir-volume&quot;<br>        mount_path = &quot;/mnt&quot;<br>      }<br>    }<br><br>    # and this is where I refer my mounts to sources<br>    # first source is the GCS template config file object<br>    volumes {<br>      name = &quot;bucket&quot;<br>      gcs {<br>        bucket    = google_storage_bucket.default.name<br>        read_only = false<br>      }<br>    }<br><br>    # and the second is the in-memory destination rendered config file<br>    volumes {<br>      name = &quot;empty-dir-volume&quot;<br>      empty_dir {<br>        medium     = &quot;MEMORY&quot;<br>        size_limit = &quot;50Mi&quot;<br>      }<br>    }<br>  }<br>}<br><br># This policy allows unauthenticated access to my Cloud Run instance<br># via the allUsers member<br>resource &quot;google_cloud_run_service_iam_member&quot; &quot;invoker&quot; {<br>  project  = &quot;&lt;project_id&gt;&quot;<br><br>  service  = google_cloud_run_v2_service.default.name<br>  location = google_cloud_run_v2_service.default.location<br>  role     = &quot;roles/run.invoker&quot;<br>  member   = &quot;allUsers&quot;<br>}<br><br># minimal bucket provisioning - the Org policy automatically makes and ensures<br># my buckets stay private<br>resource &quot;google_storage_bucket&quot; &quot;default&quot; {<br>  project                     = &quot;&lt;project_id&gt;&quot;<br><br>  name                        = &quot;poc-gcs-volumes-test-bucket-76125312&quot;<br>  location                    = &quot;EU&quot;<br>  uniform_bucket_level_access = true<br><br>  versioning {<br>    enabled = true<br>  }<br>}<br><br># my Cloud Run designated Service Account needs bucket permissions to<br># read from the bucket and for Cloud Run to mount volumes to objects<br>resource &quot;google_storage_bucket_iam_member&quot; &quot;default-sa&quot; {<br>  depends_on = [google_storage_bucket.default]<br><br>  bucket     = google_storage_bucket.default.name<br>  role       = &quot;roles/storage.objectAdmin&quot;<br>  member     = &quot;serviceAccount:my-cldrun-svc-account@&lt;project_id&gt;.iam.gserviceaccount.com&quot;<br>}<br><br># the default compute engine service account in my project also needs<br># bucket access permissions (Cloud Run uses it)<br>resource &quot;google_storage_bucket_iam_member&quot; &quot;default-sa-compute&quot; {<br>  depends_on = [google_storage_bucket.default]<br><br>  bucket     = google_storage_bucket.default.name<br>  role       = &quot;roles/storage.objectAdmin&quot;<br>  member     = &quot;serviceAccount:&lt;11-digit&gt;-compute@developer.gserviceaccount.com&quot;<br>}</pre><h3>Other things</h3><h4>Test config file</h4><p>I use a test config file that I upload to Google Cloud Storage</p><pre>application_name: my_application<br><br>authentication:<br>  access-token-uri: https://login.microsoftonline.com/azureford.onmicrosoft.com/oauth2/v2.0/token<br>  # the client-secret value will be rendered by my &#39;init&#39; script from the<br>  # container&#39;s OS Environment Variable called the same way<br>  client-secret: ${CLIENT_SECRET}<br>  status:<br>    enabled: true<br><br>features:<br>  has_carrots: true<br>  # dummy test<br>  has_sugar: fal${e<br>  has_milk: null</pre><p>To push the file to the Bucket</p><pre>gcloud storage cp config.yaml gs://poc-gcs-volumes-test-bucket-76125312 --impersonate-service-account=developer@&lt;project_id&gt;.iam.gserviceaccount.com</pre><p>And really, that’s it, to test everything, I run terraform apply and access my Cloud Run service URL to check if I see my config file correctly rendered based on my Environment Variable value in my browser.</p><p>Hope this helps anyone trying to do something similar, and let me know if you have a better way of doing this :)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0f1e4e4dbb75" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[GCP Cloud Run sidecars with shared volume (Part 1) — lessons learned]]></title>
            <link>https://medium.com/@adrianarba/gcp-cloud-run-sidecars-with-shared-volume-part-1-lessons-learned-5dec1d6fa009?source=rss-7f487fa71b14------2</link>
            <guid isPermaLink="false">https://medium.com/p/5dec1d6fa009</guid>
            <category><![CDATA[containers]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[google-cloud-run]]></category>
            <category><![CDATA[best-practices]]></category>
            <category><![CDATA[gcp]]></category>
            <dc:creator><![CDATA[Adrian Arba]]></dc:creator>
            <pubDate>Sat, 16 Mar 2024 16:59:21 GMT</pubDate>
            <atom:updated>2024-03-16T22:17:04.945Z</atom:updated>
            <content:encoded><![CDATA[<h3>GCP Cloud Run sidecars with shared volume (Part 1) — PoC</h3><p>On one of the projects I worked on, I was tasked to do a PoC implementation of an Application lift and shift from on-premise to GCP Cloud Run that requires loading both secrets and a configuration file that references those secrets when it boots up.</p><p>Specifically, what was asked of me was to:</p><ul><li>perform the PoC ASAP — it had to be done “yesterday”, but it surfaced “today” so I needed to do it until “tomorrow” — this means I had to mock the application and the config to ensure the solution conceptually works;</li><li>deploy the application in Cloud Run as is (end goal), but wrap it in a container — rewriting anything in the application at this time would not be possible due to migration time constraints (and the monolithic aspect of the application);</li><li>make sure the application loads a config file located in a separate git repo when it starts — the config file must stay within that git repo to ensure changes to it are reviewable, auditable, and taggable to support the application release lifecycle;</li><li>make sure that the config file, which also contains template references to some secrets gets templated before the application starts — of course, never keep secret data versioned in git, everyone should know this by now;</li><li>make sure when there is a git update on either the app or the config repositories, it gets reflected in GCP for testing.</li></ul><p>That meant that I had to tackle a few challenges:</p><ul><li>decide if keeping the entire config file in Secrets Manager vs just the secrets and the rest of the config in something like Cloud Storage was the right way to go;</li><li>mount the config on disk as a volume, which then the application or a script can read as a file;</li><li>if a script reads the volume, load the file, and transform it before preparing it for the application ingestion;</li><li>ensure that on either application or config update, this automatically gets reflected in Cloud Run without any application downtime;</li><li>mock the application due to time constraints and test everything as part of the PoC.</li></ul><p>At the time of writing this, March 2024, I knew that GCP had released a few interesting Cloud Run beta features that would help me consider the implementation feasible. Still, I found very little support in the web space in terms of implementations or documentation.</p><p>Based on the challenges presented to me, I had to go through several considerations:</p><h4>Consideration 1 — Should I keep the entire config file in GCP Secret Manager?</h4><p>As <a href="https://cloud.google.com/secret-manager/docs">the GCP documentation</a> suggests, Secret Manager is a service used for storing anything that could be treated as a secret or sensitive, so things like API keys, passwords, certificates, and other sensitive data. It’s meant to improve security, by providing means to limit who has access to handle the secrets, you can encrypt secrets, and replicate them and it allows versioning of your secrets and secret rotation (via Pub/Sub and other services).</p><p>What it doesn&#39;t say anywhere is that you should keep other types of data in there, like config files.</p><p>The config file at hand contains 95% nonsensitive data and 5% sensitive data. I don&#39;t believe the 5% of sensitive data warrants keeping the rest of the 95% of data in Secret Manager. Not to mention that, in general, if you have high numbers of Secret Manager API calls from applications seeing a lot of user traffic, it tends to start getting expensive, Secret Manager isn’t one of the cheapest Services in Google’s repertoire.</p><p><strong>My conclusions:</strong></p><ul><li><strong>use Secret Manager as it was intended, and add only the secret data</strong> that will be loaded in the Cloud Run container as Environment Variables (while securing the access to the Container environment as best as possible);</li><li><strong>strip the config file from any “on-prem” secret value references and add Shell references to Environment Variables</strong>;</li><li><strong>deploy the config file to another GCP service</strong> which would allow me to mount it as a file in the Cloud Run container and therefore be read by the Application. Cloud Run supports volumes from multiple services. You can use Google Cloud Storage, Filestore, Firestore or Cloud SQL if your application is <strong>stateful </strong>or in-memory storage if your application is <strong>stateless</strong>. I went with Cloud Storage and in-memory volumes.</li></ul><h4>Consideration 2 — How do I parse templated “secret” config values in the config file?</h4><p>As we know, containers run only one main process throughout their lifespan. If the process dies, the container dies as well.</p><p>The Application will read the config file on startup, but it needs it templated with the expected values for different keys, it has no logic to do any templating. To me, this sounds like a multi-container implementation.</p><p>I’ve used init containers before in this manner, in Kubernetes Stateful Set resources, for example. You can define one or multiple containers that run before your main Application container and prepare the field so that the Application will then have all the details it needs to do its job.</p><p>Google’s documentation doesn&#39;t mention anything about init containers (and to my knowledge, they are not supported yet), but it does mention sidecar containers — parallel execution of multiple containers. This implementation is great for long-running secondary containers which potentially do things like data manipulation before it reaches the main container, act as firewalls or as telemetry metric gathering, etc. But they run as long as the main process container runs too.</p><p>I only need the secondary container to run first and transform a file before the main Application container starts up and uses that file.. Grrr!!</p><p>Not even ChatGPT thought what I wanted to do was possible, as its latest learning span is April 2023 — on this note, I consider ChatGPT to be a very helpful tool in the DevOps estate, as it can quickly give you links to different documentation when you start assessing the feasibility of a solution and can help you explore playing with different components before you even touch a single line of code.</p><p><strong>My conclusion</strong>: as this was the only possible option, <strong>I chose to implement a sidecar container</strong> that will run a Python script, load the templated config file as a volume, parse it, and replace any template values with the values in ENVIRONMENT variables (loaded from Secret Manager), and then copy the config file to another mounted volume to be shared with the main Application container.</p><p>That led me to...</p><h4>Consideration 3 — Can volumes be shared between sidecar containers in GCP Cloud Run?</h4><p>Google’s documentation on sidecars and volume sharing is skittish, I only found <a href="https://cloud.google.com/run/docs/configuring/services/in-memory-volume-mounts">one single reference here</a> that briefly mentions that “<em>You can use in-memory volumes to […] share an in-memory volume between different containers in one Cloud Run instance</em>”.</p><p>The only other option would have been a Filestore instance for NFS file storage, which meant provisioning and maintaining another set of Terraform resources for just a few kilobytes of data... No way!</p><p>So I set out to test the hypothesis that memory volumes can be shared between sidecar containers in Cloud Run and hey, what do you know, it worked.</p><p>A few things to consider here, as per the GCP documentation:</p><ul><li>when creating an in-memory volume, we recommend specifying a size limit. If the volume reaches its size limit, further writes will fail with an out-of-memory error. <strong>Your instance can handle this error and keep running.</strong></li><li>if you deploy multiple containers, the memory used by each write to the volume counts as memory usage for the container that wrote the data.</li></ul><p>So, really important, set a memory limit that is both adequate and also thought of when setting container resources. For my purpose, I set a limit of 50Mb for the volume.</p><h4>Consideration 4 — How do I ensure that on either an Application or a Config change, the Cloud Run end user receives the updated result?</h4><p>The team discussed how the Application should be updated on either an Application code update or a config update.</p><p>One clear thing about Cloud Run containers, just like with any other container service, is that containers are immutable. Updates should be treated as a new version of the end container.</p><p>And since the config file is being read only when the Application starts up, having it updated dynamically would not produce any effect on the Application.</p><p>Three options were discussed:</p><ul><li>Have a process poll the config file (maybe via a SHA hash) and, if it sees a change, signal the App process to die, thus forcing Cloud Run to spin up another instance that would reload the config file and serve the update to a new Application instance — not ideal, as this would affect all running instances and would lead to Application unavailability during container boot up (a few seconds);</li><li>Rewrite the Application to continuously poll the config file — not useful as the config file also contains feature flags and secrets:<br>- if a new feature is added but the Application is not redeployed, the flag would be useless;<br>- if a “secret” value is added and the Application container has not been restarted to read the new Secret Manager secret, the config update would be useless.<br>There was also not enough time to do this;</li><li>Redeploy to Cloud Run as a new Revision regardless of the update either being an Application update or a Config update. This option is the one I favored the most, as it ensures the Container immutability principle is respected + bonus, Cloud Run only switches to a new Revision if it passes all the Startup probes and can serve traffic, otherwise, it keeps the old Revision running. This means no downtime in case of any issues with the update.</li></ul><p><strong>The conclusion</strong> was to have our (Tekton) CI/CD deploy pipeline listen to triggers from both the Application and the Config container and, regardless of who has an update, <strong>redeploy to Cloud Run as a new Revision</strong>.</p><h4>Consideration 5 — What service should I use to load the config file into and run my template script?</h4><p>As I mentioned in Consideration 1, there are multiple services you could use to provide files as volumes to Cloud Run.</p><p>My conclusion was to use Google Cloud Storage to upload the raw config files into (and version them) from the git repository and then mount them in my “init” sidecar container to be rendered by the templating script with actual values. It’s cheap and does the job, I only read from it, I don’t write to it and I only need the file in one step.</p><p>In <a href="https://medium.com/@adrianarba/gcp-cloud-run-sidecars-with-shared-volume-part-2-implementation-0f1e4e4dbb75">Part 2</a>, I’m going to go through the code I used to do the PoC. It’s going to include:</p><ul><li>writing 2 Python scripts to render the templates in the config file and mock a Flask application for me to see the actual config file being rendered when I access the Cloud Run instance URL;</li><li>building 2 Docker container images (one for the template rendering script and one for the mocked application);</li><li>pushing the container images to Google Artifact Registry;</li><li>writing the Terraform code for the Cloud Run instance, including the latest Terraform Cloud Run provider google_cloud_run_v2_service, the BETA launch stage, bucket provisioning, and permissions;</li><li>mounting the Cloud Storage saved config file in the Application container and sharing an in-memory Volume between the “init” rendering container and the Application container.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5dec1d6fa009" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Terraform — how I split my monolithic state]]></title>
            <link>https://medium.com/@adrianarba/terraform-how-i-split-my-monolithic-state-490916343dba?source=rss-7f487fa71b14------2</link>
            <guid isPermaLink="false">https://medium.com/p/490916343dba</guid>
            <category><![CDATA[infrastructure-as-code]]></category>
            <category><![CDATA[terraform]]></category>
            <dc:creator><![CDATA[Adrian Arba]]></dc:creator>
            <pubDate>Fri, 01 Mar 2024 15:02:26 GMT</pubDate>
            <atom:updated>2024-03-01T15:02:26.296Z</atom:updated>
            <content:encoded><![CDATA[<h3>Terraform — how I split my monolithic state</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*e_dat67xDVGiR4PmgO0S8g.png" /></figure><p>When project requirements are not clear from the start and/or you start working on a PoC for a large-scale infrastructure environment, you may find later on in the development process that among other things, you may run into difficulties scaling the infrastructure as new requirements come up or become clear and different components need to either stay tightly coupled or need to fan out into individual isolated components.</p><p>If not nipped in the bud early, this has the potential to become a cumbersome task the longer it sits in the backlog, becoming an ever-increasing tech debt.</p><p>Terraform code becomes harder to read, variables become harder to maintain, the state file keeps on growing and releases start taking more and more time as more and more components need to be validated against the ever-increasing single-state file.</p><p>You may also find that to tackle isolated components, you have to run through the same pipeline where all potential infrastructure changes are checked for just a small real change to that specific component.</p><p>Other pipelines/devs can’t interact with that state in parallel and efficiently to test out features or bugfixes and at one point, actually doing the change/split of state file components becomes a task that you have to perform in waves, have to plan for and, since so little people are doing it and so rarely, does not reassure you in terms of predictability.</p><p>As a good practice, try to make sure when you start building out the infra, that the requirements are clear, identify what resources need to be tightly coupled, and use different state files and data resources wherever possible where components can exist individually. As soon as you have more details, do the split of state, if necessary, don’t postpone it up to the point it becomes a scary action to do (live Production with tens or hundreds or even more components).</p><h3>Lessons learned</h3><p>On a project I worked on, we started by having to perform a PoC that turned out so well, that it became a template for onboarding other applications into the infrastructure we were building.</p><p>The problem was that we were tight on time budget and as soon as the PoC was presented, work was prioritized to start for the next phases which meant building out more infrastructure ASAP.</p><p>For the PoC, we implemented a state file for applications and a state file for the supporting infrastructure, with both tightly coupled components as well as components shareable between different applications and teams.</p><p>The new requirement was that all of these applications and teams would need to exist in their subset of services and live independently as much as possible, and the current PoC to immediately serve Production workloads.</p><p>In the infrastructure team, we immediately realized that if we scaled the infrastructure (herein terraform state) horizontally, we would slowly pave the path to chaos.</p><p>So, the lessons learned here were:</p><ul><li>Make sure you understand the elasticity of your future infrastructure, clarify if it needs to scale horizontally or vertically in terms of Terraform resources</li><li>Understand the needs and urgency of the teams relying on the infrastructure you provision — what can be destroyed/changed without a big impact</li><li>In general, don’t hold all your eggs in one basket unless explicitly asked to do so (lesson learned from upgrading Terraform from version 0.11 ..)</li><li>Don’t leave infrastructure rework in the backlog if possible</li><li>Don’t take PoCs to Production, ensure you have enough time to evaluate the work done and ask yourself what would you do better in version 2</li></ul><p>Coming back to our problem, we had two main options:</p><ul><li><strong>Option A</strong>: request for a bit more time to destroy the Production (and all other) environments (pause the dev work) rewrite the terraform components, and rebuild everything;</li><li><strong>Option B</strong>: work on addressing the issue ad-hoc, one environment at a time, rewriting the terraform components, creating new state files per component, and splitting the current monolithic state by migrating components to the new state files, and accounting for only small interruptions to dev work.</li></ul><p>I’m going to talk about <strong>Option B</strong>, the one we chose so that we limit the impact on dev work and balance introducing new infra features with splitting the old infra state on the fly as well as maintaining data added during the PoC unfold.</p><p>Below, without going into any other nonessential details, I’ll highlight how we managed to achieve this, at a terraform state level, with the following details:</p><ul><li>infrastructure was GCP-heavy</li><li>state files were stored in Cloud Storage buckets, versioned</li><li>we used Terragrunt to distinguish between environments, so the commands I used all started with Terragrunt, but bear in mind this is all Terrafrom functionality behind the hood</li><li>I used bash to run through the commands</li></ul><h3><strong>Step 1 — Add the code of the new component</strong></h3><p>As a first step, we started writing the new Terraform components in the way that we wanted, accounting for tightly coupled components, reusability, splitting shared components into a new folder, cleaning up variables, and reworking the Terragrunt folder structure to account for each component.</p><p>To simplify, I’ll use a tree level that contains only in-scope files (but imagine these paths contain a lot of terraform files):</p><pre># pre migration single component<br><br>└── project<br>    └── terraform_monolith<br>        ├── application.tf<br>        ├── database.tf<br>        └── variables.tf<br><br><br># intermediary step (where we are now)<br># this is where you would start adding the new terraform structure - &#39;database&#39;<br><br>└── project<br>    ├── terraform_monolith<br>    │   ├── application.tf<br>    │   ├── database.tf<br>    │   └── variables.tf<br>    │<br>    └── database<br>        ├── database.tf<br>        └── variables.tf<br><br><br># post migration components<br># &#39;terraform_monolith&#39; is renamed to &#39;application&#39; and cleaned up<br><br>└── project<br>    ├── application<br>    │   ├── application.tf<br>    │   └── variables.tf<br>    │<br>    └── database<br>        ├── database.tf<br>        └── variables.tf</pre><h3>Step 2 — Don’t remove any components from the monolithic component (yet)</h3><p>The initial monolithic component was left untouched because we need the state file in its full to be able to ‘move’ infrastructure from the monolithic state to the new states.</p><p>So any changes to the monolithic component will happen later, for now, just copy out whatever you need from here to the new components and edit in the new components folders (including the variables.tf files).</p><h3>Step 3 — Download the monolithic state file locally</h3><p>To interact with the monolithic state, first we downloaded it locally as a backup and only then we started running Terragrunt commands against the cloud stored one.</p><pre># pull the current backend state locally, as a backup<br># you have multiple choices:<br>#    - download from Cloud console<br>#    - dwnload via Cloud cli<br>#    - use Terraform&#39;s functionality (safest, because it locks the state in the bucket during the operation)<br><br># in your monolithic component<br>cd project/terraform_monolith<br><br># make sure everything is up to date and you can access the backend<br>terragrunt init<br><br># get a list of resources in this state file and save it in a text file<br># this helps you select individual resources to move<br>terragrunt state list<br><br># pull the state to a separate location<br>terragrunt state pull &gt; state_migration/terraform_monolith/monolith.tfstate</pre><p><strong><em>Note:</em></strong><em> If you use a remote bucket as a state backend, versioned, you will see that as you move resources from one state to another, your original state will become smaller and smaller. It is a move action, not a copy action. If you need to restore to the initial state (maybe something goes wrong), you will need to restore an older version of your bucket state, that contained all the resources in your monolithic component. Or, if you have a local backup that you know can’t be stale, you can work up from that.</em></p><p><strong><em>Note:</em></strong><em> Terraform keeps a state interaction counter in state files. Each time a state file is changed, the counter is incremented. When working with state files directly, make sure that you take care of any counter issues as well.</em></p><h3>Step 4 — Provision the state backends for the new components</h3><p>In this step, we needed to make sure that a Terragrunt init can be run against the empty components, with a state backend being created in the Cloud. To move the state around, besides the Terraform resources, you also need an output state file and an input state file.</p><p>So make sure you run this successfully in your new components (Terragrunt can create the state bucket for you in the Cloud estate, which is nice):</p><pre># switch to you new terraform component&#39;s root folder (where all the .tf files are).<br>cd project/database<br><br># run Terragrunt to initialize the backend, download all modules and create a state file in your designated bucket.<br># if the command is successfull, you are ready to migrate to this new state file<br>terragrunt init</pre><h3>Step 5 — Move the resources in the state from monolithic to split components</h3><p>Remember that list of resources we pulled at Step 3? We will need it here.</p><pre># in the same new component folder, continue with the next command<br># we are going to pull down the empty, newly initialized state file of the new component<br>terragrunt state pull &gt; state_migration/database/database.tfstate<br><br># switch back to the monolithic component<br>cd project/terraform_monolith<br><br># and start moving state to the new component using the destination local state file<br><br>  # using a few resource as examples<br>    # a module <br>terragrunt state mv -state-out=state_migration/database/database.tfstate &quot;module.pubsub.google_pubsub_subscription.pull_subscriptions[\&quot;my_pull_subscription\&quot;]&quot; &quot;module.pubsub.google_pubsub_subscription.pull_subscriptions[\&quot;my_pull_subscription\&quot;]&quot;<br><br>    # a regular resource<br>terragrunt state mv -state-out=state_migration/database/database.tfstate &quot;google_pubsub_topic_iam_member.subscriber&quot; &quot;google_pubsub_topic_iam_member.subscriber&quot;<br><br>    # a list element resource<br>terragrunt state mv -state-out=state_migration/database/database.tfstate &quot;google_service_account.app_service_account[0]&quot; &quot;google_service_account.app_service_account[0]&quot;</pre><p>Each run of <em>state mv</em> will remove that resource from the source state.</p><p>Let’s break down the command:</p><pre>terragrunt state mv                   # or terraform state mv - the command to poll the component&#39;s backend state and move one resource to another state<br><br>  -state-out=/path/to/file.tfstate      # the destination state file<br><br>  1st &quot;cloud_resource.name&quot;             # original state resource name<br><br>  2nd &quot;cloud_resource.name&quot;             # destination state resource name</pre><h3>Step 6 — Update the new state component’s backend state</h3><p>So far, the only state file that got changed after all the above commands was the terraform_monolith component state (hence the initial backup, just in case something goes terribly wrong..).</p><p>But the new components state (database) was only updated at a local file state, the backend state has not yet been updated, and that’s the last piece of the puzzle, really:</p><pre># switch paths to your new component&#39;s Terraform configuration<br>cd project/database<br><br># simply run a state push using the destination state local file<br>terragrunt state push state_migration/database/database.tfstate<br><br># and validate everything works fine<br>terragrunt init<br><br># you should see all of your defined resources correspondinf moved states<br># with no changes to be performed<br>terragrunt plan</pre><h3>Step 7 (Optional)— Delete/keep what you need from your monolithic component</h3><p>At this stage, you can choose to keep your trimmed-down terraform_monolith component, rename it, or delete it if you migrated everything.</p><p>During the state migration, you can choose to migrate an entire state file to another state file, split a state file into multiple new state files, or move parts of a state file to a new state file and keep what’s left in the original one.</p><p>You can choose to rename resources in the destination state file as well or delete resources entirely if not needed.</p><p>Hope this tutorial helps you stay on top of you infrastructure :)</p><p>Here’s another source I found helpful when building out my solution, the steps are really well explained — <a href="https://www.maxivanov.io/how-to-move-resources-and-modules-in-terragrunt/">https://www.maxivanov.io/how-to-move-resources-and-modules-in-terragrunt/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=490916343dba" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Terraform “dynamic” inside “dynamic” blocks — an implementation example]]></title>
            <link>https://medium.com/@adrianarba/terraform-dynamic-inside-dynamic-blocks-an-implementation-example-c129f62135bb?source=rss-7f487fa71b14------2</link>
            <guid isPermaLink="false">https://medium.com/p/c129f62135bb</guid>
            <category><![CDATA[gcp]]></category>
            <category><![CDATA[foreach]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[cloud]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Adrian Arba]]></dc:creator>
            <pubDate>Mon, 06 Nov 2023 21:05:44 GMT</pubDate>
            <atom:updated>2023-11-12T12:09:22.342Z</atom:updated>
            <content:encoded><![CDATA[<h3>Terraform “dynamic” inside “dynamic” blocks — an implementation example</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sCQP6zeuPgyi_Kke5rs0Vw.png" /></figure><p>Sometimes, to tackle seemingly easy tasks, one needs to venture into uncharted territory to find a solution.</p><p>Such is the case with one implementation I had to add for my project, where we decided to use GCP’s Artifact Registry (new-ish) service to store the project artifacts — Docker images.</p><p>A simple enough implementation by itself, where if you have all the necessary permissions for your deploy user, all you need in Terraform is a resource block similar to the one below:</p><pre>resource &quot;google_artifact_registry_repository&quot; &quot;my-repo&quot; {<br>  provider               = google-beta<br><br>  location               = var.gcp_region<br>  project                = var.gcp_project_id<br>  repository_id          = var.repository_id<br>  description            = var.description<br>  format                 = var.format<br>  labels                 = var.labels<br>}</pre><p>Now, things became complicated when we noticed we kept pushing images to the repo, but there was no cleanup process set up to get rid of the unused artifacts.</p><p>There were multiple possible implementations, such as the gcloud cli or maybe potentially through Google’s other container handling tool, gcrane , or even directly through the Console, but, let’s face it, none of these options are “cool” enough for a seasoned engineer.</p><p>I needed something that could be deployed once and then I could forget about it trusting it will do its job.</p><p>Reading about possible options, I found out that the google_artifact_registry_repository resource actually supports defining cleanup policies as terraform code, although the feature is still in Beta — have a read <a href="https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/artifact_registry_repository#example-usage---artifact-registry-repository-cleanup">here</a>.</p><p>Which brings me to the point of this post. These cleanup policies are optional, but, if you do add them, you need to follow specific structures because in case a policy contains a value, it cannot contain other values. Like, for instance, actions and conditions. Per a specific action, only specific conditions can be declared.</p><p>Both cleanup policies and conditions can or cannot exist and conditions are nested within a policy.</p><p>Long story short, I get to the point where, for something pretty simple, I need to do some complicated Terraform logic, using nested dynamic blocks.</p><p>So I’m going to detail in this article how I added these optional policies with nested dynamic blocks, using maps, and objects, and lists to build my policies variable.</p><p>Let’s kick off with the variable itself and some inline explanations:</p><pre># variables.tf<br><br># the policies variable<br>variable &quot;cleanup_policies&quot; {<br>  description = &quot;Object containing details about the cleanup policies for an Artifact Registry repository.&quot;<br>  type = map(object({<br>    id     = string<br>    action = string<br>    # since a policy can contain either a condition or a most_recent_versions<br>    # block, these fields are all marked as `optional`<br>    condition = optional(list(object({<br>      # since not all condition blocks support all of the 4 parameters<br>      # they are all optional as well<br>      tag_state             = optional(string)<br>      tag_prefixes          = optional(list(string))<br>      older_than            = optional(string)<br>      package_name_prefixes = optional(list(string))<br>    })))<br>    most_recent_versions = optional(list(object({<br>      package_name_prefixes = optional(list(string))<br>      keep_count            = optional(number)<br>    })))<br>  }))<br><br>  # the default ramains null because I want to ensure I don&#39;t break<br>  # previous implementations where policies do not exist<br>  default = null<br>}<br><br># the dry run vs live toggle for the policies<br>variable &quot;cleanup_policies_dry_run&quot; {<br>  description = &quot;If set to true, the cleanup pipeline is prevented from deleting artifacts in this repository.&quot;<br>  type        = bool<br>  default     = null<br>}</pre><p>Now, I went with a map of objects, because I didn’t want to complicate with a list, where I needed to do some further transformations in the dynamic blocks using a for loop, but you can go via that implementation as well..</p><pre># main.tf<br><br>resource &quot;google_artifact_registry_repository&quot; &quot;my-repo&quot; {<br>  provider               = google-beta<br><br>  location               = var.gcp_region<br>  project                = var.gcp_project_id<br>  repository_id          = var.repository_id<br>  description            = var.description<br>  format                 = var.format<br>  labels                 = var.labels<br><br>  # cleanup policies under this line<br>  cleanup_policy_dry_run = var.cleanup_policies_dry_run<br><br>  # the first dynamic block will decide if the resource will be executed with<br>  # cleanup policies based on the variable being null or not<br>  dynamic &quot;cleanup_policies&quot; {<br>    # if it is null, an empty “cleanup_policies” dynamic block will be created<br>    # without any actual effect on the resource<br>    for_each = var.cleanup_policies == null ? {} : var.cleanup_policies<br><br>    content {<br>      id     = cleanup_policies.value.id<br>      action = cleanup_policies.value.action<br><br>      # second level of dynamic blocks<br><br>      # the dynamic blocks for_each conditional evaluations are set to operate<br>      # based on what the terraform resource supports as parameters<br>      dynamic &quot;condition&quot; {<br>        for_each = ( cleanup_policies.value.condition != null &amp;&amp; cleanup_policies.value.action == &quot;KEEP&quot; ) ? [cleanup_policies.value.condition] : []<br><br>        content {<br>          # each condition is a list with exactly one element, hence [0] but<br>          # in a scenario where the list has many elements, you can use [*]<br>          tag_state             = condition.value[0].tag_state<br>          tag_prefixes          = condition.value[0].tag_prefixes<br>          package_name_prefixes = condition.value[0].package_name_prefixes<br>        }<br>      }<br><br>      dynamic &quot;condition&quot; {<br>        for_each = ( cleanup_policies.value.condition != null &amp;&amp; cleanup_policies.value.action == &quot;DELETE&quot; ) ? [cleanup_policies.value.condition] : []<br><br>        content {<br>          tag_state    = condition.value[0].tag_state<br>          tag_prefixes = condition.value[0].tag_prefixes<br>          older_than   = condition.value[0].older_than<br>        }<br>      }<br><br>      dynamic &quot;most_recent_versions&quot; {<br>        for_each = ( cleanup_policies.value.most_recent_versions != null &amp;&amp; cleanup_policies.value.action == &quot;KEEP&quot; ) ? [cleanup_policies.value.most_recent_versions] : []<br><br>        content {<br>          package_name_prefixes = most_recent_versions.value[0].package_name_prefixes<br>          keep_count            = most_recent_versions.value[0].keep_count<br>        }<br>      }<br>    }<br>  }<br>}</pre><p>So a lot of complications for something seemingly easy. Especially since I need to go via each sub-dynamic block based on the rules of the resource API and through for_each evaluations.. Luckily, there are only 3 possible scenarios, at the point of writing and I decided to stop the logic there and not go into making even more rules for the optional parameters configurations. I rely on a solid API Terraform error for that if I misconfigure a resource that badly.</p><p>Aaaand.. a variable example to tie everything together:</p><pre># input.tfvars<br><br># let&#39;s remove the dry run mode - it can take up to one day for a cleanup to<br># occur on your repository, so be midful of that<br>cleanup_policies_dry_run = false<br><br># and a set of 3 policies<br>cleanup_policies = {<br>  cleanup_stale_versions = {<br>    id     = &quot;delete-tagged-older-than-20-days&quot;<br>    action = &quot;DELETE&quot;<br>    condition = [{<br>      tag_state    = &quot;ANY&quot;<br>      older_than   = &quot;1728000s&quot;<br>    }]<br>  },<br>  keep_tagged_versions = {<br>    id     = &quot;keep-tagged-versions&quot;<br>    action = &quot;KEEP&quot;<br>    condition = [{<br>      tag_state             = &quot;TAGGED&quot;<br>      tag_prefixes          = [&quot;latest&quot;, &quot;Release&quot;]<br>      package_name_prefixes = [&quot;backend&quot;, &quot;frontend&quot;]<br>    }]<br>  },<br>  keep_minimum_versions = {<br>    id     = &quot;keep-minimum-of-versions&quot;<br>    action = &quot;KEEP&quot;<br>    most_recent_versions = [{<br>      package_name_prefixes = [&quot;backend&quot;, &quot;frontend&quot;]<br>      keep_count            = 5<br>    }]<br>  }<br>}</pre><p>A few things to note:</p><ul><li>Cleanup jobs occur roughly once a day, so be patient after you apply the code, wait a day to check if the policies worked successfully (or you can go via a dry run and check what would be deleted — details in the <a href="https://cloud.google.com/artifact-registry/docs/repositories/cleanup-policy#dry-run">GCP documentation</a>);</li><li>Prefixes mean prefixes, which is, add the predictable prefix of a tag or package and GCP will select anything starting with that prefix — in my case, I reference the Release tag prefix, which means that Google will target anything starting with Release so Release_Dec or Release202312 and so on;</li><li>If you go with a list of objects (type = list(object({ .. }))) instead of a map, you could do something like this in the code, where you take an element of your object and you transform it into a map with its value as a key:</li></ul><pre>  dynamic &quot;cleanup_policies&quot; {<br>    for_each = var.cleanup_policies == null ? {} : { for policy in var.cleanup_policies : policy.id =&gt; policy }<br>    <br>    content {<br>      id     = policy.key<br>      action = policy.value.action<br><br>      ...</pre><p>Let me know if this helped you in your endeavours and please offer any improvement suggestions in the comments, let’s all grow together :)</p><p>Thank you!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c129f62135bb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Tekton — multiple GitHub repositories Triggers serving one EventListener]]></title>
            <link>https://medium.com/@adrianarba/tekton-multiple-github-repositories-triggers-serving-one-eventlistener-93aa3bd7c332?source=rss-7f487fa71b14------2</link>
            <guid isPermaLink="false">https://medium.com/p/93aa3bd7c332</guid>
            <category><![CDATA[ci-cd-pipeline]]></category>
            <category><![CDATA[tekton]]></category>
            <category><![CDATA[github]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[listener]]></category>
            <dc:creator><![CDATA[Adrian Arba]]></dc:creator>
            <pubDate>Sun, 06 Aug 2023 11:27:07 GMT</pubDate>
            <atom:updated>2023-10-14T20:38:25.063Z</atom:updated>
            <content:encoded><![CDATA[<h3>Tekton — multiple GitHub repositories triggering one EventListener</h3><p>While working on one project, as a DevOps engineer, I was given an interesting task, connect multiple GitHub repositories to one EventListener in Tekton, to serve the CI/CD needs of our project team, composed of backend, frontend, and infrastructure engineers. The project was allocated a Tekton space with a limitation to 2 EventListeners, and other teams were supposed to join the project soon.</p><p>As a general practice, and to avoid complicated logic, a Tekton EventListener should listen to one repository and should operate as a pair, allowing the Interceptors &amp; Triggers to respond to specific events from that repository. However, this was not the case for my task.</p><p>So let’s see what one way of resolving this task might look like in this article.</p><p>Now, I did not set up the environment, I just operate the provided Tekton space, so I don’t have any input on the installation of Tekton or the provisioning of the GitHub secret used in conjunction with the webhook, but I want to point out briefly the components of this ecosystem, using the below diagram as visual:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TaUISpF2QzoewDvqv83czA.png" /><figcaption>The high-level view of the general Tekton ecosystem.</figcaption></figure><p>From left to right, we have:</p><ul><li>GitHub webhooks — these are pre-configured payload messages, usually in a JSON format, that GitHub can send to other applications based on different events, like a pull_request or a push . For webhooks to be able to send these payloads to your system, you need to provide the URL of a service that can receive and parse the payload, together with a valid Token — if you use Tekton, that service takes the form of a Routeresource;</li><li>Routes act like an ingress, exposing a URL outside of the Openshift or Kubernetes cluster to which webhooks connect and send payloads. Just like an ingress resource, Routes then map to a service inside the cluster that sends the payloads to the EventListenerPod.</li><li>The EventListener defines Triggers and Interceptors that meet GitHub payloads and then, based on filters, the payloads can be scanned, transformed, and forwarded to TriggerBindings.</li><li>TriggerBindings bridge the EventListener to TriggerTemplates and also expose Parameters from the webhook body message to be used throughout the Pipeline execution lifecycle.</li><li>TriggerTemplates define how PipelineRuns will be created to kick off eventual Pipelines automatically, assigning Volumes and populating Parameters with payload information to be used later on by Tasks.</li><li>Tasks are a list of actions within the Pipeline.</li><li>Steps are the atomic component of Tekton and one Task can have one or multiple Steps defined, each Step creating its own Pod, loading the defined Container.</li></ul><p>Of course, the discussion is broader, but this should give you an idea of how Tekton operates. We will mostly focus on the first 3 bullet points in this article.</p><h3>GitHub Webhooks</h3><p>In GitHub, you need to make sure you are an Admin of the repository you want to link to Tekton. You should have access to the Settings &gt; Hooks section, like so:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YgIY_LnjodadbR7jdPMPqA.png" /></figure><p>I have configured an HTTPS Route to be linked to this webhook and which will be informed based on pull_request and push events, but you can select from a multitude of different event types. A green mark in front of the URL shows that everything is working correctly (the URL is reachable and can be connected via the supplied Tekton secret token).</p><p><strong><em>Note:</em></strong><em> Sometimes, when adding the URL and the Token, you may see a red X instead of the green checkmark in front of the URL, even after a few minutes, appearing like the route is not working. Try doing something on a branch, like pushing a commit or a similar git action, so that an actual payload will be sent out to your </em><em>Route. The mark will change from Red to Green (fingers crossed :D).</em></p><p>In terms of configuring the webhook, this is how I set it up, and I’ll go over each point:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JmMsTzFsMd6NncKrYG7OQw.png" /><figcaption>Webhook configuration</figcaption></figure><ol><li>Payload URL — this is where you paste in your Route’s exposed URL (you have a choice of creating an HTTP or HTTPS Route via Tekton);</li><li>Since we are using Tekton, we’re expecting to receive content of the type application/json so that’s what I’ve selected;</li><li>In the Secret section, I’ve pasted in my Tekton secret configured for this specific GitHub webhooks integration;</li><li>Since I&#39;ve set up an HTTPS-type Route, I leave SSL verification Enabled;</li><li>Since I only want to handle pull_request and push event types, I select them individually from the list of events.</li></ol><h3><strong>The Route</strong></h3><p>The rest of the resources that we need to create will be created on the Tekton side. Since Tekton is basically an Operator in Kubernetes that leverages CRDs, so will our Resources be defined as yaml Kubernetes resource files, and the Route is no exception.</p><p>However, although I am using yaml files, you can configure everything from the Tekton Dashboard GUI interface.</p><p>Now, in order to expose an HTTPS route, you will need to configure a Domain and have the route created based on your private domain, if you are targeting a Production grade deployment of the solution, and I already have something set up by my admins, so I’ll only showcase how the Route resource looks like.</p><p><strong><em>Note:</em></strong><em> a Route is not actually a Tekton resource, but is the Openshift ingress resource available out of the box. Clear enough if we look at the </em><em>apiVersion field.</em></p><pre>apiVersion: route.openshift.io/v1<br>kind: Route<br>metadata:<br>  name: my-webhook<br>  namespace: my-namespace<br>  labels:<br>    app: my-cicd-pipeline<br>    app.kubernetes.io/managed-by: EventListener<br>    app.kubernetes.io/part-of: Triggers<br>    eventlistener: my-cicd-eventlistener<br>  annotations:<br>    openshift.io/host.generated: &#39;true&#39;<br>spec:<br><br>  # This will be your exposed webhook URL, prefixed by https://<br>  host: my-webhook.example.com<br><br>  # This service will be created automatically and will link the Route to the EventListener Pod<br>  to:<br>    kind: Service<br>    name: my-webhook-service<br>    weight: 100<br><br>  port:<br>    targetPort: http-listener<br><br>  wildcardPolicy: None<br><br>  # This part relates to enabling HTTPS on your Route<br>  tls:<br>    termination: edge<br>    insecureEdgeTerminationPolicy: Redirect</pre><h3>The EventListener</h3><p>For context, let’s imagine we have 2 teams in the project, each with an individual repository that we will set up a Trigger for:</p><pre>TEAM      RESOURCE    GITHUB_ORG/GITHUB_REPO  <br>Team 1    teamone     example-corp/team-one-repo<br>Team 2    teamtwo     example-corp/team-two-repo</pre><p>We’ll take a look at the EventListener‘s yaml file next:</p><pre>apiVersion: triggers.tekton.dev/v1beta1<br>kind: EventListener<br>metadata:<br>  name: my-cicd-eventlistener<br>  namespace: my-namespace<br>  labels:<br>    app: my-cicd-pipeline<br>spec:<br>  serviceAccountName: pipeline<br>  namespaceSelector: {}<br>  resources:<br>    kubernetesResource:<br>      spec:<br>        template:<br>          metadata:<br>            creationTimestamp: null<br>          spec:<br>            containers:<br>              - resources:<br>                  limits:<br>                    cpu: 25m<br>                    ephemeral-storage: 150Mi<br>                    memory: 96Mi<br>                  requests:<br>                    cpu: 5m<br>                    ephemeral-storage: 75Mi<br>                    memory: 48Mi<br>            nodeSelector:<br>              node-role.kubernetes.io/el: &quot;&quot;<br>            serviceAccountName: pipeline<br>            tolerations:<br>              - effect: NoSchedule<br>                key: node-role.kubernetes.io/el<br>                operator: Exists<br><br>  # Each trigger suits one team and consequently one team&#39;s repository.<br>  triggers:<br><br>    #########################<br>    #### Team 1 triggers ####<br>    #########################<br><br>    # The pull-request trigger gets fired on opening or updating a pull-request<br>    - name: teamone-pull-request<br><br>      # Bindings reference the TriggerBinding resource the trigger will call on when it is fired<br>      bindings:<br>        - kind: TriggerBinding<br><br>          # This is another resource I create as a yaml file<br>          ref: teamone-github-pullrequest-trigger-binding<br><br>      # And this references the TriggerTemplate<br>      template:<br><br>        # Which is another resource I create as a yaml file<br>        ref: teamone-github-pullrequest-trigger-template<br><br>      # And then we have the interceptors which do the heavy lifting<br>      interceptors:<br><br>        # It always helps to give them a name<br>        - name: &quot;Defines the event type&quot;<br>          ref:<br>            kind: ClusterInterceptor<br>            name: github<br>          params:<br><br>            # We pass on the same Token as the one used by GitHub&#39;s webhook<br>            - name: secretRef<br>              value:<br>                secretKey: secretToken<br>                secretName: github-webhooks-secret<br>    <br>            # I&#39;m only interested in pull_request events from this team<br>            - name: eventTypes<br>              value:<br>                - pull_request<br><br>        - name: &quot;Checks the type of incoming event and fires only on defined types&quot;<br>          ref:<br><br>            # You should reallly read about the `cel` interceptor<br>            kind: ClusterInterceptor<br>            name: cel<br>          params:<br>            <br>            # Filters are the way we triage requests coming from different repositories<br>            #   body.repository.full_name and body.action are parts of the GitHub payload json<br>            #   .matches(&#39;repo_name&#39;) is a function to search for a string in the body of the<br>            #     payload field &quot;full_name&quot; and we map it to the actual repository name<br>            #   in [&#39;string1&#39;,&#39;string2&#39;] is how we filter out the GitHub event action type<br>            - name: filter<br>              value: &gt;-<br>                body.repository.full_name.matches(&#39;example-corp/team-one-repo&#39;) &amp;&amp;<br>                body.action in [&#39;opened&#39;, &#39;reopened&#39;, &#39;edited&#39;, &#39;synchronize&#39;]<br><br>            # Overlays are CEL expressions added by the Trigger to the event payload in the<br>            #  top-level extensions field. Overlays are accessible from TriggerBindings.<br>            # Here I&#39;m exposing the label key with the value of a transformed repository name,<br>            #   to use later in the Pipeline.<br>            - name: overlays<br>              value:<br>                - expression: body.repository.full_name.lowerAscii().replace(&#39;/&#39;,&#39;-&#39;).replace(&#39;.&#39;, &#39;-&#39;).replace(&#39;_&#39;, &#39;-&#39;)<br>                  key: label<br><br>    <br><br>    # And similar for Team 2:<br><br>    #########################<br>    #### Team 2 triggers ####<br>    #########################<br><br>    # The pull-request trigger gets fired on opening or updating a pull-request<br>    - name: teamtwo-pull-request<br>      bindings:<br>        - kind: TriggerBinding<br>          ref: teamtwo-github-pullrequest-trigger-binding<br>      template:<br>        ref: teamtwo-github-pullrequest-trigger-template<br>      interceptors:<br><br>        - name: &quot;Defines the event type&quot;<br>          ref:<br>            kind: ClusterInterceptor<br>            name: github<br>          params:<br>            - name: secretRef<br>              value:<br>                secretKey: secretToken<br>                secretName: github-webhooks-secret<br>            - name: eventTypes<br>              value:<br>                - pull_request<br><br>        - name: &quot;Checks the type of incoming event and fires only on defined types&quot;<br>          ref:<br>            kind: ClusterInterceptor<br>            name: cel<br>          params:<br>            - name: filter<br>              value: &gt;-<br>                body.repository.full_name.matches(&#39;example-corp/team-two-repo&#39;) &amp;&amp;<br>                body.action in [&#39;opened&#39;, &#39;reopened&#39;, &#39;edited&#39;, &#39;synchronize&#39;]<br>            - name: overlays<br>              value:<br>                - expression: body.repository.full_name.lowerAscii().replace(&#39;/&#39;,&#39;-&#39;).replace(&#39;.&#39;, &#39;-&#39;).replace(&#39;_&#39;, &#39;-&#39;)<br>                  key: label</pre><p>The main thing to note is I use body.repository.full_name.matches(‘org/repo_name’) to run a filter per each repository and link it to other conditions using &amp;&amp; or || — Tekton interpreters — and I then provide the logic on how Triggers will fire requests.</p><p>The drawback is I have to do this for each filter in particular, duplicating code lines and there are some limitations to these filters, however, for my purpose, this does the job.</p><p>If you need to do any troubleshooting on the filters, remember that the EventListener runs under a Pod in Kubernetes, so you can view the log of that Pod and check which Triggers misfire.</p><p>Let me know if you have tackled this challenge in a different way, really curious about how I can improve this task further.</p><p>Thank you for your read! :)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=93aa3bd7c332" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Serve Single-Page Applications from S3 through CloudFront with external WAF — a Terraform guide…]]></title>
            <link>https://medium.com/@adrianarba/serve-single-page-applications-from-s3-through-cloudfront-with-external-waf-a-terraform-guide-9f5c58bc21d2?source=rss-7f487fa71b14------2</link>
            <guid isPermaLink="false">https://medium.com/p/9f5c58bc21d2</guid>
            <category><![CDATA[terraform-modules]]></category>
            <category><![CDATA[aws-lambda]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[cloudfront]]></category>
            <category><![CDATA[terraform]]></category>
            <dc:creator><![CDATA[Adrian Arba]]></dc:creator>
            <pubDate>Sat, 10 Jun 2023 14:11:05 GMT</pubDate>
            <atom:updated>2023-06-10T14:11:05.044Z</atom:updated>
            <content:encoded><![CDATA[<h3>Serve Single-Page Applications from S3 through CloudFront with external WAF — a Terraform guide (Part 2)</h3><p>Up next, configurations to the modules :) It’s how I put the modules into practice and leverage what they represent, a template for multiple flavors of the infrastructure.</p><p>Going back to the Repository structure, the focus in Part 2 will be on the configurations folder, namely the single_page_app folder:</p><pre>├── repository_a<br>│   └── (...) # terragrunt - out of scope<br>│<br>└── repository_b<br>    │<br>    ├── <strong>configurations</strong><br>    │   │<br>    │   └── <strong>single_page_app</strong><br>    │       │<br>    │       ├── backend.tf<br>    │       ├── cf.tf<br>    │       ├── data.tf<br>    │       ├── script.py<br>    │       ├── iam.tf<br>    │       ├── kms.tf<br>    │       ├── lambda.tf<br>    │       ├── locals.tf<br>    │       ├── outputs.tf<br>    │       ├── s3.tf<br>    │       └── variables.tf<br>    │<br>    └── modules<br>        │<br>        ├── cloudfront<br>        │   │<br>        │   ├── main.tf<br>        │   ├── outputs.tf<br>        │   └── variables.tf<br>        │<br>        ├── s3_app<br>        │   │<br>        │   ├── main.tf<br>        │   ├── outputs.tf<br>        │   └── variables.tf<br>        │<br>        └── lambda<br>            │<br>            ├── data.tf<br>            ├── main.tf<br>            ├── outputs.tf<br>            ├── providers.tf<br>            └── variables.tf</pre><p>The backend.tf file contains the following code, taking into consideration that Lambda@EDGE functions run only in the us-east-1 region:</p><pre>terraform {<br>  required_version = &quot;~&gt; 1.3.0&quot;<br><br>  required_providers {<br>    aws = {<br>      source  = &quot;hashicorp/aws&quot;<br>      version = &quot;~&gt; 4.0&quot;<br>    }<br><br>    local = {<br>      source  = &quot;hashicorp/local&quot;<br>      version = &quot;2.4.0&quot;<br>    }<br><br>    # I use archive because I will be uploading the function code<br>    # during the Terraform execution<br>    archive = {<br>      source  = &quot;hashicorp/archive&quot;<br>      version = &quot;2.3.0&quot;<br>    }<br>  }<br>}<br><br># The general AWS provider<br>provider &quot;aws&quot; {<br>  region = &quot;eu-west-1&quot;<br>}<br><br># I define the provider for the Lambda@EDGE function as an alias<br>provider &quot;aws&quot; {<br>  alias  = &quot;us-east-1&quot;<br>  region = &quot;us-east-1&quot;<br>}<br><br># The backend bucket information is configured in Terragrunt<br>terraform {<br>  backend &quot;s3&quot; {}<br>}</pre><p>The cf.tf file contains the Cloudfront configuration:</p><pre># I&#39;m creating the OAC at this level since the CloudFront module cand<br># be used both with OAI and OAC<br>resource &quot;aws_cloudfront_origin_access_control&quot; &quot;oac&quot; {<br>  name        = var.c_cf_oac_name<br>  description = var.c_cf_oac_description<br><br>  origin_access_control_origin_type = var.c_cf_oac_origin_type<br>  signing_behavior                  = var.c_cf_oac_signing_behaviour<br>  signing_protocol                  = var.c_cf_oac_signing_protocol<br>}<br><br>module &quot;distribution&quot; {<br>  depends_on = [<br>    module.bucket,<br>    aws_cloudfront_origin_access_control.oac<br>  ]<br><br>  source     = &quot;../../modules/cloudfront&quot;<br><br>  m_cf_target_origin_id                 = module.static_web_bucket.bucket_id<br>  m_cf_origins                          = local.c_cf_origins<br>  m_cf_default_cache_lambda_association = local.c_cf_default_cache_lambda_association<br>  m_cf_comment                          = var.c_cf_oac_name<br>  m_cf_aliases                          = var.c_cf_alternate_domain_names<br>  m_cf_cloudfront_default_certificate   = var.c_cf_cloudfront_default_certificate<br>  m_cf_default_cache_methods            = var.c_cf_default_cache_methods<br>  m_cf_geo_location                     = var.c_cf_geo_location<br>  m_cf_geo_restriction_type             = var.c_cf_geo_restriction_type<br>  m_cf_ssl_method                       = var.c_cf_ssl_method<br>  m_cf_minimum_tls_version              = var.c_cf_minimum_tls_version<br>  m_cf_cloudfront_certificate_arn       = var.c_cf_cloudfront_certificate_arn<br>}</pre><p>Here’s an example of how I would pass the input for the c_cf_default_cache_methods variable, to feed it to the m_cf_default_cache_methods module variable (which are the same type, type = object({}):</p><pre>c_cf_default_cache_methods = {<br>  path_pattern      = &quot;&quot;<br>  allowed_methods   = [&quot;DELETE&quot;, &quot;GET&quot;, &quot;HEAD&quot;, &quot;OPTIONS&quot;, &quot;PATCH&quot;, &quot;POST&quot;, &quot;PUT&quot;]<br>  cache_methods     = [&quot;GET&quot;, &quot;HEAD&quot;]<br>  viewer_policy     = &quot;https-only&quot;<br>  min_ttl           = 0<br>  default_ttl       = 3600<br>  max_ttl           = 86400<br>  forwarded_headers = &quot;&quot;<br>  cookies_forward   = &quot;none&quot;<br>}</pre><p>I’m using data.tf mainly to create policies for the bucket, kms key and oac permissions. It’s a big file as we wanted to be very granular and particular with our permissions, but here’s a small excerpt:</p><pre># Using a pgp key to make sure that the IAM users we create specifically<br># for this configuration have their keys stored secretly and ecnrypted into<br># the terraform state file. We have a pgp key created with which we manipulate<br># this resource<br>data &quot;local_file&quot; &quot;pgp_key&quot; {<br>  filename = var.pgpkey<br>}<br><br># Getting details about the user currently deploying the infrastructure<br>data &quot;aws_caller_identity&quot; &quot;current&quot; {}<br><br>(...)<br><br># The IAM policy for the IAM designated user<br>data &quot;aws_iam_policy_document&quot; &quot;put_object&quot; {<br>  statement {<br>    sid    = &quot;UserPutObjectsInBucket&quot;<br>    effect = &quot;Allow&quot;<br>    actions = [<br>      &quot;s3:PutObject&quot;,<br>      &quot;s3:PutObjectLegalHold&quot;<br>    ]<br>    resources = [&quot;${module.s3_bucket.bucket_arn}/*&quot;]<br>  }<br>}<br><br># Bucket level policies<br>data &quot;aws_iam_policy_document&quot; &quot;bucket_policy&quot; {<br>  statement {<br>    sid       = &quot;AllowCloudFrontServicePrincipalReadOnly&quot;<br>    effect    = &quot;Allow&quot;<br>    actions   = [&quot;s3:GetObject&quot;]<br>    resources = [&quot;${module.s3_bucket.bucket_arn}/*&quot;]<br><br>    principals {<br>      type        = &quot;Service&quot;<br>      identifiers = [&quot;cloudfront.amazonaws.com&quot;]<br>    }<br><br>    condition {<br>      test     = &quot;StringEquals&quot;<br>      variable = &quot;AWS:SourceArn&quot;<br>      values   = [&quot;arn:aws:cloudfront::${var.aws_account_number}:distribution/${module.distribution.cf_distribution_id}&quot;]<br>    }<br>  }<br><br>  statement {<br>    sid       = &quot;AllowPutObjects&quot;<br>    effect    = &quot;Allow&quot;<br>    actions   = [&quot;s3:PutObject&quot;]<br>    resources = [&quot;${module.s3_bucket.bucket_arn}/*&quot;]<br><br>    principals {<br>      type        = &quot;AWS&quot;<br>      identifiers = [&quot;arn:aws:iam::${var.aws_account_number}:user/${aws_iam_user.iam_user.name}&quot;]<br>    }<br>  }<br><br>  statement {<br>    sid     = &quot;AllowSSLRequestsOnly&quot;<br>    effect  = &quot;Deny&quot;<br>    actions = [&quot;s3:*&quot;]<br>    resources = [<br>      &quot;${module.s3_bucket.bucket_arn}&quot;,<br>      &quot;${module.s3_bucket.bucket_arn}/*&quot;<br>    ]<br><br>    principals {<br>      type        = &quot;*&quot;<br>      identifiers = [&quot;*&quot;]<br>    }<br><br>    condition {<br>      test     = &quot;Bool&quot;<br>      variable = &quot;aws:SecureTransport&quot;<br>      values   = [&quot;false&quot;]<br>    }<br>  }<br><br>  statement {<br>    sid       = &quot;RequireKMSEncryption&quot;<br>    effect    = &quot;Deny&quot;<br>    resources = [&quot;${module.s3_bucket.bucket_arn}/*&quot;]<br>    actions   = [&quot;s3:PutObject&quot;]<br><br>    principals {<br>      type        = &quot;*&quot;<br>      identifiers = [&quot;*&quot;]<br>    }<br><br>    condition {<br>      test     = &quot;StringNotEquals&quot;<br>      variable = var.s3_encryption_variable_test_one<br>      values   = [&quot;${var.c_cf_kms_key_encryption}&quot;]<br>    }<br>  }<br><br>  statement {<br>    sid       = &quot;RequireSpecificKMSKey&quot;<br>    effect    = &quot;Deny&quot;<br>    resources = [&quot;${module.s3_bucket.bucket_arn}/*&quot;]<br>    actions   = [&quot;s3:PutObject&quot;]<br><br>    principals {<br>      type        = &quot;*&quot;<br>      identifiers = [&quot;*&quot;]<br>    }<br><br>    condition {<br>      test     = &quot;StringNotLikeIfExists&quot;<br>      variable = var.c_cf_s3_encryption_var<br>      values   = [&quot;arn:aws:kms:${var.aws_region}:${var.aws_account_number}:key/${aws_kms_key.bucket_key.id}&quot;]<br>    }<br>  }<br><br>  statement {<br>    sid       = &quot;DenyUnEncryptedObjectUploads&quot;<br>    effect    = &quot;Deny&quot;<br>    resources = [&quot;${module.s3_bucket.bucket_arn}/*&quot;]<br>    actions   = [&quot;s3:PutObject&quot;]<br><br>    condition {<br>      test     = &quot;Null&quot;<br>      variable = var.c_cf_s3_encryption_var<br>      values   = [&quot;true&quot;]<br>    }<br><br>    principals {<br>      type        = &quot;*&quot;<br>      identifiers = [&quot;*&quot;]<br>    }<br>  }<br>}<br><br>data &quot;aws_iam_policy_document&quot; &quot;lambda_assume_role&quot; {<br>  statement {<br>    effect = &quot;Allow&quot;<br><br>    principals {<br>      type = &quot;Service&quot;<br>      identifiers = [<br>        &quot;lambda.amazonaws.com&quot;,<br>        &quot;edgelambda.amazonaws.com&quot;<br>      ]<br>    }<br><br>    actions = [&quot;sts:AssumeRole&quot;]<br>  }<br>}<br><br>data &quot;aws_iam_policy_document&quot; &quot;oac_kms_key_access&quot; {<br>  statement {<br>    sid = &quot;Delegate IAM for non-data KMS permissions.&quot;<br>    actions = [<br>      &quot;kms:Describe*&quot;,<br>      &quot;kms:Get*&quot;,<br>      &quot;kms:List*&quot;,<br>      (many kms: others),<br>      &quot;kms:Sign&quot;<br>    ]<br>    resources = [&quot;*&quot;]<br>    principals {<br>      type        = &quot;AWS&quot;<br>      identifiers = [&quot;arn:aws:iam::${data.aws_caller_identity.current.account_id}:root&quot;]<br>    }<br>  }<br>  statement {<br>    sid       = &quot;AllowCFServicePrincipalSSE-KMS&quot;<br>    effect    = &quot;Allow&quot;<br>    resources = [&quot;*&quot;]<br><br>    actions = [<br>      &quot;kms:Decrypt&quot;,<br>      &quot;kms:Encrypt&quot;,<br>      &quot;kms:ReEncrypt*&quot;,<br>      &quot;kms:GenerateDataKey*&quot;,<br>    ]<br><br>    condition {<br>      test     = &quot;StringEquals&quot;<br>      variable = &quot;AWS:SourceArn&quot;<br>      values   = [&quot;arn:aws:cloudfront::${var.aws_account_number}:distribution/${module.distribution.cf_distribution_id}&quot;]<br>    }<br><br>    principals {<br>      type        = &quot;Service&quot;<br>      identifiers = [&quot;cloudfront.amazonaws.com&quot;]<br>    }<br>  }<br>  statement {<br>    sid    = &quot;AllowUser&quot;<br>    effect = &quot;Allow&quot;<br>    actions = [<br>      &quot;kms:Decrypt&quot;,<br>      &quot;kms:Encrypt&quot;,<br>      &quot;kms:ReEncrypt*&quot;,<br>      &quot;kms:GenerateDataKey*&quot;,<br>    ]<br>    resources = [&quot;*&quot;]<br><br>    principals {<br>      type        = &quot;AWS&quot;<br>      identifiers = [&quot;arn:aws:iam::${var.aws_account_number}:user/${aws_iam_user.iam_user.name}&quot;]<br>    }<br>  }<br>}</pre><p>All of these permissions and data resources ensure that CloudFront and the designated IAM user to put objects in the bucket have enough and correct permissions to interact with the bucket. It is important to establish that the OAC you create has access to the bucket. Consult the official documentation for this, <a href="https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-restricting-access-to-s3.html">here</a>.</p><p>You won’t need all the actions as above, I just went very granular with the implementation.</p><p>In iam.tf, I create a user and assign this user to a group. Pretty standard implementation, I used these resources:</p><pre>resource &quot;aws_iam_user&quot; &quot;iam_user&quot; {<br>  name = var.c_cf_iam_username<br>}<br><br># Here&#39;s where the pgp key helps in keeping the keys encrypted in state<br>resource &quot;aws_iam_access_key&quot; &quot;iam_user_key&quot; {<br>  user    = aws_iam_user.iam_user.name<br>  pgp_key = data.local_file.pgp_key.content<br>}<br><br>resource &quot;aws_iam_group&quot; &quot;iam_group&quot; {<br>  name = var.c_cf_iam_group<br>}<br><br>resource &quot;aws_iam_group_membership&quot; &quot;iam_group_membership&quot; {<br>  name = var.c_cf_group_membership<br><br>  users = [<br>    aws_iam_user.iam_user.name<br>  ]<br>  group = aws_iam_group.iam_group.name<br>}<br><br>resource &quot;aws_iam_group_policy&quot; &quot;iam_group_policy&quot; {<br>  name  = var.c_cf_assume_policy<br>  group = aws_iam_group.iam_group.name<br><br>  policy = data.aws_iam_policy_document.put_object.json<br>}</pre><p>In kms.tf I create the KMS key that will encrypt objects in the bucket:</p><pre>resource &quot;aws_kms_key&quot; &quot;bucket_key&quot; {<br>  description             = &quot;This key is used to encrypt the bucket objects.&quot;<br>  deletion_window_in_days = var.c_cf_key_deletion_window<br>}<br><br>resource &quot;aws_kms_alias&quot; &quot;s3&quot; {<br>  name          = &quot;alias/${var.c_cf_bucket_name}-kms-key&quot;<br>  target_key_id = aws_kms_key.bucket_key.key_id<br>}<br><br>resource &quot;aws_kms_key_policy&quot; &quot;oac_kms_key_access&quot; {<br>  key_id = aws_kms_key.bucket_key.id<br>  policy = data.aws_iam_policy_document.oac_kms_key_access.json<br>}</pre><p>In lambda.tf I leverage the lambda module to create the function:</p><pre>resource &quot;aws_iam_role&quot; &quot;edge_lambda&quot; {<br>  name               = var.c_cf_lambda_iam_role_name<br>  assume_role_policy = data.aws_iam_policy_document.lambda_assume_role.json<br>}<br><br># I attach a pre-defined service-role<br>resource &quot;aws_iam_role_policy_attachment&quot; &quot;edge_lambda&quot; {<br>  role       = aws_iam_role.edge_lambda.name<br>  policy_arn = &quot;arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole&quot;<br>}<br><br>module &quot;edge_lambda&quot; {<br>  source = &quot;../../modules/lambda&quot;<br><br>  # Here&#39;s where that additional aws provider alias comes in handy<br>  providers = {<br>    aws = aws.us-east-1<br>  }<br><br>  m_lmb_at_edge          = true<br>  m_lmb_name             = var.c_lmb_name<br>  m_lmb_runtime          = var.c_lmb_runtime<br>  m_lmb_archiver         = var.c_lmb_archiver<br>  m_lmb_source_file_name = var.c_lmb_source_file_name<br>  m_lmb_handler          = var.c_lmb_handler<br>  m_lmb_script_extension = var.c_lmb_script_extension<br>  m_lmb_timeout          = var.c_lmb_timeout<br>  m_lmb_iam_role_arn     = aws_iam_role.edge_lambda.arn<br>}</pre><p>locals.tf is where I do some locals magic in the config (this can probably be moved to Terragrunt, as we didn’t use the locals and inputs option at the time):</p><pre>locals {<br><br>  # I&#39;m only defining an s3 origin here, but we could define multiple in this<br>  # list and in this format<br>  c_cf_origins = [<br>    {<br>      origin_id                = module.bucket.bucket_id<br>      domain_name              = module.bucket.bucket_regional_domain_name<br>      origin_access_control_id = aws_cloudfront_origin_access_control.oac.id<br>      custom_headers           = null<br>    }<br>  ]<br><br>  c_cf_default_cache_lambda_association = {<br>    event_type   = &quot;viewer-request&quot;<br>    lambda_arn   = &quot;${module.edge_lambda.function_arn}:${module.edge_lambda.version}&quot;<br>    include_body = false<br>  }<br>}</pre><p>A few things that I’m outputting, to pass to other teams:</p><pre>output &quot;access_key_id&quot; {<br>  value = aws_iam_access_key.iam_user_key.id<br>}<br><br>output &quot;secret_access_key&quot; {<br>  value = aws_iam_access_key.iam_user_key.encrypted_secret<br>}</pre><p>Can’t forget the s3 module, so s3.tf looks like this:</p><pre>module &quot;bucket&quot; {<br>  source = &quot;../../modules/s3_app&quot;<br><br>  m_s3_bucket_name       = var.c_s3_bucket_name<br>  m_s3_bucket_versioning = var.c_s3_bucket_versioning<br>  m_s3_sse_algorithm     = var.c_s3_sse_algorithm<br>  m_s3_kms_master_key_id = aws_kms_key.bucket_key.arn<br>  m_s3_m_s3_ol_configs   = var.c_s3_ol_configs<br>}<br><br># Applying the bucket level policies<br>resource &quot;aws_s3_bucket_policy&quot; &quot;allow_access_from_cloudfront&quot; {<br>  bucket = module.sbucket.bucket_id<br>  policy = data.aws_iam_policy_document.bucket_policy.json<br>}</pre><p>Here’s how the variable c_s3_ol_config might look like as input for the module:</p><pre>c_s3_ol_config = {<br>  object_lock_enabled = &quot;Enabled&quot;<br>  rule = {<br>    default_retention = {<br>      mode = &quot;GOVERNANCE&quot;<br>      days = 30<br>    }<br>  }<br>}</pre><p>And we are left with variables.tf which contains all the variables needed to tie everything in. I’m not going into details here, as they are similar to the variables declared in the modules in Part 1, the only difference is that I prefix them with c_ instead of m_ for sanity of configuration vs module error keeping.</p><p>This is for Part 2. I hope it is clear and will help in building something similar. If you liked this article, don’t forget to leave a clap, or a message and maybe subscribe to my Notifications, it helps me keep going with articles such as this one. Have a great day!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9f5c58bc21d2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Serve Single-Page Applications from S3 through CloudFront with external WAF — a Terraform guide…]]></title>
            <link>https://medium.com/@adrianarba/serve-single-page-applications-from-s3-through-cloudfront-with-external-waf-a-terraform-guide-73281304c7e6?source=rss-7f487fa71b14------2</link>
            <guid isPermaLink="false">https://medium.com/p/73281304c7e6</guid>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[terraform-modules]]></category>
            <category><![CDATA[lambda]]></category>
            <category><![CDATA[cloudfront]]></category>
            <dc:creator><![CDATA[Adrian Arba]]></dc:creator>
            <pubDate>Fri, 12 May 2023 06:01:55 GMT</pubDate>
            <atom:updated>2023-06-10T13:57:19.997Z</atom:updated>
            <content:encoded><![CDATA[<h3>Serve Single-Page Applications from S3 through CloudFront with external WAF — a Terraform guide (Part 1)</h3><p>I’m going to present how I tackled an interesting challenge, of deploying an infrastructure stack through Terraform, to serve Single Page Applications (SPA) in AWS from an S3 bucket, through a CloudFront distribution using OAC (Origin Access Control) and connected to external DNS and WAF providers.</p><p>This solution ties into a globally accessible application that calls these SPAs on application user request. Using AWS (or any Cloud provider) to serve the SPAs brings benefits such as low costs and high availability (serverless solution, S3 storage), security (always using the latest security standards in AWS), and speed (CDN serves cached content globally) with very little administrative overhead (just update the Terraform code for changes).</p><p>For this solution, I used generally available terraform resources that I implemented as modules. I won’t be covering all the code I used (this article would become pretty sizeable) but I want to point out some quirks I met and tackled while working on this, especially since most articles I’ve found mention static websites in S3 or using OAI instead of OAC and do not specify why you may be getting some weird error messages along the way.</p><p>It’s going to be a 2 part article, one focusing on the modules and another one focusing on the configuration applied to the modules.</p><p>In part 1, we are going to take a look at the solution architecture, using the diagram below, and at the asks and needs of the feature, and then I’ll outline the structure of the repository and I’ll focus on the modules, with explanations along the way.</p><p>Since the team is using an external WAF provider, I won’t be touching that</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uHqSqII5V_v6bZio1Ao-QA.png" /><figcaption>Proposed solution diagram</figcaption></figure><p>Reading the diagram:</p><ul><li>Clients accessing the SPAs would have to use the domain name URL going through an external WAF provider, instead of the CloudFront URL, this means there’s a need to decide if a connection is allowed/dropped before actually reaching CloudFront — we decided to use Lambda@Edge functions for this purpose;</li><li>Certificates would be added to ACM by a separate team, we would just reference them in the Terraform code to link them to the distribution;</li><li>CloudFront distribution to use Origin Access Control to serve the private S3 bucket objects;</li><li>Encryption in transit is enabled at all steps;</li><li>All objects in the bucket would benefit from the following:<br>- versioning<br>- object locking<br>- MFA delete protection<br>- legal holds automatically applied<br>- encryption with a dedicated KMS key</li><li>All applied policies are as limiting as possible, to allow only the designated actions to happen;</li><li>A CI/CD pipeline process would require a specific IAM user with very limited permissions in order to deploy new (versions of) objects to the bucket (the CI/CD pipeline part is out of the scope of this article);</li><li>CloudFront and S3 logging be enabled (out of the scope of this article);</li><li>Configuration of the external DNS and WAF to point to CloudFront (out of the scope of this article).</li></ul><p>DevOps tooling I used:</p><ul><li>Terraform code to build and deploy the infrastructure — v1.3.8;</li><li>Terragrunt v0.45.2 wrapper to separate the environments and pass different layers of variables down to Terraform at different levels i.e. Application level, Environment level, and Project level (out of the scope of this article);</li><li>AWS CLI v2.8.12 + Console for operating the solution services, testing, etc.</li></ul><p>The code is structured into 2 repositories:</p><ul><li>Repository A hosts the Terragrunt environments setup and is usually the place I pass values for variables from;</li><li>Repository B hosts the Terraform configuration and modules (both in one place, for simplicity) — this is also the repository I’ll be focusing on in this article;</li></ul><p>Here’s what the structure of Repository B looks like:</p><pre>├── repository_a<br>│   └── (...) # terragrunt - out of scope<br>│<br>└── repository_b<br>    │<br>    ├── configurations<br>    │   │<br>    │   └── single_page_app<br>    │       │<br>    │       ├── backend.tf<br>    │       ├── cf.tf<br>    │       ├── data.tf<br>    │       ├── script.py<br>    │       ├── iam.tf<br>    │       ├── kms.tf<br>    │       ├── lambda.tf<br>    │       ├── locals.tf<br>    │       ├── outputs.tf<br>    │       ├── s3.tf<br>    │       └── variables.tf<br>    │<br>    └── <strong>modules</strong><br>        │<br>        ├── <strong>cloudfront</strong><br>        │   │<br>        │   ├── main.tf<br>        │   ├── outputs.tf<br>        │   └── variables.tf<br>        │<br>        ├── <strong>s3_app</strong><br>        │   │<br>        │   ├── main.tf<br>        │   ├── outputs.tf<br>        │   └── variables.tf<br>        │<br>        └── <strong>lambda</strong><br>            │<br>            ├── data.tf<br>            ├── main.tf<br>            ├── outputs.tf<br>            ├── providers.tf<br>            └── variables.tf</pre><p>I’m going to present each file below, per folder, but I won’t be going into details for each variable, only what I consider to have been more complex than a generic “string” or “list”.</p><p>Module s3_apps:</p><pre># modules/s3_app/main.tf<br><br>resource &quot;aws_s3_bucket&quot; &quot;this&quot; {<br>  bucket = var.m_s3_bucket_name<br><br>  # Defining object lock configurations, if any<br>  dynamic &quot;object_lock_configuration&quot; {<br>    for_each = var.m_s3_ol_configs == null ? [] : [var.m_s3_ol_configs]<br><br>    # Enabling object lock<br>    content {<br>      object_lock_enabled = m_s3_ol_configs.value.object_lock_enabled<br><br>      dynamic &quot;rule&quot; {<br>        for_each = m_s3_ol_configs.value.rule == null ? [] : [m_s3_ol_configs.value.rule]<br><br>        content {<br><br>          # Setting the default object lock retention mode and time<br>          dynamic &quot;default_retention&quot; {<br>            for_each = rule.value.default_retention == null ? [] : [rule.value.default_retention]<br><br>            content {<br>              mode  = default_retention.value.mode<br>              days  = default_retention.value.days<br>              years = default_retention.value.years<br>            }<br>          }<br>        }<br>      }<br>    }<br>  }<br>}<br><br># Disabling ACLs<br>resource &quot;aws_s3_bucket_ownership_controls&quot; &quot;this&quot; {<br>  bucket = aws_s3_bucket.this.id<br><br>  rule {<br>    object_ownership = var.m_s3_acl_config<br>  }<br>}<br><br># Enabling bucket versioning and configuring possible mfa_delete protection (disabled here)<br>resource &quot;aws_s3_bucket_versioning&quot; &quot;this&quot; {<br>  bucket = aws_s3_bucket.this.id<br><br>  versioning_configuration {<br>    status     = var.m_s3_bucket_versioning<br>    mfa_delete = var.m_s3_mfa_delete<br>  }<br><br>  lifecycle {<br>    ignore_changes = [<br>      versioning_configuration[&quot;mfa_delete&quot;]<br>    ]<br>  }<br>}<br><br># Making sure the bucket is privately accessible only<br>resource &quot;aws_s3_bucket_public_access_block&quot; &quot;this&quot; {<br>  bucket = aws_s3_bucket.this.id<br><br>  block_public_acls       = var.m_s3_block_public_acls<br>  block_public_policy     = var.m_s3_block_public_policy<br>  ignore_public_acls      = var.m_s3_ignore_public_acls<br>  restrict_public_buckets = var.m_s3_restrict_public_buckets<br>}<br><br># Enabling encryption at rest using a specific KMS key<br>resource &quot;aws_s3_bucket_server_side_encryption_configuration&quot; &quot;this&quot; {<br>  bucket = aws_s3_bucket.this.id<br><br>  rule {<br>    apply_server_side_encryption_by_default {<br>      sse_algorithm     = var.m_s3_algorithm<br>      kms_master_key_id = var.m_s3_kms_key_id<br>    }<br>  }<br>}</pre><pre><br># modules/s3_app/main.tf<br><br>resource &quot;aws_s3_bucket&quot; &quot;this&quot; {<br>  bucket = var.m_s3_bucket_name<br><br>  # Defining object lock configurations, if any<br>  dynamic &quot;object_lock_configuration&quot; {<br>    for_each = var.m_s3_ol_configs == null ? [] : [var.m_s3_ol_configs]<br><br>    # Enabling object lock<br>    content {<br>      object_lock_enabled = m_s3_ol_configs.value.object_lock_enabled<br><br>      dynamic &quot;rule&quot; {<br>        for_each = m_s3_ol_configs.value.rule == null ? [] : [m_s3_ol_configs.value.rule]<br><br>        content {<br>          # Setting the default object lock retention mode and time<br>          dynamic &quot;default_retention&quot; {<br>            for_each = rule.value.default_retention == null ? [] : [rule.value.default_retention]<br><br>            content {<br>              mode  = default_retention.value.mode<br>              days  = default_retention.value.days<br>              years = default_retention.value.years<br>            }<br>          }<br>        }<br>      }<br>    }<br>  }<br>}<br><br># Handling ACLs<br>resource &quot;aws_s3_bucket_ownership_controls&quot; &quot;this&quot; {<br>  bucket = aws_s3_bucket.this.id<br><br>  rule {<br>    object_ownership = var.m_s3_acl_config<br>  }<br>}<br><br># Setting ACLs as private<br>resource &quot;aws_s3_bucket_acl&quot; &quot;this&quot; {<br>  depends_on = [aws_s3_bucket_ownership_controls.this]<br><br>  bucket = aws_s3_bucket.this.id<br>  acl    = var.m_s3_acl<br>}<br><br># Enabling bucket versioning and configuring possible mfa_delete protection (disabled here)<br>resource &quot;aws_s3_bucket_versioning&quot; &quot;this&quot; {<br>  bucket = aws_s3_bucket.this.id<br><br>  versioning_configuration {<br>    status     = var.m_s3_bucket_versioning<br>    mfa_delete = var.m_s3_mfa_delete<br>  }<br><br>  lifecycle {<br>    ignore_changes = [<br>      versioning_configuration[&quot;mfa_delete&quot;]<br>    ]<br>  }<br>}<br><br># Making sure the bucket is privately accessible only<br>resource &quot;aws_s3_bucket_public_access_block&quot; &quot;this&quot; {<br>  bucket = aws_s3_bucket.this.id<br><br>  block_public_acls       = var.m_s3_block_public_acls<br>  block_public_policy     = var.m_s3_block_public_policy<br>  ignore_public_acls      = var.m_s3_ignore_public_acls<br>  restrict_public_buckets = var.m_s3_restrict_public_buckets<br>}<br><br># Enabling encryption at rest using a specific KMS key<br>resource &quot;aws_s3_bucket_server_side_encryption_configuration&quot; &quot;this&quot; {<br>  bucket = aws_s3_bucket.this.id<br><br>  rule {<br>    apply_server_side_encryption_by_default {<br>      sse_algorithm     = var.m_s3_algorithm<br>      kms_master_key_id = var.m_s3_kms_key_id<br>    }<br>  }<br>}</pre><p>Note the “dynamic” blocks, these get populated only if the “for_each” condition does not evaluate to a “null” variable, otherwise, they get ignored, meaning if another configuration does not want to handle object locking, this module can still be called for that use case too.</p><p>On the “object_lock_configuration” dynamic block, this can be replaced with the newer “aws_s3_bucket_object_lock_configuration” resource — <a href="https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_object_lock_configuration">https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_object_lock_configuration</a> as it will be deprecated soon, but leaving it as is here, because I prefer to have it declared dynamically.</p><pre># modules/s3_app/variables.tf<br><br># ignoring most of the variables as they are just defined and contain a default<br># keep in mind that all variables used within the module need to be declared in a module .tf file such as this one<br>(...)<br><br>variable &quot;m_s3_ol_configs&quot; {<br>  type = object({<br>    object_lock_enabled = string<br>    rule = object({<br>      default_retention = object({<br>        days  = optional(number)<br>        years = optional(number)<br>        mode  = string<br>      })<br>    })<br>  })<br><br>  default = null<br>}<br><br>variable &quot;m_s3_acl&quot; {<br>  type    = string<br>  default = &quot;private&quot;<br>}<br><br>variable &quot;m_s3_acl_config&quot; {<br>  type    = string<br>  default = &quot;BucketOwnerEnforced&quot;<br>}<br><br>variable &quot;m_s3_mfa_delete&quot; {<br>  type    = string<br>  default = &quot;Disabled&quot;<br>}</pre><p>I’ve kept an example of the more complex “m_s3_ol_configs” variable used in the “for_each” loop and some defaults for other used variables.</p><p>You don’t have to go to such complex lengths when designing the variables like it’s the case for “m_s3_ol_configs&quot;, it’s only a matter of how dynamic your module needs to be to serve specific purposes, as we will see later, with cases where you want to have specific resource blocks added or not based on the variable values.</p><pre># modules/s3_apps/outputs.tf<br><br>output &quot;bucket_id&quot; {<br>    value = element(concat(aws_s3_bucket.this.*.id, [&quot;&quot;]), 0)<br>}<br><br>output &quot;bucket_arn&quot; {<br>    value = element(concat(aws_s3_bucket.this.*.arn, [&quot;&quot;]), 0)<br>}<br><br>output &quot;bucket_regional_domain_name&quot; {<br>    value = element(concat(aws_s3_bucket.this.*.bucket_regional_domain_name, [&quot;&quot;]), 0)<br>}</pre><p>The outputs file contains a few elements we will be using in the configurations to link CloudFront to the bucket as well as other resources.</p><p>Module cloudfront:</p><pre># modules/cloudfront/main.tf<br><br>resource &quot;aws_cloudfront_distribution&quot; &quot;this&quot; {<br>  comment             = var.m_cf_comment<br>  enabled             = var.m_cf_enable_distribution<br>  is_ipv6_enabled     = var.m_cf_ipv6_enable<br>  price_class         = var.m_cf_pc<br><br>  # it generally takes 15 minutes for the distribution to be deployed, you can ask terraform to wait for completion<br>  wait_for_deployment = var.m_cf_wait<br><br>  # the root object is optional, usually maps to index.html, you can leave it blank if you want<br>  default_root_object = var.m_cf_default_root_obj<br><br>  # aliases are optional but if you plan to use a custom SSL certificate and a friendly domain name, set them in a variable<br>  aliases             = var.m_cf_aliases<br><br>  # if there are multiple origins, they will be added dynamically<br>  dynamic &quot;origin&quot; {<br>    for_each = var.m_cf_origins<br><br>    content {<br>      origin_id                = origin.value.origin_id<br>      domain_name              = origin.value.domain_name<br>      origin_access_control_id = origin.value.origin_access_control_id<br>      # we don&#39;t use the origin_path, but uncomment and set if used<br>      # origin_path              = coalesce(origin.value.origin_path, &quot;&quot;)<br><br>      # same for any custom headers per origin<br>      dynamic &quot;custom_header&quot; {<br>        for_each = origin.value.custom_headers == null ? [] : origin.value.custom_headers<br>        iterator = header<br><br>        content {<br>          name  = header.value.name<br>          value = header.value.value<br>        }<br>      }<br>    }<br>  }<br><br>  default_cache_behavior {<br>    allowed_methods  = var.m_cf_default_cache_methods.allowed_methods<br>    cached_methods   = var.m_cf_default_cache_methods.cache_methods<br>    target_origin_id = var.m_cf_target_origin_id<br><br>    forwarded_values {<br>      query_string = var.m_cf_query_string<br><br>      cookies {<br>        forward = var.m_cf_default_cache_methods.cookies_forward<br>      }<br>    }<br><br>    min_ttl                = var.m_cf_default_cache_methods.min_ttl<br>    default_ttl            = var.m_cf_default_cache_methods.default_ttl<br>    max_ttl                = var.m_cf_default_cache_methods.max_ttl<br>    viewer_protocol_policy = var.m_cf_default_cache_methods.viewer_policy<br><br>    # if there are any lambda associations for the default cache, they will be added dynamically<br>    dynamic &quot;lambda_function_association&quot; {<br>      for_each = var.m_cf_default_cache_lambda_associations == null ? [] : [var.m_cf_default_cache_lambda_associations]<br><br>      content {<br>        event_type   = lambda_function_association.value.event_type<br>        lambda_arn   = lambda_function_association.value.lambda_arn<br>        include_body = lambda_function_association.value.include_body<br>      }<br>    }<br>  }<br><br>  # if there are any ordered cache behaviours, they will be added dynamically<br>  dynamic &quot;ordered_cache_behavior&quot; {<br>    for_each = var.m_cf_ordered_cache_behaviors<br>    iterator = behavior<br><br>    content {<br>      target_origin_id          = behavior.value.target_origin_id<br>      path_pattern              = behavior.value.cache_behavior.path_pattern<br>      viewer_protocol_policy    = behavior.value.cache_behavior.viewer_protocol_policy<br><br>      allowed_methods           = behavior.value.cache_behavior.allowed_methods<br>      cached_methods            = behavior.value.cache_behavior.cached_methods<br>      compress                  = behavior.value.cache_behavior.compress<br>      trusted_signers           = behavior.value.cache_behavior.trusted_signers<br>      smooth_streaming          = behavior.value.cache_behavior.smooth_streaming<br>      realtime_log_config_arn   = lookup(behavior.value.cache_behavior, &quot;realtime_log_config_arn&quot;, null)<br>      field_level_encryption_id = behavior.value.cache_behavior.field_level_encryption_id<br><br>      # set these to null to enable use of cache-control headers.<br>      # if using cache-control headers, TTLs must be set as: default: 1day; min: 0s; max: 365days<br>      min_ttl                   = behavior.value.cache_behavior.min_ttl<br>      default_ttl               = behavior.value.cache_behavior.default_ttl<br>      max_ttl                   = behavior.value.cache_behavior.max_ttl<br><br>      # same for any forward rules per cache behaviour<br>      dynamic &quot;forwarded_values&quot; {<br>        for_each = behavior.value.cache_behavior.forwarded_values[*]<br><br>        content {<br>          query_string            = forwarded_values.value.query_string<br>          query_string_cache_keys = forwarded_values.value.query_string_cache_keys<br>          headers                 = forwarded_values.value.headers<br><br>          cookies {<br>            forward           = forwarded_values.value.cookie_forward_type<br>            whitelisted_names = forwarded_values.value.cookie_whitelisted_names<br>          }<br>        }<br>      }<br>    }<br>  }<br><br>  restrictions {<br>    geo_restriction {<br>      restriction_type = var.m_cf_geo_restriction_type<br>      locations        = var.m_cf_geo_locations<br>    }<br>  }<br><br>  viewer_certificate {<br>    # these 2 options will only be enabled if not using the default CloudFront certificate<br>    minimum_protocol_version       = var.m_cf_minimum_tls_version<br>    ssl_support_method             = var.m_cf_ssl_method<br><br>    # this should be set to disabled if you use your own domain certificate for the CF distribution instead of a default one<br>    cloudfront_default_certificate = var.m_cf_cloudfront_default_certificate<br><br>    # a valid domain certificate needs to be uploaded to ACM and the ARN needs to be retrieved and set here<br>    acm_certificate_arn            = var.m_cf_cloudfront_certificate_arn<br>  }<br>}</pre><p>I tried keeping the module as dynamic as possible for the task at hand, meaning it can be reused in multiple setups and potentially with static websites as well, or multiple origins and cache behaviors.</p><p>Few important notes:</p><ul><li>setting a default root object (i.e. “index.html) is optional and it works by you being pointed to that file if you don’t specify anything after the CloudFront URL address. And if you do set one, you will constantly get an access denied message until you actually put that object inside the bucket:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JUNBhNGydNsEEvS-E_BlCQ.png" /></figure><ul><li>aliases are also optional, and you can leave the list blank if, for instance, you are only testing the scenario and you plan to use the default CloudFront certificate generated automatically, but you can populate them if you plan to use your own production-grade domain certificate, uploaded to ACM, to benefit from friendly names. Below is what a default vs custom certificate would look like:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ku5lVVt1TrCnqDR3-IOQrg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fFCIoxyoGSdgih1WtjHw1w.png" /></figure><ul><li>if you do use a default CloudFront SSL certificate instead of your own, some security features like the latest TLS policy will not be enabled, so bare this in mind, such deployments should not be considered production worthy.</li><li>if you don’t have any geo-restrictions in place, you will have to add these values to the following variables:<br>- <strong>m_cf_geo_restriction_type: “none”</strong><br>- <strong>m_cf_geo_locations: []</strong></li><li>you can check all the features enabled in your CloudFront distribution with this CLI command:</li></ul><pre>aws --profile &lt;profile_name&gt; cloudfront get-distribution-config --id &lt;CFDISTID&gt;</pre><p>Moving on with the terraform code, here are a few variables examples</p><pre># module/cloudfront/variables.tf<br><br>(...)<br><br>variable &quot;m_cf_origins&quot; {<br>  type          = list(object({<br>    origin_id                = string<br>    domain_name              = string<br>    # origin_path              = string<br>    origin_access_control_id = string           # [required when using OAC] OAC ID<br>    custom_headers           = list(object({<br>      name  = string<br>      value = string<br>    }))<br>  }))<br><br>  default = []<br>}<br><br>variable &quot;m_cf_default_cache_methods&quot; {<br>  type = object({<br>    path_pattern      = string<br>    allowed_methods   = list(string)<br>    cache_methods     = list(string)<br>    viewer_policy     = string<br>    min_ttl           = number<br>    default_ttl       = number<br>    max_ttl           = number<br>    forwarded_headers = string<br>    cookies_forward   = string<br>  })<br><br>  default = null<br>}<br><br>variable &quot;m_cf_default_cache_lambda_associations&quot; {<br>  type        = list(object({<br>    event_type   = string        # values: viewer-request, origin-request, viewer-response, origin-response<br>    lambda_arn   = string<br>    include_body = string<br>  }))<br><br>  default = null<br>}<br><br>variable &quot;m_cf_ordered_cache_behaviors&quot; {<br>  type = list(object({<br>    target_origin_id          = string<br>    # (...) long list of parameters you can find in the resource documentation<br>    max_ttl                   = number<br>    forwarded_values = object({<br>      query_string             = bool<br>      query_string_cache_keys  = list(string)<br>      headers                  = list(string)<br>      cookie_forward_type      = string<br>      cookie_whitelisted_names = list(string)<br>    })<br>  }))<br><br>  default = []<br>}<br><br>(...)</pre><p>I only chose the ones that are the most complex, but this file should contain all the variables used in the module.</p><pre># module/cloudfront/outputs.tf<br><br>output &quot;cf_distribution_domain_name&quot; {<br>  value = aws_cloudfront_distribution.this.domain_name<br>}<br><br>output &quot;cf_distribution_arn&quot; {<br>  value = aws_cloudfront_distribution.this.arn<br>}<br><br>output &quot;cf_distribution_id&quot; {<br>  value = aws_cloudfront_distribution.this.id<br>}</pre><p>Module lambda:</p><pre># module/lambda/main.tf<br><br>resource &quot;aws_lambda_function&quot; &quot;this&quot; {<br>    <br>  # required set to True for Edge functions<br>  publish = var.m_lmb_at_edge ? true : var.m_lmb_publish<br><br>  timeout          = var.m_lmb_at_edge ? min(var.m_lmb_timeout, 30) : var.m_lmb_timeout<br>  filename         = &quot;${var.m_lmb_name}.zip&quot;<br>  function_name    = var.m_lmb_name<br>  role             = var.m_lmb_iam_role_arn<br>  handler          = &quot;${var.m_lmb_source_file_name}.${var.m_lmb_handler}&quot;<br>  runtime          = var.m_lmb_runtime<br>  source_code_hash = data.archive_file.lambda.output_base64sha256<br><br>  # environment variables work on regular functions, not on edge functions<br>  # we define it as dynamic so that if we do use edge functions, this section will get ignored<br>  dynamic &quot;environment&quot; {<br>    for_each = length(keys(var.m_lmb_env_vars)) == 0 ? [] : [true]<br>      content {<br>        variables = var.m_lmb_env_vars<br>      }<br>  }<br>}</pre><p>We have a lightweight module here, with the following gotchas:</p><ul><li>in order for a Lambda function to become a Lambda@EDGE function, we need to specify the “publish = true” parameter in the resource;</li><li>roles need to be created and assigned to Lambdas — and we create the role in the configurations, so that any specific permissions get templated at that level, per needs, and not at module level;</li><li>the “handler” parameter takes the value of “lambda_function_file_name” without an extension + the defined function name within that file, in this case, “handler” (as we will see below);</li><li>we use “source_code_hash” to trigger updates to the function (important especially when testing your code);</li><li>environment variables are not supported in edge functions, so we declare a “dynamic” environment section that does not get populated for edge function, but, if we use the module with regular functions, it will be there to allow taking in any environment variables — I’ll show you how I pass variables to the function when getting to the configurations section.</li></ul><pre># module/lambda/variables.tf<br><br>(...)<br><br>variable &quot;m_lmb_env_vars&quot; {<br>  description = &quot;Map of environment variables to be passed to the lambda&quot;<br>  type        = map(string)<br>}<br><br>variable &quot;m_lmb_publish&quot; {<br>  description = &quot;Whether to publish creation/change as new Lambda Function Version.&quot;<br>  type        = bool<br>  default     = false<br>}<br><br>(...)</pre><p>These are the most complex variables in the Lambda module :D</p><pre># module/lambda/outputs.tf<br><br>output &quot;function_arn&quot; {<br>  value = aws_lambda_function.this.arn<br>}</pre><p>We will use the function’s ARN to link the function to the CloudFront distribution later.</p><p>The data.tf file designates uploading a local zip file:</p><pre>data &quot;archive_file&quot; &quot;lambda&quot; {<br>  type        = var.m_lmb_archiver<br>  output_path = &quot;${var.m_lmb_name}.${var.m_lmb_archiver}&quot;<br><br>  source_file = &quot;path/to/your/script.py&quot;<br>}</pre><p>I’ve also added a python script at the configuration level, that drops all connections going directly to the CloudFront distribution, but allows connections coming from the external WAF:</p><pre>def handler(event, context):<br>    request = event[&quot;Records&quot;][0][&quot;cf&quot;][&quot;request&quot;]<br>    host_key = request[&quot;headers&quot;][&quot;host&quot;][0][&quot;key&quot;]<br>    host_value = request[&quot;headers&quot;][&quot;host&quot;][0][&quot;value&quot;]<br>    distribution = event[&quot;Records&quot;][0][&quot;cf&quot;][&quot;config&quot;][&quot;distributionDomainName&quot;]<br><br>    ## Drop the request if it&#39;s going directly to the CloudFront URL<br>    if host_key == &quot;Host&quot; and distribution in host_value:<br>        return {<br>            &quot;status&quot;: &quot;403&quot;,<br>            &quot;statusDescription&quot;: &quot;Forbidden&quot;,<br>            &quot;body&quot;: &quot;Direct Access To the Cloudfront URL is Denied&quot;,<br>        }<br><br>    ## Else return the request unchanged<br>    return request</pre><p>I used this <a href="https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-event-structure.html">AWS page</a> to understand how a viewer request event looks like to set up the basic script above (basic, but hey, it does its job :) ).</p><p>The providers.tf section contains only the AWS provider:</p><pre>terraform {<br>  required_providers {<br>    aws = {<br>      source  = &quot;hashicorp/aws&quot;<br>    }<br>  }<br>}</pre><p>That’s it for the 3 modules I used for this solution. Hopefully, you won’t have any difficulties filling in the blanks with the more simple variables used.</p><p>In part 2 — the configuration, we will add parametrization on top of the modules and we will also pass on things like policies, lambda code, etc to deploy the entire solution.</p><p>Leave a message or a clap if you enjoyed or found this article useful and see you in part 2!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=73281304c7e6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[CI/CD defined through terraform using AWS CodePipeline, AWS CodeCommit, and AWS CodeBuild]]></title>
            <link>https://medium.com/@adrianarba/ci-cd-defined-through-terraform-using-aws-codepipeline-aws-codecommit-and-aws-codebuild-12ade4d9cfa3?source=rss-7f487fa71b14------2</link>
            <guid isPermaLink="false">https://medium.com/p/12ade4d9cfa3</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[how-to]]></category>
            <category><![CDATA[ci-cd-pipeline]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Adrian Arba]]></dc:creator>
            <pubDate>Mon, 21 Feb 2022 11:44:39 GMT</pubDate>
            <atom:updated>2023-08-22T05:44:19.201Z</atom:updated>
            <content:encoded><![CDATA[<p>I am a big fan of popular CI/CD solutions such as Jenkins, GitHub Actions, Azure DevOps, and I’ve even dabbed with things like Rundeck in the past, but I never had a chance to try out AWS’s CodePipeline before, so when this challenge arose on my project, I was excited to get the opportunity to try this service out and finally be able to compare with solutions I am familiar with. This ended up being a bigger challenge than I had hoped for, because, as it turned out, the services that AWS currently provides to tie into the solution I was looking for aren’t yet as mature as individually developed solutions are.</p><p>If you take AWS CodeCommit, for example, and compare it to Github or Gitlab or Bitbucket, or even Azure’s repositories, it currently lags behind in terms of customization, functionality, granularity, and ease of use. And although I am sure AWS’ engineers are actively working on improving services such as CodeCommit, or CodePipeline, they still represent a challenge if you are familiar with more mature, community preferred alternatives.</p><p>Having said this, I work with AWS on a daily basis as part of my job, so I was excited to get on top of this.</p><p>Before delving into details, let’s first take a look at the larger picture. I won’t be discussing everything today, only a small part of it, namely the CI/CD part, but I think some extra context always sets the scene right.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*4uJMsrXJlo01D3AX" /><figcaption>Grand diagram of things</figcaption></figure><p>As far as code functionality, my piece handles the following requirements:</p><ul><li>a way to provision <a href="https://docs.aws.amazon.com/singlesignon/latest/userguide/permissionsetsconcept.html">Permission Sets</a> into the AWS Account that hosts the AWS SSO service;</li><li>a way to link these Permission Sets to Groups and AWS Accounts;</li><li>a CI/CD to deploy the Permission Sets automatically when a new set or Group/Account link is configured and pushed to the code repository;</li><li>documentation of the solutions;</li><li>the code translates into infrastructure which will be built by running terraform commands.</li></ul><p>The environment that is being built will be used to vend Accounts in AWS from a Management account that relies on AWS SSO to allow user access. Users assume Roles (shaped as Permission Sets) assigned to Groups that they are part of, which, in turn, are assigned to AWS Accounts.</p><p>Permission Sets dictate the level of access a User has to the plethora of available services.</p><p>Different teams will have different Permission Sets assigned for different Accounts they need access to and all of their Users will access AWS through one AWS SSO portal which will be available through a URL link (this and many other components are provisioned separately).</p><p>The challenge comes from making this process a repeatable one with as little human interaction as possible. While I’m writing this post, AWS has not yet added any API functionality for the SSO service to allow provisioning Users and Groups programmatically through calls, so this step needs to be performed from the AWS SSO Console, but the rest of the steps can be done programmatically through either the AWS cli or through terraform code.</p><p>Since I’m a DevOps engineer by trade, I chose to continue with the terraform implementation and the bit this article will focus on today is the CI/CD components from the grand diagram of things above.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/572/0*iu0pLyI6jX5a0fZ3" /><figcaption>AWS CI/CD</figcaption></figure><p>My code needs a home, and a way to have it executed and deployed. AWS provides solutions for all of my needs:</p><ul><li><strong>AWS CodeCommi</strong>t acts as a git server, providing repositories to store code and allows interaction with your code through the git cli;</li><li><strong>AWS CodeBuild</strong> acts as a Build environment/engine and is used to execute instructions in multiple stages, to build and pack code in the shape of a usable artifact;</li><li><strong>AWS CodePipeline</strong> is the CI/CD framework that links the other two services together (as well as others) through executable Stages;</li><li><strong>AWS S3</strong> to keep any artifacts that result out of a successful Build Stage, for later use and posterity.</li></ul><h3>AWS CodeCommit</h3><p>Because it is a one-time task, I provision the <strong>AWS CodeCommit</strong> repository manually, from the AWS console. I pretty much leave everything as is, maybe one thing to consider changing would be the main branch name (which currently defaults to <em>master</em>).</p><p>One thing we need here is to get the Clone URL to be able to clone the repository later on. For that, click on the repository name and then open up the Clone URL list button and select Clone HTTPS (GRC):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7ejDHFofcsmfaJECIFTsCw.png" /></figure><p>We are interested in getting the green box that shows the repository connection URL on top of your page. Take note of the fact that the Region you are in gets prefixed to your repository name in the generated URL:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*z6eQxiSdxFtgdc6VL0i-IQ.png" /></figure><h3>Local Setup</h3><p>In order to work with terraform code locally and test any changes before committing them to the repository, we need to set up our local environment.</p><p>As always, there are several ways to do this, and I am going to give you my way as an example.</p><p>I have been granted SSO access to AWS and use an SSO landing page of the form URL <a href="https://d-&lt;hash&gt;.awsapps.com/start#/">https://d-&lt;hash&gt;.awsapps.com/start#/</a> to log in to the AWS Accounts consoles through my browser. I need something similar for my terminal, where I’ll be running terraform from.</p><ul><li><strong>requirement:</strong> aws cli v2</li></ul><pre>➜  aws --version                       <br>aws-cli/2.4.15 Python/3.8.8 Darwin/20.6.0 exe/x86_64 prompt/off</pre><p>Using the aws cli, I set up the SSO login and save my configuration under the form of an AWS local profile.</p><pre>➜ aws configure sso --profile myprofile</pre><p>This process points me to answer some questions such as my AWS account number, the Region I am working in, the desired cli output etc and then opens up a browser that points to the SSO start URL and allows me to sign in with my credentials, as well as it allows me to select the Account and Role I will assume whenever I need to log in through the terminal.</p><p>I can run this command as many times as I need if I make mistakes, and I can find the generated configuration under <strong>~/.aws/config</strong> if I want to adjust anything manually.</p><p>The login command will then become:</p><pre>➜ aws sso login --profile myprofile</pre><p>With this done, the next step is to install terraform.</p><ul><li><strong>requirement:</strong> terraform ≥ 0.15</li></ul><p>Terraform has added a lot of new features in version 12 and also overhauled its HCL scripting language. For the purpose of this tutorial, I am using terraform version ≥ 0.15, allowing me to have full support for the latest aws providers.</p><pre>➜ terraform --version<br>Terraform v0.15.5<br>on darwin_amd64</pre><p>I’ve installed it using <strong>tfenv</strong>, allowing me to quickly switch between terraform versions if needed, as I work on multiple projects with different requirements.</p><pre>➜ tfenv list      <br>* 0.15.5 (set by /usr/local/Cellar/tfenv/2.2.2/version)<br>  0.14.11</pre><p>One more thing is missing here. For me to interact with my AWS CodeCommit repository, I need to have <strong>git</strong> installed. There are multiple ways to clone and work with a CodeCommit repository and I’m going to show you the way I did it, by using the HTTPS GRC URL.</p><ul><li><strong>requirement:</strong> git ≥ 1.7.9</li></ul><pre>➜ git --version<br>git version 2.34.1</pre><ul><li><strong>requirement:</strong> <em>git-remote-codecommit</em> python3 package</li></ul><pre>➜ sudo pip install git-remote-codecommit</pre><p>More info on this here: <a href="https://docs.aws.amazon.com/codecommit/latest/userguide/setting-up-git-remote-codecommit.html">https://docs.aws.amazon.com/codecommit/latest/userguide/setting-up-git-remote-codecommit.html</a></p><p>As the last step, make sure you have your favorite IDE at hand for writing the actual code. I use <a href="https://code.visualstudio.com/"><strong>VS Code</strong></a><strong> </strong>with the terraform extension and found it a perfect match for the job at hand.</p><p>We have everything set up now, the last thing to dois to clone the empty CodeCommit repository locally (replace the profile and repository name along with your region as needed, in the command below):</p><pre>➜ git clone codecommit::eu-west-2://&lt;myprofile&gt;@&lt;myrepo&gt;</pre><p>We open our IDE to this location and we are ready to code.</p><pre>➜ cd &lt;myrepo&gt; &amp;&amp; code .</pre><p>We are now ready to build our code for provisioning the CI/CD Pipeline.</p><p>This is what the repository looks like and highlighted in blue are the files I’ll be discussing in this post (screenshot from VS Code):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/764/1*g4bBmtMNHsO2djQ4MVMv1w.png" /></figure><p>I’m using a <strong>configurations</strong> folder to keep all of my configurations, focusing on the <strong>permission_sets_pipeline</strong> configuration and I also have some terraform files directly under configurations (<strong>global.tf</strong>, <strong>output.tf</strong>, <strong>provider.tf</strong>, <strong>version.tf</strong> and <strong>state.tf</strong>) which act as universal configuration files and which I soft link inside my configurations folders (notice the small bent arrow mark to the right of some of the files).</p><p><strong>NOTE:</strong> AWS CodeBuild struggles with the concept of soft links, so I ended up copying the files over their soft links when I had to execute terraform commands, to avoid errors (later on, in the buildspec file definition).</p><p><strong>global.tf</strong> is where I define my universal variables and locals, which I then call on to from different configurations or pieces of code.</p><p>I’m using standard AWS resources from the <a href="https://registry.terraform.io/">Terraform.io registry</a> as well as two buildspec yaml files which will be used by AWS CodeBuild later on.</p><h3>AWS CodePipeline</h3><p>My <strong>codepipeline.tf</strong> file contains the definition of what I want the CI/CD pipeline to look like.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7b98326cb8dc6f1aebb8f8e62a5e0cea/href">https://medium.com/media/7b98326cb8dc6f1aebb8f8e62a5e0cea/href</a></iframe><p>I have 4 stages configured:</p><ul><li>The <strong>Clone</strong> stage clones my AWS CodeCommit repository (the exact same one, because I have multiple configurations inside it);</li><li>The <strong>Plan</strong> stage runs AWS CodeBuild and installs and runs terraform over my <strong>permission_sets</strong> configuration; it also saves the plan to AWS S3 as an artifact for later use;</li><li>The <strong>Manual-Approval</strong> stage allows me to check the planned terraform actions from the Plan stage and decide if I want to go forward with applying the changes or discarding them;</li><li>The <strong>Apply</strong> stage retrieved the S3 saved plan and applies the changes to my infrastructure.</li></ul><p>Each <strong>stage{}</strong> section contains an <strong>action{}</strong> section which allows describing what we want our stage to do.</p><p><strong>input_artifacts[]</strong> and <strong>output_artifacts[]</strong> allow us to specify if the current stage will use available artifacts and if it will produce any artifacts, and they accept lists of artifact names.</p><p>One important thing here, make sure you don’t use hyphens in your artifact name (don’t name it <em>terraform-plan</em>) because this generates an ambiguous terraform error.</p><p>Call artifacts with <strong>camelCase</strong>, <strong>snake_case,</strong> or <strong>PascalCase</strong> naming conventions (i.e.<em> CodeWorkspace</em> or <em>TerraformPlanFile</em>).</p><p>Per each type of Stage, different extra configurations are available under <strong>configuration{}</strong>, and I wanted to point out a few that I found useful:</p><ul><li>for the Plan and Deploy stages, I needed to expose a CodePipeline specific variable to AWS CodeBuild, because I wanted to save terraform plan files with the commitID and the pipelineID added to their name. This way, there would be no confusion as to which infrastructure changes get applied on each Pipeline execution.<br>I managed to do this through the <em>EnvironmentVariables</em> configuration, making the variable PIPELINE_EXECUTION_ID available to CodeBuild, having the pipeline execution ID as a value:</li></ul><pre>configuration = {<br>(...)</pre><pre>  EnvironmentVariables = jsonencode([<br>    {<br>      name  = &quot;PIPELINE_EXECUTION_ID&quot;<br>      value = &quot;#{codepipeline.PipelineExecutionId}&quot;<br>      type  = &quot;PLAINTEXT&quot;<br>    }<br>}</pre><ul><li>worth noting here that CodePipeline variables are available for being called through the syntax <strong>codepipeline.&lt;variable&gt;</strong> and are documented on the official AWS docs here <a href="https://docs.aws.amazon.com/codepipeline/latest/userguide/reference-variables.html">https://docs.aws.amazon.com/codepipeline/latest/userguide/reference-variables.html</a></li><li>the Deploy stage uses two input artifacts, the CodeBuild workspace and my terraform plan file, and I needed to specify a primary source used by CodeBuild for execution. I chose to use the CodeWorkspace artifacts as the main source and add in the terraform plan file to this workspace afterward:</li></ul><pre>configuration = {<br>(...)</pre><pre>  PrimarySource = &quot;CodeWorkspace&quot;<br>}</pre><ul><li>the rest of the Stages have some configurations attached, feel free to browse them, and here’s the AWS documentation page to help you navigate all the configurations: <a href="https://docs.aws.amazon.com/codepipeline/latest/userguide/reference-pipeline-structure.html">https://docs.aws.amazon.com/codepipeline/latest/userguide/reference-pipeline-structure.html</a></li></ul><p>The <strong>iam.tf</strong> file contains the required permissions for the CI/CD environment to run. In my case, they mostly revolve around codecommit and codebuild actions. Here’s my example:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a5bd685ee99d71840c68f4f1edfcb0dd/href">https://medium.com/media/a5bd685ee99d71840c68f4f1edfcb0dd/href</a></iframe><p>I do need to specify some access for AWS CodeBuild as well, so I reference the CodeBuild resources which we will discuss next.</p><h3>AWS CodeBuild</h3><p>Since AWS CodePipeline needs to execute its Stages somewhere, let’s say a Build Engine, we have CodeBuild available for us to handle this task. It employs multiple types of computes to run code, and I found I am comfortable using Containers as a Build setup.</p><p>The two resources I defined reflect the <em>terraform plan</em> and the <em>terraform apply</em> CodePipeline Stages, and I need to pass them in because I want to load the buildspec yaml files which, in turn, define what steps to run to deploy the terraform code. We’ll see that in a minute.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d3e2993eaf72cd97f52e7afc186896d3/href">https://medium.com/media/d3e2993eaf72cd97f52e7afc186896d3/href</a></iframe><p>The variables I use in the <strong>environment{}</strong> section reflect the compute type and the container image name and type that CodeBuild will use to run my code. These are further defined in my <strong>global.tf</strong> file:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fee1f28686973ffce48c84f41101f38d/href">https://medium.com/media/fee1f28686973ffce48c84f41101f38d/href</a></iframe><p>As mentioned, with each <strong>aws_codebuild_project</strong> resource, I call in the buildspec yaml file.</p><p>The <strong>buildspec-plan.yml</strong> file downloads and installs terraform, switches to the CodeBuild source directory, and runs the terraform plan commands which give me an out plan file that I then save to AWS S3 to be loaded later, in the apply Stage.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b097763a5d7886c7060228f0c958ee27/href">https://medium.com/media/b097763a5d7886c7060228f0c958ee27/href</a></iframe><p>I make use of a few variables here:</p><p><strong>NOTE:</strong> AWS CodeBuild specific variables are prefixed with <strong>CODEBUILD_</strong></p><ul><li><strong>CODEBUILD_SRC_DIR</strong> is available by default into AWS CodeBuild and reflects the main build directory inside the container;</li><li><strong>CODEBUILD_RESOLVED_SOURCE_VERSION</strong> is the calculated commit hash for the latest AWS CodeCommit push/merge;</li><li><strong>TF_VERSION</strong> &amp; <strong>PERMISSION_SETS_DIR </strong>I expose at the top under <em>env.variables</em> and they can be templated from somewhere else, like Ansible, if needed;</li><li><strong>PIPELINE_EXECUTION_ID</strong> is the variable exported by the AWS CodePipeline Plan and Apply Stages configurations above and represents the unique hash ID of each pipeline execution.</li></ul><p>I also save my terraform plan generated file <em>tfplan_commitid_${CODEBUILD_RESOLVED_SOURCE_VERSION}_pipelineid_${PIPELINE_EXECUTION_ID}</em> as a artifact called <strong>TerraformPlanFile</strong>.</p><p>The <strong>bildspec-apply.yml</strong> file does mostly the same thing as the plan buildspec, but instead of running terraform plan, it runs terraform apply and deploys my infrastructure changes to AWS.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c763d6afc3a245d25828f29993c56438/href">https://medium.com/media/c763d6afc3a245d25828f29993c56438/href</a></iframe><p>I use the same variables as for the plan bit. One caveat to note here, and this one is important and is the stuff of nightmares:</p><ul><li><strong>CODEBUILD_SRC_DIR_TerraformPlanFile</strong> -&gt; when handling your “secondary” artifact (in this Stage, my first artifact was <em>CodeWorkspace</em>, as noted earlier), you need to add its <em>artifacts.name</em> value to the CODEBUILD_SRC_DIR variable;</li><li><strong>CODEBUILD_SRC_DIR</strong> on its own will still point to your initial source / Artifact.</li></ul><p>I, of course, need to create IAM permissions for CodeBuild as well, and these ones mostly revolve around SSO and DynamoDB:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4e0cf95314ed90aa0db05edf1f369de4/href">https://medium.com/media/4e0cf95314ed90aa0db05edf1f369de4/href</a></iframe><h3>AWS S3</h3><p>The last piece of the puzzle is the creation of an S3 bucket with a specific purpose to store build artifacts.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ebc41afd7c0a8f20570ba98752efb8b8/href">https://medium.com/media/ebc41afd7c0a8f20570ba98752efb8b8/href</a></iframe><p>Noting fancy here, just making the bucket Private and encrypting the content for good measure.</p><p>The end result:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/474/1*eA5zdradEie4lfXA-wpHfg.png" /></figure><h3>Cost</h3><p>The total monthly cost of the solution taking into account one execution per day: <strong>$3.01</strong></p><p>This was pretty long, but I hope the extra details help anyone wanting to do something similar, CI/CD fully integrated into AWS.</p><p>Any thoughts? Drop a comment! Like the article? Clap a few times below and let me know how much you enjoyed it! :)</p><p>Thank you!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=12ade4d9cfa3" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>