<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Oliver Schenk on Medium]]></title>
        <description><![CDATA[Stories by Oliver Schenk on Medium]]></description>
        <link>https://medium.com/@oliver.schenk?source=rss-159faff569ab------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*29HrMj51al_9XPsONLkcKg.jpeg</url>
            <title>Stories by Oliver Schenk on Medium</title>
            <link>https://medium.com/@oliver.schenk?source=rss-159faff569ab------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 14:20:49 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@oliver.schenk/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[AWS Multi-account Multi-region Bootstrapping with Terraform]]></title>
            <link>https://levelup.gitconnected.com/aws-multi-account-multi-region-bootstrapping-with-terraform-39aeed097ad2?source=rss-159faff569ab------2</link>
            <guid isPermaLink="false">https://medium.com/p/39aeed097ad2</guid>
            <category><![CDATA[terragrunt]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[bootstrapping]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[aws-organizations]]></category>
            <dc:creator><![CDATA[Oliver Schenk]]></dc:creator>
            <pubDate>Fri, 14 Apr 2023 13:41:09 GMT</pubDate>
            <atom:updated>2023-04-16T13:44:04.731Z</atom:updated>
            <cc:license>http://creativecommons.org/publicdomain/zero/1.0/</cc:license>
            <content:encoded><![CDATA[<p>This is a detailed, step-by-step guide that runs through one possible method of bootstrapping a multi-account, multi-region AWS environment using Terraform and a bit of bash.</p><p>Source code can be found <a href="https://github.com/oliverschenk/aws-multi-account-multi-region-bootstrapping-terraform">on Github</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*W4XJ7DxGiUX-IkmV" /><figcaption>Photo by <a href="https://unsplash.com/@alinnnaaaa?utm_source=medium&amp;utm_medium=referral">Alina Grubnyak</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>High Level Architecture</h3><p>Let me first outline what kind of architecture you will end up with after following this guide. Most of it will be deployed using Terraform.</p><ul><li>AWS Organizations for member account management</li><li>Account OU structure following suggested best practice</li><li>5 member accounts — deployment, shared services, security, logging, development</li><li>Configurable billing alarm notification</li><li>AWS Identity Center for user/group management and easy SSO access to accounts and CLI credentials</li><li>Centralised logging for CloudTrail for each account</li><li>Terraform deployment CI/CD pipeline for this project including a CodeCommit repo, CodeBuild and CodePipeline with approval stage and notification</li><li>Terraform remote state managed in S3 and appropriate roles and policies for Terraform state management and Terraform deployment into each account</li></ul><p>This sample project deploys all of the above using minimal manual intervention and bootstraps the foundational architecture from your local machine into AWS. You can then clone the repository that is pushed to CodeCommit and make changes as you see fit. Anything you commit and push into the repository will be automatically built and deployed.</p><h3>Alternative Solutions</h3><p>It is important to be aware that the solution presented here is only one of many ways to deal with multi-account and multi-region environments. It all comes down to trade-offs and what best fits your requirements in terms of architecture. The following list suggests some other ways you can build a similar environment, but with different trade-offs.</p><h4>Manual configuration using AWS Console</h4><p>Everything that is defined in this guide can be configured using the AWS Console without any infrastructure code. This method is almost never recommended for production systems as it takes a lot of time, it is difficult to maintain and difficult to reproduce.</p><h4>AWS CloudFormation and CDK</h4><p>This guide uses Terraform, but there is no reason why a similar outcome could not be achieved using CloudFormation and CDK. It depends on the skill sets available to you and the teams you may be working with. CloudFormation and CDK are maintained by AWS so it’s more likely that the latest features would be supported.</p><h4>AWS Control Tower</h4><p>This guide presents a multi-account and multi-region strategy by deploying and configuring a range of AWS services to achieve an outcome. When it comes to this specific problem, it is also possible to use a managed service called <a href="https://docs.aws.amazon.com/controltower/latest/userguide/what-is-control-tower.html">AWS Control Tower</a>. This service is specifically designed to offer a mechanism for creating best-practice environments and uses services such as AWS Organizations, AWS Service Catalog and IAM Identity Center under the hood. This can be a good starting point for many organisations as it takes away some of the heavy lifting. There is of course a cost involved and it has <a href="https://docs.aws.amazon.com/controltower/latest/userguide/limits.html">some limitations</a>.</p><h3>Well Architected Framework</h3><p>Let’s just briefly go over a bit of background and the inspiration for this project.</p><p>When it comes to cloud development and adoption, there is an endless supply of “getting started” tutorials that don’t always have much substance. I wanted to build something that is more practical, and not just theoretical. Something that considers a few best practices.</p><p>It is easy to get started by logging into the Console where you can create some resources and start hacking away. This is, after all, one of the Cloud’s benefits. You can leverage its power for quick prototyping and then tear it all down again at almost no cost.</p><p>Once you start having to build more substantial, permanent infrastructure and services, it is important to step back and put some thought into how it will all be managed. In fact, AWS provides their own guidance to help you in your thinking through its <a href="https://aws.amazon.com/architecture/well-architected/">Well-Architected Framework</a>.</p><p>This guide leverages the <a href="https://docs.aws.amazon.com/organizations/latest/userguide/orgs_introduction.html">AWS Organizations</a> service to provide features such as centralised <a href="https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_accounts.html">account management</a>, <a href="https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/consolidated-billing.html">consolidated billing</a>, <a href="https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_ous.html">organizational units</a> (OUs), <a href="https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_policies_scps.html">service control policies</a> (SCPs), <a href="https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_policies_tag-policies.html">tag policies</a>, and integration with <a href="https://docs.aws.amazon.com/organizations/latest/userguide/orgs_integrate_services_list.html">many other services</a>.</p><p>Using multiple accounts is a good method of isolation and provides natural boundaries for security, access and billing. This is explained in a <a href="https://aws.amazon.com/blogs/mt/best-practices-for-organizational-units-with-aws-organizations/">great AWS blog post</a>. This post also details recommended account structures using OUs to group and segregate accounts. By attaching common policies to OUs rather than individual accounts means your policies become easier to manage.</p><p>In a whitepaper called <a href="https://docs.aws.amazon.com/whitepapers/latest/organizing-your-aws-environment/organizing-your-aws-environment.html">Organizing Your AWS Environment Using Multiple Accounts</a>, AWS sets out some best practices for how your environment should be organised. This also includes <a href="https://docs.aws.amazon.com/whitepapers/latest/organizing-your-aws-environment/recommended-ous-and-accounts.html">Recommended OUs and accounts</a> as well as <a href="https://docs.aws.amazon.com/whitepapers/latest/organizing-your-aws-environment/patterns-for-organizing-your-aws-accounts.html">Patterns for organizing your AWS accounts</a>.</p><p>Now, as is the case with most guides and tutorials out there, for most of you, this will not be <em>exactly</em> what you are looking for, but I hope I can at least contribute towards filling some of your gaps.</p><p>Whilst I tried to create something a little bit more than a Hello World tutorial, there are still things you will need to consider on your own. You’ll need to harden the solution to your own security needs by implementing the policies you need. For example, you’ll want to restrict the management account TerraformDeployment policy a bit more. You’ll want to implement more targeted SCPs. You’ll want to tweak the CloudTrail settings and maybe implement other security services.</p><h3>Multi-account Environments</h3><p>There are many ways to structure your environment, depending on whether you are setting up for an organisation, for a SaaS platform or for a specific project (just to name a few examples). AWS Accounts are a good isolation method — some examples include creating separate accounts for:</p><ul><li>Management account (required)</li><li>Development</li><li>Test/Staging/UAT</li><li>Production</li><li>Security and access management</li><li>Auditing</li><li>Logging</li></ul><p>The topic of designing and managing multi-account environments is huge, but at a high level the idea is to use accounts as a way to create natural security and data contexts. Accounts are a natural security instrument, because one account can only talk to another account if such access is specifically granted on both sides.</p><p>You’ll find many articles online for these topics. AWS also provides services that help manage an environment with many accounts.</p><ul><li><a href="https://aws.amazon.com/organizations/getting-started/best-practices/">AWS Organizations</a></li><li><a href="https://docs.aws.amazon.com/whitepapers/latest/organizing-your-aws-environment/organizing-your-aws-environment.pdf">Organizing your AWS environment using multiple accounts</a></li><li><a href="https://docs.aws.amazon.com/controltower/latest/userguide/what-is-control-tower.html">AWS Control Tower</a></li></ul><h3>Starting point</h3><p>In order to keep the complexity in this article manageable, we are going to focus on creating the scripts and Terraform code necessary to bootstrap an environment that includes the following accounts:</p><ul><li>Deployment — CI/CD and Terraform state</li><li>Security — Security and identity</li><li>Development</li><li>Shared Services</li><li>Logging</li></ul><p>You can then add your own Production account when you are ready to do so and have considered the security configuration that you need.</p><p>Please note that removing accounts is not as easy as creating accounts. I’ve added a section at the end of this article on what you can do to remove resources and close accounts.</p><p><strong>Remember that the email address you used to create an account can never be used again!</strong></p><h3>Prerequisites</h3><p>You’ll need a Linux based environment like Ubuntu. I generally run Windows with a Ubuntu based WSL instance. You’ll need at least the following tools to get going:</p><ul><li>Terragrunt</li><li>Terraform</li><li>AWS CLI</li><li>Bash shell</li><li>git with <a href="https://docs.aws.amazon.com/codecommit/latest/userguide/setting-up-git-remote-codecommit.html">git-remote-codecommit</a></li><li><a href="https://stedolan.github.io/jq/">jq</a></li><li><a href="https://keybase.io/">Keybase</a></li></ul><h3>Preparing the Management Account</h3><p>There are a couple of things that need to be done manually before we can get started on creating member accounts and resources.</p><h4>Creating a Management account</h4><p>The Management account must be set up manually as this is where your billing and payment information will be stored and this account acts as the over-arching management account with consolidated billing.</p><p>It would be recommended to deploy the resources in this guide into a brand new Management account as this would be the safest method of deployment and would avoid complicating any issues and errors.</p><h4>Turning on IAM Identity Center</h4><p>The IAM Identity Center, which was previously known as AWS Single Sign-On, is not yet full supported in Terraform when it comes to some of the underlying settings. This is one of those services that is best enabled in the Console, because it does some things under the hood that make it work correctly.</p><ol><li>Log-in as the root user into your Management account and head over to IAM Identity Center.</li><li>Make sure you are in the desired region.</li><li>Click on the <strong>Enable</strong> button to turn on IAM Identity Center.</li></ol><p>In the settings tab I left the default settings for the Identity source, but I customised my AWS access portal URL. In the Authentication tab I enabled the OTP for users created via API, turned on MFA requirements to always-on and require them to register an MFA device at sign-in.</p><p>There is still a step we can do later regarding delegation of administrative rights to another account so that we don’t use the Management account going forward. We can do that after all our member accounts are created and we have a <em>shared services</em> account we can use.</p><h4>Create Terraform initialisation user</h4><p>In this step we are going to create a user in the management account with specific permissions required to deploy the initial bootstrapping resources. The primary purpose of this user is to deploy the initial accounts, roles and policies. More information on this shortly.</p><p>In the management account create a new user called terraform-init and create an inline policy, or a separate policy, with the permissions provided in TerraformInit-IAM-Policy.json. Once created, generate a set of access keys and keep them handy.</p><p>This user will be deleted again after bootstrapping is completed.</p><p>At this point you should have the following setup.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/440/1*kzLf2TI1AUSPJLhhsKsXnA.png" /><figcaption>Management account with terraform-init user</figcaption></figure><h3><strong>Configuration</strong></h3><p>When using Terraform/Terragrunt, a useful strategy for gathering configuration parameters is to use an external YAML file and load it using the yamldecode and file functions. This mechanism and how this is used is discussed later, but for now the important thing is to adjust the configuration to suit your needs.</p><p>In this project you’ll find common_vars.yaml as the global configuration file that you should configure.</p><p>The namespace and name parameters are used throughout the Terraform code to attach an appropriate prefix to each resources. This ensures your resources can easily be identified by name and tags. You should change this to your own organisation’s namespace and project name.</p><p>The other important parameters are the account email addresses and the notification email addresses for billing and builds as well as setting the region that you want to work in. I suggest you use the + notation in order to use random email addresses as once you use an email address no other account can use it again unless you change it before closing an account. Even if an account is closed, the email address cannot be reused.</p><p>To be honest there’s probably some more things that I could have made configurable. I’ll leave that up to you!</p><h3>The Deployment Script</h3><p>The deployment script deploy.sh takes care of bootstrapping and deploying initial resources, including the CI/CD pipeline that will then take care of any future changes within the member accounts.</p><h4>What does it do</h4><p>The deployment script can be divided into roughly five steps. Note however that depending on what options you specify when running the script some of the steps will be different or skipped. This is explained in the next section.</p><ol><li>Deploy resources defined in the bootstrap/organization folder using local state first and then switch to remote Terraform state in S3.</li><li>Generate a temp-admin user in the Deployment account, decrypt the credentials and store as variables.</li><li>Deploy member account resources defined the accounts folder.</li><li>Push the entire repository to CodeCommit.</li><li>Destroy the temp-admin user.</li></ol><h4>Usage</h4><pre>DESCRIPTION:<br>  Script for initializing an AWS account structure. See README for more details.<br>  *** MUST BE RUN WITH ADMIN CREDENTIALS FOR terraform-init USER IN THE MANAGEMENT ACCOUNT ***<br><br>USAGE:<br>  deploy.sh -a terraform_init_access_key -s terraform_init_secret_key -k keybase_profile<br>  [-r default_region] [-l] [-t tf_account_id] [-p]<br><br>OPTIONS<br>  -l   skip using local state, can be used after the inital run, must provide tf_account_id<br>  -p   push code to remote repo, should only be used once to bootstrap CodeCommit<br>  -r   the default region for this deployment<br>  -t   account ID where Terraform state is stored, used with -l option</pre><h4>Running the script for the first time</h4><p>If running the script for the first time you should specify the -p flag in order to push the code to the CodeCommit repository.</p><pre>./deploy.sh \<br>  -a &lt;terraform_init_access_key&gt; \<br>  -s &lt;terraform_init_secret_key&gt; \<br>  -k &lt;your_keybase_account&gt;\<br>  -p</pre><h4>Running the script subsequent times</h4><p>If you’ve run the script before and the Terraform state and the accounts have already been created then you should skip creating local state and specify the Deployment account ID. This can be done using the -l flag and the -t flag with the account ID. Also, if the code was already pushed to CodeCommit you should no longer specify the -p flag.</p><pre>./deploy.sh \<br>  -a &lt;terraform_init_access_key&gt; \<br>  -s &lt;terraform_init_secret_key&gt; \<br>  -k &lt;your_keybase_account&gt; \<br>  -l \<br>  -t &lt;deployment_account_id&gt;</pre><h4>Terraform state</h4><p>When you run the deployment script the first time it will use local Terraform state. Normally you would only run the deployment script once, but if you encounter some issues or you need to adjust anything in the bootstrap folder, then you’ll have to run it more than once.</p><p>The problem is that once Step 1 above is successful, your Terraform state will be stored remotely in an S3 bucket in the Deployment account. At that point you should use the -l flag and specify the Deployment account ID with the -t flag so that it uses remote Terraform state for any subsequent deployments that you might want to do.</p><h4>CodeCommit repository</h4><p>The deployment script will push the code into a CodeCommit repository if the -p flag is provided. This should be used for the initial run only. After that the -p flag is no longer needed as your code will already be stored in CodeCommit.</p><p>After the first deployment you should clone the remote CodeCommit repository and make any changes in there so that you can use git to push changes into the CI/CD pipeline.</p><h4>Deployment Step 1 — Bootstrap organization resources</h4><p>When you run the deployment script for the first time (and without the -l flag) it will force Terraform to use local state for the resources in the bootstrap/organization folder.</p><pre>echo &quot;Copying local state override file from overrides/backend-local.override.tf&quot;<br>cp overrides/backend-local-override.tf .</pre><p>It will then run the init and apply commands using local state.</p><pre>echo &quot;Running terragrunt init using local state&quot;<br>terragrunt init --terragrunt-config terragrunt-local.hcl<br>echo &quot;Running terragrunt apply using local state&quot;<br>terragrunt apply --terragrunt-config terragrunt-local.hcl</pre><p>Once applied, it will export the Deployment account ID into the TF_AWS_ACCT environment variable.</p><pre>echo &quot;Exporting Terraform state account&quot;<br>DEPLOYMENT_AWS_ACCT=$(terraform output -json account_ids | jq -r &#39;.&quot;deployment&quot;&#39;)<br>export TF_AWS_ACCT=${DEPLOYMENT_AWS_ACCT}</pre><p>This is needed in terragrunt.hcl when generating the remote state block so that the OrganizationAccountAccessRole role is assumed from the context of the Deployment account. This is because we want all Terraform remote state to go into an S3 bucket in the Deployment account.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/da6e5367b407e944164a2124b5911cb7/href">https://medium.com/media/da6e5367b407e944164a2124b5911cb7/href</a></iframe><p>The script will now remove the reference to local state and run terragrunt init again to initialise remote state.</p><pre>echo &quot;=== COPYING LOCAL STATE TO S3 ===&quot;<br>echo &quot;Removing backend-local-override.tf&quot;<br>rm ./backend-local-override.tf || true<br>sleep 10 # give AWS some time for the IAM policy to take effect<br>echo &quot;Running terragrunt init to initialise Terraform state S3 backend&quot;<br>terragrunt init</pre><p>At this point we now have the organization resources deployed and remote state managed in the Deployment account. You will receive a whole bunch of emails from AWS welcoming you for each account.</p><h4>Deployment Step 2 — Create temp-admin user</h4><p>Now that the organization level resources have been deployed, the script exports all of the account IDs into a JSON file and prints it out. This can be read by Terragrunt if needed by any of the Terraform code.</p><pre>echo &quot;Outputting account IDs into accounts.json&quot;<br>terraform output -json account_ids | jq &gt; ../../accounts.json<br>cat ../../accounts.json</pre><p>The deployment script will now create the temp-admin user in the bootstrap/temp-admin folder. This is a user that lives in the Deployment account context and is given access to assume the OrganizationAccountAccessRole roles in all of the member accounts so that it can bootstrap the initial member account resources like the CI/CD pipeline, CloudTrail buckets, account aliases and create a billing alarm.</p><pre>echo &quot;Running terragrunt init&quot;<br>terragrunt init<br>echo &quot;Running terragrunt apply using Terraform state account $TF_AWS_ACCT and keybase profile $KEYBASE_PROFILE&quot;<br>terragrunt apply -var terraform_state_account_id=${TF_AWS_ACCT} -var keybase=${KEYBASE_PROFILE}</pre><p>The neat thing here is that it uses Keybase to encrypt the access keys. These are then decrypted and stored as variables. You’ll need your Keybase account credentials at this stage.</p><pre>echo &quot;Storing and decrypting temp-admin access keys&quot;<br>ADMIN_ACCESS_KEY=$(terraform output -raw temp_admin_access_key)<br>ADMIN_SECRET_KEY=$(terraform output -raw temp_admin_secret_key | base64 --decode | keybase --pinentry=none pgp decrypt)</pre><h4>Deployment Step 3 — Bootstrap account resources</h4><p>The script now exports the credentials for the temporary admin user and deploys all of the account resources in the accounts folder using the recursive power of Terragrunt’s run-all command.</p><pre>echo &quot;=== APPLYING ALL ACCOUNT RESOURCES ===&quot;<br>export_admin_keys<br><br>echo &quot;=== APPLYING INFRASTRUCTURE FOR EACH ACCOUNT ===&quot;<br>pushd ./accounts<br>terragrunt run-all init<br>terragrunt run-all apply</pre><p>The script now just exports the CodeCommit repository name so that it can be used in the next step.</p><pre>pushd ./ap-southeast-2/deployment/codecommit<br>REPO_NAME=$(terragrunt output -raw repository_name)<br>echo &quot;Stored CodeCommit repository name: $REPO_NAME&quot;</pre><h4>Deployment Step 4 — Push code to CodeCommit</h4><p>All account resources and the CI/CD pipeline is now ready to go, we just need to push the first commit into it from our local source code.</p><p>As explained earlier, you should only use the -p flag when deploying these resource for the first time. Once your CodeCommit repository is updated you shouldn’t push into it again in this way.</p><p>If you’ve provided the -p flag, the script will do a few things to clean up unnecessary files, clone the empty CodeCommit repo into a repository folder and then copy all of the source in this project into that repository folder. It will then commit and push this to CodeCommit. Finally, it will delete the temporary repository folder.</p><pre>if [[ -n &quot;${PUSH_CODE}&quot; ]]; then<br>    echo &quot;=== PREPARING TO PUSH CODE TO CODE COMMIT REPOSITORY ===&quot;<br>    echo &quot;Cleaning up Terragrunt and Terraform cache&quot;<br>    find . -type d -name &quot;.terragrunt-cache&quot; -prune -exec rm -rf {} \; # clean Terragrunt cache data<br>    find . -type d -name &quot;.terraform&quot; -prune -exec rm -rf {} \; # clean Terraform cache data<br>    echo &quot;Preparing source code folder&quot;<br>    rm -rf repository    # clean if already exists<br>    mkdir repository<br>    echo &quot;Cloning CodeCommig repository: codecommit://$REPO_NAME in to repository folder&quot;<br>    git clone codecommit://$REPO_NAME repository<br>    echo &quot;Transfering source code into cloned repository folder&quot;<br>    rsync -av . ./repository --exclude repository<br>    pushd ./repository<br>    echo &quot;Running git add and commit for initial commit&quot;<br>    git --git-dir=.git add .<br>    git --git-dir=.git commit -m &quot;Bootstrap commit&quot;<br>    echo &quot;Pushing to remote repository&quot;<br>    git --git-dir=.git push<br>    popd<br>    echo &quot;Cleaning up repository folder&quot;<br>    rm -rf repository<br>fi</pre><p>Once this deployment script has finished, you now have a CodeCommit repository that you can clone and treat as live code. Anything you push into this repository will trigger a CodePipeline execution.</p><h4>Deployment Step 5 — Destroy temp-admin user</h4><p>The temp-admin user can now be destroyed as the CI/CD pipeline now assumes responsibility for executing changes.</p><pre>pushd ./src/bootstrap/temp-admin<br>export_management_keys<br>terragrunt destroy -var terraform_state_account_id=${TF_AWS_ACCT} -var keybase=${KEYBASE_PROFILE}</pre><h4>Cleanup</h4><p>Assuming that the deployment was successful you should now delete the terraform-init user in the Management account.</p><h3>Users, Providers and Roles</h3><p>One of the tricky things to get right is making sure that the correct resources are deployed to the correct account when using Terragrunt and Terraform.</p><p>As you saw in the deployment script in the previous section, there are multiple phases in the script. It starts with the terraform-init credentials that are passed to it for the organization deployment, but it then creates a temp-admin user for the account deployments. In amongst all that is also the Terraform state and making sure it gets stored in the Deployment account.</p><p>On top of that you then also have the OrganizationAccountAccessRole, the Terraform roles and the Deployment roles. It can get quite confusing! To be able to understand all this it is best to break it down into manageable steps.</p><h4>Bootstrap organization phase</h4><p>In the first part of the script the terraform-init user is used. The credentials of that user are within the Management account context. At this point neither the AWS Organization nor the member accounts exist yet. Everything that we create in this first step is in the Management account.</p><p>However, once the AWS Organization is created and the accounts are created your environment will look something like this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/541/1*aXEPHDvjae0GhDkFSEAJcg.png" /><figcaption>AWS Organizations and member accounts</figcaption></figure><p>The magic sauce here is the OrganizationAccountAccessRole that exists in each of the member accounts. This is a role that AWS Organizations creates automatically. The key is that this role is pre-configured with a trust policy such that a principal in the Management account is allowed to assume it.</p><p>If you look at the Terraform-IAM-Policy.json file you’ll see that we’ve given the terraform-init user access to this role in any account.</p><pre>{<br>    &quot;Sid&quot;: &quot;AllowRoleAssumptionToOrgAccounts&quot;,<br>    &quot;Effect&quot;: &quot;Allow&quot;,<br>    &quot;Action&quot;: [<br>        &quot;sts:AssumeRole&quot;<br>    ],<br>    &quot;Resource&quot;: [<br>        &quot;arn:aws:iam::*:role/OrganizationAccountAccessRole&quot;<br>    ]<br>}</pre><p>That’s why the terraform-init user is able to assume the OrganizationAccountAccessRole in each member account and therefore have Administrator level permissions in each member account.</p><p>Boom!</p><p>In the following diagram you can see what is deployed and what role that is assumed using the credentials of the terraform-init user. The red colour is the Management account context, the green colour is the Deployment account context and the blue is a simplified view of the other accounts (to avoid duplication on the drawing).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/733/1*euH1yRu7HrgFJaj_SNSovw.png" /><figcaption>Account overview with deployment roles and policies</figcaption></figure><h4>Deployment strategy</h4><p>In the previous diagram you can see that most of the action regarding Terraform and deployments happens in the Deployment account. This is where Terraform remote state is stored and this is the account from which the CI/CD pipeline is executed.</p><p>To enable the pipeline to deploy resources into each member account (including the Deployment account itself) and into the Management account, the necessary cross-account permissions must exist.</p><p>That’s what the TerraformDeployment role in each account is for. These roles are only allowed to be assumed from the Deployment account. The other side of the cross-account equation is on the the Deployment account side. In the Deployment account you’ll see an AllowDeploymentToAssumeTerraformDeploymentIn* policy for each of the accounts. As we’ll see later, once the CI/CD pipeline is created this policy will be attached to the execution role of the pipeline.</p><p>We can visualise this for one of the accounts in the following diagram. The same will apply to each account.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/472/1*0dPvMrGipgbEBi5kXOAqlQ.png" /><figcaption>Deployment account allowed to assume TerraformDeployment role in Development account</figcaption></figure><h4>Terraform state</h4><p>An important part of the deployment strategy is how the Terraform state is managed. Unlike the resources deployed to each individual member account, we want Terraform state to be stored centrally but still individually for each account and region. In other words it should be in one S3 bucket, but each account/region combination should be stored with a unique prefix.</p><p>You’ll see two roles — TerraformAdministrator and TerraformReader in the previous diagram. The sole purpose of the first roles is to allow Terraform to create and update remote state configurations. The second role on the other hand is used to allow Terraform to reference existing state objects where needed in a read-only fashion.</p><p>You can visualise Terraform state management as follows.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/745/1*HFHrFTq-WpIGnw239HBx4w.png" /><figcaption>Terraform state management</figcaption></figure><h4>The temp-admin user</h4><p>So what is the purpose of the temp-admin user in the deployment script?</p><p>Well, at this stage our CI/CD pipeline hasn’t been bootstrapped yet, which is why we need the temporary temp-admin user in the Deployment account and attach each of the AllowDeploymentToAssumeTerraformDeploymentIn* policies so that we can deploy all of the resources in the accounts folder. In addition to that it also needs the Terraform Administrator role to be able to write the Terraform state.</p><p>The follow diagram shows the effective permissions of the temp-admin user and the roles that it will be allowed to assume.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/572/1*2Stvd3O1YMoEZsBTgdmbWQ.png" /><figcaption>temp-admin user effective permissions</figcaption></figure><p>We’ll talk about how Terragrunt generators and Terraform providers are used to assume all of these different roles and manage Terraform state in the correct account very soon.</p><h4>From deploy.sh to CodePipeline</h4><p>It is worth mentioning here again, mainly for clarity, that the main purpose of the deploy.sh script is to deal with the chicken and the egg situation of bootstrapping a CI/CD pipeline. Something has to create the pipeline first before it can exist and manage itself!</p><p>The deploy.sh script simply makes use of a temporary temp-admin user in the Deployment account with the same permissions that the CI/CD pipeline will have once deployed.</p><p>This is why all of the Terragrunt code, all of the permissions and Terraform state work just the same whether they are run from deploy.sh or from CI/CD. They both run within the Deployment account context and access the same Terraform source code.</p><p>The only difference is that deploy.sh is executed manually by a human and contains a static set of commands, whereas the CI/CD pipeline runs automatically and uses the instructions defined in the buildspec YAML files.</p><h3>CI/CD pipeline</h3><p>The first part of the deployment script takes care of bootstrapping all of the necessary accounts, roles and policies to facilitate cross-account deployments. The second part then deploys the infrastructure into the member accounts, which includes a basic CI/CD pipeline.</p><p>The pipeline is based on CodeCommit for the source, a CodeBuild stage for the Terraform plan phase, a CodeBuild stage for the Terraform apply phase and CodePipeline as the overall orchestrator. This configuration also requires various roles and policies in order to detect code Changes in CodeCommit and start pipeline execution via an EventBridge event, store artifacts and build logs in S3 and send an email notification via an SNS topic and subscription. All of this can be visualised as follows.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/756/1*-FNJAIDMVAPZlG_2qTHtug.png" /><figcaption>CI/CD pipeline overview</figcaption></figure><p>Note that the names of the resources in the above diagram may not match exactly with what is produced by the deployment. The names in this diagram have been simplified.</p><h4>Scope of deployment</h4><p>It is worth noting again that the scope of the automated build and deployment only includes the source code in the accounts folder. The bootstrap folder is only used by the deployment script when first deploying the initial account structure.</p><p>If you need to make changes to AWS Organizations or the underlying accounts, such as creating new accounts, this is best done in a manually controlled fashion by updating the bootstrap code and running it using the deploy.sh script. This will bootstrap any new accounts with the necessary TerraformDeployment roles.</p><p>You’ll then have to create new sub-folders in the accounts folder and relevant region for any account that you create.</p><p>You can run the deploy.sh script using the -l and -t flags so that it skips local state and skips uploading the whole repo to CodeCommit.</p><h4>Management account deployment</h4><p>Another thing worth mentioning is that in this project the Management account has also been given a TerraformDeployment role that the Deployment account can assume. This is currently configured with an Administrator policy attached.</p><p>This was done to facilitate creating billing alarms.</p><p>In a production setting it would be recommended to harden this using either a more limited TerraformDeployment policy or by applying Service Control Policies (SCPs) via AWS Organizations.</p><h4>CodeCommit source control</h4><p>As was explained earlier, the deployment script pushes the initial code to CodeCommit as part of the deployment process. From that point on you should clone the repository and use that to make any further changes as the CI/CD pipeline is now live.</p><p>CodeCommit was designed to be used as a private repository. In order to facilitate the use of standard git commands over HTTPS, you should use the git-remote-codecommit git extension as <a href="https://docs.aws.amazon.com/codecommit/latest/userguide/setting-up-git-remote-codecommit.html">explained here</a>.</p><p>Using this means you can use AWS credential profiles together with standard git commands such as clone, push and pull. The repository URL will look something like this.</p><pre>git clone codecommit://CodeCommitProfile@MyDemoRepo my-demo-repo</pre><p>The CodeCommitProfile is your configured AWS credentials for the Deployment account. You should customise this to your needs and only give sufficient access to perform the tasks you need in CodeCommit. You may also want to apply your own branching strategy and pull request methodology.</p><h4>CodeCommit source change detection</h4><p>When there is a source code change in the master branch of CodeCommit we want AWS to automatically trigger pipeline execution to deploy the changes. There are two ways to achieve this:</p><ol><li>Polling — CodePipeline can poll the CodeCommit repository and look for changes. This is an internal mechanism.</li><li>CloudWatch Event — The CloudWatch method is to use an event rule to detect the CodeCommit referenceCreated and referenceUpdated events on a particular branch name and then set CodePipeline as a target for the rule. CodePipeline will start execution when it receives the event.</li></ol><p>Option 1 is no longer recommended by AWS as it does not provide any flexibility or customisation. The second option is more powerful as it allows you to customise the conditions under which a CodePipeline execution is triggered and you’ll also have a full audit history if you use CloudTrail.</p><p>In this deployment we configured option 2. You’ll find that in CodePipeline the PollForSourceChanges has been set to false to disable automatic polling. To make the even rule work we need the following elements.</p><p>A CloudWatch event rule which looks for the aws.codecommit source event with the given CodeCommit ARN and the referenceCreated and referenceUpdated events on the given branch name. You’ll find the <a href="https://docs.aws.amazon.com/codecommit/latest/userguide/monitoring-events.html">CodeCommit events reference in the docs</a>.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d1b41fd1d4825acd617bb39ff780076a/href">https://medium.com/media/d1b41fd1d4825acd617bb39ff780076a/href</a></iframe><p>With this we now have the ability to detect and capture the events that we are interested in from CodeCommit. The next step is to send them to CodePipeline. For this we need a rule target by specifying the rule created above and the ARN of CodePipeline.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/eb77dc18882fb8d916fa0a287330ea89/href">https://medium.com/media/eb77dc18882fb8d916fa0a287330ea89/href</a></iframe><p>You’ll notice that there is also a role ARN that needs to be specified. As with any action in AWS, we need to give the originating principal the permission to start the pipeline execution. In this case the principal is events.amazonaws.com and it needs to be able to assume the role that has the codepipeline:StartPipelineExecution action allowed on the CodePipeline.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/79b29132915ddfb2e4b44a57e1a5fbdc/href">https://medium.com/media/79b29132915ddfb2e4b44a57e1a5fbdc/href</a></iframe><p>With this we now have the ability to push changes into CodeCommit and automatically trigger CodePipeline execution.</p><h4>CodePipeline stages</h4><p>This particular project is primarily for the purpose of deploying resources using Terraform and thus it makes sense to tailor the pipeline towards the Terraform plan and apply stages. For this reason you’ll find that the CodePipeline is configured with four distinct stages.</p><ol><li>Source — The pipeline fetches the source code from the specified CodeCommit repository and branch.</li><li>Plan — The pipeline executes the CodeBuild project for the plan stage, which is based on the commands in the buildspec-plan.yaml file. This stage outputs a number of artifacts that can then be reviewed including the Terraform plan output, the plan output in JSON format and a visualisation of Terraform changes using a library called terraform-visual.</li><li>Approval — The pipeline sends a notification email to the specified notification email address containing the link to approval and build outputs. A user has to approve or reject the build after reviewing the plan outputs.</li><li>Apply — If the approval stage is passed, the pipeline runs the CodeBuild project for the apply stage. This applies or destroys resources as necessary.</li></ol><p>The pipeline is designed such that the source fetched from CodeCommit in the Source stage feeds into the input of the Plan stage. The output of the Plan stage feeds into the Apply stage if the Approval stage is approved.</p><p>Feeding the Plan stage into the Apply stage means that the Terraform plan outputs can be used by the Apply stage without having to calculate all of the infrastructure differences again. This avoids unnecessary API calls and shortens the build time.</p><p>The pipeline must also be assigned a role that it can assume that gives it the necessary permissions to fetch source from CodeCommit, start CodeBuild builds, store artifacts in S3 and trigger an SNS notification.</p><h4>A note on Terragrunt run-all</h4><p>The plan and apply buildspec commands make use of the Terragrunt run-all command, however some special considerations are needed to ensure that it correctly applies and destroys resources.</p><p>The short version of how to achieve this is to place an empty file with the filename .destroy in the folder where the terragrunt.hcl file is that you wish to destroy.</p><p>You can find out more about the chosen strategy in the <a href="https://medium.com/gitconnected/using-terragrunt-run-all-in-aws-ci-cd-df2877bad198">Using Terragrunt run-all in AWS CI/CD</a> article.</p><h4>CodeBuild</h4><p>There are two separate CodeBuild projects — one for the plan stage and one for the apply stage. From a Terraform point of view both of them are created using the same re-usable CodeBuild Terraform module.</p><p>The difference is that they use different buildspec files and only the plan stage generates output artifacts.</p><p>Let’s first look at the roles and policies required by CodeBuild. A CodeBuild role is configured that CodeBuild is allowed to assume with permissions to send logs to CloudWatch and an S3 logging bucket, to store output artifacts in S3, to assume the TerraformDeployment roles in each member account and the TerraformAdministrator and TerraformReader role in the Deployment account.</p><p>You may also note that because CodeBuild is integrated with CodePipeline, the source artifacts and output artifacts are simply defined as type = CODEPIPELINE, which essentially means that CodeBuild will receive the source from CodePipeline and pass the primary output artifact back to CodePipeline when finished.</p><p>Note that any secondary-artifacts defined in the buildspec file must also be defined in CodeBuild by passing in the secondary_artifacts variable with the relevant artifact identifier and S3 paths.</p><p>In other words, the primary artifact is passed back to CodePipeline whereas the secondary artifacts are written to S3 by CodeBuild directly. The primary artifact is what then gets passed to the Apply stage.</p><h4>Codebuild plan stage</h4><p>This is where things get a little bit more interesting. It gets quite tricky to keep track of how resources actually end up in the correct account and how Terragrunt/Terraform is able to assume the correct roles to achieve this.</p><p>The purpose of the buildspec-plan.yaml file is to define all the commands that CodeBuild will execute in the plan stage. Let’s now go through some of the buildspec details.</p><p>The overall goal of the plan stage is to give you feedback as to what will be created, updated or destroyed so that you can review this before accepting the change and moving onto the Apply stage. This process uses the run-all command, but with a few issues to consider.</p><ol><li>We need to be able to detect whether a destroy marker was placed into a folder so that a resource can be planned with the -destroy flag. If we find a .destroy file next to a terragrunt.hcl module then we need the script to run with a -destroy flag. With Terragrunt, removing a module or deleting a module folder doesn’t mean that this resource gets destroyed, because if there no longer a Terragrunt module there so how can we then run apply against it!</li><li>We want to output both a binary plan file and a JSON file.</li><li>We want to generate the <a href="https://github.com/hieven/terraform-visual">Terraform Visual report</a>.</li><li>We need to arrange all of the outputs into specific primary and secondary artifacts and place them in the correct output folders so that they get moved to S3 correctly.</li></ol><p>First we should note the shell: bash environment parameter at the beginning of the buildspec file. This ensures that CodeBuild executes commands in a bash context. Some of the commands that are used in the buildspec file require this.</p><p>The script will first process modules that need to be destroyed. It looks for the .destroy file and builds a string that tells Terragrunt which folder paths to include.</p><pre># Find all Terragrunt modules to destroy<br>- tg_modules_to_destroy=$(find . -maxdepth 4 -name terragrunt.hcl -execdir [ -e .destroy ] \; -printf &quot; --terragrunt-include-dir %h&quot;)<br><br># Plan to destroy modules no longer needed<br>- |<br>  if [[ -n &quot;${tg_modules_to_destroy}&quot; ]]; then<br>    terragrunt run-all plan $tg_modules_to_destroy -destroy -no-color -out=tfplan.binary<br>  fi</pre><p>The script will then process modules that need to be applies or updated. It looks for all the terragrunt.hcl modules that do not have an associated .destroy file and again builds a string that tells Terragrunt which folder paths to include.</p><pre># Find all Terragrunt modules that do not contain a .destroy file<br>- tg_modules=$(find . -maxdepth 4 -name terragrunt.hcl -execdir [ ! -e .destroy ] \; -printf &quot; --terragrunt-include-dir %h&quot;)<br><br># Plan all changes<br>- |<br>  if [[ -n &quot;${tg_modules}&quot; ]]; then<br>    terragrunt run-all plan $tg_modules -no-color -out=tfplan.binary<br>  fi</pre><p>At this stage we’ve performed the main planning phase of Terraform. However, for review purposes, we want to generate a JSON file of the plan and convert this into a visual HTML report using <a href="https://github.com/hieven/terraform-visual">Terraform Visual</a>.</p><p>To do this, the script will loop through all of the folders that contain a terragrunt.hcl file so that it can find the tfplan.binary file that is then used to generate a JSON output.</p><p>The script needs to first creates some variables that we can use later to know the module name, account name and region name.</p><pre>module_dir=$(dirname $module_path)<br>module_dir=$(echo &quot;$module_dir&quot; | sed &#39;s/\(.*\)\/\.terragrunt-cache\/.*/\1/&#39;)<br><br>module=$(basename $module_dir)<br>account_dir=$(dirname $module_dir)<br>account=$(basename $account_dir)<br>region_dir=$(dirname $account_dir)<br>region=$(basename $region_dir)</pre><p>To output the plan file as a JSON file we use the terragrunt show command.</p><pre>terragrunt show -json $accounts_path/$module_dir/tfplan.binary \<br>  --terragrunt-working-dir $module_dir \<br>  --terragrunt-no-auto-init &gt; $module_dir/plan.json</pre><p>Now generate the HTML report.</p><pre>terraform-visual --plan $module_dir/plan.json --out $module_dir</pre><p>The problem now is that this generates a file called index.html, a folder with some dependencies and a plan.js file that contains the change set. This is not ideal, because these will be duplicated for each module and the file names won’t tell you what module it belongs to.</p><p>Instead, what we will do is move these artifacts into a common folder so that the dependencies are only required once and we’ll rename the files based on the module path and adjust the HTML source code accordingly to point to the renamed plan JavaScript file.</p><p>Let’s build some strings that we can reuse and then rename the HTML file.</p><pre># Prefix that will be used when renaming Terraform Visual reports<br>prefix=$region-$account-$module<br># The path where Terraform Visual reports are currently stored<br>report_dir=$module_dir/terraform-visual-report<br><br># Rename the index.html file to include prefix<br>mv $report_dir/index.html $report_dir/$prefix-terraform-visual.html</pre><p>You can see that reports stored in the terraform-visual-report folder will be moved to $report_dir/$prefix-terraform-visual.html which also renames the HTML file.</p><p>We now need to rename the plan.js file as well.</p><pre>mkdir $report_dir/plans<br>mv $report_dir/plan.js $report_dir/plans/$prefix-plan.js</pre><p>Now that the files have been renamed, we just need to fix the reference inside the HTML file to point to the correct relative location of the plan JavaScript file.</p><pre>sed -i -e &quot;s@\./plan\.js@\./plans/$prefix\-plan\.js@g&quot; $report_dir/$prefix-terraform-visual.html</pre><p>Finally, copy the entire Terraform Visual report structure to the common reports artifact folder.</p><pre>cp -a $report_dir/* $reports_artifacts_path/</pre><p>At this point any common dependency files that already exist will be overwritten, but that’s fine as long as the HTML and JS file have unique file names for each module. This does the job in minimising duplication.</p><p>The final part of the buildspec file is specifying the artifacts. The primary artifact is listed first. This is configured to includes all files except for the reporting files as these are already included in the secondary artifacts.</p><p>In this implementation the .terragrunt-cache folders that Terragrunt generates have been left in place and will be passed to the Apply stage as part of the input artifact. This cache folder can be rather large and will result in an artifact that is greater than 1GB.</p><p>You can either let the S3 lifecycle rules take care of purging these files (currently set to 30 days) or you can exclude the .terragrunt-cache folder from the primary artifact by adding an exclusion to buildspec-plan.yaml. If you exclude it, it just means that the Apply stage will need to download all of the dependencies again and re-create the cache and will thus take longer.</p><pre>artifacts:<br>  files:<br>    - &quot;**/*&quot;<br>  exclude-paths:<br>    - &quot;**/.terragrunt-cache/**/*&quot;<br>    - &quot;**/terraform-visual-report/**/*&quot;<br>    - &quot;reports/**/*&quot;</pre><p>The two secondary artifacts that are defined output the plan.json file, which may be useful for review, and the Terraform Visual report files.</p><p>As was already mentioned previously, the primary artifact will be passed back up to CodePipeline so that it can then pass it onto the Apply stage CodeBuild project, and the secondary artifacts will be stored directly in S3.</p><h4>CodeBuild apply stage</h4><p>By the time we get to this stage, the CodePipeline Approval stage has been approved and it is now time to actually apply the resource changes.</p><p>The purpose of the buildspec-apply.yaml file is to define all the commands that CodeBuild will execute in the Apply stage and execute the change.</p><p>The commands for the Apply stage are actually almost identical to the Plan stage. The main difference is that we don’t generate a binary file using the -out parameter.</p><pre># Find all Terragrunt modules to destroy<br>- tg_modules_to_destroy=$(find . -maxdepth 4 -name terragrunt.hcl -execdir [ -e .destroy ] \; -printf &quot; --terragrunt-include-dir %h&quot;)<br><br># Destroy modules no longer needed<br>- |<br>  if [[ -n &quot;${tg_modules_to_destroy}&quot; ]]; then<br>    terragrunt run-all apply $tg_modules_to_destroy -destroy -no-color<br>  fi<br><br># Find all Terragrunt modules that do not contain a .destroy file<br>- tg_modules=$(find . -maxdepth 4 -name terragrunt.hcl -execdir [ ! -e .destroy ] \; -printf &quot; --terragrunt-include-dir %h&quot;)<br><br># Apply all changes<br>- |<br>  if [[ -n &quot;${tg_modules}&quot; ]]; then<br>    terragrunt run-all apply $tg_modules -no-color<br>  fi</pre><p>The one thing that I was unable to work out using Terragrunt and the run-all command is how to pass the tfplan.binary file. I tried for a long time, but it always errored out saying that the file could not be found even though it was right there!</p><h3>The magic of Terragrunt</h3><p>Let’s just talk a little about how Terragrunt works and how it assumes the correct roles for each account so that resources and Terraform state are created and updated in the correct locations.</p><p>First thing to understand is that Terragrunt is a light wrapper around Terraform. All of the code that defines actual resources is still plain old Terraform. Terragrunt provides an additional level of tooling that allows us to keep our code DRY and deploy things a bit more efficiently.</p><p>Let’s break it down into pieces.</p><h4>Terraform Modules</h4><p>When you write Terraform it is important to try to think in a modular way. Each module should be re-usable and should make use of variables and outputs so that it can be configurable. It is a way of enforcing certain standards within your team or organisation.</p><p>In this project you’ll find all of the Terraform code in the modules folder. This is mainly for simplicity of giving you the sample code for this article.</p><p>If your company is more mature and has existing Terraform modules then an alternative option would be to keep them in a separate repository. This forms a kind of Terraform library. On the extreme end you could store each Terraform module in a separate repository. Both Terraform and Terragrunt can reference Terraform modules directly from Github and other providers. One of the reasons for having separate repositories, other than to avoid code duplication, is that you can version your modules using git tags and reference them accordingly in your Terraform/Terragrunt code as remote sources. Other developers can then simply reference the module.</p><p>At the end of the day you should strive for re-use, but it’s about how far do you want to go. A good balance would be to have the most specific modules required for this project in this repository, and any generic modules in a separate repository to act as a Terraform module library.</p><p>The key point is that all Terraform should go into the modules folder, because, as well see shortly, referencing and instantiating these modules and deploying them into the correct account will be a job for Terragrunt.</p><p>You can think of the Terraform modules as <em>classes </em>in programming and the Terragrunt modules as <em>instances</em>. For example, you can create an S3 bucket Terraform module (class) and create as many Terragrunt modules (instances) based on this Terraform module as you would with a class.</p><h4>Account folder structure</h4><p>Let’s now have a look at the accounts folder structure, which in fact also has something to do with how Terragrunt works.</p><p>The first level of folders is arranged as a folder per region and one for global resources. The next level down is one folder per account. The next and final level down is one folder per module.</p><pre>accounts<br>  |- ap-southeast-2<br>  |    |- deployment<br>  |    |    |- _env<br>  |    |    |- artifacts-bucket<br>  |    |    |    &#39;- terragrunt.hcl<br>  |    |    |- codebuild-apply<br>  |    |    |    &#39;- terragrunt.hcl<br>  |    |    |- codebuild-plan<br>  |    |    |    &#39;- terragrunt.hcl<br>  |    |    |- codecommit<br>  |    |    |    &#39;- terragrunt.hcl<br>  |    |    |- codepipeline<br>  |    |    |    &#39;- terragrunt.hcl<br>  |    |    |- logging-bucket<br>  |    |    |    &#39;- terragrunt.hcl<br>  |    |    |- random-pipeline-prefix<br>  |    |    |    &#39;- terragrunt.hcl<br>  |    |    &#39;- account.hcl<br>  |    |- development<br>  |    |- logging<br>  |    |- management<br>  |    |- security<br>  |    |- shared-services<br>  |    &#39;- region.hcl<br>  |- global<br>  |    |- deployment<br>  |    |- development<br>  |    |- logging<br>  |    |- management<br>  |    |- security<br>  |    |- shared-services<br>  |    &#39;- region.hcl<br>  &#39;- us-east-1<br>       |- management<br>       &#39;- region.hcl</pre><p><strong><em>Note 1:</em></strong><em> In the above folder structure I’ve only expanded the </em><em>ap-southeast-2/deployment folders. Have a look at Github if you want to see the structure of everything.</em></p><p><strong><em>Note 2:</em></strong><em> I’ve chosen a region/account/module structure, but it would also be perfectly valid if you wanted to go for an account/region/module structure.</em></p><p><strong><em>Note 3:</em></strong><em> The _env folder is where you can place a common Terragrunt module definition with inputs that are the same across multiple modules. You can then include this using the </em><em>include statement in each relevant module. See the buckets in this project for an example.</em></p><p>This folder structure means you can create as many Terragrunt module folders as you need in whatever region and account that you want. The CI/CD pipeline will find them all.</p><h4>Terragrunt modules</h4><p>When running Terragrunt it can either be run individually from within the folder where each terragrunt.hcl file is located or from the accounts folder using the run-all command. The second option will recursively find all terragrunt.hcl files and run the plan or apply command as specified.</p><p>In each terragrunt.hcl file you’ll find the following piece of code.</p><pre>include &quot;root&quot; {<br>  path = find_in_parent_folders()<br>}</pre><p>What this does it starts going up the folder chain to find a parent terragrunt.hcl file. The one that it will find is the src/terragrunt.hcl file. This is where a lot of the magic happens, but this is also where it gets very confusing if you’re not familar with Terragrunt. Let’s call this the <em>parent module</em>.</p><p>What’s important to note is that Terragrunt runs all commands <strong>relative to the path of the terragrunt.hcl file</strong>. In other words, all the commands in the parent terragrunt.hcl module are actually run <strong>relative to the terragrunt.hcl</strong> module in each individual module folder.</p><p>The primary purpose of the parent terragrunt.hcl file is to keep your Terragrunt code DRY. There are the key things that this parent module does:</p><ol><li>Loads the variables defined in common_vars.yaml, region.hcl, account.hcl and accouns.json files, merges them and then passes them onto Terraform as inputs.</li><li>Generate the provider block file — this is what causes Terraform to assume the TerraformDeployment role in the correct account to ensure the resources are deployed into the correct account.</li><li>Generate the remote_state block file — this is what allows Terraform to assume the TerraformAdministrator role in the Deployment account for managing Terraform remote state in S3.</li></ol><h4>Loading input files</h4><p>When using Terraform it is often useful to load some static configuration parameters using external files that are maintained as part of the project source code.</p><p>The parent Terragrunt module can use the find_in_parent_folder function, but as was discussed previously, this will start searching from the <em>child’s</em> module folder <em>up</em> the folder hierarchy until it finds the specified file.</p><p>In the case of this project there are four things that it loads:</p><pre>common_vars = yamldecode(file(find_in_parent_folders(&quot;common_vars.yaml&quot;)))<br>  <br>region_vars  = read_terragrunt_config(find_in_parent_folders(&quot;region.hcl&quot;, &quot;ignore&quot;), {locals = { aws_region = local.common_vars.management_aws_region}})\<br>account_vars = read_terragrunt_config(find_in_parent_folders(&quot;account.hcl&quot;), {})<br><br>account_ids = jsondecode(file(&quot;accounts.json&quot;))</pre><p>In common_vars.yaml you’ll find global project configuration parameters. As mentioned earlier in this article, this is where you configure your project specific parameters.</p><p>In region.hcl you’ll find something like:</p><pre>locals {<br>  aws_region  = &quot;ap-southeast-2&quot;<br>  environment = &quot;ap-southeast-2&quot;<br>}</pre><p>In account.hcl you’ll find something like:</p><pre>locals {<br>  stage  = &quot;deployment&quot;<br>  account_name = &quot;deployment&quot;<br>}</pre><p>It is important to note that because of the way Terragrunt searches for files, the account.hcl and region.hcl files will be found relative to the child module. So if the child module is in the ap-southeast-2 folder then that’s the values you’ll get. If it’s in the us-east-1 folder, then you’ll get the region.hcl file specific to us-east-1. Similarly for the account.hcl file.</p><p>The magic here is that your child module will always get the correct input region, stage and account name as long as it is placed in the correct sub-folder in the folder structure that represents the region and account you want those resources to be deployed in.</p><p>The last file is more for convenience, because in some cases you need to know the account ID, like for example, to build ARN strings. This accounts.json file is something that the deploy.sh script will generate for you when it creates your accounts.</p><pre>{<br>  &quot;deployment&quot;: &quot;111111111111&quot;,<br>  &quot;development&quot;: &quot;222222222222&quot;,<br>  &quot;logging&quot;: &quot;3333333333333&quot;,<br>  &quot;management&quot;: &quot;444444444444&quot;,<br>  &quot;security&quot;: &quot;555555555555&quot;,<br>  &quot;shared-services&quot;: &quot;666666666666&quot;<br>}</pre><p>Now you know where aws_region, account_name and account_idscome from. These can be referenced as follows:</p><pre>${local.region_vars.locals.aws_region}<br>${local.account_ids[local.account_vars.locals.account_name]}</pre><p>Pretty neat!</p><p>The final piece of the puzzle is to pass the inputs, or a subset of inputs, to Terraform as input variables. This is achieved using the merge function.</p><pre>inputs = merge(<br>  local.common_vars,<br>  local.account_vars.locals,<br>  local.region_vars.locals,<br>  { terraform_state_bucket_name = local.terraform_state_bucket_name }<br>)</pre><p>Remember that if you are loading a Terragrunt file (like region.hcl or account.hcl) using read_terragrunt_config you’ll need to use .locals to get an object that represents all of the variable values in that file. If you are merging a single variable, you’ll need to use object notation like the terraform_state_bucket_name variable shown above.</p><h4>Terraform provider</h4><p>Let’s move on to providers and how these are generated.</p><p>The following code generates the provider.tf file. We saw in the previous section how the various parameters are loaded.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d1574101d7d97b93f78aea3438bbff24/href">https://medium.com/media/d1574101d7d97b93f78aea3438bbff24/href</a></iframe><p>The assume_role block is built dynamically using the account and region parameters loaded in the locals section so that we get the TerraformDeployment role in the correct account based on the module currently being deployed by the run-all command.</p><p>The role_arn is built using the account name loaded from account.hcl and the accounts.json file to look up the account ID using the account name.</p><p>The region is determined from region.hcl.</p><p>You’ll note that these is a second provider with an alias noassume. This can be used where you want to do something without assuming the role of the target account, but rather use the underlying role of the CodeBuild project that is currently executing (which in the case of this project is always the Deployment account). You might use this provider in Terraform to read Terraform state data using data blocks.</p><pre>data &quot;aws_s3_object&quot; &quot;some_s3_bucket_data&quot; {<br>  bucket = &quot;some-s3-bucket-name&quot;<br>  key    = &quot;some-file-key&quot;<br><br>  provider = &quot;noassume&quot;<br>}</pre><h4>Terraform remote state</h4><p>Finally, let’s have a look at remote state configuration.</p><p>The following code generates the backend.tf file.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a1ffae0ccb9a8e940aab100b9a4dbc76/href">https://medium.com/media/a1ffae0ccb9a8e940aab100b9a4dbc76/href</a></iframe><p>Remeber that as with the provider that is generated, the remote state is also generate for every module that is run by run-all. Each one gets an independent backend.tf file generated by the above generator using the parameters specified.</p><p>There’s two main note worthy things going on here.</p><p>The key is created by taking the path relative to the Terraform module. This means the remote state stored in S3 will automatically end up with nicely separate paths for each region and account.</p><p>The role_arn is populated either by the TF_AWS_ACCT environment variable when using the deploy.sh script, or if not found, by the get_aws_account_id() function which will automatically grab the Deployment account ID when running in CI/CD as that’s the account context from which the CodeBuild projects are run. This ensures that role_arn is correct for both the script or the CI/CD method without having to change anything.</p><h3>Destroying this project</h3><p>Destroying the resources created in this project is not quite as easy as creating them, because there are a few factors involved. I’ve included a destroy.sh script, but there are a few things you need to do first before this will work.</p><p>This may sound obvious, but you should only destroy the resources in this project if you haven’t already used the accounts created for other deployments and projects!</p><p>The first issue is that AWS Organization and the member accounts are marked with a lifecycle policy where prevent_destroy = true. This will cause Terraform to throw an error if you try to destroy these resources. This is good in a real deployment, but a bit annoying if you just want to play around with the code in this article.</p><p>To get ready to destroy these resources you’ll have to make some changes and then apply those using the deploy.sh script. The first change is remove the lifecycle block from all of the relevant resources. The second change is add close_on_deletion = true in all of the aws_organizations_account resources to ensure deleting the account also results in its closure.</p><p>Another thing to note is that the AWS Organization quota will only allow you to close a maximum of 10 accounts, but this should be ok for this project. The bigger issue is only 3 accounts may be closed concurrently, so you may still end up having to login to your Management account and then remove those accounts in AWS Organizations manually.</p><p>Finally, before you can use the destroy.sh script, you’ll need the terraform-init user and access keys for this user. If you removed this user, or no longer have the access keys, you’ll need to create this user and get new access keys from the Management account.</p><p>With all this in place you can try to use the destroy.sh script. It’s hard to test for all possible deployments, but it essentially runs everything in reverse and doesn’t bother with local state.</p><p>Good luck, in the worst case you’ll have to manually delete the accounts.</p><h3>Conclusion</h3><p>This was quite a tricky article to put together so my apologies if something doesn’t work right out of the box. I tested a million times, but it’s not easy to test account creation without then having to destroy these accounts.</p><p>I hope you learned something new and had some fun along the way. Multi-account and multi-region deployments can take many forms so I’m sure you will continue learning on your own journey.</p><p>Cheers!</p><h3>Level Up Coding</h3><p>Thanks for being a part of our community! Before you go:</p><ul><li>👏 Clap for the story and follow the author 👉</li><li>📰 View more content in the <a href="https://levelup.gitconnected.com/?utm_source=pub&amp;utm_medium=post">Level Up Coding publication</a></li><li>💰 Free coding interview course ⇒ <a href="https://skilled.dev/?utm_source=luc&amp;utm_medium=article">View Course</a></li><li>🔔 Follow us: <a href="https://twitter.com/gitconnected">Twitter</a> | <a href="https://www.linkedin.com/company/gitconnected">LinkedIn</a> | <a href="https://newsletter.levelup.dev/">Newsletter</a></li></ul><p>🚀👉 <a href="https://jobs.levelup.dev/talent/welcome?referral=true"><strong>Join the Level Up talent collective and find an amazing job</strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=39aeed097ad2" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/aws-multi-account-multi-region-bootstrapping-with-terraform-39aeed097ad2">AWS Multi-account Multi-region Bootstrapping with Terraform</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using Terragrunt run-all in AWS CI/CD]]></title>
            <link>https://levelup.gitconnected.com/using-terragrunt-run-all-in-aws-ci-cd-df2877bad198?source=rss-159faff569ab------2</link>
            <guid isPermaLink="false">https://medium.com/p/df2877bad198</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[aws-codepipeline]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[terragrunt]]></category>
            <dc:creator><![CDATA[Oliver Schenk]]></dc:creator>
            <pubDate>Tue, 11 Apr 2023 08:11:45 GMT</pubDate>
            <atom:updated>2023-04-12T13:34:11.931Z</atom:updated>
            <cc:license>http://creativecommons.org/publicdomain/zero/1.0/</cc:license>
            <content:encoded><![CDATA[<p>This guide describes a solution for detecting modules that were deleted and applying the change with <strong>terragrunt run-all</strong><em> </em>in a CI/CD pipeline using CodePipeline and CodeBuild.</p><p>You’ll find the <a href="https://github.com/oliverschenk/terragrunt-run-all">source code on Github</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*d3cgdfBemOjSa3c9" /><figcaption>Photo by <a href="https://unsplash.com/@wocintechchat?utm_source=medium&amp;utm_medium=referral">Christina @ wocintechchat.com</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>The problem</h3><p>When creating or updating AWS resources in Terraform, or creating new Terragrunt modules, change detection will ensure that the plan and apply commands automatically detect and apply the appropriate change.</p><p>Everything works fine as long as a module physically exists, as it will always be able to compare the previous state to the new state. You can edit the underlying Terraform code with no major issues.</p><p>The problem comes along when an entire Terragrunt module is deleted. In other words, if a <em>terragrunt.hcl </em>file, or the folder that contains it, is deleted you might instinctively think that this resource will be destroyed.</p><p>This is not the case.</p><p>What will happen is the Terragrunt run-all command will simply skip over it as it no longer sees it. In a way this makes sense, because each Terragrunt module (a folder with a terragrunt.hcl file) is independent to any other module.</p><p>The run-all command scans for modules and then builds a dependency tree to determine how to group the executions and runs these in a parallel fashion. This command does not know whether a module did or didn’t exist previously. If you remove a module it simply won’t be actioned and your resources will remain.</p><p>The argument is that a major change as drastic as destroying a resource shouldn’t be magically handled by the apply command. The destruction of resources should be a conscious and deliberate act by a developer and should be actioned by a CI/CD pipeline in a similarly deliberate way. This also provides a level of auditability.</p><h3>Possible solutions</h3><p>Here are some possible solutions to detect if a module needs to be destroyed:</p><ol><li><strong>Create a reusable Terraform module</strong> using the null_resource resource that doesn’t actually create any resources, but ensures that you don’t delete your terragrunt.hcl module. The problem is that whilst this will destroy your resources in AWS, you’ll still have this null_resource resource hanging around in your code and state.</li><li>To <strong>flag that a module needs to be destroyed</strong>, create a file such as .destroy in the folder where the terragrunt.hcl file is located and commit this into your repo. Don’t delete the actual Terragrunt module yet. The CI/CD pipeline build process will detect the .destroy file and run the destroy command. In a future commit it is then safe to delete the folder containing both the .destroy and terragrunt.hcl file.</li><li>More sophisticated CI/CD tools can <strong>use tags or notes in commit or pull request comments</strong> to determine whether some special actions should be taken. This relies on a good CI/CD tool and good development practices with a clearly defined commit and/or pull request process.</li></ol><p>In this guide I’ll describe a solution using <strong>Option 2</strong>.</p><h3>Making a change</h3><p>The following lists the possible changes that you could make to change a resource. Once a change is made you would commit this into the source repository so that it gets picked up by the CI/CD pipeline.</p><h4>Creating a resource</h4><p>When you are creating a new resource you create the appropriately named folder in the region/account folder where you want the resource to be deployed and add a terragrunt.hcl file into it. This then references either a new or existing Terraform module.</p><h4>Updating a resource</h4><p>When you are updating a resource you either update the Terraform code in the Terraform module referenced by the terragrunt.hcl file or you change some inputs or dependencies in the terragrunt.hcl file.</p><h4>Deleting a resource</h4><p>Deleting a resource means deleting a whole Terragrunt module. To do this you create a new empty file named .destroy inside the Terragrunt module folder you want to destroy. It should sit next to the terragrunt.hcl file.</p><p>In the following example we want the test-string module to be destroyed so a .destroy file was added to this folder.</p><pre>infrastructure<br>|- ...<br>&#39;- test-string<br>     |- .destroy        &lt;--- add a .destroy file<br>     &#39;- terragrunt.hcl</pre><h3>Implementation</h3><p>In this section I’ll step through a sample CI/CD pipeline and the implementation of the buildspec for detecting and actioning destroy operations using the proposed solution.</p><h4>Infrastructure</h4><p>The infrastructure deployed in this implementation consists of the following:</p><ul><li>Artifacts bucket for CodePipeline artifacts</li><li>Logging bucket for CodeBuild logs</li><li>TerraformAdministratorRole that can be assumed by CodeBuild for managing Terraform state and locking</li><li>TerraformDeploymentRole that can be assumed by CodeBuild to deploy infrastructure into the specified account</li><li>CodeCommit repository for source control</li><li>CodeBuild project for the planning stage</li><li>CodeBuild project for the apply stage</li><li>CodePipeline for orchestration of the plan, approval and apply stages, including an event bridge rule to detect CodeCommit changes and SNS for build approval notification</li><li>An SSM Parameter for testing the destroy functionality</li></ul><p>Note that the sample code contains the Terraform module for the SSM parameter, but does not create an instance of it using Terragrunt. This article will walk through creating and destroying such an instance in the <strong>Testing</strong> section.</p><h4>CodeBuild plan phase</h4><p>The <em>plan</em> phase of the build process is responsible for running plan on your modules.</p><p>We first want to plan the modules to be destroyed. The trick here is to find all folders that contain a .destroy file and then generate a --terragrunt-include-dir parameter for each folder that was found so that only those folders are included.</p><p>If any modules were found that need to be destroyed run the terragrunt run-all plan command with the -destroy flag. This will then only include the folders provided by the $tg_modules_to_destroy string. Using the run-all command has the benefit of taking into account dependencies when if multiple things are being destroyed at the same time.</p><pre># Find all Terragrunt modules to destroy<br>- tg_modules_to_destroy=$(find . -maxdepth 4 -name terragrunt.hcl -execdir [ -e .destroy ] \; -printf &quot; --terragrunt-include-dir %h&quot;)<br><br># Plan to destroy modules no longer needed<br>- |<br>  if [[ -n &quot;${tg_modules_to_destroy}&quot; ]]; then<br>    terragrunt run-all plan $tg_modules_to_destroy -destroy -no-color<br>  fi</pre><p>Now we can do a similar thing for the folders that don’t contain a .destroy file.</p><pre># Find all Terragrunt modules that do not contain a .destroy file<br>- tg_modules=$(find . -maxdepth 4 -name terragrunt.hcl -execdir [ ! -e .destroy ] \; -printf &quot; --terragrunt-include-dir %h&quot;)<br><br># Plan all changes<br>- |<br>  if [[ -n &quot;${tg_modules}&quot; ]]; then<br>    terragrunt run-all plan $tg_modules -no-color<br>  fi</pre><h4>CodeBuild apply phase</h4><p>The <em>apply</em> phase of the build process is responsible for running apply on your modules. The code is pretty much identical.</p><pre># Find all Terragrunt modules to destroy<br>- tg_modules_to_destroy=$(find . -maxdepth 4 -name terragrunt.hcl -execdir [ -e .destroy ] \; -printf &quot; --terragrunt-include-dir %h&quot;)<br><br># Destroy modules no longer needed<br>- |<br>  if [[ -n &quot;${tg_modules_to_destroy}&quot; ]]; then<br>    terragrunt run-all apply $tg_modules_to_destroy -destroy -no-color<br>  fi<br><br># Find all Terragrunt modules that do not contain a .destroy file<br>- tg_modules=$(find . -maxdepth 4 -name terragrunt.hcl -execdir [ ! -e .destroy ] \; -printf &quot; --terragrunt-include-dir %h&quot;)<br><br># Apply all changes<br>- |<br>  if [[ -n &quot;${tg_modules}&quot; ]]; then<br>    terragrunt run-all apply $tg_modules -no-color<br>  fi</pre><h4>Why separate CodeBuild projects?</h4><p>Just a short note on why there are separate CodeBuild projects for plan and apply stages when they are pretty much identical.</p><p>The main argument against creating a single CodeBuild project is that the plan and apply phases are quite different even though this project happens to be quite simple. In a more advanced pipeline you would have all sorts of artifacts being created in the planning stage that potentially visualise your Terraform changes and perform testing of some kind. I personally recommend keeping the plan and apply stages separate.</p><p>You can still build a reusable CodeBuild module in Terraform even though there will be two Terragrunt instances of this module with different input parameters.</p><h3>Testing</h3><p>Let’s run through an example. Once you have cloned <a href="https://github.com/oliverschenk/terragrunt-run-all">the repository</a> open up the folder in your favourite CLI and code editor.</p><h4>Pre-requisites</h4><p>This project was developed in a Ubuntu environment using WSL.</p><p>To run the examples in this section your environment will need:</p><ul><li>AWS CLI</li><li>An AWS Account</li><li>Credentials for the above account with Admin rights either through access keys or SSO (see article on <a href="https://medium.com/gitconnected/aws-single-sign-on-terraform-and-terragrunt-a8c22bb7cfa8">SSO with Terraform and Terragrunt</a>)</li><li>Terragrunt</li><li>Terraform</li></ul><h4>Configuration</h4><p>You should edit the common_vars.yaml file and adjust the parameters to suit your own environment. The main parameters of interest are the deployment_account_id and the build_notification_email, as well as the namespace, name, and environment (region).</p><p>The resources will be deployed into the account specified by deployment_account_id.</p><p>It is recommended to change the namespace and name to avoid bucket naming collisions.</p><h4>Deployment</h4><p>To deploy the resources for the first time run the deploy.sh script without any options. You may need to chmod +x *.sh if your script files do not have execute permissions.</p><p>The script will first use local Terraform state. It copies the terragrunt-local.hcl file to terragrunt.hcl and then uses the run-all command to first init and then apply the resources.</p><pre>if [[ -z &quot;${SKIP_LOCAL_STATE}&quot; ]]; then<br>    echo &quot;=== RUNNING DEPLOYMENT WITH LOCAL STATE ===&quot;<br><br>    echo &quot;Putting in place temporary local state terraform module&quot;<br>    rm -f terragrunt.hcl<br>    cp terragrunt-local.hcl terragrunt.hcl<br>else<br>    echo &quot;Putting in place remote state terraform module&quot;<br>    rm -f terragrunt.hcl<br>    cp terragrunt-remote.hcl terragrunt.hcl<br>fi<br><br>pushd ./infrastructure<br><br>echo &quot;Running terragrunt init&quot;<br>terragrunt run-all init<br>echo &quot;Running terragrunt apply&quot;<br>terragrunt run-all apply</pre><p>Once this is done, it will copy the terragrunt-remote.hcl file to terragrunt.hcl so that the remote state can be initialised. Due to the parallel nature of the run-all command we can’t use this for initialising the remote state otherwise it will try to create multiple copies of the S3 bucket and DynamoDB table. Instead, the script will go to one of the modules and initialise the state there first. It will then go back and run the run-all command.</p><pre>if [[ -z &quot;${SKIP_LOCAL_STATE}&quot; ]]; then<br><br>    popd<br>    <br>    echo &quot;Putting in place remote state terraform module&quot;<br>    rm -f terragrunt.hcl<br>    cp terragrunt-remote.hcl terragrunt.hcl<br><br>    pushd ./infrastructure<br><br>    # Here we initialise remote state only in one module so that the<br>    # relevant S3 bucket and DynamoDB table is created. This is because<br>    # Terragrunt run-all runs tasks in parallel and would therefore<br>    # attempt to create more than one remote state and error out<br>    echo &quot;Initialising remote S3 state&quot;<br>    pushd ./artifacts-bucket<br>    terragrunt init -force-copy --terragrunt-non-interactive<br>    popd<br><br>    # Now that the S3 remote state bucket already exists, use it<br>    # to initialise the remaining modules<br>    terragrunt run-all init -force-copy --terragrunt-non-interactive<br><br>fi</pre><p>Once done you’ll see the name of the CodeCommit repository printed out.</p><pre>Stored CodeCommit repository name: abc-tg-run-all-ap-southeast-2-repository</pre><p>You should also have received an email where you can confirm your subscription to receive build notifications via SNS. You’ll get an email every time the CI/CD pipeline is at the Approval stage waiting for your approval before proceeding.</p><p>The last part of the script clones your CodeCommit repository into a separate folder, copies the project code into it, then uses git to add, commit and push the code into CodeCommit and finally deletes the repository folder it created. At this point the first commit was made into CodeCommit and the CI/CD pipeline will have been activated.</p><h4>Cloning the CodeCommit repository</h4><p>Now that the CodeCommit repository has been created we need to switch over to using that code base. I’m just going to clone it into a new folder. You’ll need to have the <a href="https://docs.aws.amazon.com/codecommit/latest/userguide/setting-up-git-remote-codecommit.html">git-remote-codecommit module</a> installed.</p><pre>git clone codecommit://abc-tg-run-all-ap-southeast-2-repository</pre><h4>Creating a test SSM parameter</h4><p>Let’s use a Terraform module in the modules folder called ssm-string-parameter. This is simply a reusable Terraform module that creates an SSM parameter.</p><pre>resource &quot;aws_ssm_parameter&quot; &quot;this&quot; {<br>  name  = &quot;${module.this.id}-${var.parameter_name}&quot;<br>  type  = &quot;String&quot;<br>  value = var.parameter_value<br><br>  tags = module.this.tags<br>}</pre><p>Now let’s create an instance of this module by creating a test-parameter folder in the infrastructure folder. Inside that we’ll place the terragrunt.hcl module to contain an instance of the Terraform module.</p><pre>include &quot;root&quot; {<br>  path = find_in_parent_folders()<br>}<br><br>terraform {<br>  source = &quot;../../modules//ssm-string-parameter&quot;<br>}<br><br>inputs = {<br>  parameter_name  = &quot;test-parameter-1&quot;<br>  parameter_value = &quot;foo bar&quot;<br>}</pre><p>You can now commit and push these changes. The CodePipeline project should automatically start building and you’ll soon receive a build approval notification.</p><p>Let’s check that the parameter was created.</p><pre>aws ssm get-parameter --name abc-ap-southeast-2-tg-run-all-test-parameter-1<br>{<br>    &quot;Parameter&quot;: {<br>        &quot;Name&quot;: &quot;abc-ap-southeast-2-tg-run-all-test-parameter-1&quot;,<br>        &quot;Type&quot;: &quot;String&quot;,<br>        &quot;Value&quot;: &quot;foo bar&quot;,<br>        &quot;Version&quot;: 1,<br>        &quot;LastModifiedDate&quot;: &quot;2023-04-11T13:53:10.731000+08:00&quot;,<br>        &quot;ARN&quot;: &quot;arn:aws:ssm:ap-southeast-2:111111111111:parameter/abc-ap-southeast-2-tg-run-all-test-parameter-1&quot;,<br>        &quot;DataType&quot;: &quot;text&quot;<br>    }<br>}</pre><h4>Destroying the test SSM parameter</h4><p>Now comes the important bit in our CI/CD pipeline — testing that the destroy functionality works as expected. To destroy the SSM parameter that was created previously, let’s add the required .destroy file to flag that this module should be destroyed.</p><p>Create a new empty file in the test-parameter folder next to the terrragrunt.hcl file and name it .destroy.</p><p>Push these changes to CodeCommit. Again, the pipeline will execute and you’ll need to approve the change. However, before you do, you can check the CodeBuild plan log output. You should see a plan to destroy the parameter.</p><pre>[Container] 2023/04/11 05:57:57 Running command tg_modules_to_destroy=$(find . -maxdepth 4 -name terragrunt.hcl -execdir [ -e .destroy ] \; -printf &quot; --terragrunt-include-dir %h&quot;)<br><br>[Container] 2023/04/11 05:57:57 Running command if [[ -n &quot;${tg_modules_to_destroy}&quot; ]]; then<br>  terragrunt run-all plan $tg_modules_to_destroy -destroy -no-color<br>fi<br><br>time=2023-04-11T05:57:57Z level=info msg=The stack at /codebuild/output/src067178559/src/src/infrastructure will be processed in the following order for command plan:<br>Group 1<br>- Module /codebuild/output/src067178559/src/src/infrastructure/test-parameter<br><br><br>aws_ssm_parameter.this: Refreshing state... [id=abc-ap-southeast-2-tg-run-all-test-parameter-1]<br><br>Terraform used the selected providers to generate the following execution<br>plan. Resource actions are indicated with the following symbols:<br>  - destroy<br><br>Terraform will perform the following actions:<br><br>  # aws_ssm_parameter.this will be destroyed<br>  - resource &quot;aws_ssm_parameter&quot; &quot;this&quot; {<br>      - arn       = &quot;arn:aws:ssm:ap-southeast-2:111111111111:parameter/abc-ap-southeast-2-tg-run-all-test-parameter-1&quot; -&gt; null<br>      - data_type = &quot;text&quot; -&gt; null<br>      - id        = &quot;abc-ap-southeast-2-tg-run-all-test-parameter-1&quot; -&gt; null<br>      - name      = &quot;abc-ap-southeast-2-tg-run-all-test-parameter-1&quot; -&gt; null<br>      - tags      = {<br>          - &quot;Environment&quot; = &quot;ap-southeast-2&quot;<br>          - &quot;Name&quot;        = &quot;abc-ap-southeast-2-tg-run-all&quot;<br>          - &quot;Namespace&quot;   = &quot;abc&quot;<br>        } -&gt; null<br>      - tags_all  = {<br>          - &quot;Environment&quot; = &quot;ap-southeast-2&quot;<br>          - &quot;Name&quot;        = &quot;abc-ap-southeast-2-tg-run-all&quot;<br>          - &quot;Namespace&quot;   = &quot;abc&quot;<br>        } -&gt; null<br>      - tier      = &quot;Standard&quot; -&gt; null<br>      - type      = &quot;String&quot; -&gt; null<br>      - value     = (sensitive value) -&gt; null<br>      - version   = 1 -&gt; null<br>    }<br><br>Plan: 0 to add, 0 to change, 1 to destroy.<br><br>Changes to Outputs:<br>  - arn = &quot;arn:aws:ssm:ap-southeast-2:111111111111:parameter/abc-ap-southeast-2-tg-run-all-test-parameter-1&quot; -&gt; null</pre><p>After approving and change and waiting for the pipeline to complete, you should no longer be able to access the parameter.</p><pre>aws ssm get-parameter --name abc-ap-southeast-2-tg-run-all-test-parameter-1<br><br>An error occurred (ParameterNotFound) when calling the GetParameter operation:</pre><p>And there you have it!</p><p>The test-parameter folder can now be safely removed and pushed at the next opportunity. You can leave the test-parameter folder in place as long as the .destroy file remains, it will just slow down the pipeline a bit. If you were to remove the .destroy file then the resources would get created again.</p><h3>Clean-up</h3><p>To destroy all of the resources created in this project you can use the destroy.sh script. This will destroy all of the resources including the Terraform state S3 bucket and DynamoDB table.</p><p>This script copies the terragrunt-local.hcl file to terragrunt.hcl and then executes the run-all init command to migrate the state back to local state and then runs the run-all destroy command.</p><p>The reason local state has to be used is because the roles that were used by the CI/CD pipeline (like TerraformAdministratorRole) will now be destroyed so we can’t use them anymore.</p><p>This script also removes the remote state resources.</p><h3>Thanks!</h3><p>I hope this guide on how to create CI/CD functionality for creating and destroying resources when using Terragrunt has been useful. Cheers!</p><h3>Level Up Coding</h3><p>Thanks for being a part of our community! Before you go:</p><ul><li>👏 Clap for the story and follow the author 👉</li><li>📰 View more content in the <a href="https://levelup.gitconnected.com/?utm_source=pub&amp;utm_medium=post">Level Up Coding publication</a></li><li>💰 Free coding interview course ⇒ <a href="https://skilled.dev/?utm_source=luc&amp;utm_medium=article">View Course</a></li><li>🔔 Follow us: <a href="https://twitter.com/gitconnected">Twitter</a> | <a href="https://www.linkedin.com/company/gitconnected">LinkedIn</a> | <a href="https://newsletter.levelup.dev/">Newsletter</a></li></ul><p>🚀👉 <a href="https://jobs.levelup.dev/talent/welcome?referral=true"><strong>Join the Level Up talent collective and find an amazing job</strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=df2877bad198" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/using-terragrunt-run-all-in-aws-ci-cd-df2877bad198">Using Terragrunt run-all in AWS CI/CD</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cognito Lambda Triggers in Dotnet — PostConfirmation]]></title>
            <link>https://levelup.gitconnected.com/cognito-lambda-triggers-in-dotnet-postconfirmation-f1419f4d4952?source=rss-159faff569ab------2</link>
            <guid isPermaLink="false">https://medium.com/p/f1419f4d4952</guid>
            <category><![CDATA[cognito-user-pools]]></category>
            <category><![CDATA[cognito]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[aws-lambda-trigger]]></category>
            <dc:creator><![CDATA[Oliver Schenk]]></dc:creator>
            <pubDate>Thu, 30 Mar 2023 03:46:43 GMT</pubDate>
            <atom:updated>2023-03-30T15:14:17.919Z</atom:updated>
            <cc:license>http://creativecommons.org/licenses/by/4.0/</cc:license>
            <content:encoded><![CDATA[<h3>Cognito Lambda Triggers in Dotnet — PostConfirmation</h3><p>This article is part of the <strong>Cognito Lambda Triggers in Dotnet</strong> series, dedicated to understanding and handling Cognito Lambda Triggers.</p><p>If you want to learn about Lambda triggers and how to architect a Lambda function to handle various trigger sources more generally, refer to the related parent article <a href="https://medium.com/@oliver.schenk/cognito-lambda-triggers-in-dotnet-3bf13a55eda3">Cognito Lambda Triggers in Dotnet</a>. If you head over there you’ll also find links to handle other trigger sources.</p><p>In the following sections we’ll look at the <strong>PostConfirmation</strong> trigger source. You’ll find the technical details in the <a href="https://docs.aws.amazon.com/cognito/latest/developerguide/user-pool-lambda-post-confirmation.html">Cognito documentation</a>, but what is often more important is to understand why it exists and why you might want to use it.</p><p>Here you’ll find an explanation with use cases, payload examples, handler implementations and test scripts.</p><p>You’ll find the <a href="https://github.com/oliverschenk/cognito-lambda-triggers-dotnet">source code on Github</a>. This will be updated with more examples over time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*yQFjdguHtKm8dbEa" /><figcaption>Photo by <a href="https://unsplash.com/@matthewhenry?utm_source=medium&amp;utm_medium=referral">Matthew Henry</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>PostConfirmation</h3><p>When a user signs up or requests a password reset, Cognito will send a confirmation code to the user’s email or phone number (depending on the configuration). Cognito requires this code to be provided thus confirming the user owns the email address or phone number configured in the account.</p><p>These flows will trigger the relevant PostConfirmation_* Lambda handler once the correct code is provided.</p><ul><li><strong>PostConfirmation_ConfirmSignUp </strong>— This occurs when a user has confirmed their email or phone using the correct code that was sent to them following a sign-up request.</li><li><strong>PostConfirmation_ConfirmForgotPassword </strong>— This occurs when a user has provided the correct code that was sent to them and a new password following a password reset request.</li></ul><h3>Event Payload</h3><p>The event payload passed to your trigger handler function will consist of a set of <a href="https://medium.com/@oliver.schenk/cognito-lambda-triggers-in-dotnet-3bf13a55eda3#ddd6">common parameters</a> and a request and/or response property specific to the trigger source.</p><h4>Request</h4><p>The PostConfiguration request parameter consists of the following properties (in addition to the common parameters).</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4e0682dd67c531e7f87f8a0b1c7d6dc0/href">https://medium.com/media/4e0682dd67c531e7f87f8a0b1c7d6dc0/href</a></iframe><p>The ClientMetadata property is a key-value data value that is passed into the Lambda trigger handler from data that is included when making any of the following API calls: <a href="https://docs.aws.amazon.com/cognito-user-identity-pools/latest/APIReference/API_AdminConfirmSignUp.html">AdminConfirmSignUp</a>, <a href="https://docs.aws.amazon.com/cognito-user-identity-pools/latest/APIReference/API_ConfirmForgotPassword.html">ConfirmForgotPassword</a>, <a href="https://docs.aws.amazon.com/cognito-user-identity-pools/latest/APIReference/API_ConfirmSignUp.html">ConfirmSignUp</a>, and <a href="https://docs.aws.amazon.com/cognito-user-identity-pools/latest/APIReference/API_SignUp.html">SignUp</a>.</p><p>The purpose of ClientMetadata property is to allow the client to pass further information to the PostConfirmation function. This extra information can be used by your function.</p><p>Because this data comes from the client side, from a security point of view, this data can only really be relied upon when the sign-up is done via API calls in a secured backend environment rather than via a public client.</p><p>One way to use the metadata is to pass information from the client to the backend and then reuse that information in another trigger handler.</p><h4>Response</h4><p>No additional properties are returned in the PostConfirmation response parameter.</p><h3>Use Cases</h3><p>The purpose of this trigger is to perform any necessary custom tasks <em>after</em> a user either confirms their account after signing up or after a user resets their password after requesting a password reset. You can think of this function as a kind of side effect.</p><p>Here are some examples of what you could do:</p><ul><li>If you want to store user information in a database you can use this trigger to create that user entity in a database or data store. Your system might then ask for further information once that user signs in for the first time into your application.</li><li>Once a user has been confirmed, your system can send an email or SMS to the user with information about sign-in or next steps, or simply provide a welcome message.</li><li>In the case of resetting a forgotten password, you can use this trigger to send an email to the user telling them that their password has been reset and asking them whether they requested this reset.</li><li>If your system uses any external systems or other services within your architecture you could send an event to your event bus (like EventBridge) or call an external API to communicate with an external system that needs knowledge of your user.</li><li>If you are using <a href="https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-user-pools-user-groups.html">user groups</a> in Cognito you can use this trigger to add the user to a specific group based on some custom logic.</li></ul><h3>Handler Examples</h3><p>Below you will find an example handler implementation for each PostConfirmation trigger source. The handler templates can be found in the <a href="https://github.com/oliverschenk/cognito-lambda-triggers-dotnet/tree/main/lambda-triggers/src/CognitoLambdaTriggers/Handlers">GitHub repository</a> along with some <a href="https://github.com/oliverschenk/cognito-lambda-triggers-dotnet/tree/main/lambda-triggers/test/payloads">example event payloads</a>.</p><h4>PostConfirmation_ConfirmSignUp</h4><p>In this example, after the user confirms their email address using a code, the Lambda handler will insert the user’s details into a DynamoDB table.</p><p>The following snippet shows an example event payload for the Lambda function after the user provides the correct confirmation code.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fc9efc7f1ee88d70a1a4df041a628896/href">https://medium.com/media/fc9efc7f1ee88d70a1a4df041a628896/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c5f36287039c3849c6b8c1edac17116b/href">https://medium.com/media/c5f36287039c3849c6b8c1edac17116b/href</a></iframe><p>You’ll find the relevant DynamoDB configuration and permissions in the dynamodb.tf Terraform code.</p><p>You’ll note that I made use of the <a href="https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DotNetSDKHighLevel.html">DynamoDb Object Persistence Model</a> for modelling the DynamoDB Users table.</p><h4>PostConfirmation_ConfirmForgotPassword</h4><p>The example code for this handler implements sending a custom email using AWS SES to your user’s email address telling them that their password has been reset.</p><p>The following snippet shows an example event payload for the Lambda function after the user provides the correct confirmation code and a new password.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c2258469e3b650e0423516acc47f0b06/href">https://medium.com/media/c2258469e3b650e0423516acc47f0b06/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e096c15a772bde5c1ed784a3756fedff/href">https://medium.com/media/e096c15a772bde5c1ed784a3756fedff/href</a></iframe><p>The source email address is the one you supplied when deploying the Terraform code. It is then stored as a parameter in AWS SSM and is fetched by the Lambda function using the SSM client. This demonstrates a way to avoid hard coding anything.</p><h3>Testing</h3><p>To simplify initial testing I have provided some sample scripts that can be used to create users and clean out users. Just be a bit careful with the script that deletes users, because it will literally delete all your users in the specified Cognito user pool!</p><ul><li><a href="https://github.com/oliverschenk/cognito-lambda-triggers-dotnet/blob/main/lambda-triggers/test/commands/create-user-signup.sh"><em>create-user-signup.sh </em></a>— creates a user using self-registration</li><li><a href="https://github.com/oliverschenk/cognito-lambda-triggers-dotnet/blob/main/lambda-triggers/test/commands/delete-users.sh"><em>delete-users.sh</em></a> — deletes all users in the given user pool</li></ul><p>In my samples below you’ll see I’ve put the user pool ID and user pool client ID into environment variables USER_POOL_ID and USER_POOL_CLIENT_ID respectively to avoid having to paste these values all the time.</p><h4>Comment out unused Lambda triggers in Terraform</h4><p>It would be recommended to comment out any Lambda triggers that you are not using in the Terraform code so that your user pool only triggers the handlers you are interested in.</p><p>If you leave some of the blank handlers set up you may get strange behaviour. For example you might not receive any emails if the custom sender Lambda functions are still enabled.</p><h4>Enabling Debug level logs</h4><p>To provide some additional debugging help you should make sure that the Lambda function has the AWS_LAMBDA_HANDLER_LOG_LEVEL environment variable configured and set to Debug. This will ensure that the Dotnet code outputs Debug level logging.</p><h4>ConfirmSignUp Handler</h4><p>To test this handler you’ll have to first create a new user and then confirm with a code.</p><p>You could create a new user using the aws cognito-idp sign-up CLI command, but to simplify things, I’ve created a create-user-signup.sh script to streamline creating a new random user.</p><p>To use the script pass in your user pool client ID followed by an email address. The script will automatically generate a random alias using the + symbol. In email world this will still end up at your base email address. Everything else is randomly generated.</p><p>This script does not need any AWS credentials, because it is meant to be called by unauthenticated users.</p><pre>./create-user-signup.sh 1e919xxxxxxxxxxxxxxx9k4krm your_email@account.com<br><br>Creating a user with the following details:<br>Email=your_email+015b26@account.com<br>Phone=+61424351028<br>Name=Susan30117<br>Password=269b58b723907daec510ABC!<br><br>{<br>    &quot;UserConfirmed&quot;: false,<br>    &quot;CodeDeliveryDetails&quot;: {<br>        &quot;Destination&quot;: &quot;y***@a***&quot;,<br>        &quot;DeliveryMedium&quot;: &quot;EMAIL&quot;,<br>        &quot;AttributeName&quot;: &quot;email&quot;<br>    },<br>    &quot;UserSub&quot;: &quot;4da4944d-046c-4e5f-ba9c-2b69eb79b654&quot;<br>}<br>Finished.</pre><p>You should receive a code in your email. To provide this code back to Cognito we need to use the confirm-sign-up action. You’ll once again need the client ID and also the username. This can be the email address or the UserSub value.</p><pre>aws cognito-idp confirm-sign-up \<br>  --client-id 1e919xxxxxxxxxxxxxxx9k4krm \<br>  --username your_email+015b26@account.com \<br>  --confirmation-code 751473</pre><p>At this point both the user itself and the email address is verified and the PostConfirmation Lambda function should have triggered. If you enabled debugging, as explained previously, you should see the call to Lambda and the payload in CloudWatch.</p><pre>2023-03-29T07:58:12.729Z fd589f18-d2e6-438e-97a3-8d143f9e157d info Handling Cognito event trigger source &#39;$PostConfirmation_ConfirmSignUp&#39;.<br>2023-03-29T07:58:12.768Z fd589f18-d2e6-438e-97a3-8d143f9e157d dbug <br>{<br>    &quot;request&quot;: {<br>        &quot;clientMetadata&quot;: null,<br>        &quot;userAttributes&quot;: {<br>            &quot;sub&quot;: &quot;4da4944d-046c-4e5f-ba9c-2b69eb79b654&quot;,<br>            &quot;email_verified&quot;: &quot;true&quot;,<br>            &quot;cognito:user_status&quot;: &quot;CONFIRMED&quot;,<br>            &quot;cognito:email_alias&quot;: &quot;your_email+015b26@account.com&quot;,<br>            &quot;name&quot;: &quot;Susan30117&quot;,<br>            &quot;phone_number_verified&quot;: &quot;false&quot;,<br>            &quot;phone_number&quot;: &quot;+61424351028&quot;,<br>            &quot;email&quot;: &quot;your_email+015b26@account.com&quot;<br>        }<br>    },<br>    &quot;response&quot;: {},<br>    &quot;version&quot;: &quot;1&quot;,<br>    &quot;triggerSource&quot;: &quot;PostConfirmation_ConfirmSignUp&quot;,<br>    &quot;region&quot;: &quot;ap-southeast-2&quot;,<br>    &quot;userPoolId&quot;: &quot;ap-southeast-2_H0KQ9nfDu&quot;,<br>    &quot;userName&quot;: &quot;4da4944d-046c-4e5f-ba9c-2b69eb79b654&quot;,<br>    &quot;callerContext&quot;: {<br>        &quot;awsSdkVersion&quot;: &quot;aws-sdk-unknown-unknown&quot;,<br>        &quot;clientId&quot;: &quot;1e919xxxxxxxxxxxxxxx9k4krm&quot;<br>    }<br>}</pre><p>You’ll see that cognito:user_status is CONFIRMED and email_verified is true. This means that the user is confirmed and can now login. The user will also be able to recover their account if they forget the password. We’ll test this in the next section.</p><p>Let’s now check that the user information was written into the DynamoDB Users table.</p><pre>aws dynamodb scan --table-name Users<br><br>{<br>    &quot;Items&quot;: [<br>        {<br>            &quot;Phone&quot;: {<br>                &quot;S&quot;: &quot;+61424351028&quot;<br>            },<br>            &quot;UserId&quot;: {<br>                &quot;S&quot;: &quot;4da4944d-046c-4e5f-ba9c-2b69eb79b654&quot;<br>            },<br>            &quot;Email&quot;: {<br>                &quot;S&quot;: &quot;your_email+015b26@account.com&quot;<br>            }<br>        }<br>    ],<br>    &quot;Count&quot;: 1,<br>    &quot;ScannedCount&quot;: 1,<br>    &quot;ConsumedCapacity&quot;: null<br>}</pre><p>Whoop, there it is!</p><h4>ConfirmForgotPassword Handler</h4><p>Now that we have a verified user, we can try to test the ConfirmForgotPassword handler. In this example you will receive an email confirming that your password has been reset after you reset your password successfully.</p><p>To request a password reset use the forgot_password action. You’ll once again need the client ID and also the username. This can be the email address or the UserSub value. This assumes you have an existing validated user.</p><pre>aws cognito-idp forgot-password \<br>  --client-id 1e919xxxxxxxxxxxxxxx9k4krm \<br>  --username your_email+015b26@account.com<br><br>{<br>    &quot;CodeDeliveryDetails&quot;: {<br>        &quot;Destination&quot;: &quot;y***@a***&quot;,<br>        &quot;DeliveryMedium&quot;: &quot;EMAIL&quot;,<br>        &quot;AttributeName&quot;: &quot;email&quot;<br>    }<br>}</pre><p>You should receive a code in your email. To supply the code back to Cognito you will use the confirm-forgot-password action. You’ll again need the client ID and the username, but this time you’ll also need the code and a new password.</p><pre>aws cognito-idp confirm-forgot-password \<br>  --client-id 1e919xxxxxxxxxxxxxxx9k4krm \<br>  --username your_email+015b26@account.com \<br>  --confirmation-code 155343 \<br>  --password mynewpasswordABC123!</pre><p>For security reasons the service will not give any response, but you should receive an email stating “This is a security notice that your password has been reset.”</p><p>You can also check if your ConfirmForgotPassword Lambda function was exectued in CloudWatch.</p><pre>2023-03-29T08:15:35.961Z 3b054e21-fd01-486b-abfc-077ab9f6f998 info Handling Cognito event trigger source &#39;$PostConfirmation_ConfirmForgotPassword&#39;.<br>2023-03-29T08:15:35.981Z 3b054e21-fd01-486b-abfc-077ab9f6f998 dbug <br>{<br>    &quot;request&quot;: {<br>        &quot;clientMetadata&quot;: null,<br>        &quot;userAttributes&quot;: {<br>            &quot;sub&quot;: &quot;4da4944d-046c-4e5f-ba9c-2b69eb79b654&quot;,<br>            &quot;email_verified&quot;: &quot;true&quot;,<br>            &quot;cognito:user_status&quot;: &quot;CONFIRMED&quot;,<br>            &quot;cognito:email_alias&quot;: &quot;your_email+015b26@account.com&quot;,<br>            &quot;name&quot;: &quot;Susan30117&quot;,<br>            &quot;phone_number_verified&quot;: &quot;false&quot;,<br>            &quot;phone_number&quot;: &quot;+61424351028&quot;,<br>            &quot;email&quot;: &quot;your_email+015b26@account.com&quot;<br>        }<br>    },<br>    &quot;response&quot;: {},<br>    &quot;version&quot;: &quot;1&quot;,<br>    &quot;triggerSource&quot;: &quot;PostConfirmation_ConfirmForgotPassword&quot;,<br>    &quot;region&quot;: &quot;ap-southeast-2&quot;,<br>    &quot;userPoolId&quot;: &quot;ap-southeast-2_H0KQ9nfDu&quot;,<br>    &quot;userName&quot;: &quot;4da4944d-046c-4e5f-ba9c-2b69eb79b654&quot;,<br>    &quot;callerContext&quot;: {<br>        &quot;awsSdkVersion&quot;: &quot;aws-sdk-unknown-unknown&quot;,<br>        &quot;clientId&quot;: &quot;1e919xxxxxxxxxxxxxxx9k4krm&quot;<br>    }<br>}</pre><p>To confirm that the password reset worked you can try to request a token. Note that this uses the USER_PASSWORD_AUTH flow, which would generally not be recommended for production use. I used it here, because it is the easiest way to demonstrate something using the AWS CLI.</p><pre>aws cognito-idp initiate-auth \<br>  --client-id 1e919xxxxxxxxxxxxxxx9k4krm \<br>  --auth-flow USER_PASSWORD_AUTH \<br>  --auth-parameters &quot;USERNAME=your_email+015b26@account.com,PASSWORD=mynewpasswordABC123!&quot;<br><br>{<br>    &quot;ChallengeParameters&quot;: {},<br>    &quot;AuthenticationResult&quot;: {<br>        &quot;AccessToken&quot;: &quot;eyJraWQiOiJxbkd1YjF...&quot;,<br>        &quot;ExpiresIn&quot;: 3600,<br>        &quot;TokenType&quot;: &quot;Bearer&quot;,<br>        &quot;RefreshToken&quot;: &quot;eyJjdHkiOiJKV1QiLCJ...&quot;,<br>        &quot;IdToken&quot;: &quot;eyJraWQiOiJ5b0Z6YUd...&quot;<br>    }<br>}</pre><p>Looking good!</p><h3>Other Trigger Implementations</h3><p>Watch <a href="https://awstip.com/cognito-lambda-triggers-in-dotnet-3bf13a55eda3">this space</a> for articles that are dedicated to explanations and examples on how to handle each type of trigger source.</p><p>Follow me or subscribe to receive notification of when these articles are released.</p><p>Thanks for your time!</p><h3>Level Up Coding</h3><p>Thanks for being a part of our community! Before you go:</p><ul><li>👏 Clap for the story and follow the author 👉</li><li>📰 View more content in the <a href="https://levelup.gitconnected.com/?utm_source=pub&amp;utm_medium=post">Level Up Coding publication</a></li><li>💰 Free coding interview course ⇒ <a href="https://skilled.dev/?utm_source=luc&amp;utm_medium=article">View Course</a></li><li>🔔 Follow us: <a href="https://twitter.com/gitconnected">Twitter</a> | <a href="https://www.linkedin.com/company/gitconnected">LinkedIn</a> | <a href="https://newsletter.levelup.dev/">Newsletter</a></li></ul><p>🚀👉 <a href="https://jobs.levelup.dev/talent/welcome?referral=true"><strong>Join the Level Up talent collective and find an amazing job</strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f1419f4d4952" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/cognito-lambda-triggers-in-dotnet-postconfirmation-f1419f4d4952">Cognito Lambda Triggers in Dotnet — PostConfirmation</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AWS Single Sign-on, Terraform and Terragrunt]]></title>
            <link>https://levelup.gitconnected.com/aws-single-sign-on-terraform-and-terragrunt-a8c22bb7cfa8?source=rss-159faff569ab------2</link>
            <guid isPermaLink="false">https://medium.com/p/a8c22bb7cfa8</guid>
            <category><![CDATA[security]]></category>
            <category><![CDATA[single-sign-on]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[aws-iam-identity-center]]></category>
            <dc:creator><![CDATA[Oliver Schenk]]></dc:creator>
            <pubDate>Mon, 27 Mar 2023 02:19:12 GMT</pubDate>
            <atom:updated>2023-03-30T15:13:40.347Z</atom:updated>
            <cc:license>http://creativecommons.org/publicdomain/zero/1.0/</cc:license>
            <content:encoded><![CDATA[<p>When working locally with the AWS CLI or with Terraform or Terragrunt, you will require a valid AWS configuration to be authorised to communicate with AWS services.</p><p>It is recommended to leverage features such as Single Sign-On (SSO) and Role Based Access Controls (RBAC) when using your identity as the principal for interacting with AWS services.</p><p>Using SSO ensures that you are not storing a permanent access key and secret access key and that your sessions have a limited lifetime. Using <a href="https://docs.aws.amazon.com/singlesignon/latest/userguide/permissionsetsconcept.html">permission sets</a> (a form of RBAC) ensures that you create roles with only the minimum required permissions for relevant accounts and assume them when needed. This is one aspect of implmenting the principle of least privilege (PoLP).</p><p>In this guide it is assumed that you are making use of <a href="https://docs.aws.amazon.com/singlesignon/latest/userguide/what-is.html">IAM Identity Center</a> and that you are configured as a user. You should also have access to one or more accounts using permission sets. This guide will walk through accessing at least two accounts using the same login.</p><p>Code provided in this guide is available in this <a href="https://github.com/oliverschenk/aws-sso-terraform-terragrunt">Github repository</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*VkmV8Rw_RjN2mk1U" /><figcaption>Photo by <a href="https://unsplash.com/@marvelous?utm_source=medium&amp;utm_medium=referral">Marvin Meyer</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>AWS SSO configuration</h3><p>When using the SSO functionality in Identity Center you will not be storing any access key or secret access key on your local machine. Instead, you will configure the necessary SSO profiles in ~/.aws/config and login via AWS CLI commands to create a session.</p><p>It is worth noting that you only need to login once in order to access the accounts for which you have been given access. It is then a matter of specifying the correct profile parameter to control which account is being are accessed and using what role.</p><h3>Create initial sso-session</h3><p>When setting up SSO you’ll need to know the <em>Start Url</em> to your specific login page. You’ll find this in the Identity Center labelled <em>AWS access portal URL</em>. It looks a bit like the following URL, but you’ll need to replace your-portal-name with your own.</p><pre>https://your-portal-name.awsapps.com/start#/</pre><p>First we need to create a session profile that will be used for login. Let’s assume that your local AWS config file does not currently have any SSO configurations. Remember to replace session-name, your-portal-name and the AWS region with your own details.</p><pre>aws configure sso-session<br><br>SSO session name: session-name<br>SSO start URL [None]: https://your-portal-name.awsapps.com/start#/<br>SSO region [None]: ap-southeast-2<br>SSO registration scopes [sso:account:access]:<br><br>Completed configuring SSO session: session-name<br>Run the following to login and refresh access token for this session:<br><br>aws sso login --sso-session session-name</pre><p>You will now see that the ~/.aws/config file contains the following sso session block.</p><pre>[sso-session session-name]<br>sso_start_url = https://your-portal-name.awsapps.com/start#/<br>sso_region = ap-southeast-2<br>sso_registration_scopes = sso:account:access</pre><p>This is a bit different to the profile block that you might be used to. The sso-session block is used only during login and then caches your credentials under the hood. We’ll shortly configure some profiles which then point to this sso-session block.</p><p>You can test your configuration by running the login command with the sso-session parameter.</p><pre>aws sso login --sso-session session-name</pre><p>You might see a browser window open, or, if not, you can click on the link provided or open it yourself manually. Once you provide the code that is shown in your own CLI window you will then be asked to login. This is your username and password as was created in Identity Center.</p><pre>Attempting to automatically open the SSO authorization page in your default browser.<br>If the browser does not open or you wish to use a different device to authorize this request, open the following URL:<br><br>https://device.sso.ap-southeast-2.amazonaws.com/<br><br>Then enter the code:<br><br>QGTQ-ZJXQ<br><br>Successfully logged into Start URL: https://your-portal-name.awsapps.com/start#/</pre><p>Once logged in, the CLI window will tell you that login was successful.</p><p>The AWS CLI has now stored your temporary session token in its cache folder. This is generally stored in ~/.aws/sso/cache.</p><h3>Create account profiles</h3><p>You’ve now logged in and received a temporary session token. However, we now need to set up individual profiles for each account and role combination that you want to access. These are the permission sets that have been assigned in Identity Center.</p><p>By default, you will not be able to access any account. In order to access an account, a permission set must be defined linking your user account with a role and account. Each user can be linked with more than one account, and each of these accounts may be accessed through more than one role. Make sure you have configured the permission sets for your accounts in Identity Center.</p><p>In this guide I will set up two profiles for two different accounts, but both have the same role name:</p><ul><li>Profile DevelopmentPowerUser for account 111111111111 (Development) with role name PowerUserAccess</li><li>Profile DeploymentPowerUser for account 222222222222 (Deployment) with role name PowerUserAccess</li></ul><p>One way to automatically create SSO profiles in the AWS config file is to run aws configure sso and specifying the SSO session name.</p><pre>aws configure sso<br><br>SSO session name (Recommended): session-name<br>There are 4 AWS accounts available to you.<br>Using the account ID 111111111111<br>The only role available to you is: PowerUserAccess<br>Using the role name &quot;PowerUserAccess&quot;<br>CLI default client Region [None]: ap-southeast-2<br>CLI default output format [None]: json<br>CLI profile name [PowerUserAccess-111111111111]: DevelopmentPowerUser<br><br>To use this profile, specify the profile name using --profile, as shown:<br><br>aws s3 ls --profile DevelopmentPowerUser</pre><p>The SSO session name must match exactly what was created in the previous section of this guide. In this case it was session-name. If you haven’t logged in recently, it will ask you to login, otherwise it will skip to the next question.</p><p>The CLI will ask you to select an account and a role based on your permission sets. After that you can specify the default region and CLI output format.</p><p>Finally, you just need to choose a profile name. What you choose here will become your profile parameter for any future CLI calls to AWS services. In the ~/.aws/config file you’ll see the following.</p><pre>[profile DevelopmentPowerUser]<br>sso_session = session-name<br>sso_account_id = 111111111111<br>sso_role_name = PowerUserAccess<br>region = ap-southeast-2<br>output = json</pre><p>You can now repeat the above for any other accounts and roles that you require. In the previous example I created a profile for the DevelopmentPowerUser and below I created a DeploymentPowerUser profile.</p><pre>aws configure sso<br><br>SSO session name (Recommended): session-name<br>There are 4 AWS accounts available to you.<br>Using the account ID 222222222222<br>The only role available to you is: PowerUserAccess<br>Using the role name &quot;PowerUserAccess&quot;<br>CLI default client Region [None]: ap-southeast-2<br>CLI default output format [None]: json<br>CLI profile name [PowerUserAccess-222222222222]: DeploymentPowerUser<br><br>To use this profile, specify the profile name using --profile, as shown:<br><br>aws s3 ls --profile DevelopmentPowerUser</pre><p>That’s it, now you can simply specify the profile parameter and access each of your accounts with the given role. In your ~/.aws/config file you’ll see the following.</p><pre>[profile DeploymentPowerUser]<br>sso_session = session-name<br>sso_account_id = 222222222222<br>sso_role_name = PowerUserAccess<br>region = ap-southeast-2<br>output = json</pre><h3>Testing SSO profiles</h3><p>Just to be sure it’s all working we can use the aws sts get-caller-identity command so check what account we are talking to.</p><pre>aws sts get-caller-identity --profile DevelopmentPowerUser<br>{<br>    &quot;UserId&quot;: &quot;AROAXGDMY47MMC7G7UT42:oliver.schenk@my-corp.com.au&quot;,<br>    &quot;Account&quot;: &quot;111111111111&quot;,<br>    &quot;Arn&quot;: &quot;arn:aws:sts::111111111111:assumed-role/AWSReservedSSO_PowerUserAccess_4e450d63f85909d9/oliver.schenk@my-corp.com.au&quot;<br>}<br><br>aws sts get-caller-identity --profile DeploymentPowerUser<br>{<br>    &quot;UserId&quot;: &quot;AROARLJ34IRAHVJPXC7BD:oliver.schenk@my-corp.com.au&quot;,<br>    &quot;Account&quot;: &quot;222222222222&quot;,<br>    &quot;Arn&quot;: &quot;arn:aws:sts::222222222222:assumed-role/AWSReservedSSO_PowerUserAccess_74c4a25373eded7b/oliver.schenk@my-corp.com.au&quot;<br>}</pre><p>You can see in the above output that by simply changing the profile parameter you can access the respective account with the relevant role.</p><h3>SSO with Terraform</h3><p>Let’s create a very simple SSM Parameter resource in a Terraform module that we can then re-use and deploy through Terraform or Terragrunt.</p><pre># variables.tf<br>variable &quot;name&quot; {}<br>variable &quot;value&quot; {}</pre><pre># main.tf<br>resource &quot;aws_ssm_parameter&quot; &quot;foo&quot; {<br>  name  = var.name<br>  type  = &quot;String&quot;<br>  value = var.value<br>}</pre><h4>Environment variable</h4><p>To access a single account when working with Terraform, the most convenient option is to export the profile name as an environment variable. We’ll use the Terraform module defined earlier.</p><pre>module &quot;ssm_parameter_foo&quot; {<br>  source = &quot;../modules//ssm-string-parameter&quot;<br><br>  name  = &quot;profile-type-tf-env&quot;<br>  value = &quot;Terraform with environment variable profile&quot;<br>}</pre><p>Set the AWS_PROFILE environment variable.</p><pre>export AWS_PROFILE=DevelopmentPowerUser</pre><p>You can now deploy the Terraform into the Development account by simply running terraform init and terraform apply.</p><h4>Provider profile parameter</h4><p>Another option is to provide the profile in the Terraform provider block. This is not a preferred option as it means a hard-coded profile parameter ends up in your source code and this may not match other developer if they checkout your code.</p><pre># provider.tf<br>provider &quot;aws&quot; {<br>  region  = &quot;ap-southeast-2&quot;<br>  profile = &quot;DevelopmentPowerUser&quot;<br>}</pre><p>We’ll again use the Terraform module defined earlier.</p><pre>module &quot;ssm_parameter_foo&quot; {<br>  source = &quot;../modules//ssm-string-parameter&quot;<br><br>  name  = &quot;profile-type-tf-provider&quot;<br>  value = &quot;Terraform with profile in provider block&quot;<br>}</pre><p>You can now deploy the Terraform into the Development account. Let’s also unset any environment variables just to make sure it’s not still using the environment variable.</p><pre>unset AWS_PROFILE<br>terraform init<br>terraform apply</pre><h3>SSO with Terragrunt</h3><p>When using Terragrunt, the above methods work in the same way. One slight variation is that Terragrunt can generate Terraform configuration blocks such as the provider block and you can then leverage external configuration files to feed parameters in.</p><h4>Environment variable</h4><p>The environment variable option works in exactly the same way when using Terragrunt.</p><pre>terraform {<br>  source = &quot;../modules//ssm-string-parameter&quot;<br>}<br><br>inputs = {<br>  name = &quot;profile-type-tg-env&quot;<br>  value = &quot;Terragrunt with environment variable profile&quot;<br>}</pre><p>Set the AWS_PROFILE environment variable.</p><pre>export AWS_PROFILE=DevelopmentPowerUser</pre><p>You can now deploy the Terraform into the Development account.</p><pre>terragrunt apply</pre><p>With Terragrunt you don’t have to run init as by default it automatically runs the init command each time.</p><h4>Provider profile parameter</h4><p>In this scenario we make Terragrunt generate the necessary provider. As a bonus we load the profile from an external YAML configuration file.</p><pre>terraform {<br>  source = &quot;../modules//ssm-string-parameter&quot;<br>}<br><br>locals {<br>  common_vars = yamldecode(file(find_in_parent_folders(&quot;common_vars.yaml&quot;)))<br>}<br><br># Indicate what region to deploy the resources into<br>generate &quot;provider&quot; {<br>  path = &quot;provider.tf&quot;<br>  if_exists = &quot;overwrite_terragrunt&quot;<br>  contents = &lt;&lt;EOF<br>provider &quot;aws&quot; {<br>  region = &quot;${local.common_vars.region}&quot;<br>  profile = &quot;${local.common_vars.profile}&quot;<br>}<br>EOF<br>}<br><br>inputs = {<br>  name = &quot;profile-type-tg-provider&quot;<br>  value = &quot;Terragrunt with generated provider block and config file&quot;<br>}</pre><p>You can now deploy the Terraform into the Development account. Let’s also unset any environment variables just to make sure it’s not still using the environment variable.</p><pre>unset AWS_PROFILE<br>terragrunt apply</pre><h3>Terraform and Terragrunt with S3 backend</h3><p>This is where we start to run into trouble. Everything up until now worked fine, but once you introduce an S3 remote backend you’ll start to see some problems. This is the case for both Terraform and Terragrunt.</p><p>Let’s say you have the following setup</p><pre># main.tf<br>module &quot;ssm_parameter_foo&quot; {<br>  source = &quot;../modules//ssm-string-parameter&quot;<br><br>  name  = &quot;profile-type-tf-provider-s3&quot;<br>  value = &quot;Terraform with profile in provider block with S3 backend&quot;<br>}</pre><pre># provider.tf<br>provider &quot;aws&quot; {<br>  region  = &quot;ap-southeast-2&quot;<br>  profile = &quot;DevelopmentPowerUser&quot;<br>}</pre><pre># backend.tf<br>terraform {<br>  backend &quot;s3&quot; {<br>    bucket         = &quot;terraform-provider-profile-s3-backend-state&quot;<br>    dynamodb_table = &quot;terraform-locking-table&quot;<br>    encrypt        = true<br>    key            = &quot;terraform.tfstate&quot;<br>    profile        = &quot;DevelopmentPowerUser&quot;<br>    region         = &quot;ap-southeast-2&quot;<br>  }<br>}</pre><p>When you try to run the terraform init command you will see an error.</p><pre>Error: error configuring S3 Backend: Error creating AWS session: profile &quot;DevelopmentPowerUser&quot; is configured to use SSO but is missing required configuration: sso_region, sso_start_url</pre><p>The problem is that the recommended way of creating AWS config profiles does not seem to work with Terraform as it does not recognise SSO configurations where the sso_session parameter of the [profile ...] points to a [sso-session ...] block. Rather it wants the sso_start_url and the sso_region to be part of the profile block.</p><p>The only way for Terraform to be happy is to create a profile without the sso_session parameter and include the sso_start_url and the sso_region.</p><pre>[profile DevelopmentPowerUserTerraform]<br>sso_start_url = https://your-portal-name.awsapps.com/start#/<br>sso_region = ap-southeast-2<br>sso_account_id = 111111111111<br>sso_role_name = PowerUserAccess<br>region = ap-southeast-2<br>output = json</pre><p>You should then use aws sso login using the profile parameter.</p><pre>aws sso login --profile DevelopmentPowerUserTerraform</pre><p>Finally, update any references to the profile so that it is now called DevelopmentPowerUserTerraform.</p><p>This issue is still being tracked in a <a href="https://github.com/hashicorp/terraform-provider-aws/issues/28263">Github issue</a>.</p><h3>CI/CD considerations</h3><p>The methods presented in this guide is specifically for a user who is manually deploying infrastructure with Terraform or Terragrunt from their local development machine.</p><p>In a proper CI/CD environment you might use this method only for bootstrapping the initial CI/CD pipeline and roles. Any future change would be done through Git commits and the CI/CD pipeline (for example CodePipeline and CodeBuild) would then assume the necessary roles to automatically deploy infrastructure using Terraform or Terragrunt.</p><h3>Level Up Coding</h3><p>Thanks for being a part of our community! Before you go:</p><ul><li>👏 Clap for the story and follow the author 👉</li><li>📰 View more content in the <a href="https://levelup.gitconnected.com/?utm_source=pub&amp;utm_medium=post">Level Up Coding publication</a></li><li>💰 Free coding interview course ⇒ <a href="https://skilled.dev/?utm_source=luc&amp;utm_medium=article">View Course</a></li><li>🔔 Follow us: <a href="https://twitter.com/gitconnected">Twitter</a> | <a href="https://www.linkedin.com/company/gitconnected">LinkedIn</a> | <a href="https://newsletter.levelup.dev/">Newsletter</a></li></ul><p>🚀👉 <a href="https://jobs.levelup.dev/talent/welcome?referral=true"><strong>Join the Level Up talent collective and find an amazing job</strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a8c22bb7cfa8" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/aws-single-sign-on-terraform-and-terragrunt-a8c22bb7cfa8">AWS Single Sign-on, Terraform and Terragrunt</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Protect Your AWS Access Keys]]></title>
            <link>https://levelup.gitconnected.com/protect-your-aws-access-keys-7505e6f4db0f?source=rss-159faff569ab------2</link>
            <guid isPermaLink="false">https://medium.com/p/7505e6f4db0f</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[aws-vault]]></category>
            <category><![CDATA[security]]></category>
            <category><![CDATA[keybase]]></category>
            <dc:creator><![CDATA[Oliver Schenk]]></dc:creator>
            <pubDate>Wed, 15 Feb 2023 06:12:59 GMT</pubDate>
            <atom:updated>2023-03-01T01:47:51.652Z</atom:updated>
            <cc:license>http://creativecommons.org/publicdomain/zero/1.0/</cc:license>
            <content:encoded><![CDATA[<p>This is a quick guide on how I manage and secure my AWS credentials on my development machine running either Ubuntu natively or in Windows 10/11 using WSL2 (I’m just going to call it WSL) and <a href="https://github.com/99designs/aws-vault">aws-vault</a>.</p><p>In this article, I explain the Windows with WSL combination, because this is where I had the most trouble getting things right. I believe that running Ubuntu natively may have a few less issues, but the steps should be similar.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YhPLzisGPzhAYokMHh-wrg.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@markusspiske?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Markus Spiske</a> on <a href="https://unsplash.com/photos/gkmSu3dUjTg?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><h3>High Level Steps</h3><ol><li>Install Ubuntu WSL or upgrade to latest LTS version</li><li>Install pre-requisites</li><li>Configure encryption keys</li><li>Configure pass</li><li>Configure aws-vault</li><li>Add credentials to aws-vault</li><li>Test aws-vault usage</li></ol><h3>Ubuntu and WSL</h3><p>I won’t linger on this point for too long, but just for the sake of completeness, when you are running Ubuntu in WSL it is usually installed via the Microsoft store and will appear in your installed applications list in the control panel. You can see the <a href="https://learn.microsoft.com/en-us/windows/wsl/install">official documentation</a> for more information.</p><p>If you have an old version of Ubuntu you can do a distribution upgrade or uninstall Ubuntu completely and re-install the latest version from the store. Just beware that if you uninstall <strong>you will lose any and all data</strong> that was in your WSL environment.</p><p>At the time of writing I installed a fresh version of Ubuntu 22.04.1 LTS (Jammy Jellyfish) from the Microsoft store.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6dEYeQVm_W_PwfcNKs8IPw.png" /></figure><p>After installation you should be able to start wsl from your favourite CLI. On first launch, it will ask you to create a username and password (which does not need to match your Windows username).</p><h3>Install pre-requisites</h3><p>Make sure the following packages have been installed in your system.</p><pre>sudo apt-get update<br>sudo apt-get upgrade<br>sudo apt-get install -y pass gnupg</pre><ul><li>pnupg will give you the gpg tool for generating new key pairs</li><li>pass is used by aws-vault as a backend to store your encrypted credentials and password protect them</li></ul><p>There is an extra environment variable required to get gpg to work properly when asking for passwords. Run the following commands to fix this issue.</p><pre>echo &#39;export GPG_TTY=$(tty)&#39; &gt;&gt; ~/.profile<br>export GPG_TTY=$(tty)</pre><p>I would also recommend installing Homebrew to manage packages such as aws-vault (and other AWS tools you may use like AWS CLI or Terraform) that are not available, or not up-to-date, using apt.</p><pre>/bin/bash -c &quot;$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)&quot;</pre><p>This can take a while. Once installed run the following in order to complete the recommended Homebrew setup.</p><pre>echo &#39;# Set PATH, MANPATH, etc., for Homebrew.&#39; &gt;&gt; ~/.profile<br>echo &#39;eval &quot;$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)&quot;&#39; &gt;&gt; ~/.profile<br>eval &quot;$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)&quot;<br>sudo apt-get install -y build-essential<br>brew install gcc</pre><h3>Configure encryption keys</h3><p>When aws-vault stores your AWS credentials it uses what is called a backend. There are a number of supported backends as <a href="https://github.com/99designs/aws-vault#vaulting-backends">listed in the documentation</a>.</p><p>The one that we will be using in this guide is the pass backend, which is the standard unix password manager. For this to work properly it needs to be configured with public key encryption as it makes use of gpg for encryption and decryption.</p><p>There are three scenarios we can consider:</p><ol><li>You already have a public/private key pair in your system that you want to use.</li><li>You already have a public/private key pair stored somewhere else (such as in Keybase) and you need to import it.</li><li>You don’t yet have a public/private key pair.</li></ol><h4>You already have a key pair</h4><p>In this scenario you should be good to go. You can confirm with gpg -k that you have a valid key listed and that it is trusted.</p><h4>You have a key pair in Keybase</h4><p>This scenario is a bit different in that I’m using a <a href="https://keybase.io/">Keybase account</a> for managing my identity and encryption key. In my case I already had a key stored in Keybase. This scenario assumes you also have a Keybase account and have previously imported a key into Keybase.</p><p>If you haven’t yet installed Keybase you can do so <a href="https://keybase.io/docs/the_app/install_linux">following the instructions</a> on the Keybase website.</p><p>Assuming your Keybase setup is good to go, you can list they keys in your Keybase account as follows:</p><pre>$ keybase pgp list<br>Keybase Key ID:  0101ed2034cdea4ca787...<br>PGP Fingerprint: e2f65096845b529a0f13...<br>PGP Identities:<br>   Oliver Schenk &lt;my_name@email.com&gt;</pre><p>To import a key into your local gpg trust database you can use the following command:</p><pre>$ keybase pgp export -s | gpg --allow-secret-key-import --import<br>gpg: key 7D2E1...: public key &quot;Oliver Schenk &lt;my_name@email.com&gt;&quot; imported<br>gpg: key 7D2E1...: secret key imported<br>gpg: Total number processed: 1<br>gpg:               imported: 1<br>gpg:       secret keys read: 1<br>gpg:   secret keys imported: 1</pre><p>It may ask you for the Keybase account password and then it will ask you for a new password to password protect your key.</p><p>The last step is to make this key a trusted key. You will see if you list your keys that the trust level is unknown.</p><pre>$ gpg -k<br>/home/oliverschenk/.gnupg/pubring.kbx<br>-------------------------------------<br>pub   rsa4096 2023-02-14 [SC] [expires: 2039-02-10]<br>      E2F6509684...<br>uid           [ unknown] Oliver Schenk &lt;my_name@email.com&gt;<br>sub   rsa4096 2023-02-14 [E] [expires: 2039-02-10]</pre><p>To fix this use the following command (replace the hex part with your own key signature from the gpg -k output) and set the trust level to 5:</p><pre>$ gpg --edit-key E2F65096845B52... trust<br>...<br>Please decide how far you trust this user to correctly verify other users&#39; keys<br>(by looking at passports, checking fingerprints from different sources, etc.)<br><br>  1 = I don&#39;t know or won&#39;t say<br>  2 = I do NOT trust<br>  3 = I trust marginally<br>  4 = I trust fully<br>  5 = I trust ultimately<br>  m = back to the main menu<br><br>Your decision? 5<br>Do you really want to set this key to ultimate trust? (y/N) y<br>...</pre><p>If you do gpg -k again you should see the trust is now ultimate.</p><h4>You don’t have a key pair</h4><p>You can see what keys you have in your system with the command gpg -k which will list your keys. If you’ve never used gpg before you’ll see something like this, but only the first time.</p><pre>$ gpg -k<br>gpg: /home/oliverschenk/.gnupg/trustdb.gpg: trustdb created</pre><p>If you get no other results then you don’t have any local keys.</p><p>To generate a key you can follow these steps. It will ask you some questions along the way. I’ved used the --generate-key flag but you can also use --full-generate-key which gives you more customisation to suit your needs.</p><pre>$ gpg --generate-key<br>...<br><br>Real name: Oliver Schenk<br>Email address: my_name@email.com<br>You selected this USER-ID:<br>    &quot;Oliver Schenk &lt;my_name@email.com&gt;&quot;<br><br>...<br>gpg: key E2D45551B0B51E1E marked as ultimately trusted<br>gpg: directory &#39;/home/oliverschenk/.gnupg/openpgp-revocs.d&#39; created<br>gpg: revocation certificate stored as &#39;/home/oliverschenk/.gnupg/openpgp-revocs.d/BE9D74317B9F88...51B0B51E1E.rev&#39;<br>public and secret key created and signed.<br><br>pub   rsa3072 2023-02-15 [SC] [expires: 2025-02-14]<br>      BE9D74317B9F883...<br>uid                      Oliver Schenk &lt;my_name@email.com&gt;<br>sub   rsa3072 2023-02-15 [E] [expires: 2025-02-14]</pre><p>The key that was generated is considered ultimately trusted which means it has the highest level of trust in your system. You can now list your keys using gpg -k again.</p><pre>$ gpg -k<br>gpg: checking the trustdb<br>gpg: marginals needed: 3  completes needed: 1  trust model: pgp<br>gpg: depth: 0  valid:   1  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 1u<br>gpg: next trustdb check due at 2025-02-14<br>/home/oliverschenk/.gnupg/pubring.kbx<br>-------------------------------------<br>pub   rsa3072 2023-02-15 [SC] [expires: 2025-02-14]<br>      BE9D74317B9F8...<br>uid           [ultimate] Oliver Schenk &lt;my_name@email.com&gt;<br>sub   rsa3072 2023-02-15 [E] [expires: 2025-02-14]</pre><h3>Configure pass</h3><p>In this step we need to initialise pass to use the key prepared earlier. As you would have seen a few times, the gpg -k command will list your key and you’ll find your key ID as a relatively long hex string. Use this string when initialising pass.</p><pre>$ pass init D2F6505D842D529B0F13D27B7C2E217E20215AA0<br>mkdir: created directory &#39;/home/oliverschenk/.password-store/&#39;<br>Password store initialized for E2F65056842B529B0F13D27B7C2E117E202052A9</pre><h3>Configure aws-vault</h3><p>Let’s now make sure aws-vault is installed and configured correctly.</p><p>To install aws-vault you can use brew.</p><pre>$ brew install aws-vault</pre><p>You’re almost ready to store your AWS credentials in aws-vault. There’s one more configuration step that you have to do in order to make sure aws-vault uses the pass backend.</p><pre>echo &#39;# Set aws-vault configuration env variables.&#39; &gt;&gt; ~/.profile<br>echo &#39;export AWS_VAULT_BACKEND=pass&#39; &gt;&gt; ~/.profile</pre><p>You can test that the profile script works by closing the WSL CLI and launching it again. With the following commands you should be able to prove that the configuration is now correct.</p><pre>$ echo $AWS_VAULT_BACKEND<br>pass<br>$ echo $GPG_TTY<br>/dev/pts/1<br>$ aws-vault list --debug<br>2023/02/14 23:04:36 aws-vault 6.6.2-Homebrew<br>2023/02/14 23:04:36 [keyring] Considering backends: [pass]<br>2023/02/14 23:04:36 Loading config file /home/&lt;your username&gt;/.aws/config<br>2023/02/14 23:04:36 Parsing config file /home/&lt;your username&gt;/.aws/config<br>2023/02/14 23:04:36 Unrecognised ini file section: DEFAULT<br>Profile                  Credentials              Sessions<br>=======                  ===========              ========</pre><p>You’ll see that the selected backend is pass which is what we want. It even created a config file in the .aws folder for us.</p><h3>Add credentials to aws-vault</h3><p>You are now ready to add a set of AWS credentials to aws-vault. You’ll need a user AWS access key and secret access key that you want to store in aws-vault.</p><pre>$ aws-vault add my-aws-user<br>Enter Access Key ID: AKIAT7S2FKU6...<br>Enter Secret Access Key:<br>Added credentials to profile &quot;my-aws-user&quot; in vault</pre><p>This will add an entry to ~/.aws/config but you also need to make sure there is a default region configured. It should look something like this.</p><pre>$ cat ~/.aws/config<br>[profile my-aws-user]<br>region=ap-southeast-2</pre><p>You can replace ap-southeast-2 with your own <a href="https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html#concepts-available-regions">region identifier</a>. This isn’t strictly necessary, but if you don’t provide it here you’ll have to provide the region with every AWS CLI call.</p><h3>Test aws-vault usage</h3><p>Let’s give it a try using the AWS CLI!</p><p>If the following command is working correctly you should be asked for a password. This is the password you used to protect your private key.</p><pre>$ aws-vault exec my-aws-user -- aws sts get-caller-identity<br>{<br>    &quot;UserId&quot;: &quot;A1DAX7S2F2U64ADAZFHWF&quot;,<br>    &quot;Account&quot;: &quot;123456789101&quot;,<br>    &quot;Arn&quot;: &quot;arn:aws:iam::123456789101:user/my-aws-user&quot;<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/563/1*GXKC-etGX0T7Hk6CpZlBsg.png" /></figure><p>Hurrah! 👌</p><p>Note that your sessions will be cached. You can see how much time is left on the session with the list command.</p><pre>$ aws-vault list<br>Profile                  Credentials              Sessions<br>=======                  ===========              ========<br>my-aws-user              my-aws-user              sts.GetSessionToken:54m21s</pre><p>To kill any existing sessions use the clear command.</p><pre>$ aws-vault clear<br>Cleared 1 sessions.</pre><p>However, note that this only clears the session held by aws-vault, but there is also a timeout on the system asking you for your key password again (I believe the default is 10 minutes). This means after 10 minutes of inactivity, it should ask you for a password again. For best protection always lock your operating system before leaving your computer!</p><h3>Summary</h3><p>You’ve now hopefully got a working Windows / WSL / aws-vault development environment that will allow you to securely store and use your AWS credentials.</p><p>As always, let me know if there are any issues, errors or omissions.</p><p>Have a great day!</p><h3>Level Up Coding</h3><p>Thanks for being a part of our community! Before you go:</p><ul><li>👏 Clap for the story and follow the author 👉</li><li>📰 View more content in the <a href="https://levelup.gitconnected.com/?utm_source=pub&amp;utm_medium=post">Level Up Coding publication</a></li><li>💰 Free coding interview course ⇒ <a href="https://skilled.dev/?utm_source=luc&amp;utm_medium=article">View Course</a></li><li>🔔 Follow us: <a href="https://twitter.com/gitconnected">Twitter</a> | <a href="https://www.linkedin.com/company/gitconnected">LinkedIn</a> | <a href="https://newsletter.levelup.dev/">Newsletter</a></li></ul><p>🚀👉 <a href="https://jobs.levelup.dev/talent/welcome?referral=true"><strong>Join the Level Up talent collective and find an amazing job</strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7505e6f4db0f" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/protect-your-aws-access-keys-7505e6f4db0f">Protect Your AWS Access Keys</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cognito Lambda Triggers in Dotnet — PreSignUp]]></title>
            <link>https://levelup.gitconnected.com/cognito-lambda-triggers-in-dotnet-presignup-4c465c6f81a?source=rss-159faff569ab------2</link>
            <guid isPermaLink="false">https://medium.com/p/4c465c6f81a</guid>
            <category><![CDATA[aws-lambda-trigger]]></category>
            <category><![CDATA[cognito-user-pools]]></category>
            <category><![CDATA[cognito]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Oliver Schenk]]></dc:creator>
            <pubDate>Wed, 31 Aug 2022 14:11:54 GMT</pubDate>
            <atom:updated>2022-10-13T02:57:29.381Z</atom:updated>
            <cc:license>http://creativecommons.org/licenses/by/4.0/</cc:license>
            <content:encoded><![CDATA[<h3>Cognito Triggers Deep Dive— PreSignUp</h3><p>This article is part of the <strong>Cognito Lambda Triggers in Dotnet</strong> series, dedicated to understanding and handling Cognito Lambda Triggers.</p><p>If you want to learn about Lambda triggers and how to architect a Lambda function to handle various trigger sources more generally, refer to the related parent article <a href="https://medium.com/@oliver.schenk/cognito-lambda-triggers-in-dotnet-3bf13a55eda3">Cognito Lambda Triggers in Dotnet</a>. If you head over there you’ll also find links to handle other trigger sources.</p><p>In the following sections we’ll look at the <strong>PreSignUp</strong> trigger source. You’ll find the technical details in the Cognito <a href="https://docs.aws.amazon.com/cognito/latest/developerguide/user-pool-lambda-pre-sign-up.html">documentation</a>, but what is often more important is to understand why it exists and why you might want to use it.</p><p>Here you’ll find an explanation with use cases, payload examples, handler implementations and test scripts.</p><p>You’ll find the <a href="https://github.com/oliverschenk/cognito-lambda-triggers-dotnet">source code on Github</a>. This will be updated with more examples over time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KGLJ0-8oENh4CyHItqSvXQ.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@moneyphotos?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">regularguy.eth</a> on <a href="https://unsplash.com/s/photos/password?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><h3>PreSignUp</h3><p>The PreSignUp_* triggers are in a category of triggers that are triggered just before Cognito signs up a new user when a new user registration request is received.</p><p>There are three possible triggers sources depending on where the request to create a new user came from.</p><ul><li><strong>PreSignUp_SignUp </strong>— User self registration process via the <a href="https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-user-pools-app-integration.html">Hosted UI</a> or the <a href="https://docs.aws.amazon.com/cognito-user-identity-pools/latest/APIReference/API_SignUp.html">SignUp</a> API call</li><li><strong>PreSignUp_AdminCreateUser </strong>— User was created by the system using the <a href="https://docs.aws.amazon.com/cognito-user-identity-pools/latest/APIReference/API_AdminCreateUser.html">AdminCreateUser</a> API call (say by an Administrator of a system)</li><li><strong>PreSignUp_ExternalProvider</strong> — User signed in for the first time using an external provider such as a social login (e.g. Facebook, Google, etc…), SAML (e.g. Active Directory) or OIDC (e.g. Salesforce). <br>Note that setting this up is a bit more complicated, because it involves linking the third party system with Cognito. See “<a href="https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-user-pools-identity-federation.html">Adding user pool sign-in through a third party</a>” for reference.</li></ul><h3>Event Payload</h3><p>The event payload passed to your trigger handler function will consist of a set of <a href="https://medium.com/@oliver.schenk/cognito-lambda-triggers-in-dotnet-3bf13a55eda3#ddd6">common parameters</a> and a request and/or response property specific to the trigger source.</p><h4>Request</h4><p>The PreSignUp request parameter consists of the following properties.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c5c74c2a11497c2b4aed0849e1718f2d/href">https://medium.com/media/c5c74c2a11497c2b4aed0849e1718f2d/href</a></iframe><p>The ClientMetadata and ValidationData properties are key-value data that are passed into the Lambda trigger handler from data that is included when making either the AdminCreateUser or SignUpUser API call.</p><p>The purpose of ClientMetadata property is to allow the client to pass further information to the PreSignUp function. The purpose of the ValidationData is to allow the client to pass further information about user validation. This extra information can be used by your function.</p><p>Because this data comes from the client side, from a security point of view, this data can only really be relied upon when the sign-up is done via API calls in a secured backend environment rather than via a public client.</p><p>One way to use the metadata is to pass information from the client to the backend and then reuse that information in another trigger handler.</p><h4>Response</h4><p>The PreSignUp response parameter consists of the following properties.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/449f35a3cc6e6863eef19c587a80b40c/href">https://medium.com/media/449f35a3cc6e6863eef19c587a80b40c/href</a></iframe><p><strong>AutoConfirmUser</strong> — Set this to true to automatically set a user’s confirmation status to CONFIRMED rather than sending out a verification email or SMS. They will then be able to login immediately.</p><p><strong>AutoVerifyPhone </strong>— Set this to true to mark the phone number as verified.</p><p><strong>AutoVerifyEmail</strong> — Set this to true to mark the email address as verified.</p><h3>Use Cases</h3><p>The purpose of this trigger is to determine whether a user should be allowed to sign up, and if so, whether they need to verify their details or if the user, or their email or phone number details, can be forced to a verified state.</p><p>Here are some examples of what you could do:</p><ul><li>You can prevent a user from being created by throwing an exception. You could do something as simple as checking the length of a user attribute or as complicated as calling some other service to check whether that user already exists, that the email address domain is within an allowed domain or that the phone number is within a specific country and so on.</li><li>Normally a new user will be in an UNCONFIRMED state. However, you can force a user into the CONFIRMED state as soon as they are created by setting the AutoConfirmUser parameter to true. For example, you might validate an enterprise client based on their domain email address and there is no need for them to validate themselves. If you do this no verification code will be sent.</li><li>You can force email and/or phone attributes to be verified automatically by setting the AutoVerifyEmail or AutoVerifyPhone parameter to true. This is usually used in conjunction with AutoConfirmUser.</li><li>If you are using more than one external provider where the same email address is used you might use the pre-signup trigger to link the two providers to the same Cognito user. See the following <a href="https://stackoverflow.com/questions/59635482/aws-cognito-best-practice-to-handle-same-user-with-same-email-address-signing">stackoverflow post</a>.</li></ul><h3>Important Tips</h3><p>There are some things you need to be aware of:</p><ul><li>The AutoConfirmUser, AutoVerifyEmail and AutoVerifyPhone parameters are ignored by Cognito when using the AdminCreateUser API call.</li><li>Users created via the AdminCreateUser API call are automatically confirmed and the status of the user becomes FORCE_CHANGE_PASSWORD. The user then has to change their password after or at first login.</li><li>Be careful when setting AutoConfirmUser to true and not verifying the user’s email address or phone number. A user will not be able to recover their password if they have forgotten it without a verified email or phone number.</li></ul><p>Regarding email and phone number verification the <a href="https://docs.aws.amazon.com/cognito/latest/developerguide/user-pool-lambda-pre-sign-up.html">Cognito documentation</a> also provides some important tips.</p><blockquote><strong>Note: </strong>If an alias with the same phone number already exists, the alias will be moved to the new user, and the previous user’s phone_number will be marked as unverified. The same is true for email addresses. To prevent this from happening, you can use the user pools <a href="https://docs.aws.amazon.com/cognito-user-identity-pools/latest/APIReference/API_ListUsers.html">ListUsers API</a> to see if there is an existing user who is already using the new user&#39;s phone number or email address as an alias.</blockquote><h3>Handler Examples</h3><p>Below you will find an example handler implementation for each PreSignUp trigger source. The templates can be found in the <a href="https://github.com/oliverschenk/cognito-lambda-triggers-dotnet/tree/main/lambda-triggers/src/CognitoLambdaTriggers/Handlers">GitHub repository</a>.</p><h4>PreSignUp_SignUp</h4><p>In this example the trigger handler automatically confirms the user, but assumes a different process will deal with email/phone verification.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/713126d22591da2aac341b636a623067/href">https://medium.com/media/713126d22591da2aac341b636a623067/href</a></iframe><h4>PreSignUp_AdminCreateUser</h4><p>In this example the trigger handler throws an exception and denies the user sign-up.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c1b4df0f36cb272ae840ec5f2e7a7f88/href">https://medium.com/media/c1b4df0f36cb272ae840ec5f2e7a7f88/href</a></iframe><h4>PreSignUp_ExternalProvider</h4><p>In this example the trigger handler checks if a user with the given username already exists and, if so, links the new provider to this existing user.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4493d1e47fb7ec0675a89d457b2c8a7a/href">https://medium.com/media/4493d1e47fb7ec0675a89d457b2c8a7a/href</a></iframe><h3>Testing</h3><p>To simplify initial testing I have provided some sample scripts that can be used to create users and clean out users. Just be a bit careful with the script that deletes users, because it will literally delete all your users in the specified Cognito user pool!</p><ul><li><a href="https://github.com/oliverschenk/cognito-lambda-triggers-dotnet/blob/main/lambda-triggers/test/commands/admin-create-user.sh"><em>admin-create-user.sh</em></a> — creates a user using AdminCreateUser API call</li><li><a href="https://github.com/oliverschenk/cognito-lambda-triggers-dotnet/blob/main/lambda-triggers/test/commands/create-user-signup.sh"><em>create-user-signup.sh </em></a>— creates a user using self-registration</li><li><a href="https://github.com/oliverschenk/cognito-lambda-triggers-dotnet/blob/main/lambda-triggers/test/commands/delete-users.sh"><em>delete-users.sh</em></a> — deletes all users in the given user pool</li></ul><p>In my samples below you’ll see I’ve put the user pool ID and user pool client ID into environment variables USER_POOL_ID and USER_POOL_CLIENT_ID respectively to avoid having to paste these values all the time.</p><h4>SignUp Handler</h4><p>The SignUp handler can be triggered using the aws cognito-idp sign-up CLI command. To simplify things, I’ve created a create-user-signup.sh script to streamline creating a new random user.</p><p>To use the script pass in your user pool client ID followed by an email address. The script will automatically generate a random alias using the + symbol. In email world this will still end up at your base email address. Everything else is randomly generated.</p><p>This script does not need any AWS credentials, because it is meant to be called by unauthenticated users.</p><pre>$ export USER_POOL_CLIENT_ID=&lt;your_user_pool_client_id&gt;</pre><pre>$ ./create-user-signup.sh $USER_POOL_CLIENT_ID your_email@account.com</pre><pre>Creating a user with the following details:<br><a href="/service/mailto:Email=oliver.schenk+3aedd2@gmail.com">Email=your_email+3aedd2@account.com</a><br>Phone=+61437340252<br>Name=Susan6599<br>Password=0117e007b8cdce52dd98ABC!</pre><pre>UserConfirmed: true<br>UserSub: 9ff6b3de-0f13-48b1-8e29-af4190a1cf2f<br>Finished.</pre><p>You will notice that if you’ve set AutoConfimUser to true in the Lambda trigger handler you will see the Cognito call will respond with UserConfirmed: true and you will not receive any confirmation email. However, your email address will remain unverified.</p><p>We can prove the user status as follows.</p><pre>$ aws cognito-idp admin-get-user --user-pool-id $USER_POOL_ID --username 9ff6b3de-0f13-48b1-8e29-af4190a1cf2f<br>Enabled: true<br>UserAttributes:<br>- Name: sub<br>  Value: 9ff6b3de-0f13-48b1-8e29-af4190a1cf2f<br>- Name: email_verified<br>  Value: &#39;false&#39;<br>- Name: name<br>  Value: Susan6599<br>- Name: phone_number_verified<br>  Value: &#39;false&#39;<br>- Name: phone_number<br>  Value: &#39;+61437340252&#39;<br>- Name: email<br>  Value: <a href="/service/mailto:Email=oliver.schenk+3aedd2@gmail.com">your_email+3aedd2@account.com</a><br>UserCreateDate: &#39;2022-08-29T22:15:07.324000+08:00&#39;<br>UserLastModifiedDate: &#39;2022-08-29T22:15:07.324000+08:00&#39;<br>UserStatus: CONFIRMED<br>Username: 9ff6b3de-0f13-48b1-8e29-af4190a1cf2f</pre><p>As you can see, the UserStatus is CONFIRMED, but the email_verified field is false. This means the user can login using the username (in this case email or phone number) and password, but if the user were to forget the password then it would not be possible to recover the account.</p><p>We can try resetting the password.</p><pre>$ aws cognito-idp forgot-password --client-id $USER_POOL_CLIENT_ID --username 9ff6b3de-0f13-48b1-8e29-af4190a1cf2f<br>CodeDeliveryDetails:<br>  AttributeName: email<br>  DeliveryMedium: EMAIL<br>  Destination: 9***@h***</pre><p>You might be wondering why we’re getting a successful response? Well, if you look closely, it isn’t actually a successfully response, but rather it is pretending to be.</p><p>This is actually due to the way Cognito handles error responses. In the Terraform code for the user pool client the prevent_user_existence_errors parameter is set to ENABLED which means Cognito will mask the fact that a user with the given username does not exist or that the email and/or phone number has not been verified. This is a security feature <a href="https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-user-pool-managing-errors.html">documented here</a>.</p><h4>AdminCreateUser Handler</h4><p>The AdminCreateUser handler can be triggered using the aws cognito-idp admin-create-user CLI command. To simplify things, I’ve created an admin-create-user.sh script to streamline creating a new random user.</p><p>Unlike the SignUp call, the AdminCreateUser call needs AWS credentials, because it is meant to be used by a user or service that manages the Cognito user pool and its users.</p><p>The following example is a response received if the handler throws an exception. The exception message will be included in the CLI error message.</p><pre>$ ./admin-create-user.sh $USER_POOL_ID <a href="/service/mailto:oliver.schenk@gmail.com">your_email@account.com</a><br>Creating a user with the following details:<br><a href="/service/mailto:Email=oliver.schenk+fb08c0@gmail.com">Email=</a><a href="/service/mailto:Email=oliver.schenk+3aedd2@gmail.com">your_email</a><a href="/service/mailto:Email=oliver.schenk+fb08c0@gmail.com">+fb08c0@account.com</a><br>Phone=+61453330042<br>Name=Susan1348<br>Password=0ddad52a3ead8331ea6bABC!</pre><pre>An error occurred (UserLambdaValidationException) when calling the AdminCreateUser operation: PreSignUp failed with error User could not be created..</pre><p>Let’s see what happens if we don’t throw an exception in the handler (commented out the code that throws an exception in the handler and re-deployed the Lambda).</p><pre>./admin-create-user.sh $USER_POOL_ID <a href="/service/mailto:oliver.schenk@gmail.com">your_email@account.com</a><br>Creating a user with the following details:<br><a href="/service/mailto:Email=oliver.schenk+8ebc46@gmail.com">Email=your_email+8ebc46@account.com</a><br>Phone=+61418115255<br>Name=Susan25233<br>Password=ef7dfcf02365818ee008ABC!</pre><pre>User:<br>  Attributes:<br>  - Name: sub<br>    Value: 74740674-e7aa-4bf7-9bbf-74ff3d80ca98<br>  - Name: name<br>    Value: Susan25233<br>  - Name: phone_number<br>    Value: &#39;+61418115255&#39;<br>  - Name: email<br>    Value: <a href="/service/mailto:Email=oliver.schenk+8ebc46@gmail.com">your_email+8ebc46@account.com</a><br>  Enabled: true<br>  UserCreateDate: &#39;2022-08-31T20:47:00.770000+08:00&#39;<br>  UserLastModifiedDate: &#39;2022-08-31T20:47:00.770000+08:00&#39;<br>  UserStatus: FORCE_CHANGE_PASSWORD<br>  Username: 74740674-e7aa-4bf7-9bbf-74ff3d80ca98<br>Finished.</pre><p>This time the user was created. The user is confirmed, but the status is FORCE_CHANGE_PASSWORD meaning the user must change their password on next login.</p><p>The AdminCreateUser API call has many possible parameters that you can use to control the sign-up and validation flow. The key point is that the trigger response does not control the behaviour of the validation of user attributes. It is the parameters you pass into the AdminCreateUser API call that determines whether an invitation message is sent or not, whether user attributes are automatically verified or not and so on.</p><p>The best way to learn is to experiment with different settings.</p><h4>ExternalProvider Handler</h4><p>The ExternalProvider handler is only called once when a user first signs in using an external provider.</p><p>I’m not going to spend a lot of time here on how to set up an external provider, because that would be another whole article and there are already some out there.</p><p>The key point in terms of the trigger is that this gives you an opportunity to take some action when an external provider is used for the first time.</p><p>You might, for example, check whether the username or email already exists in the user pool and, if so, link it using the <a href="https://docs.aws.amazon.com/cognito-user-identity-pools/latest/APIReference/API_AdminLinkProviderForUser.html">AdminLinkProviderForUser</a> API call.</p><p>You might auto verify the email address, but you might have to manually ask for and verify a phone number, because this wasn’t provided by the external provider. You’ll see many apps these days will use a multi-step user on-boarding process rather than just one large sign-up form, especially because an external provider might only give you the email address and nothing much else.</p><p>Again, this kind of topic is quite a large topic in itself.</p><h3>Other Trigger Implementations</h3><p>Watch <a href="https://awstip.com/cognito-lambda-triggers-in-dotnet-3bf13a55eda3">this space</a> for articles that are dedicated to explanations and examples on how to handle each type of trigger source.</p><p>Follow me or subscribe to receive notification of when these articles are released.</p><p>Thanks for your time!</p><h3>Level Up Coding</h3><p>Thanks for being a part of our community! Before you go:</p><ul><li>👏 Clap for the story and follow the author 👉</li><li>📰 View more content in the <a href="https://levelup.gitconnected.com/?utm_source=pub&amp;utm_medium=post">Level Up Coding publication</a></li><li>🔔 Follow us: <a href="https://twitter.com/gitconnected">Twitter</a> | <a href="https://www.linkedin.com/company/gitconnected">LinkedIn</a> | <a href="https://newsletter.levelup.dev/">Newsletter</a></li></ul><p>🚀👉 <a href="https://jobs.levelup.dev/talent/welcome?referral=true"><strong>Join the Level Up talent collective and find an amazing job</strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4c465c6f81a" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/cognito-lambda-triggers-in-dotnet-presignup-4c465c6f81a">Cognito Lambda Triggers in Dotnet — PreSignUp</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cognito Lambda Triggers in Dotnet]]></title>
            <link>https://levelup.gitconnected.com/cognito-lambda-triggers-in-dotnet-3bf13a55eda3?source=rss-159faff569ab------2</link>
            <guid isPermaLink="false">https://medium.com/p/3bf13a55eda3</guid>
            <category><![CDATA[cognito-user-pools]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[lambda]]></category>
            <dc:creator><![CDATA[Oliver Schenk]]></dc:creator>
            <pubDate>Fri, 19 Aug 2022 15:07:41 GMT</pubDate>
            <atom:updated>2023-03-30T03:49:37.084Z</atom:updated>
            <cc:license>http://creativecommons.org/licenses/by/4.0/</cc:license>
            <content:encoded><![CDATA[<p><a href="https://aws.amazon.com/cognito/">Amazon Cognito</a> is a user sign-up, sign-in and access control service with many <a href="https://aws.amazon.com/cognito/details/">features</a>. One feature that you will almost certainly need to use in any serious solution is the ability to handle Cognito events using <a href="https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-user-identity-pools-working-with-aws-lambda-triggers.html">Lambda triggers</a>.</p><p>This article provides general guidance and an implementation on how to build, deploy, configure and handle Cognito triggers using Dotnet and Terraform.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Eg5vX4yvXFiuSBel8_DjAQ.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@jdent?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Jason Dent</a> on <a href="https://unsplash.com/s/photos/vault?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>The <a href="https://github.com/oliverschenk/cognito-lambda-triggers-dotnet">source code</a> for this project includes a complete Terraform definition for creating a Cognito User Pool and Cognito User Pool Client ID, however these details are explained in more detail in my previous article <a href="https://levelup.gitconnected.com/amazon-cognito-with-sms-user-verification-8c0d90121faa">Amazon Cognito with SMS user verification</a>. In this article the focus will be on what is required to get Lambda triggers up and running using Dotnet.</p><p>The following trigger event deep-dive articles are now available:</p><ul><li><a href="https://medium.com/@oliver.schenk/cognito-lambda-triggers-in-dotnet-presignup-4c465c6f81a">PreSignUp</a></li><li><a href="https://medium.com/@oliver.schenk/cognito-lambda-triggers-in-dotnet-postconfirmation-f1419f4d4952">PostConfirmation</a></li></ul><p><em>I’m working on implementations and examples for each trigger type, so watch this space! More links coming soon …</em></p><h3>Prerequisites</h3><p>To deploy the infrastructure and Cognito trigger Lambda function you will at least need the following:</p><ul><li>An AWS account with admin level access credentials for deploying resources</li><li><a href="https://dotnet.microsoft.com/en-us/download">Dotnet 6.0 SDK</a> with <a href="https://docs.aws.amazon.com/lambda/latest/dg/csharp-package-cli.html">Lambda tools</a></li><li><a href="https://www.terraform.io/downloads">Terraform</a></li></ul><h3>General Tips</h3><p>The <a href="https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-user-identity-pools-working-with-aws-lambda-triggers.html">Amazon Cognito documentation</a> provides a lot of information about triggers. You can read the detailed documentation in your own time, but at a very high level the following considerations are important with respect to a Lambda trigger function:</p><ol><li>It must complete and respond within <strong>5 seconds</strong>. Cognito will attempt <strong>3 retries</strong> and then timeout.</li><li>Except for Custom Sender Lambda triggers, Amazon Cognito invokes Lambda functions <strong>synchronously</strong>.</li><li>It must have a resource-based policy attached that <strong>grants the Congito service permission to invoke</strong> the function.</li><li>Any error messages it generates and returns will be <strong>visible to users</strong> if using the <em>Hosted UI</em> feature.</li><li>The event payload consists of a set of <strong>common parameters</strong> that apply to all triggers and zero or more trigger specific parameters.</li><li>The <a href="https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-user-identity-pools-working-with-aws-lambda-triggers.html#cognito-user-identity-pools-working-with-aws-lambda-trigger-sources"><strong>trigger source</strong></a> parameter is a string that identifies the purpose or category of trigger event.</li></ol><h3>Trigger Event</h3><p>The trigger event is an object that is passed to your trigger handler Lambda function and provides the context of the event and the request information that you can then use in your logic.</p><p>All events have a minimum set of parameters that are called the <a href="https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-user-identity-pools-working-with-aws-lambda-triggers.html#cognito-user-pools-lambda-trigger-syntax-shared">common parameters</a>. These are shown in the code snippet below. Many triggers will have additional request and response parameters.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4999cbbf666483a9e88c8ac6777b84b8/href">https://medium.com/media/4999cbbf666483a9e88c8ac6777b84b8/href</a></iframe><h4>Trigger Source</h4><p>Cognito can be configured to invoke a Lambda trigger handler function for one or more of the available events that Cognito triggers. In other words, you can assign a function to handle just one of the triggers or you can assign a function to more than one trigger to allow that function to handle many different events.</p><p>In addition to there being different triggers, there can also be more than one underlying reason for invoking a trigger. These are documented — although the documentation is a bit messy. The Cognito service and documentation refers to this as the <a href="https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-user-identity-pools-working-with-aws-lambda-triggers.html#cognito-user-identity-pools-working-with-aws-lambda-trigger-sources"><em>Trigger Source</em></a><em>.</em> The trigger source value can be determined through the triggerSource parameter in the trigger event object.</p><p>The decision on which approach to take will depend on your architecture, but most of the time it is safe to say that for each user pool where you have users within a common concern you can keep all of the sign-up, sign-in and related trigger code in one Lambda function.</p><p>The sample project provided as part of this article takes the approach of defining strongly typed handlers for each type of trigger source by inheriting</p><h4>Request and Response</h4><p>When one of the documented Cognito events occurs and you have configured a Lambda trigger handler for that event, Cognito will invoke your function and pass a Cognito event object to your function.</p><p>The request parameters are contained in the request field of the Cognito event object. Based on the trigger source, you must then perform your custom code using the given request data (if relevant) and update the necessary parameters in the response field (if relevant). You must then return the entire Cognito event object.</p><p>There is no Dotnet library provided by AWS for a strongly typed Cognito event object so it is up to us to look at the triggerSource field and based on that deserialize the object to something we can use. The documentation is pretty good at describing the structure of each type of payload based on the trigger source. The main object that can be used in Dotnet is the JsonElement, but we’ll go into more detail shortly.</p><p>The <a href="https://github.com/oliverschenk/cognito-lambda-triggers-dotnet">Github project</a> also includes a sample JSON payload for each type of trigger source.</p><h3>Implementation</h3><p>A Dotnet implementation of trigger event objects can be found in the project code. It provides an implementation of the event object and an associated handler for every single type of trigger source.</p><p>It is up to you to implement the handlers that you need as explained below. You can implement just one of them or more than one.</p><h4>Handler Factory</h4><p>The core purpose of the Lambda function handler and the CognitoEventHandlerFactory is to find and instantiate the appropriate handler with a strongly typed event context at run-time and run the handler logic.</p><p>All you need to do to implement a Cognito Lambda trigger is find the appropriate handler class and inside it override the HandleTriggerEvent method, write your custom logic, set any appropriate values in TriggerEvent.Response (if applicable) and finally return using base.HandletriggerEvent().</p><p>This will automatically serialize the trigger event object back into a JsonElement and return it so that the Lambda function can return it back to Cognito.</p><p>Below is an example of the PreSignUp_PreSignUp trigger handler.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5a2d0baee149ec0ebabaede8620735ea/href">https://medium.com/media/5a2d0baee149ec0ebabaede8620735ea/href</a></iframe><p>If your handler method requires async/await, you can override the HandleTriggerEventAsync method instead.</p><p>If for whatever reason a handler can’t be loaded (maybe AWS invents a new trigger source suddenly), the fallback is the NoActionHandler. This simply returns the original event object without modification.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/de46680b3925573e6a7a8e1ec2fe331b/href">https://medium.com/media/de46680b3925573e6a7a8e1ec2fe331b/href</a></iframe><h4>Common Parameters</h4><p>The common parameters are provided as a CognitoTriggerEventBase implementation. Note that this class is not abstract or generic, because it is used to deserialize the Lambda event context into a strongly typed object. This is then used in the logic to find the correct handler at run-time based on matching the TriggerSource.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/247c71e08ea78bcb68e14d7688c15b96/href">https://medium.com/media/247c71e08ea78bcb68e14d7688c15b96/href</a></iframe><h4>Request and Response Parameters</h4><p>Once the type of a handler is dynamically determined based on the trigger source, the correct implementation of the handler is loaded. Each handler implements CognitoTriggerHander&lt;T&gt; with a specified trigger event.</p><p>This trigger event in turn is an implementation of CognitoTriggerEvent&lt;TRequest, TResponse&gt; where TRequest and TResponse are the strongly typed request and response objects for each handler type.</p><p><em>In summary, you now have a handler that knows exactly what the type of the request and the response is for each trigger source. Pretty cool!</em></p><p>In fact, when you implement a handler, you can simply access the request object using TriggerEvent.Request and the response object using TriggerEvent.Response and all of the parameters will be available and strongly typed.</p><h3>Terraform Infrastructure</h3><p>On the infrastructure side you could use either Cloud Formation or Terraform. I’ve used Terraform in the sample project.</p><p><strong><em>Note:</em></strong><em> I tried to use the </em><a href="http://serverless.com/"><em>Serverless</em></a><em> framework for this, but I had problems with the deployment. The Cloud Formation that Serverless produces did not work for me due to some permission errors that I never managed to resolve.</em></p><p>The main things that you need to do are:</p><ol><li>Create the Lambda zip package. (see deployment script)</li><li>Define a KMS key block.</li><li>Define a Lambda function block.</li><li>Define the triggers in the Cognito User Pool.</li><li>Define permissions for Lambda invocation and usage of the KMS key.</li></ol><p>In the aws_cognito_user_pool configuration you’ll find the lambda_config block, which is where the triggers are set to the Lambda ARN.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/adbee9c174af2949d514919b0ee90365/href">https://medium.com/media/adbee9c174af2949d514919b0ee90365/href</a></iframe><p>Note that technically the KMS key is only required for the <a href="https://docs.aws.amazon.com/cognito/latest/developerguide/user-pool-lambda-custom-sender-triggers.html">Custom Sender Triggers</a>, so if you’re not implementing custom_email_sender or custom_sms_sender then you will not need a KMS key.</p><p>The KEY_ID environment variable is passed to the Lambda function so that you can use this if you need to decrypt the secret code for the custom sender triggers.</p><h4>A note about custom senders</h4><p>Just be aware that if you specify a custom sender Lambda function then you take on the responsibility of actually sending an email or SMS.</p><p>The example in this code only provided limited implementations for some functions. The other handlers are all empty. It would therefore be advisable to comment out any Lambda functions that you are not using in the Terraform code and only enable those that you are testing, otherwise you might get some strange behaviour. For example, you will not receive any emails for sending confirmation codes.</p><h3>Deployment Script</h3><p>A deployment bash script is included in the sample project. It will take care of building, packaging and deployment to AWS.</p><p>You will need to supply an email address you want to use for testing. This is because some of the example code uses AWS SES.</p><p>If you want to supply an AWS credential profile you can use the -p flag.</p><pre># example using -p flag<br>./deploy.sh -e your_email@account.com -p &lt;aws-profile-name&gt;</pre><p>There is also a destroy.sh script to tear down the AWS infrastructure.</p><h3>Trigger Implementations</h3><p>Watch this space for articles that are dedicated to explanations and examples on how to handle each type of trigger source.</p><p>The following trigger event deep-dive articles are now available:</p><ul><li><a href="https://medium.com/@oliver.schenk/cognito-lambda-triggers-in-dotnet-presignup-4c465c6f81a">PreSignUp</a></li><li><a href="https://medium.com/@oliver.schenk/cognito-lambda-triggers-in-dotnet-postconfirmation-f1419f4d4952">PostConfirmation</a></li></ul><p>Follow me or subscribe to receive notification of when these articles are released.</p><p>Thanks for your time!</p><h3>Level Up Coding</h3><p>Thanks for being a part of our community! Before you go:</p><ul><li>👏 Clap for the story and follow the author 👉</li><li>📰 View more content in the <a href="https://levelup.gitconnected.com/?utm_source=pub&amp;utm_medium=post">Level Up Coding publication</a></li><li>🔔 Follow us: <a href="https://twitter.com/gitconnected">Twitter</a> | <a href="https://www.linkedin.com/company/gitconnected">LinkedIn</a> | <a href="https://newsletter.levelup.dev/">Newsletter</a></li></ul><p>🚀👉 <a href="https://jobs.levelup.dev/talent/welcome?referral=true"><strong>Join the Level Up talent collective and find an amazing job</strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3bf13a55eda3" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/cognito-lambda-triggers-in-dotnet-3bf13a55eda3">Cognito Lambda Triggers in Dotnet</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AWS SaaS — Useful Development Resources]]></title>
            <link>https://medium.com/@oliver.schenk/aws-saas-useful-development-resources-15958a998387?source=rss-159faff569ab------2</link>
            <guid isPermaLink="false">https://medium.com/p/15958a998387</guid>
            <category><![CDATA[saas]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Oliver Schenk]]></dc:creator>
            <pubDate>Wed, 27 Jul 2022 11:48:15 GMT</pubDate>
            <atom:updated>2022-08-01T23:36:16.807Z</atom:updated>
            <cc:license>http://creativecommons.org/publicdomain/zero/1.0/</cc:license>
            <content:encoded><![CDATA[<h3>AWS SaaS — Useful Development Resources</h3><p>I’ve always been interested in how SaaS solutions and platforms are architected and built, especially within the world of AWS. There’s a lot of generic learning material out there to start understanding the theory and building blocks of SaaS. Below is a list of information that helped me get a handle on what building blocks are required for a SaaS solution.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HKaB6k2whF0myXHDhTIADw.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@charles_forerunner?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Charles Forerunner</a> on <a href="https://unsplash.com/s/photos/platform?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><h3>Articles and Websites</h3><ul><li><a href="https://docs.aws.amazon.com/wellarchitected/latest/saas-lens/wellarchitected-saas-lens.pdf">SaaS Lens: Well Architected Framework</a></li><li><a href="https://docs.aws.amazon.com/whitepapers/latest/saas-solutions-tenant-isolation-architectures/welcome.html">SaaS Solutions on AWS: Tenant Isolation Architectures</a></li><li><a href="https://d1.awsstatic.com/whitepapers/saas-tenant-isolation-strategies.pdf">SaaS Tenant Isolation Strategies</a></li><li><a href="https://aws.amazon.com/partners/programs/saas-factory/">AWS SaaS Factory Program</a></li><li><a href="https://d1.awsstatic.com/whitepapers/saas-journey-framework.pdf">Saas Journey Framework</a></li><li><a href="https://aws.amazon.com/blogs/apn/architecting-successful-saas-understanding-cloud-based-software-as-a-service-models/">Architecting Successful SaaS: Understanding Cloud-Based Software-as-a-Service Models</a></li><li><a href="https://aws.amazon.com/blogs/apn/architecting-successful-saas-interacting-with-your-saas-customers-cloud-accounts/">Architecting Successful SaaS: Interacting with Your SaaS Customer’s Cloud Accounts</a></li><li><a href="https://catalog.us-east-1.prod.workshops.aws/workshops/b0c6ad36-0a4b-45d8-856b-8a64f0ac76bb/en-US">AWS Serverless SaaS Workshop</a></li><li><a href="http://www.saasresources.com/">SaaSresources.com</a></li><li><a href="https://dev.to/gunnargrosch/resources-for-building-your-saas-product-on-aws-40ke">Resources for building your SaaS product on AWS</a></li></ul><h3>YouTube Videos</h3><ul><li><a href="https://www.youtube.com/watch?v=j7Sqt8GpYl0">AWS re:Invent 2021 — SaaS architecture patterns: From concept to implementation</a></li><li><a href="https://www.youtube.com/watch?v=8Z5zBsKgTxY">AWS re:Invent 2021 — Inside a working serverless SaaS reference solution</a></li><li><a href="https://www.youtube.com/watch?v=fuDZq-EspNA">AWS re:Invent 2019: SaaS tenant isolation patterns (ARC372-P)</a></li><li><a href="https://www.youtube.com/watch?v=AOfuZN5yo38">AWS re:Invent 2019: [REPEAT 1] Microservices decomposition for SaaS environments (ARC210-R1)</a></li><li><a href="https://www.youtube.com/watch?v=mwQ5lipGTBI">AWS re:Invent 2018: Deconstructing SaaS: Building Multi-Tenant Solutions on AWS (ARC418-R1)</a></li><li><a href="https://www.youtube.com/watch?v=mPXxypb5UJg">How to Build Modern SaaS Applications on AWS</a></li><li><a href="https://www.youtube.com/watch?v=rOcjdFdTEAs">AWS Summit Online ASEAN 2021 | Building a successful SaaS startup on AWS</a></li><li><a href="https://www.youtube.com/watch?v=kfSFODrdVus">Building SaaS on AWS — Deep Dive: Architecting multi-tenant SaaS on EKS</a></li></ul><h3>Sample Code</h3><ul><li><a href="https://github.com/awslabs/aws-saas-boost">AWS SaaS Boost Github</a></li><li><a href="https://github.com/aws-samples/aws-saas-factory-serverless-workshop">AWS SaaS Factory Monolith to Serverless SaaS Workshop</a></li><li><a href="https://github.com/aws-samples/aws-serverless-saas-workshop">AWS Serverless SaaS Workshop</a></li><li><a href="https://github.com/aws-samples/aws-saas-factory-eks-reference-architecture">EKS SaaS — Reference Solution</a></li><li><a href="https://github.com/async-labs/saas">SaaS Boilerplate Book — Repo</a></li></ul><h3>Books</h3><ul><li><a href="https://builderbook.org/book">SaaS Boilerplate Book</a></li><li><a href="https://www.amazon.com.au/SaaS-Storage-Strategies-AWS-Whitepaper-ebook/dp/B01MRKZSOG">SaaS Storage Strategies (AWS Whitepaper)</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=15958a998387" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AWS Cloud9 multi-account and role based access]]></title>
            <link>https://levelup.gitconnected.com/aws-cloud9-multi-account-and-role-based-access-c5dfe6c88956?source=rss-159faff569ab------2</link>
            <guid isPermaLink="false">https://medium.com/p/c5dfe6c88956</guid>
            <category><![CDATA[cloud9]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[aws-iam]]></category>
            <dc:creator><![CDATA[Oliver Schenk]]></dc:creator>
            <pubDate>Wed, 13 Jul 2022 06:12:35 GMT</pubDate>
            <atom:updated>2022-09-06T11:27:03.030Z</atom:updated>
            <cc:license>http://creativecommons.org/publicdomain/zero/1.0/</cc:license>
            <content:encoded><![CDATA[<p>I have multiple workstations and I often switch between Linux and Windows, because there are different things I like to do in each OS. It’s a bit of a pain to have to keep the development tools and environments up-to-date on each of these systems.</p><p>So … in the last few days, I decided to give <a href="https://aws.amazon.com/cloud9/">Cloud9</a> a go.</p><p>For those who haven’t heard of it, it is AWS’s answer to a cloud hosted Integrated Development Environment (IDE).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*51Qq3C92NZ11C9GATWv3ug.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@riku?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Riku Lu</a> on <a href="https://unsplash.com/s/photos/ide?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>As with many AWS services, it’s pretty easy to get started and everything works quite well for basic use cases. If you have an AWS account you simply login to the console, create the Cloud9 environment and you’ll be able to access services within your account based on what IAM policies are attached to your user account. All of that right in your browser!</p><p>However, I had a use case that took a bit of work to get going and Google wasn’t much of a friend this time.</p><p>Here’s my situation:</p><ol><li>I don’t just have one account, I have a <em>management</em> account, an <em>infosec</em> account and then some <em>dev</em>, <em>test</em> and <em>prod</em> accounts.</li><li>Users and user groups only exist in the <em>infosec</em> account. All other access is assumed using role based access.</li><li>Cloud9 is deployed in the <em>infosec</em> account, because that’s where users login and that’s the central account for such things.</li><li>I normally use <a href="https://github.com/99designs/aws-vault">aws-vault</a> for executing AWS CLI commands on my local machine, which supports logging in to the console as a specific role.</li></ol><p>I wanted to host my Cloud9 instance in the <em>infosec</em> account and then assume roles in other accounts (mainly <em>dev</em>) to deploy resources. One way to get around this of course would be to create a dedicated sandbox account for each developer in which to host both the Cloud9 instance and any resources created as part of development.</p><p>Deployments to the <em>test</em> and <em>prod </em>accounts should ideally be done using a CI/CD pipeline outside of Cloud9, if not to the <em>dev </em>account as well.</p><h4>Don’t create a Cloud9 environment using aws-vault login</h4><p>The first thing that really stuffed me up was to use aws-vault to login to the console and then create a Cloud9 environment.</p><p>The issue is that when aws-vault launches the browser window and logs you into the console, it does so using a federated user and the ARN of this role is temporary and short lived. Once your session expires, the next time you login, the ARN will be different. You can see in the image below that the ARN contains a timestamp so this will change each time you login again.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/754/1*EcOd2Y74ZuKhoQturibIfQ.png" /></figure><p>When creating a Cloud9 instance it will work fine whilst the session is active, but because the instance <em>owner</em> is an ARN that will never exist again you will not be able to launch your Cloud9 instance ever again either.</p><p>Initially it will look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/962/1*CD3VscvUc1BBBYtKDa9tvQ.png" /></figure><p>But after logging out and back in again, the ARN will be different and will no longer match the <em>Owner Arn</em> shown above. The <em>Permissions</em> will be <em>None</em> and the <em>Open IDE</em> button in the console will be greyed out.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/962/1*yTlFh9U2aJ9knZkvQAQ01w.png" /></figure><p>What I had to do in the end was login to the console using a normal IAM user without using aws-vault. At least that got me into the <em>infosec</em> account as myself and not as a short lived entity.</p><h4>Attach the <em>AWSCloud9Administrator</em> policy</h4><p>The issue now is that even though I’m logged in, I still don’t really have any permissions. I need to give myself the <em>AWSCloud9Administrator</em> or <em>AWSCloud9User </em>permission so that I can create an environment for myself.</p><p>To do that I assumed an <em>Administrator</em> role in the <em>infosec</em> account and attached the <em>AWSCloud9Administrator </em>policy to my user account. (I was cutting corners — normally I would create a user group and then attach the policy to this and add myself to that group.)</p><p>Now that I have this permission, I can switch back out of the Administrator role that I assumed and go ahead to the Cloud9 console to create a new environment.</p><h4>AWS managed temporary credentials</h4><p>AWS is trying to help you out by managing credentials for you. What Cloud9 does is it rotates credentials automatically every 5 minutes. These are called <a href="https://docs.aws.amazon.com/cloud9/latest/user-guide/security-iam.html#auth-and-access-control-temporary-managed-credentials">AWS managed temporary credentials</a>. It’s a similar concept to aws-vault, but it does it all behind the scenes.</p><p>The first problem is that these credentials have the same level of access as the currently logged in user, which, in my case, means almost no permissions at all.</p><p>This is what I get in the Cloud9 environment when I query my own identity:</p><pre>OliverSchenk-Admin:~ $ aws sts get-caller-identity<br>{<br>    &quot;UserId&quot;: &quot;xxxxxxxxxxxxxxxxxxxxx&quot;,<br>    &quot;Account&quot;: &quot;xxxxxxxxxxxx&quot;,<br>    &quot;Arn&quot;: &quot;arn:aws:iam::xxxxxxxxxxxx:user/OliverSchenk-Admin&quot;<br>}</pre><p>The second problem is that these credentials are restricted and only allow a limited set of AWS service calls. The <a href="https://docs.aws.amazon.com/cloud9/latest/user-guide/security-iam.html#auth-and-access-control-temporary-managed-credentials">documentation lists the calls that are allowed to be made</a> and assuming a role is not one of them. If I try to assume a role it will fail.</p><pre>OliverSchenk-Admin:~ $ aws sts assume-role --role-arn arn:aws:iam::&lt;dev_account_id&gt;:role/Administrator --role-session-name OliverSchenk-Admin-Dev</pre><pre>An error occurred (InvalidClientTokenId) when calling the AssumeRole operation: The security token included in the request is invalid</pre><h4>Next best option</h4><p>Given assuming a role is not possible with the default environment credentials, AWS advises that the next best thing is to turn off the managed temporary credentials and instead attach an EC2 instance profile to the EC2 instance running the Cloud9 environment.</p><p>What this effectively means is that Cloud9 will no longer generate the credentials for us based on its allowed permissions, but instead we can create our own IAM role and associated policy to govern what our Cloud9 environment is allowed to access. We can then use the metadata of the EC2 instance as a credentials source. This is still better than hard coding a fixed set of AWS credentials in a configuration file.</p><h4>Turn off managed temporary credentials</h4><p>Turn off managed temporary credentials in the <em>Preferences</em> tab by clicking on the cog icon in the top right corner. In <em>AWS Settings &gt; Credentials</em> turn off <em>AWS managed temporary credentials</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TGedaYRoKibvhmevE9v8cA.png" /></figure><p>This will remove the ~/.aws/credentials file and Cloud9 will no longer have any credentials configured. When I try to call an AWS service I now get something like this:</p><pre>OliverSchenk-Admin:~ $ aws sts get-caller-identity</pre><pre>Unable to locate credentials. You can configure credentials by running &quot;aws configure&quot;.</pre><h4>Create and attach an instance profile</h4><p>The next thing to do is to create a new IAM policy and role and then attach this to the EC2 instance in the EC2 console. You will obviously configure whatever permissions you want your environment to have, but all I really wanted was to be able to assume a role in the <em>dev</em> environment. You can of course attach as many policies as you need to your role.</p><p>The following policy allows me to assume the Administrator role in the <em>dev</em> account. (This assumes I’ve correctly set up cross-account policies already — something for another post!)</p><pre>{<br>    &quot;Statement&quot;: [<br>        {<br>            &quot;Action&quot;: &quot;sts:AssumeRole&quot;,<br>            &quot;Effect&quot;: &quot;Allow&quot;,<br>            &quot;Resource&quot;: &quot;arn:aws:iam::&lt;dev_account_id&gt;:role/Administrator&quot;,<br>            &quot;Sid&quot;: &quot;AssumeDevAdministratorRole&quot;<br>        }<br>    ],<br>    &quot;Version&quot;: &quot;2012-10-17&quot;<br>}</pre><p>I attached the above to a new role and also had to set up a trust relationship to allow the EC2 instance to assume the role:</p><pre>{<br>    &quot;Version&quot;: &quot;2012-10-17&quot;,<br>    &quot;Statement&quot;: [<br>        {<br>            &quot;Effect&quot;: &quot;Allow&quot;,<br>            &quot;Principal&quot;: {<br>                &quot;Service&quot;: &quot;ec2.amazonaws.com&quot;<br>            },<br>            &quot;Action&quot;: &quot;sts:AssumeRole&quot;<br>        }<br>    ]<br>}</pre><p>Now all that’s left to do is to go into the EC2 instance, find the Cloud9 instance, select it then click on <em>Actions &gt; Security &gt; Modify IAM Role</em> and select the role I just created and <em>Update IAM Role</em>.</p><p>In the Cloud9 environment I can now see the role that was assumed by the EC2 instance.</p><pre>OliverSchenk-Admin:~ $ aws sts get-caller-identity<br>{<br>    &quot;UserId&quot;: &quot;AROAYIR3NLRT3B7FQ7XGV:i-05a3b97d2929b28d0&quot;,<br>    &quot;Account&quot;: &quot;&lt;infosec_account_id&gt;&quot;,<br>    &quot;Arn&quot;: &quot;arn:aws:sts::&lt;infosec_account_id&gt;:assumed-role/OliverSchenk-Dev-Account-Cloud9-Instance-Profile/i-05a3b97d2929b28d0&quot;<br>}</pre><h4>Set up an AWS profile</h4><p>I created an AWS configuration profile that will assume the role in the <em>dev</em> account. In the Cloud9 terminal I edited the file ~/.aws/config and added something like this:</p><pre>[profile OliverSchenk-Admin-Dev]<br>region = ap-southeast-2<br>output = yaml<br>role_arn = arn:aws:iam::&lt;dev_account_id&gt;:role/Administrator<br>credential_source = Ec2InstanceMetadata</pre><p>The credential_source takes the credentials of the underlying EC2 instance and the role_arn tells it what role to assume.</p><p>I tested the caller identity again using the --profile parameter. I got something like this:</p><pre>OliverSchenk-Admin:~ $ aws sts get-caller-identity --profile OliverSchenk-Admin-Dev<br>Account: &#39;&lt;dev_account_id&gt;&#39;<br>Arn: arn:aws:sts::&lt;dev_account_id&gt;:assumed-role/Administrator/botocore-session-1657689540<br>UserId: AROAQD4TDLBZ2Wxxxxxxx:botocore-session-1657689540</pre><p>Looks like I have now successfully assumed a role in the <em>dev</em> account from the Cloud9 environment running in the <em>infosec</em> account.</p><p>Rather than using the --profile parameter it is also possible to set the AWS_PROFILE environment variable so that any script that contains AWS CLI commands will run under the given profile.</p><pre>OliverSchenk-Admin:~ $ export AWS_PROFILE=OliverSchenk-Admin-Dev<br>OliverSchenk-Admin:~ $ aws sts get-caller-identity<br>Account: &#39;&lt;dev_account_id&gt;&#39;<br>Arn: arn:aws:sts::&lt;dev_account_id&gt;:assumed-role/Administrator/botocore-session-1657689540<br>UserId: AROAQD4TDLBZ2Wxxxxxxx:botocore-session-1657689540</pre><h4>View the Cloud9 IDE and AWS consoles in other accounts at the same time</h4><p>One of the problems I encountered was that I couldn’t login as two different AWS entities or accounts in the same browser at the same time. In other words I couldn’t login to the Cloud9 instance and then also login to the <em>dev </em>account<em>.</em></p><p>There are a few ways to get around this. You can either use different browsers, you can use incognito mode, you can use different ‘people’ configurations in Chrome or you can use browser plugins such as Firefox multi-account containers.</p><h4>AWS CDK CLI needs a credentials file</h4><p>Unlike the AWS CLI, the AWS CDK CLI has some issues when it comes to detecting credentials. Because I turned off temporary managed credentials and I’m using EC2 instance credentials together with a role based approach I did not have a ~/.aws/credentials file in my environment. AWS CDK does not like this so just create an empty one with touch ~/.aws/credentials and that should sort you out.</p><p><em>And that’s all for today, thanks! Leave me some comments if you have corrections, tips or suggestions.</em></p><h3>Level Up Coding</h3><p>Thanks for being a part of our community! Before you go:</p><ul><li>👏 Clap for the story and follow the author 👉</li><li>📰 View more content in the <a href="https://levelup.gitconnected.com/?utm_source=pub&amp;utm_medium=post">Level Up Coding publication</a></li><li>🔔 Follow us: <a href="https://twitter.com/gitconnected">Twitter</a> | <a href="https://www.linkedin.com/company/gitconnected">LinkedIn</a> | <a href="https://newsletter.levelup.dev/">Newsletter</a></li></ul><p>🚀👉 <a href="https://jobs.levelup.dev/talent/welcome?referral=true"><strong>Join the Level Up talent collective and find an amazing job</strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c5dfe6c88956" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/aws-cloud9-multi-account-and-role-based-access-c5dfe6c88956">AWS Cloud9 multi-account and role based access</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Amazon Cognito with SMS user verification]]></title>
            <link>https://levelup.gitconnected.com/amazon-cognito-with-sms-user-verification-8c0d90121faa?source=rss-159faff569ab------2</link>
            <guid isPermaLink="false">https://medium.com/p/8c0d90121faa</guid>
            <category><![CDATA[angular]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[cognito]]></category>
            <dc:creator><![CDATA[Oliver Schenk]]></dc:creator>
            <pubDate>Mon, 20 Jun 2022 14:39:45 GMT</pubDate>
            <atom:updated>2022-08-03T14:15:14.492Z</atom:updated>
            <cc:license>http://creativecommons.org/publicdomain/zero/1.0/</cc:license>
            <content:encoded><![CDATA[<h4>Includes a working Angular SPA sample app</h4><p><a href="https://aws.amazon.com/cognito/">Amazon Cognito</a> is a secure and scalable service to manage sign-up, sign-in and access control for mobile and web apps. There are many <a href="https://aws.amazon.com/cognito/details/">features</a> and the full breadth of the service can be discovered in the <a href="https://docs.aws.amazon.com/cognito/latest/developerguide/what-is-amazon-cognito.html">documentation</a>.</p><p>In this article, we are going to create complete solution that demonstrates user sign-up and sign-in through an Angular front-end app and a Cognito User Pool back-end deployed using Terraform.</p><p>The source code is available in <a href="https://github.com/oliverschenk/aws-cognito-angular/tree/amazon-cognito-with-sms-user-verification">Github</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/681/1*XKC4s2Wh_Bi7k1R7L5EKcw.png" /></figure><p>It is worth noting that Cognito provides a lot of functionality, flexibility and customisation. However, this also comes at the cost of complexity.</p><p>Rather than trying to dive into every possible feature, it is usually best to think about what it is that you want to achieve in your app and then research from there. Think business process, then data and application requirements and finally choose the technology that meets those requirements. Cognito is of course not the only solution on the market — there’s also <a href="https://auth0.com/">Auth0</a>, <a href="https://www.okta.com/au/">Okta</a>, <a href="https://firebase.google.com/">Firebase</a>, <a href="https://azure.microsoft.com/en-au/services/active-directory/">Azure AD</a>, etc…</p><p>As an alternative approach, AWS provides a service called <a href="https://aws.amazon.com/amplify/">AWS Amplify</a>, which, alongside a whole raft of common app features, includes user management and authentication as one of its abstractions. It is designed to hide away the complexities of Cognito and promote a rapid development style.</p><p>The cool thing is that we can still leverage the power of the Amplify libraries without actually having to go for the full Amplify service. In this article we will make use of the AWS Amplify Auth library.</p><h3><strong>What we will build</strong></h3><p>The features that we are looking to implement include the following:</p><ul><li>Self-service user sign-up based on phone number</li><li>Account activation using SMS verification code</li><li>User sign-in and sign-out</li><li>Custom user attributes</li><li>View user profile</li><li>Password reset</li></ul><h3>Solution architecture</h3><p>The solution consists of two main parts. The first is the back-end Cognito infrastructure deployed in AWS using Terraform and the second is the front-end Angular app that will interact with the Cognito service.</p><p>A diagram of what will be built in this solution and what processes it will support is shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/674/1*iA8eRF6_kGF9C5V6GpKGjw.png" /></figure><h4>Back-end</h4><p>The primary component in the back-end is a Cognito User Pool and the associated configuration to support the features we’ve specified above.</p><p>Cognito can be configured for many different use cases. Any technology that requires strong, yet customisable, security will inevitably have a higher degree of complexity both conceptually and practically.</p><p>All of the features that we want in regards to unauthenticated and authenticated user management can be achieved using the following AWS resources:</p><ul><li><strong>Cognito User Pool<br></strong>This is the primary user directory resource in Cognito.</li><li><strong>Cognito User Pool App Client<br></strong>This is represents a client application that can make Cognito API calls as an unauthenticated client such as registering, sign-in and password resets. In our case the user pool client is the Angular front-end app.</li><li><strong>SMS Role<br></strong>This is a role that Cognito will assume to send an SMS via Amazon SNS.</li></ul><p>We will go into each of the above in more detail in the implementation below.</p><h4>Front-end</h4><p>The front-end represents the user facing application. In this guide the front-end has been created as a demonstration of the features that relate specifically to interactions with the Cognito service.</p><p>The application in this guide was written using Angular, however the same principles apply to any “public” client that runs in a browser or mobile app. Cognito also supports app clients that can be considered “confidential” such as server-side rendered applications or back-end scripts.</p><p>The application uses a custom UI design rather than the <a href="https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-user-pools-app-integration.html">Cognito Hosted UI</a>. This means the communication with Cognito has to be handled manually. This was achieved using the <a href="https://www.npmjs.com/package/@aws-amplify/auth">AWS Amplify Auth</a> library, which under the hood makes use of the <a href="https://www.npmjs.com/package/amazon-cognito-identity-js">amazon-cognito-identity-js</a> library.</p><h3>Prerequisites</h3><ul><li><a href="https://www.terraform.io/intro">Terraform</a></li><li><a href="https://nodejs.org/">Node.JS</a></li><li>[optional] <a href="https://github.com/99designs/aws-vault">aws-vault</a> (for credentials management)</li><li>[optional] Redux DevTools Chrome extension (for visualising NgRx actions and state)</li><li>A <a href="https://docs.aws.amazon.com/sns/latest/dg/sns-sms-sandbox.html">verified phone number</a> in the SMS Sandbox in your AWS account</li></ul><h3>Back-end implementation</h3><p>Alright, let’s get onto building some infrastructure using Terraform. This section assumes that you are at least somewhat familiar with Terraform and how files are generally defined and organised.</p><p>In the root of your repo create the <em>backend</em> folder with some some empty files.</p><pre>mkdir backend<br>cd backend<br>touch variables.tf main.tf outputs.tf provider.tf</pre><p>First of all let’s set up the provider in the provider.tf file. If you want to load credentials from a specific profile you can provide the profile = &lt;credentials_profile&gt; in the provider block. However, for better security you might want to look at something like <a href="https://github.com/99designs/aws-vault">aws-vault</a> and assume roles to access resources in the desired account.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d93b9cbb00817e754a1ebd8a0db0a7c0/href">https://medium.com/media/d93b9cbb00817e754a1ebd8a0db0a7c0/href</a></iframe><p>Now define some variables in variables.tf.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9190c3d934ccd517daf5b03d9a31390d/href">https://medium.com/media/9190c3d934ccd517daf5b03d9a31390d/href</a></iframe><p>Now let’s define our main resources in main.tf.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/bdcc4883abd75b1e3c9096588b173612/href">https://medium.com/media/bdcc4883abd75b1e3c9096588b173612/href</a></iframe><p>Finally, define the outputs in outputs.tf, which are needed to configure the front-end.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/56bdbf84c1a691b0a985290971313646/href">https://medium.com/media/56bdbf84c1a691b0a985290971313646/href</a></iframe><h4>Locals</h4><p>In the locals section of the main.tf file I’ve defined an id that is used throughout for resource naming purposes. You can name your resources whatever you want by adjusting the project_name and stage variables. It is good practice to include a stage variable as it clearly shows whether a resource is, for example, dev or prod.</p><h4><strong>Cognito user pool</strong></h4><p>The User Pool in Cognito represents the main user directory. This is where users are created and managed. The following list highlights the important aspects of the configuration:</p><ul><li>The username_attributes property allows you to specify which <a href="https://docs.aws.amazon.com/cognito/latest/developerguide/user-pool-settings-attributes.html">user pool attributes</a> are allowed to be used as the username when signing up. This can be email, phone_number, both (meaning either can be used, but then your app will need more sophisticated logic for authentication) or empty (meaning your username is explicitly provided by your users as a string). In our case we want the user to sign-up using their phone number so we have set this to phone_number.</li><li>In apps where users are able to register themselves, the email address and/or phone number is often a required field and is then verified in order to help mitigate against bogus or fake accounts being activated. In Cognito there are a few ways to achieve this. One way is to implement a <a href="https://docs.aws.amazon.com/cognito/latest/developerguide/user-pool-lambda-pre-sign-up.html">pre sign-up Lambda trigger</a> and write your own logic to send out and handle the verification of a user. The other way is to use Cognito’s auto_verified_attributes property to let Cognito send out the necessary verification email or SMS depending on whether email or phone_number is specified. In our case we want this to be set to phone_number so that Cognito deals with creating a verification code and then uses SNS to send an SMS to the given phone number.</li><li>To ensure that users are allowed to register themselves, rather than just via the admin console or the API, the allow_admin_create_user_only property should be set to false.</li><li>For an explanation of the sms_configuration configuration block see the SNS role section below.</li><li>The password_policy configuration block defines the password policy and the lifetime of any temporary password or code. This is <a href="https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-cognito-userpool-passwordpolicy.html">documented here</a>.</li><li>The account_recovery_setting configuration block is used to define password recovery mechanism. In our case the name property should be set to verified_phone_number to ensure that the user can recover their password using their verified phone number.</li><li>The schema configuration blocks are used to define the user attributes. This includes both <a href="https://docs.aws.amazon.com/cognito/latest/developerguide/user-pool-settings-attributes.html">standard and custom attributes</a>. Note that in the Terraform source code it is not possible to tell which attributes are standard attributes and which are custom attributes. However, once the user pool is created the custom attributes will automatically be prefixed by custom:in Cognito and need to be referenced as such in our code.</li></ul><h4><strong>SNS IAM role</strong></h4><p>In order to allow Cognito to send SMS messages it must be allowed to assume a role that includes the relevant policy for publishing via Amazon SNS. The ARN of the role is given in the sms_configuration configuration block sns_caller_arn of the user pool.</p><p>For additional security, the role is configured with an sts:ExternalId parameter that is also given to the user pool sms_configuration object via the external_id parameter. What this means is that when Cognito assumes the role it will pass the ExternalID parameter, which will then be validated against the role’s expected ExternalID. In our case the external ID is randomly generated by Terraform using the random_string resource.</p><h4>Cognito user pool client</h4><p>A Cognito user pool supports a variety of <a href="https://awscli.amazonaws.com/v2/documentation/api/latest/reference/cognito-idp/index.html#cli-aws-cognito-idp">API calls</a>. Some of these API calls can be performed via an application in an unauthenticated user state. Sign-up, email/phone number confirmation and password reset are examples of such operations. These calls must be done within the context of a Cognito user pool client by providing the Client ID as a parameter in an API call.</p><p>The purpose of a user pool client configuration is to specify what authentication flows are allowed to be used by a particular client, how token expiry is managed, what callback URLs are used and other OAuth related settings.</p><p>A user pool can have many user pool client configurations attached to it. This is because even though different types of apps might access the same user pool, they can, and probably will, have different authentication requirements depending on the nature of the app (e.g. front-end or back-end, third party integration, etc…)</p><p>The following choices were made for the solution being built in this guide:</p><ul><li>The <a href="https://docs.aws.amazon.com/cognito-user-identity-pools/latest/APIReference/API_CreateUserPoolClient.html#CognitoUserPools-CreateUserPoolClient-request-ExplicitAuthFlows">explicit auth flows</a> are set to ALLOW_USER_SRP_AUTH and ALLOW_REFRESH_TOKEN_AUTH. This allows for a secure username and password authentication method as well as refresh tokens to obtain new tokens when existing tokens have expired.</li><li>Setting the parameter enable_token_revocation to true allows for token revocation via a number of methods as <a href="https://docs.aws.amazon.com/cognito/latest/developerguide/token-revocation.html">documented</a>.</li><li>Setting the parameters prevent_user_existence_errors to ENABLED ensures that error messages do not give away the fact that a user does not exist. This is documented further <a href="https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-user-pool-managing-errors.html">here</a>. It is more secure to simply say “incorrect username or password” than to say “user not found”, because it allows real phone numbers to be discovered.</li></ul><h4>Testing</h4><p>To test the Cognito User Pool we can use the AWS CLI. The first thing we can do is try to register a new user.</p><p><em>Note that I’m using aws-vault for credentials management.</em></p><pre>aws-vault exec OliverSchenk-Admin-Dev -- aws cognito-idp sign-up --client-id 9ocqafeo835vplo3lns7pc7ub --username +61400000000 --password TestPassword1 --user-attributes Name=&quot;name&quot;,Value=&quot;Oliver Schenk&quot; Name=&quot;custom:company&quot;,Value=&quot;ACME&quot; Name=&quot;custom:role_name&quot;,Value=&quot;Inventor&quot;</pre><ul><li>Replace the client-id value with your own client ID</li><li>Replace the username value with your own test phone number</li><li>Replace the password value with whatever password you want</li><li>Replace the other values for name, custom:company and custom:role_name with whatever values you want</li></ul><p>At this point you should get an SMS message on your registered sandbox phone number containing a 6 digit code. If not, check your phone number includes the country code and that your AWS account has the SMS sandbox enabled.</p><p>You can now verify the phone number by providing the code with another AWS CLI command.</p><pre>aws-vault exec OliverSchenk-Admin-Dev -- aws cognito-idp confirm-sign-up --client-id 9ocqafeo835vplo3lns7pc7ub --username +61400000000 --confirmation-code 123456</pre><ul><li>Replace the client-id value with your own client ID</li><li>Replace the username value with your own phone number</li><li>Replace the confirmation code with the code you received in the SMS message</li></ul><p>At this point the user is now verified.</p><p>We will not be able to easily test the authentication part using the AWS CLI, because the configured authentication method uses SRP which is not easily calculated without using code and a library. In the next section we’ll see how to use the <a href="https://www.npmjs.com/package/amazon-cognito-identity-js">amazon-cognito-identity-js</a> library to interact with the Cognito user pool.</p><h3>Front-end implementation</h3><p>Let’s now focus on the front-end. To make things a little bit more interesting I used the <a href="https://ionicframework.com/">Ionic</a> framework and <a href="https://ngrx.io/">NgRx</a> for reactive style app state management. You will find the entire source code in the <a href="https://github.com/oliverschenk/aws-cognito-angular/tree/amazon-cognito-with-sms-user-verification">GitHub repo</a>.</p><p>I’m not going to paste every single source code file in the article body, but I provided some of the important parts that relate to the Cognito implementation and anything else that I encountered that I had to resolve.</p><h4>Cognito user pool and Cognito client ID</h4><p>As an output of the Cognito User Pool and User Pool Client resources created using Terraform you will get a Cognito user pool ID and a Cognito user pool client ID. These should be provided in the environment.ts file as follows. You should replace these with your own details.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/990a4a3e68888e21dad51e402a421d27/href">https://medium.com/media/990a4a3e68888e21dad51e402a421d27/href</a></iframe><p>You will most likely have a dev and prod stage at least and therefore the production Cognito IDs would go into environment.prod.ts.</p><h4>Running the sample app</h4><p>You can run the sample app by installing Ionic CLI and serving up the app using the following commands.</p><pre>npm install -g @ionic/cli</pre><pre>cd frontend/aws-cognito-angular<br>ionic serve</pre><p>It is best to use the browser’s development tool and run the application in mobile app view size or resize the browser window.</p><h4>Auth module</h4><p>All of the authentication related logic and UI is contained within the authentication module stored in the src\app\core\auth folder. This consists of the following:</p><ul><li><strong>components</strong> — The Angular reusable components used by the auth pages.</li><li><strong>pages</strong> — The Angular components that represent Ionic pages.</li><li><strong>services</strong> — The main auth services and guards.</li><li><strong>state</strong> — The NgRx actions, effects, models, reducers and selectors for authentication actions and page state actions.</li><li><strong>auth.module </strong>— The Angular module definition and the root of the NgRx imports for the auth feature store and effects.</li><li><strong>auth.routes</strong> — The page routing definition.</li></ul><h4>Auth service</h4><p>The interaction with Cognito in this sample app is being handled through a service that makes use of the <a href="https://www.npmjs.com/package/@aws-amplify/auth">@aws-amplify/auth</a> library and the documentation on the <a href="https://docs.amplify.aws/lib/auth/getting-started/q/platform/js/#option-2-call-authentication-apis-manually">Amplify website</a>.</p><p>The service provides a uniform authentication and user management API that hides a few of the Cognito details and communications. It provides all the standard functions that would be expected from an app including sign-up, sign-in, sign-out, password management and user profile management.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a4dc9cc4a48f6816fbe4b336b57d9a03/href">https://medium.com/media/a4dc9cc4a48f6816fbe4b336b57d9a03/href</a></iframe><h4>Effects and reducers</h4><p>Whilst the auth service is responsible for providing a simplified API to communicate with Cognito, the NgRx effects are responsible for handling any actions that are dispatched from the UI and from other effects.</p><p>For example, the sign-in page dispatches a signIn action when the Login button is clicked. This is handed by the signIn$ effect. This in turn calls the authService.signIn method and, if the result is successful, dispatches the signInSuccessful action. This in turn is handled by the signInSuccessful$ effect, which navigates to the root path of the site (e.g. the profile page).</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/05e081a26fddad7892c18d5a056516b1/href">https://medium.com/media/05e081a26fddad7892c18d5a056516b1/href</a></iframe><p>As each action is dispatched, the reducers also set the relevant state. For example loading, showError and isLoggedIn. These states are then reflected in the UI dynamically as they are all Observables.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9d4edd14b71a2f64b886c41d30a59d25/href">https://medium.com/media/9d4edd14b71a2f64b886c41d30a59d25/href</a></iframe><h4>Profile module</h4><p>The profile module is responsible for the main profile page and the profile edit page. You can look at the source code to understand this part of the app. I this secured part of the app rather simple as the main focus on in this article are the Cognito related operations.</p><p>The profile pages were added to demonstrate a protected page using an AuthGuard and the currently authenticated user to make Cognito API calls. The profile data itself is read and updated using the Cognito API via the profile and auth services.</p><h3>Tips</h3><p>Below are various tips regarding issues that I encountered with Angular when working through this solution.</p><h4>Tip 1 — Fix EsLint error</h4><p>In order to prevent eslint from showing errors in every Typescript source code file I had to follow <a href="https://stackoverflow.com/questions/64933543/parsing-error-cannot-read-file-tsconfig-json-eslint">this information on Stack Overflow</a> in order to resolve this issue. I had to rename .eslintrc.json to .eslintrc.js and then in that file add the following to the parserOptions configuration:</p><pre>“tsconfigRootDir”: __dirname</pre><h4>Tip 2 — Fix ‘global is not defined’ error</h4><p>When serving up the ionic project you will most likely get an error in the browser as follows:</p><pre>Uncaught ReferenceError: global is not defined</pre><p>To resolve this add the following at the end of the polyfills.ts file:</p><pre>(window as any).global = window;</pre><h4>Tip 3 — CommonJS and AMD build warnings</h4><p>When building or serving the Angular SPA a number of warnings appear due to the use of the AWS Amplify Auth library. These can be muted in angular.json by adding the following to the build options:</p><pre>&quot;options&quot;: {<br>  ...<br>  &quot;allowedCommonJsDependencies&quot;: [<br>    &quot;crypto-js&quot;,<br>    &quot;buffer&quot;,<br>    &quot;isomorphic-unfetch&quot;,<br>    &quot;url&quot;,<br>    &quot;@aws-crypto/sha256-js&quot;,<br>    &quot;@aws-crypto/sha256-browser&quot;,<br>    &quot;uuid&quot;<br>  ]<br>}</pre><h4>Tip 4— Redux DevTools</h4><p>Make sure you make use of the Redux DevTools Chrome Extension to view the state of the NgRx Store as various actions are performed.</p><h3>Level Up Coding</h3><p>Thanks for being a part of our community! More content in the <a href="https://levelup.gitconnected.com/">Level Up Coding publication</a>. <br>Follow: <a href="https://twitter.com/gitconnected">Twitter</a>, <a href="https://www.linkedin.com/company/gitconnected">LinkedIn</a>, <a href="https://newsletter.levelup.dev/">Newsletter</a><br><strong>Level Up is transforming tech recruiting ➡️ </strong><a href="https://jobs.levelup.dev/talent/welcome?referral=true"><strong>Join our talent collective</strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8c0d90121faa" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/amazon-cognito-with-sms-user-verification-8c0d90121faa">Amazon Cognito with SMS user verification</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>