Introduction

Welcome to this step-by-step guide on how to automate the deployment and management of a web server infrastructure on AWS using Terraform and Ansible. In this project, you will:

• Learn to use Terraform to create a VPC and three EC2 instances and put them behind an Application Load Balancer
• Export the public IP addresses of the instances to a file called host-inventory
• Learn how to use AWS Route53 to set up a domain name and an A record for a subdomain that points to the ALB IP address
• Use Ansible to automate the configuration of the servers, including: Installing Apache, setting the time zone to Africa/Lagos, and displaying a simple HTML page that can be used to identify the servers.

This project is designed for beginners who are familiar with the basic concepts of AWS and have some experience creating resources via the AWS console, and Ansible. If you are new to Terraform, this project will be a great starting point for you to learn how to use these automation tools to deploy and manage your web server infrastructure on AWS.

Pre-requisites

1. An AWS account with an IAM user with:

• AmazonVPCFullAccess
• AmazonEC2FullAccess
• AmazonRoute53FullAccess

The Access Key and Secret Access Key are required

2. Terraform installed.

3. Ansible installed.

4. A Linux OS (I used Ubuntu 20.04); Ansible does not run on Windows. You can use an EC2 instance, Vagrant, or a Mac.

5. A Name.com account; This is not necessary, but it is needed if you want to automate the process of adding the nameservers generated by Route53 to your Name.com domain (if this is your preferred domain provider).

What is Terraform?

Terraform is an open-source infrastructure as code software tool that enables users to provision and manage infrastructure resources. It is platform-agnostic and can be used to provision resources on various cloud providers, such as AWS, Azure, and GCP. Terraform’s configuration files are written in HashiCorp Configuration Language (HCL), which is a simple and easy-to-learn language. Terraform is declarative, meaning that you describe the desired state of your infrastructure, and the tool figures out how to reach that state. You don’t need to specify the step-by-step process, Terraform handles that for you.

What is Ansible?

Ansible is an open-source automation tool that enables users to automate the configuration, deployment, and management of servers. It is agentless and uses a simple, human-readable language to define automation tasks. Ansible uses YAML (YAML Ain’t Markup Language) to write its playbooks, which is also easy to understand.

By using Terraform to provision the infrastructure resources and Ansible to automate the configuration of the servers, you will be able to achieve a high level of automation and consistency in your web server infrastructure deployment and management.

Automating Your Infrastructure

In this section of the article, we will delve into the process of automating the infrastructure for our web server. This includes creating a VPC, setting up EC2 instances, configuring an Application Load Balancer (ALB), and setting up a subdomain for the resources in Route53. While all of these configurations can be done in one main.tf file, it is better to separate them for better organisation and maintainability. Terraform is smart enough to understand how the resources are related, so by separating the configurations into different files, we can make it easier to manage and understand the infrastructure.

Additionally, we will be using a variables.tf file to define variables that will be used across the configurations and a terraform.tfvars file to store the actual values of the variables. This allows for better management of sensitive information and flexibility in changing variable values across different environments. By the end of this section, you will have a fully automated and configured infrastructure.

Before we begin, you need to add your AWS Access Key and Secret Key as environment variables in your terminal. This is because when using Terraform to manage AWS resources, it needs your AWS access key and secret key to communicate with the AWS API. These keys are used to authenticate and authorise Terraform to access your AWS account. To keep sensitive information separate, it’s best to add your keys as environment variables instead of hardcoding them in Terraform configuration files or scripts. This way, you can safely store your Terraform code in a version control system without risking exposing sensitive information.

To do this, run the following commands:

export AWS_ACCESS_KEY_ID="youraccesskey"
export AWS_SECRET_ACCESS_KEY="yoursecretkey"

Step 1: Create the providers.tf file

In Terraform, providers are plugins that interact with specific services or platforms. They are defined in the Terraform configuration files and are necessary for Terraform to create, update, and delete resources. A providers.tf file is used to organise and manage these providers. It allows you to specify the provider name, version, and any required authentication or configuration settings. It also makes it easy to maintain consistency across different environments and parts of the codebase.

Note: You can also include provider configurations in the main.tf file, but using a separate providers.tf file is recommended for better organisation.

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.0"
    }
  }
}

provider "aws" {
  region = var.region
}

Here, we’ve added AWS to the required providers block, which includes the source and version of the provider, in this case AWS versions 4.0 to 4.9.

We’ve also included an “aws” provider block, which specifies the authentication credentials to access the AWS API and the region in which Terraform should provision our infrastructure. Terraform pulls the access key and secret key from the environment variable we defined earlier, the region is defined in our variables.tf file.

Step 2: Create the vpc.tf file

A Virtual Private Cloud (VPC) is a virtual network that is dedicated to your AWS account. It enables you to launch AWS resources in a virtual network that you’ve defined. With a VPC, you can define your own network space, and control how your network and the Amazon EC2 resources inside it are exposed to the Internet.

In the vpc.tf file, you will create:

• A VPC
• Three subnets
• An Internet Gateway (IGW)
• A Route table with the subnets associated
• Two Security groups — one for the Load balancer and the other for the instances

Firstly, you will create thevariables.tf file and define the variables to be used in the vpc.tf file.

variable "region" {
  description = "AWS region"
  type = string
  default = "us-east-1"
}

variable "vpc_block" {
  description = "cidr block and name tag for the vpc"
  type = object({
    cidr_block = string
    name = string
  })
}

variable "subnet_blocks" {
  description = "cidr, availability zone and name tag for the subnets"
  type = map(object({
    cidr = string
    az = string
  }))
}

variable "igw" {
  description = "internet gateway name"
  type = string
}

variable "rtb" {
  description = "cidr block and name tag for route table"
  type = object({
    cidr_block = string
    name = string
  })
}

variable "sg" {
  description = "Application load balancer and instances security group name"
  type = list(string)
}

variable "inbound_ports" {
  description = "Security groups inbound ports"
  type = list(number)
  default = [80,443]
}

The variables.tf file contains variable blocks. Each variable block has the following:

• A variable name in quotes
• A “type” keyword that specifies the type of the variable. It could either be string, number, bool, list, map, tuple or object
• A “description” keyword that describes what the variable represents. It is good practice to always add a description in the variable block
• A “default” keyword which specifies a default value for the variable. If this is not provided the value of the variable ia assigned from the terraform.tfvars file or can be passed when you run terraform plan or terraform apply .

Secondly, you will assign values to the defined variables in the terraform.tfvars file

vpc_block = {cidr_block = "172.17.0.0/16", name = "project_vpc"}
subnet_blocks = {
  subnet_01 = {
    cidr = "172.17.1.0/24"
    az = "us-east-1a"
  }
  subnet_02 = {
    cidr = "172.17.2.0/24"
    az = "us-east-1b"
  }
  subnet_03 = {
    cidr = "172.17.3.0/24"
    az = "us-east-1c"
  }
}
igw = "project_igw"
rtb = {cidr_block = "0.0.0.0/0", name = "project_rtb"}
sg = ["project_alb_sg", "project_instances_sg"]

The file is in a key-value format, where key is the variable name, which is the same as the one defined in the variables.tf file, and value is the desired value for that variable according to the type defined in the variables.tf file.

Finally, the vpc.tf file

#vpc.tf

resource "aws_vpc" "main" {
  cidr_block            = var.vpc_block.cidr_block
  enable_dns_hostnames  = true
  tags = {
    Name                = var.vpc_block.name
  }
}

resource "aws_subnet" "subnets" {
  for_each              = var.subnet_blocks

  vpc_id                = aws_vpc.main.id
  cidr_block            = each.value["cidr"]
  availability_zone     = each.value["az"]

  tags = {
    Name                = each.key
  }
}

resource "aws_internet_gateway" "main" {
  vpc_id                = aws_vpc.main.id

  tags = {
    Name                = var.igw
  }
}

resource "aws_route_table" "main" {
  vpc_id                = aws_vpc.main.id

  route {
    cidr_block          = var.rtb.cidr_block
    gateway_id          = aws_internet_gateway.main.id
  }

  tags = {
    Name        = var.rtb.name
  }
}

resource "aws_route_table_association" "main" {
  for_each              = aws_subnet.subnets

  subnet_id             = each.value.id
  route_table_id        = aws_route_table.main.id
}

resource "aws_security_group" "alb_sg" {
  name                  = var.sg[0]
  description           = "Allow HTTPS and HTTP inbound traffic for application load balancer"
  vpc_id                = aws_vpc.main.id

  dynamic "ingress" {
    for_each            = var.inbound_ports

    content {
      from_port         = ingress.value
      to_port           = ingress.value
      protocol          = "tcp"
      cidr_blocks       = ["0.0.0.0/0"]
    }
  }

  egress {
    from_port        = 0
    to_port          = 0
    protocol         = "-1"
    cidr_blocks      = ["0.0.0.0/0"]
  }
}

resource "aws_security_group" "instance_sg" {
  name                  = var.sg[1]
  description           = "Allow HTTPS and HTTP inbound traffic for the instances"
  vpc_id                = aws_vpc.main.id

  dynamic "ingress" {
    for_each            = var.inbound_ports

    content {
      from_port         = ingress.value
      to_port           = ingress.value
      protocol          = "tcp"
      security_groups   = [aws_security_group.alb_sg.id]
    }
  }

  ingress {
    description         = "Allow SSH"
    from_port           = 22
    to_port             = 22
    protocol            = "tcp"
    cidr_blocks         = ["0.0.0.0/0"]
  }

  egress {
    from_port        = 0
    to_port          = 0
    protocol         = "-1"
    cidr_blocks      = ["0.0.0.0/0"]
  }
}

A resource block describes a specific resource that you want to create, update, or delete. It defines the provider that is responsible for managing the resource with the type of resource (e.g., aws_vpc ), the name of the resource (e.g., “main”), and any configuration options that are specific to that resource (e.g., all the configurations between the curly braces).

The for_each keyword in the aws_subnet and aws_route_table_association resources is a special argument that allows you to create multiple resources of the same type. It is useful when you need to create multiple resources that are similar but have different properties. The for_each argument takes a list or map data structure, and Terraform will create a new resource for each element in the list or map. For example, in the case of the aws_subnet resource, we needed to create three subnets with different CIDR blocks and availability zones. To accomplish this, we defined a variable in the variables.tf file with a type of map(object) and the properties for each subnet were assigned in the terraform.tfvars file. We then used the for_each argument to loop through the map and create the three subnets with the defined properties. Similarly, in the aws_route_table_association resource, we used the for_each argument to loop through the subnets created and extract the subnet ID for each one, in order to associate them with the route table.

You can reference an attribute in a resource that has not been created yet by using the resource type, name, and attribute. For example, to reference the ID of the VPC in the aws_subnet resource block, use “aws_vpc.main.id”. This links the ID of the VPC to the subnet resource after it is created.

In the aws_security_group resource block, we use a dynamic block to create multiple ingress rules that allow traffic on ports 80 and 443. This allows us to efficiently create multiple ingress rules with the same configuration, rather than defining each rule individually. The dynamic block functions similarly to thefor_each feature, which can be used to create multiple resources with the same configuration.

You can use ingress rules to allow incoming traffic on specific ports or from specific IP addresses, and use egress rules to allow outgoing traffic on specific ports or to specific IP addresses.

Step 3: Create the instances.tf file

An EC2 instance is a virtual server in Amazon Web Services’ Elastic Compute Cloud (EC2) for running applications on the Amazon Web Services infrastructure. EC2 instances can be launched with various configurations of CPU, memory, storage, and networking capacity and can be running different operating systems. You can configure security and networking, and you can also launch multiple instances to create a scalable and fault-tolerant application.

In this file, you will do the following:

• Retrieve the latest Ubuntu20.04 AMI(Amazon Machine Image) from AWS in order to create the EC2 instances
• Create an SSH key-pair in order for Ansible to connect to the instances and configure the webserver
• Save the private key generated in the project directory
• Create three EC2 instances and export their public IP addresses to a file called host-inventory
• Execute the ansible playbook to configure the servers

Firstly, you’ll update the variables.tf file to define variables for the instances.tf file

#variables.tf

variable "ssh_key" {
  description = "SSH Key name"
  type = string
}

Secondly, you’ll update the terraform.tfvars file to assign values to the defined variables

#terraform.tfvars

ssh_key = "project_key"

Thirdly, the instances.tf file

#instances.tf

data "aws_ami" "latest_ubuntu_image" {
  most_recent = true
  owners = ["amazon"]

  filter {
    name = "name"
    values = ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-*"]
  }

  filter {
    name = "virtualization-type"
    values = ["hvm"]
  }
}

resource "tls_private_key" "main" {
  algorithm = "RSA"
  rsa_bits = 4096
}

resource "aws_key_pair" "generated_key" {
  key_name = var.ssh_key
  public_key = tls_private_key.main.public_key_openssh
}

resource "local_file" "ssh_key" {
  content = tls_private_key.main.private_key_pem
  filename = "${var.ssh_key}.pem"
  file_permission = "0400"
}

resource "aws_instance" "main" {
  for_each                     = aws_subnet.subnets

  ami                           = data.aws_ami.latest_ubuntu_image.id
  instance_type                 = "t2.micro"
  subnet_id                     = each.value.id
  vpc_security_group_ids        = [aws_security_group.instance_sg.id]
  associate_public_ip_address   = true
  key_name                      = var.ssh_key

  provisioner "local-exec" {
    command = "echo '${self.public_ip}' >> ./host-inventory"
  }
}

resource "null_resource" "ansible-playbook" {
  provisioner "local-exec" {
    command = "ansible-playbook --private-key ${var.ssh_key}.pem main.yml"
  }

  depends_on = [aws_instance.main]
} 

Data sources access existing infrastructure and retrieve information about it, such as the Amazon Machine Image (AMI). This information can be used in Terraform configuration files to make decisions about how to manage that infrastructure. An AMI includes the necessary information to launch an instance, including the operating system, application server, and application. It’s considered good practice to source the AMI directly from AWS because it ensures that the image is the latest version and optimized for the region in which you’re launching the instance. This is particularly important because AMI IDs can change over time and are different in each region. Data sources are defined using the data block. Data blocks follow the same syntax as resource blocks.

We’ll start by using the data block to retrieve information about the AMI.

Next, we’ll use the tls_private_key resource to generate a secure private key in PEM and OPENSSH PEM formats. You can use this private key for SSH connections. SSH keys come in pairs, with a public key used to encrypt data sent to the server and a private key used to decrypt data received from the server. The private key is typically kept on the client side, while the public key is uploaded to the server. The aws_key_pair resource uploads the public key, generated with the tls_private_key resource, to enable logging into the instances created. The local_file resource stores the private key generated on the local machine.

Terraform does not encourage using tls_private_key in a production environment because the SSH key is stored as plain text in the state file and that can pose a security risk. However, you can use it in this case. Do not commit your state file to version control.

You’ll use the aws_instance resource to create three instances in each of the three subnets that you created earlier using the for_each loop. To execute a command on the local machine where you are running Terraform, use the local_exec provisioner. This provisioner is typically used within a resource block and can only function within the context of a resource. This means that the command will be executed after the resource is created or updated. You can use the provisioner to export the public IP of the instances you created to the host-inventory file. The self variable allows you to reference the current resource and its attributes within the same resource block. The attribute in this case is the public IP of each of the instances you created.

The host-inventory file lists the hosts, their IP addresses or hostnames, and their associated groups to be managed by Ansible

The null_resource block is a resource type that does not create any actual underlying resources. It is used to trigger provisioners and perform tasks. It uses the local_exec provisioner to run the Ansible playbook created in main.yml on the local machine. depends_on is a property that is used to specify dependencies between resources in Terraform. This means that Terraform will make sure that the aws_instance.main resource is created or updated before the “ansible-playbook” resource is executed.

Next, you’ll have to create the main.yml file referenced in the null_resource block in order to configure the servers (instances) created.

#main.yml

---

- hosts: all
  tasks:

  - name: update cache
    apt:
      update_cache: yes

  - name: Install Apache2    
    apt:
      name: apache2
      state: present

  - name: Set timezone to Africa/Lagos
    timezone:
      name: Africa/Lagos

  - name: Display hostname
    shell: echo "<h1>Hello world!! This is $(hostname -f) and the date is $(date)</h1>" > /var/www/html/index.html
    notify: restart_apache

  handlers:
  - name: restart_apache
    systemd:
      name: apache2
      state: restarted

You’ll also need to create theansible.cfg file that contains settings for Ansible, such as the location of the Ansible configuration directory, the location of the inventory file, and the default user when connecting to remote hosts.

#ansible.cfg

[defaults]
inventory=host-inventory
remote_user=ubuntu
host_key_checking=false

[privilege_escalation]
become=yes
become_method=sudo

Step 4: Create the load_balancer.tf file

A Load balancer is a service that distributes incoming network traffic across multiple EC2 instances, ensuring that no single instance is overwhelmed with too much traffic. This improves the overall performance and availability of your application. Target groups are used in conjunction with a load balancer to distribute incoming traffic across multiple resources. When you create a target group, you specify the protocol and port number for the traffic, and then you register the resources that you want to receive traffic. The load balancer then uses the target group to route incoming traffic to the registered resources.

In this file, you will do the following:

• Create the load balancer
• Create the target group
• Attach the instances to the target group
• Create an HTTP listener in your load balancer.

Firstly, update the variables.tf file to define variables for the load_balancer.tf file

#variables.tf

variable "lb_name" {
  description = "Load balancer name"
  type = string
}

variable "tg_name" {
  description = "Target group name"
  type = string
}

Next, update the terraform.tfvars file to assign values to the defined variables

#terraform.tfvars

lb_name = "project-alb"
tg_name = "project-tg"

Next, create the load_balancer.tf file

#load_balancer.tf

resource "aws_lb" "main" {
  name = var.lb_name
  internal = false
  load_balancer_type = "application"
  security_groups = [aws_security_group.alb_sg.id]
  subnets = [for subnet in aws_subnet.subnets : subnet.id]
}

resource "aws_lb_target_group" "main" {
  name = var.tg_name
  port = 80
  protocol = "HTTP"
  vpc_id = aws_vpc.main.id

  health_check {
    port = 80
    protocol = "HTTP"
    path = "/"
    matcher = "200-299"
  }
}

resource "aws_lb_target_group_attachment" "main" {
  for_each = aws_instance.main

  target_group_arn = aws_lb_target_group.main.arn
  target_id = each.value.id
  port = 80
}

resource "aws_lb_listener" "http" {
  load_balancer_arn = aws_lb.main.arn
  port = 80
  protocol = "HTTP"

  default_action {
    type = "forward"
    target_group_arn = aws_lb_target_group.main.arn
  }
}

We’ll start by creating the Application Load Balancer (ALB) resource using the aws_lb resource block.

Next, we’ll create the target group resource. You’ll also define a health check for the target group, setting the port, protocol, and path for the check, as well as the status code range that is considered healthy.

Next, we’ll attach the instances we created to the target group using the aws_lb_target_group_attachment resource.

We’ll also create the ALB listener on port 80 with a default action to forward the traffic to the target group you created earlier.

Step 5: Create the route53.tf file

Amazon Route 53 is a highly available and scalable Domain Name System (DNS) web service. It is a way to route end users to Internet applications by translating human-friendly domain names into IP addresses.

In this file, you will do the following:

• Create a route53 hosted zone

A hosted zone is a container for records in the Amazon Route 53 DNS service that you can use to route traffic for a domain and its subdomains.

• Set up an “A” record to point your ALB to your subdomain

An “A” record (Address record) in Amazon Route 53 is a type of DNS record that maps a domain name to an IPv4 address or Alias, such as an ALB. This allows you to associate a domain name with a specific IP address, so that when users enter the domain name into their web browser, they are directed to the correct website

A subdomain is a domain that is part of a larger domain. It is created by adding a prefix to the root domain name, separated by a dot.

Firstly, you’ll update your variables.tf file

#variables.tf

variable "domain_names" {
  description = "Describes the domain and subdomain names"
  type = map(string)
}

Next, you’ll assign values to the declared variables in your terraform.tfvars file

#terraform.tfvars

domain_names = {domain_name = "sarahligbe.live", subdomain_name = "terraform-test.sarahligbe.live"}

Finally, you’ll create the route53.tf file

#route53.tf

resource "aws_route53_zone" "main" {
  name = var.domain_names.domain_name
}

resource "aws_route53_record" "subdomain_a_record" {
  zone_id = aws_route53_zone.main.zone_id
  name = var.domain_names.subdomain_name
  type = "A"

  alias {
    name = aws_lb.main.dns_name
    zone_id = aws_lb.main.zone_id
    evaluate_target_health = true
  }
}

After creating a hosted zone, AWS Route53 automatically creates a set of default records, including a Start of Authority (SOA) record and four name server (NS) records. You can then add or update the name servers with your domain provider, such as Name.com, to route traffic for your domain and subdomains. This brings us to our final step, which is to automate the addition of our Route53 generated name servers to our domain provider, which is Name.com in this case.

Step 6: Create the namedotcom.tf file

This is an optional step. If you’re using a different domain provider, you can Google whether or not Terraform has a provider for them.

To use this provider, you should generate a token at https://www.name.com/account/settings/api

You can give the token whatever name you like.

Next, you’ll whitelist your IP address. Whitelisting your IP address is a security measure that helps ensure that only authorized users are able to access the API. This is because when you whitelist an IP address, you are essentially creating a list of trusted IP addresses that are allowed to access the API. This helps to prevent unauthorised access to the API from unknown or untrusted sources.

Your current IP address is displayed there, you can copy it and add it.

Your public IP address may change, however, for a couple of reasons. One common reason is that your internet service provider (ISP) uses dynamic IP addressing, which assigns a new IP address to your device when it connects to the internet. This is done to conserve IP addresses and ensure that they are used efficiently. Additionally, your IP address may change if you are using a mobile device and switch between cellular and WiFi connections, or if you are connected to a VPN. You can update your new IP address on Name.com if it changes and delete the old one.

It’s not good practice to leave sensitive information, such as your token, in your variables.tf file or terraform.tfvars file, hence we will be storing our token generated and our username as environment variables and then referencing them within our variables.tf file.

You can store your Terraform variables as environment variables by prefixing the variable name with TF_VAR. Example:

export TF_VAR_token="yourtokenname"
export TF_VAR_username="yourusername"

Next, you’ll update your variables.tf file

#variables.tf

variable "token" {
  description = "Name.com API token"
  type = string
}

variable "username" {
  description = "Name.com username"
  type = string
}

Next, we’ll update the providers.tf file to include the namedotcom provider

#providers.tf

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.0"
    }

    namedotcom = {
      source = "lexfrei/namedotcom"
      version = "1.2.4"
    }
  }
}

provider "aws" {
  region = var.region
}

provider "namedotcom" {
  token = var.token
  username = var.username
}

Here, we’ve added namedotcom to the required providers block, which includes the source and version of the provider.

We’ve also added a namedotcom provider block, which specifies the authentication credentials to access the Name.com API. It requires a token and a username to be passed as variables (which we already defined). These credentials are used to access the API and make changes to domain names registered on Name.com.

Finally, create the namedotcom.tf file

#namedotcom.tf

resource "namedotcom_domain_nameservers" "domain_name" {
  domain_name = var.domain_names.domain_name
  nameservers = [
    aws_route53_zone.main.name_servers[0],
    aws_route53_zone.main.name_servers[1],
    aws_route53_zone.main.name_servers[2],
    aws_route53_zone.main.name_servers[3]
  ]
}

Here, we use the namedotcom_domain_nameservers resource block to add the four nameservers that were created using the aws_route53_zone resource block.

Step 7: Build the infrastructure

Now that we’ve successfully written the Terraform and Ansible scripts to provision our infrastructure and configure the servers, the next thing to do is proceed with building the infrastructure.

Terraform init

The first Terraform command you’ll run is terraform init . This command is used to initialise a working directory that contains Terraform configuration files, and initialise the backend if one is set. It sets up the necessary infrastructure to run Terraform, such as downloading plugins for the providers specified in your Terraform scripts. By running this command, you can ensure that Terraform is prepared to run correctly before you start building your infrastructure.

You should run this command whenever you set up a new Terraform project or after making changes to your Terraform configuration.

terraform init creates a .terraform directory and a .terraform.lock.hcl file.

The .terraform directory stores the Terraform plugins and other information required for Terraform to properly manage and configure your infrastructure. It is regenerated whenever you run terraform init so there’s no need to commit it to version control.

The .terraform.lock.hcl lock file is used to enforce consistency between Terraform runs and prevent unintended changes to your infrastructure. This should be added to version control, as stated in the image above.

Terraform plan

terraform plan is an optional but advisable command to run. This command shows what changes Terraform will make to your infrastructure based on the configuration files you have defined. The plan is an important step before applying changes to your infrastructure, as it provides an opportunity to review and confirm the changes Terraform will make.

Terraform apply

terraform apply is the final command to run in order to implement the changes declared in your Terraform configuration files. This command takes the plan generated by terraform plan and applies it to create, update, or delete the infrastructure components you have defined.

The command will prompt you to confirm the changes before actually applying them. You can, however, run terraform apply — auto-approve to apply the changes without getting the prompt.

After you run terraform apply, Terraform creates a state file. Terraform uses the state file to keep track of the infrastructure it manages. It stores the state of the resources, including their attributes and relationships to other resources.

The state file is important because it enables Terraform to determine what actions are necessary to bring the infrastructure into the desired state defined in Terraform configuration files. By default, the state file is stored locally, but it can also be stored in a remote backend, such as a S3 bucket, for increased security and collaboration in a team environment. Do not commit your state file to version control, as it contains information about the infrastructure that Terraform manages and could be used maliciously if it falls into the wrong hands.

Whenever you change, add, or remove any resources, Terraform creates a state backup file that stores the previous state of the resources before the changes.

Terraform destroy

One of the great things about Terraform is how you can easily destroy all the infrastructure resources you manage with it. This ensures that there are no unused resources that are consuming your cloud provider’s resources and accruing charges.

Commit to Version Control

You can commit your Terraform configuration files to version control, such as Git. However, you shouldn’t commit every file to version control, as they may contain sensitive information and secrets that can be accessed by unauthorised parties, leading to potential security risks.

Do not commit your terraform.tfvars file if it contains any sensitive information.

Use .gitignore file to exclude the following files that shouldn’t be committed:

#.gitignore

#The .terraform directory
.terraform/*

#The tf state file and any other backup
*.tfstate
*.tfstate.*

Conclusion

We’ve been able to automate and configure our infrastructure using Terraform and Ansible, commit to version control, and adhere to some security best practices. You can learn more about Terraform by checking out their official documentation.

If you have any questions, please leave them in the comment section, and I’ll be happy to answer. By following this process, you should now have a solid foundation for automating your infrastructure and keeping it secure.

Additional Resources

1. Terraform AWS documentation
2. Terraform best practices
3. Terraform namedotcom documentation
4. Terraform for_each meta-argument
5. How to find an AMI

Introduction

For Altschool Africa’s holiday challenge, students in the Cloud Engineering track were required to deploy a highly available AWS infrastructure to host a simple web app that displays the hostname of each server (that’s the gist of it).

Planning for High Availability

A highly available infrastructure is one that is designed to handle failures and continue to operate without interruption.

There are several key considerations that should be taken into account to ensure that the application can handle failures and continue to operate without interruption. Some of these include:

• Redundancy: One of the most important considerations is to ensure that there is redundancy built into the system so that if one component fails, there is another component that can take over and keep the system running. This can include implementing load balancers and deploying the servers in different availability zones. Load balancers come with built-in redundancy, this means that when one server fails, the load balancer will divert all the traffic to another server without affecting the user.
• Failover: Which is the mechanism in place that can automatically switch to a backup system or component in the event of a failure. Load balancers have automatic failover.
• Scaling: This is the ability to effectively scale up or down based on demand. This can include implementing autoscaling groups for the automatic scaling of servers.

It is important to have a security plan in place that covers all aspects of the infrastructure, including network security and encryption. AWS Certificate Manager allows you to request a public certificate to secure your web application.

Taking these considerations into account, you can design a highly available web application that is resilient to failures, secure, and can continue to operate in the event of an unexpected downtime.

By using Ansible, it will be easier to automate many of these processes and provide a more consistent and reliable approach.

The Architecture Diagram

Pre-requisites

This article guides beginners on automating the provisioning and configuration of AWS resources with Ansible. Before starting, ensure you have a basic understanding of creating AWS resources through the AWS Management Console and some experience using Ansible for automation. If you lack knowledge in either of these areas, familiarise yourself with the basics before proceeding. We will not cover the basics of creating IAM roles or the process of selecting a CIDR range for your VPC and subnets.

The following steps should be taken before proceeding with the playbook:

1. Create an IAM role with:

• AmazonVPCFullAccess
• AmazonEC2FullAccess
• AmazonRoute53FullAccess
• AwsCertificateManagerFullAccess.

2. Create an EC2 instance (Ubuntu 20.04, preferably) that will serve as the ansible control node and attach the IAM role to it. This way, there will be no need to include an access key or secret key in our playbook. (This is where you will run the playbook from so it should be able to connect to the internet)

3. You should have the following software and dependencies installed:

• Python3 >= 3.8.10
• Ansible >= 2.12.0. Run ansible --version to see the version of ansible-core you have installed.
• pip3 (in order to install the latest version of boto3)
• boto3 (my version is 1.26.43) — Ansible uses boto3 to communicate with AWS API to allow users manage AWS infrastructure through Ansible modules.

sudo pip3 install boto3

You can upgrade to the latest version of boto3 by running:

sudo pip3 install boto3 --upgrade

• unzip — To unzip the AWS CLI that will be downloaded
• AWS CLI v2 — this is because ansible currently does not have an AWS Certificate Manager(ACM) module for requesting public certificates, hence the need for the AWS CLI.

After successful installation of the AWS CLI, configure it by running:

aws configure

The AWS CLI prompts you for the following:

• Access key ID — This should be left blank
• Secret access key — This should also be left blank
• Region — Enter your preferred region (e.g., us-east-1)
• Output format — json

The Ansible Playbook

The Ansible project tree is going to look like this:

• ansible.cfg contains our configuration settings

[defaults]
inventory=host-inventory
remote_user=ubuntu
host_key_checking=false

[privilege_escalation]
become=yes
become_method=sudo 

• host-inventory contains the list of hosts in which the modules and tasks in the playbook will operate

[local]
localhost

• host_vars contains the variables associated with our host
• roles is a directory that contains a collection of tasks, templates which makes it easier to organize and structure the playbook.
• main.yml is where each of these individual roles are consolidated to make up the entire play.

Step One: Provisioning the VPC

From the architecture diagram and project tree above, you will notice that you are creating a VPC with:

• A public and private subnet each in two availability zones
• An Internet gateway
• Two NAT gateways in each public subnet (this is ideal for high availability, although only one is created)
• A route table to route the public subnet traffic
• A route table to route the private subnet traffic
• A security group for the Application load balancer
• A security group for the instances.

Defining the VPC variables

• Create your project directory
• Create host_vars/localhost directory in the root of your project directory
• Create a vars.yml where all the variables needed to run the playbook will be defined.

---
# host_vars/localhost/vars.yml

# VPC variables

vpc_name: "assignment-VPC"   
vpc_cidr: "172.20.0.0/16"                  #VPC CIDR block
aws_region: "us-east-1"
az_01: "us-east-1a"                        #Availability zone 1
az_02: "us-east-1b"                        #Availability zone 2
pub_cidr_01: "172.20.1.0/24"              #CIDR for the first public subnet
pub_cidr_02: "172.20.2.0/24"               #CIDR for the second public subnet
priv_cidr_01: "172.20.3.0/24"               #CIDR for the first private subnet
priv_cidr_02: "172.20.4.0/24"              #CIDR for the second private subnet
pub_subnet_name_01: "Assignment-pub-01"
pub_subnet_name_02: "Assignment-pub-02"
priv_subnet_name_01: "Assignment-priv-01"
priv_subnet_name_02: "Assignment-priv-02"
igw_name: "Assignment-IGW"                #Internet gateway name
igw_rt_name: "Assignment-IGW-RT"         #Internet gateway route table name
nat_name: "Assignment-NAT"                #NAT gateway name
nat_rt_name: "Assignment-NAT-RT"          #NAT gateway route table name
alb_sg: "ALB-SG"                 #Application load balancer security group name
instances_sg: "Instance-SG"          #EC2 security group name

Next you will move on to provisioning the VPC itself

1. Create the roles/ directory in the root of your project directory.

2. Create the provision_vpc/tasks subdirectory in the roles directory.

3. Create the vpc.yml file

---
#roles/provision_vpc/tasks/vpc.yml

# Provision the vpc

  - name: Create a VPC
    amazon.aws.ec2_vpc_net:
      name: '{{ vpc_name }}'      
      cidr_block: '{{ vpc_cidr }}'        
      region: '{{ aws_region }}'   
      dns_support: true            
      dns_hostnames: true          
      state: present
    register: new_vpc
    tags:
      - vpc
      - first

  - name: save the vpc id
    set_fact:
      vpc_id: '{{ new_vpc.vpc.id }}'
      cacheable: yes

The amazon.aws.ec2_vpc_net module creates a VPC with the following parameters listed:

• name — to specify the name of the VPC
• cidr_block — to specify the cidr block for the VPC
• region — to specify the region in which the VPC will be created
• dns_support and dns_hostnames — to enable DNS support and DNS hostname support respectively
• state — to specify the state of the VPC, use ‘present’ to create the VPC or ‘absent’ to delete it.

You will use register to capture the output to the new_vpcvariable.

Theset_fact module allows you to set or create new variables and their values during the execution of the playbook. Here it stores the vpc id of the VPC parsed from new_vpc so that it can be used in other parts of the playbook.

4. Create the subnets.yml file

---
#roles/provision_vpc/tasks/subnets.yml

# Creates two public and two private subnets and stores their subnet ids


  - name: Create two public and two private subnets
    amazon.aws.ec2_vpc_subnet:
      az: '{{ item.az }}'
      cidr: '{{ item.cidr }}'
      vpc_id: '{{ vpc_id }}'
      region: '{{ aws_region }}'
      state: present
      resource_tags:
        Name: '{{ item.name }}'
    loop:
      - { az: '{{ az_01 }}', cidr: '{{ pub_cidr_01 }}', name: '{{ pub_subnet_name_01 }}' }
      - { az: '{{ az_02 }}', cidr: '{{ pub_cidr_02 }}', name: '{{ pub_subnet_name_02 }}' }
      - { az: '{{ az_01 }}', cidr: '{{ priv_cidr_01 }}', name: '{{ priv_subnet_name_01 }}' }
      - { az: '{{ az_02 }}', cidr: '{{ priv_cidr_02 }}', name: '{{ priv_subnet_name_02 }}' }
 

  - name: Gather facts about the subnets
    amazon.aws.ec2_vpc_subnet_info:
      region: '{{ aws_region }}'
      filters:
        "tag:Name":
          - '{{ pub_subnet_name_01 }}'
          - '{{ pub_subnet_name_02 }}'
          - '{{ priv_subnet_name_01 }}'
          - '{{ priv_subnet_name_02 }}'
    register: subnets_info
    tags: first

  - name: Get the subnet ids
    set_fact:
      priv_01_subnet_id: "{{ subnets_info|community.general.json_query(priv_01)|join(\"\") }}"
      priv_02_subnet_id: "{{ subnets_info|community.general.json_query(priv_02)|join(\"\") }}"
      pub_01_subnet_id: "{{ subnets_info|community.general.json_query(pub_01)|join(\"\") }}"
      pub_02_subnet_id: "{{ subnets_info|community.general.json_query(pub_02)|join(\"\") }}"
    vars:
      priv_01: "subnets[?tags.Name == '{{ priv_subnet_name_01 }}'].id"
      priv_02: "subnets[?tags.Name == '{{ priv_subnet_name_02 }}'].id"
      pub_01: "subnets[?tags.Name == '{{ pub_subnet_name_01 }}'].id"
      pub_02: "subnets[?tags.Name == '{{ pub_subnet_name_02 }}'].id"

Theamazon.aws.ec2_vpc_subnet module creates subnets in the VPC with the following parameters:

• az — to specify the availability zone of each subnet
• cidr — to specify a distinct CIDR in each subnet (It is important that the CIDR of the subnets do not overlap)
• vpc_id — to specify the ID of the VPC in which the subnets are created. The variable {{vpc_id}}listed as the value is from the variable defined with the set_fact module in vpc.yml
• resource_tags — is used to specify tags that will be associated with the subnet such as the name of the subnet.

Theloop keyword is used to iterate over a list of dictionaries(or hashes) where each item in the list contains subkeys that represent the availability zone, CIDR, and name of each of the four subnets to be created.

In Ansible, list of dictionaries can also be written as:

loop: [{ key1:value1, key2:value2 }, { key3:value3, key4:value4 }]

Theamazon.aws.ec2_vpc_subnet_info module is used to gather information on the subnets created.

• The filters parameter is used to filter the list of the subnets returned by the module based on specific criteria. In this case the criteria is the names of the individual subnets.

Theset_fact module is setting four new variables, as seen above, for later use in the playbook. The community.general.json_query filter — a form of data manipulation in Ansible used to extract specific values from a complex data structure in JSON format, is used to extract the specific subnet ids from the subnet_info variable that was used to capture the information gathered on the subnets. join(\"\") is used to display the subnet id as a string.vars is used to make the filter more readable by defining the JMESPath query separately.

5. Create the igw.yml file

---
#roles/provision_vpc/tasks/igw.yml

# Creates the Internet gateway

  - name: Create an internet gateway
    amazon.aws.ec2_vpc_igw:
      vpc_id: '{{ vpc_id }}'
      region: '{{ aws_region }}'
      state: present
      resource_tags:
        Name: '{{ igw_name }}'
    register: igw

  - name: store the gateway id
    set_fact: 
      igw_gateway_id: '{{ igw.gateway_id }}'

Theamazon.aws.ec2_vpc_igw module creates an internet gateway with parameters that define the VPC ID, region, state, and name of the internet gateway.

The internet gateway ID is extracted from the igw variable defined with register and stored in the igw_gateway_id variable to be used later in the playbook.

6. Create the nat.yml file

---
#roles/provision_vpc/tasks/nat.yml

# Create NAT gateway and allocate an elastic ip

  - name: Create a NAT gateway and allocate an elastic IP
    amazon.aws.ec2_vpc_nat_gateway:
      state: present
      subnet_id: '{{ pub_01_subnet_id }}'
      if_exist_do_not_create: true
      wait: yes
      region: '{{ aws_region }}'
    register: nat_gateway
    tags: nat

# making it cacheable in order to use it in the delete playbook
  - name: store nat gateway id
    set_fact:
      nat_gate_id: '{{ nat_gateway.nat_gateway_id }}'
      

Theamazon.aws.ec2_vpc_nat_gateway module creates the NAT gateway with the following parameters:

• state and region to specify the state and region in which the NAT gateway is created respectively.
• subnet_id —to specify the subnet id to create the subnet in. The NAT gateway is usually created in a public subnet.
• if_exist_do_not_create — to specify whether or not to create a NAT gateway if it already exists in a subnet id. This also allows idempotency.
• wait — is used to control whether or not the module should wait for the NAT gateway to be created and be in the “available” state before continuing with the playbook.

The NAT gateway ID is stored in the nat_gate_id variable for later use in the playbook.

7. Create the igw_rt.yml file

---
#roles/provision_vpc/tasks/igw_rt.yml

# Create a route table for the internet gateway and associates the public subnets

  - name: Create IGW route table and associate the public subnets
    amazon.aws.ec2_vpc_route_table:
      vpc_id: '{{ vpc_id }}'
      region: '{{ aws_region }}'
      subnets:
        - '{{ pub_01_subnet_id }}'
        - '{{ pub_02_subnet_id }}'
      routes:
        - dest: 0.0.0.0/0
          gateway_id: "{{ igw_gateway_id }}"
      resource_tags:
        Name: '{{ igw_rt_name }}'
    register: IGW_RT
 

Theamazon.aws.ec2_vpc_route_table module creates a route table for controlling the flow of network traffic within the VPC by using the following parameters:

• vpc_id and region to specify the VPC ID and region in which the route table is to be created.
• subnets — to specify the subnets to be associated with the route table. Here the public subnets are associated.
• routes — to specify the routes to be added to the route table. It is specified as a list of dictionaries representing a route. Each route dictionary should contain a dest and gateway_id key. dest being the IPV4 network range for the destination address of the route. gateway_id in this case will be the Internet gateway ID defined in the igw.yml file with the set_fact module since this is a public route table.

8. Create the nat_rt.yml file

---
#roles/provision_vpc/tasks/nat_rt.yml

# Creates a NAT route table and associates the private subnets

  - name: Create NAT route table
    amazon.aws.ec2_vpc_route_table:
      vpc_id: '{{ vpc_id }}'
      region: '{{ aws_region }}'
      subnets:
        - '{{ priv_01_subnet_id }}'
        - '{{ priv_02_subnet_id }}'
      routes:
        - dest: 0.0.0.0/0
          gateway_id: "{{ nat_gate_id }}"
      resource_tags:
        Name: '{{ nat_rt_name }}'
    register: NAT_RT

Here the private subnets are associated, and the NAT gateway ID is used for the gateway_id key in the routes parameter.

9. Create the sg.yml file

---
#roles/provision_vpc/tasks/sg.yml

# Creates security groups for the ec2 instances and the application load balancer

  - name: Load balancer security group
    amazon.aws.ec2_group:
      name: '{{ alb_sg }}'
      description: Security group for the ALB
      vpc_id: '{{ vpc_id }}'
      region: '{{ aws_region }}'
      rules:
        - proto: tcp
          ports:
            - 80
            - 443
          cidr_ip: 0.0.0.0/0
    register: alb_sg

  - name: store the alb security group name
    set_fact: 
      alb_group_id: '{{ alb_sg.group_id }}'
      alb_group_name: '{{ alb_sg.group_name }}'

  - name: EC2 security group
    amazon.aws.ec2_group:
      name: '{{ instances_sg }}'
      description: Security group for the instances
      vpc_id: '{{ vpc_id }}'
      region: '{{ aws_region }}'
      rules:
        - proto: tcp
          ports:
            - 80
            - 443
          group_name: '{{ alb_group_name }}'
    register: instance_sg
    

  - name: store the instances security group id
    set_fact:
      instance_group_id: '{{ instance_sg.group_id }}'

Theamazon.aws.ec2_group module creates security groups, which are a virtual firewall that controls inbound and outbound network traffic to and from your EC2 instances.

• name ,vpc_id , and region parameters specify the name of the security group, the VPC ID in which the security group is created and the region respectively.
• rules — specifies the set of inbound rules to enforce in the security group. It is a list of dictionaries. Each rule dictionary should contain the following keys: proto which defines the protocol the rules applies to such as “tcp”. ports which defines the port traffic is going to such as “80” for HTTP, and “443” for HTTPS. cidr_ip which is the IPV4 CIDR range traffic is coming from and group_name which is the name of the Security group traffic is coming from.
• description — specifies the description of the Security group.

The application load balancer’s security group is first created, and the security group name and security group id are extracted and stored using the set_fact module. Then the EC2 instance security group is created, which allows ports 80 and 443 from the application load balancer’s security group.

10. Create the main.yml file

---
#roles/provision_vpc/tasks/main.yml

- include_tasks: vpc.yml
- include_tasks: subnets.yml
- include_tasks: igw.yml
- include_tasks: nat.yml
- include_tasks: igw_rt.yml
- include_tasks: nat_rt.yml
- include_tasks: sg.yml

The main.yml file consolidates all the individual tasks that have been created in the provision_vpc role. It makes use of the include_tasks module to achieve this.

Step Two: Create AutoScaling Group

To create the autoscaling group, you have to create the launch template first. Launch templates are a way to store and version the launch parameters of an EC2 instance. AutoScaling groups automatically increase or decrease the number of instances in the group based on user-defined policies.

First, you will update host_vars/localhost/vars.yml to include the AutoScaling group variables.

#host_vars/localhost/vars.yml

# Autoscaling group variables

lt_name: "Assignment-LT"            #Launch template name
lt_key_name: "exam"                  #Launch template key name for SSH
lt_instance_type: "t2.micro"          #Launch template instance type
lt_image_id: "ami-0b93ce03dcbcb10f6"     #ubuntu20.04 
asg_name: "Assignment-ASG"            #Auto Scaling group name

# User data variables

nginx_display_directory: /var/www/assignment
nginx_source_file: /etc/nginx/sites-available/assignment.conf
nginx_dest_file: /etc/nginx/sites-enabled
domain: yourdomainname

Next, you will move on to creating the launch template and autoscaling group.

1. Create the autoscaling/ directory in the roles directory.
2. Create a tasks and templates directory in the autoscaling directory. The templates directory will store the user data script you will use to deploy the web application in Jinja2 format which can include variables defined in the variables file.
3. In the templates directory, create a user-data.sh.j2 file. The .j2 extension indicates that this file is a Jinja2 template and should be processed as such.

#!/bin/bash

#roles/autoscaling/templates/user-data.sh.j2

# script to install nginx using user data

apt update -y
apt install nginx php7.4 php7.4-fpm -y
systemctl start nginx
mkdir -p {{ nginx_display_directory }}

echo "?php echo "Hello world! This is '; echo gethostname(); ?>" | tee {{ nginx_display_directory }}/index.php

printf "server {\n
       listen 80 default_server;\n
       listen [::]:80 default_server;\n
       server_name {{ domain }};\n
       root {{ nginx_display_directory }};\n
       index index.php;\n
       location / {\n
               try_files \$uri \$uri/ =404;\n
       }\n

       location ~ \.php$ {\n
           include snippets/fastcgi-php.conf;\n
           fastcgi_pass unix:/var/run/php/php7.4-fpm.sock;\n
      }\n

}" | tee -a {{ nginx_source_file }}
ln -s {{ nginx_source_file }} {{ nginx_dest_file }}
rm {{ nginx_dest_file }}/default
systemctl reload nginx

The script above will install Nginx and PHP on our servers and then display the hostname of each of the servers.

4. Change directories to the tasks directory and create the launch_template.yml file.

---
#roles/autoscaling/tasks/launch_template.yml

# Creates the launch template

  - name: Create the launch template
    community.aws.ec2_launch_template:
      name: '{{ lt_name }}'
      image_id: '{{ lt_image_id }}'
      key_name: '{{ lt_key_name }}'
      instance_type: '{{ lt_instance_type }}'
      security_group_ids: '{{ instance_group_id }}'
      user_data: "{{ lookup('ansible.builtin.template', 'user-data.sh.j2')| b64encode }}"
      region: '{{ aws_region }}'
    register: launch_temp
    tags: lt

  - name: store the launch template id
    set_fact:
      launch_temp_id: '{{ launch_temp.latest_template.launch_template_id }}'
      

Thecommunity.aws.ec2_launch_template module can create launch templates with the following parameters:

• name and region— specifies the name and region of the launch template respectively
• image_id — specifies the image ID of the AMI to be used in the launch template
• key_name — specifies the name of the SSH key for the launch template. There is no need to add the .pem extension
• instance_type — specifies the instance type of the launch template such as “t2.micro”
• security_group_ids — specifies the security group to be associated with the instances created from the launch template. The security group ID value is the instance_group_id variable created with the set_fact module in the sg.yml file
• user_data — specifies the user data to be used by the instances. lookup plugin is being used to retrieve the contents the user-data.sh.j2 file from the templates directory. The contents of this file are then passed through the ansible.builtin.template lookup plugin, which processes the Jinja2 template code, replacing any variables with their corresponding values. The resulting output is then passed through the b64encode filter, which encodes the data in base64 (the user data must be base64 encoded). The final output, which is the base64 encoded user data, is then passed as the value of the user_data parameter to the community.aws.ec2_launch_template module.

The launch template ID from the output is stored in the launch_temp_id variable.

5. Create the asg.yml file

---
#roles/autoscaling/tasks/asg.yml

# Create an autoscaling group in the private subnets using the launch template

  - name: Create autoscaling group
    community.aws.ec2_asg:
      name: '{{ asg_name }}'
      availability_zones: [ '{{ az_01 }}', '{{ az_02 }}' ]
      launch_template:
        launch_template_id: "{{ launch_temp_id }}"
      min_size: 2
      max_size: 5
      desired_capacity: 2
      vpc_zone_identifier: [ '{{ priv_01_subnet_id }}', '{{ priv_02_subnet_id }}' ]
      health_check_type: ELB
      health_check_period: 120
      wait_for_instances: true
      region: '{{ aws_region }}'
    register: autosg


  - name: store the instance id
    set_fact:
      asg_instance_id: '{{ autosg.instances }}'
   

( The community.aws.ec2_asg module redirects to the amazon.aws.autoscaling_group module. If your ansible-core version is between 2.12.0 and 2.13.0, you can use community.aws.ec2_asg . If it is >2.14.0, use amazon.aws.autoscaling_group instead ).

The community.aws.ec2_asgmodule creates auto-scaling groups with the following parameters:

• name and region specify the name and region of the AutoScaling group
• availability_zones — specifies the availability zones of the instances in the AutoScaling group. From the architecture diagram, the AutoScaling group spans two availability zones.
• launch_template — specifies the launch template to be used. It has a sub-parameter which specifies the launch template ID(which was stored in the launch_temp_id variable).
• min_size — specifies the minimum number of instances that should be running in the AutoScaling group at any given time.
• max_size — specifies the maximum number of instances that should be running in the AutoScaling group at any given time. The AutoScaling group will not launch more instances than this number.
• desired_capacity — specifies the number of instances that the AutoScaling group should try to maintain.
• vpc_zone_identifier — specifies which subnets the AutoScaling group instances should be launched in. It is a list of subnet IDs, where each subnet corresponds to a subnet in a VPC within a particular availability zone. The private subnet IDS are used.
• health_check_type — specifies whether or not an instance is running and responding to requests. The AutoScaling group uses the “ELB” option to know whether the instance is healthy as reported by the Elastic load balancer
• health_check_period — specifies the length of time, in seconds, that the Auto Scaling group should wait between checking the health status of an instance
• wait_for_instances — specifies whether or not the module should wait for the instances to be in a ready state before continuing with the playbook.

The instance IDs of the instances extracted are stored in the asg_instance_id variable (it is stored as a list since there are multiple instances).

6. Create the main.yml file

---
#roles/autoscaling/tasks/main.yml

- include_tasks: launch_template.yml
- include_tasks: asg.yml

Step Three: Create the Application Load Balancer

To create the application load balancer, you must first create a target group. Target groups are used in conjunction with load balancers to help route traffic to the resources in the group. These resources could be EC2 instances.

To begin, you’ll update the host_vars/localhost/vars.yml file to include the load balancer variables

#host_vars/localhost/vars.yml

# Application load balancer variables

tg_name: "Assignment-TG"
alb_name: "Assignment-ALB"

Next, you will move on to creating the target group and then the application load balancer.

1. In the roles/ directory, create the load_balancer/tasks directory
2. cd into tasks/ and create the target_groups.yml file

---
#roles/load_balancer/tasks/target_groups.yml

# Creates a target group for the application load balancer

  - name: Create the target group for the load balancer
    community.aws.elb_target_group:
      name: '{{ tg_name }}'
      protocol: http
      port: 80
      vpc_id: '{{ vpc_id }}'
      health_check_protocol: http
      health_check_path: /
      successful_response_codes: "200,250-260"
      targets:
        - Id: '{{ asg_instance_id[0] }}'
          Port: 80
        - Id: '{{ asg_instance_id[1] }}'
          Port: 80
      state: present
      region: '{{ aws_region }}'
    register: tg
    
  - name: store the target group name
    set_fact:
      tg_group_name: '{{ tg.target_group_name }}'
      tg_group_arn: '{{ tg.target_group_arn }}'
      cacheable: yes

Thecommunity.aws.elb_target_group module creates and manages target groups for the application load balancer with the following parameters:

• name and region — specifies the name and region in which the target group is created respectively
• protocol and port—specifies the protocol that routes incoming traffic to the target group, and the port on which the target group should listen for incoming traffic respectively
• health_check_protocol and health_check_path— specifies the protocol the load balancer uses to check the health of registered targets, and the path on the targets for the health checks respectively
• successful_response_codes — specifies the HTTP response code that should be considered a success response from your targets
• targets — specifies the targets (EC2 instances in this case) that should be registered with the target group. The value of this parameter is a list of dictionaries, where each dictionary represents a target. The subkey id defines the ID of the instances (you stored the instance ID from asg.yml output in a list asg_instance_id. You can reference each instance based on their index). The port value should match the port parameter defined earlier
• state — specifies the state of the target group.

The target group name and Amazon Resource Name (ARN) are stored for later use in the playbook using the set_fact module.

3. Create the alb.yml file

---
#roles/load_balancer/tasks/alb.yml

# Creates the application load balancer in the public subnets for the two availability zones in the vpc

  - name: Create the load balancer
    community.aws.elb_application_lb:
      name: '{{ alb_name }}'
      security_groups: '{{ alb_group_id }}'
      subnets:
        - '{{ pub_01_subnet_id }}'
        - '{{ pub_02_subnet_id }}'
      listeners:
        - Protocol: HTTP
          Port: 80
          DefaultActions:
            - Type: forward
              TargetGroupName: '{{ tg_group_name }}'
      state: present
      region: '{{ aws_region }}'
      wait: true
      wait_timeout: 300
    register: new_alb
    
  - name: store the alb dns name and hosted zone id
    set_fact:
      alb_dns: '{{ new_alb.dns_name }}'
      alb_hosted_zone_id: '{{ new_alb.canonical_hosted_zone_id }}'

  - name: Add the target group to the autoscaling group
    community.aws.ec2_asg:
      name: '{{ asg_name }}'
      target_group_arns: '{{ tg_group_arn }}'
      region: '{{ aws_region }}'
   

( The community.aws.elb_application_lb module redirects to the amazon.aws.elb_application_lb module. If your ansible-core version is between 2.12.0 and 2.13.0, you can use community.aws.elb_application_lb . If it is >2.14.0, use amazon.aws.elb_application_lb instead).

Thecommunity.aws.elb_application_lb module creates and manages the application load balancer with the following parameters:

• name and region — specify the name and region in which the load balancer is created respectively
• security_group — specifies the security group to be associated with the load balancer. The value references the security group ID you created in asg.yml
• subnets — specifies the subnet IDs in which the load balancer should be created in. The subnet IDs should span at least two availability zones to ensure high availability. The public subnet IDs are used
• listeners — specifies the listeners to be associated with the Application load balancer. The value of this parameter is a list of dictionaries. The key-value pair of each dictionary is case sensitive.

Use the set_fact module to store the dns_name and canonical_hosted_zone_id return values, which specify the DNS name and the Amazon Route 53 hosted zone ID you want to use as the alias target for the Application Loadthelancer.

4. Create the main.yml file

---
#roles/load_balancer/tasks/main.yml

- include_tasks: target_groups.yml
- include_tasks: alb.yml

Step Four: Create Route53 Records

AWS Route53 is a simple and reliable way for end users to access internet applications by translating domain names to the IP addresses of the servers hosting those applications.

To create a Route53 record for the Application load balancer, you must first create a hosted zone.

1. In the roles/ directory, create the route53/tasks/ directory.
2. cd into the tasks/ directory and create the main.yml file (yes, you can also explicitly define your tasks in the main.yml file instead of using the include_tasks module).

---
#roles/route53/tasks/main.yml

# Create a route53 hosted zone and an A record for the application load balancer

  - name: Create route53 hosted zone
    community.aws.route53_zone:
      zone: '{{ domain }}'
    register: hosted_zone_name
    tags: route53

  - name: Create an alias record that points to the ALB
    community.aws.route53:
      state: present
      hosted_zone_id: '{{ hosted_zone_name.zone_id }}'
      record: '{{ domain }}'
      type: A
      overwrite: true
      value: '{{ alb_dns }}'
      alias: True
      alias_hosted_zone_id: '{{ alb_hosted_zone_id }}'

  - name: get nameservers
    community.aws.route53:
      zone: '{{ domain }}'
      record: '{{ domain }}'
      type: A
      state: get
    register: nameserver
    tags: route53

  - name: store the credentials 
    set_fact:
      nameservers_1: '{{ nameserver.nameservers }}'

  - name: display nameservers
    debug:
      msg: "Here are your nameservers: {{ nameservers_1 }}"
    tags: route53

( The community.aws.route53_zone and community.aws.route53 modules redirect to amazon.aws.route53_zone and amazon.aws.route53 respectively. If your ansible-core version is > 2.14.0, use amazon.aws.route53_zone and amazon.aws.route53 instead).

The community.aws.route53_zone creates the Route53 public hosted zone with the following parameters:

• zone — specifies the domain name of the hosted zone to be created.

Theregister module captures the output of the module in order to use it with the community.aws.route53 module.

The community.aws.route53 creates the A record in your defined hosted zone with the following parameters:

• zone — specifies the domain name in which the record is created
• record — specifies the full DNS record to create. The value can either be a subdomain or just the domain name itself
• type — specifies the type of record to create. Yours is an “A” record
• overwrite — specifies whether or not to overwrite an existing record. Initially, the value was set to false but running the playbook again resulted in an error because the record already existed so overwrite was set to true, it doesn’t really break anything since the record is still pointing to the same Application Load Balancer
• alias — specifies whether or not this is an alias record. It is, since you’re not translating the domain name to an IP address but the load balancer’s DNS name
• alias_hosted_zone_id — specifies the target hosted zone ID. We defined this in the alb.yml file using the set_fact module.

The next task sets thestate parameter to “get” in order to capture the output from the hosted zone created. The nameservers are then saved using the set_fact module.

The last task displays the nameservers of the hosted zone. Use these nameservers to update your domain provider (e.g., Name.com), which will tell Internet service providers around the world to route traffic aimed at your domain name to AWS Route53’s nameservers. This process is known as DNS Propagation.

Step Five: Request a Public Certificate from AWS Certificate Manager and update the Application Load Balancer Listeners

AWS Certificate Manager (ACM) is a service that allows you to easily provision, manage, and deploy Secure Sockets Layer or Transport Layer Security (SSL or TLS) certificates for use with AWS services.

You have the option to either import a certificate or request a new public certificate from AWS. Since Ansible does not have a module for requesting a new public certificate using ACM at the time of writing, you’ll use the AWS CLI to request the certificate through Ansible’s shell module. Ensure that you have already configured the AWS CLI as stated in the prerequisites

1. In the roles/ directory, create the ssl/tasks/ directory
2. In the tasks/ directory, create the main.yml file

---
#roles/ssl/tasks/main.yml

# Ansible does not have a module to request a public certificate from aws certificate manager, hence the use of the cli

  - name: Request a public certificate
    shell: |
      aws acm request-certificate \
      --domain-name "{{ domain }}" \
      --validation-method DNS \
      --region "{{ aws_region }}" \
      --domain-validation-options DomainName={{ domain }},ValidationDomain={{ domain }} \
      --idempotency-token prevail
    register: new_acm
  

# Create a CNAME record while the certificate is still pending validation

  - name: Get the Certificate ARN
    set_fact:
certificate_arn: "" (new_acm.stdout|from_json). CertificateArn }}'

  - name: Retrieve information about the requested certificate
    shell: |
      aws acm describe-certificate --certificate-arn '{{ certificate_arn }}' --region '{{ aws_region }}'
    register: certificate_details
    retries: 13
    delay: 10
    until: (certificate_details.stdout|from_json). (certificate_details.stdout|from_json). Certificate.DomainValidationOptions[0].ResourceRecord.Name is defined
    

  - name: Create DNS(CNAME) record in route 53
    community.aws.route53:
      state: present
      ttl: 120
zone:'(certificate_details.stdout | from_json)'{{ (certificate_details.stdout|from_json). (certificate_details.stdout|from_json). Certificate.DomainValidationOptions[0].ValidationDomain }}'
record: " (certificate_details.stdout|from_json). (certificate_details.stdout|from_json). Certificate.DomainValidationOptions[0].ResourceRecord.Name }}'
type: "" (certificate_details.stdout|from_json). Certificate.DomainValidationOptions[0].ResourceRecord.Type }}'
value: "" (certificate_details.stdout|from_json). Certificate.DomainValidationOptions[0].ResourceRecord.Value }}'
      wait: yes
    

  - name: Wait for the certificate's status to become ISSUED
    shell: |
      aws acm describe-certificate --certificate-arn '{{ certificate_arn }}' --region '{{ aws_region }}'
    register: certificate_details
    retries: 85
    delay: 20
    until: (certificate_details.stdout|from_json). (certificate_details.stdout|from_json). Certificate.Status == "ISSUED"
    

# Update the application load balancer with https listeners and add a rule that redirects http

  - name: Update the ALB to redirect http traffic to https
    community.aws.elb_application_lb:
      name: '{{ alb_name }}'
      region: '{{ aws_region }}'
      security_groups: '{{ alb_group_id }}'
      subnets:
        - '{{ pub_01_subnet_id }}'
        - '{{ pub_02_subnet_id }}'
      listeners:
        - Protocol: HTTPS
          Port: 443
          SslPolicy: ELBSecurityPolicy-2016-08
          DefaultActions:
            - Type: forward
              TargetGroupName: '{{ tg_name }}'
          Certificates:
            - CertificateArn: '{{ certificate_arn }}'
        - Protocol: HTTP
          Port: 80
          DefaultActions:
            - Type: forward
              TargetGroupName: '{{ tg_name }}'
          Rules:
            - Conditions:
                - Field: path-pattern
                  Values: 
                    - "/*"
              Priority: '1'
              Actions:
                - Type: redirect
                  RedirectConfig:
                    Protocol: "HTTPS"
                    Port: "443"
                    Host: "#{host}"
                    Path: "/#{path}"
                    Query: "#{query}"
                    StatusCode: "HTTP_301"

The shell module is used to run the aws acm commands.

The aws acm request-certificate command requests a certificate for you from ACM with the following options:

• --domain_name — specifies the domain name to request the certificate for. This could be a subdomain or your domain itself
• --validation-method — specifies the method to validate that the domain name belongs to you. You could choose either DNS or Email validation. DNS is a better choice
• --region — specifies the region to request the certificate in
• --domain-validation-options — specifies the options that are required in order to validate your domain name. The list includes the DomainName which is your domain name and ValidationDomain which could be the same as your domain name or a superdomain if you are requesting the certificate for a subdomain
• --idempotency-token — specifies a token generated by you to ensure idempotency between request calls for a certificate. It typically times out after one hour.

The register module captures the output from running the command so that it can be used in other parts of the playbook.

The Certificate’s Amazon Resource Name(ARN) is stored with the set_fact module from the standard output (stdout) of the variable defined with the register module.

Remember earlier you configured the AWS CLI to output in JSON format, hence why we’re using the from_json ansible filter to manipulate the data to extract the ARN.

You will need to retrieve information about the just created certificate in order to create a DNS CNAME record in Route53.

You will use the aws acm describe-certificate command in the ansible shell module to achieve this with the following options:

• --certificate-arn — specifies the ARN of the certificate created (which we saved with the set_fact module). This option has to be specified to run the command
• --region — specifies the region in which the certificate was created in.

The register module captures the output of the command.

The retries module is used to retry a task until a condition is met. This module works hand-in-hand with delay which specifies the time, in seconds, the tasks is to be retried, and until which specifies the condition to be met.

The task is to be retried 13 times, every 10 seconds until the ResourceRecord Name in the output of the command is defined ( That’s 130 seconds = 2 minutes, 10 seconds).

You will then create the DNS record in Route53 using the community.aws.route53 module. The ttl parameter (time-to-live) specifies the time in seconds that a record should be cached by resolvers and other DNS clients. The zone , record , type , and value parameters are parsed from the output of the aws acm describe-certificate command captured with the register module.

Now you will wait for the certificate to become issued. You’ll use the aws acm describe-certificate along with the retries , delay , and until modules to wait for the output’s “status” to change from PENDING_VALIDATION to ISSUED.

The task is to be retried 85 times, every 20 seconds until the Status equals ISSUED. (That’s about 1700 seconds = 28 minutes 20 seconds)

Finally, you’ll update your Application Load Balancer to include a https listener and redirect http to https.

You’ll have to specify the name , region , subnets , and security_groups parameters again in the community.aws.elb_application_lb module to enable Ansible recognise which Application Load Balancer you want to update.

In the first item in the listeners parameter list, the SslPolicy and Certificates subkeys are added along side the Protocol , Port , and DefaulActions subkeys.

• The SslPolicy — specifies the security configurations that determine how the load balancer should handle SSL/TLS connections. AWS recommends ELBSecurityPolicy-2016–08
• Certificates — specifies the SSL certificate for an Application Load Balancer. We’ll use the ACM certificate ARN.

In the second item in the listeners parameter list, you will specify the Protocol , Port , and DefaultActions subkeys of the HTTP listener that you want to redirect to HTTPS. You will use the Rules subkey to define a set of rules for the redirection.

The Rules subkey is defined with the following parameters:

• Conditions — specifies the conditions to be met before the rules can be applied. The Field key specifies the criteria used to match incoming requests to the rule, such as the host header, path, or query string. You’re matching incoming requests to the “/*” path condition which means that the rule will match any incoming request regardless of its path
• Priority — specifies the order in which rules are evaluated, with the rule having the highest priority being evaluated first. Rules with the lowest value have the highest priority, while the default rule is always evaluated last. The priority value is set to ‘1’, hence this rule will be evaluated first.
• Actions — the action that should be taken when a request matches the conditions specified in the Conditions subkey. The Type of action is set to redirect . This basically redirects the HTTP traffic to the HTTPS traffic with the same Host , Path , and Query parameters. The StatusCode is set to "HTTP_301", which is used for permanent redirects.

Step Six: Include the Roles in the main.yml file

The last step in writing the playbook is to include all the roles in the main.yml file

---
#main.yml

- name: Provisioning AWS resources using Ansible
  hosts: localhost
  connection: local
  gather_facts: False

  roles:
    - name: Create a logical network
      role: provision_vpc
    - name: Create an autoscaling group
      role: autoscaling
    - name: Create an application load balancer
      role: load_balancer
    - name: Link the load balancer to route53
      role: route53
    - name: Create a certificate manager, add it to the load balancer and update load balancer rules
      role: ssl

Now that you have your complete playbook, the next thing is to run the playbook.

ansible-playbook main.yml 

After running the playbook you should have all the necessary AWS resources provisioned and your domain name should be up and running.