<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Irfan Danish on Medium]]></title>
        <description><![CDATA[Stories by Irfan Danish on Medium]]></description>
        <link>https://medium.com/@irdanish?source=rss-4f1b8073abf9------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*mHg9w7Ml-SDx6ClNR-cI2A.jpeg</url>
            <title>Stories by Irfan Danish on Medium</title>
            <link>https://medium.com/@irdanish?source=rss-4f1b8073abf9------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 22:15:30 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@irdanish/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Unlocking RDS PostgreSQL in a Private Subnet: Exploring Different Connection Approaches]]></title>
            <link>https://towardsaws.com/unlocking-rds-postgresql-in-a-private-subnet-exploring-different-connection-approaches-7101cc52040f?source=rss-4f1b8073abf9------2</link>
            <guid isPermaLink="false">https://medium.com/p/7101cc52040f</guid>
            <category><![CDATA[aws-rds]]></category>
            <category><![CDATA[ssh-tunnel]]></category>
            <category><![CDATA[postgres]]></category>
            <category><![CDATA[rds-connection]]></category>
            <category><![CDATA[rds]]></category>
            <dc:creator><![CDATA[Irfan Danish]]></dc:creator>
            <pubDate>Sat, 10 Jun 2023 12:02:56 GMT</pubDate>
            <atom:updated>2023-08-16T09:44:42.568Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2hwncCZMfnmDU7E15A1_bw.png" /></figure><p>In my earlier years when I was starting my development career I really find it hard to connect to RDS Databases in private subnet to setup my local environment so I can run some SQL queries on the databases, as I didn’t find any proper guide. In this article we will look into different ways how we can connect to a RDS PostgreSQL database.</p><figure><a href="https://bmc.link/irfandanish"><img alt="" src="https://cdn-images-1.medium.com/max/170/0*OBC8Mo-L-Pbb61uc.png" /></a></figure><p>I assume that you have setup your SSH key for your bastion host, if you haven’t gone through my earlier articles I would recommend checking on we have created <a href="https://towardsaws.com/creating-a-bastion-host-for-secure-access-to-your-aws-infrastructure-with-terraform-17ee287bb3d">EC2 Bastion Host</a> and a <a href="https://towardsaws.com/rds-deploying-scalable-and-resilient-relational-databases-using-terraform-iac-5fbf5e4d717b">RDS PostgreSQL</a> using Terraform. In the article EC2 Bastion Host article we have created SSH Key pair to access bastion host, but that was generic key, if you want to create a user specific SSH key, you can follow the following steps:</p><p>Create SSH Key pair in your local system using following command:</p><pre>ssh-keygen ~/.ssh/&lt;key-name&gt;</pre><p>Then add the contents of the public key (file with .pub extension) to the authorized keys by copying and and pasting the contents in the following file on the EC2 Bastion host using following command:</p><pre>vi ~/.ssh/authorized_keys<br><br>or <br><br>nano ~/.ssh/authorized_keys</pre><p>Now in order to connect to EC2 Bastion Host if your EC2 bastion host is restricted (not allowing access from anywhere on the internet), then in that case either add your IPv4 address Security Group of EC2 Bastion host or if you/your organization uses VPN to connect to AWS, then make sure that your are using that VPN to connect to the EC2 Bastion host.</p><p>From now on I will assume that infrastructure that we have provisioned in the previous articles. We have created EC2 Bastion host using Amazon Linux so our username for our bastion host will be ec2-user.</p><h4>Connecting from your Terminal:</h4><p>In order to connect from your terminal we will use following command:</p><pre>nohup ssh -i &quot;&lt;key-name&gt;&quot; -NL 8886:&lt;postgres-rds-endpoint&gt;:5432 \<br>ec2-user@&lt;bastion-public-ipv4-address&gt; &amp;</pre><p>In the above command we are using nohup to execute the command in the background and ensure it continues running even if the terminal session is closed. -NL 8886:&lt;postgres-rds-endpoint&gt;:5432 sets up a local port forwarding. It binds the local port 8886 of EC2 Bastion host to the &lt;postgres-rds-endpoint&gt; and port 5432 on the bastion host. This allows communication between the local machine and the PostgreSQL database running on the remote server.</p><p>By running this command, you create a tunnel through which you can securely access the PostgreSQL database hosted on the remote server using a local client on your machine. The local port 8886 is forwarded to the PostgreSQL service running on the remote server, enabling you to interact with the database as if it were running locally.</p><p>Now to check if the tunnel is established correctly or not we can use following command:</p><pre>ps -ef | grep ssh<br><br>or<br><br>netstat -nta tcp | grep -i LISTEN | grep 8886</pre><p>To test connection to RDS use following command and provide the password:</p><pre>psql -h &lt;rds-endpoint&gt; -p 5432 -U &lt;user-name&gt; -d postgres</pre><h4>Connecting from Python:</h4><p>In order to connect from python we will be creating an SSH Tunnel using python, in order to connect to RDS PostgreSQL from python we need to have following dependencies installed on virtual environment:</p><pre>pip install sshtunnel psycopg2-binary</pre><p>We can also use psycopg2 instead of psycopg2-binary, but usually it requires to install some extra build dependencies because psycopg2 wheel is built from source during installation on case of simple psycopg2.</p><p>Once these packages are installed we connect to RDS PostgreSQL using following code:</p><pre>import psycopg2<br>from typing import Tuple<br>from sshtunnel import SSHTunnelForwarder<br><br># Type Aliases<br>Connection = psycopg2.extensions.connection<br>TunnelConnection = Tuple[SSHTunnelForwarder, Connection]<br><br><br>def get_tunnel_db_connection(bastion_host_ip: str, bastion_user: str,<br>                             bastion_pkey: str, db_name: str, db_password: str,<br>                             db_user: str, rds_host: str) -&gt; TunnelConnection:<br>    bastion_port = 22<br>    rds_port = 5432<br>    server = SSHTunnelForwarder(<br>        (bastion_host_ip, bastion_port),<br>        ssh_username=bastion_user,<br>        ssh_pkey=bastion_pkey,<br>        remote_bind_address=(rds_host, rds_port))<br>    server.start()<br>    connection = psycopg2.connect(<br>        host=&quot;127.0.0.1&quot;,<br>        port=server.local_bind_port,<br>        dbname=db_name,<br>        password=db_password,<br>        user=db_user<br>    )<br>    return server, connection<br><br><br># creates the tunnel and connection to the database<br>tunnel, conn = get_tunnel_db_connection(<br>    bastion_host_ip=&quot;&lt;public-ipv4-ec2-bastion-host&gt;&quot;,<br>    bastion_user=&quot;ec2-user&quot;,<br>    bastion_pkey=&quot;path/to/key/ec2-bastion-key-pair.pem&quot;,<br>    db_name=&quot;postgres&quot;,<br>    db_password=&quot;&lt;db-password&gt;&quot;,<br>    db_user=&quot;infrafypostgres&quot;,<br>    rds_host=&quot;rds-endpoint&quot;<br>)<br><br># execute the query<br>cursor = conn.cursor()<br>cursor.execute(&#39;SELECT datname FROM pg_database&#39;)<br>result = cursor.fetchall()<br>print(f&quot;Result: \n{result}&quot;)<br># close the tunnel<br>tunnel.stop()</pre><p>In the above code we are using sshtunnel library to create the tunnel and the psycopg2 library to interact with the database.</p><p>The get_tunnel_db_connection function sets up an SSH tunnel between the local machine and a bastion host, forwarding traffic to the specified PostgreSQL database. It then establishes a connection to the database using psycopg2, configuring it to use the local host and the port assigned to the tunnel. The function returns the SSH tunnel object and the database connection.</p><p>By calling the get_tunnel_db_connection function and providing the necessary parameters, you can establish a secure connection to the PostgreSQL database. The code then executes a query on the database, retrieves the results, and prints them. Finally, the SSH tunnel is closed to terminate the connection. This approach enables secure access to the database through an SSH tunnel, allowing you to interact with the PostgreSQL database from your local machine.</p><h4>Connecting using pgAdmin4 by creating SSH tunnel:</h4><ol><li>In order to connect to your RDS PostgreSQL database from pgAdmin4, open your pgAdmin4 application and select Servers/Register/Server as shown below:</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/966/1*EMTEHOgx-hfqs2S72WwJhg.png" /></figure><p>2. By following the above step a pop window will open as shown below, enter any name in the field Name. This name will be shown in the pgAdmin4.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/652/1*0gDLYXJHI05lB1XF9ISODA.png" /></figure><p>3. Then click on the Connectiontab as shown below, in the field Host name/address add the endpoint of your RDS PostgreSQL database. In the Port field enter 5432 which is the port on which PostgreSQL runs. In the usernameand password fields add your username and password, as shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/637/1*U37RtXfO_61MwtHY-DDHuw.png" /></figure><p>4. Then click on the SSH Tunnel tab, here we will enter the details of EC2 Bastion host. In the Tunnel host field add the IPv4 of EC2 Bastion host. Add 22 to the tunnel port, add ec2-user as user name (if bastion host is ubuntu server then add ubuntu). Then in the Identity file field add/browse the path to your private SSH key and click on save as shown below. Once you will click on save it will start connection process to your database.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/638/1*5SrPogoEAVb4lZ2CMhob3g.png" /></figure><p>5. Once you are successfully connected to your database it will appear as shown below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2YCskpdcCGpt-7CQ104FiA.png" /></figure><h3>Conclusion:</h3><p>In conclusion, we have explored three different approaches to connect to an RDS PostgreSQL database in a private subnet. The first approach involved establishing an SSH tunnel using the terminal or command line, providing flexibility and secure access. The second approach utilized the sshtunnel library in Python, allowing us to create an SSH tunnel programmatically and interact with the database using Python. Lastly, we explored connecting through pgAdmin4, offering a graphical interface for managing the database. By understanding these approaches, you can choose the most suitable method for connecting to your RDS PostgreSQL database based on your preferences and specific use case.</p><h3>Recommended Readings:</h3><ul><li><a href="https://towardsaws.com/devops-1-setting-up-a-terraform-development-environment-for-aws-5134274dde5a">DevOps: 1 — Setting up a Terraform Development Environment for AWS</a></li><li><a href="https://medium.com/geekculture/devops-2-aws-vpc-virtual-private-cloud-part-i-412ad4ea0cc9">Building a Secure AWS VPC with Terraform: Subnets, Internet Gateways, and More — Part-I</a></li><li><a href="https://towardsaws.com/building-a-secure-aws-vpc-with-terraform-subnets-inetnet-gateways-and-more-part-ii-14369aac292">Building a Secure AWS VPC with Terraform: Subnets, Internet Gateways, and More — Part-II</a></li><li><a href="https://towardsaws.com/creating-a-bastion-host-for-secure-access-to-your-aws-infrastructure-with-terraform-17ee287bb3d">Creating a Bastion Host for Secure Access to Your AWS Infrastructure with Terraform</a></li><li><a href="https://medium.com/towards-aws/rds-deploying-scalable-and-resilient-relational-databases-using-terraform-iac-5fbf5e4d717b">RDS: Deploying Scalable and Resilient Relational Databases Using Terraform IaC</a></li></ul><h3>Final Note</h3><figure><a href="https://miro.medium.com/v2/resize:fit:340/format:webp/0*D2TPjtsNBM15fYXS.png"><img alt="" src="https://cdn-images-1.medium.com/max/170/0*V5sH11NEn2llWs28.png" /></a></figure><p>If you enjoyed this article and found it useful, be sure to follow me on <a href="https://irdanish.medium.com/">Medium</a> and <a href="https://github.com/irdanish11">GitHub</a> for more content like this. On Medium, you can find more articles on Cloud Computing, DevOps, Machine Learning and other related topics. On GitHub, you can find my open-source projects and code samples. By following me on these platforms, you can stay up-to-date with my latest work and learn more about best practices for managing infrastructure with Terraform and other cloud tools. Thanks for reading!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7101cc52040f" width="1" height="1" alt=""><hr><p><a href="https://towardsaws.com/unlocking-rds-postgresql-in-a-private-subnet-exploring-different-connection-approaches-7101cc52040f">Unlocking RDS PostgreSQL in a Private Subnet: Exploring Different Connection Approaches</a> was originally published in <a href="https://towardsaws.com/">Towards AWS</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[RDS: Deploying Scalable and Resilient Relational Databases Using Terraform IaC]]></title>
            <link>https://towardsaws.com/rds-deploying-scalable-and-resilient-relational-databases-using-terraform-iac-5fbf5e4d717b?source=rss-4f1b8073abf9------2</link>
            <guid isPermaLink="false">https://medium.com/p/5fbf5e4d717b</guid>
            <category><![CDATA[aws-rds]]></category>
            <category><![CDATA[postgresql]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[multi-az]]></category>
            <category><![CDATA[rds]]></category>
            <dc:creator><![CDATA[Irfan Danish]]></dc:creator>
            <pubDate>Mon, 29 May 2023 06:43:36 GMT</pubDate>
            <atom:updated>2023-05-30T06:58:27.417Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1001/1*hN_-iBWY0s60QNb2njZpuQ.png" /></figure><figure><a href="https://bmc.link/irfandanish"><img alt="" src="https://cdn-images-1.medium.com/max/170/1*HZd8ZXyxRuTIm4MAEsJBYw.png" /></a></figure><p>Greetings, dear readers ☺️! I hope you’re all doing well and enjoying your time. In some of our previous articles, we explored the creation of a custom managed <a href="https://towardsaws.com/building-a-secure-aws-vpc-with-terraform-subnets-inetnet-gateways-and-more-part-ii-14369aac292">VPC</a> and an <a href="https://towardsaws.com/creating-a-bastion-host-for-secure-access-to-your-aws-infrastructure-with-terraform-17ee287bb3d">EC2 Bastion Host</a>. Now, armed with this knowledge, we are prepared to take the next step: provisioning a 3-tier application architecture. Building upon our previous work, we will dive into the exciting world of designing and deploying a robust and scalable infrastructure for hosting multi-tier applications. So, fasten your seatbelts and get ready to embark on this fascinating journey!</p><h3>What is a 3-tier architecture?</h3><p>A three-tier architecture is a widely adopted design pattern in software development that separates an application into three distinct layers:</p><ul><li>Presentation Layer (Web Tier).</li><li>Application or business logic (App Tier).</li><li>Data Storage Layer (Database Tier).</li></ul><p>The presentation layer is responsible for the user interface, allowing users to interact with the application. The application layer contains the core functionality and business logic, processing user requests and implementing the application’s rules. The data storage layer handles data persistence, ensuring that data is securely stored and retrieved when needed.</p><p>A three-tier architecture is essential for real-world applications due to its promotion of modularity, code reusability, scalability, and performance optimization. By separating the application into distinct layers, developers can work independently on each layer, making maintenance and updates easier while minimizing bugs and unintended consequences. Each layer can be scaled individually to handle increased traffic, and specific optimizations can be applied. Overall, this architecture empowers developers to build robust, flexible, and efficient applications that are easier to develop, maintain, and scale in real-world scenarios.</p><p>We will build a complete 3-tier application architecture in our upcoming articles, starting from today where we will look into Database Tier or Data Storage Layer. In this article we will look in AWS’s managed relational database service RDS, and provision a highly available and fault tolerant RDS Database using Terraform.</p><h3>AWS RDS — Relational Database Services</h3><p>Amazon RDS (Relational Database Service) is a managed database service by AWS that makes it easier to set up, operate, and scale relational databases in the cloud. RDS supports popular database engines such as MySQL, PostgreSQL, Oracle, and SQL Server, offering a reliable and highly available solution for storing and managing structured data.</p><p>With AWS RDS, users can offload the burden of database administration tasks, such as provisioning, patching, and backups, to AWS, allowing them to focus on developing applications. AWS RDS also provides high availability through multi-AZ (Availability Zone) deployments, where data is automatically replicated across multiple geographically separate AZs to provide failover support. This ensures that databases remain accessible even in the event of an infrastructure failure. RDS also offers automated backups, enabling point-in-time recovery and eliminating the need for manual backups.</p><p>RDS offers built-in security features such as encryption at rest and in transit, network isolation through Amazon VPC, and integration with AWS Identity and Access Management (IAM) for fine-grained access control.</p><p>With this brief introduction of RDS, now we will move towards provisioning a RDS Database using Terraform. For this article we are going to deploy AWS RDS database with PostgreSQL engine in Multi-AZ setting.</p><h3>Provisioning Multi-AZ RDS PostgreSQL in Terraform</h3><p>We will be building on our existing IaC which has the terraform manifests for VPC, and Bastion Host. You can find all of the code for this article on my <a href="https://github.com/irdanish11">GitHub</a> <a href="https://github.com/irdanish11/terraform-iac/tree/main/aws/infrastructure/04-rds-database">Terraform Repository</a>.</p><h4>Variables:</h4><p>First of all we create variables required for our RDS Database. Add the following to variables.tf.</p><pre>## RDS variables<br>variable &quot;rds-postgres-db-username&quot; {<br>  type = string<br>}<br><br>variable &quot;rds-postgres-db-password&quot; {<br>  type = string<br>}<br><br>variable &quot;rds-postgres-db-name&quot; {<br>  type = string<br>}<br><br>variable &quot;rds-postgres-db-port&quot; {<br>  type = number<br>}<br><br>variable &quot;rds-postgres-db-maintenance-window&quot; {<br>  type = string<br>}</pre><p>Then add following to the terraform.tfvars:</p><pre>## RDS Variables<br>rds-postgres-db-username           = &quot;dbusername&quot;<br>rds-postgres-db-password           = &quot;dbpassword&quot;<br>rds-postgres-db-port               = 5432<br>rds-postgres-db-name               = &quot;dbname&quot;<br>rds-postgres-db-maintenance-window = &quot;Sun:00:00-Sun:03:00&quot;</pre><p>Here we have configured some of the credentials for our RDS Database, a more secure approach is to manage you RDS secrets and credentials using AWS Secrets Manager, which we will demonstrate in our upcoming articles.</p><h4>Database Subnet Group:</h4><p>First of all we will provsion a Database (DB) subnet group. A DB subnet group is a logical grouping of subnets within a virtual private cloud (VPC) that is specifically designed for hosting database instances. The purpose of a DB subnet group is to define the subnets where the RDS instance will be deployed. RDS ensures that the DB instance is provisioned within the specified subnets.</p><p>DB subnet groups enable high availability and fault tolerance for RDS instances. By distributing the RDS instances across multiple availability zones (AZs) within the selected subnets, it ensures that the database remains accessible even in the event of an AZ failure. This helps to minimize downtime and maintain data durability.</p><pre>resource &quot;aws_db_subnet_group&quot; &quot;db-subnet-group&quot; {<br>  name = &quot;${var.project}-db-subnet-group-${var.environment}&quot;<br>  subnet_ids = [<br>    aws_subnet.vpc-private-subnet-1.id,<br>    aws_subnet.vpc-private-subnet-2.id,<br>    aws_subnet.vpc-private-subnet-3.id<br>  ]<br>  tags = {<br>    Name = &quot;${var.project}-db-subnet-group-${var.environment}&quot;<br>  }<br>}</pre><p>️Here we have created the DB subnet group and provided the subnet ids of our private subnets.</p><h4>Security Configurations:</h4><p>Next we will configure some security measures for our RDS instance. First we will create security group for our RDS instance.</p><pre>## Security Group for RDS Postgres<br>resource &quot;aws_security_group&quot; &quot;rds-postgres-sg&quot; {<br>  name        = &quot;${var.project}-rds-postgres-sg-${var.environment}&quot;<br>  description = &quot;Security group for RDS Postgres&quot;<br>  vpc_id      = aws_vpc.kodetronix-vpc.id<br>  ingress {<br>    description     = &quot;Postgres port&quot;<br>    from_port       = var.rds-postgres-db-port<br>    to_port         = var.rds-postgres-db-port<br>    protocol        = &quot;tcp&quot;<br>    security_groups = [aws_security_group.ec2-bastion-sg.id]<br>  }<br>}</pre><p>As of now as you can see that only from our EC2 Bastion Host we would be able to connect to our RDS Database, as ingress is only allowed from the security group of Bastion Host.</p><p>Next we will create KMS key that will be used to encrypt replicas and a parameter group fro PostgreSQL which will allow only SSL connections to the database.</p><pre>## KMS Key for RDS Postgres<br>resource &quot;aws_kms_key&quot; &quot;db_key&quot; {<br>  description         = &quot;KMS key for RDS Postgres&quot;<br>  key_usage           = &quot;ENCRYPT_DECRYPT&quot;<br>  enable_key_rotation = true<br>}<br><br>## Parameter Group for RDS Postgres<br>resource &quot;aws_db_parameter_group&quot; &quot;rds-postgres-pg&quot; {<br>  name        = &quot;${var.project}-rds-postgres-pg-${var.environment}&quot;<br>  family      = &quot;postgres14&quot;<br>  description = &quot;Custom Parameter Group for RDS Postgres&quot;<br>  parameter {<br>    name         = &quot;rds.force_ssl&quot;<br>    value        = &quot;1&quot;<br>    apply_method = &quot;immediate&quot;<br>  }<br>  tags = {<br>    Name = &quot;${var.project}-rds-postgres-pg-${var.environment}&quot;<br>  }<br>}</pre><h4>Create RDS Database Instance:</h4><p>Now we will provision our PostgreSQL RDS Database Instance:</p><pre>resource &quot;aws_db_instance&quot; &quot;rds-postgres&quot; {<br>  identifier                            = &quot;${var.project}-rds-postgres-db-${var.environment}&quot;<br>  engine                                = &quot;postgres&quot;<br>  engine_version                        = &quot;14.6&quot;<br>  instance_class                        = &quot;db.t4g.large&quot;<br>  allocated_storage                     = 20<br>  max_allocated_storage                 = 100<br>  storage_type                          = &quot;gp2&quot;<br>  storage_encrypted                     = true<br>  kms_key_id                            = aws_kms_key.db_key.arn<br>  db_name                               = var.rds-postgres-db-name<br>  username                              = var.rds-postgres-db-username<br>  password                              = var.rds-postgres-db-password<br>  port                                  = var.rds-postgres-db-port<br>  multi_az                              = true<br>  network_type                          = &quot;IPV4&quot;<br>  db_subnet_group_name                  = aws_db_subnet_group.db-subnet-group.id<br>  vpc_security_group_ids                = [aws_security_group.rds-postgres-sg.id]<br>  deletion_protection                   = false<br>  allow_major_version_upgrade           = false<br>  auto_minor_version_upgrade            = true<br>  apply_immediately                     = false<br>  backup_retention_period               = 30<br>  backup_window                         = &quot;21:00-23:00&quot;<br>  maintenance_window                    = var.rds-postgres-db-maintenance-window<br>  copy_tags_to_snapshot                 = true<br>  delete_automated_backups              = true<br>  enabled_cloudwatch_logs_exports       = [&quot;postgresql&quot;, &quot;upgrade&quot;]<br>  performance_insights_enabled          = true<br>  performance_insights_kms_key_id       = aws_kms_key.db_key.arn<br>  performance_insights_retention_period = 7<br>  publicly_accessible                   = false<br>  parameter_group_name                  = aws_db_parameter_group.rds-postgres-pg.id<br>  skip_final_snapshot                   = false<br>  final_snapshot_identifier             = &quot;${var.project}-rds-postgres-db-final-snapshot-${var.environment}&quot;<br>}</pre><p>For our RDS Database we select PostgreSQL engine version 14.6, for DB instance we are using db.t4g.large, which is a general purpose burstable instance hat provide a baseline level of CPU performance with the ability to burst CPU usage at any time for as long as required. It has 2 vCPU based on AWS Graviton2 architecture and 8 GB of memory.</p><p>In our configuration, we assign 20 GB of storage to the RDS instance, and we enable storage autoscaling, which enables the RDS to dynamically adjust its storage capacity as needed, with a maximum limit of 100 GB. Additionally, we enable Multi-AZ, which ensures that the database data is asynchronously replicated across instances located in different availability zones (AZs).</p><p>This replication provides fault tolerance and high availability, as the system can automatically failover to an instance in another AZ in the event of infrastructure failures or other outages. By utilizing these features, we enhance the scalability and resilience of our RDS database, ensuring data durability and minimizing downtime.</p><blockquote><strong>Caution! Here we have set the </strong><strong>deletion_protection to </strong><strong>false, this is only for demo purposes so we can tear down infrastructure easily. In a production setting you should enable the deletion protection.</strong></blockquote><p>For PostgreSQL engine version upgrades, we have restricted it to minor versions only, while major version upgrades are disabled. As for automated backups, they are retained for a period of 30 days. It is crucial to note that the backup window and maintenance window should never overlap.</p><p>In our setup, backups are scheduled between 9 PM and 11 PM UTC, while the maintenance window is set for every Sunday between 12 AM and 03 AM. To gain in-depth performance insights and analysis, we have also enabled performance insights, which will provide detailed insights at the query and user levels. With these configurations in place, we have successfully met the minimum requirements for deploying our RDS database.</p><h4>Database Observability and Telemetry:</h4><p>Having fulfilled the basic prerequisites for provisioning our RDS database, it’s time to enhance its observability and telemetry. Our next step involves setting up CloudWatch alarms to monitor the CPU, Memory, and Storage utilization of our RDS database. Additionally, we’ll create an SNS topic and set up a subscription for that topic.</p><p>Whenever any of the mentioned metrics exceed the alarm threshold, the SNS topic will promptly send an email notification to our subscribed email address, enabling us to take appropriate actions in a timely manner. This enhanced monitoring and notification system ensures that we stay informed about the health and performance of our RDS database and empowers us to respond effectively whenever necessary.</p><pre>resource &quot;aws_sns_topic&quot; &quot;databases-sns-topic&quot; {<br>  name = &quot;${var.project}-databases-sns-topic-${var.environment}&quot;<br>  tags = {<br>    Name = &quot;${var.project}-databases-sns-topic-${var.environment}&quot;<br>  }<br>}<br><br>resource &quot;aws_sns_topic_subscription&quot; &quot;db-alerts-sns-subscription&quot; {<br>  topic_arn = aws_sns_topic.databases-sns-topic.arn<br>  protocol  = &quot;email&quot;<br>  endpoint  = &quot;dbalerts@kodetronix.com&quot;<br>}</pre><p>In above code snippet we first create SNS topic and then a SNS topic subscription which will send the email notification.</p><pre>## Cloudwatch alarm for RDS CPU utilization<br>resource &quot;aws_cloudwatch_metric_alarm&quot; &quot;rds-postgres-cpu-utilization-alarm&quot; {<br>  alarm_name          = &quot;${var.project}-rds-postgres-cpu-utilization-alarm-${var.environment}&quot;<br>  comparison_operator = &quot;GreaterThanOrEqualToThreshold&quot;<br>  evaluation_periods  = &quot;1&quot;<br>  metric_name         = &quot;CPUUtilization&quot;<br>  namespace           = &quot;AWS/RDS&quot;<br>  period              = &quot;300&quot;<br>  statistic           = &quot;Average&quot;<br>  threshold           = &quot;80&quot;<br>  alarm_description   = &quot;This metric monitors RDS Postgres CPU utilization&quot;<br>  alarm_actions       = [aws_sns_topic.databases-sns-topic.arn]<br>  ok_actions          = [aws_sns_topic.databases-sns-topic.arn]<br>  dimensions = {<br>    DBInstanceIdentifier = aws_db_instance.rds-postgres.id<br>  }<br>}<br> ## Cloudwatch alarm for RDS Memory utilization<br>resource &quot;aws_cloudwatch_metric_alarm&quot; &quot;rds-postgres-memory-utilization-alarm&quot; {<br>  alarm_name          = &quot;${var.project}-rds-postgres-memory-utilization-alarm-${var.environment}&quot;<br>  comparison_operator = &quot;GreaterThanOrEqualToThreshold&quot;<br>  evaluation_periods  = &quot;1&quot;<br>  metric_name         = &quot;FreeableMemory&quot;<br>  namespace           = &quot;AWS/RDS&quot;<br>  period              = &quot;300&quot;<br>  statistic           = &quot;Average&quot;<br>  threshold           = &quot;6400&quot;<br>  alarm_description   = &quot;This metric monitors RDS Postgres memory utilization&quot;<br>  alarm_actions       = [aws_sns_topic.databases-sns-topic.arn]<br>  ok_actions          = [aws_sns_topic.databases-sns-topic.arn]<br>  dimensions = {<br>    DBInstanceIdentifier = aws_db_instance.rds-postgres.id<br>  }<br>}<br><br>## Cloudwatch alarm for RDS Storage utilization<br>resource &quot;aws_cloudwatch_metric_alarm&quot; &quot;rds-postgres-storage-utilization-alarm&quot; {<br>  alarm_name          = &quot;${var.project}-rds-postgres-storage-utilization-alarm-${var.environment}&quot;<br>  comparison_operator = &quot;GreaterThanOrEqualToThreshold&quot;<br>  evaluation_periods  = &quot;1&quot;<br>  metric_name         = &quot;FreeStorageSpace&quot;<br>  namespace           = &quot;AWS/RDS&quot;<br>  period              = &quot;3600&quot;<br>  statistic           = &quot;Average&quot;<br>  threshold           = &quot;16000&quot;<br>  alarm_description   = &quot;This metric monitors RDS Postgres storage utilization&quot;<br>  alarm_actions       = [aws_sns_topic.databases-sns-topic.arn]<br>  ok_actions          = [aws_sns_topic.databases-sns-topic.arn]<br>  dimensions = {<br>    DBInstanceIdentifier = aws_db_instance.rds-postgres.id<br>  }<br>}</pre><p>We establish thresholds of 80 percent for CPU, memory, and storage utilization. When the CPU usage consistently exceeds 80 percent, the CPU CloudWatch alarm is triggered. Similarly, the Memory CloudWatch alarm is activated when memory usage remains above 6.4 GB continuously for more than 5 minutes. For storage, the alarm is triggered when 16 GB of storage is used and remains at that level for more than one hour. By setting these thresholds and alarms, we can proactively monitor the performance of our system and receive timely notifications when any of these metrics reach critical levels, allowing us to take prompt action and ensure the smooth operation of our infrastructure.</p><h3>Conclusion:</h3><p>In conclusion, we have explored the concept of a 3-tier architecture, understanding its significance in building scalable and modular applications. By separating our application into distinct layers, we achieve improved maintainability and scalability. Leveraging Terraform, we provisioned an Amazon RDS instance, configuring its storage, backups, and other parameters. With features like storage autoscaling, Multi-AZ deployment, and performance insights, we enhanced the resilience, availability, and performance of our database. By combining these principles and tools, we have established a solid foundation for building robust and scalable applications, ready to meet the demands of modern systems.</p><h4>Recommended Readings:</h4><ul><li><a href="https://towardsaws.com/devops-1-setting-up-a-terraform-development-environment-for-aws-5134274dde5a">DevOps: 1 — Setting up a Terraform Development Environment for AWS</a></li><li><a href="https://medium.com/geekculture/devops-2-aws-vpc-virtual-private-cloud-part-i-412ad4ea0cc9">Building a Secure AWS VPC with Terraform: Subnets, Internet Gateways, and More — Part-I</a></li><li><a href="https://towardsaws.com/building-a-secure-aws-vpc-with-terraform-subnets-inetnet-gateways-and-more-part-ii-14369aac292">Building a Secure AWS VPC with Terraform: Subnets, Internet Gateways, and More — Part-II</a></li><li><a href="https://towardsaws.com/creating-a-bastion-host-for-secure-access-to-your-aws-infrastructure-with-terraform-17ee287bb3d">Creating a Bastion Host for Secure Access to Your AWS Infrastructure with Terraform</a></li></ul><h3>Final Note</h3><figure><a href="https://bmc.link/irfandanish"><img alt="" src="https://cdn-images-1.medium.com/max/170/0*D2TPjtsNBM15fYXS.png" /></a></figure><p>If you enjoyed this article and found it useful, be sure to follow me on <a href="https://irdanish.medium.com/">Medium</a> and <a href="https://github.com/irdanish11">GitHub</a> for more content like this. On Medium, you can find more articles on Cloud Computing, DevOps, Machine Learning and other related topics. On GitHub, you can find my open-source projects and code samples. By following me on these platforms, you can stay up-to-date with my latest work and learn more about best practices for managing infrastructure with Terraform and other cloud tools. Thanks for reading!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5fbf5e4d717b" width="1" height="1" alt=""><hr><p><a href="https://towardsaws.com/rds-deploying-scalable-and-resilient-relational-databases-using-terraform-iac-5fbf5e4d717b">RDS: Deploying Scalable and Resilient Relational Databases Using Terraform IaC</a> was originally published in <a href="https://towardsaws.com/">Towards AWS</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Creating a Bastion Host for Secure Access to Your AWS Infrastructure with Terraform]]></title>
            <link>https://towardsaws.com/creating-a-bastion-host-for-secure-access-to-your-aws-infrastructure-with-terraform-17ee287bb3d?source=rss-4f1b8073abf9------2</link>
            <guid isPermaLink="false">https://medium.com/p/17ee287bb3d</guid>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[bastion-host]]></category>
            <category><![CDATA[ec2]]></category>
            <category><![CDATA[iac]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Irfan Danish]]></dc:creator>
            <pubDate>Wed, 10 May 2023 06:50:42 GMT</pubDate>
            <atom:updated>2023-05-30T06:57:22.474Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3lyIUK_cboNyGTZzFKj4vA.png" /><figcaption>Photo by Author Irfan Danish</figcaption></figure><figure><a href="https://bmc.link/irfandanish"><img alt="" src="https://cdn-images-1.medium.com/max/170/1*WfbwTpxi0c0k2Vwzp3s5yg.png" /></a></figure><p>In my previous article <a href="https://medium.com/geekculture/devops-2-aws-vpc-virtual-private-cloud-part-i-412ad4ea0cc9">Building a Secure AWS VPC with Terraform: Subnets, Internet Gateways, and More — Part-I</a> and <a href="https://towardsaws.com/building-a-secure-aws-vpc-with-terraform-subnets-inetnet-gateways-and-more-part-ii-14369aac292">Part-II</a> , we explored the process of provisioning a secure Virtual Private Cloud (VPC) infrastructure on AWS. We discussed various components like subnets, security groups, and network ACLs to ensure a robust and protected environment. Building upon that foundation, today we will delve into the concept of an EC2 bastion host.</p><p>In today’s interconnected world, ensuring the security of our cloud infrastructure is paramount. As organizations increasingly adopt cloud technologies, it becomes crucial to implement robust security measures to safeguard sensitive data and systems. One such security measure is the use of an <strong><em>bastion host</em></strong> — an essential component of a secure infrastructure setup. In this article, we will explore the concept of an EC2 bastion host, its significance, the benefits it offers in enhancing security within your AWS environment and walk through the step-by-step process of setting it up. So, if you’re ready to strengthen the security of your infrastructure and manage your resources more effectively, let’s dive into the world of EC2 bastion hosts.</p><h3>Demystifying the Bastion Host:</h3><p>A bastion host, also known as a jump box or a jump server, acts as a fortified gateway between external networks (e.g., the internet) and internal private networks (private subnets). While this isolation adds an extra layer of security, it can pose challenges when administrators or developers require remote access for maintenance, debugging, or troubleshooting purposes. This is where a bastion host comes into play, it serves as a secure and controlled access point, allowing authorized users to connect to resources within the private network from remote locations. By acting as a single entry point, the bastion host minimizes the exposure of your internal network, reducing the attack surface and strengthening the overall security posture.</p><h3>Benefits of Using an EC2 Bastion Host:</h3><ol><li><strong><em>Enhanced Security:</em></strong> The bastion host acts as a gatekeeper, granting access to specific users or IP addresses while enforcing strong authentication and secure protocols. It helps protect your private instances from direct exposure to the internet, reducing the risk of unauthorized access and potential security breaches.</li><li><strong><em>Centralized Access Control:</em></strong> With an EC2 bastion host in place, you can enforce fine-grained access control policies, limiting the number of users who have direct access to the private instances. This centralized approach ensures better governance and auditing capabilities, making it easier to track and monitor access to critical resources.</li><li><strong><em>Simplified Networking:</em></strong> By utilizing an EC2 bastion host, you can streamline your network architecture. Rather than exposing multiple instances to the public internet, you consolidate remote access through a single entry point, reducing the complexity of managing and securing multiple access points.</li><li><strong><em>Monitoring and Auditing:</em></strong> An EC2 bastion host provides a centralized location to collect logs, monitor user activity, and track SSH sessions. This facilitates better visibility into who is accessing your resources and enables auditing capabilities to meet compliance requirements.</li></ol><h3>Provisioning EC2 Bastion Host using Terraform</h3><p>To continue our journey towards building a secure and well-managed AWS infrastructure using Terraform, we will now focus on provisioning an EC2 bastion host. As a prerequisite, we will be utilizing the VPC infrastructure that we have set up in our previous article, <a href="https://towardsaws.com/building-a-secure-aws-vpc-with-terraform-subnets-inetnet-gateways-and-more-part-ii-14369aac292">Building a Secure AWS VPC with Terraform: Subnets, Internet Gateways, and More — Part II</a>. In case you missed it, we highly recommend giving it a read as it lays the groundwork for the bastion host setup.</p><p>All of the code for this article can be found on my <a href="https://github.com/irdanish11">GitHub</a> <a href="https://github.com/irdanish11/terraform-iac/tree/main/aws/infrastructure/03-ec2-bastion-host">Terraform Repository</a></p><h4>Variables for EC2 Bastion Host</h4><ol><li>variables.tf file.</li></ol><pre># EC2 Bastion Host variables<br>variable &quot;ec2-bastion-public-key-path&quot; {<br>  type = string<br>}<br><br>variable &quot;ec2-bastion-private-key-path&quot; {<br>  type = string<br>}<br><br>variable &quot;ec2-bastion-ingress-ip-1&quot; {<br>  type = string<br>}</pre><p>2. terraform.tfvars file.</p><pre>## EC2 Bastion Host Variables<br>ec2-bastion-public-key-path = &quot;../secrets/ec2-bastion-key-pair.pub&quot;<br>ec2-bastion-private-key-path = &quot;../secrets/ec2-bastion-key-pair.pem&quot;<br>ec2-bastion-ingress-ip-1 = &quot;0.0.0.0/0&quot;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/236/1*5QH8Jybe6b1C7NDdVzK5ag.png" /><figcaption>Photo by <a href="https://unsplash.com/@markusspiske?utm_source=medium&amp;utm_medium=referral">Markus Spiske</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p><strong>Caution! </strong>In the terraform.tfvars file we have temporarily setup the value of ec2-bastion-ingress-ip-1 (incoming traffic range) to 0.0.0.0/0 for demonstration purposes. This configuration allows unrestricted access to the EC2 bastion host from the public internet using your SSH key. However, it’s important to note that this approach is highly insecure. In a real-world scenario, it is strongly recommended to restrict incoming traffic only to trusted sources, such as your organization’s VPN or the CIDR range of your organization’s network. By implementing these restrictions, you ensure a much higher level of security for your EC2 bastion host and protect it from unauthorized access.</p><h4>Generating SSH Key Pair:</h4><p>Our next step is to generate SSH Key pair for our EC2 Bastion host and create AWS SSH Key Pair resource as well. There are two ways you can create and manage SSH Key Pair for EC2 Bastion Host, we demonstrate both:</p><ul><li>Using Terraform tls_private_key resource:</li></ul><pre>## Generate PEM (and OpenSSH) formatted private key.<br>resource &quot;tls_private_key&quot; &quot;ec2-bastion-host-key-pair&quot; {<br>  algorithm = &quot;RSA&quot;<br>  rsa_bits = 4096<br>}<br><br>## Create the file for Public Key<br>resource &quot;local_file&quot; &quot;ec2-bastion-host-public-key&quot; {<br>  depends_on = [ tls_private_key.ec2-bastion-host-key-pair ]<br>  content = tls_private_key.ec2-bastion-host-key-pair.public_key_openssh<br>  filename = var.ec2-bastion-public-key-path<br>}<br><br>## Create the sensitive file for Private Key<br>resource &quot;local_sensitive_file&quot; &quot;ec2-bastion-host-private-key&quot; {<br>  depends_on = [ tls_private_key.ec2-bastion-host-key-pair ]<br>  content = tls_private_key.ec2-bastion-host-key-pair.private_key_pem<br>  filename = var.ec2-bastion-private-key-path<br>  file_permission = &quot;0600&quot;<br>}<br><br>## AWS SSH Key Pair<br>resource &quot;aws_key_pair&quot; &quot;ec2-bastion-host-key-pair&quot; {<br>  depends_on = [ local_file.ec2-bastion-host-public-key ]<br>  key_name = &quot;${var.project}-ec2-bastion-host-key-pair-${var.environment}&quot;<br>  public_key = tls_private_key.ec2-bastion-host-key-pair.public_key_openssh<br>}</pre><p>Terraform provides the tls_private_key resource, which allows you to generate an SSH key pair within your Terraform configuration. This option is convenient if you prefer managing the key generation process directly in your Terraform code. You can define the key pair parameters, such as the algorithm, length, and format, within the resource block.</p><p>You can see from above code snippet we are using tls_private_key resource to generate the SSH private and public keys, and then we use local_file and local_sensitive_file resources to store and manage the generated private key securely. The local_file resource allows you to save the private key to a local file on your machine, which can be useful for accessing it outside of Terraform. However, since the private key contains sensitive information, such as authentication credentials, it&#39;s crucial to protect it. Here, the local_sensitive_file resource comes into play by encrypting the private key and storing it securely.</p><p>By using both local_file and local_sensitive_file resources, you can ensure that the private key is safely stored and easily accessible when needed, striking a balance between convenience and security in managing your SSH key pair.</p><ul><li>Using ssh-keygen:</li></ul><p>First we need to create the SSH Key Pair using following command:</p><pre>ssh-keygen -t rsa -C &quot;you.email@example.com&quot; -b 4096</pre><p>It will prompt you to enter the complete path to the file in which to save the key enter the path e.g path-to-repo/terraform-iac/aws/infrastructure/secrets/ec2-bastion-key-pair. Then it will ask for password enter a secure password for key. Then change the permissions of the SSH private key so that only your user can access the key by running the following command:</p><pre>chmod 600 path-to-repo/terraform-iac/aws/infrastructure/secrets/ec2-bastion-key-pair</pre><p>Once the SSH Key Pair is generated we will write our AWS Key Pair resource as follows:</p><pre>## AWS SSH Key Pair<br>resource &quot;aws_key_pair&quot; &quot;ec2-bastion-host-key-pair&quot; {<br>  key_name = &quot;${var.project}-ec2-bastion-host-key-pair-${var.environment}&quot;<br>  public_key = file(var.ec2-bastion-public-key-path)<br>}</pre><h4>Networking (Security Group &amp; Elastic IP):</h4><p>Now we will create a Security Group to control the inbound and outbound traffic for EC2 bastion host instance:</p><pre>resource &quot;aws_security_group&quot; &quot;ec2-bastion-sg&quot; {<br>  description = &quot;EC2 Bastion Host Security Group&quot;<br>  name = &quot;${var.project}-ec2-bastion-sg-${var.environment}&quot;<br>  vpc_id = aws_vpc.kodetronix-vpc.id<br>  ingress {<br>    from_port = 22<br>    to_port = 22<br>    protocol = &quot;tcp&quot;<br>    cidr_blocks = [var.ec2-bastion-ingress-ip-1]<br>    description = &quot;Open to Public Internet&quot;<br>  }<br>  egress {<br>    from_port = 0<br>    to_port = 0<br>    protocol = &quot;-1&quot;<br>    ipv6_cidr_blocks = [&quot;::/0&quot;]<br>    description = &quot;IPv6 route Open to Public Internet&quot;<br>  }<br>  egress {<br>    from_port = 0<br>    to_port = 0<br>    protocol = &quot;-1&quot;<br>    cidr_blocks = [&quot;0.0.0.0/0&quot;]<br>    description = &quot;IPv4 route Open to Public Internet&quot;<br>  }<br>}</pre><p>In the above terraform resource block we create a security group in our VPC (that we have created in our previous article). For ingress traffic, the security group allows TCP traffic on port 22 (SSH) from a specific IP address defined by the variable var.ec2-bastion-ingress-ip-1, you can also add multiple IP addresses as well. This configuration allows SSH access to the EC2 bastion host from the specified IP address, typically used for secure remote administration.</p><p>Regarding egress traffic, the security group allows all traffic (protocol “-1”) to be sent out to the public internet. This includes both IPv4 (cidr_blocks = [&quot;0.0.0.0/0&quot;]) and IPv6 (ipv6_cidr_blocks = [&quot;::/0&quot;]) traffic. These rules ensure that the EC2 bastion host can communicate with external resources if necessary.</p><p>Next we will create Elastic IP for our EC2 bastion host. Using an Elastic IP (EIP) for a bastion host offers several benefits and is considered a best practice in secure infrastructure design. By associating an EIP with the bastion host, you ensure that its public IP address remains consistent even if the instance is stopped, restarted or replaced. This provides stability for accessing the bastion host, as you can rely on a fixed IP address instead of having to constantly update your access rules or DNS records.</p><pre>## EC2 Bastion Host Elastic IP<br>resource &quot;aws_eip&quot; &quot;ec2-bastion-host-eip&quot; {<br>    vpc = true<br>    tags = {<br>    Name = &quot;${var.project}-ec2-bastion-host-eip-${var.environment}&quot;<br>    }<br>}</pre><h4>EC2 Instance &amp; Elastic IP Association:</h4><p>Finally now we will create EC2 instance that we will be using as bastion host.</p><pre>resource &quot;aws_instance&quot; &quot;ec2-bastion-host&quot; {<br>    ami = &quot;ami-0d76271a8a1525c1a&quot;<br>    instance_type = &quot;t2.micro&quot;<br>    key_name = aws_key_pair.ec2-bastion-host-key-pair.key_name<br>    vpc_security_group_ids = [ aws_security_group.ec2-bastion-sg.id ]<br>    subnet_id = aws_subnet.vpc-public-subnet-2.id<br>    associate_public_ip_address = false<br>    user_data                   = file(var.bastion-bootstrap-script-path)<br>    root_block_device {<br>      volume_size = 8<br>      delete_on_termination = true<br>      volume_type = &quot;gp2&quot;<br>      encrypted = true<br>      tags = {<br>        Name = &quot;${var.project}-ec2-bastion-host-root-volume-${var.environment}&quot;<br>      }<br>    }<br>    credit_specification {<br>      cpu_credits = &quot;standard&quot;<br>    }<br>    tags = {<br>        Name = &quot;${var.project}-ec2-bastion-host-${var.environment}&quot;<br>    }<br>    lifecycle {<br>      ignore_changes = [ <br>        associate_public_ip_address,<br>       ]<br>    }<br>}</pre><p>The provided Terraform code snippet represents the definition of an AWS EC2 instance resource named “ec2-bastion-host” in the Terraform configuration. This resource is responsible for provisioning and managing an EC2 bastion host within the specified AWS environment.</p><p>The EC2 instance is created with the Amazon Linux Amazon Machine Image (AMI) using the ami attribute and has an instance type of t2.micro. The SSH key pair for authentication is specified with key_name using the aws_key_pair.ec2-bastion-host-key-pair.key_name reference that we have created earlier. The EC2 instance is launched in the specified public subnet 2, to ensure it resides in the desired network segment. The option associate_public_ip_address is set to false, indicating that the EC2 instance won&#39;t have a public IP address associated with it.</p><p>The configuration also defines the root block device attributes, including the volume size, deletion on termination, volume type, encryption, and associated tags. The credit_specification section sets the CPU credits to &quot;standard&quot; for the instance. Lastly, the lifecycle block is included to ignore changes in the associate_public_ip_address attribute during updates to prevent unnecessary modifications.</p><p>To access our EC2 bastion host now we are going to associate Elastic IP that we have created earlier with our EC2 instance using following terraform resource block:</p><pre>## EC2 Bastion Host Elastic IP Association<br>resource &quot;aws_eip_association&quot; &quot;ec2-bastion-host-eip-association&quot; {<br>    instance_id = aws_instance.ec2-bastion-host.id<br>    allocation_id = aws_eip.ec2-bastion-host-eip.id<br>}</pre><p>Now we are ready to apply the infrastructure, once the infrastructure is provisioned you can use you private key and the public IP of your EIP to access the EC2 bastion host as follows:</p><pre>ssh -i ec2-bastion-key-pair.pem ec2-user@&lt;public-ipv4-of-eip&gt;</pre><h3>Conclusion:</h3><p>In conclusion, an EC2 Bastion Host is an important component of a secure AWS architecture as it provides a secure and controlled way to access instances within a private subnet. Provisioning the EC2 Bastion Host using Terraform allows for easy management of infrastructure as code. In this article, we went through the process of provisioning an EC2 Bastion Host using Terraform and also highlighted the importance of using best practices, such as limiting incoming traffic and using an Elastic IP address. By following these best practices, we can ensure a secure and manageable infrastructure that is easy to maintain and scale as required. We hope this article has provided valuable insights into the EC2 Bastion Host and how to provision it using Terraform. With EC2 Bastion Host setup in our next article <a href="https://medium.com/towards-aws/rds-deploying-scalable-and-resilient-relational-databases-using-terraform-iac-5fbf5e4d717b">RDS: Deploying Scalable and Resilient Relational Databases Using Terraform IaC</a> you can check how to deploy a relational database using AWS RDS.</p><h4>Recommended Readings:</h4><ul><li><a href="https://towardsaws.com/devops-1-setting-up-a-terraform-development-environment-for-aws-5134274dde5a">DevOps: 1 — Setting up a Terraform Development Environment for AWS</a></li><li><a href="https://medium.com/geekculture/devops-2-aws-vpc-virtual-private-cloud-part-i-412ad4ea0cc9">Building a Secure AWS VPC with Terraform: Subnets, Internet Gateways, and More — Part-I</a></li><li><a href="https://towardsaws.com/building-a-secure-aws-vpc-with-terraform-subnets-inetnet-gateways-and-more-part-ii-14369aac292">Building a Secure AWS VPC with Terraform: Subnets, Internet Gateways, and More — Part-II</a></li><li><a href="https://medium.com/towards-aws/rds-deploying-scalable-and-resilient-relational-databases-using-terraform-iac-5fbf5e4d717b">RDS: Deploying Scalable and Resilient Relational Databases Using Terraform IaC</a></li></ul><h3>Final Note</h3><figure><a href="https://bmc.link/irfandanish"><img alt="" src="https://cdn-images-1.medium.com/max/170/1*WfbwTpxi0c0k2Vwzp3s5yg.png" /></a></figure><p>If you enjoyed this article and found it useful, be sure to follow me on <a href="https://irdanish.medium.com/">Medium</a> and <a href="https://github.com/irdanish11">GitHub</a> for more content like this. On Medium, you can find more articles on Cloud Computing, DevOps, Machine Learning and other related topics. On GitHub, you can find my open-source projects and code samples. By following me on these platforms, you can stay up-to-date with my latest work and learn more about best practices for managing infrastructure with Terraform and other cloud tools. Thanks for reading!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=17ee287bb3d" width="1" height="1" alt=""><hr><p><a href="https://towardsaws.com/creating-a-bastion-host-for-secure-access-to-your-aws-infrastructure-with-terraform-17ee287bb3d">Creating a Bastion Host for Secure Access to Your AWS Infrastructure with Terraform</a> was originally published in <a href="https://towardsaws.com/">Towards AWS</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a Secure AWS VPC with Terraform: Subnets, Inetnet Gateways, and More— Part-II]]></title>
            <link>https://towardsaws.com/building-a-secure-aws-vpc-with-terraform-subnets-inetnet-gateways-and-more-part-ii-14369aac292?source=rss-4f1b8073abf9------2</link>
            <guid isPermaLink="false">https://medium.com/p/14369aac292</guid>
            <category><![CDATA[aws-vpc]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[vpc]]></category>
            <category><![CDATA[infrastructure-as-code]]></category>
            <dc:creator><![CDATA[Irfan Danish]]></dc:creator>
            <pubDate>Tue, 02 May 2023 00:54:35 GMT</pubDate>
            <atom:updated>2023-05-10T09:01:47.449Z</atom:updated>
            <content:encoded><![CDATA[<h3>Building a Secure AWS VPC with Terraform: Subnets, Internet Gateways, and More — Part-II</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xc4FjXDa0jv81p3ewfwDoA.png" /><figcaption>Image From Author</figcaption></figure><figure><a href="https://bmc.link/irfandanish"><img alt="" src="https://cdn-images-1.medium.com/max/170/1*WfbwTpxi0c0k2Vwzp3s5yg.png" /></a></figure><p>In our previous article <a href="https://medium.com/@irdanish/devops-2-aws-vpc-virtual-private-cloud-part-i-412ad4ea0cc9">AWS VPC (Virtual Private Cloud) — Part-I</a>, we delved into the basics of AWS VPC (Virtual Private Cloud), exploring concepts like subnets, route tables, internet and NAT gateways, and how they come together to form a VPC. In this article, we’ll dive into provisioning a secure and fully functional VPC, complete with all the necessary components. So, let’s jump right in!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/478/1*CI4OJCZm9jWC8q74I6gKEg.gif" /><figcaption>Source: giphy.com</figcaption></figure><p>All of the code for this article can be found on my <a href="https://github.com/irdanish11">GitHub</a> <a href="https://github.com/irdanish11/terraform-iac/tree/main/aws/infrastructure/02-vpc">Terraform Repository</a></p><h3>Prerequisites:</h3><p>To start working with Terraform and AWS, we need to set up the provider and backend configuration. In our previous article, <a href="https://medium.com/geekculture/devops-1-setting-up-a-terraform-development-environment-for-aws-5134274dde5a">Setting up a Terraform Development Environment for AWS</a>, we created an S3 Bucket and a DynamoDB table to manage our Terraform state. If you haven’t read that article yet, we recommend checking it out. Once we have the backend configured, we can write our provider block in the providers.tf file.</p><pre>terraform {<br>  required_providers {<br>    aws = {<br>      source = &quot;hashicorp/aws&quot;<br>      version = &quot;4.62.0&quot;<br>    }<br>  }<br>}<br><br>provider &quot;aws&quot; {<br>  # Configuration options<br>  region = &quot;eu-west-2&quot;<br>  profile = &quot;aws-access-key-profile&quot;<br>}<br><br># Configuring S3 and DynamoDB backend for Terraform state<br># terraform init -reconfigure<br>terraform {<br>  backend &quot;s3&quot; {<br>    bucket = &quot;infrafy-backend-tfstate-bucket-dev&quot;<br>    key    = &quot;terraform.tfstate&quot;<br>    region = &quot;eu-west-2&quot;<br>    profile = &quot;aws-access-key-profile&quot;<br>    dynamodb_table = &quot;infrafy-tfstate-lock-dev&quot;<br>  }    <br>}</pre><p>In the Terraform manifest above, we begin by setting our cloud provider to <strong><em>AWS</em></strong> and specifying its version. Next, we configure the default region that will be used for infrastructure provisioning, as well as the AWS Access Keys profile. Finally, we configure our <strong><em>S3</em></strong> and <strong><em>DynamoDB</em></strong> as the backend to store and manage the Terraform state.</p><h3>Infrastructure for VPC:</h3><p>Now we can summarize the infrastructure that we are going to create.</p><ul><li>VPC</li><li>Intenet Gateway</li><li>NAT Gateway</li><li>3 Public Subnets in three different Availability Zones</li><li>3 Private Subnets in three different Availability Zones</li><li>Separate Route Tables for each of the 3 public and private subnets (Total 6 Route Tables).</li></ul><p>By creating separate route tables for each of our subnets, we gain more control and flexibility in managing traffic for each subnet individually. Although we could create two route tables and associate them with all three public and private subnets respectively, having a dedicated route table for each subnet allows for more fine-grained management of network traffic.</p><h4>Variables:</h4><p>Here we will create variables that are required for the provisioning of the VPC infrastructure.</p><ul><li>variables.tf</li></ul><pre>## VPC variables<br>variable &quot;vpc-cidr-block&quot; {<br>  type = string<br>}<br><br>variable &quot;public-subnets-cidr-blocks&quot; {<br>    type = list(string)<br>}<br><br>variable &quot;private-subnets-cidr-blocks&quot; {<br>    type = list(string)<br>}<br><br>variable &quot;public-subnet-1-cidr-block&quot; {<br>  type = string<br>}<br><br>variable &quot;public-subnet-2-cidr-block&quot; {<br>  type = string<br>}<br><br>variable &quot;public-subnet-3-cidr-block&quot; {<br>  type = string<br>}<br><br>variable &quot;private-subnet-1-cidr-block&quot; {<br>  type = string<br>}<br><br>variable &quot;private-subnet-2-cidr-block&quot; {<br>  type = string<br>}<br><br>variable &quot;private-subnet-3-cidr-block&quot; {<br>  type = string<br>}</pre><ul><li>terraform.tfvars</li></ul><pre>## VPC variables<br>vpc-cidr-block = &quot;10.20.0.0/16&quot;<br>public-subnets-cidr-blocks = [&quot;10.20.0.0/24&quot;, &quot;10.20.1.0/24&quot;, &quot;10.20.2.0/24&quot;]<br>private-subnets-cidr-blocks = [&quot;10.20.3.0/24&quot;, &quot;10.20.4.0/24&quot;, &quot;10.20.5.0/24&quot;]<br><br>public-subnet-1-cidr-block = &quot;10.20.0.0/24&quot;<br>public-subnet-2-cidr-block = &quot;10.20.1.0/24&quot;<br>public-subnet-3-cidr-block = &quot;10.20.2.0/24&quot; <br><br>private-subnet-1-cidr-block = &quot;10.20.3.0/24&quot;<br>private-subnet-2-cidr-block = &quot;10.20.4.0/24&quot;<br>private-subnet-3-cidr-block = &quot;10.20.5.0/24&quot;</pre><p>First we have CIDR Block for VPC which is 10.20.0.0/16 , that means that we can have 65,531 IP address in our VPC.</p><blockquote>exponent = 32–16 = 16</blockquote><blockquote>Number of IPs In VPC = 2¹⁶ — 5 = 65,531 IP Addresses</blockquote><p>Each subnet is having a CIDR Block with the range of 8 bits (32–24=8), meaning each subnet can have total 251 IP addresses (5 IP addresses are reserved in each CIDR Block). For more information please visit <a href="https://docs.aws.amazon.com/vpc/latest/userguide/subnet-sizing.html">AWS Documentation</a>.</p><h4>VPC:</h4><p>In the vpc.tf file, we can find all the necessary resources for creating a VPC. Let&#39;s begin by writing the null label and VPC resource block, we will set theinstance_tenancy to default in VPC resource block. This attribute determines whether EC2 instances launched in the VPC use the tenancy attribute specified at instance launch or use a dedicated tenancy, which incurs an additional per-region fee of $2 per hour, plus an hourly per-instance usage fee. While using dedicated instance_tenancy offers more security and may be necessary for compliance, default instance_tenancy is typically sufficient for most use cases.</p><pre>## Null label for VPC <br>module &quot;label-kodetronix-vpc&quot; {<br>  source = &quot;../../modules/terraform-null-label&quot;<br><br>  namespace   = var.project<br>  environment = var.environment<br>  name        = &quot;vpc&quot;<br><br>  label_order = [&quot;namespace&quot;, &quot;name&quot;, &quot;environment&quot;]<br><br>  additional_tag_map = {<br>    ManagedBy = &quot;Terraform&quot;<br>  }<br>}<br><br>## VPC<br>resource &quot;aws_vpc&quot; &quot;kodetronix-vpc&quot; {<br>  cidr_block           = var.vpc-cidr-block<br>  instance_tenancy = &quot;default&quot;<br>  tags = {<br>    Name = &quot;${module.label-kodetronix-vpc.id}&quot;<br>  }<br>}</pre><h4>Internet Gateway:</h4><p>Now we will write resource blocks for Internet Gateway:</p><pre>## VPC Internet Gateway<br>resource &quot;aws_internet_gateway&quot; &quot;vpc-igw&quot; {<br>  vpc_id = aws_vpc.kodetronix-vpc.id<br>  tags = {<br>    Name = &quot;${var.project}-IGW&quot;<br>  }<br>}</pre><p>Here we have created a resource block for AWS Internet Gateway and provided the VPC ID with which IGW will be associated and tags for the name of the IGW resource.</p><h4>Public Subnets:</h4><p>In the next step of our VPC provisioning using Terraform, we will define resource blocks for our public subnets in three availability zones: a, b and c. We will also create three route tables for these subnets and add a route to the Internet Gateway, which will allow us to make the subnets public. This will ensure that our resources in these subnets can communicate with the internet, enabling us to host public-facing services.</p><pre>############# Public Subnet 1 #############<br>## Public Subnet 1<br>resource &quot;aws_subnet&quot; &quot;vpc-public-subnet-1&quot; {<br>  vpc_id = aws_vpc.kodetronix-vpc.id<br>  cidr_block = var.public-subnet-1-cidr-block<br>  availability_zone = &quot;${var.region}a&quot;<br>  tags = {<br>    Name = &quot;${var.project}-public-subnet-01&quot;<br>  }<br>}<br><br>## Route Table for Public Subnet 1<br>resource &quot;aws_route_table&quot; &quot;public-subnet-1-rtb&quot; {<br>  vpc_id = aws_vpc.kodetronix-vpc.id<br>    route {<br>        cidr_block = &quot;0.0.0.0/0&quot;<br>        gateway_id = aws_internet_gateway.vpc-igw.id<br>    }<br>  tags = {<br>    Name = &quot;${var.project}-rtb-public-subnet-01&quot;<br>  }<br>}<br><br>## Route Table Association for Public Subnet 1<br>resource &quot;aws_route_table_association&quot; &quot;public-subnet-1-rtb-association&quot; {<br>  subnet_id = aws_subnet.vpc-public-subnet-1.id<br>  route_table_id = aws_route_table.public-subnet-1-rtb.id<br>}</pre><p>In the above code snippet, we begin by creating a resource block for the subnet, where we set the VPC ID and CIDR block variable for the subnet and designate it to availability zone a. Following this, we create a resource block for the route table and add a route to the public internet 0.0.0.0/0 along with the Internet Gateway ID. This ensures that all traffic meant for the public internet is routed through the Internet Gateway. Lastly, we create a resource block for the route table association to attach the previously created route table to the subnet.</p><p>In order to keep this article concise, the code for the remaining two public subnets has not been provided. However, you can access the complete code for this article by visiting my <a href="https://github.com/irdanish11/terraform-iac/tree/main/aws/infrastructure/02-vpc">Terraform GitHub Repository</a>.</p><h4><strong>NAT Gateway:</strong></h4><pre>## Elastic IP for NAT Gateway<br>resource &quot;aws_eip&quot; &quot;nat-gateway-eip&quot; {<br>  vpc = true<br>  tags = {<br>    Name = &quot;${var.project}-nat-gateway-eip&quot;<br>  }<br>}<br>  <br>## VPC NAT(Network Address Translation) Gateway<br>resource &quot;aws_nat_gateway&quot; &quot;vpc-nat-gateway&quot; {<br>  allocation_id = aws_eip.nat-gateway-eip.id<br>  subnet_id = aws_subnet.vpc-public-subnet-1.id<br>  tags = {<br>    Name = &quot;${var.project}-nat-gateway&quot;<br>  }<br>}</pre><p>The first resource block is for AWS Elastic IP, Elastic IP addresses are required for NAT Gateways in AWS as they provide a fixed public IP address that can be associated with a NAT Gateway even after it’s replaced. NAT Gateways provide internet access to resources within private subnets of a VPC, and they need to have a fixed public IP address that remains unchanged even after the gateway is replaced, as any change in the public IP address would disrupt the connections of resources using the NAT Gateway. Elastic IP addresses allow for this fixed public IP address assignment, making NAT Gateways more reliable and easier to manage.</p><p>After creating Elastic IP for NAT Gateway we write NAT Gateway Resource Block, as it required the NAT Gateway has to be created in one of the public subnets, so we provide the subnet ID of public subnet 1 and also provide the Elastic IP ID. While we’re only creating one NAT Gateway in this example, for high availability, it’s recommended to create three separate NAT Gateways, one in each of the public subnets. This ensures that even if one availability zone goes down, traffic from the private subnets can still reach the internet.</p><h4>Private Subnets:</h4><p>In the final step of our VPC provisioning, we will create private subnets to isolate resources from the public internet. Creating private subnets involves creating the same three resource blocks as we did for public subnets: subnet, route table, and route table association. However, the main difference is that private subnets do not have a route to the public internet. Instead, they have a route to the NAT Gateway. In the route table for private subnets, we add a route that routes all traffic outside of the VPC range to the NAT Gateway. This ensures that private resources can access the internet through the NAT Gateway, but the internet cannot directly access the private resources.</p><pre>############# Private Subnet 1 #############<br>## Private Subnet 1<br>resource &quot;aws_subnet&quot; &quot;vpc-private-subnet-1&quot; {<br>  vpc_id = aws_vpc.kodetronix-vpc.id<br>  cidr_block = var.private-subnet-1-cidr-block<br>  availability_zone = &quot;${var.region}a&quot;<br>  tags = {<br>    Name = &quot;${var.project}-private-subnet-01&quot;<br>  }<br>}<br><br>## Route Table for Private Subnet 1<br>resource &quot;aws_route_table&quot; &quot;private-subnet-1-rtb&quot; {<br>  vpc_id = aws_vpc.kodetronix-vpc.id<br>  route {<br>    cidr_block = &quot;0.0.0.0/0&quot;<br>    nat_gateway_id = aws_nat_gateway.vpc-nat-gateway.id<br>  }<br>  tags = {<br>    Name = &quot;${var.project}-rtb-private-subnet-01&quot;<br>  }<br>}<br><br>## Route Table Association for Private Subnet 1<br>resource &quot;aws_route_table_association&quot; &quot;private-subnet-1-rtb-association&quot; {<br>  subnet_id = aws_subnet.vpc-private-subnet-1.id<br>  route_table_id = aws_route_table.private-subnet-1-rtb.id<br>}</pre><p>We have not included the creation of Security Groups and NACLs in this article as we plan to cover them in our upcoming articles where we will also discuss the different solution architectures and there will be an appropriate need for Security Groups and NACLs in those solutions.</p><p>In our upcoming article titled <a href="https://medium.com/towards-aws/creating-a-bastion-host-for-secure-access-to-your-aws-infrastructure-with-terraform-17ee287bb3d"><em>Creating a Bastion Host for Secure Access to Your AWS Infrastructure with Terraform</em></a>, we will demonstrate how to provision an EC2 instance as a Bastion Host. This Bastion Host will act as a secure and controlled entry point for users to access resources in the private network.</p><h3>Final Note</h3><figure><a href="https://bmc.link/irfandanish"><img alt="" src="https://cdn-images-1.medium.com/max/170/1*WfbwTpxi0c0k2Vwzp3s5yg.png" /></a></figure><p>If you enjoyed this article and found it useful, be sure to follow me on <a href="https://irdanish.medium.com/">Medium</a> and <a href="https://github.com/irdanish11">GitHub</a> for more content like this. On Medium, you can find more articles on Cloud Computing, DevOps, Machine Learning and other related topics. On GitHub, you can find my open-source projects and code samples. By following me on these platforms, you can stay up-to-date with my latest work and learn more about best practices for managing infrastructure with Terraform and other cloud tools. Thanks for reading!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=14369aac292" width="1" height="1" alt=""><hr><p><a href="https://towardsaws.com/building-a-secure-aws-vpc-with-terraform-subnets-inetnet-gateways-and-more-part-ii-14369aac292">Building a Secure AWS VPC with Terraform: Subnets, Inetnet Gateways, and More— Part-II</a> was originally published in <a href="https://towardsaws.com/">Towards AWS</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[DevOps: 2 AWS VPC (Virtual Private Cloud) — Part-I]]></title>
            <link>https://medium.com/geekculture/devops-2-aws-vpc-virtual-private-cloud-part-i-412ad4ea0cc9?source=rss-4f1b8073abf9------2</link>
            <guid isPermaLink="false">https://medium.com/p/412ad4ea0cc9</guid>
            <category><![CDATA[subnet]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[vpc]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[virtual-private-cloud]]></category>
            <dc:creator><![CDATA[Irfan Danish]]></dc:creator>
            <pubDate>Fri, 28 Apr 2023 19:01:46 GMT</pubDate>
            <atom:updated>2023-08-28T11:38:35.541Z</atom:updated>
            <content:encoded><![CDATA[<h3>Building a Secure AWS VPC with Terraform: Subnets, Internet Gateways, and More— Part-I</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xc4FjXDa0jv81p3ewfwDoA.png" /></figure><figure><a href="https://bmc.link/irfandanish"><img alt="" src="https://cdn-images-1.medium.com/max/170/1*WfbwTpxi0c0k2Vwzp3s5yg.png" /></a></figure><p>Building a secure and robust infrastructure in the cloud is one of the most critical tasks for any organization. AWS offers various services to help you achieve this goal, and the most fundamental building block of AWS networking is the Virtual Private Cloud (VPC). In this article we will cover VPC and some of its key components. In our last article we have discussed how to <a href="https://medium.com/@irdanish/devops-1-setting-up-a-terraform-development-environment-for-aws-5134274dde5a">Setup Terraform Developement Environment for AWS</a> and in this two part article first we learn some concepts about VPC and in the second part we will use Terraform to provision our own VPC with public and private subnets as shown in the above diagram.</p><h3><strong>What is AWS VPC?</strong></h3><p>Amazon Virtual Private Cloud (VPC) is a fundamental building block of the AWS cloud infrastructure, which allows you to provision a logically isolated section of the Amazon Web Services (AWS) Cloud where you can launch Amazon Elastic Compute Cloud (EC2) instances, Amazon Relational Database Service (RDS) instances, and other AWS resources in a virtual network that you define.</p><p>AWS VPC is required to provide a secure and isolated environment for your applications to run on the cloud. It allows you to create a private network in the cloud where you can launch resources such as EC2 instances, RDS databases, and Elastic Load Balancers, etc. VPC enables you to define and control the network topology, the IP address range, and the network gateways, making it easy to configure and manage the network infrastructure. With VPC, you can create multiple subnets, each with its own security group, which provides a layer of security to your resources. You can also use Network Access Control Lists (NACLs) to add an additional layer of security to your network.</p><p>VPC provides the ability to create a private network in the cloud, which provides a more secure and isolated environment for your applications. VPC also enables you to connect your on-premises network to the cloud using VPN or Direct Connect. With VPC, you can also create subnets, which provide a way to divide your network into smaller, more manageable parts.</p><p>VPC consists of a lot of different components and concepts, below we will briefly take a look at some of them.</p><h3>1. CIDR:</h3><p>CIDR stands for Classless Inter-Domain Routing (CIDR) is a way of specifying IP addresses and their associated routing prefix. CIDR notation allows us to represent an IP address range using a single IP address and a slash followed by the number of bits in the network prefix. For example, 10.0.0.0/16 represents the entire range of IP addresses from 10.0.0.0 to 10.0.255.255 and the CIDR block 10.0.1.0/24 represents the range from 10.0.1.0 to 10.0.0.255. The number after slash tells you how many IP address you will get for that CIDR block. To calculate the number of addresses in a CIDR block we have to subtract the number after the slash from 32 and then raise it to the power of 2 which gives the total number of IP addresses in a CIDR Block. An example calculation is shown below for 10.0.0.0/24:</p><blockquote>n = 32–24 = 8</blockquote><blockquote>Total IP addresses = 2⁸ — 5 = 251</blockquote><p>Similarly 10.0.0.0/16 will have (2¹⁶ — 5 ) means 65,531 addresses and so on. In each CIDR block first 4 IP addresses and last IP address cannot be used and are reserved.</p><h3>2. Subnet:</h3><p>A subnet is a range of IP addresses in your VPC that you can allocate to your resources. Subnets are created within a VPC and are associated with a particular Availability Zone (AZ). By default, each subnet can have up to 256 IP addresses. Subnets are used to segregate resources in a VPC and apply network security policies.</p><h3>3. Route Table:</h3><p>A route table contains a set of rules, called routes, that are used to determine where network traffic is directed. Each subnet in a VPC must be associated with a route table, and the rules in the route table are used to determine how traffic is routed within the VPC. The default route table in a VPC allows all resources in the VPC to communicate with each other. When we create a VPC a default route table is associate to the VPC, but we can also create route table for each of our subnet.</p><h3>4. Internet Gateway:</h3><p>An Internet Gateway (IGW) is a horizontally scaled, redundant, and highly available VPC component that allows communication between instances in your VPC and the internet. An IGW enables your instances to communicate with the internet, and vice versa. The IGW is attached to your VPC, and it provides a target for all internet-bound traffic in the VPC.</p><h3>5. Public vs Private Subnet:</h3><p>In a VPC, you can create subnets that are either public or private. Public subnets are associated with a route table that has a route to the internet via an IGW. Instances in a public subnet can have a public IP address and can be accessed from the internet. Private subnets, on the other hand, are associated with a route table that does not have a route to the internet. Instances in a private subnet can access the internet through a NAT Gateway.</p><h3>6. NAT Gateway:</h3><p>A Network Address Translation (NAT) Gateway is a highly available AWS-managed service that allows instances in a private subnet to access the internet. NAT Gateway enables instances in a private subnet to connect to the internet without exposing their private IP addresses to the internet. The NAT Gateway maps the private IP addresses of instances to public IP addresses. The NAT Gateways are usually deployed in one of the public subnets so that they can directly access the internet.</p><h3>7. NACL:</h3><p>A Network Access Control List (NACL) is a stateless firewall that controls traffic to and from subnets in a VPC. NACLs are associated with a VPC and are used to filter traffic based on rules that you define. You can use NACLs to allow or deny traffic based on protocol, port, and source/destination IP addresses.</p><h3>8. Security Groups:</h3><p>A Security Group is a virtual firewall that controls inbound and outbound traffic for instances in a VPC. Security Groups are associated with a VPC and are used to allow or deny traffic based on rules that you define. You can use Security Groups to specify which ports are open for incoming traffic, which IPs are allowed to access your instances, and more.</p><p>Above we have briefly described what is a VPC and its key componnets. In conclusion, AWS VPC is a critical component of AWS cloud infrastructure that provides a secure and isolated environment for your applications to run on the cloud. It gives you complete control over your network infrastructure, including network topology, IP address ranges, and network gateways. With VPC, you can create a private network in the cloud that is highly secure and flexible, enabling you to launch resources like EC2 instances, RDS databases, and ELBs, etc.</p><p>In our next article <a href="https://towardsaws.com/building-a-secure-aws-vpc-with-terraform-subnets-inetnet-gateways-and-more-part-ii-14369aac292"><em>Building a Secure AWS VPC with Terraform: Subnets, Internet Gateways, and More — Part-II</em></a>, we will provision our own VPC with Public and Private Subnets and create route tables for each of the subnet and also we will provision a NAT Gateway for our private subnets.</p><h3>Final Note</h3><figure><a href="https://bmc.link/irfandanish"><img alt="" src="https://cdn-images-1.medium.com/max/170/1*WfbwTpxi0c0k2Vwzp3s5yg.png" /></a></figure><p>If you enjoyed this article and found it useful, be sure to follow me on <a href="https://irdanish.medium.com/">Medium</a> and <a href="https://github.com/irdanish11">GitHub</a> for more content like this. On Medium, you can find more articles on Cloud Computing, DevOps, Machine Learning and other related topics. On GitHub, you can find my open-source projects and code samples. By following me on these platforms, you can stay up-to-date with my latest work and learn more about best practices for managing infrastructure with Terraform and other cloud tools. Thanks for reading!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=412ad4ea0cc9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/geekculture/devops-2-aws-vpc-virtual-private-cloud-part-i-412ad4ea0cc9">DevOps: 2 AWS VPC (Virtual Private Cloud) — Part-I</a> was originally published in <a href="https://medium.com/geekculture">Geek Culture</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[DevOps: 1 - Setting up a Terraform Development Environment for AWS]]></title>
            <link>https://towardsaws.com/devops-1-setting-up-a-terraform-development-environment-for-aws-5134274dde5a?source=rss-4f1b8073abf9------2</link>
            <guid isPermaLink="false">https://medium.com/p/5134274dde5a</guid>
            <category><![CDATA[terraform-cloud]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[terraform-state]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[terraform-setup]]></category>
            <dc:creator><![CDATA[Irfan Danish]]></dc:creator>
            <pubDate>Fri, 07 Apr 2023 21:00:22 GMT</pubDate>
            <atom:updated>2023-05-10T09:05:31.164Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qHBbT0XzTctJY21lVZ_t5w.png" /><figcaption>Terraform IaC on AWS</figcaption></figure><figure><a href="https://bmc.link/irfandanish"><img alt="" src="https://cdn-images-1.medium.com/max/170/1*WfbwTpxi0c0k2Vwzp3s5yg.png" /></a></figure><p>Terraform is a popular open-source tool for provisioning and managing cloud infrastructure. If you’re new to Terraform, the first step is to set up a development environment. In this tutorial, we’ll walk through the process of creating the infrastructure required to manage the Terraform state on AWS backend and not only that we will also create some other prerequisite infrastructure that would help us in the future when we are provisioning new infrastructure.</p><p>The other pieces of infrastructure that we will create include a <em>Hello World </em>ECR repository that we can use as a first image whenever we are provisioning an Image Lambda Function and the second piece is a Route53 Hosted Zone which we will use as a default DNS zone whenever we provision a new Route53 record.</p><p>All of the code for this article can be found on my <a href="https://github.com/irdanish11">GitHub</a> <a href="https://github.com/irdanish11/terraform-iac/tree/main/aws/infrastructure/01-terraform-prerequisite-configurations">Terraform Repository</a></p><h3>Prerequisites:</h3><p>Before getting started, you’ll need to have the following:</p><ul><li>An AWS account</li><li>AWS CLI installed and AWS Access Key Configured</li><li>Terraform installed</li></ul><h3>Infrastructure for Terraform Backend State Management</h3><p>Terraform state is the representation of the resources created by Terraform, including their properties, dependencies, and other metadata. This state is used by Terraform to plan and apply changes to the infrastructure. To manage Terraform state for AWS, we need to consider several factors. In this article, we’ll explore the best practices for managing Terraform state in AWS, including creating an S3 bucket and DynamoDB table, configuring the backend, and managing state changes.</p><p>While each can be used independently, using both services together provides additional benefits, such as enhanced security and data consistency. By encrypting the S3 bucket and enabling versioning, you can ensure the security of the Terraform state information, while also being able to track and manage changes over time. By using DynamoDB to store metadata, you can ensure that Terraform always has access to the latest state information, even if a specific version of the S3 object is not available.</p><p>In this tutorial we will be using following <a href="https://registry.terraform.io/namespaces/terraform-aws-modules">Terraform AWS modules</a> from <a href="https://github.com/antonbabenko">Anton Babenko</a> to create our required infrastructure.</p><ul><li><a href="https://registry.terraform.io/modules/terraform-aws-modules/s3-bucket/aws/latest">S3 Bucket Module</a></li><li><a href="https://registry.terraform.io/modules/terraform-aws-modules/dynamodb-table/aws/latest">DynamoDB Module</a></li><li><a href="https://registry.terraform.io/modules/terraform-aws-modules/ecr/aws/latest">ECR Module</a></li><li><a href="https://registry.terraform.io/modules/terraform-aws-modules/route53/aws/latest">Route53 Module</a></li><li><a href="https://registry.terraform.io/modules/cloudposse/label/null/latest">Null Label Module</a></li></ul><h4>1. Setting up Terraform variables for required infrastructure</h4><p>First we will setup the variables.tf file which will contain all the variable definitions.</p><pre>variable &quot;project&quot; {<br>  type = string<br>}<br><br>variable &quot;environment&quot; {<br>  type = string<br>}<br><br>variable &quot;region&quot; {<br>  type = string<br>}<br><br>variable &quot;profile&quot; {<br>  type = string<br>}<br><br>variable &quot;domain_name&quot; {<br>  type = string<br>}</pre><p>Then we will create terraform.tfvars file that contains the values for the variables used in the Terraform configuration.</p><pre>##Variables<br>region      = &quot;us-east-1&quot;<br>project     = &quot;infrafy&quot;<br>environment = &quot;dev&quot; <br>domain_name = &quot;infrafy.codetronix.xyz&quot; #Format =&gt; &lt;project&gt;.&lt;company-name&gt;.&lt;domain-extensions&gt;<br>profile     = &quot;aws-access-key-profile-name&quot; # </pre><p>Here the variable project can be considered as a product of a your company codetronix . Change the above values accordingly for your use case.</p><p>Then setup the provider in main.tf file:</p><pre>provider &quot;aws&quot; {<br>  region = &quot;us-east-1&quot;<br>  profile = &quot;aws-access-key-profile-name&quot;<br>}</pre><h4>2. Create an S3 Bucket</h4><p>First, we’ll create an S3 bucket to store the Terraform state file. In the following code block first we are creating a null label for our S3 bucket (i.e. name for our S3 bucket) and then we use terraform-aws-s3 module to create the S3 bucket, in the we will block all public access and deny all the access to the bucket without HTTPS and also enable the versioning for the bucket.</p><pre>## Label for S3 bucket for backend<br>module &quot;s3-label&quot; {<br>  source = &quot;../../modules/terraform-null-label&quot;<br><br>  namespace   = var.project<br>  environment = var.environment<br>  name        = &quot;backend-tfstate&quot;<br><br>  label_order = [&quot;namespace&quot;, &quot;name&quot;, &quot;environment&quot;]<br><br>  additional_tag_map = {<br>    ManagedBy = &quot;Terraform&quot;<br>  }<br>}<br><br>## S3 bucket for backend<br>module &quot;s3_bucket&quot; {<br>  source = &quot;../../modules/terraform-aws-s3&quot;<br><br>  bucket                                = module.s3-label.id<br>  acl                                   = &quot;private&quot;<br>  attach_deny_insecure_transport_policy = true<br>  block_public_acls                     = true<br>  block_public_policy                   = true<br>  ignore_public_acls                    = true<br>  restrict_public_buckets               = true<br><br>  versioning = {<br>    enabled = true<br>  }<br><br>}</pre><h4>3. Create a DynamoDB Table</h4><p>Next, we’ll create a DynamoDB table to store the locking information. The DynamoDB table used for Terraform state is designed to store the state information in a structured format that can be easily queried and updated. Each row in the table represents a resource managed by Terraform, and the columns store information about the resource, including its configuration and dependencies.</p><p>One of the benefits of using DynamoDB as a backend is that it provides built-in support for high availability and durability. DynamoDB automatically replicates data across multiple availability zones, ensuring that the state information is always available even in the event of a regional outage.</p><pre>## Label for dynamodb table for backend<br>module &quot;dynamodb-label&quot; {<br>  source = &quot;../../modules/terraform-null-label&quot;<br><br>  namespace   = var.project<br>  environment = var.environment<br>  name        = &quot;tfstate-lock&quot;<br><br>  label_order = [&quot;namespace&quot;, &quot;name&quot;, &quot;environment&quot;]<br><br>  additional_tag_map = {<br>    ManagedBy = &quot;Terraform&quot;<br>  }<br>}<br><br>## dynamodb table for backend<br>module &quot;dynamodb_table&quot; {<br>  source = &quot;../../modules/terraform-aws-dynamodb-table&quot;<br><br>  name     = module.dynamodb-label.id<br>  hash_key = &quot;LockID&quot;<br><br>  attributes = [<br>    {<br>      name = &quot;LockID&quot;<br>      type = &quot;S&quot;<br>    }<br>  ]<br>}</pre><p>In the above code block first we have created the null label for DynamoDB table and then we have created the DynamoDB table.</p><p>With that it completes the infrastructure required to store the Terraform state in the backend.</p><h3>Other pieces of Infrastructure (optional):</h3><p>The other two pieces that we are going to work on are totally optional.</p><h4>1. Setting Hello World ECR Repository:</h4><p>Now we are going to create an ECR repository that will serve us as a go to image for our Image Lambda Functions. In order to do that make sure you have Docker installed on your machine.</p><p>First we will create the ECR Repository;</p><pre>## Sample hello world ECR repository<br>module &quot;ecr&quot; {<br>  source = &quot;../../modules/terraform-aws-ecr&quot;<br><br>  name                    = &quot;sample-hello-world&quot;<br>  scan_images_on_push     = false<br>  enable_lifecycle_policy = true<br>  max_image_count         = &quot;1&quot;<br>  image_tag_mutability    = &quot;MUTABLE&quot;<br><br>  tags = {<br>    Project     = var.project<br>    Environment = var.environment<br>  }<br>}</pre><p>Then we will create a Dockerfile in the same directory as these terraform manifests with the following contents:</p><pre>FROM hello-world<br>CMD [&quot;echo&quot;, &quot;hello world&quot;]</pre><p>Finally we will call the terraform module that will build and upload the Docker Image to our ECR Repository:</p><pre>## Data source to get current account id<br>data &quot;aws_caller_identity&quot; &quot;current&quot; {}<br><br>## Sample hello world image to ECR<br>module &quot;ecr_image&quot; {<br>  source             = &quot;../../modules/terraform-aws-ecr-image&quot;<br>  dockerfile_dir     = &quot;./&quot;<br>  ecr_repository_url = module.ecr.repository_url<br>  region             = var.region<br>  profile            = var.profile<br>  account_id         = data.aws_caller_identity.current.account_id<br>}</pre><h4>2. Route53 Zone</h4><p>The final piece of infrastruture that we are going to create is a hosted Route53 zone to use it as a default DNS zone whenever we provision a new Route53 record. Hosted zone is a container for DNS records for a specific domain. When you create a hosted zone in Route 53, you are creating a container that holds information about the domain name and its associated DNS records.</p><pre>module &quot;zones&quot; {<br>  source = &quot;../../modules/terraform-aws-route53/zones&quot;<br><br>  zones = {<br>    &quot;main-domain&quot; = {<br>      domain_name = &quot;${var.domain_name}&quot;<br>    }<br>  }<br>}</pre><h4>3. Outputs File</h4><p>In the final step we will create the outputs.tf file for the infrastructure that we have created. In Terraform, the outputs.tffile is a file that defines the output values that are generated by your infrastructure. These output values can be used by other parts of your infrastructure or by external systems.</p><pre> output &quot;s3_bucket_id&quot; {<br>  value = module.s3_bucket.s3_bucket_id<br>}<br><br>output &quot;dynamodb_table_id&quot; {<br>  value = module.dynamodb_table.dynamodb_table_id<br>}<br><br>output &quot;repository_url&quot; {<br>  value = module.ecr.repository_url<br>}<br><br>output &quot;repository_arn&quot; {<br>  value = module.ecr.repository_arn<br>}<br><br>output &quot;ecr_image_url&quot; {<br>  value = module.ecr_image.ecr_image_url<br>}<br><br>output &quot;route53_zone_name_servers&quot; {<br>  value = module.zones.route53_zone_name_servers<br>}</pre><p>With that we complete our infrastructure and now its time to apply these terraform manifests.</p><h3>Apply the Terraform manifests:</h3><p>Now we will provision the infrastructure that using the terraform manifests file that we have created.</p><p>First we initialize the terraform manifests directory which will download the necessary provider plugins and sets up the backend configuration for storing the state.</p><pre>terraform init</pre><p>Then we will validate the syntax and format of the Terraform configuration files. It checks for any errors or warnings in your code.</p><pre>terraform validate</pre><p>In the third step we will do the plan which will create an execution plan that shows what Terraform will do when you apply your configuration. It shows the resources that will be created, modified or destroyed.</p><pre>terraform plan -var-file=./terraform.tfvars</pre><p>Then finally after checking and verifying the execution plan we will apply the changes defined.</p><pre>terraform apply -var-file=./terraform.tfvars</pre><p>With this final step we have completed this tutorial and now we are ready to use our newly created infrastructure as our terrform state backend. In the next article we will look how we can use this terraform state backend to create the and store the our infrastructure state.</p><h3>Conclusion:</h3><p>In conclusion, setting up a development environment for Terraform on AWS requires careful consideration of the infrastructure required to manage the Terraform state, including an S3 bucket and DynamoDB table. Additionally, creating a Route53 hosted zone and an ECR repository can further enhance your infrastructure provisioning workflow. By using the best practices outlined in this article, you can improve your efficiency and maintainability when working with Terraform on AWS.</p><p>As always, it’s important to regularly review and update your infrastructure code to ensure it remains in sync with your evolving infrastructure requirements. With the powerful features of Terraform and AWS, you can create a flexible and scalable infrastructure that can be easily managed and updated over time.</p><h3>Final Note</h3><figure><a href="https://bmc.link/irfandanish"><img alt="" src="https://cdn-images-1.medium.com/max/170/1*WfbwTpxi0c0k2Vwzp3s5yg.png" /></a></figure><p>If you enjoyed this article and found it useful, be sure to follow me on <a href="https://irdanish.medium.com/">Medium</a> and <a href="https://github.com/irdanish11">GitHub</a> for more content like this. On Medium, you can find more articles on Cloud Computing, DevOps, Machine Learning and other related topics. On GitHub, you can find my open-source projects and code samples. By following me on these platforms, you can stay up-to-date with my latest work and learn more about best practices for managing infrastructure with Terraform and other cloud tools. Thanks for reading!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5134274dde5a" width="1" height="1" alt=""><hr><p><a href="https://towardsaws.com/devops-1-setting-up-a-terraform-development-environment-for-aws-5134274dde5a">DevOps: 1 - Setting up a Terraform Development Environment for AWS</a> was originally published in <a href="https://towardsaws.com/">Towards AWS</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Object Detection YOLOv4: How to train and test YOLOv4 on Ubuntu Linux 20.04]]></title>
            <link>https://medium.com/geekculture/object-detection-yolov4-how-to-train-and-test-yolov4-on-ubuntu-linux-20-04-50f1d3b4e4f1?source=rss-4f1b8073abf9------2</link>
            <guid isPermaLink="false">https://medium.com/p/50f1d3b4e4f1</guid>
            <category><![CDATA[darknet]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[yolo]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[object-detection]]></category>
            <dc:creator><![CDATA[Irfan Danish]]></dc:creator>
            <pubDate>Tue, 19 Jan 2021 15:42:38 GMT</pubDate>
            <atom:updated>2021-01-19T15:42:38.915Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/768/0*ERSKYbfWgS0HLaE1" /><figcaption>YOLOv4 — Darknet</figcaption></figure><p>Hello folks, in today&#39;s story we will learn about how to run <a href="https://github.com/AlexeyAB/darknet">Darknet</a> aka YOLOv4 repository on your Linux machine.</p><p>The first step in building/installing YOLO v4 on Ubuntu is installing its dependencies. Here I am assuming that you have a freshly installed Ubuntu 20.04 installation which is only having default installed packages.</p><h3>Installing YOLO Prerequisites</h3><p>Here are the important prerequisites of YOLO.</p><p>We can run YOLO either in <strong>CPU </strong>or run with <strong>GPU </strong>acceleration. In order to do GPU acceleration, you may need a good NVIDIA based graphics card with <a href="https://www.lifewire.com/what-is-nvidia-cuda-834095">CUDA cores</a>.</p><p>Use the following command to install the prerequisites, you can find the explanation of each one after that.</p><ul><li><strong>CMake &gt;= 3.8</strong></li><li><strong>CUDA 10.1 (For GPU)</strong></li><li><strong>OpenCV &gt;= 2.4 (For CPU and GPU)</strong></li><li><strong>cuDNN &gt;= 7.x for CUDA 10.1 (for GPU)</strong></li><li><strong>OpenMP (for CPU)</strong></li><li><strong>Other Dependencies: make, git, g++</strong></li></ul><h4>CMake &gt;= 3.8 (for modern CUDA support)(CPU and GPU)</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*fiZNvzHe2QNTdyFc" /><figcaption>CMake: <a href="https://blog.kitware.com/kitware-increases-android-support-in-cmake-3-7/">blog.kitware.com</a></figcaption></figure><p>The Ubuntu 20.04 is having CMake version 3.16, we can install this using apt package manager.</p><p>Installing CMake in Ubuntu 20.04</p><pre>sudo apt install cmake</pre><p>For checking your CMake version, you can use the following command</p><pre>cmake --version</pre><p>It will give you following output.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/942/1*aKSJZ_9OEQP-XqOxRLXXyQ.png" /><figcaption>cmake version</figcaption></figure><h4>CUDA 10.1 (For GPU)</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*gnF2rRKFUx6YnMnf" /><figcaption>nvidia cuda toolkit</figcaption></figure><p>You can install CUDA if you have a GPU from NVIDIA, adding GPU for YOLO will speed up the object detection process.</p><p>CUDA is a parallel computing platform and application programming interface model created by Nvidia. It allows software developers and software engineers to use a CUDA-enabled graphics processing unit for general purpose processing.</p><p>Here are the tutorials to install CUDA 10 on Ubuntu.</p><p><a href="https://developer.nvidia.com/cuda-toolkit-archive"><strong>Download and install CUDA 10 Toolkit</strong></a></p><p><a href="https://linuxconfig.org/how-to-install-cuda-on-ubuntu-20-04-focal-fossa-linux">How to install CUDA on Ubuntu 20.04 Focal Fossa Linux</a></p><p>For checking your Cuda toolkit version, you can use the following command</p><pre>nvcc --version</pre><p>It will give you following output.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/962/1*znkIeSLQzZhPaLE0LXYtaQ.png" /><figcaption>cuda toolkit version</figcaption></figure><h4><a href="https://opencv.org/">OpenCV </a>&gt;= 2.4 (For CPU and GPU)</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/474/0*OvEWkvG4S2a9VX_p" /><figcaption>opencv</figcaption></figure><p><a href="https://opencv.org/">OpenCV </a>is a library of programming functions mainly aimed at real-time computer vision. Originally developed by Intel, it was later supported by Willow Garage then Itseez. The library is cross-platform and free for use under the open-source BSD license.</p><blockquote><strong>Note</strong>: <em>The OpenCV is an optional install YOLO, but if you install it, you will get a window system to display the output image from YOLO detection. Otherwise, the output image from YOLO will be saved as an image file. I have enabled OpenCV for this tutorial so that you can see the output of YOLO in a window.</em></blockquote><p>You can install OpenCV in Ubuntu using the <strong>apt </strong>package manager or using compiling the source code.</p><p><strong>Installing OpenCV using package manager</strong></p><p>Here is the command to quickly install OpenCV and its Python extension using the apt package manager.</p><pre>sudo apt install libopencv-dev python3-opencv</pre><p><strong>Installing OpenCV using source compilation</strong></p><p>If you want to customize your OpenCV installation, you can try the source compilation.</p><p><a href="http://www.codebind.com/linux-tutorials/install-opencv-ubuntu-18-04-lts-c-cpp-linux/">Install OpenCV in Ubuntu</a></p><p>After installing OpenCV, you can find the existing version of OpenCV using the following command</p><pre>opencv_version</pre><h4>cuDNN &gt;= 7.x for CUDA 10.1 (for GPU)</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*9FR9wEkg1xs6U6m-" /><figcaption>cuDNN</figcaption></figure><p>The NVIDIA CUDA® Deep Neural Network library (cuDNN) is a GPU-accelerated library of primitives for deep neural networks. cuDNN provides highly tuned implementations for standard routines such as forward and backward convolution, pooling, normalization, and activation layers.</p><p>Here is the installation procedure for cuDNN.</p><p><a href="https://docs.nvidia.com/deeplearning/sdk/cudnn-archived/cudnn_741/cudnn-install/index.html">Installing cuDNN On Linux</a></p><h4>OpenMP (for CPU)</h4><p><a href="https://www.openmp.org/"><strong>OpenMP </strong></a><strong>uses</strong> a portable, scalable model that gives programmers a simple and flexible interface for developing parallel applications for platforms ranging from the standard desktop computer to the supercomputer.</p><p>Here is the command to install OpenMP in Ubuntu 20.04</p><pre>sudo apt install libomp-dev</pre><h4>Other Dependencies (For CPU and GPU)</h4><p>Here are other dependencies used for CPU and GPU configuration.</p><pre>sudo apt install make git g++</pre><h3>Downloading YOLO v4</h3><p>You can download the YOLO v4 using following command</p><pre>git clone <a href="https://github.com/AlexeyAB/darknet">https://github.com/AlexeyAB/darknet</a></pre><p><a href="https://github.com/AlexeyAB/darknet">Yolo v4 Github</a></p><h3>Building YOLO v4</h3><p>You can build the YOLO v4 using Make and CMake command.</p><p><strong>Building YOLO v4 using Make</strong></p><p>Switch to the <strong>darknet </strong>folder after download. Open the <em>Makefile </em>in the <em>darknet </em>folder. You can see some of the variables at the beginning of the <em>Makefile</em>. If you want to compile darknet for CPU, you can use the following flags.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/889/1*vkGcT4zcCVvoSHcRqJ6yCA.png" /><figcaption>Makefile</figcaption></figure><p><strong>For CPU build</strong></p><p>Set <strong>AVX=1</strong> and <strong>OPENMP=1</strong> to speedup on CPU (if an error occurs then set AVX=0), <br>Set <strong>LIBSO=1 will create the shared library of the </strong>darknet, ‘<em>libdarknet.so</em>‘, which is used to interface darknet and <strong>Python</strong>.</p><pre>GPU=0<br>CUDNN=0<br>CUDNN_HALF=0<br>OPENCV=1<br>AVX=1<br>OPENMP=1<br>LIBSO=1  </pre><p><strong>For GPU build</strong></p><p>set <strong>GPU=1</strong> and <strong>CUDNN=1</strong> to speedup on GPU</p><p>set <strong>CUDNN_HALF=1</strong> to further speedup 3 x times (Mixed-precision on Tensor Cores) GPU: Volta, Xavier, Turing and higher</p><pre>GPU=1<br>CUDNN=1<br>CUDNN_HALF=1<br>OPENCV=1<br>AVX=0<br>OPENMP=0<br>LIBSO=1  </pre><p>After doing these changes, just execute the following command from the <strong>darknet</strong> folder.</p><pre>make</pre><p>If everything works fine, you may get an output as shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/892/1*ot7mp0iKnxglBzdfH4g3aw.png" /><figcaption>darknet build</figcaption></figure><p>After build, you can able find <em>darknet</em> and <em>libdarknet.so</em> in the build path.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/894/1*bFtKNK3lWLsMq3CBM9zUaA.png" /><figcaption>build files</figcaption></figure><p><strong>Building YOLO v4 using CMake</strong></p><p>You can build darknet using CMake build. Just follow the commands below in order to build from CMake.</p><p><strong>Note</strong>: The commands should be executed from inside the <strong>darknet </strong>folder.</p><pre>mkdir build_release<br>cd build_release<br>cmake ..<br>make</pre><p>If the build is successful, it will create the following files inside the <strong>build_release </strong>folder.</p><ol><li>darknet</li><li>libdark.so</li></ol><p>After building those files, copy the <em>darknet</em> and <em>libdark.so</em> from the <strong>build_release folder</strong> to the <strong>darknet </strong>folder. You also have to rename</p><p><em>libdark.so</em> -&gt; <em>libdarknet.so</em></p><p>to test the darknet Python scripts.</p><h3>Testing YOLO v4</h3><p>After building YOLO, let’s test the working of YOLO v4. To test the darknet, first, we have to download a pre-trained model. The following model is trained for the <a href="http://cocodataset.org/#home">MS COCO dataset</a>.</p><p><a href="https://drive.google.com/file/d/1cewMfusmPjYWbrnuJRuKhPMwRe_b9PaT/view">Download YOLO v4 Model</a></p><p>After downloading the <em>yolov4.weights</em>, copy to the <strong>darknet </strong>folder. Now make sure that<strong> you have the </strong>following files in the <strong>darknet </strong>folder.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/892/1*VW30aUCBoftm4b4ucDrr4Q.png" /><figcaption>4 weight file for MS CoCo Dataset</figcaption></figure><p>Now open a terminal from the darknet folder by right-clicking on the folder and execute the following commands.</p><h3>Testing YOLO v4 in a single image</h3><p>The command below is for running YOLO in a single image. Both of the commands mentioned below do the same functions. The first one is for detection from one image, the second one is for multiple use cases, for eg. detection from video and webcam.</p><pre>Version #1</pre><pre>./darknet detect cfg/yolov4.cfg yolov4.weights data/person.jpg</pre><pre>Version #2<br> <br>./darknet detector test cfg/coco.data cfg/yolov4.cfg yolov4.weights data/person.jpg -i 0 -thresh 0.25</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/897/1*xTMNL3ZrJdj3rwMs8djQ7A.png" /><figcaption>darknet in action</figcaption></figure><p>The <em>darknet</em> is the executable that we are getting when we build the darknet source code. Using this executable we can directly perform object detection in an image, video, camera, and network video stream.</p><p>Here <em>yolov4.weights</em> is the pre-trained model, <em>cfg/yolov4.cfg</em> is the configuration file of the model. The <em>data/person.jpg</em> is the input image of the model.</p><p>The 2nd command is providing the configuration file of COCO dataset <strong><em>cfg/coco.d</em>ata</strong>, the ‘<strong><em>i=0</em></strong>‘ mentioning the GPU number, and ‘<strong><em>thresh</em></strong>‘ is the threshold of detection. The accuracy of the detection will vary if you vary this value. By default, YOLO only displays objects detected with a confidence of .25 or higher.</p><p>You may get following output if everything is successful</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/915/1*kioiiJp8RBUaaamJXSh3Kw.png" /><figcaption>successfull yolov4 test</figcaption></figure><p>You can able to see the confidence of detection from the terminal.</p><p>Instead<strong> of <em>data/person.jpg</em>, </strong>you can try with <br><strong><em>data/eagle.jpg</em> </strong><br><strong><em>data/dog.jpg</em></strong><br><strong><em>data/person.jpg</em></strong><br><strong><em>data/horses.jpg</em></strong></p><h3>Testing YOLO v4 in a Video</h3><p>Download a test video from the following link</p><p><a href="https://drive.google.com/file/d/1w4emvTuxVhDWUGsQ6uwWPXhjL74sxOIJ/view">Download Video Sample</a></p><p>Copy the test video <em>test50.mp4</em> into the darknet folder, and test YOLO in a video using the following command.</p><pre>./darknet detector demo cfg/coco.data cfg/yolov4.cfg yolov4.weights test50.mp4 -i 0 -thresh 0.25</pre><p>You may get an output as shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/771/0*2eu2yidgBa_CJwdE" /><figcaption>sample</figcaption></figure><p>So that was it for this story, in the next story we will learn how to train custom YOLOv4 object detector. See you in the next story.</p><p>To see my work you can follow me on G<a href="http://github.com/irdanish11">ithub</a> &amp; <a href="https://irdanish.medium.com/">Medium</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=50f1d3b4e4f1" width="1" height="1" alt=""><hr><p><a href="https://medium.com/geekculture/object-detection-yolov4-how-to-train-and-test-yolov4-on-ubuntu-linux-20-04-50f1d3b4e4f1">Object Detection YOLOv4: How to train and test YOLOv4 on Ubuntu Linux 20.04</a> was originally published in <a href="https://medium.com/geekculture">Geek Culture</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Learning TensorFlow 2: Use tf.function and Forget About tf.Session]]></title>
            <link>https://medium.com/geekculture/learning-tensorflow-2-use-tf-function-and-forget-about-tf-session-a8117158edd9?source=rss-4f1b8073abf9------2</link>
            <guid isPermaLink="false">https://medium.com/p/a8117158edd9</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[keras]]></category>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[neural-networks]]></category>
            <dc:creator><![CDATA[Irfan Danish]]></dc:creator>
            <pubDate>Sun, 17 Jan 2021 17:45:11 GMT</pubDate>
            <atom:updated>2021-01-19T15:37:10.201Z</atom:updated>
            <content:encoded><![CDATA[<h4><a href="https://towardsai.net/p/category/machine-learning">Machine Learning</a></h4><h3>Learning TensorFlow 2: Use tf.function and Forget About tf.Session</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fMK7Y6XKoeHlJ5o9wGrMVQ.png" /><figcaption>TF 2</figcaption></figure><p>In these series, we will be analyzing strength and subtleties of TensorFlow 2. You will be learning about the new features and methods that will make your problems easier to solve. But if you are still afraid of TensorFlow and confused about the frameworks such as TensorFlow, PyTorch and Keras you should definitely check out this story, <a href="https://medium.com/towards-artificial-intelligence/tensorflow-2-for-all-the-giant-of-deep-learning-80427cceff05"><strong>TensorFlow 2 for All: The Giant of Deep Learning</strong></a>, it will definitely help you out.</p><p>So let’s start today’s story. As TensorFlow 2 came out we said goodbye to a lot of things such as Placeholders, Sessions, and all the creepy stuff that most of the people didn’t like at all because that was not pythonic at all. But TF 2, addressed this problem TF2 embraced the python completely, and along with we’ve got a lot of new features.</p><p>And AutoGraph is one of the most exciting new features of Tensorflow 2: it allows transforming a subset of Python syntax into its portable, high-performance, and language agnostic graph representation bridging the gap between Tensorflow 1.x and the 2.0 release based on eager execution.</p><p>As often happens all that glitters is not gold: although powerful, AutoGraph hides some subtlety that is worth knowing; this article will guide you through them using an error-driven approach.</p><h3>Session execution</h3><p>The reader familiar with Tensorflow 1.x already knows that the standard workflow to get a callable graph (or better, define a graph with nodes that can be executed within a tf.Session) is:</p><ol><li>Create the tf.Graph object and set it as the default graph for the current scope.</li><li>Describe the computation using the Tensorflow API (e.g. y = tf.matmul(a,x) + b).</li><li>Think in advance about variable sharing and define the variables’ scope accordingly.</li><li>Create and configure the tf.Session.</li><li>Build the concrete graph and load it into the tf.Session.</li><li>Initialize all the variables.</li><li>Use the tf.Session.run method to start the computation. The node execution will trigger a backtracking procedure from the chosen nodes (.run input parameters) to their inputs, in order to resolve the dependencies and compute the result.</li></ol><p>All these points can be translated in code with this minimal example:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/647cf4a7830b9eab047ed2bf3fe3802d/href">https://medium.com/media/647cf4a7830b9eab047ed2bf3fe3802d/href</a></iframe><p>Tensorflow 2.0, defaulting on eager execution follows a completely different approach based on the direct execution of what the user wants.</p><ul><li>Remove the graph definition.</li><li>Remove the session execution.</li><li>Remove variables initialization.</li><li>Remove the variable sharing via scopes.</li><li>Remove the tf.control_dependencies to execute sequential operation not connected by a dependency relation.</li></ul><p>Just write the code and run it:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/485be64b9b8316b081d8a9356a700f76/href">https://medium.com/media/485be64b9b8316b081d8a9356a700f76/href</a></iframe><p>The eager counterpart of any Tensorflow 1.x source code is usually slower since it relies on the Python interpreter to run the computation and there are a lot of optimizations that are only possible on DataFlow graphs.</p><p>The bridge among the two versions that allow creating computational graphs even in Tensorflow 2.0 is tf.function.</p><h3>tf.function, not tf.Session</h3><p>One of the major changes in Tensorflow 2.0 is the removal of the tf.Session object (see <a href="https://github.com/tensorflow/community/blob/master/rfcs/20180918-functions-not-sessions-20.md"><strong>RFC: Functions, not Sessions</strong></a>). This change forces the user to organize the code in a better way: no more a tf.Session object to pass around, but just Python functions that can be accelerated with a simple decoration.</p><p>In order to define a graph in Tensorflow 2.0, we need to define a Python function and decorate it with @tf.function.</p><blockquote><strong><em>Note</em></strong><em>: the speed-up is not guaranteed. There are certain tasks in which is not worth converting the function to its graph representation, as is the case of this simple matrix multiplication we are performing here. However, for computationally intensive tasks like the optimization of a deep neural network the Graph conversion provides a huge performance boost.</em></blockquote><p>The automatic conversion from Python code to its graph representation is called AutoGraph.</p><p>In Tensorflow 2.0, AutoGraph is automatically applied to a function when it is decorated with <a href="https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/function"><strong>@tf.function</strong></a>; this decorator creates callable graphs from Python functions.</p><h3>tf.function: layman explanation</h3><p>On the first call of a tf.function decorated function:</p><ul><li>The function is executed and traced. Eager execution is disabled in this context, therefore every tf. method just define a tf.Operation node that produces a tf.Tensor output, Tensorflow 1.x like.</li><li>AutoGraph is used to detect Python constructs that can be converted to their graph equivalent (while → tf.while, for → tf.while, if → tf.cond, assert → tf.assert, …).</li><li>From the function trace + autograph, the graph representation is built. In order to preserve the execution order in the defined graph, tf.control_dependencies is automatically added after every statement, in order to condition the line <em>i</em>+1 on the execution of line <em>i.</em></li><li>The tf.Graph object has now been built.</li><li>Based on the function name and the input parameters a unique ID is created and associated with the graph. The graph is cached into a map: map[id] = graph.</li><li>Any function call will just re-use the defined graph if the key matches.</li></ul><p>The next sections will guide you through the required steps to migrate a 1.x snippet to its eager and graph-accelerated version.</p><h3>Conversion to eager execution</h3><p>To use tf.function the first thing to do is to refactor the old 1.x code, wrapping the code we want to execute into a session.</p><p>In general, where first there was a session execution, now there is Python function.</p><blockquote><strong>Note</strong>: this is a huge advantage since the software architecture it allows defining is cleaner, and easy to maintain and document.</blockquote><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/99d9551f75705f5674877557cab58a55/href">https://medium.com/media/99d9551f75705f5674877557cab58a55/href</a></iframe><p>What happens now? <strong>Nothing</strong>. TensorFlow 2 works in eager mode by default, this means that we just defined a standard Python function and if we evaluate it:</p><pre>print(f().numpy())</pre><p>We get the expected result:</p><pre>[[22. 22.]<br> [23. 13.]]</pre><h3>From eager to tf.function: the need to refactor</h3><p>Let’s just add the @tf.function decoration to the f function. For the sake of clarity (and to debug in the old-school print driven way) let’s add even a print and a tf.print statement inside the function body:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/65a5115e94ba67302a52bf1b8cf77058/href">https://medium.com/media/65a5115e94ba67302a52bf1b8cf77058/href</a></iframe><p>What happens now?</p><ol><li>The annotation @tf.function wrapped the f function in a tensorflow.python.eager.def_function.Function object. The Python function is assigned to the .python_function property of the object.</li><li>Until the object is called ( f() ): nothing happens.</li><li>When f() is called the process of graph building starts. At this stage, only the Python code is executed and the behavior of the function is traced, in order to collect the required data to build the graph. Thus the only output we get is:</li></ol><p>PRINT: Tensor(&quot;add:0&quot;, shape=(2, 2), dtype=float32)</p><p>The tf.print call is not evaluated as any other tf.* method, since Tensorflow already knows everything about those statements and it can use them as they are to build the graph.</p><p>4. <strong>Fail</strong> during the first and only invocation of the function, the following exception has been raised</p><pre>ValueError: tf.function-decorated function tried to create variables on non-first call.</pre><p>@tf.function failed to build the graph.</p><p>One may expect an execution flow like:</p><p><strong>First call</strong>:</p><pre>f()</pre><p><em>Graph definition and execution since this is the first time the function f is called.</em></p><p><strong>Any other call</strong>:</p><pre>f() #again</pre><p>Failure:</p><pre>ValueError: tf.function-decorated function tried to create variables on non-first call.</pre><p>This can happen because there is no guarantee about the number of times tf.function evaluates the Python function while converting it to Graph. Therefore the behavior described above is exactly what happens under the hood.</p><p>However, it still remains shady when this second function call is performed and why there is no second output from the print call (that it should be executed since is before the tf.Variable definition).</p><p>As it’s easy to understand, the exception is raised because the function contains a tf.Variable definition. In fact, a tf.Variable in eager mode is just a plain Python object, that gets destroyed as soon as it goes out of scope. While a tf.Variable object defines a persistent object if the function is decorated: in fact, the eager mode is disabled and the tf.Variable object defines a node in a persistent Graph (a Graph that exists even after the session execution).</p><p>Hence, the same function that in eager mode is perfectly valid (and in fact the same function without annotation works), when annotated with @tf.function stops working. Thus this is the <strong>first lesson</strong>:</p><blockquote>Converting a function that works in eager mode to its Graph representation requires to think about the Graph even though we are working in eager mode.</blockquote><p>So now, what we have to do in order to go on with the analysis of tf.function? There are 3 options:</p><ol><li>Declare f as a function that accepts an input parameter: the parameter can be a tf.Variable or any other input type.</li><li>Create a function that inherits the Python variable from the parent scope, and check in the function body if it has already been declared (if b != None).</li><li>Wrap everything inside a class. The __call__ method is the function we want to execute and the variable is declared as a private attribute (self._b). The same declaration check of point 2 has to be performed. In practice, this is the Object Oriented solution that is functionally equivalent to the one suggested in point 2.</li></ol><p>In order to understand if there are differences among these methods, all of them are going to be analyzed.</p><h3>Handling states breaking the function scope</h3><p>Points 2 and 3 described above have the same behavior, but the Object Oriented solution is way better from the software engineering point of view. Just compare these two implementations:</p><p><strong>The ugly solution with global variables</strong> (highly discouraged):</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1308665fc2417e8ba44ced378f009685/href">https://medium.com/media/1308665fc2417e8ba44ced378f009685/href</a></iframe><p><strong>Object Oriented solution</strong> (recommended):</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d8434564a8757da883dd930f5bca8e18/href">https://medium.com/media/d8434564a8757da883dd930f5bca8e18/href</a></iframe><p>The Object Oriented solution is superior: no global variables, the class F can always be instantiated and called without having to worry about the global b variable that every other function sees.</p><p>So far so good, we solved the problem of functions that create states by breaking the scope. In fact, once executed the previous script returns the same values of the eager execution.</p><p>From this the <strong>second lesson</strong>:</p><blockquote>When defining a function you want to accelerate converting it to its graph representation, you have to define its body thinking about the Graph is being built. There is no 1:1 match between eager execution and the graph built by <em>@tf.function</em>; thanks to AutoGraph there is no need to worry about the order of the operation execution, but special attention is required when definition function with objects that can create a state (<em>tf.Variable</em>).</blockquote><p>Another option to solve the problem (that we discussed in point 1) is to move the variable outside the function body.</p><h3>Handling states using input parameters</h3><p>We can refactor the f function making it accept b as an input parameter.</p><p>It should be pretty clear that tf.function do not allow to simply wrap a function that works in eager mode and accelerate it - it requires to think about how the conversion is performed, what happens when converting Python to graph operations, and take care of a lot of subtleties.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/368bbde9c9709f4f393f617d3fcc6c2e/href">https://medium.com/media/368bbde9c9709f4f393f617d3fcc6c2e/href</a></iframe><p>As in the previous section, the function produces the expected behavior. Moreover, being passed by reference the status (the variable value) can be updated from inside the graph accelerated function while being available from the outside. In fact, the following code produces 1,2,3, with the help of <a href="https://www.tensorflow.org/api_docs/python/tf/compat/v1/assign_add">assign_add</a>.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cf21fdf7ebb4f44c5161ec496c1bee0a/href">https://medium.com/media/cf21fdf7ebb4f44c5161ec496c1bee0a/href</a></iframe><h3>Conclusions</h3><p>This is the end of part 1. The article is divided into 2 parts because there are a lot of things to write about tf.function and its subtleties and a single article is going to be too long.</p><p>In part 1 we learned how to convert a 1.x code to its eager version, how to convert the eager version to its graph representation concluding with the problems to face when working with functions that create a state.</p><p>In the next part, we’ll study what happens when instead of a tf.Variable we pass a tf.Tensor or a Python value as input to a decorated function, together with the analysis of the tf.function behavior when the Python code is executed in the first function call: are we sure everything is going to be converted to the Graph representation we expect?</p><p>Stay tuned for part 2!</p><p>To see my work you can follow me on G<a href="http://github.com/irdanish11">ithub</a> &amp; <a href="https://irdanish.medium.com/">Medium</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a8117158edd9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/geekculture/learning-tensorflow-2-use-tf-function-and-forget-about-tf-session-a8117158edd9">Learning TensorFlow 2: Use tf.function and Forget About tf.Session</a> was originally published in <a href="https://medium.com/geekculture">Geek Culture</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>