<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ishan Sharma on Medium]]></title>
        <description><![CDATA[Stories by Ishan Sharma on Medium]]></description>
        <link>https://medium.com/@ishansharma_97521?source=rss-ef523eeeac44------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*SHr41pxFJQhm2EB7.</url>
            <title>Stories by Ishan Sharma on Medium</title>
            <link>https://medium.com/@ishansharma_97521?source=rss-ef523eeeac44------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 05:47:08 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ishansharma_97521/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Multi Hybrid Cloud infrastructure Creation]]></title>
            <link>https://medium.com/@ishansharma_97521/multi-hybrid-cloud-infrastructure-creation-41d8f3f73b66?source=rss-ef523eeeac44------2</link>
            <guid isPermaLink="false">https://medium.com/p/41d8f3f73b66</guid>
            <dc:creator><![CDATA[Ishan Sharma]]></dc:creator>
            <pubDate>Fri, 18 Sep 2020 14:45:55 GMT</pubDate>
            <atom:updated>2020-09-18T14:45:55.382Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Er1hU2c1B2ls3PGeAWWkQw.png" /></figure><p>Deploy the Wordpress application on Kubernetes and AWS using terraform including the following steps;</p><p>1. Write an Infrastructure as code using terraform, which automatically deploy the Wordpress application</p><p>2. On AWS, use RDS service for the relational database for Wordpress application.</p><p>3. Deploy the Wordpress as a container either on top of Minikube or EKS or Fargate service on AWS</p><p>4. The Wordpress application should be accessible from the public world if deployed on AWS or through workstation if deployed on Minikube.</p><p>Try each step first manually and write Terraform code for the same to have a proper understanding of workflow of task.</p><p>The code for the following infrastructure is given below:</p><pre>//Here, I used minikube, a single node k8s cluster.<br>provider “kubernetes” {<br> config_context_cluster = “minikube”<br>}//AWS provider credentials<br>provider &quot;aws&quot; {<br>  region     = &quot;ap-south-1&quot;<br>  profile    = &quot;arnav&quot;<br>}//Database instance<br>resource &quot;aws_db_instance&quot; &quot;mysqldb&quot; {<br>  allocated_storage    = 10<br>  engine               = &quot;mysql&quot;<br>  engine_version       = &quot;5.7&quot;<br>  instance_class       = &quot;db.t2.micro&quot;<br>  storage_type         = &quot;gp2&quot;<br>  name                 = &quot;mydb&quot;<br>  username             = &quot;aditi&quot;<br>  password             = &quot;longpassword&quot;<br>  port                 = &quot;3306&quot;<br>  publicly_accessible  = true<br>  skip_final_snapshot  = true<br>  parameter_group_name = &quot;default.mysql5.7&quot;tags = {<br>    Name = &quot;Wp-database&quot;<br>  }<br>}//Creating a deployment resource for our application<br>resource “kubernetes_deployment” “example” {<br> metadata {<br> name = “wordpressdeployment”<br> labels = {<br> test = “MyExampleApp”<br> }<br> }spec {<br> replicas = 3selector {<br> match_labels = {<br> test = “MyExampleApp”<br> }<br> }template {<br> metadata {<br> labels = {<br> test = “MyExampleApp”<br> }<br> }spec {<br> container {<br> image = “wordpress:4.8-apache”<br> name = “wordpresscontainer”}<br> }<br> }<br> }<br>}//creating a service, to expose our application to outside world<br>resource “kubernetes_service” “wp-expose” {<br> metadata {<br> name = “terraform-wp-service”<br> }<br> spec {<br> selector = {<br> test = “MyExampleApp”<br> }<br> //valid ports range from:30000-32767<br> port {<br> node_port = 32765<br> port = 80<br> target_port = 80<br> }type = “NodePort”<br> }<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ZlDe011B8OvGPkDM.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*GenntNyU1fXIdtaZ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*oufVu8XMO-XGsIzs.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*b1yR9YRq-_pojscP.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*e7Z8i0RYd1AH8f6B.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/634/0*ZmtxrUn1WN4XtRwN.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/634/0*T0W3xy57V5yyzOiN.png" /></figure><p>For destroying the complete infrastructure in simnple steps you just need to type single command and then the magic will happen</p><pre>terrafrom destroy</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*9ERNPRUMNvHpbY0y.png" /></figure><p>Following code portrays the whole framework:</p><pre>provider “aws” {<br> region = “ap-south-1”<br> profile = “arnav”<br>}//Database instance<br>resource “aws_db_instance” “mysqldb” {<br> allocated_storage = 10<br> engine = “mysql”<br> engine_version = “5.7”<br> instance_class = “db.t2.micro”<br> storage_type = “gp2”<br> name = “mydb”<br> username = “arnav”<br> password = “longpassword”<br> port = “3306”<br> publicly_accessible = true<br> skip_final_snapshot = true<br> parameter_group_name = “default.mysql5.7”<br>tags = {<br> Name = “Wp-database”<br> }<br>}provider &quot;google&quot; {<br> project = &quot;qwiklabs-gcp-02-4aba9d16c1f2&quot;<br> region = &quot;asia-southeast1&quot;<br> <br>}resource &quot;google_compute_network&quot; &quot;vpc_network&quot; {<br> name = &quot;wp-vpc&quot;<br> auto_create_subnetworks = false<br> routing_mode = &quot;REGIONAL&quot;<br>}resource &quot;google_compute_subnetwork&quot; &quot;subnet&quot; {<br> network = google_compute_network.vpc_network.id<br> name = &quot;wp-lab&quot;<br> ip_cidr_range = &quot;10.0.11.0/24&quot;<br> region = &quot;asia-southeast1&quot;<br> depends_on = [google_compute_network.vpc_network]<br>}resource &quot;google_compute_firewall&quot; &quot;firewall&quot; {<br> name = &quot;wp-firewall&quot;<br> network = google_compute_network.vpc_network.name<br> source_ranges = [ &quot;0.0.0.0/0&quot; ]<br> allow {<br> protocol = &quot;all&quot;<br> }<br> depends_on = [google_compute_subnetwork.subnet]<br>}resource &quot;google_container_cluster&quot; &quot;gce&quot; {<br> name = &quot;wp-cluster&quot;<br> location = &quot;asia-southeast1&quot;<br> remove_default_node_pool = true<br> initial_node_count = 1<br> network = google_compute_network.vpc_network.name<br> subnetwork = google_compute_subnetwork.subnet.name<br> depends_on = [google_compute_firewall.firewall]}resource &quot;google_container_node_pool&quot; &quot;node_pool&quot; {<br> location = &quot;asia-southeast1&quot;<br> name = &quot;wp-node&quot;<br> cluster = google_container_cluster.gce.name<br> node_count = 1<br>node_config {<br> machine_type = &quot;n1-standard-1&quot;<br> }<br> depends_on = [google_container_cluster.gce]<br>}resource &quot;null_resource&quot; &quot;one&quot; {<br> provisioner &quot;local-exec&quot; {<br> command =&quot;gcloud container clusters get-credentials wp-cluster - region=asia-southeast1&quot;<br> }<br> <br> depends_on=[google_container_node_pool.node_pool,]<br>}provider &quot;kubernetes&quot; {}<br>resource &quot;kubernetes_deployment&quot; &quot;example&quot; {<br> metadata {<br> name = &quot;wordpressdeployment&quot;<br> labels = {<br> test = &quot;MyExampleApp&quot;<br>    }<br>  }<br>spec {<br> replicas = 3<br>selector {<br> match_labels = {<br> test = &quot;MyExampleApp&quot;<br>   }<br>  }<br>template {<br> metadata {<br> labels = {<br> test = &quot;MyExampleApp&quot;<br> }<br> }<br>spec {<br> container {<br> image = &quot;wordpress:4.8-apache&quot;<br> name = &quot;wordpresscontainer&quot;<br>}<br> }<br> }<br> }<br> depends_on=[null_resource.one]<br>}resource &quot;kubernetes_service&quot; &quot;wp-expose&quot; {<br> metadata {<br> name = &quot;terraform-wp-service&quot;<br> }<br> spec {<br> selector = {<br> test = &quot;MyExampleApp&quot;<br> }<br> <br> port {<br> <br> port = 8080<br> target_port = 80<br> }<br>type = &quot;LoadBalancer&quot;<br> }<br> depends_on = [kubernetes_deployment.example]<br>}<br>output &quot;WordPress-Address&quot; {<br> value = &quot;${kubernetes_service.wp-expose.load_balancer_ingress.0.ip}&quot;<br>}</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=41d8f3f73b66" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Configuring Bastion Host for Securing AWS Infrastructure]]></title>
            <link>https://medium.com/@ishansharma_97521/configuring-bastion-host-for-securing-aws-infrastructure-721ba78fab62?source=rss-ef523eeeac44------2</link>
            <guid isPermaLink="false">https://medium.com/p/721ba78fab62</guid>
            <dc:creator><![CDATA[Ishan Sharma]]></dc:creator>
            <pubDate>Fri, 18 Sep 2020 14:41:43 GMT</pubDate>
            <atom:updated>2020-09-18T14:41:43.265Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Wu06q0EST5YpV6vMJ5BXWw.png" /></figure><p>Counting bastion host in your VPC condition empowers you to safely interface with your occurrences without presenting your condition to the Internet. After you set up your bastion has, you can get to different examples in your VPC through Secure Shell (SSH) associations on Linux.</p><p>Here, I attempted to send the given whole foundation through Terraform As IAC.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*AY6Nu75-1RqsAJnO.png" /></figure><pre>The following code can deploy your entire infrastructure:provider “aws” {<br> region= “ap-south-1”<br> profile=”dhairya”<br>}//Creating our own VPC giving your own cidr block<br>resource “aws_vpc” “main” {<br> cidr_block = “192.168.0.0/16”<br> instance_tenancy = “default”<br> enable_dns_hostnames = true<br> tags = {<br> Name = “myvpc”<br> }<br>}//creating public subnet having public IP mapped to each instance launched here<br>resource “aws_subnet” “main1” {<br> vpc_id = aws_vpc.main.id<br> cidr_block = “192.168.0.0/24”<br> availability_zone = “ap-south-1a”<br> map_public_ip_on_launch = true<br> tags = {<br> Name = “my_public_subnet_1a”<br> }<br> depends_on=[aws_vpc.main,]<br>}//creating private subnet that dont have public Ip to the instances launched here<br>resource “aws_subnet” “main2” {<br> vpc_id = aws_vpc.main.id<br> cidr_block = “192.168.1.0/24”<br> availability_zone = “ap-south-1b”<br> tags = {<br> Name = “my_private_subnet_1b”<br> }<br> depends_on=[aws_vpc.main,]<br>}//creating an internet gateway so that internal routers can go outside and vice-versa<br>resource “aws_internet_gateway” “gw” {<br> vpc_id = aws_vpc.main.idtags = {<br> Name = “my_igw”<br> }<br> depends_on=[aws_vpc.main,]<br>}//Creating a Routing Table and adding route<br>resource “aws_route_table” “r” {<br> vpc_id = aws_vpc.main.idroute {<br> cidr_block = “0.0.0.0/0”<br> gateway_id = aws_internet_gateway.gw.id<br> }tags = {<br> Name = “my_routing_table_public”<br> }<br> depends_on=[aws_vpc.main,aws_internet_gateway.gw,]<br>}//Associating the routing table to our public subnetresource “aws_route_table_association” “a” {<br> subnet_id = aws_subnet.main1.id<br> route_table_id = aws_route_table.r.id<br> depends_on=[aws_route_table.r,aws_subnet.main1,]<br>}//Creating EIP for NAT gateway<br>resource “aws_eip” “eip” {<br>vpc = true<br>}//Creating NAT gateway for private instances<br>resource “aws_nat_gateway” “gw” {<br> allocation_id = aws_eip.eip.id<br> subnet_id = aws_subnet.main1.idtags = {<br> Name = “NAT-gw”<br> }<br>depends_on=[aws_subnet.main1, aws_eip.eip]<br>}//Creating a routing table for NAT gateway <br>resource “aws_route_table” “r2” {<br> vpc_id = aws_vpc.main.idroute {<br> cidr_block = “0.0.0.0/0”<br> gateway_id = aws_internet_gateway.gw.id<br> }tags = {<br> Name = “my_routing_table_private”<br> }<br> depends_on=[aws_vpc.main,aws_internet_gateway.gw,]<br>}//Associating the routing table to our private subnetresource “aws_route_table_association” “a2” {<br> subnet_id = aws_subnet.main2.id<br> route_table_id = aws_route_table.r2.id<br> depends_on=[aws_route_table.r2,aws_subnet.main2,]<br>}//Creating the security group for wordpress app<br>resource “aws_security_group” “sg1” {<br> name = “wp-sg”<br> description = “Allow ssh and http “<br> vpc_id = aws_vpc.main.idingress {<br> description = “allow ssh”<br> from_port = 22<br> to_port = 22<br> protocol = “tcp”<br> cidr_blocks = [“0.0.0.0/0”]<br> }<br> ingress {<br> description = “allow custom tcp”<br> from_port = 81<br> to_port = 81<br> protocol = “tcp”<br> cidr_blocks = [“0.0.0.0/0”]<br> }<br> ingress {<br> description = “allow http”<br> from_port = 80<br> to_port = 80<br> protocol = “tcp”<br> cidr_blocks = [“0.0.0.0/0”]<br> }<br> egress {<br> from_port = 0<br> to_port = 0<br> protocol = “-1”<br> cidr_blocks = [“0.0.0.0/0”]<br> }tags = {<br> Name = “wp-sg”<br> }<br> depends_on=[aws_vpc.main,]}//Creating security group for Bashion Host<br>resource “aws_security_group” “sg2” {<br> name = “bashion-sg”<br> description = “Allow ssh “<br> vpc_id = aws_vpc.main.idingress {<br> description = “allow ssh”<br> from_port = 22<br> to_port = 22<br> protocol = “tcp”<br> cidr_blocks = [“0.0.0.0/0”]<br> }<br> <br> egress {<br> from_port = 0<br> to_port = 0<br> protocol = “-1”<br> cidr_blocks = [“0.0.0.0/0”]<br> }tags = {<br> Name = “bashion_host-sg”<br> }<br> depends_on=[aws_vpc.main,]<br>}//Creating the security group for database<br>resource “aws_security_group” “sg3” {<br> name = “mysql-sg”<br> description = “Allow security groups and check connectivity”<br> vpc_id = aws_vpc.main.idingress {<br> description = “allow wp-sg”<br> from_port = 3306<br> to_port = 3306<br> protocol = “tcp”<br> security_groups = [aws_security_group.sg1.id]<br> }<br> ingress {<br> description = “allow bashion-sg”<br> from_port = 22<br> to_port = 22<br> protocol = “tcp”<br> security_groups = [aws_security_group.sg2.id]<br> }<br> ingress {<br> description = “Allowing ICMP Bashion”<br> from_port = -1<br> to_port = -1<br> protocol = “icmp”<br> cidr_blocks = [aws_security_group.sg2.id]<br> }<br> <br> egress {<br> from_port = 0<br> to_port = 0<br> protocol = “-1”<br> cidr_blocks = [“0.0.0.0/0”]<br> }tags = {<br> Name = “mysql-sg”<br> }<br> depends_on=[aws_vpc.main,aws_security_group.sg1,aws_security_group.sg2,]<br>}//creating instance for Bashion Host<br>resource “aws_instance” “web” {<br> ami = “ami-08706cb5f68222d09”<br> instance_type = “t2.micro”<br> key_name = “tera”<br> security_groups = [aws_security_group.sg2.id]<br> subnet_id = aws_subnet.main1.id<br> tags = {<br> Name = “bashion-host”<br> }<br> depends_on = [aws_security_group.sg2,aws_subnet.main1,]<br>}//Creating instance for WordPress<br>resource “aws_instance” “web1” {<br> ami = “ami-0ebc1ac48dfd14136”<br> instance_type = “t2.micro”<br> key_name = “tera”<br> security_groups = [aws_security_group.sg1.id]<br> subnet_id = aws_subnet.main1.id<br> /*connection {<br> type = “ssh”<br> user = “ec2-user”<br> private_key = file(“C:/Users/HP/Downloads/tera.pem”)<br> host = aws_instance.web.public_ip<br> }<br> provisioner “remote-exec” {<br> inline = [<br> <br> “sudo yum install docker -y”,<br> “sudo systemctl start docker”,<br> “sudo systemctl enable docker”,<br> “sudo docker run -dit — name web -p 81:80 wordpress”,<br> ]<br> }*/tags = {<br> Name = “Wordpress”<br> }<br> depends_on = [aws_security_group.sg1,aws_subnet.main1,]<br>}//Creating instance for MySql<br>resource “aws_instance” “web2” {<br> ami = “ami-0ebc1ac48dfd14136”<br> instance_type = “t2.micro”<br> key_name = “tera”<br> vpc_security_group_ids= [aws_security_group.sg3.id]<br> subnet_id = aws_subnet.main2.id<br> associate_public_ip_address = false<br> user_data = &lt;&lt;EOF<br> #! /bin/bash<br> sudo yum install docker -y<br> sudo systemctl start docker<br> sudo docker run — name mydb -e MYSQL_ROOT_PASSWORD=redhat -e MYSQL_USER=muskan -e MYSQL_PASSWORD=redhat -e MYSQL_DATABASE=wpdb -p 3306:3306 -d mysql:5.7<br> EOF<br> <br> tags = {<br> Name = “mysql”<br> }<br> depends_on=[aws_security_group.sg3,aws_subnet.main2,]<br>}output “mysql_private_ip” {<br>value=aws_instance.web2.private_ip<br>}output “public_ip” {<br>value=aws_instance.web1.public_ip<br>}</pre><p>To create the whole infrastructure in your AWS account using terraform</p><pre>//To download the required plugins:<br>terraform init<br>//to deploy the whole infrastructure:<br>terraform apply</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*xoaC3GS2iYR-XTBQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*7gFdsO-KM68lzMyr.png" /></figure><p>At output we got the public IP of of our web-app , that is WordPress, and the database connected to it is secured totally, as no connectivity to outside world, and has no public IP.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/0*geoBGINpS3eh3c98.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/603/0*D4DeAQugs0UkPnkY.png" /></figure><p>Our web application is set up and now the clients can use our application and they can be very sure about there data, as it is fully secured now.</p><p>Presently in the event that we need to do a few updates in the framework which is facilitating our database , we have Bastion Host. Since, our information base dont have Public IP we cannot interface with outside world and do a few updates. Be that as it may, to unravel this , we utilized NAT passage and connected an Elastic IP to it. Presently the steering table of private subnet is refreshed and through SSHing from Bastion Host to information base host, all the administration purposes can be comprehended without any problem.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*L_eCDwIa_rmP7Qfo.png" /></figure><p>We can connect to our Bashion Host, and there we can save the key to login our database Host. As we aloowed our Bashion Host to enter the database through SSH and ICMP.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*tnPVGnZ3nFSVtggK.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*9LBpNDK4f0LEVaNY.png" /></figure><p>AWS has also given us a solution to solve this issue, we can construct this entire infrastructure in AWS VPC in a single click.. the above code just gave the idea behind this infrastructure that how it is created.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*4_iR3DLTugzsR_PD.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*BgLUVylWIoMLG2Zl.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=721ba78fab62" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Personal VPC creation with some customization]]></title>
            <link>https://medium.com/@ishansharma_97521/personal-vpc-creation-with-some-customization-908d363e77b7?source=rss-ef523eeeac44------2</link>
            <guid isPermaLink="false">https://medium.com/p/908d363e77b7</guid>
            <dc:creator><![CDATA[Ishan Sharma]]></dc:creator>
            <pubDate>Fri, 18 Sep 2020 14:38:31 GMT</pubDate>
            <atom:updated>2020-09-18T14:38:31.775Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ekb4VIjc-g2xA-psWsIPrA.jpeg" /></figure><p>Imagine creating your whole workstation from scratch according to your requirements and customizing it accordingly where you decide which region you want to setup your workstation which is VPC, you decide which laptop or subnet will be public facing and private, you assign the range of IP addresses your DHCP will provide. How amazing it will be.</p><p>Lets make this entire setup by automating it with terraform and using the power of AWS</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*jOofN-K8nDGB8p3D.png" /></figure><p>First of all, we create a VPC. Inside our VPC we create subnets. Both the subnets inside the VPC will have connectivity. For this, we dont have to do anything, AWS manages this entirely itself. Without Public IP, NATing, extra router we have connectivity between the instances situated at different AZs. They will have local connectivity because of internal routers which are managed by AWS.</p><p>The following code creates the VPC and Subnets inside AWS. We give our own range of IP address that we want to assign to our instances. Then we create two subnets at two different AZ.</p><pre>provider “aws” {<br> region = “ap-south-1”<br> profile = “arnav”<br>}resource “aws_vpc” “main” {<br> cidr_block = “192.168.0.0/16”<br> instance_tenancy = “default”<br> enable_dns_hostnames = true<br> tags = {<br> Name = “myvpc”<br> }<br>}resource “aws_subnet” “main1” {<br> vpc_id = aws_vpc.main.id<br> cidr_block = “192.168.0.0/24”<br> availability_zone = “ap-south-1a”<br> map_public_ip_on_launch = true<br> tags = {<br> Name = “my_subnet_1a”<br> }<br>}resource “aws_subnet” “main2” {<br> vpc_id = aws_vpc.main.id<br> cidr_block = “192.168.1.0/24”<br> availability_zone = “ap-south-1b”<br> tags = {<br> Name = “my_subnet_1b”<br> }<br>}</pre><p>To have outside connectivity to our instances inside this VPC we will attach a public facing router which has DNAT rule enabled already . This router is known as <strong>Internet gateway</strong>. For one VPC we attach only one internet gateway router. As soon as we create a Internet Gateway all the internal routers get connected to it automatically.</p><pre>resource “aws_internet_gateway” “gw” {<br> vpc_id = aws_vpc.main.idtags = {<br> Name = “my_igw”<br> }<br>}</pre><p>As we know that the most secure system is the one which does not has outside connectivity. Here, we want to create a secure environment for our Database server suppose, so we will put it in the subnet which is private that is no outside connectivity. And similarly, if we want to create a web-app so that outside client can visit it and use it we will put our web-application server in public subnet. We are making my_subnet_1a public , so we will assign a public IP to all the instances launched in this subnet. And this public IP is actually assigned to Internet Gateway router .</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1010/0*U5aqOPjxoHiETlaI.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1017/0*cc8_PtZ-4KL39saI.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1020/0*_PXlkidJI-Lsc2De.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1020/0*fG8bHjNMcnRdZNEQ.png" /></figure><p>Now, if any instance wants to go to to the public world or reply to a public IP it must know its Internet Gateway as internal routers will not help in this case. Hence, we give a routing table to the instance at the public subnet so that they know, if they want to connect to the world, this is the gateway for that . DHCP is the one that give this information to the instances at the respective subnet. Hence , with the following code we create a routing table and update the DHCP of our public subnet. (here: my_subnet_1a)</p><pre>resource “aws_route_table” “r” {<br> vpc_id = aws_vpc.main.idroute {<br> cidr_block = “0.0.0.0/0”<br> gateway_id = aws_internet_gateway.gw.id<br> }tags = {<br> Name = “my_routing_table”<br> }<br>}resource “aws_route_table_association” “a” {<br> subnet_id = aws_subnet.main1.id<br> route_table_id = aws_route_table.r.id<br>}</pre><p>Associating the Routing table to my_subnet_1a (its Dedicated DHCP) we get:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1018/0*cLCJ3oVEOEOJW7LD.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1020/0*uElYtyvP3wdiLgkg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1017/0*5zUe3ug6U9qOw2U0.png" /></figure><p>Firewalls at system adds an additional security layer to any system as it control the ingress and outgress traffic to any system. For this we create security groups in our VPC to assign it to instances launched there. suppose we want to create a web-application ex : WordPress , we enable port 80 ingress traffic, and port 22 for management purpose.</p><pre>resource “aws_security_group” “sg1” {<br> name = “wp-sg”<br> description = “Allow ssh and http “<br> vpc_id = aws_vpc.main.idingress {<br> description = “allow ssh”<br> from_port = 22<br> to_port = 22<br> protocol = “tcp”<br> cidr_blocks = [“0.0.0.0/0”]<br> }<br> ingress {<br> description = “allow http”<br> from_port = 80<br> to_port = 80<br> protocol = “tcp”<br> cidr_blocks = [“0.0.0.0/0”]<br> }<br> egress {<br> from_port = 0<br> to_port = 0<br> protocol = “-1”<br> cidr_blocks = [“0.0.0.0/0”]<br> }tags = {<br> Name = “wp-sg”<br> }<br>}</pre><p>And now for suppose our data base server we want that only our web application can access this server , since our web application can undergo scaling, so the IP address allowing for ingress might change but the security group assigned to the web-application at each instance must remain same, so here we decide the inbound traffic according to the security groups at port 3306.</p><pre>resource “aws_security_group” “sg3” {<br> name = “mysql-sg”<br> description = “Allow security groups”<br> vpc_id = aws_vpc.main.idingress {<br> description = “allow wp-sg”<br> from_port = 3306<br> to_port = 3306<br> protocol = “tcp”<br> security_groups = [aws_security_group.sg1.id]<br> }<br> egress {<br> from_port = 0<br> to_port = 0<br> protocol = “-1”<br> cidr_blocks = [“0.0.0.0/0”]<br> }tags = {<br> Name = “mysql-sg”<br> }<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/843/0*SNkqqGCiT34ZmKsN.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/846/0*paryLlAukLb2Y2w1.png" /></figure><p>Now we launch our instances at required subnet, that is database server ex MySQL at private subnet and Web-application ex WordPress at public subnet.</p><pre>resource “aws_instance” “web” {<br> ami = “ami-08706cb5f68222d09”<br> instance_type = “t2.micro”<br> key_name = “mykey-new”<br> security_groups = [aws_security_group.sg3.id]<br> subnet_id = aws_subnet.main2.id<br> tags = {<br> Name = “MySQL”<br> }<br>}<br>resource “aws_instance” “web1” {<br> ami = “ami-7e257211”<br> instance_type = “t2.micro”<br> key_name = “mykey-new”<br> security_groups = [aws_security_group.sg1.id]<br> subnet_id = aws_subnet.main1.id<br> tags = {<br> Name = “wordpress”<br> }<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1017/0*RiFL3HHvNW--5NoH.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1014/0*8fgk9XHqkvxLduMk.png" /></figure><p>Now running the entire code in terraform we can launch the entire infrastructure that we created according to our own customization.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*4O3qyDeDaqVbV-qC.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ghSWqf9feSMeWhaP.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/919/0*JkidZD0-M9fWqmaG.png" /></figure><p>And there you go! You entirely created your infrastructure. Ready to use your Web-app which is completely secured.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1017/0*quGybXhyYhROtA4m.png" /></figure><p>Now in just a single click you can destroy your entire infrastructure thats the beauty of terraform.</p><pre>terraform destroy -auto-approve</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=908d363e77b7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cloud Infrastructure set up using EFS]]></title>
            <link>https://medium.com/@ishansharma_97521/cloud-infrastructure-set-up-using-efs-ab052799bb3d?source=rss-ef523eeeac44------2</link>
            <guid isPermaLink="false">https://medium.com/p/ab052799bb3d</guid>
            <dc:creator><![CDATA[Ishan Sharma]]></dc:creator>
            <pubDate>Fri, 18 Sep 2020 13:52:07 GMT</pubDate>
            <atom:updated>2020-09-18T13:52:07.840Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*3adB1e7AVssEe1OjV1iLlw.jpeg" /></figure><blockquote><em>1.Create Security group which allow the port 80.</em></blockquote><blockquote><em>2. Launch EC2 instance.</em></blockquote><blockquote><em>3. In this Ec2 instance use the existing key or provided key and security group which we have created in step 1.</em></blockquote><blockquote><em>4. Launch one Volume using the EFS service and attach it in your vpc, then mount that volume into /var/www/html</em></blockquote><blockquote><em>5. Developer have uploded the code into github repo also the repo has some images.</em></blockquote><blockquote><em>6. Copy the github repo code into /var/www/html</em></blockquote><blockquote><em>7. Create S3 bucket, and copy/deploy the images from github repo into the s3 bucket andchange the permission to public readable.</em></blockquote><blockquote><em>8 Create a Cloudfront using s3 bucket(which contains images) and use the Cloudfront URL to update in code in /var/www/html</em></blockquote><p><strong>Terraform Code for setting up Security Group for ingress and egress ports on AWS</strong></p><pre>resource &quot;aws_security_group&quot; &quot;sg&quot; {<br>  name        = &quot;sg&quot;<br>  description = &quot;created using terraform&quot;<br>  vpc_id      = &quot;vpc-7a899412&quot;<br>  ingress {<br>    description = &quot;ssh inbound rule using terraform&quot;<br>    from_port   = 22<br>    to_port     = 22<br>    protocol    = &quot;tcp&quot;<br>    cidr_blocks = [&quot;0.0.0.0/0&quot;]<br>    ipv6_cidr_blocks=[&quot;::/0&quot;]<br>  }<br>  ingress {<br>    description = &quot;http inbound rule using terraform&quot;<br>    from_port   = 80<br>    to_port     = 80<br>    protocol    = &quot;tcp&quot;<br>    cidr_blocks = [&quot;0.0.0.0/0&quot;]<br>    ipv6_cidr_blocks=[&quot;::/0&quot;]<br>  }<br>  ingress {<br>    description = &quot;custom tcp inbound rule using terraform&quot;<br>    from_port   = 81<br>    to_port     = 81<br>    protocol    = &quot;tcp&quot;<br>    cidr_blocks = [&quot;0.0.0.0/0&quot;]<br>    ipv6_cidr_blocks=[&quot;::/0&quot;]<br>  }<br>  <br>  ingress {<br>    description = &quot;NFS&quot;<br>    from_port   = 2049<br>    to_port     = 2049<br>    protocol    = &quot;tcp&quot;<br>    cidr_blocks = [&quot;0.0.0.0/0&quot;]<br>    ipv6_cidr_blocks=[&quot;::/0&quot;]<br>  }<br>  egress {<br>    from_port   = 0<br>    to_port     = 0<br>    protocol    = &quot;-1&quot;<br>    cidr_blocks = [&quot;0.0.0.0/0&quot;]<br>  }<br>  tags = {<br>    Name = &quot;sg&quot;<br>  }<br>}</pre><p><strong>IAC for EFS is given below:</strong></p><pre>resource &quot;aws_efs_file_system&quot; &quot;efs_new&quot; {<br>  depends_on = [<br>    aws_security_group.sg,<br>  ]<br>  <br>  creation_token = &quot;EFS_token&quot;<br>  tags = {<br>    Name = &quot;EFS_token&quot;<br>  }<br>}</pre><p><strong>To mount the EFS storage to EC2 instance the IAC is given below:</strong></p><pre>resource &quot;aws_efs_mount_target&quot; &quot;mnt&quot; {<br> <br>  depends_on = [<br>    aws_efs_file_system.efs1,<br>  ]<br>  file_system_id = aws_efs_file_system.efs1.id<br>  subnet_id      = &quot;subnet-366b007a&quot;<br>  security_groups = [aws_security_group.sg.id]<br>}resource &quot;aws_efs_mount_target&quot; &quot;mt_new&quot; {<br>  depends_on = [<br>    aws_efs_mount_target.mnt,<br>  ]<br>  file_system_id = aws_efs_file_system.efs1.id<br>  subnet_id      = &quot;subnet-a2e954d9&quot;<br>  security_groups = [aws_security_group.sg.id]  <br>}resource &quot;aws_efs_mount_target&quot; &quot;mnt_new&quot; {<br>  depends_on = [<br>    aws_efs_mount_target.mt_new,<br>  ]<br>  file_system_id = aws_efs_file_system.efs1.id<br>  subnet_id      = &quot;subnet-a6e5dfce&quot;<br>  security_groups = [aws_security_group.sg.id]<br>}</pre><p>Now that we have created a Key Pair, a Security Group and also an Elastic File System or EFS mounted on our subnets so we can now use them in launching our instance.</p><p>So as to utilize the EFS we need to initially mount it to our occasion by running some particular orders in any case, this should be possible by composing those orders in the client information square of our Instance creation code which will run the orders we give along making the Instance.</p><pre>resource &quot;aws_instance&quot;  &quot;i2&quot; {<br>   depends_on = [<br>    aws_efs_mount_target.mnt_new,<br>	aws_cloudfront_distribution.sbc,<br>  ]<br>  <br>  ami           = &quot;ami-005956c5f0f757d37&quot;<br>  instance_type = &quot;t2.micro&quot;<br>  key_name	= &quot;key&quot;<br>  security_groups =  [ &quot;sg&quot; ] <br>  availability_zone = &quot;ap-south-1a&quot;<br>  <br>  user_data = &lt;&lt;-EOF<br>		 #!/bin/bash<br>		 sudo yum install -y amazon-efs-utils<br>		 sudo yum install -y nfs-utils<br>		 file_system_id_1=&quot;${aws_efs_file_system.efs1.id}&quot;<br>         mkdir /var/www<br>		 mkdir /var/www/html<br>         mount -t efs $file_system_id_1:/ /var/www/html<br>		 echo $file_system_id_1:/ /var/www/html efs defaults,_netdev 0 0 &gt;&gt; /etc/fstab<br>	EOF<br>	<br>  tags = {<br>    Name = &quot;terra&quot;<br>  }<br>}</pre><p>So as to utilize Images in our site page we would need to utilize S3 Bucket i.e Amazon Simple Storage Services, every single item transferred to the pail has its own article url which can be utilized in our website page to get the picture document and use it.</p><p><strong>IAC TO CREATE S3 BUCKET :</strong></p><pre>Blocking all public access enables us to make our S3 bucket and it contents inaccessible from the outside world making them more secure.resource &quot;aws_s3_bucket&quot; &quot;s3b&quot; {<br>  bucket = &quot;bucket911&quot;<br>  acl    = &quot;private&quot;<br>  tags = {<br>    Name        = &quot;Terra-bucket&quot;<br>    Environment = &quot;Dev&quot;<br>  }<br>}//BLOCK PUBLIC ACCESS : resource &quot;aws_s3_bucket_public_access_block&quot; &quot;sbb&quot; {<br>   depends_on = [<br>    aws_s3_bucket.s3b,<br>  ]<br>  <br>  bucket = aws_s3_bucket.s3b.id<br>  block_public_acls   = true<br>  block_public_policy = true<br>  ignore_public_acls  = true<br>  restrict_public_buckets = true<br>}</pre><p><strong>IAC TO CREATE CLOUDFRONT</strong> :</p><pre>locals {<br>  s3_origin_id = aws_s3_bucket.s3b.bucket<br>}<br>resource &quot;aws_cloudfront_distribution&quot; &quot;sbc&quot; {<br>	<br>	 depends_on = [<br>   aws_s3_bucket_object.sbo,<br>  ]<br>  <br>  origin {<br>    domain_name = &quot;${aws_s3_bucket.s3b.bucket}.s3.amazonaws.com&quot;<br>    origin_id   = aws_s3_bucket.s3b.bucket<br>    s3_origin_config {<br>      origin_access_identity = &quot;origin-access-identity/cloudfront/E3NRGG0HS1Z25F&quot;<br>    }<br>  }<br>  enabled             = true<br>  is_ipv6_enabled     = true<br>  comment             = &quot;&quot;<br>  default_root_object = &quot;&quot;<br>  <br>  default_cache_behavior {<br>    allowed_methods  = [&quot;GET&quot;, &quot;HEAD&quot;]<br>    cached_methods   = [&quot;GET&quot;, &quot;HEAD&quot;]<br>    target_origin_id = aws_s3_bucket.s3b.bucket<br>    forwarded_values {<br>      query_string = false<br>      cookies {<br>        forward = &quot;none&quot;<br>      }<br>    }<br>    viewer_protocol_policy = &quot;redirect-to-https&quot;<br>    min_ttl                = 0<br>    default_ttl            = 86400<br>    max_ttl                = 31536000<br>  }<br>  price_class = &quot;PriceClass_All&quot;<br>  restrictions {<br>    geo_restriction {<br>      restriction_type = &quot;none&quot;<br>      locations        = []<br>    }<br>  }<br>  tags = {<br>    Environment = &quot;TerraCloud&quot;<br>  }<br>  viewer_certificate {<br>    cloudfront_default_certificate = true<br>  }<br>}</pre><p><strong>The code to add a S3 Bucket Policy regarding Cloudfront :-</strong></p><pre>resource &quot;aws_s3_bucket_policy&quot; &quot;sbp&quot; {<br>  depends_on = [<br>   aws_s3_bucket_public_access_block.sbb,<br>  ]<br>  <br>  bucket = aws_s3_bucket.s3b.id<br>  policy = &lt;&lt;EOF<br>{<br>  &quot;Version&quot;: &quot;2008-10-17&quot;,<br>    &quot;Id&quot;: &quot;PolicyForCloudFrontPrivateContent&quot;,<br>    &quot;Statement&quot;: [<br>        {<br>            &quot;Sid&quot;: &quot;1&quot;,<br>            &quot;Effect&quot;: &quot;Allow&quot;,<br>            &quot;Principal&quot;: {<br>                &quot;AWS&quot;: &quot;arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity E3NRPP0HT1Z25F&quot;<br>            },<br>            &quot;Action&quot;: &quot;s3:GetObject&quot;,<br>            &quot;Resource&quot;: &quot;arn:aws:s3:::${aws_s3_bucket.s3b.bucket}/*&quot;<br>        }<br>    ]<br>}<br>EOF<br>}</pre><p>In the above advances we set up the assets we needed to fabricate our Infrastructure.</p><p>In this we will Configure Apache httpd Web-Server in our case/far off machine running on the cloud, again we can do this in two different ways possibly we can utilize ssh from our cli or we can utilize Putty, yet on the other hand that would need to be done physically which isn’t what we need to do.</p><p>This cycle should be possible by utilizing Terraform Provisioner which can access and run orders on your Remote machine CLI by means of ssh association and furthermore on your nearby machine CLI.</p><p><strong>The code to utilize Provisioner for Remote-Execution and Local-Execution</strong> :-</p><pre>resource &quot;null_resource&quot; &quot;nullresource&quot;  {<br> depends_on = [<br>   aws_instance.i2,<br>  ]<br>    connection {<br>    type     = &quot;ssh&quot;<br>    user     = &quot;ec2-user&quot;<br>    private_key = file(&quot;C:/Users/One Drive/Desktop/tera/Task2/key.pem&quot;)<br>    host     = aws_instance.i2.public_ip<br>  }<br>provisioner &quot;remote-exec&quot; {<br>    inline = [<br>      &quot;sudo yum install httpd  php git -y&quot;,<br>	  &quot;sudo service httpd start&quot;,<br>      &quot;sudo rm -rf /var/www/html&quot;,<br>      &quot;sudo git clone <a href="https://github.com/Phoenix918/Repo1.git">https://github.com/dhairya2019/Repo.git</a> /var/www/html&quot;,<br>	  &quot;sudo sed -i &#39;s/old_domain/${aws_cloudfront_distribution.sbc.domain_name}/g&#39; /var/www/html/ocean.html&quot; <br>    ]<br>  }<br>}<br>resource &quot;null_resource&quot; &quot;ncl&quot;  {<br>depends_on = [<br>    null_resource.nullresource,<br>  ]<br>	provisioner &quot;local-exec&quot; {<br>	    command = &quot;chrome  http://${aws_instance.i2.public_ip}/ocean.html&quot;<br>  	}<br>}</pre><p><strong>Here comes the final code stacked up together to create this complete infrastructure in one go. Isn’t it great!</strong></p><pre>provider &quot;aws&quot; {<br>  region                  = &quot;ap-south-1&quot;<br>  profile                 = &quot;dhairya&quot;<br>}<br>resource &quot;aws_s3_bucket&quot; &quot;s3b&quot; {<br>  bucket = &quot;bucket911&quot;<br>  acl    = &quot;private&quot;<br>  tags = {<br>    Name        = &quot;Terra-bucket&quot;<br>    Environment = &quot;Development&quot;<br>  }<br>}<br>resource &quot;aws_s3_bucket_public_access_block&quot; &quot;sbb&quot; {<br>   depends_on = [<br>    aws_s3_bucket.s3b,<br>  ]<br>  <br>  bucket = aws_s3_bucket.s3b.id<br>  block_public_acls   = true<br>  block_public_policy = true<br>  ignore_public_acls  = true<br>  restrict_public_buckets = true<br>}<br>resource &quot;aws_security_group&quot; &quot;sg&quot; {<br>  name        = &quot;sg&quot;<br>  description = &quot;created using terraform&quot;<br>  vpc_id      = &quot;vpc-7a899412&quot;<br>  ingress {<br>    description = &quot;ssh inbound rule using terraform&quot;<br>    from_port   = 22<br>    to_port     = 22<br>    protocol    = &quot;tcp&quot;<br>    cidr_blocks = [&quot;0.0.0.0/0&quot;]<br>    ipv6_cidr_blocks=[&quot;::/0&quot;]<br>  }<br>  ingress {<br>    description = &quot;http inbound rule using terraform&quot;<br>    from_port   = 80<br>    to_port     = 80<br>    protocol    = &quot;tcp&quot;<br>    cidr_blocks = [&quot;0.0.0.0/0&quot;]<br>    ipv6_cidr_blocks=[&quot;::/0&quot;]<br>  }<br>  ingress {<br>    description = &quot;custom tcp inbound rule using terraform&quot;<br>    from_port   = 81<br>    to_port     = 81<br>    protocol    = &quot;tcp&quot;<br>    cidr_blocks = [&quot;0.0.0.0/0&quot;]<br>    ipv6_cidr_blocks=[&quot;::/0&quot;]<br>  }<br>  <br>  ingress {<br>    description = &quot;NFS&quot;<br>    from_port   = 2049<br>    to_port     = 2049<br>    protocol    = &quot;tcp&quot;<br>    cidr_blocks = [&quot;0.0.0.0/0&quot;]<br>    ipv6_cidr_blocks=[&quot;::/0&quot;]<br>  }<br>  egress {<br>    from_port   = 0<br>    to_port     = 0<br>    protocol    = &quot;-1&quot;<br>    cidr_blocks = [&quot;0.0.0.0/0&quot;]<br>  }<br>  tags = {<br>    Name = &quot;sc6&quot;<br>  }<br>}<br>resource &quot;aws_key_pair&quot; &quot;key&quot; {<br>  key_name   = &quot;key&quot;<br>  public_key = &quot;ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEA4u7diIn9xz4yqlecyA1ep6Y0ERczDnVFW12Y71YgTlgt8FwXxLyFUGZKPvs41Ul3uGwZT26WBU1YyLkBvEIi0lNtClYhqw2B3Cp7PJStG+wHoV2b5ob23kdv5K0S7zq0ZY6XG1Gmu9ZwXYntawAST2O82NB2xPyFvGHw0w8fj16otPMuCWp4lv3LKwpaPsxrcLCUlqGjrkIcaQEWJWoNDza2ewJyRCs3mZELgEfmOQ6GrJOMtXs/KUczvgjtr71QYLsrk3NEn+z2T0bT0/68AH2CSQKIHWILjmltgkBObymHooCxwSsowK8aznnfRAcD/LlLLA9yoHenHNmMwX/eoQ== rsa-key-20200715&quot;<br>}<br>resource &quot;aws_efs_file_system&quot; &quot;efs1&quot; {<br>  depends_on = [<br>    aws_security_group.sg,<br>  ]<br>  <br>  creation_token = &quot;TEFS_1&quot;<br>  tags = {<br>    Name = &quot;TEFS_1&quot;<br>  }<br>}<br>resource &quot;aws_efs_mount_target&quot; &quot;mnt1&quot; {<br> <br>  depends_on = [<br>    aws_efs_file_system.efs1,<br>  ]<br>  file_system_id = aws_efs_file_system.efs1.id<br>  subnet_id      = &quot;subnet-366b007a&quot;<br>  security_groups = [aws_security_group.sg.id]<br>}<br>resource &quot;aws_efs_mount_target&quot; &quot;mt_)new&quot; {<br>  depends_on = [<br>    aws_efs_mount_target.mnt,<br>  ]<br>  file_system_id = aws_efs_file_system.efs1.id<br>  subnet_id      = &quot;subnet-a2e954d9&quot;<br>  security_groups = [aws_security_group.sg.id]  <br>}resource &quot;aws_efs_mount_target&quot; &quot;mnt_new&quot; {<br>  depends_on = [<br>    aws_efs_mount_target.mt_new,<br>  ]<br>  file_system_id = aws_efs_file_system.efs1.id<br>  subnet_id      = &quot;subnet-a6e5dfce&quot;<br>  security_groups = [aws_security_group.sg.id]<br>}<br>resource &quot;aws_instance&quot;  &quot;i2&quot; {<br>   depends_on = [<br>    aws_efs_mount_target.mnt_new,<br>	aws_cloudfront_distribution.sbc,<br>  ]<br>  <br>  ami           = &quot;ami-005956c5f0f757d37&quot;<br>  instance_type = &quot;t2.micro&quot;<br>  key_name	= &quot;key&quot;<br>  security_groups =  [ &quot;sg&quot; ] <br>  availability_zone = &quot;ap-south-1a&quot;<br>  <br>  user_data = &lt;&lt;-EOF<br>		 #!/bin/bash<br>		 sudo yum install -y amazon-efs-utils<br>		 sudo yum install -y nfs-utils<br>		 file_system_id_1=&quot;${aws_efs_file_system.efs1.id}&quot;<br>         mkdir /var/www<br>		 mkdir /var/www/html<br>         mount -t efs $file_system_id_1:/ /var/www/html<br>		 echo $file_system_id_1:/ /var/www/html efs defaults,_netdev 0 0 &gt;&gt; /etc/fstab<br>	EOF<br>	<br>  tags = {<br>    Name = &quot;Terraos_2&quot;<br>  }<br>}<br>locals {<br>  s3_origin_id = aws_s3_bucket.s3b.bucket<br>}<br>resource &quot;aws_cloudfront_distribution&quot; &quot;sbc&quot; {<br>	<br>	 depends_on = [<br>   aws_s3_bucket_object.sbo,<br>  ]<br>  <br>  origin {<br>    domain_name = &quot;${aws_s3_bucket.s3b.bucket}.s3.amazonaws.com&quot;<br>    origin_id   = aws_s3_bucket.s3b.bucket<br>    s3_origin_config {<br>      origin_access_identity = &quot;origin-access-identity/cloudfront/E3NRGG0HS1Z25F&quot;<br>    }<br>  }<br>  enabled             = true<br>  is_ipv6_enabled     = true<br>  comment             = &quot;&quot;<br>  default_root_object = &quot;&quot;<br>  <br>  default_cache_behavior {<br>    allowed_methods  = [&quot;GET&quot;, &quot;HEAD&quot;]<br>    cached_methods   = [&quot;GET&quot;, &quot;HEAD&quot;]<br>    target_origin_id = aws_s3_bucket.s3b.bucket<br>    forwarded_values {<br>      query_string = false<br>      cookies {<br>        forward = &quot;none&quot;<br>      }<br>    }<br>    viewer_protocol_policy = &quot;redirect-to-https&quot;<br>    min_ttl                = 0<br>    default_ttl            = 86400<br>    max_ttl                = 31536000<br>  }<br>  price_class = &quot;PriceClass_All&quot;<br>  restrictions {<br>    geo_restriction {<br>      restriction_type = &quot;none&quot;<br>      locations        = []<br>    }<br>  }<br>  tags = {<br>    Environment = &quot;TerraCloud&quot;<br>  }<br>  viewer_certificate {<br>    cloudfront_default_certificate = true<br>  }<br>}<br>resource &quot;aws_s3_bucket_policy&quot; &quot;sbp&quot; {<br>  depends_on = [<br>   aws_s3_bucket_public_access_block.sbb,<br>  ]<br>  <br>  bucket = aws_s3_bucket.s3b.id<br>  policy = &lt;&lt;EOF<br>{<br>  &quot;Version&quot;: &quot;2008-10-17&quot;,<br>    &quot;Id&quot;: &quot;PolicyForCloudFrontPrivateContent&quot;,<br>    &quot;Statement&quot;: [<br>        {<br>            &quot;Sid&quot;: &quot;1&quot;,<br>            &quot;Effect&quot;: &quot;Allow&quot;,<br>            &quot;Principal&quot;: {<br>                &quot;AWS&quot;: &quot;arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity E3NRGG0HS1Z25F&quot;<br>            },<br>            &quot;Action&quot;: &quot;s3:GetObject&quot;,<br>            &quot;Resource&quot;: &quot;arn:aws:s3:::${aws_s3_bucket.s3b.bucket}/*&quot;<br>        }<br>    ]<br>}<br>EOF<br>}<br>resource &quot;aws_s3_bucket_object&quot; &quot;sbo&quot; {<br>  depends_on = [<br>   aws_s3_bucket_policy.sbp,<br>  ]<br>  <br>  bucket = aws_s3_bucket.s3b.id<br>  key    = &quot;s3upload.jpg&quot;<br>  source = &quot;/Users/OneDrive/Downloads/s3upload.jpg&quot;<br>  content_type = &quot;image/jpeg&quot;<br>  content_disposition = &quot;inline&quot;<br>}<br>resource &quot;null_resource&quot; &quot;nullresource&quot;  {<br> depends_on = [<br>   aws_instance.i2,<br>  ]<br>    connection {<br>    type     = &quot;ssh&quot;<br>    user     = &quot;ec2-user&quot;<br>    private_key = file(&quot;C:/Users/OneDrive/Desktop/tera/Task2/key.pem&quot;)<br>    host     = aws_instance.i2.public_ip<br>  }<br>provisioner &quot;remote-exec&quot; {<br>    inline = [<br>      &quot;sudo yum install httpd  php git -y&quot;,<br>	  &quot;sudo service httpd start&quot;,<br>      &quot;sudo rm -rf /var/www/html&quot;,<br>      &quot;sudo git clone <a href="https://github.com/Phoenix918/Repo1.git">https://github.com/dhairya2019/repo.git</a> /var/www/html&quot;,<br>	  &quot;sudo sed -i &#39;s/old_domain/${aws_cloudfront_distribution.sbc.domain_name}/g&#39; /var/www/html/ocean.html&quot; <br>    ]<br>  }<br>}<br>resource &quot;null_resource&quot; &quot;ncl&quot;  {<br>depends_on = [<br>    null_resource.nullresource,<br>  ]<br>	provisioner &quot;local-exec&quot; {<br>	    command = &quot;chrome  http://${aws_instance.i2.public_ip}/ocean.html&quot;<br>  	}}</pre><p>And finally its completed.</p><p>To run it run the following commands</p><pre>terrafrom initterraform apply</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*_4doM3kFlxOcNoa8.png" /></figure><p>After creating the infrastructure you need to destroy it to protect yourself from being charged</p><p>use this command to destroy the complete infrastructure</p><pre>terraform destroy</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*CgnyNrW-QiiQETQd.png" /></figure><p>This is accomplished by utilizing a few assets/administrations like in AWS-Cloud one would require assets/administrations like EC2, S3, Elastic File System or EFS, Cloudfront, Security Groups, key pair, and more as and when required i.e we would require an Infrastructure.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ab052799bb3d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[CloudFront Distribution in AWS using Terraform]]></title>
            <link>https://medium.com/@ishansharma_97521/cloudfront-distribution-in-aws-using-terraform-837296e11987?source=rss-ef523eeeac44------2</link>
            <guid isPermaLink="false">https://medium.com/p/837296e11987</guid>
            <dc:creator><![CDATA[Ishan Sharma]]></dc:creator>
            <pubDate>Fri, 18 Sep 2020 13:48:22 GMT</pubDate>
            <atom:updated>2020-09-18T13:48:22.200Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/750/1*Yh-h1NOjC3P7t4SenKmqxg.jpeg" /></figure><p>CloudFront is widely used service provided by AWS. CloudFront uses edge locations to load additional website data. To explain this further, consider this example. Suppose you launch your website using EC2. So normally you would store all the website code as well as website data(i.e images, videos etc.) in the same EC2 machine. But this is not well optimized setup and latency is high. This is because EC2 is a regional service and can’t use edge locations. What we can do instead is create a S3 bucket and store all the website data on it. S3 is a global service so we are using CloudFront for S3 we can use edge locations to load our website data fast with reduced latency.</p><p>We will create the setup along the lines of this setup. We will Terraform to do so. Terraform is used to automate the AWS process.</p><p><strong>A brief overview of what this article achieves.</strong></p><ol><li>Create the key and security group which allow the port 80.</li><li>Launch EC2 instance.</li><li>In this EC2 instance use the key and security group which we have created in step 1.</li><li>Launch one Volume (EBS) and mount that volume into /var/www/html</li><li>Developer have uploded the code into github repo also the repo has some images.</li><li>Copy the github repo code into /var/www/html</li><li>Create S3 bucket, and copy/deploy the images from github repo into the s3 bucket and change the permission to public readable.</li><li>Create a Cloudfront using s3 bucket(which contains images) and use the Cloudfront URL to update in code in /var/www/html</li></ol><p>In the following steps I created my own security group. Then I created an EC2 instance and installed httpd server on it. This EC2 will act as our web server. I created one more EBS. We will mount /var/www/html folder to this EBS. Mounting ensures that even if our server(i.e EC2 fails the website code is safe). We then clone our GitHub code to /var/www/html folder.</p><p><strong>Specifying the provider</strong></p><p>Specifier in our case is AWS. Specifying the region, ap-south-1 is Mumbai.</p><pre>provider &quot;aws&quot; {<br>  region  = &quot;ap-south-1&quot;<br>  <br>}</pre><p><strong>Creating a Security allowing traffic to port number 80</strong></p><p>The following code is used to create security group.</p><pre>resource &quot;aws_security_group&quot; &quot;allow_tls&quot; {<br>  name        = &quot;Security_01&quot;<br>  description = &quot;Allow SSH &amp; HTTP inbound traffic&quot;ingress {<br>    description = &quot;allowing HTTP&quot;<br>    from_port   = 80<br>    to_port     = 80<br>    protocol    = &quot;tcp&quot;<br>    cidr_blocks = [&quot;0.0.0.0/0&quot;]<br>    <br>  }<br>ingress {<br>    description = &quot;allowing SSH&quot;<br>    from_port   = 22<br>    to_port     = 22<br>    protocol    = &quot;tcp&quot;<br>    cidr_blocks = [&quot;0.0.0.0/0&quot;]<br>  }egress {<br>    from_port   = 0<br>    to_port     = 0<br>    protocol    = &quot;-1&quot;<br>    cidr_blocks = [&quot;0.0.0.0/0&quot;]<br>  }tags = {<br>    Name = &quot;Security_01&quot;<br>  }<br>}output &quot;sec_op&quot;{<br> value = aws_security_group.allow_tls.name<br>}</pre><p><strong>Launching Ec2 instance using security group we created</strong></p><pre>resource &quot;aws_instance&quot; &quot;web&quot; {<br>  ami           = &quot;ami-0447a12f28fddb066&quot;<br>  instance_type = &quot;t2.micro&quot;<br>  key_name = &quot;d&quot;<br>  security_groups =[ &quot;Security_01&quot; ]<br> <br> connection {<br>    type     = &quot;ssh&quot;<br>    user     = &quot;ec2-user&quot;<br>    private_key = file(&quot;C:/Users/asus/Desktop/d.pem&quot;)<br>    host     = aws_instance.web.public_ip<br>  }provisioner &quot;remote-exec&quot; {<br>    inline = [<br>      &quot;sudo yum install httpd php git -y&quot;,<br>      &quot;sudo systemctl start httpd&quot;,<br>      &quot;sudo systemctl enable httpd&quot;,<br>     ]<br>  }tags = {<br>    Name = &quot;project_os&quot;<br>  }}</pre><p><strong>Creating EBS volume and attaching it to Ec2 instance</strong></p><pre>resource &quot;aws_ebs_volume&quot; &quot;example&quot; {<br>  availability_zone = aws_instance.web.availability_zone<br>  size              = 1tags = {<br>    Name = &quot;project_vol&quot;<br>  }<br>}output &quot;ebs_vol&quot;{<br> value =aws_ebs_volume.example<br>}// 6. attaching the volume.resource &quot;aws_volume_attachment&quot; &quot;ebs_att&quot; {<br>  device_name = &quot;/dev/sdh&quot;<br>  volume_id   = &quot;${aws_ebs_volume.example.id}&quot;<br>  instance_id = &quot;${aws_instance.web.id}&quot;<br>  force_detach= true<br>}</pre><p><strong>Configuring EBS Volume and cloning from GitHub to /var/www/html</strong></p><pre>resource &quot;null_resource&quot; &quot;remote_01&quot;  {<br> <br> depends_on = [<br> aws_volume_attachment.ebs_att,<br> ]<br> <br> connection {<br>    type     = &quot;ssh&quot;<br>    user     = &quot;ec2-user&quot;<br>    private_key = file(&quot;C:/Users/asus/Desktop/d.pem&quot;)<br>    host     = aws_instance.web.public_ip<br>  }provisioner &quot;remote-exec&quot; {<br>   <br>  inline = [<br>      &quot;sudo mkfs.ext4  /dev/xvdh&quot;,<br>      &quot;sudo mount /dev/xvdh /var/www/html&quot;,<br>      &quot;sudo rm -rf /var/www/html/*&quot;, <br>      &quot;sudo git clone <a href="https://github.com/web-crawler/terra.git">https://github.com/web-crawler/terra.git</a>  /var/www/html &quot;<br>     ]<br>  }}</pre><p>Now in the following I will show how I created S3 bucket and CloudFront.</p><p><strong>Creating S3 and uploading additional website data into it.</strong></p><p>In this case website data is a local image which I am uploading to S3.</p><pre>resource &quot;aws_s3_bucket&quot; &quot;bucket1w2w2w2w2w2&quot; {<br>  <br>  acl    = &quot;public-read&quot;<br>  versioning {<br>enabled=true<br>}<br>}resource &quot;aws_s3_bucket_object&quot; &quot;bucket1&quot; {<br>   bucket = aws_s3_bucket.bucket1w2w2w2w2w2.bucket<br>   key = &quot;mypic&quot;<br>   acl = &quot;public-read&quot;<br>   source=&quot;C:/Users/asus/Desktop/web.png&quot;<br>   etag = filemd5(&quot;C:/Users/asus/Desktop/web.png&quot;)tags = {<br>    Name        = &quot;My bucket&quot;<br>    Environment = &quot;Dev&quot;<br>  }<br>}</pre><p><strong>Creating CloudFront Distribution for our S3 bucket</strong></p><pre>resource &quot;aws_cloudfront_distribution&quot; &quot;s3_distribution&quot; {<br>depends_on = [<br>   null_resource.remote_01,<br>  ]  <br>origin {<br>    domain_name = &quot;${aws_s3_bucket.bucket1w2w2w2w2w2.bucket_regional_domain_name}&quot;<br>    origin_id   = &quot;my_first_origin&quot;}enabled             = true<br>  is_ipv6_enabled     = true<br>  comment             = &quot;Some comment&quot;<br>  default_root_object = &quot;mypic&quot;default_cache_behavior {<br>    allowed_methods  = [&quot;DELETE&quot;, &quot;GET&quot;, &quot;HEAD&quot;, &quot;OPTIONS&quot;, &quot;PATCH&quot;, &quot;POST&quot;, &quot;PUT&quot;]<br>    cached_methods   = [&quot;GET&quot;, &quot;HEAD&quot;]<br>    target_origin_id = &quot;my_first_origin&quot;forwarded_values {<br>      query_string = falsecookies {<br>        forward = &quot;none&quot;<br>      }<br>    }viewer_protocol_policy = &quot;allow-all&quot;<br>    min_ttl                = 0<br>    default_ttl            = 3600<br>    max_ttl                = 86400<br>  }# Cache behavior with precedence 0<br>  ordered_cache_behavior {<br>    path_pattern     = &quot;/content/immutable/*&quot;<br>    allowed_methods  = [&quot;GET&quot;, &quot;HEAD&quot;, &quot;OPTIONS&quot;]<br>    cached_methods   = [&quot;GET&quot;, &quot;HEAD&quot;, &quot;OPTIONS&quot;]<br>    target_origin_id = &quot;my_first_origin&quot;forwarded_values {<br>      query_string = false<br>      headers      = [&quot;Origin&quot;]cookies {<br>        forward = &quot;none&quot;<br>      }<br>    }min_ttl                = 0<br>    default_ttl            = 86400<br>    max_ttl                = 31536000<br>    compress               = true<br>    viewer_protocol_policy = &quot;allow-all&quot;<br>  }# Cache behavior with precedence 1<br>  ordered_cache_behavior {<br>    path_pattern     = &quot;/content/*&quot;<br>    allowed_methods  = [&quot;GET&quot;, &quot;HEAD&quot;, &quot;OPTIONS&quot;]<br>    cached_methods   = [&quot;GET&quot;, &quot;HEAD&quot;]<br>    target_origin_id  = &quot;my_first_origin&quot;forwarded_values {<br>      query_string = falsecookies {<br>        forward = &quot;none&quot;<br>      }<br>    }min_ttl                = 0<br>    default_ttl            = 3600<br>    max_ttl                = 86400<br>    compress             = true<br>    viewer_protocol_policy = &quot;redirect-to-https&quot;<br>  }price_class = &quot;PriceClass_200&quot;restrictions {<br>    geo_restriction {<br>      restriction_type = &quot;whitelist&quot;<br>      locations        = [&quot;US&quot;, &quot;CA&quot;, &quot;GB&quot;, &quot;IN&quot;]<br>    }<br>  }tags = {<br>    Environment = &quot;production&quot;<br>  }viewer_certificate {<br>    cloudfront_default_certificate = true<br>  }</pre><p><strong>Updating CloudFront URL in our web server</strong></p><p>Our web server is hosted on EC2. So I updated this URL to our main code.</p><pre>connection {<br>        type    = &quot;ssh&quot;<br>        user    = &quot;ec2-user&quot;<br>        private_key = file(&quot;C:/Users/asus/Desktop/d.pem&quot;)<br>     host     = aws_instance.web.public_ip<br>    }<br>provisioner &quot;remote-exec&quot; {<br>        inline  = [<br>            &quot;sudo su &lt;&lt; EOF&quot;,<br>            &quot;echo \&quot;&lt;center&gt;&lt;img src=&#39;<a href="https://medium.com/$%7Bself.domain_name%7D/$%7Baws_s3_bucket_object.bucket1.key%7D&#39;">http://${self.domain_name}/${aws_s3_bucket_object.bucket1.key}&#39;</a> height=&#39;400px&#39; width=&#39;400px&#39;&gt;&lt;/center&gt;\&quot; &gt;&gt; /var/www/html/index.php&quot;,<br>            &quot;EOF&quot;<br>        ]<br>    }}</pre><p><strong>Running the code</strong></p><p>Save this file with .tf extension as this a terraform file. Now we will run this code on Command Prompt.</p><pre>terraform init<br>terraform apply -auto-approve</pre><p>Now whole of the setup will be created using this single file.</p><p>We can delete the whole setup created using a single command.</p><pre>terraform destroy</pre><p><strong><em>Snaps of the AWS services created by Terraform</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ArXDbT6tDB1O0vdu.png" /><figcaption>EC2</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*h9bQFxc_WpBVnQDv.png" /><figcaption>S3 bucket</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*0g19qHj-h9DP7Xiv.png" /><figcaption>Security Groups</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*99dc59VSD9IPfI_o.png" /><figcaption>CloudFront Distribution</figcaption></figure><p>This above task automates the process of setting up the web server automatically along with setting up CloudFront distribution. Any additional views and improvements are welcome</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=837296e11987" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Job Creation Using Jenkinsfile Approach]]></title>
            <link>https://medium.com/@ishansharma_97521/job-creation-using-jenkinsfile-approach-2c1fe1cbc778?source=rss-ef523eeeac44------2</link>
            <guid isPermaLink="false">https://medium.com/p/2c1fe1cbc778</guid>
            <dc:creator><![CDATA[Ishan Sharma]]></dc:creator>
            <pubDate>Thu, 23 Jul 2020 10:09:41 GMT</pubDate>
            <atom:updated>2020-07-23T12:07:12.022Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*x0Eb1pJ3v95XOe9sumWUhg.png" /></figure><p>Perform third task with the help of Jenkins coding file ( called as jenkinsfile approach ) and perform the with following phases:</p><p>1. Create container image that’s has Jenkins installed using dockerfile Or You can use the Jenkins Server on RHEL 8/7</p><p>2. When we launch this image, it should automatically starts Jenkins service in the container.</p><p>3. Create a job chain of job1, job2, job3 and job4 using build pipeline plugin in Jenkins</p><p>4. Job2 ( Seed Job ) : Pull the Github repo automatically when some developers push repo to Github.</p><p>5. Further on jobs should be pipeline using written code using Groovy language by the developer</p><p>6. Job1 :</p><p>1. By looking at the code or program file, Jenkins should automatically start the respective language interpreter installed image container to deploy code on top of Kubernetes ( eg. If code is of PHP, then Jenkins should start the container that has PHP already installed )</p><p>2. Expose your pod so that testing team could perform the testing on the pod</p><p>3. Make the data to remain persistent using PVC ( If server collects some data like logs, other user information )</p><p>7. Job3 : Test your app if it is working or not.</p><p>8. Job4 : if app is not working , then send email to developer with error messages and redeploy the application after code is being edited by the developer</p><p><strong>SOLUTION :-</strong></p><p><strong>TO ENABLE GROOVY</strong></p><ol><li>Go to Manage Jenkins</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/60/0*f7NK_QMXSQ9MpmHT" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/678/0*VPSCvtTefqty_V2i.png" /></figure><p>2. Go to Global Security Settings and Change Settings</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/60/0*n0q3Cf_m9KLhwOyS" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/670/0*hbNDjH2BTA019EQp.png" /></figure><p>Now we will create a seed_job as per the name it will work as seed for other jobs and create the other jobs . We will use Git as SCM we will upload our code in Github and then we will then Jenkins will fetch code from there and create jobs</p><pre>job(“task6_DSL_job1”)</pre><pre>{<br>description(“this job will copy the file in you os version and push image to docker hub”<br>)</pre><pre>scm </pre><pre>{<br>github(‘https://github.com/ishan/devops.git&#39; , ‘master’)<br>}</pre><pre>triggers </pre><pre>{<br>scm(“* * * * *”)<br>}</pre><pre>steps </pre><pre>{</pre><pre>shell(‘’’</pre><pre>sudo cp -rvf * /root/webpages/</pre><pre>if sudo ls /root/webpages | sudo grep .html</pre><pre>then</pre><pre>sudo cp -rvf /root/webpages/*.html /root/devops_task6/html/pages/*</pre><pre>sudo docker build -t ishansharma/html_os_1:v1 /root/devops_task6/html/.</pre><pre>sudo docker push ishansharma/html_os_1:v1</pre><pre>else</pre><pre>sudo cp -rvf /root/webpages/*.php /root/devops_task6/php/pages/*</pre><pre>sudo docker build -t ishansharma/php_os_1:v1 /root/devops_task6/php/.</pre><pre>sudo docker push ishansharma/php_os_1:v1<br>‘’’)</pre><pre>}</pre><pre>}</pre><pre>job(“task6_DSL_job2”) </pre><pre>{<br>triggers </pre><pre>{<br>upstream(‘task6_DSL_job1’, ‘SUCCESS’)<br>}</pre><pre>job(“task6_DSL_job3”) </pre><pre>{<br>triggers </pre><pre>{<br>upstream(‘task6_DSL_job2’, ‘SUCCESS’)<br>}</pre><pre>steps </pre><pre>{<br>shell(‘’’</pre><pre>if </pre><pre>sudo kubectl get deploy | grep html_os</pre><pre>then</pre><pre>pass</pre><pre>else</pre><pre>sudo kubectl create -f /root/devops_task6/task6_html.yaml</pre><pre>fi</pre><pre>if </pre><pre>sudo kubectl get deploy | grep php_os</pre><pre>then</pre><pre>pass</pre><pre>else</pre><pre>sudo kubectl create -f /root/devops_task6/task6_php.yaml</pre><pre>fi <br>‘’’)</pre><pre>}</pre><pre>}</pre><pre>job(“task6_DSL_job4”) </pre><pre>{</pre><pre>description (“It will test if pod is running else send a mail”)</pre><pre>triggers </pre><pre>{<br>upstream(‘task6_DSL_job2’, ‘SUCCESS’)}steps </pre><pre>{</pre><pre>shell(‘’’</pre><pre>if </pre><pre>sudo kubectl get deployment | grep html_os</pre><pre>then</pre><pre>echo “send html_os to production”</pre><pre>else</pre><pre>echo “sending html_os back to developer”</pre><pre>exit 1<br>fi</pre><pre>if </pre><pre>sudo kubectl get deployment | grep php_os</pre><pre>then</pre><pre>echo “send php_os to production”</pre><pre>else</pre><pre>echo “sending php_os back to developer”</pre><pre>exit 1</pre><pre>fi‘’’)</pre><pre>}</pre><pre>}</pre><pre>buildPipelineView(“task6_DSL_BPV”) </pre><pre>{</pre><pre>filterBuildQueue(true)<br>filterExecutors(false)<br>title(‘task6_DSL_BPV’)<br>displayedBuilds(1)<br>selectedJob(‘task6_DSL_job1’)<br>alwaysAllowManualTrigger(false)<br>showPipelineParameters(true)<br>refreshFrequency(1)}}</pre><p>This is the code will create all the jobs , and one build pipeline view</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/646/0*jxX6VyIqNfO0rQ50.png" /></figure><p>Now all the things will as done automatically as we have given all the info in the job creation and we have other codes also</p><p><strong>Code for html deployment </strong>:</p><pre>apiVersion: v1<br>kind: Service<br>metadata:<br>name: html_os<br>labels:<br>app: html_os<br>spec:<br>ports:<br>- port: 80<br>selector:<br>app: html_os<br>type: LoadBalancer<br>apiVersion: v1<br>kind: PersistentVolumeClaim<br>metadata:<br>name: html-pv-claim<br>labels:<br>app: html_pvc<br>spec:<br>accessModes:<br>- ReadWriteOnce<br>resources:<br>requests:<br>storage: 20Gi<br> — -<br>apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2<br>kind: Deployment<br>metadata:<br>name: html_os<br>labels:<br>app: html_os<br>spec:<br>selector:<br>matchLabels:<br>app: html_os<br>strategy:<br>type: Recreate<br>template:<br>metadata:<br>labels:<br>app: html_os<br>spec:<br>containers:<br>- image: ishansharma/html_os_1:v1<br>name: html_os<br>ports:<br>- containerPort: 80<br>name: html_os<br>volumeMounts:<br>- name: html-persistent-storage<br>mountPath: /var/www/html<br>volumes:<br>- name: html-persistent-storage<br>persistentVolumeClaim:<br>claimName: html-pv-claim</pre><p><strong>Code for PHP deployment</strong> :</p><pre>apiVersion: v1<br>kind: Service<br>metadata:<br>name: php_os<br>labels:<br>app: php_os<br>spec:<br>ports:<br>- port: 80<br>selector:<br>app: php_os<br>type: LoadBalancer <br>— -<br>apiVersion: v1<br>kind: PersistentVolumeClaim<br>metadata:<br>name: php-pv-claim<br>labels:<br>app: php_pvc<br>spec:<br>accessModes:<br>- ReadWriteOnce<br>resources:<br>requests:<br>storage: 20Gi<br>apiVersion: apps/v1 <br>kind: Deployment<br>metadata:<br>name: php_os<br>labels:<br>app: php_os<br>spec:<br>selector: match<br>Labels:<br>app: php_os<br>strategy:<br>type: Recreate<br>template:<br>metadata:<br>labels:<br>app: php_os<br>spec:<br>containers:<br>- image: ishansharma/php_os_1:v1<br>name: php_os<br>ports:<br>- containerPort: 80<br>name: html_os<br>volumeMounts:<br>- name: php-persistent-storage<br>mountPath: /var/www/html<br>volumes:<br>- name: php-persistent-storage<br>persistentVolumeClaim:<br>claimName: php-pv-claim</pre><p><strong>Console Output Seed Job</strong> :</p><p>Now these pods will be launched and we can connect to them and see web pages .</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2c1fe1cbc778" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Prometheus and Grafana with Persistent Storage]]></title>
            <link>https://medium.com/@ishansharma_97521/prometheus-and-grafana-with-persistent-storage-6bdff75a0efa?source=rss-ef523eeeac44------2</link>
            <guid isPermaLink="false">https://medium.com/p/6bdff75a0efa</guid>
            <dc:creator><![CDATA[Ishan Sharma]]></dc:creator>
            <pubDate>Thu, 23 Jul 2020 08:22:09 GMT</pubDate>
            <atom:updated>2020-07-23T10:20:00.878Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eQ9WXI5w9S4eDqkYoKsBwA.jpeg" /></figure><p>Integrate Prometheus and Grafana and perform in following way:</p><ol><li>Deploy them as pods on top of Kubernetes by creating resources Deployment, ReplicaSet, Pods or Services</li></ol><p>2. And make their data to be remain persistent</p><p>3. And both of them should be exposed to outside world</p><p>Integrating Prometheus and Grafana with the container management tool Kubernetes. This makes the Prometheus and Grafana containers easily manageable. Prometheus needs to be provided with some node exporter software so that Prometheus can collect the metrics whenever required from this exporter software installed inside various operating systems. Run the below mentioned commands to download and start the node exporter software inside the operating system.</p><pre>wget https://github.com/prometheus/node_exporter/releases/download/v1.0.0/node_exporter-1.0.0.rc.1.linux-amd64.tar.gz</pre><pre>tar xvfz node_exporter-1.0.0.rc.1.linux-amd64.tar.gz</pre><pre>cd node_exporter-1.0.0.rc.1.linux-amd64</pre><pre>./node_exporter</pre><p>The downloaded files of the Prometheus are copied into a separate folder so that they can be edited or managed accordingly without any error. For creating the containers of Prometheus , create a docker image using Dockerfile so that whenever the container has been launched , Prometheus services launch inside the container Use the following commands to create the Prometheus image.</p><pre>ROM centos</pre><pre>RUN yum install wget tar -y</pre><pre>RUN wget https://github.com/prometheus/prometheus/releases/download/v2.19.2/prometheus-2.19.2.linux-amd64.tar.gz</pre><pre>RUN mkdir /yashprom</pre><pre>RUN tar -xzf prometheus-2.19.2.linux-amd64.tar.gz</pre><pre>RUN cp -rf  prometheus-2.19.2.linux-amd64/* /ishan</pre><pre>RUN cp /ishan/prometheus /usr/bin</pre><pre>CMD prometheus --config.file=/ishan/prometheus.yml</pre><p>Use the commands to build the image and push to the docker hub if required.</p><pre>docker build -t (name of image) (path of Dockerfile)<br>docker push (image name)</pre><p>Similarly create the Grafana image following the below mentioned commands:</p><pre>FROM centos</pre><pre>RUN yum install wget -y</pre><pre>RUN wget https://dl.grafana.com/oss/release/grafana-7.0.6-1.x86_64.rpm</pre><pre>RUN yum install grafana-7.0.6-1.x86_64.rpm -y</pre><pre>WORKDIR /usr/share/grafana</pre><pre>ENTRYPOINT [&quot;/usr/sbin/grafana-server&quot;, &quot;--config=/etc/grafana/grafana.ini&quot;, &quot;cfg:default.paths.logs=/var/log/grafana&quot;, &quot;cfg:default.paths.data=/var/lib/grafana&quot;, &quot;LimitNOFILE=10000&quot;]</pre><pre>CMD prometheus --config.file=/ishan/prometheus.yml</pre><p>It is necessary to provide the details of nodes that are to be managed inside the configuration file of the Prometheus. Here the role of ConfigMap resource of the Kubernetes becomes useful. The details of the nodes that are to be monitored are mentioned inside this ConfigMap file as given below: ( Here I took 2 nodes with so mentioned IP addresses with the node exporter services running inside the nodes)</p><pre>kind: ConfigMap</pre><pre>apiVersion: v1</pre><pre>metadata:</pre><pre>name: prometheus-nodes</pre><pre>data:</pre><pre>prometheus.yml: |</pre><pre>global:</pre><pre>scrape_interval: 15s</pre><pre>evaluation_interval: 15s</pre><pre>alerting:</pre><pre>alertmanagers:</pre><pre>- static_configs:</pre><pre>- targets:</pre><pre>rule_files:</pre><pre>scrape_configs:</pre><pre>- job_name: &#39;rhelnode&#39;</pre><pre>static_configs:</pre><pre>- targets: [&#39;192.168.225.80:9100&#39;]</pre><pre>- job_name: &#39;clinode&#39;</pre><pre>static_configs:</pre><pre>- targets: [&#39;192.168.225.124:9100&#39;]</pre><p>Now it is up to creating the Kubernetes resources for the Prometheus and Grafana ;</p><p>It is required to provide persistent storage to the Prometheus pod so as to keep the data permanent. Also it is mandatory to attach the ConfigMap resource to this pod. Finally the deployment service is provided to the pod so that there is no tension in the downtime or unexpected deletion of the pods. This deployment is exposed so that it can be accessed by the admins. This whole setup can be created by using a single YAML file.</p><pre>apiVersion: v1</pre><pre>kind: Service</pre><pre>metadata:</pre><pre>name: prometheus</pre><pre>spec:</pre><pre>selector:</pre><pre>env: deploy</pre><pre>type: NodePort</pre><pre>ports:</pre><pre>- port: 9090</pre><pre>targetPort: 9090</pre><pre>---</pre><pre>apiVersion: v1</pre><pre>kind: PersistentVolumeClaim</pre><pre>metadata:</pre><pre>name: prometheus-pvc</pre><pre>spec:</pre><pre>accessModes:</pre><pre>- ReadWriteOnce</pre><pre>resources:</pre><pre>requests:</pre><pre>storage: 2Gi</pre><pre>---</pre><pre>apiVersion: apps/v1</pre><pre>kind: Deployment</pre><pre>metadata:</pre><pre>name: prometheus</pre><pre>labels:</pre><pre>env: deploy</pre><pre>spec:</pre><pre>replicas: 1</pre><pre>selector:</pre><pre>matchLabels:</pre><pre>env: deploy</pre><pre>template:</pre><pre>metadata:</pre><pre>name: prometheus-pod</pre><pre>labels:</pre><pre>env: deploy</pre><pre>spec:</pre><pre>containers:</pre><pre>- name: promcont1</pre><pre>image: ishan/prometheus:x2</pre><pre>volumeMounts:</pre><pre>- name: logs</pre><pre>mountPath: /yashprom</pre><pre>volumeMounts:</pre><pre>- name: ymlnodes</pre><pre>mountPath: /ishan/prometheus.yml</pre><pre>subPath: prometheus.yml</pre><pre>volumes:</pre><pre>- name: logs</pre><pre>persistentVolumeClaim:</pre><pre>claimName: prometheus-pvc</pre><pre>- name: ymlnodes</pre><pre>configMap:</pre><pre>name: prometheus-nodes</pre><pre>defaultMode: 0744</pre><p>YAML file for Grafana. Provide the PVC to make the data persistent and expose the deployment to make the pod accessible and there is no room for downtime or unexpected failures.</p><pre>apiVersion: v1</pre><pre>kind: Service</pre><pre>metadata:</pre><pre>name: grafana</pre><pre>spec:</pre><pre>selector:</pre><pre>env: deploy</pre><pre>type: NodePort</pre><pre>ports:</pre><pre>- port: 3000</pre><pre>targetPort: 3000</pre><pre>---</pre><pre>apiVersion: v1</pre><pre>kind: PersistentVolumeClaim</pre><pre>metadata:</pre><pre>name: grafana-pvc</pre><pre>spec:</pre><pre>accessModes:</pre><pre>- ReadWriteOnce</pre><pre>resources:</pre><pre>requests:</pre><pre>storage: 2Gi</pre><pre>---</pre><pre>apiVersion: apps/v1</pre><pre>kind: Deployment</pre><pre>metadata:</pre><pre>name: grafana</pre><pre>spec:</pre><pre>replicas: 1</pre><pre>selector:</pre><pre>matchLabels:</pre><pre>env: deploy</pre><pre>template:</pre><pre>metadata:</pre><pre>name: grafana-pod</pre><pre>labels:</pre><pre>env: deploy</pre><pre>spec:</pre><pre>containers:</pre><pre>- name: g1</pre><pre>image: ishan/grafana:x1</pre><pre>volumeMounts:</pre><pre>- name: data</pre><pre>mountPath: /var/lib/grafana</pre><pre>volumes:</pre><pre>- name: data</pre><pre>persistentVolumeClaim:</pre><pre>claimName: grafana-pvc</pre><p>Creating using these YAML files-</p><pre>kubectl create -f [YAML file name]</pre><p>With the whole setup deployed on top of the private Kubernetes cluster , now its the time for testing</p><p>The configured two nodes are up and data is successfully retrieved by the Prometheus. Even after the pod is deleted , since it is a deployment service of the Prometheus the deleted pods are automatically launched by the K8s with the data remaining persistent , older data can be accessed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/778/0*Wj7xjy53Gt6CnaBi.png" /></figure><p>Moving to Grafana , initially creating a dashboard inside with a simple prom query</p><p>Now onto prometheus, enter url given by k8s.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/764/0*pcR1R1nb690OmfjO.png" /></figure><p>Moving on to graphana and adding data source.</p><p>Images are explaining the steps.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/727/0*WZZtE1zagDag11WQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/633/0*511mdGLkJ6QNNETM.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/661/0*g64By21iwBdMj9Jj.png" /><figcaption>Creating a basic graph in Grafana</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/687/0*Zmv7omCDxc_TyRJK.png" /></figure><p>Testing whether the data is persistent or not</p><p>Deleting the grafana pod using following command-</p><pre>kubectl delete pods grafana-7b7dc95846</pre><p>As the service used is deployment , the deleted pod is again up.</p><p>Even after deleting the pod, the data is persistent and hence the practical is successful.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6bdff75a0efa" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Dynamic Jenkins Cluster]]></title>
            <link>https://medium.com/@ishansharma_97521/dynamic-jenkins-cluster-a1fbea7018b4?source=rss-ef523eeeac44------2</link>
            <guid isPermaLink="false">https://medium.com/p/a1fbea7018b4</guid>
            <dc:creator><![CDATA[Ishan Sharma]]></dc:creator>
            <pubDate>Thu, 23 Jul 2020 08:11:38 GMT</pubDate>
            <atom:updated>2020-07-23T08:11:38.394Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/750/1*E0yMoRnmr-Si1Z-9YaQn5Q.png" /></figure><p><strong>OBJECTIVE:-</strong></p><p>Create A dynamic Jenkins cluster and perform task-3 using the dynamic Jenkins cluster.</p><p>Steps to proceed as:</p><p>1. Create container image that’s has Linux and other basic configuration required to run Slave for Jenkins. ( example here we require kubectl to be configured )</p><p>2. When we launch the job it should automatically starts job on slave based on the label provided for dynamic approach.</p><p>3. Create a job chain of job1 &amp; job2 using build pipeline plugin in Jenkins</p><p>4. Job1 : Pull the Github repo automatically when some developers push repo to Github and perform the following operations as:</p><p>&lt;1. Create the new image dynamically for the application and copy the application code into that corresponding docker image</p><p>&lt; 2. Push that image to the docker hub (Public repository)</p><p>( Github code contain the application code and Dockerfile to create a new image )</p><p>5. Job2 ( Should be run on the dynamic slave of Jenkins configured with Kubernetes kubectl command): Launch the application on the top of Kubernetes cluster performing following operations:</p><p>&lt;1. If launching first time then create a deployment of the pod using the image created in the previous job. Else if deployment already exists then do rollout of the existing pod making zero downtime for the user.</p><p>&lt;2. If Application created first time, then Expose the application. Else don’t expose it.</p><p><strong>SOLUTION:-</strong></p><p>We need kubectl configured and we make use of dockerfile.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/648/0*9zW1uapuO_NTHizh.png" /></figure><p>Now we can start creating jobs , Job1 is to create a dynamic image as the code is ; if the code is HTML then a html image should be created if the code is PHP so the image with php-interpreter should launch but i have these images created already i only need the code if you want to create a full image then create using Dockerfile and any base OS.</p><p>I have two images “httpd” and ‘apache-webserver-php” which areused as base images to create images for deployment pods.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/30/0*TwjAKr2MW2pZcmQm" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/795/0*EEigYas3D4TcMcqp.png" /></figure><p>As this image is created it will be pushed to the dockerhub automatically from job-1 . When this job completes, we will have either php or html, depending on the developer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/0*vPVStw7ec_3t0-Ej.png" /></figure><p>Job2 will run on the slave node and it will launch the pods in the K8S cluster , this job will be triggered by job1. This job will use the old images and it will deploy pods and expose them .</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/0*ldT3YpyabXUHGxE2.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a1fbea7018b4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Automated Testing in Devops using Kubernetes]]></title>
            <link>https://medium.com/@ishansharma_97521/automated-testing-in-devops-using-kubernetes-e67f325cb704?source=rss-ef523eeeac44------2</link>
            <guid isPermaLink="false">https://medium.com/p/e67f325cb704</guid>
            <dc:creator><![CDATA[Ishan Sharma]]></dc:creator>
            <pubDate>Thu, 23 Jul 2020 08:02:16 GMT</pubDate>
            <atom:updated>2020-07-23T10:28:55.122Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/638/1*ZuB9WaI2qTj8l1m4VyTwEg.jpeg" /></figure><p><strong>OBJECTIVE:-</strong></p><p>Perform second task on top of Kubernetes where we use Kubernetes resources like Pods, ReplicaSet, Deployment, PVC and Service.</p><p>1. Create container image that’s has Jenkins installed using dockerfile Or You can use the Jenkins Server on RHEL 8/7</p><p>2. When we launch this image, it should automatically starts Jenkins service in the container.</p><p>3. Create a job chain of job1, job2, job3 and job4 using build pipeline plugin in Jenkins</p><p>4. Job1 : Pull the Github repo automatically when some developers push repo to Github.</p><p>5. Job2 :</p><p>1. By looking at the code or program file, Jenkins should automatically start the respective language interpreter installed image container to deploy code on top of Kubernetes ( eg. If code is of PHP, then Jenkins should start the container that has PHP already installed )</p><p>2. Expose your pod so that testing team could perform the testing on the pod</p><p>3. Make the data to remain persistent ( If server collects some data like logs, other user information )</p><p>6. Job3 : Test your app if it is working or not.</p><p>7. Job4 : if app is not working , then send email to developer with error messages and redeploy the application after code is being edited by the developer</p><p><strong>SOLUTION:</strong>-</p><p>We need the kubectl rpm or command to be installed. We need files or password and authentication. A configuration file of the Kubernetes which will configure the kubectl.</p><p>First we need the command or the executable file so that we need that for that we can use curl command</p><pre>curl -LO <a href="https://storage.googleapis.com/kubernetes-release/release/%60curl">https://storage.googleapis.com/kubernetes-release/release/`curl</a> -s <a href="https://storage.googleapis.com/kubernetes-release/release/stable.txt%60/bin/linux/amd64/kubectl">https://storage.googleapis.com/kubernetes-release/release/stable.txt`/bin/linux/amd64/kubectl</a></pre><p>This command will give you the executable file of the kubectl this command will make changes or execute in OS so we have to give it the executable permission.</p><pre>chmod +x ./kubectl</pre><p>But as we know that all of the commands in a RHEL8 OS is present in a specific folder so have to copy this file to that folder.</p><pre>FOR RHEL8 OS :-sudo mv ./kubectl /usr/bin/kubectlFOR Ubantu OS :-sudo mv ./kubectl /usr/local/bin/kubectl</pre><p>Now we can access the kubectl command from any folder but it will give error that no config so first we have to configure it. For this we have to make a config file but we will need some files first.I have my K8S cluster in my windows so i have these three files at these locations you can use them as it is for your system.</p><pre>File Name:client.key Path:C:\Users\ishan\.minikube\profiles\minikubeFile Name:client.crt Path:C:\Users\ishan\.minikube\profiles\minikubeFile Name:ca.crt Path:C:\Users\ishan\.minikube\ca.crt</pre><p>Now we need a config file that will be used by kubectl to connect to our K8S cluster we will make a folder from name “.kube” and we will paste this file in that folder.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/576/0*GuaIM_Pf5_0G5pV3.png" /></figure><p>Now our kubectl command is configured and we can use it in RHEL8 to access our cluster.</p><p>Now our task begins as we have done the similar task in task2 so we will not discuss it so deeply. So now we will just start jenkins and start building jobs.</p><p>Job1 will have simplest work to get code from the git hub it will download the code and store in a folder.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/0*vFneVdH80pkdTvfC.png" /></figure><p>Now job2 we will launch pods we will launch them and we have to launch pod in respective pod so that they have perfect interpreter.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/678/0*YHPKB5qisusQHx9q.png" /></figure><p>We can deploy our code like this in this the reader.py file will give output as html then we deploy a pod from httpd image and if the reader.py gives php then we will use a image which has php installed.I have created 3 replicas you can create according to your requirement.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/685/0*s4zj6dcQ7eamo9bG.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/674/0*I36rj_77JXIfb0V0.png" /></figure><p>Job3 will have a remote which will be given to QAT team so they can test our code and redeploy of there is any problem i have written some simple code just to make sure pod is running.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e67f325cb704" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Facial Recognition Using Transfer Learning]]></title>
            <link>https://medium.com/@ishansharma_97521/facial-recognition-using-transfer-learning-bf556d8c6ac5?source=rss-ef523eeeac44------2</link>
            <guid isPermaLink="false">https://medium.com/p/bf556d8c6ac5</guid>
            <dc:creator><![CDATA[Ishan Sharma]]></dc:creator>
            <pubDate>Mon, 13 Jul 2020 02:07:31 GMT</pubDate>
            <atom:updated>2020-07-13T02:13:31.858Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/825/1*QyJo3Gx5T5xFd5BbiUYAxA.jpeg" /></figure><h3>OBJECTIVE</h3><p>Create a project using transfer learning solving various problems like Face Recognition, Image Classification, using existing Deep Learning models like VGG16, VGG19, ResNet, etc.</p><p><strong>Fine Tuning</strong></p><p>We can give the new dataset to fine tune the pre-trained CNN model. Here the new dataset (for ex: human face) is almost similar to the original dataset (from the images on imagenet) used for pre-training. Since the new dataset is similar, the same weights can be used for extracting the features from the new dataset.</p><p>1. If the new dataset is very small, it’s better to train only the final layers of the network to avoid over-fitting, keeping all other layers fixed. So remove the final layers of the pre-trained network, add new layers and retrain only the new layers.(This helps to those who doesn’t have high-end computation resources and want to consume less time for better results.)</p><p>2. If the new dataset is very much large, we can retrain the whole network with initial weights from the pre trained model.</p><p>So in this article I will explain how to create a face recognition model using Transfer Learning with very limited amount of dataset. I have used MobileNet model and then tuning the last layers to make my face recognition model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/515/0*1gB5hZcJ3tr1Ds_V.png" /></figure><p>Above is the overall architecture of the MobileNet, having 30 layers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/0*NgEt_n9QorpMtqQ9.png" /></figure><p>As in the above image you can see that I used keras.applications with tensorflow as the backend, importing MobileNet model and weights from <a href="http://www.image-net.org/"><strong>imagenet</strong></a>. So in this code I freeze all the layers so that I won’t have to train all the layers again (preserving weights) and just fine tuning the last few can help me to classify my own datasets. It saves time and hardware resources to train.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/0*DibNKKx2VAv0CR6x.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/0*zXHJMsPhMlLpTMKB.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/0*j-tELu-XjV20DgUm.png" /></figure><p>Data augmentation and categorizing them into train and validation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/0*_4PXtkecJApF_1UW.png" /></figure><p>In the above image I made checkpoint and earlystopping code and also training the model with saving the minimum val_loss.</p><p>In 7th epoch I got the best result i.e 91% accuracy.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/0*UW2ZvvMCixm3SeGG.png" /></figure><p>Here the code is predicting using cv2 and labelling it using own style.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/0*OvKBxAHrXPhYb8SD.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/0*yWY37RgPV4Z8-Ajp.png" /></figure><p>Below is the few results of my code in predicting the face. It is me and my friend.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/595/0*bmtxJW1hABlTQmfo.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/594/0*iy6t8PSSKHsGRvxS.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/558/0*auvei8E3jwYyXfyR.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/593/0*b-jrNVaqcW2UHXki.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bf556d8c6ac5" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>