<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Dominik Zygarski on Medium]]></title>
        <description><![CDATA[Stories by Dominik Zygarski on Medium]]></description>
        <link>https://medium.com/@dominikzygarski_88070?source=rss-b563547878e3------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*tGS7RGQqfXdOMBAO2ii-Fw@2x.jpeg</url>
            <title>Stories by Dominik Zygarski on Medium</title>
            <link>https://medium.com/@dominikzygarski_88070?source=rss-b563547878e3------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 08:40:38 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@dominikzygarski_88070/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Project: Kubernetes for Modern Data Engineering.]]></title>
            <link>https://medium.com/@dominikzygarski_88070/project-kubernetes-for-modern-data-engineering-7e1d6b49e56a?source=rss-b563547878e3------2</link>
            <guid isPermaLink="false">https://medium.com/p/7e1d6b49e56a</guid>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[apache]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Dominik Zygarski]]></dc:creator>
            <pubDate>Sun, 18 May 2025 22:20:57 GMT</pubDate>
            <atom:updated>2025-06-25T10:54:15.247Z</atom:updated>
            <content:encoded><![CDATA[<h3>🔧 Environment Setup</h3><p>Before diving into the core of the project, we need to prepare the local environment where our Kubernetes cluster and related services will run. This project is built on a local setup using <strong>Docker Desktop</strong>, which provides integrated <strong>Kubernetes</strong> support. This makes it possible to work with a real Kubernetes cluster without relying on cloud providers.</p><h4>kubectl</h4><p>kubectl is the official command-line tool for interacting with Kubernetes clusters. It&#39;s included with Docker Desktop, but you can also install it manually:</p><p>brew install kubectl</p><h4>Helm</h4><p>Helm is the package manager for Kubernetes, similar to apt or brew, but designed for Kubernetes resources. It simplifies the installation of tools like Airflow, Prometheus, or PostgreSQL.</p><p>brew install helm</p><h3>Docker Desktop</h3><p>Docker Desktop allows you to run containers locally and includes optional support for Kubernetes. It is available for macOS and Windows, and includes Docker Engine, Docker CLI, Docker Compose, and a built-in Kubernetes cluster.</p><h4>Enable Kubernetes:</h4><p>Once Docker Desktop is installed and running:</p><ol><li>Open the Docker Desktop settings/preferences.</li><li>Go to the <strong>Kubernetes</strong> tab.</li><li>Enable <strong>“Enable Kubernetes”</strong>.</li><li>Click <strong>“Apply &amp; Restart”</strong>.</li></ol><p>Docker will take a few minutes to initialize the local Kubernetes cluster. Once it’s ready, you can confirm it by running in your terminal:</p><p>kubectl cluster-info</p><p>If the Kubernetes control plane URL is displayed, everything is working correctly.</p><h4>Verifying Kubernetes Cluster and Running Pods</h4><p>After enabling Kubernetes in Docker Desktop and confirming that it’s running, it’s a good idea to verify that your cluster is actually up and operational.</p><p>To see which context is currently active (useful when managing multiple clusters), run:</p><p>kubectl config get-contexts</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NmmSZTCkQQ4wg-YSW1AorA.png" /></figure><h3>Accessing the Kubernetes Dashboard locally</h3><p>Once my cluster was up and running, I wanted to check if the Kubernetes Dashboard was accessible. Since I’m using Docker Desktop with Kubernetes enabled, the quickest way to open the Dashboard locally was by running:</p><p>kubectl proxy</p><p>This starts a local server at http://127.0.0.1:8001, which allows secure access to internal cluster services including the Dashboard, without exposing anything publicly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9i2Pz4P3Ov3Hfd7ZUhouJQ.png" /></figure><p>After starting the proxy, I was able to access the Kubernetes Dashboard through a local address.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Be0xjuNzKyh0ZcQSyTy-TA.png" /></figure><p>When I opened the Dashboard, it asked for a token. I didn’t have one yet, so I created a separate service account with admin permissions and generated the token manually.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rj2mv0DpB9zTPxD3QrHThQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IkDuPjsUUJmv_jHXtGAVqQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kDS1zaFkeMKePH_3GiRYPw.png" /></figure><p>I wrote the YAML files in VS Code, one for the service account, one for the role binding, and one for the secret. Looked fine at first glance, so I ran:</p><pre>kubectl apply -f dashboard-adminuser.yaml</pre><p>And nothing. Just an error about no matches for kind &quot;ServiceAccount&quot;.</p><p>Turns out I made a typo or messed up the syntax. Took me a couple of minutes to spot it and fix the structure.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IkDuPjsUUJmv_jHXtGAVqQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Y336xWeA5Vimtx6C7-U_-w.png" /></figure><p>and</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rj2mv0DpB9zTPxD3QrHThQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QVPNiHNwA0DVSPVLwJC7Kw.png" /></figure><p>Once I fixed the YAML, everything applied without issues. I ran the files one by one:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Zae42MzIFPa4pG2Lik0idA.png" /></figure><p>Once the secret was created, I pulled the token using:</p><pre>kubectl get secret admin-user -n kubernetes-dashboard -o jsonpath={&quot;.data.token&quot;} | base64 -d</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QSMBgg6vKjmCdcwz6mN79Q.png" /></figure><p>Opened the Dashboard, dropped in the token, and that was it. Ready to go.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*paHeC3cBhHoHMW7gYdIS2w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nw0U8AfirIcodwbChYJNQA.png" /></figure><h3><strong>Installing Apache Airflow with Helm</strong></h3><p>To install Apache Airflow, I used Helm. I ran the following command:</p><pre>helm install airflow apache-airflow/airflow --namespace airflow --create-namespace --debug</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nweW4-F0ZjzhZhUPmJ6fXg.png" /></figure><p>Once the installation finished and all the pods were running, I wanted to open the Airflow web interface in my browser. Since this is a local Kubernetes cluster running via Docker Desktop, the easiest way to expose a service is by using kubectl port-forward.</p><p>I ran the following command:</p><pre>kubectl port-forward svc/airflow-webserver 8080:8080 --namespace airflow</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*p_ZAdXf9DG0bOfNBilq78Q.png" /></figure><p>After opening the Airflow UI, I noticed a warning message:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4StILUj_woRyXY40Ln4t4Q.png" /></figure><blockquote>Adding Static Webserver and Fernet Keys</blockquote><p>After exporting the default values to a values.yaml file, I updated it to include two important security settings: a static webserverSecretKey and a fernetKey. These are used by Airflow to sign session cookies and encrypt variables stored in metadata DB.</p><p>I pulled the current Fernet key from the running Kubernetes secret:</p><pre>kubectl get secret --namespace airflow airflow-fernet-key \<br>  -o jsonpath=&quot;{.data.fernet-key}&quot; | base64 --decode</pre><p>Then I updated the values.yaml file:</p><pre>fernetKey: VlpOMnV6bUpEdWpXSnFHVkQxTkhXUGE0ZGlFNGJja00=<br>webserverSecretKey: VlpOMnV6bUpEdWpXSnFHVkQxTkhXUGE0ZGlFNGJja00=<br>executor: &quot;KubernetesExecutor&quot;</pre><p>To apply the changes, I ran:</p><pre>helm upgrade --install airflow apache-airflow/airflow --namespace airflow --create-namespace -f values.yaml</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GRVU_j5vGhJJ63essmhS1Q.png" /></figure><h3>Connecting Apache Airflow DAGs to Kubernetes</h3><p>I started by creating a local directory for my DAGs:</p><pre>mkdir dags<br>touch dags/hello.py</pre><p>Then I installed Apache Airflow locally with pip to get access to modules and test DAG code outside the cluster:</p><pre>pip install apache-airflow</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wLkQUlH1yyPc7pABJuslOw.png" /></figure><p>To test that my DAG integration with Kubernetes was working correctly, I created a minimal Airflow DAG in the dags/ directory:</p><pre>from airflow import DAG<br>from airflow.operators.bash import BashOperator<br>from datetime import datetime, timedelta<br><br>default_args = {<br>    &#39;owner&#39;: &#39;dominikzygarski.com&#39;,<br>    &#39;start_date&#39;: datetime(2025, 5, 18),<br>    &#39;catchup&#39;: False<br>}<br><br>dag = DAG(<br>    &#39;hello_world&#39;,<br>    default_args = default_args,<br>    schedule = timedelta(days=1)<br>)<br><br>t1 = BashOperator(<br>    task_id = &#39;hello_world&#39;,<br>    bash_command = &#39;echo &quot;Hello World&quot;&#39;,<br>    dag = dag<br>)<br><br>t2 = BashOperator(<br>    task_id = &#39;hello_dml&#39;,<br>    bash_command = &#39;echo &quot;Hello Data Mastery Lab&quot;&#39;,<br>    dag = dag<br>)<br><br>t1 &gt;&gt; t2 </pre><p>This DAG runs two simple tasks in sequence, just to confirm that Airflow was picking up custom DAG definitions correctly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*d_AEG7R2BWCM_Zl6Ln-HPA.png" /></figure><p>After writing my first DAG, I noticed that VS Code was highlighting the airflow import in red. That usually means the module isn’t installed or isn’t detected in the current Python environment.</p><p>Apparently, one of Airflow’s dependencies required a Rust compiler, and my pip version wasn’t up to date to handle prebuilt wheels. To fix it, I ran the following commands:</p><pre>pip install --upgrade pip<br>curl --proto &#39;=https&#39; --tlsv1.2 -sSf https://sh.rustup.rs | sh<br>source $HOME/.cargo/env<br>pip install apache-airflow</pre><p>This upgraded pip, installed Rust, and finally let me install Airflow without issues. Once that was done, the import error in VS Code disappeared and I was able to test my DAG logic locally.</p><h3>Loading DAGs from GitHub with Git-Sync</h3><p>To automatically sync my DAGs into Airflow, I added a Git-Sync configuration to my values.yaml file:</p><pre>gitSync:<br>  enabled: true<br>  repo: &quot;https://github.com/ogzyzy/Kubernetes-For-Modern-Data-Engineering.git&quot;<br>  branch: &quot;main&quot;<br>  rev: &quot;HEAD&quot;<br>  depth: 1<br>  maxFailures: 0<br>  subPath: &quot;dags&quot;</pre><p>This tells Airflow to pull only the dags/ folder from my GitHub repo and keep it up to date inside the Kubernetes pod.</p><p>With my values.yaml file ready, I deployed the final version of Airflow using Helm:</p><pre>helm upgrade --install airflow apache-airflow/airflow --namespace airflow --create-namespace -f values.yaml</pre><p>I opened the Airflow UI to check if everything was working correctly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*p3jgYBJbUHs5m_EksYAKeg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*chAjByGJyjNv_oWojpUsSg.png" /></figure><h3>Using PythonOperator to Load and Process External Data</h3><p>Now I created a DAG that fetches a CSV file from a public GitHub repository, processes it using pandas, and prints a summary of total sales per category directly to the Airflow logs.</p><p>The workflow consists of two PythonOperator tasks:</p><ul><li>The first task (get_data) downloads the data and pushes it to XCom as a JSON string.</li><li>The second task (preview_data) pulls the data from XCom, transforms it using pandas, and prints the top 20 categories by total sales.</li></ul><p>Here’s the full DAG definition:</p><pre>from datetime import datetime, timedelta<br><br>from airflow import DAG<br>from airflow.operators.python import PythonOperator<br><br><br>def get_data(**kwargs):<br>    import requests<br>    import pandas as pd<br><br>    url = &#39;https://raw.githubusercontent.com/airscholar/ApacheFlink-SalesAnalytics/main/output/new-output.csv&#39;<br>    response = requests.get(url)<br><br>    if response.status_code == 200:<br>        df = pd.read_csv(url, header=None, names=[&#39;Category&#39;, &#39;Price&#39;, &#39;Quantity&#39;])<br><br>        #convert dataframe to json string from xcom<br>        json_data = df.to_json(orient=&#39;records&#39;)<br><br>        kwargs[&#39;ti&#39;].xcom_push(key=&#39;data&#39;, value=json_data)<br>    else:<br>        raise Exception(f&#39;Failed to get data, HTTP status code: {response.status_code}&#39;)<br><br><br>def preview_data(**kwargs):<br>    import pandas as pd<br>    import json<br><br>    output_data = kwargs[&#39;ti&#39;].xcom_pull(key=&#39;data&#39;, task_ids=&#39;get_data&#39;)<br>    print(output_data)<br>    if output_data:<br>        output_data = json.loads(output_data)<br>    else:<br>        raise ValueError(&#39;No data received from XCom&#39;)<br><br>    # Create Dataframe from JSON data<br>    df = pd.DataFrame(output_data)<br><br>    #Compute total sales<br>    df[&#39;Total&#39;] = df[&#39;Price&#39;] * df[&#39;Quantity&#39;]<br><br>    df = df.groupby(&#39;Category&#39;, as_index=False).agg({&#39;Quantity&#39;: &#39;sum&#39;, &#39;Total&#39;: &#39;sum&#39;})<br><br>    #sort by total sales<br>    df = df.sort_values(by=&#39;Total&#39;, ascending=False)<br><br>    print(df[[&#39;Category&#39;, &#39;Total&#39;]].head(20))<br><br><br>default_args = {<br>    &#39;owner&#39;: &#39;dominikzygarski.com&#39;,<br>    &#39;start_date&#39;: datetime(2025, 5, 18),<br>    &#39;catchup&#39;: False<br>}<br><br>dag = DAG(<br>    &#39;fetch_and_preview&#39;,<br>    default_args = default_args,<br>    schedule=timedelta(days=1)<br>)<br><br>get_data_from_url = PythonOperator(<br>    task_id=&#39;get_data&#39;,<br>    python_callable=get_data,<br>    dag=dag<br>)<br><br>preview_data_from_url = PythonOperator(<br>    task_id =&#39;preview_data&#39;,<br>    python_callable=preview_data,<br>    dag=dag<br>)<br><br>get_data_from_url &gt;&gt; preview_data_from_url</pre><p>After writing the DAG, I committed the changes and pushed everything to GitHub.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*j3MeSxhhPb3Pp6D6zSPq7A.png" /></figure><p>After pushing the updated DAG to GitHub, it was automatically pulled into the running Airflow deployment via gitSync.</p><p>Now I manually triggered the fetch_and_preview DAG from the Airflow UI. Both tasks: get_data and preview_data completed successfully.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*seocPOZzHanAl_jq-Epa0Q.png" /></figure><p>At this point, I noticed that the task logs in the Airflow UI were not displaying. Instead, an error related to HTTPConnectionPool and NameResolutionError was shown. To fix this, I enabled persistent logging and configured it to use my cluster’s default hostPath storage class.</p><p>I updated my values.yaml file with the following configuration:</p><pre>logs:<br>  persistence:<br>    enabled: true<br>    existingClaim: &quot;&quot;<br>    storageClassName: hostpath<br>    size: 1Gi</pre><p>This change ensures that logs are written to a persistent volume, making them accessible in the UI even after pod restarts.</p><p>After saving the file, I re-deployed the chart using:</p><pre>helm upgrade --install airflow apache-airflow/airflow --namespace airflow --create-namespace -f values.yaml</pre><p>Once the components restarted, the logs began showing correctly in the Airflow UI.</p><p>Thank you for reading this blog!</p><p>See you in the next post!</p><p>The structure and direction of this project follow the approach shared by <a href="https://medium.com/u/e88756b73479">Yusuf Ganiyu</a>.</p><p>Let’s connect via <a href="https://www.linkedin.com/in/dominik-zygarski/"><strong>LinkedIn</strong></a><strong>.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7e1d6b49e56a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Project: Implementation of a Scalable Architecture with a Load Balancer in AWS.]]></title>
            <link>https://medium.com/@dominikzygarski_88070/implementation-of-a-scalable-architecture-with-a-load-balancer-in-aws-3a7282ca4be3?source=rss-b563547878e3------2</link>
            <guid isPermaLink="false">https://medium.com/p/3a7282ca4be3</guid>
            <category><![CDATA[infrastructure-as-code]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Dominik Zygarski]]></dc:creator>
            <pubDate>Wed, 19 Mar 2025 21:40:02 GMT</pubDate>
            <atom:updated>2025-05-19T09:22:33.226Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/916/1*nBdra7XrJ7Rsrs0OGK8d_A.png" /></figure><p>In this article, I will describe the process of creating a scalable cloud infrastructure using AWS CloudFormation. I developed a template that automates the deployment of an Application Load Balancer (ALB), an Auto Scaling Group, and the necessary infrastructure components, such as a VPC, subnets, and security groups. The goal of this project was to design a highly available architecture that can dynamically scale based on CPU utilization, monitored by CloudWatch alarms. Leveraging Infrastructure as Code (IaC) principles ensures consistency and repeatability of deployments, which is a critical skill for aspiring DevOps and Cloud Engineers.</p><h3><strong>Overview of the Architecture</strong></h3><p>The architecture I designed is centered around high availability and scalability, leveraging AWS services to ensure seamless traffic distribution and resource management. At its core, the setup includes an Application Load Balancer (ALB) that distributes incoming HTTP traffic across two public subnets in different Availability Zones . These subnets host EC2 instances managed by an Auto Scaling Group, which dynamically adjusts the number of instances based on CPU utilization. A CloudWatch alarm monitors the CPU usage of the instances, triggering the Auto Scaling Group to scale out when the average CPU exceeds 85% over a specified period. The entire infrastructure is deployed within a custom VPC, connected to the internet via an Internet Gateway, with security groups configured to allow HTTP traffic.</p><pre>AWSTemplateFormatVersion: &quot;2010-09-09&quot;<br>Description: &quot;Automatic Load Balancer&quot;<br><br>Resources:<br><br># VPC<br>  VPC:<br>    Type: &quot;AWS::EC2::VPC&quot;<br>    Properties:<br>      CidrBlock: &quot;10.0.0.0/16&quot;<br>      EnableDnsHostnames: true<br>      EnableDnsSupport: true<br>      Tags:<br>        - Key: Name<br>          Value: VPC<br>  <br># Public Subnet 1A<br>  PublicSubnet1A:<br>    Type: &quot;AWS::EC2::Subnet&quot;<br>    Properties:<br>      VpcId: !Ref VPC<br>      AvailabilityZone: !Select [0, !GetAZs &#39;&#39;]<br>      CidrBlock: &quot;10.0.1.0/24&quot;<br>      MapPublicIpOnLaunch: true<br>      Tags: <br>        - Key: Name<br>          Value: PublicSubnet1A<br><br># Public Subnet 1B<br>  PublicSubnet1B:<br>    Type: &quot;AWS::EC2::Subnet&quot;<br>    Properties:<br>      VpcId: !Ref VPC<br>      AvailabilityZone: !Select [1, !GetAZs &#39;&#39;]<br>      CidrBlock: &quot;10.0.2.0/24&quot;<br>      MapPublicIpOnLaunch: true<br>      Tags: <br>        - Key: Name<br>          Value: PublicSubnet1B<br><br># Internet Gateway<br>  InternetGateway:<br>    Type: &quot;AWS::EC2::InternetGateway&quot;<br>    Properties:<br>      Tags:<br>        - Key: Name<br>          Value: InternetGateway<br><br># InternetGatewayAttachment<br>  InternetGatewayAttachment:<br>    Type: &quot;AWS::EC2::VPCGatewayAttachment&quot;<br>    Properties:<br>      InternetGatewayId: !Ref InternetGateway<br>      VpcId: !Ref VPC<br><br># RouteTable<br>  RouteTable:<br>    Type: &quot;AWS::EC2::RouteTable&quot;<br>    Properties:<br>      VpcId: !Ref VPC<br>      Tags:<br>        - Key: Name<br>          Value: RouteTable<br><br># Route<br>  Route: <br>    Type: &quot;AWS::EC2::Route&quot;<br>    DependsOn: InternetGatewayAttachment<br>    Properties:<br>      RouteTableId: !Ref RouteTable<br>      GatewayId: !Ref InternetGateway<br>      DestinationCidrBlock: &quot;0.0.0.0/0&quot;<br><br># PublicSubnet1ARoute<br>  PublicSubnet1ARoute:<br>    Type: &quot;AWS::EC2::SubnetRouteTableAssociation&quot;<br>    Properties:<br>      RouteTableId: !Ref RouteTable<br>      SubnetId: !Ref PublicSubnet1A<br><br># PublicSubnet1BRoute<br>  PublicSubnet1BRoute:<br>    Type: &quot;AWS::EC2::SubnetRouteTableAssociation&quot;<br>    Properties:<br>      RouteTableId: !Ref RouteTable<br>      SubnetId: !Ref PublicSubnet1B<br>  <br># SecurityGroup<br>  SecurityGroup: <br>    Type: &quot;AWS::EC2::SecurityGroup&quot;<br>    Properties:<br>      VpcId: !Ref VPC<br>      GroupDescription: &quot;Allow HTTP traffic&quot;<br>      SecurityGroupIngress: <br>        - IpProtocol: tcp<br>          FromPort: 80<br>          ToPort: 80<br>          CidrIp: &quot;0.0.0.0/0&quot;<br>      Tags: <br>        - Key: Name<br>          Value: SecurityGroup<br><br># LoadBalancer<br>  LoadBalancer:<br>    Type: &quot;AWS::ElasticLoadBalancingV2::LoadBalancer&quot;<br>    Properties:<br>      Subnets: <br>        - !Ref PublicSubnet1A<br>        - !Ref PublicSubnet1B<br>      SecurityGroups:<br>        - !Ref SecurityGroup<br>      Tags:<br>        - Key: Name<br>          Value: LoadBalancer<br><br># LoadBalancerListener:<br>  LoadBalancerListener:<br>    Type: &quot;AWS::ElasticLoadBalancingV2::Listener&quot;<br>    Properties:<br>      LoadBalancerArn: !Ref LoadBalancer<br>      DefaultActions: <br>        - Type: &quot;forward&quot;<br>          TargetGroupArn: !Ref TargetGroup<br>      Protocol: HTTP<br>      Port: 80<br><br># TargetGroup<br>  TargetGroup: <br>    Type: &quot;AWS::ElasticLoadBalancingV2::TargetGroup&quot;<br>    Properties:<br>      VpcId: !Ref VPC<br>      Protocol: HTTP<br>      Port: 80<br>      Tags:<br>        - Key: Name<br>          Value: TargetGroup<br><br># LaunchTemplate<br>  LaunchTemplate:<br>    Type: &quot;AWS::EC2::LaunchTemplate&quot;<br>    Properties:<br>      LaunchTemplateData:<br>        ImageId: &quot;ami-0b74f796d330ab49c&quot;<br>        InstanceType: &quot;t2.micro&quot;<br>        SecurityGroupIds:<br>          - !Ref SecurityGroup<br>        UserData:<br>          Fn::Base64: !Sub |<br>            #!/bin/bash<br>            yum install -y httpd<br>            systemctl start httpd<br>            systemctl enable httpd<br>            echo &quot;Testing Auto Scaling Group Instance&quot; &gt; /var/www/html/index.html<br><br># AutoScalingGroup<br>  AutoScalingGroup:<br>    Type: &quot;AWS::AutoScaling::AutoScalingGroup&quot;<br>    Properties: <br>      LaunchTemplate: <br>        LaunchTemplateId: !Ref LaunchTemplate<br>        Version: !GetAtt LaunchTemplate.LatestVersionNumber<br>      TargetGroupARNs:<br>        - !Ref TargetGroup<br>      VPCZoneIdentifier:<br>        - !Ref PublicSubnet1A<br>        - !Ref PublicSubnet1B<br>      MinSize: 1<br>      DesiredCapacity: 2<br>      MaxSize: 3<br>      Tags: <br>        - Key: Name<br>          Value: AutoScalingGroup<br>          PropagateAtLaunch: true <br>      <br># ScalingPolicy<br>  ScaleOutPolicy:<br>    Type: &quot;AWS::AutoScaling::ScalingPolicy&quot;<br>    Properties:<br>      AutoScalingGroupName: !Ref AutoScalingGroup<br>      PolicyType: SimpleScaling<br>      AdjustmentType: ChangeInCapacity<br>      ScalingAdjustment: 1<br>      Cooldown: 300<br>    <br># CloudWatch Alarm for High CPU Utilization<br>  HighCPUAlarm:<br>    Type: &quot;AWS::CloudWatch::Alarm&quot;<br>    Properties:<br>      AlarmDescription: &quot;Alarm when CPU exceeds 85%&quot;<br>      Namespace: &quot;AWS/EC2&quot;<br>      MetricName: CPUUtilization<br>      Statistic: Average<br>      Threshold: 85<br>      ComparisonOperator: GreaterThanThreshold<br>      Period: 300<br>      EvaluationPeriods: 2<br>      AlarmActions:<br>        - !Ref ScaleOutPolicy<br>      Dimensions:<br>        - Name: AutoScalingGroupName<br>          Value: !Ref AutoScalingGroup</pre><h4>VPC Configuration</h4><p>The VPC resource defines a virtual network with a <strong>CIDR block</strong> of <strong>10.0.0.0/16</strong>, creating a broad address space for the infrastructure. It has <strong>EnableDnsHostnames</strong> and <strong>EnableDnsSupport</strong> set to true, allowing instances within the VPC to resolve domain names effectively.</p><pre># VPC<br>  VPC:<br>    Type: &quot;AWS::EC2::VPC&quot;<br>    Properties:<br>      CidrBlock: &quot;10.0.0.0/16&quot;<br>      EnableDnsHostnames: true<br>      EnableDnsSupport: true<br>      Tags:<br>        - Key: Name<br>          Value: VPC</pre><h4>Public Subnets Configuration</h4><p>The template defines two public subnets, PublicSubnet1A and PublicSubnet1B, within the VPC. PublicSubnet1A uses a CIDR block of <strong>10.0.1.0/24</strong> and is placed in the first Availability Zone of the region, selected dynamically with <strong>!Select [0, !GetAZs ‘’]</strong>. Similarly, PublicSubnet1B has a CIDR block of <strong>10.0.2.0/24</strong> and is placed in the second Availability Zone using <strong>!Select [1, !GetAZs ‘’]</strong>. Both subnets have <strong>MapPublicIpOnLaunch</strong> set to true, ensuring that instances launched in these subnets receive a public IP address.</p><pre># Public Subnet 1A<br>  PublicSubnet1A:<br>    Type: &quot;AWS::EC2::Subnet&quot;<br>    Properties:<br>      VpcId: !Ref VPC<br>      AvailabilityZone: !Select [0, !GetAZs &#39;&#39;]<br>      CidrBlock: &quot;10.0.1.0/24&quot;<br>      MapPublicIpOnLaunch: true<br>      Tags: <br>        - Key: Name<br>          Value: PublicSubnet1A<br><br># Public Subnet 1B<br>  PublicSubnet1B:<br>    Type: &quot;AWS::EC2::Subnet&quot;<br>    Properties:<br>      VpcId: !Ref VPC<br>      AvailabilityZone: !Select [1, !GetAZs &#39;&#39;]<br>      CidrBlock: &quot;10.0.2.0/24&quot;<br>      MapPublicIpOnLaunch: true<br>      Tags: <br>        - Key: Name<br>          Value: PublicSubnet1B</pre><h4>Internet Gateway Setup</h4><p>The resource creates an Internet Gateway. It enables internet access for the VPC. The <strong>InternetGatewayAttachment</strong> resource then attaches this Internet Gateway to the VPC using <strong>InternetGatewayId: !Ref InternetGateway</strong> and, establishing a connection that allows resources within the VPC to communicate with the internet.</p><pre># Internet Gateway<br>  InternetGateway:<br>    Type: &quot;AWS::EC2::InternetGateway&quot;<br>    Properties:<br>      Tags:<br>        - Key: Name<br>          Value: InternetGateway<br><br># InternetGatewayAttachment<br>  InternetGatewayAttachment:<br>    Type: &quot;AWS::EC2::VPCGatewayAttachment&quot;<br>    Properties:<br>      InternetGatewayId: !Ref InternetGateway<br>      VpcId: !Ref VPC</pre><h4>Routing Configuration</h4><p>The RouteTable resource creates a route table for the VPC and is associated with the VPC using <strong>VpcId: !Ref VPC</strong>. The Route resource adds a default route to this table, directing all outbound traffic (<strong>DestinationCidrBlock: “0.0.0.0/0”</strong>) to the <strong>Internet Gateway</strong> via <strong>GatewayId: !Ref InternetGateway</strong>, with a <strong>dependency on InternetGatewayAttachment</strong>. The PublicSubnet1ARoute and PublicSubnet1BRoute resources then associate this route table with PublicSubnet1A and PublicSubnet1B, ensuring that instances in these subnets can access the internet.</p><pre># RouteTable<br>  RouteTable:<br>    Type: &quot;AWS::EC2::RouteTable&quot;<br>    Properties:<br>      VpcId: !Ref VPC<br>      Tags:<br>        - Key: Name<br>          Value: RouteTable<br><br># Route<br>  Route: <br>    Type: &quot;AWS::EC2::Route&quot;<br>    DependsOn: InternetGatewayAttachment<br>    Properties:<br>      RouteTableId: !Ref RouteTable<br>      GatewayId: !Ref InternetGateway<br>      DestinationCidrBlock: &quot;0.0.0.0/0&quot;<br><br># PublicSubnet1ARoute<br>  PublicSubnet1ARoute:<br>    Type: &quot;AWS::EC2::SubnetRouteTableAssociation&quot;<br>    Properties:<br>      RouteTableId: !Ref RouteTable<br>      SubnetId: !Ref PublicSubnet1A<br><br># PublicSubnet1BRoute<br>  PublicSubnet1BRoute:<br>    Type: &quot;AWS::EC2::SubnetRouteTableAssociation&quot;<br>    Properties:<br>      RouteTableId: !Ref RouteTable<br>      SubnetId: !Ref PublicSubnet1B</pre><h4>Security Group Configuration</h4><p>The SecurityGroup resource defines a security group within the VPC. It <strong>allows inbound HTTP traffic</strong> on port <strong>80</strong> from any IP address (<strong>CidrIp: “0.0.0.0/0”</strong>) using the <strong>TCP protocol</strong>. We specify TCP because security groups operate at the <strong>transport layer</strong> (<strong>Layer 4</strong>) of the <strong>OSI model</strong>, controlling traffic based on protocols like TCP or UDP, along with port numbers. This setup ensures that the Application Load Balancer and EC2 instances can receive web traffic.</p><pre># SecurityGroup<br>  SecurityGroup: <br>    Type: &quot;AWS::EC2::SecurityGroup&quot;<br>    Properties:<br>      VpcId: !Ref VPC<br>      GroupDescription: &quot;Allow HTTP traffic&quot;<br>      SecurityGroupIngress: <br>        - IpProtocol: tcp<br>          FromPort: 80<br>          ToPort: 80<br>          CidrIp: &quot;0.0.0.0/0&quot;<br>      Tags: <br>        - Key: Name<br>          Value: SecurityGroup</pre><h4>Load Balancer Setup</h4><p>The LoadBalancer resource creates an Application Load Balancer and is deployed across PublicSubnet1A and PublicSubnet1B for <strong>high availability</strong>. It uses the previously defined SecurityGroup to manage traffic. The <strong>LoadBalancerListener listens</strong> on port <strong>80</strong> for <strong>HTTP</strong> traffic and forwards requests to the TargetGroup using the <strong>forward action</strong>. HTTP is used here because the Application Load Balancer operates at the <strong>application layer (Layer 7)</strong> of the <strong>OSI model</strong>, allowing it to inspect and route traffic based on HTTP-specific attributes like headers or paths, which is ideal for web applications. The TargetGroup, operates within the VPC, handling HTTP traffic on port 80 and directing it to the associated EC2 instances.</p><pre># LoadBalancer<br>  LoadBalancer:<br>    Type: &quot;AWS::ElasticLoadBalancingV2::LoadBalancer&quot;<br>    Properties:<br>      Subnets: <br>        - !Ref PublicSubnet1A<br>        - !Ref PublicSubnet1B<br>      SecurityGroups:<br>        - !Ref SecurityGroup<br>      Tags:<br>        - Key: Name<br>          Value: LoadBalancer<br><br># LoadBalancerListener:<br>  LoadBalancerListener:<br>    Type: &quot;AWS::ElasticLoadBalancingV2::Listener&quot;<br>    Properties:<br>      LoadBalancerArn: !Ref LoadBalancer<br>      DefaultActions: <br>        - Type: &quot;forward&quot;<br>          TargetGroupArn: !Ref TargetGroup<br>      Protocol: HTTP<br>      Port: 80<br><br># TargetGroup<br>  TargetGroup: <br>    Type: &quot;AWS::ElasticLoadBalancingV2::TargetGroup&quot;<br>    Properties:<br>      VpcId: !Ref VPC<br>      Protocol: HTTP<br>      Port: 80<br>      Tags:<br>        - Key: Name<br>          Value: TargetGroup</pre><h4>Auto Scaling Configuration</h4><p>The LaunchTemplate resource defines the configuration for EC2 instances, using the t2.micro instance type and the AMI ami-0b74f796d330ab49c. It applies the SecurityGroup and includes a <strong>UserData script</strong> that <strong>installs Apache</strong>, starts the web server, and creates a test page.</p><p>The AutoScalingGroup resource uses this template to manage instances across PublicSubnet1A and PublicSubnet1B, with a <strong>minimum of 1</strong>, <strong>desired capacity of 2</strong>, and <strong>maximum of 3 instances.</strong> It integrates with the TargetGroup for load balancing. The ScaleOutPolicy enables simple scaling, <strong>adding one instance</strong> (ScalingAdjustment: 1) with a <strong>300-second cooldown period</strong>.</p><pre># LaunchTemplate<br>  LaunchTemplate:<br>    Type: &quot;AWS::EC2::LaunchTemplate&quot;<br>    Properties:<br>      LaunchTemplateData:<br>        ImageId: &quot;ami-0b74f796d330ab49c&quot;<br>        InstanceType: &quot;t2.micro&quot;<br>        SecurityGroupIds:<br>          - !Ref SecurityGroup<br>        UserData:<br>          Fn::Base64: !Sub |<br>            #!/bin/bash<br>            yum install -y httpd<br>            systemctl start httpd<br>            systemctl enable httpd<br>            echo &quot;Testing Auto Scaling Group Instance&quot; &gt; /var/www/html/index.html<br><br># AutoScalingGroup<br>  AutoScalingGroup:<br>    Type: &quot;AWS::AutoScaling::AutoScalingGroup&quot;<br>    Properties: <br>      LaunchTemplate: <br>        LaunchTemplateId: !Ref LaunchTemplate<br>        Version: !GetAtt LaunchTemplate.LatestVersionNumber<br>      TargetGroupARNs:<br>        - !Ref TargetGroup<br>      VPCZoneIdentifier:<br>        - !Ref PublicSubnet1A<br>        - !Ref PublicSubnet1B<br>      MinSize: 1<br>      DesiredCapacity: 2<br>      MaxSize: 3<br>      Tags: <br>        - Key: Name<br>          Value: AutoScalingGroup<br>          PropagateAtLaunch: true <br>      <br># ScalingPolicy<br>  ScaleOutPolicy:<br>    Type: &quot;AWS::AutoScaling::ScalingPolicy&quot;<br>    Properties:<br>      AutoScalingGroupName: !Ref AutoScalingGroup<br>      PolicyType: SimpleScaling<br>      AdjustmentType: ChangeInCapacity<br>      ScalingAdjustment: 1<br>      Cooldown: 300</pre><h4>CloudWatch Alarm for Scaling</h4><p>The HighCPUAlarm resource sets up a <strong>CloudWatch</strong> alarm to monitor CPU usage of the AutoScalingGroup. It tracks the <strong>CPUUtilization metric</strong> in the <strong>AWS/EC2 namespace</strong>, calculating the average over a 300-second period. If the average CPU usage exceeds <strong>85%</strong> for two consecutive periods (EvaluationPeriods: 2), the alarm <strong>triggers the ScaleOutPolicy to add an instance.</strong></p><pre># CloudWatch Alarm for High CPU Utilization<br>  HighCPUAlarm:<br>    Type: &quot;AWS::CloudWatch::Alarm&quot;<br>    Properties:<br>      AlarmDescription: &quot;Alarm when CPU exceeds 85%&quot;<br>      Namespace: &quot;AWS/EC2&quot;<br>      MetricName: CPUUtilization<br>      Statistic: Average<br>      Threshold: 85<br>      ComparisonOperator: GreaterThanThreshold<br>      Period: 300<br>      EvaluationPeriods: 2<br>      AlarmActions:<br>        - !Ref ScaleOutPolicy<br>      Dimensions:<br>        - Name: AutoScalingGroupName<br>          Value: !Ref AutoScalingGroup</pre><p>Thank you for reading this blog !</p><p>See you in the next post!</p><p>Let’s connect via <a href="https://www.linkedin.com/in/dominik-zygarski/"><strong>LinkedIn</strong></a><strong>.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3a7282ca4be3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[IAM with CloudFormation.]]></title>
            <link>https://medium.com/@dominikzygarski_88070/cloud-engineer-academy-module-5-6-part-3-iam-with-cloudformation-b929646f7598?source=rss-b563547878e3------2</link>
            <guid isPermaLink="false">https://medium.com/p/b929646f7598</guid>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Dominik Zygarski]]></dc:creator>
            <pubDate>Fri, 08 Nov 2024 16:09:30 GMT</pubDate>
            <atom:updated>2025-03-17T18:09:16.632Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QSDVzVKMdqfXkUuo6UnaiQ.png" /></figure><p>AWS Identity and Access Management (IAM) is a core security service in Amazon Web Services that allows you to manage access to AWS resources. With IAM, you can create and manage users, groups, roles, and access policies. CloudFormation, on the other hand, is an infrastructure-as-code tool that enables you to define AWS resources in YAML or JSON files, making it possible to replicate environments across AWS accounts or regions.</p><p>In this post, we will demonstrate how to use IAM and CloudFormation to create a user, group, roles, and policies. Finally, we will showcase a complete CloudFormation template to build this IAM setup.</p><h4>IAM Users</h4><p>IAM User is a <strong>unique account assigned to a person or application, allowing access to cloud resources</strong>. Each IAM User has a unique identifier and can be assigned to groups and have individual policies. IAM User is the ideal choice when we need dedicated access for individuals or applications within a single AWS account.</p><p><strong>Creating an IAM User with CloudFormation</strong></p><p>Sample CloudFormation template to create an IAM User:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/740/1*OE3UC9WvmSz1LFGootdTVg.png" /></figure><p>In CloudFormation, the ManagedPolicyArns field in resources of type AWS::IAM::User is used to assign managed policies to a user in the form of a list of ARNs (Amazon Resource Names).</p><p>Managed policies are predefined sets of permissions that can be assigned to various IAM resources (users, groups, or roles). They allow easy and centralized management of permissions, which is especially useful when the same set of permissions needs to be assigned to multiple resources.</p><p>In the example shown:</p><pre>ManagedPolicyArns:<br>  - &#39;arn:aws:iam::aws:policy/AdministratorAccess&#39;</pre><p>the user DominikCFN is assigned the managed policy AdministratorAccess, which grants full administrative access to all AWS resources. This policy is defined at the AWS level and is available globally (it begins with arn:aws:iam::aws:policy/).</p><p>Assigning ManagedPolicyArns enables:</p><ul><li>quick permission assignment without the need to create custom policies,</li><li>centralized management and updating of policies at the AWS level,</li><li>easier maintenance of consistency in permissions for users and other IAM resources.</li></ul><p>In the context of automation and standardization, using managed policies helps simplify and unify the permission assignment process within CloudFormation.</p><h4>IAM Groups</h4><p>An IAM Group is a <strong>logical grouping of users that simplifies permission management</strong>. Instead of assigning permissions to individual users, you can add users to a group and assign permissions to the group. This makes it easier to manage permissions, especially for larger teams.</p><p>For example, we will create a group named “Administrators” and then assign the user to this group. This way, every user in the group inherits the permissions assigned to the group. Here’s the CloudFormation code:</p><p><strong>Creating an IAM Group with CloudFormation</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sEc34TPZqr0w1uLmX7As8w.png" /></figure><h4>Assigning User to Group</h4><p>To allow a user to inherit the permissions of a group, we need to assign them to the group. We’ll use the <strong>AWS::IAM::UserToGroupAddition</strong> resource, which allows us to add a user to a group automatically when setting up the environment with CloudFormation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Fli6H4k-n3FQ9UuBqT1nGQ.png" /></figure><p>Here:</p><ul><li><strong>GroupName</strong> references the group we want to add the user to.</li><li><strong>Users</strong> specifies a list of users to be added to the group (in this case, only the “DominikFCN” user).</li></ul><h4>IAM Roles</h4><p>An IAM Role is an <strong>identity that can be assigned to resources like EC2 instances, allowing them to access other AWS resources.</strong> Roles are particularly useful for applications or services that need access to AWS resources using temporary credentials.</p><p>Example: We will create a role that can be assumed by EC2 instances to access AWS resources. In this case, we’ll assign the <strong>PowerUserAccess</strong> policy, which provides full permissions except for IAM and organization-level administrative permissions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cUgoEirIAB2VgcRldqD0Yw.png" /></figure><p><strong>Explanation of Each Element in </strong><strong>Statement</strong></p><p><strong>Effect: “Allow”</strong></p><ul><li>Effect specifies whether the action is <strong>allowed</strong> (Allow) or <strong>denied</strong> (Deny).</li><li>Here, Effect is set to &quot;Allow&quot;, which means that we are permitting the action specified in Action.</li></ul><p><strong>Principal</strong></p><ul><li>Principal defines <strong>who</strong> has the permission to perform the action specified in this policy.</li><li>In this case, Principal is set to Service: &quot;ec2.amazonaws.com&quot;, which means that only <strong>EC2 instances</strong> (the ec2.amazonaws.com service) are allowed to use this role. Essentially, we’re telling AWS that EC2 instances have the right to &quot;assume&quot; (or get access to) this role.</li></ul><p><strong>Action</strong></p><ul><li>Action defines the specific action that is allowed within this policy.</li><li>sts:AssumeRole is the action that allows resources, such as EC2 instances, to <strong>assume the IAM role</strong>. By doing this, the EC2 instance can use the permissions attached to this role.</li></ul><h4>Custom Policy</h4><p>A Custom Policy is a <strong>custom-defined policy that allows you to specify detailed access rules</strong>. Custom policies are useful when you need specific permissions not covered by AWS’s managed policies. In our example, we create a policy that allows access to S3 objects.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6Zh0k13KUSRGV3kvarszqw.png" /></figure><p>In this code:</p><ul><li><strong>PolicyName</strong> defines the policy name as “CustomPolicy”.</li><li><strong>PolicyDocument</strong> contains the access rules; in this case, it allows retrieving objects from any S3 resource.</li><li><strong>Roles</strong> assigns this policy to the previously defined role.</li></ul><h3>Complete Solution in CloudFormation</h3><p>The above elements can be combined into a single CloudFormation template that automatically creates a user, group, assigns the user to the group, and defines an IAM role and policy. This template enables fast and repeatable IAM resource creation, which is particularly useful in larger environments.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ff8tmmA-aZI0N2i6odaF5Q.png" /></figure><p>A complete solution enables the automated setup of an IAM structure, making the process fast, consistent, and aligned with best practices.</p><p>Thank you for reading this blog post! I hope this guide has helped you understand the basics of managing access in AWS with IAM and CloudFormation.</p><p>See you in the next post!</p><p>Let’s connect via <a href="https://www.linkedin.com/in/dominik-zygarski/"><strong>LinkedIn</strong></a><strong>.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b929646f7598" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Project: Building a Secure and Communicative VPC with AWS CloudFormation.]]></title>
            <link>https://medium.com/@dominikzygarski_88070/project-building-a-secure-and-communicative-vpc-with-aws-cloudformation-b8111ae56995?source=rss-b563547878e3------2</link>
            <guid isPermaLink="false">https://medium.com/p/b8111ae56995</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[cloudformation]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <dc:creator><![CDATA[Dominik Zygarski]]></dc:creator>
            <pubDate>Wed, 10 Jul 2024 13:05:55 GMT</pubDate>
            <atom:updated>2024-07-10T13:05:55.193Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*cZetZPRM8P3BjFAgpyf5XQ.png" /></figure><p>In this blog post, I will describe the process of deploying a <strong>Bastion Host using AWS CloudFormation</strong>. CloudFormation is a powerful tool that allows you to automate the management of AWS infrastructure using template files. By leveraging CloudFormation, I was able to quickly and effectively create a Bastion Host, significantly simplifying the process of configuring and managing infrastructure.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bCyxDE4Cx-bRvWwYT9xZwg.png" /></figure><h3>Writing the CloudFormation Template</h3><p>To write the CloudFormation template, I used the Visual Studio Code (VSCode) editor. VSCode is a convenient tool that offers many extensions to facilitate working with CloudFormation templates, such as syntax highlighting and auto-completion. This made the process of creating and editing template files much faster and more efficient.</p><h4>Creating a VPC</h4><p>The first step in creating the template was writing a <strong><em>YAML</em> file</strong> named cloudformation-vpc2.yaml. At the beginning of the template, I specified the <strong>template version</strong> and a brief <strong>description</strong>:</p><pre>Resources:<br>  BastionHostVPC:<br>    Type: &#39;AWS::EC2::VPC&#39;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*agg4hWbBDglhJZQJsSYYrA.png" /></figure><p>Next, I began defining the <strong>resources</strong>, starting with the <strong>VPC</strong> where all other resources will reside. A VPC is the primary organizational unit in AWS that allows for the creation of isolated networks. In my template, I defined a VPC named BastionHostVPC with the <strong>CIDR block</strong> of 183.14.0.0/16, <strong>DNS support enabled</strong>, and <strong>DNS hostnames enabled</strong>. Here is a snippet of the template code:</p><pre>Resources:<br>  BastionHostVPC:<br>    Type: &#39;AWS::EC2::VPC&#39;<br>    Properties:<br>      CidrBlock: &#39;183.14.0.0/16&#39;<br>      EnableDnsSupport: true<br>      EnableDnsHostnames: true<br>      Tags:<br>        - Key: &#39;Name&#39;<br>          Value: &#39;BastionHostVPC&#39;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1iVtV6eDltaedZVu1lHQYw.png" /></figure><h4>Creating Public Subnet 1A</h4><p>The next step was to create the PublicSubnet1A in (<strong>AZ-1a</strong>). This subnet was defined with the <strong>CIDR block</strong> 183.14.1.0/24 and assigned to the <strong>first Available Zone in the region</strong>. In the template, I also specified that instances launched in this subnet would <strong>automatically receive public IP addresses</strong>.</p><pre># Public Subnet 1A<br>PublicSubnet1A:<br>  Type: &#39;AWS::EC2::Subnet&#39;<br>  Properties:<br>    VpcId: !Ref &#39;BastionHostVPC&#39;<br>    CidrBlock: &#39;183.14.1.0/24&#39;<br>    AvailabilityZone: !Select [0, !GetAZs &#39;&#39;]<br>    MapPublicIpOnLaunch: true<br>    Tags:<br>      - Key: &#39;Name&#39;<br>        Value: &#39;PublicSubnet1A</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*456KVGHp_ge3iIZ3tJeyEg.png" /></figure><h4>Creating Private Subnet 1A</h4><p>The next step was to deploy the<strong> Private Subnet</strong> AppPrivateSubnet1A (<strong>AZ-1a</strong>). This subnet was defined with the <strong>CIDR block</strong> of 183.14.2.0/24.</p><pre># App Private Subnet 1A<br>AppPrivateSubnet1A:<br>  Type: &#39;AWS::EC2::Subnet&#39;<br>  Properties:<br>    VpcId: !Ref &#39;BastionHostVPC&#39;<br>    CidrBlock: &#39;183.14.2.0/24&#39;<br>    AvailabilityZone: !Select [0, !GetAZs &#39;&#39;]<br>    Tags:<br>      - Key: &#39;Name&#39;<br>        Value: &#39;AppPrivateSubnet1A&#39;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fxOp_nwFYG73AQnPv-4alw.png" /></figure><h4>Creating Data Private Subnet 1A</h4><p>The next step was to create the<strong> Private Subnet </strong>DataPrivateSubnet1A <br>(<strong>AZ-1a</strong>). This subnet was defined with <strong>the CIDR block</strong> 183.14.3.0/24.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mgbeaiEYGS8QevtJ_Koh7Q.png" /></figure><h4>Creating Subnets in the Second Availability Zone</h4><p>Next, I repeated these steps for the<strong> second Availability Zone</strong> (<strong>AZ-1b</strong>). In AWS, the first Availability Zone is indexed as 0 and the second as 1. This is because AWS indexing starts from zero, so when selecting Availability Zones using the !Select function, we use 0 for the first zone and 1 for the second. I created the<strong> Public Subnet, App Private Subnet, </strong>and<strong> Data Private Subnet</strong> in (<strong>AZ-1b</strong>).</p><pre># Public Subnet 1B<br>PublicSubnet1B:<br>  Type: &#39;AWS::EC2::Subnet&#39;<br>  Properties:<br>    VpcId: !Ref &#39;BastionHostVPC&#39;<br>    CidrBlock: &#39;183.14.4.0/24&#39;<br>    AvailabilityZone: !Select [1, !GetAZs &#39;&#39;]<br>    MapPublicIpOnLaunch: true<br>    Tags:<br>      - Key: &#39;Name&#39;<br>        Value: &#39;PublicSubnet1B&#39;<br><br># App Private Subnet 1B<br>AppPrivateSubnet1B:<br>  Type: &#39;AWS::EC2::Subnet&#39;<br>  Properties:<br>    VpcId: !Ref &#39;BastionHostVPC&#39;<br>    CidrBlock: &#39;183.14.5.0/24&#39;<br>    AvailabilityZone: !Select [1, !GetAZs &#39;&#39;]<br>    Tags:<br>      - Key: &#39;Name&#39;<br>        Value: &#39;AppPrivateSubnet1B&#39;<br><br># Data Private Subnet 1B<br>DataPrivateSubnet1B:<br>  Type: &#39;AWS::EC2::Subnet&#39;<br>  Properties:<br>    VpcId: !Ref &#39;BastionHostVPC&#39;<br>    CidrBlock: &#39;183.14.6.0/24&#39;<br>    AvailabilityZone: !Select [1, !GetAZs &#39;&#39;]<br>    Tags:<br>      - Key: &#39;Name&#39;<br>        Value: &#39;DataPrivateSubnet1B&#39;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AxY1iryaQXO8ZLIlv1q1yA.png" /></figure><h4>Creating the Internet Gateway</h4><p>The next step was to define and deploy the <strong>Internet Gateway</strong>. The Internet Gateway <strong>allows instances</strong> in the Public Subnet <strong>to communicate with the internet</strong>. In my template, I defined the it and added appropriate tags for easier identification.</p><pre># Internet Gateway<br>InternetGateway:<br>  Type: &#39;AWS::EC2::InternetGateway&#39;<br>  Properties:<br>    Tags:<br>      - Key: &#39;Name&#39;<br>        Value: &#39;InternetGateway&#39;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Z73oa5p-dbykuSagxVMcMQ.png" /></figure><h4>Creating the Gateway Attachment</h4><p>The next step was to create the<strong> Gateway Attachment</strong>, which <strong>assigns the Internet Gateway to our VPC</strong>. This connection <strong>is necessary to allow instances in the Public Subnets to access the internet</strong>. In my template, I specified that the VPC to which the Internet Gateway should be attached is BastionHostVPC.</p><pre># Gateway Attachment<br>AttachGateway:<br>  Type: &#39;AWS::EC2::VPCGatewayAttachment&#39;<br>  Properties:<br>    VpcId: !Ref &#39;BastionHostVPC&#39;<br>    InternetGatewayId: !Ref &#39;InternetGateway&#39;</pre><h4>Creating the Route Table</h4><p>The next step was to create the<strong> Route Table</strong>. The Route Table <strong>specifies routing rules for network traffic within the VPC</strong>. In my template, I defined the Route Table and assigned it to BastionHostVPC, using appropriate tags to facilitate its identification.</p><pre># Route Table<br>RouteTable:<br>  Type: &#39;AWS::EC2::RouteTable&#39;<br>  Properties:<br>    VpcId: !Ref &#39;BastionHostVPC&#39;<br>    Tags:<br>      - Key: &#39;Name&#39;<br>        Value: &#39;RouteTable&#39;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jRhgCAwYHPtiNNzl1IgF5A.png" /></figure><h4>Defining the Public Route</h4><p>The next step was to define the route (<strong>Public Route</strong>) in our <strong>Route Table</strong>. In this entry, <strong>I specified which Route Table it should refer to</strong> and <strong>where it should direct the traffic</strong>, which is the Internet Gateway. I also <strong>opened the network traffic to the internet</strong> by defining the <strong>CIDR block</strong> 0.0.0.0/0, <strong>allowing incoming and outgoing traffic from the internet</strong>.</p><pre># Public Route<br>PublicRoute:<br>  Type: &#39;AWS::EC2::Route&#39;<br>  DependsOn: &#39;AttachGateway&#39;<br>  Properties:<br>    RouteTableId: !Ref &#39;RouteTable&#39;<br>    GatewayId: !Ref &#39;InternetGateway&#39;<br>    DestinationCidrBlock: &#39;0.0.0.0/0&#39;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*38jS4VvJKc6uXqH-q6AstA.png" /></figure><h4>Route Table Association</h4><p>The next step was to <strong>associate the Route Table with the respective subnets</strong>. This <strong>ensures that my subnets do not use the default routing tables</strong>. I associated the Route Table with the Public Subnets PublicSubnet1A and PublicSubnet1B to enable them to use the custom routing table defined earlier.</p><pre># Route Table Association for Public Subnet 1A<br>PublicSubnet1ARouteTableAssociation:<br>  Type: &#39;AWS::EC2::SubnetRouteTableAssociation&#39;<br>  Properties:<br>    SubnetId: !Ref &#39;PublicSubnet1A&#39;<br>    RouteTableId: !Ref &#39;RouteTable&#39;<br><br># Route Table Association for Public Subnet 1B<br>PublicSubnet1BRouteTableAssociation:<br>  Type: &#39;AWS::EC2::SubnetRouteTableAssociation&#39;<br>  Properties:<br>    SubnetId: !Ref &#39;PublicSubnet1B&#39;<br>    RouteTableId: !Ref &#39;RouteTable&#39;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*H_izHsy9BQ4pGfurkapXSw.png" /></figure><h3>Placing the Bastion Host in Public Subnet 1A</h3><p>The next part of the project was <strong>to place the Bastion Host</strong> in PublicSubnet1A, similar to what I described in my previous blog post using the AWS console.</p><p>I created an EC2 instance and assigned it to Availability Zone 1A. Then, I <strong>specified the instance type</strong> as t2.micro and <strong>the image</strong> (<strong>AMI</strong>) from which the system would be built. The instance type defines resources like <strong>CPU and memory available to the instance</strong>.</p><p>Next, I assigned a <strong>Security Group</strong> to the instance, which I will define in the next section of the code, and <strong>specified the SSH key that will be used for connections.</strong></p><pre># Bastion Host Instance<br>BastionHostInstance:<br>  Type: &#39;AWS::EC2::Instance&#39;<br>  Properties:<br>    AvailabilityZone: !Select [0, !GetAZs &#39;&#39;]<br>    SubnetId: !Ref &#39;PublicSubnet1A&#39;<br>    InstanceType: &#39;t2.micro&#39;<br>    ImageId: &#39;ami-0910ce22fbfba68e1&#39;<br>    SecurityGroupIds:<br>      - !Ref &#39;BastionHostSG&#39;<br>    KeyName: &#39;BastionHostKeyPair&#39;<br>    Tags:<br>      - Key: &#39;Name&#39;<br>        Value: &#39;BastionHostInstance&#39;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6-Wk7_OMWycftfursYucVA.png" /></figure><p>I used an AMI image obtained from the <strong>AMI Catalog in the AWS console</strong>. An AMI (<strong>Amazon Machine Image</strong>) is a <strong>pre-configured environment that includes an operating system and other software</strong>, which can be used to launch EC2 instances.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*l02-OqOvaO2uYj6K4rj1Cw.png" /></figure><h4>Defining the Security Group for Bastion Host</h4><p>This part of the code creates the <strong>Security Group</strong> I mentioned earlier. I named it EnableSSH because it <strong>allows SSH connections from my computer</strong>. In the template, I defined the <strong>group name</strong>, the <strong>VPC it belongs to</strong>, and the <strong>inbound traffic rules</strong>. The <strong>protocol</strong> used for the connection is <strong>TCP</strong>, the <strong>IP address of my computer is specified</strong>, and <strong>port 22</strong> is opened for <strong>SSH connections</strong>.</p><pre># Bastion Security Group<br>BastionHostSG:<br>  Type: &#39;AWS::EC2::SecurityGroup&#39;<br>  Properties:<br>    GroupDescription: &#39;EnableSSH&#39;<br>    GroupName: &#39;BastionHostSecurityGroup&#39;<br>    VpcId: !Ref &#39;BastionHostVPC&#39;<br>    SecurityGroupIngress:<br>      - IpProtocol: &#39;tcp&#39;<br>        CidrIp: &#39;83.24.86.53/32&#39;<br>        FromPort: 22<br>        ToPort: 22</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*k3QhXRcZHlQtcrOJarny7w.png" /></figure><h4>Coding App Instance 1A</h4><p>Coding the AppInstance1A, which will be located in the AppPrivateSubnet1A subnet, <strong>is similar to coding the Bastion Host</strong>. In the CloudFormation template, we define similar properties: Availability Zone, Subnet, Instance type, AMI Image, Security Group, and SSH key.</p><pre># App Instance 1A<br>AppInstance1A:<br>  Type: &#39;AWS::EC2::Instance&#39;<br>  Properties:<br>    AvailabilityZone: !Select [0, !GetAZs &#39;&#39;]<br>    SubnetId: !Ref &#39;AppPrivateSubnet1A&#39;<br>    InstanceType: &#39;t2.micro&#39;<br>    ImageId: &#39;ami-0910ce22fbfba68e1&#39;<br>    SecurityGroupIds:<br>      - !Ref &#39;AppInstance1ASG&#39;<br>    KeyName: &#39;BastionHostKeyPair&#39;<br>    Tags:<br>      - Key: &#39;Name&#39;<br>        Value: &#39;AppInstance1A&#39;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rTRuMWDDlxmBy1KplIoVFA.png" /></figure><h4>Defining the Security Group for App Instance 1A</h4><p>Next, we define the <strong>Security Group for this instance</strong>. It differs from the previous Security Group in that this time <strong>we do not specify a specific IP</strong> for SSH connections but use SourceSecurityGroupId. This means that instead of allowing traffic from a specific IP address, <strong>we allow traffic from instances assigned to a specified SG</strong>, in this case, from the Bastion Host.</p><pre># App Instance 1A Security Group<br>AppInstance1ASG:<br>  Type: &#39;AWS::EC2::SecurityGroup&#39;<br>  Properties:<br>    GroupDescription: &#39;Allow SSH from Bastion&#39;<br>    VpcId: !Ref &#39;BastionHostVPC&#39;<br>    SecurityGroupIngress:<br>      - IpProtocol: &#39;tcp&#39;<br>        SourceSecurityGroupId: !Ref &#39;BastionHostSG&#39;<br>        FromPort: 22<br>        ToPort: 22<br>    Tags:<br>      - Key: &#39;Name&#39;<br>        Value: &#39;AppInstance1ASG&#39;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8LoiqcrMSSPn2Ry8dszyLQ.png" /></figure><h4>Coding App Instance 1B</h4><p>Similarly to the previous instance, we now set up AppInstance1B, which will be in the AppPrivateSubnet1B subnet, <strong>in another Availability Zone</strong> <br>(<strong>AZ-1b</strong>). In the CloudFormation template, we define the following properties: Availability Zone, Subnet, Instance Type, AMI Image, Security Group, and SSH key.</p><pre># App Instance 1B<br>AppInstance1B:<br>  Type: &#39;AWS::EC2::Instance&#39;<br>  Properties:<br>    AvailabilityZone: !Select [1, !GetAZs &#39;&#39;]<br>    SubnetId: !Ref &#39;AppPrivateSubnet1B&#39;<br>    InstanceType: &#39;t2.micro&#39;<br>    ImageId: &#39;ami-0910ce22fbfba68e1&#39;<br>    SecurityGroupIds:<br>      - !Ref &#39;AppInstance1BSG&#39;<br>    KeyName: &#39;BastionHostKeyPair&#39;<br>    Tags:<br>      - Key: &#39;Name&#39;<br>        Value: &#39;AppInstance1B&#39;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uih5US6SSxI4UbSUbKXEzA.png" /></figure><h4>Defining the Security Group for App Instance 1B</h4><p>For the AppInstance1B instance, we define a Security Group that allows <strong>ICMP traffic</strong> from the AppInstance1A instance <strong>located in another Availability Zone</strong> (<strong>AZ-1a</strong>). ICMP (<strong>Internet Control Message Protocol</strong>) is often used for <strong>network diagnostics</strong>, such as the <strong>ping command</strong>, which checks if an instance is reachable. By using SourceSecurityGroupId, <strong>we allow ICMP traffic only from instances assigned to a specific SG</strong>, in this case, AppInstance1ASG.</p><pre># App Instance 1B Security Group<br>AppInstance1BSG:<br>  Type: &#39;AWS::EC2::SecurityGroup&#39;<br>  Properties:<br>    GroupDescription: &#39;Allow ICMP from AppInstance1A&#39;<br>    VpcId: !Ref &#39;BastionHostVPC&#39;<br>    SecurityGroupIngress:<br>      - IpProtocol: &#39;icmp&#39;<br>        SourceSecurityGroupId: !Ref &#39;AppInstance1ASG&#39;<br>        FromPort: -1<br>        ToPort: -1<br>    Tags:<br>      - Key: &#39;Name&#39;<br>        Value: &#39;AppInstance1BSG&#39;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*K99SNHxpvtPy7uroNdn0Yw.png" /></figure><h3>Deploying VPC Using VS Code</h3><p>To deploy a VPC using CloudFormation, I use <strong>VS Code terminal</strong>.</p><p><strong>Creating the stack</strong>:</p><ul><li>In the VS Code terminal, I used the command to create the stack:</li></ul><pre>aws cloudformation create-stack --stack-name BastionHostVPC --template-body file://cloudformation-vpc2.yaml</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EzvNRgfTbd-oEq5rrOsS6w.png" /></figure><p><strong>Command output</strong>:</p><ul><li>After executing the above command, I received c<strong>onfirmation of the stack creation along with the stack ID</strong>:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*a1lDKMGQuNSnKguK3rP4mQ.png" /></figure><p><strong>Checking the status</strong>:</p><ul><li>To <strong>check the status</strong> of the created stack, I used the command:</li></ul><pre>aws cloudformation describe-stacks --stack-name BastionHostVPC</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*z1gWvQSPJ1v97_E0e4bZ1Q.png" /></figure><ul><li>I received information that the <strong>stack is currently being created</strong>:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WHm7ixcnpRW7aIKiMkJS8w.png" /></figure><ul><li>After a few minutes, I <strong>rechecked</strong> the stack status using the same command and received information that the <strong>stack was successfully created</strong>:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*STYe58o22oGKJu_12k0v2w.png" /></figure><h3>Confirmation of Stack Creation in AWS CloudFormation</h3><p>After creating the stack using CloudFormation in VS Code, <strong>I logged into the AWS console to confirm that the stack was successfully created</strong>.</p><p>In the “Stacks” section, I could see the created stack named BastionHostVPC with the status CREATE_COMPLETE.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NYMM9rcqf16VX5iuaR-e8g.png" /></figure><p>I clicked on the BastionHostVPC stack to view its details and navigated to the &quot;<strong>Events</strong>&quot; tab.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wAz75jcoKA4kTVP9JSAr6A.png" /></figure><p>I could see there that <strong>all resources were successfully created</strong>.</p><h3>Validating SSH Connection</h3><p>To validate the SSH connection to our Bastion Host, <strong>we navigate to the EC2 panel in the AWS console</strong>.</p><p>In the <strong>instances table</strong>, I found our BastionHostInstance and checked its <strong>public IP address</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*of6q5VAmRPtz8M0xNcFzWg.png" /></figure><p>Next, using the terminal, I attempted to connect to the Bastion Host using the following command:</p><pre>ssh -i BastionHostKeyPair.pem ec2-user@3.71.114.139</pre><p>However, I encountered an Operation timed out error. This is likely due to a change in my public IP address, <strong>as I had restarted my router in the meantime, causing the IP address to change</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6s10qHBA5Hi-EKcoOXS0Vw.png" /></figure><h4>Debugging Process</h4><p>After encountering the SSH connection error, I decided to debug the issue.</p><p>I went to <a href="https://whatismyipaddress.com/"><strong>whatismyipaddress.com</strong></a> and <strong>checked my current IP address</strong>.</p><p>Then, I changed the IP address in the code responsible for the Bastion Host Security Group to reflect the new IP address.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*x0WmGWicdiXzkW046q_0ig.png" /></figure><p>After <strong>saving the modified template</strong>, I used the terminal in VSCode to update the stack with the following command:</p><pre>aws cloudformation update-stack --stack-name BastionHostVPC --template-body file://cloudformation-vpc2.yaml</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*94jXOx2eXIvtdsMDOPW9xQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FVWbTqQtjpW4Vb8Tud41fQ.png" /></figure><p>After a few moments, I checked the stack status using the describe-stacks command and saw that the <strong>update was completed successfully</strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*35TyAcwC6HS968ESYt_DOQ.png" /></figure><h4>Rechecking the SSH Connection</h4><p>When I attempted to reconnect to the BastionHost via SSH, <strong>I encountered a permissions error with the key file</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F9tuHbCWDyMby-GHc5zczw.png" /></figure><p>This error occurred because the <strong>file permissions were too open</strong>, making the <strong>private key file accessible to other users</strong>.</p><p>To resolve this issue, I changed the file permissions using the command:</p><pre>chmod 400 BastionHostKeyPair.pem</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-71q_FvzVJeLWV3y_ld4dw.png" /></figure><h4>Copying Over Key</h4><p>Next, I <strong>transferred the private key to the BastionHost to connect to the next instance from there</strong>. I used the following SCP command to copy the key from my local machine to the BastionHost:</p><pre>scp -i BastionHostKeyPair.pem BastionHostKeyPair.pem ec2-user@3.71.114.139:~/</pre><p>~/ denotes the home directory of the user on the remote server. In this case, scp (<strong>secure copy</strong>) transfers the SSH key file from the local machine to the <strong>home directory</strong> of the ec2-user on the BastionHost.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*frsoqTeGlmc8UDT8PWKDqA.png" /></figure><h4>Connecting to Instance in AppPrivateSubnet1A</h4><p>After connecting to BastionHost, I could now connect to the instance in AppPrivateSubnet1A. I used the <strong>private IP address of this instance</strong>, which I found in the AWS console.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_2HpQ9hddd5rAhtl1pmQSQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4WhcsJRsk_e5ySnAChJG_w.png" /></figure><h3>Checking ICMP Connection to Instance in AppPrivateSubnet1B</h3><p>Next, I <strong>checked the ICMP connection</strong> (<strong>ping</strong>) to the instance located in <strong>AppPrivateSubnet1B</strong>.</p><p>Already being connected to the instance in AppPrivateSubnet1A, I used the private IP address of the AppPrivateSubnet1B instance, which I obtained from the AWS console.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bP9ZUQonrGGAuFODqdvstQ.png" /></figure><p>I used the following command to perform the ping:</p><pre>ping 183.14.5.19 -c 4</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZPhyYcqeONfN3mtikKZDLw.png" /></figure><h3>Summary</h3><p>EC2 instances <strong>located in different availability zones</strong> (AZs) within my VPC network <strong>can communicate with each other</strong>. I successfully confirmed the ICMP connection between the instances in AppPrivateSubnet1A and AppPrivateSubnet1B, <strong>proving that the VPC configuration, including subnets, route tables, and security groups, was done correctly</strong>.</p><p>Thank you for reading my blog! I had a lot of fun working on this project and learned a lot of new things.</p><p>Looking forward to seeing you in the next post!</p><p>Dominik</p><p>Let’s connect via <a href="https://www.linkedin.com/in/dominik-zygarski/"><strong>LinkedIn</strong></a><strong>.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b8111ae56995" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploying a Bastion Host.]]></title>
            <link>https://medium.com/@dominikzygarski_88070/cloud-engineer-academy-module-5-part-2-deploying-a-bastion-host-e7cbfe9aa45b?source=rss-b563547878e3------2</link>
            <guid isPermaLink="false">https://medium.com/p/e7cbfe9aa45b</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Dominik Zygarski]]></dc:creator>
            <pubDate>Sun, 09 Jun 2024 14:05:42 GMT</pubDate>
            <atom:updated>2025-03-17T18:11:27.011Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/580/1*iu9ttFsNM_3ZVh3YReBMuQ.png" /></figure><h3><strong>Introduction</strong></h3><p>As part of the Cloud Engineer Academy coursework, we were assigned the task of setting up a Virtual Private Cloud (VPC) with a bastion host on AWS. The goal of this exercise was to create a secure access point to EC2 instances in private subnets. First, I configured the VPC and launched a bastion host in Availability Zone 1 (AZ1). Then, I connected to it via SSH to transfer a private key, which allowed me to access another EC2 instance in the private subnet within the same zone. Finally, I tested the connection to another EC2 instance located in a private subnet in AZ2.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rD7QS0Lwvm7x3hJvxcTJug.png" /></figure><h4>Creating a VPC with Two Availability Zones</h4><p>The first step was to create a Virtual Private Cloud (VPC) with two availability zones (AZ1 and AZ2). In each availability zone, I created two subnets: one public and one private. The public subnet in AZ1 will host a bastion host, serving as an access point to the private subnets in both zones. The private subnets in AZ1 and AZ2 will contain EC2 instances, which can communicate with each other.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*X0uZCv_T-_ZfZVY5-xkQNQ.png" /></figure><h4><strong>Creating a Bastion Host</strong></h4><p>The next step was to create the bastion host. I launched an EC2 instance and assigned it to the <strong>public subnet in Availability Zone 1 (AZ1)</strong>. The bastion host will act as an intermediary, providing secure access to the EC2 instances located in the private subnets. The configuration of the EC2 instance included setting up appropriate security group rules to allow SSH connections only from <strong>my IP addresses</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OcJNPgvUCoTwRyDSXQRb4g.png" /></figure><p>It was also important to configure the <strong>key pair</strong> and download it to my computer to ensure secure access to the instance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jwUsy_hac79MN8zpKdnRag.png" /></figure><h4><strong>Launching EC2 Instances in Private Subnets</strong></h4><p>The next step was to launch <strong>two EC2 instances in the private subnets.</strong> I placed the first instance in the private subnet in Availability Zone 1 (AZ1) and the second instance in the private subnet in AZ2. These instances will be able to communicate with each other within the VPC but will not be directly accessible from the internet. The configuration of private subnets adds an extra layer of security by isolating the EC2 instances from public access. For the instance in AZ1, it was also important to configure and download the <strong>key pair</strong>, though I did not assign them a <strong>public IP</strong> addresses as with the bastion host.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HA7CKBqSEylBQmplFLLnLg.png" /></figure><h4><strong>Connecting to the Bastion Host</strong></h4><p>Next, from my local machine, I tried to connect via SSH to the bastion host using the previously downloaded key pair.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7NGAtK28eRn5UssN9kyk9g.png" /></figure><p>However, I encountered an issue where the file permissions for the key pair were too open. To resolve this, I changed the file permissions using the chmod command. I set the file to permissions <strong>400</strong>, meaning that only the owner of the file has <strong>read access</strong>, and no one else can write to or execute the file.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BnpW3SnVmGef8PqqFvxvfA.png" /></figure><h4><strong>Modifying Security Group Rules</strong></h4><p>After successfully connecting to the bastion host, the next step was to modify the security group rules for the EC2 instance in the private subnet in AZ1. To allow SSH connections from the bastion host, I added a rule permitting inbound traffic on <strong>port 22 (SSH)</strong> from the <strong>bastion host’s IP address.</strong> This enabled secure access to the EC2 instance in the private subnet from the bastion host.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xUoEa9gp5LneeJYaGnjagw.png" /></figure><h4><strong>Transferring Key Pair to Bastion Host</strong></h4><p>After configuring the security group rules, I transferred the previously downloaded key pair file for the EC2 instance in the private subnet in AZ1 to our bastion host. Before transferring, I had to change the permissions of the key pair file to <strong>400</strong>, similar to the previous key pair, to ensure only the owner could read the file. After changing the permissions, I used the scp command to securely copy the file between my local machine and the bastion host. This enabled me to use the bastion host to connect to the EC2 instance in the private subnet.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-fLiz2dQWkdC66IoKdU6oQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qtxIWXa4nT15vWTBC1iU4w.png" /></figure><h4><strong>Connecting via SSH to the EC2 in the Private Subnet</strong></h4><p>The next step was to connect via SSH to the EC2 instance in the private subnet in AZ1. Using the bastion host as an intermediary, I logged into the EC2 instance with the appropriate private key.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TNvRJQD6LlGActVClEutyQ.png" /></figure><h4><strong>Modifying Security Group Settings for EC2 in Private Subnet in AZ2</strong></h4><p>Next, I had to change the security group settings for the EC2 instance in the private subnet in AZ2 to allow pinging. I added a rule permitting ICMP (ping) traffic from the IP address of the EC2 instance in the private subnet in AZ1. This allowed me to verify connectivity between these two EC2 instances.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZXizpfRsZvhFwychZXYeyA.png" /></figure><h4><strong>Verifying Connectivity Between EC2 Instances</strong></h4><p>After configuring the security rules, I executed the ping command from the EC2 instance in the private subnet in AZ1 to the EC2 instance in the private subnet in AZ2. The <strong>output showed successful connectivity between the instances</strong>, confirming the correct network setup.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3o_UUENc1PjzNpUz94VZRw.png" /></figure><p>Thank you for reading my blog!</p><p>See you next time!</p><p>Dominik</p><p>Let’s connect via <a href="https://www.linkedin.com/in/dominik-zygarski/"><strong>LinkedIn</strong></a><strong>.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e7cbfe9aa45b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Exploring AWS VPC Infrastructure & IAM.]]></title>
            <link>https://medium.com/@dominikzygarski_88070/cloud-engineer-academy-module-5-part-1-exploring-aws-vpc-infrastructure-iam-cb4406906bf1?source=rss-b563547878e3------2</link>
            <guid isPermaLink="false">https://medium.com/p/cb4406906bf1</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <dc:creator><![CDATA[Dominik Zygarski]]></dc:creator>
            <pubDate>Mon, 03 Jun 2024 17:39:23 GMT</pubDate>
            <atom:updated>2025-03-17T18:10:25.721Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HwAZ0KxTXK7f6mQxZx70yw.jpeg" /></figure><p>In <strong>Module 5</strong> of the course at the Cloud Engineer Academy, I familiarized myself with the <strong>infrastructure</strong> of AWS Virtual Private Cloud (<strong>VPC</strong>). This module helped me understand the key components and services that enable the creation and management of isolated network environments in the cloud. AWS VPC offers the flexibility, security, and control necessary for effective cloud resource management. In the following sections of this post, I will share details about elements such as Subnets, IAM, Route Tables, and other important AWS functions and services.</p><h4>AWS Regions</h4><p>AWS Regions are geographic locations where Amazon Web Services (AWS) has data centers. Each of these regions is isolated from the others, ensuring a high level of reliability and stability of AWS services. Regions are divided into Availability Zones, which are separate data centers within a single region, connected by fast, private links. This architecture allows AWS to offer high availability and fault tolerance. Choosing the right region is crucial for application performance and compliance with local data storage regulations.</p><h4>AWS Availability Zones</h4><p>Availability Zones are distinct data centers located within a single AWS Region. Each Availability Zone is housed in separate buildings or facilities, providing physical separation and independence. This design ensures high availability and fault tolerance, as a failure in one Availability Zone does not impact the others within the same region. Availability Zones are connected by fast, private links, allowing for data and application synchronization between them. Utilizing multiple Availability Zones enhances application availability and minimizes the risk of downtime.</p><h4>AWS Local Zones</h4><p>Local Zones are extensions of AWS Regions that are placed closer to end users in selected locations. Local Zones provide low latency and high performance, which is crucial for applications requiring immediate data processing, such as online gaming, video streaming, and virtual and augmented reality applications. Local Zones enable the deployment of compute and storage resources closer to where they are needed, significantly enhancing service quality for end users. Utilizing Local Zones is also beneficial for compliance with data residency regulations.</p><h4>Public Subnet</h4><p>A public subnet is a subnet within a VPC that is accessible from the internet. For a subnet to be considered public, it must have a route table associated with it that directs traffic to an Internet Gateway (IGW). Public subnets typically house resources that need to be directly accessible from the internet, such as web servers that host websites or APIs. Public subnets are essential for applications that require direct communication with users or other services available on the internet.</p><h4>Private Subnet</h4><p>A private subnet is a subnet within a VPC that is not directly accessible from the internet. For a subnet to be considered private, it should not have a route in its route table that directs traffic to an Internet Gateway (IGW). Resources placed in a private subnet communicate with the internet through a NAT Gateway located in a public subnet. Private subnets are ideal for resources that should not be publicly accessible, such as databases, application servers, or other internal services that require a high level of security.</p><h4>AWS VPC</h4><p>VPC (Virtual Private Cloud) is a virtual network in the cloud that is logically isolated from other networks in the cloud. It allows you to create and manage your own network within cloud resources, providing control over network configuration, including selecting IP address ranges, creating subnets, and configuring route tables and gateways. VPC also enables secure connections to other local networks through VPN (Virtual Private Network) or AWS Direct Connect. With VPC, you can build a scalable and secure cloud environment that meets specific business and technical needs.</p><h4>Internet Gateway</h4><p>An Internet Gateway (IGW) is a scalable, highly available gateway that enables communication between instances in a VPC and the internet. The Internet Gateway acts as a bridge, allowing instances in public subnets to receive and send traffic to and from the internet. Each VPC can have only one Internet Gateway, which is redundantly and independently scalable. The IGW requires no special configuration and automatically handles incoming and outgoing traffic. Combined with appropriate routing rules and security settings, the IGW enables the creation of publicly accessible web applications and services.</p><h4>NAT Gateway</h4><p>A NAT Gateway (Network Address Translation Gateway) is a managed service that allows resources in private subnets to connect to the internet or other AWS services without exposing those resources to the internet. NAT Gateway enables instances in private subnets to initiate outbound traffic while preventing inbound traffic that is not a response to the outbound requests. This service is scalable and highly available, automatically adjusting to network traffic loads. NAT Gateway eliminates the need for manually configuring and managing NAT instances, simplifying network management and enhancing security.</p><h4>Security Groups</h4><p>Security Groups are virtual firewalls that control inbound and outbound traffic for resources within a VPC. Each resource, such as an EC2 instance, can be associated with one or more security groups. Security Groups define rules that specify what traffic is allowed or denied. These rules can be created for both inbound (ingress) and outbound (egress) traffic. Security Groups are stateful, meaning they automatically track the state of allowed connections and apply the appropriate rules. By using Security Groups, you can precisely control access to cloud resources, thereby enhancing the security of your applications.</p><h4>Route Tables</h4><p>Route tables are a critical component in managing network traffic within a VPC. Route tables define how network traffic is directed within the VPC and between the VPC and other networks. Each route table contains a set of rules (routes) that specify destination IP addresses and the corresponding gateways or network devices through which the traffic should be routed. In AWS, each subnet must be associated with one route table, allowing control over the traffic to and from that subnet. By configuring route tables appropriately, you can ensure proper data flow, control access to resources, and enhance network security and performance.</p><h4>CIDR Blocks</h4><p>CIDR (Classless Inter-Domain Routing) Blocks are a method for allocating and managing IP addresses in a network. CIDR allows for more flexible and efficient allocation of IP addresses compared to traditional class-based methods. A CIDR block is defined by an IP address and a network prefix (e.g., 192.168.0.0/16), where the number after the slash indicates the number of bits in the network mask. CIDR Blocks are crucial in managing virtual networks, such as VPCs in AWS, as they allow for precise determination of IP address ranges for subnets, which helps in optimizing IP address usage and providing better network scalability.</p><h4>NACLs</h4><p>NACLs (Network Access Control Lists) are optional layers of security for VPCs that control traffic to and from one or more subnets. Unlike Security Groups, which are assigned to individual resources, NACLs are associated with subnets and act as stateless firewalls. NACLs contain ordered rules for both inbound (ingress) and outbound (egress) traffic. Each rule is evaluated based on its number, and NACLs process the rules in ascending order, stopping at the first matching rule. NACLs are stateless, meaning rules must be defined separately for inbound and outbound traffic. NACLs are useful for securing entire subnets and providing an additional layer of network traffic control.</p><h4>AWS IAM</h4><p>IAM (Identity and Access Management) is a cloud service that allows you to manage access to cloud resources. IAM enables you to create and manage users, groups, and roles, and assign them appropriate permissions to AWS resources. With IAM, you can precisely control who has access to which resources and to what extent. This facilitates secure management of cloud resources, minimizing the risk of unauthorized access. IAM also offers the ability to configure policies that define permissions at a very granular level, allowing for security customizations tailored to the specific needs of an organization.</p><p><strong>Crucial components:</strong></p><p><strong>Users</strong></p><p>IAM Users are individual identities assigned to specific people or applications that need access to AWS resources. Each IAM user has a unique name and can have long-term credentials, such as a password for the AWS Management Console and access keys for using AWS APIs. Users can be added to IAM groups and have roles and policies assigned to them that define their permissions.</p><p><strong>Groups</strong></p><p>IAM Groups are collections of IAM users that make it easier to manage permissions for multiple users at once. Groups allow you to assign permissions collectively, meaning that the permissions granted to the group are inherited by all its members. This simplifies access management, especially in large organizations.</p><p><strong>Roles</strong></p><p>IAM Roles are sets of permissions that can be assigned to any AWS identity, such as IAM users, AWS services, or external accounts. IAM Roles allow for granting access without sharing long-term credentials. Roles are particularly useful for situations where temporary authorization is needed, such as for applications running on EC2 instances.</p><p><strong>Policies</strong></p><p>IAM Policies are JSON documents that define sets of permissions for users, groups, and roles. Policies specify what actions are allowed or denied on specified resources. Policies can be centrally managed (managed policies) or created for specific users (inline policies). With IAM Policies, you can precisely control access to AWS resources according to the organization’s security policies.</p><p>To conclude, I would like to present the design of a VPC infrastructure that illustrates a web application. The primary focus was not on the application itself, but on designing the VPC to ensure a secure, scalable, and efficient environment for the web application. This design includes various components such as public and private subnets, an Internet Gateway, a NAT Gateway, route tables and security groups. These components work together to provide the necessary level of security and functionality for the web application hosted on AWS.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*99tbidqrHA94LpMtnHSPNA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*j5Y6HMAyV2CSsAlL_HszIw.png" /></figure><p><strong>Internet Gateway (IGW):</strong></p><ul><li>Traffic from the web application passes through the Internet Gateway, enabling communication between VPC resources and the internet.</li></ul><p><strong>Public Subnet A:</strong></p><ul><li>Public Subnet A contains resources such as a Load Balancer, which receives incoming traffic from users on the internet.</li></ul><p><strong><em>Security Group 1:</em></strong></p><ul><li>Security rules allow inbound and outbound traffic on all ports and protocols to and from any IP address (0.0.0.0/0).</li></ul><p><strong>Private Subnet B:</strong></p><ul><li>Resources in Private Subnet B handle application logic and are accessible only through secure connections, with no direct internet access.</li></ul><p><strong><em>Security Group 2:</em></strong></p><ul><li>Subnet B allows mutual <strong>communication between resources</strong> within Subnet B, <strong>inbound and outbound </strong>communication with Subnet C, and <strong>outbound</strong> traffic to<strong> </strong>Subnet A.</li><li>An <strong>SSH port</strong> is open for a specified <strong>administrator’s IP.</strong></li></ul><p><strong>Private Subnet C:</strong></p><ul><li>Databases in Private Subnet C are accessible only by EC2 instances in private subnets or other internal VPC resources.</li></ul><p><strong><em>Security Group 3:</em></strong></p><ul><li>Subnet C allows mutual <strong>communication between its resources</strong>, <strong>inbound and outbound</strong> communication with Subnet B, and <strong>outbound</strong> traffic to Subnet A.</li><li>An <strong>SSH port</strong> is open for a specified <strong>administrator’s IP.</strong></li></ul><p><strong>IAM (Identity and Access Management):</strong></p><ul><li>IAM manages the identities of users and resources, as well as permissions that control access to different parts of the VPC infrastructure.</li><li>IAM users and roles have policies assigned that specify what actions they can perform on AWS resources, further securing the entire infrastructure.</li></ul><p>I recommend familiarizing yourself with the AWS Official Documentation, as it was very helpful to me throughout this project.</p><p><a href="https://docs.aws.amazon.com/vpc/latest/userguide/vpc-security-groups.html">https://docs.aws.amazon.com/vpc/latest/userguide/vpc-security-groups.html</a></p><p>Thank you for reading my blog!</p><p>See you next time!</p><p>Dominik</p><p>Let’s connect via <a href="https://www.linkedin.com/in/dominik-zygarski/"><strong>LinkedIn</strong></a><strong>.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cb4406906bf1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Docker Adventure- Exploring New DevOps Horizons.]]></title>
            <link>https://medium.com/@dominikzygarski_88070/docker-adventure-exploring-new-devops-horizons-part-1-decf90acbc0d?source=rss-b563547878e3------2</link>
            <guid isPermaLink="false">https://medium.com/p/decf90acbc0d</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <dc:creator><![CDATA[Dominik Zygarski]]></dc:creator>
            <pubDate>Tue, 30 Apr 2024 07:39:59 GMT</pubDate>
            <atom:updated>2025-03-17T18:09:36.356Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/934/1*tLQStKX0WnNZdMEq0Xf0eg.png" /></figure><p>During my journey of learning at the Cloud Engineer Academy, I noticed the need for a brief break from the intense focus on AWS. In search of new skills and tools to complement my knowledge in cloud engineering and DevOps, I stumbled upon Docker. Inspired by the potential of this tool, I decided to delve into its intricacies by enrolling in a course on the Udemy platform.</p><p>Docker is an open-source platform that enables automation of deploying, scaling, and managing applications in containers. Containers are portable software units that contain everything needed to run an application, including code, dependencies, system libraries, and tools. With Docker, developers can easily package their applications into containers, ensuring a consistent runtime environment regardless of the platform on which the application will run. Docker also facilitates rapid application development, testing, and deployment, speeding up the project lifecycle and increasing deployment efficiency.</p><h4><strong>Containers.</strong></h4><p>Containers are software units that contain all the necessary resources to run an application, including code, dependencies, system libraries, and tools. Through containerization technology, applications are isolated from each other and the environment, enabling portability and consistency across different platforms. Containers are lighter than virtual machines, allowing for faster startup times and more efficient hardware resource utilization.</p><h4><strong>Installation.</strong></h4><p>To install Docker on macOS, you first need to download the appropriate version from the official Docker Hub website. After downloading and installing Docker, you can check if the installation was successful by running the command docker -v. Since I already have it installed on my system, I can show you my Docker version using this command.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*T-FXaiNjFtF-rYGxqpbbhQ.png" /></figure><p>If you’re on macOS and want to install Docker, visit the <a href="https://docs.docker.com/engine/install/"><strong>official Docker website</strong></a> and follow the installation instructions for your operating system.</p><h4><strong>Basic Docker Commands and DockerHUB.</strong></h4><p>The first step in working with Docker is understanding basic commands, starting with docker run. However, before we run our first application in a container, we need access to the appropriate images. Docker Hub is the central repository for public Docker images, containing thousands of ready-to-use images of various applications, tools, and operating systems.</p><p>For instance, we will install CentOS, so for that purpose, we need to visit the <a href="https://hub.docker.com/"><strong>Docker Hub website</strong></a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yDPW1DX9VEBzL70hnIh6Qg.png" /></figure><p>To pull the CentOS image from Docker Hub, we’ll use the docker pull centos command, which will download the latest version of the CentOS image from the official repository. Then, we&#39;ll be able to use this image to run a container on our system.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WXsljPNTHUBuWlumL-KETw.png" /></figure><p>Containers can be launched using the docker run command, and adding the -it option, as in docker run -it, enables interactive connection to the container. For example, using docker run -it centos bash allows us to immediately execute commands in the Bash shell of the container.</p><p>Now we can exit from CentOS.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sIpC4jPQvQMZVK8c75fzTA.png" /></figure><p>The next useful command is docker ps, which displays a list of active containers. However, it&#39;s worth noting that if we previously stopped our container, it won&#39;t be visible in the list.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*USYL17gLoa2v7A012ql0aA.png" /></figure><p>If we want the container to be visible in the output of the docker ps command, you can run it in the background using the docker run -d command. Additionally, to set its execution for a specified amount of time, we can add the sleep command to our docker run command. For example: docker run -d centos sleep infinity.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*I9PUZKAqEO_fKfB5ctDMNQ.png" /></figure><p>If we want to stop a container, we use the docker stop command and provide the container&#39;s identifier. For example: docker stop container_id. This command allows for the controlled termination of the container, which can be useful, for instance, when interrupting the application&#39;s operation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xMaRlcBCZjQ96XpLq3dwNw.png" /></figure><p>If we want to display a list of all containers, we use the docker ps -a command. This command will show both active and stopped containers, allowing us to better manage them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ztYVG7-AkaCxEC1z_M55cw.png" /></figure><p>If we no longer need those containers visible on the list displayed by docker ps -a, we can remove them using the docker rm command.</p><p>Additionally, we can delete multiple containers at once.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*76waH5di7paVucakvPavxA.png" /></figure><p>Similarly to how the docker ps command is used to display a list of Docker containers, the docker images command is used to display a list of Docker images. Docker images are predefined software packages containing all the necessary resources to run an application. By using the docker images command, we can see the available images on our system, along with their names, identifiers, sizes, and versions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1mcl9Vl9aJ4C6h_92Lus0A.png" /></figure><p>Removing Docker images is done using the docker rmi command, where rmi stands for &quot;remove image&quot;. This command allows you to delete specific images from the local Docker repository. We can specify the image identifiers or their names as arguments for this command, and we can also delete multiple images at once by providing several identifiers or names.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2z4H9tv3ciOC3kq7JylqPA.png" /></figure><p>In this case, I used a container ID shortcut to delete it, which is also something we can do with containers.</p><p>Today’s blog post covered the basic Docker commands essential for working with containers and images. We discussed the process of launching containers, managing them, as well as removing images and containers. We also learned about the ability to use shortcuts for container and image identifiers for convenience in managing them. Exploring these commands has provided me with a better understanding of the fundamentals of working with Docker, enabling me to use this technology more effectively in my projects.</p><p>I would like to thank you for reading this post and invite you to visit the next ones. I hope you will find the following content equally interesting and helpful.</p><p>See you next time!</p><p>Dominik</p><p>Let’s connect via <a href="https://www.linkedin.com/in/dominik-zygarski/"><strong>LinkedIn</strong></a><strong>.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=decf90acbc0d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Linux & Bash.]]></title>
            <link>https://medium.com/@dominikzygarski_88070/cloud-engineer-academy-module-4-linux-bash-d1bf95c505a1?source=rss-b563547878e3------2</link>
            <guid isPermaLink="false">https://medium.com/p/d1bf95c505a1</guid>
            <category><![CDATA[linux]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <dc:creator><![CDATA[Dominik Zygarski]]></dc:creator>
            <pubDate>Wed, 20 Mar 2024 15:07:22 GMT</pubDate>
            <atom:updated>2025-03-17T18:10:07.074Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HtUwp3yKNczD_AA5kOqCEQ.png" /></figure><p>My journey in IT began with learning Linux on the Udemy platform. This was my first step towards becoming a cloud technology specialist. Thanks to this, I was able to quickly and efficiently get through the <strong>Linux &amp; Bash module</strong> at the <strong>Cloud Engineer Academy</strong>. Starting with the basics on Udemy, I built a strong foundation that proved to be extremely helpful in further learning and practice.</p><h4>Introduction to Linux.</h4><p>Linux is a family of open-source operating systems widely used in the field of computer science and technology. Its popularity stems from several factors, including stability, flexibility, and security. Linux also offers a wide range of tools and applications that allow users to customize the system to their individual needs.</p><h4>What is Bash?</h4><p>Bash, short for <strong>Bourne Again Shell</strong>, is one of the most widely used command-line interpreters in Unix and Linux systems. It is an environment where users can input and execute text commands, allowing them to manage files, run programs, and manipulate processes. Bash extends the capabilities of its predecessor, <strong>sh (Bourne Shell)</strong>, offering enhanced script programming features such as program flow control, conditional testing, and loops. With these features, Bash has become the standard for terminal work in Linux and Unix-based environments, serving not only as a tool for performing basic tasks but also as a powerful environment for task automation and advanced system management.</p><h4>Linux in AWS: Why It’s Key?</h4><p>Knowledge of Linux is essential for effectively using AWS, as many AWS services are based on Linux systems. Linux provides the flexibility, security, and stability that are crucial for cloud environments. Moreover, proficiency in Linux allows for better management and configuration of cloud instances, invaluable for building scalable and efficient applications. Therefore, for anyone looking to fully leverage the capabilities offered by AWS, an understanding of Linux is absolutely fundamental.</p><h4>Virtual Linux Machines on AWS: Accessibility and Flexibility Without Straining Your Own Resources.</h4><p>Thanks to AWS, I can launch an <strong>EC2 instance with the Linux system</strong> of my choice at any moment, giving me the freedom to experiment and learn new things. Connecting through an <strong>external terminal</strong> is straightforward and secure, which further facilitates working on diverse projects without the need for investing in expensive and advanced hardware.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5KMNdYT77vJVqvKdJT4DNg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eNwlTjr4f6UfzP07zxt-HQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5SZP4-Cav8JIYcbam8V6kw.png" /></figure><p>To leverage all these capabilities, it’s essential to have and configure a key pair. AWS uses these keys to secure access to EC2 instances. When creating a new instance, we can generate a new RSA key pair, one of the most secure authentication methods available. The public key is automatically installed on our EC2 instance, while the private key must be downloaded and kept in a secure location. It’s important to remember that the private key is displayed only once, at the time of the key pair creation. If we lose it, we won’t be able to connect to our instance, which would force us to generate a new key pair or use another access method.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wnZnRWMMGhFZXFOKsfTq5Q.png" /></figure><p>We also need to set up SSH access in the AWS security group by adding a rule for traffic on port 22, allowing secure connections to our EC2 instance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jDFpsmhmS8onqoB5wZjRzg.png" /></figure><p>Additionally, we can configure a static IP for our EC2 instance using the Elastic IP service in AWS. This allows us to connect to the instance with a single, unchanging IP address, which is especially useful when we turn the instance off and on, as otherwise, its IP address could change. While <strong>Elastic IP is a paid service</strong>, the costs are very low. Without a configured Elastic IP, it’s important to remember that the instance’s IP address may change with each restart.</p><h4>Connecting to an EC2 Instance via an External Terminal.</h4><p>I personally use <strong>Termius</strong> for connecting to EC2 instances, but there are many tools available for the same purpose. Mac users can take advantage of the built-in terminal, which offers SSH support directly from the command line. For Windows users, a popular choice is <strong>PuTTY</strong>, a free SSH client that facilitates a secure connection to a Linux server. Regardless of the operating system you’re using, it’s important to choose a tool that best fits your needs and provides secure access to remote machines.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*S8j-dR0axa1dwNVgFCsmuw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EiEte40d9z2jOO5IrRak6A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qaTk1cV0Pe97RerOsRkrQw.png" /></figure><p>Through the integration of AWS and applications like Termius, I have gained seamless access to Linux systems directly from my own computer. This opens up new avenues for deepening knowledge and developing technical skills. I can freely explore the intricate aspects of Bash, as well as dive into advanced tools such as <strong>Docker</strong>, which are pivotal for my IT career, without overly burdening my computer’s system.</p><h4>Basic Bash Commands: The Key to Efficient Work with Linux.</h4><p>In my Linux journey through Bash, I’ve learned many basic commands that proved to be incredibly useful. They allowed me to efficiently manage files, directories, and processes, as well as customize my working environment to suit my needs. Now I can easily navigate the system, manipulate data, and control application behavior, which is invaluable in my daily work with Linux.</p><p>ls: Lists the files and directories in the current directory.<br>cd: Changes the current directory.<br>pwd: Prints the current working directory.<br>touch: Creates a new file.<br>grep: Searches for patterns in text files and prints matching lines.<br>mkdir: Creates a new directory.<br>rm: Removes (deletes) a file or directory.<br>chmod: Changes the permissions of a file or directory.<br>vim: Opens the Vim text editor.<br>more: Displays the contents of a text file one screen at a time.<br>man: Displays the manual pages for a command.<br>cat: Concatenates and displays the content of files.<br>cp: Copies files or directories.<br>mv: Moves or renames files or directories.<br>rm: Removes (deletes) files or directories.<br>ln: Creates links between files.<br>which: Displays the path of an executable.<br>expr: Evaluates expressions.<br>bc: Launches a calculator.<br>alias: Creates aliases for commands.<br>du: Displays disk usage of files and directories.<br>df: Displays disk space usage of file systems.<br>ps: Displays information about processes.<br>pstree: Displays a tree of processes.<br>top: Displays real-time information about system processes.<br>kill: Terminates processes by sending signals.<br>/dev/null: A special file that discards all data written to it.</p><p>Summarizing, in today’s blog post, I discussed the basic Linux commands I’ve learned and shared my experience with configuring EC2 instances in the AWS cloud and using the Termius tool for remote connections to EC2 instances. Grasping these fundamental commands and learning to navigate cloud services have empowered me to further explore Linux and deepen my understanding of it.</p><p>Thank you for taking the time to read my blog! I trust you found this post informative and useful.</p><p>Looking forward to seeing you in the next post!</p><p>Dominik</p><p>Let’s connect via <a href="https://www.linkedin.com/in/dominik-zygarski/"><strong>LinkedIn</strong></a><strong>.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d1bf95c505a1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Project: Video Sharing Platform- Architecture Design.]]></title>
            <link>https://medium.com/@dominikzygarski_88070/project-video-sharing-platform-architecture-design-400e718f77b3?source=rss-b563547878e3------2</link>
            <guid isPermaLink="false">https://medium.com/p/400e718f77b3</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[youtube]]></category>
            <dc:creator><![CDATA[Dominik Zygarski]]></dc:creator>
            <pubDate>Sun, 17 Mar 2024 19:45:42 GMT</pubDate>
            <atom:updated>2024-03-20T15:07:59.775Z</atom:updated>
            <content:encoded><![CDATA[<p>I recently took on designing a serverless architecture for a <strong>video sharing application</strong>. It was a fantastic opportunity for me to dive deep into cloud technology and brush up on modern tech. Now, I’d like to share this project with you, tell you all about my design and how it all comes together into a cohesive whole that could actually work in the real world.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LGVm-28miyIrC4hX35xu8g.png" /></figure><h4>Frontend:</h4><p>The frontend is a <strong>static website</strong> hosted on <strong>AWS S3</strong>, serving as the user interface for the video sharing application. It is designed with simplicity, speed, and reliability in mind, allowing users to easily upload, browse, and play video content. Utilizing modern frameworks such as React or Angular, the site is responsive and provides a seamless user experience across various device types.</p><h4><strong>Video Processing:</strong></h4><p>A pivotal component of the project is the video processing system, which automatically converts uploaded content into various formats and resolutions. I utilized <strong>Kafka</strong>, a streaming platform, to manage the queue of video processing tasks. Kafka enables efficient real-time processing of large volumes of data, essential in a video-sharing application where users expect quick access to uploaded content. Upon video upload, it is automatically directed to the Kafka system, which then ensures it’s transformed into formats suitable for different devices and connection speeds, guaranteeing optimal video quality for every user.</p><h4>Video Analysis:</h4><p>To ensure all uploaded content is appropriate and safe for the community, I incorporated video analysis using <strong>AWS Recognition</strong> into the process. This service utilizes advanced machine learning algorithms to identify objects, people, text, scenes, and activities, as well as detect inappropriate content. When a user uploads a video, it is automatically sent to AWS Recognition, which analyzes the material for potential threats or inappropriate content. This solution allows me to quickly address any materials that could violate platform policies, making it safer for all users.</p><h4><strong>Databases for Metadata:</strong></h4><p>For storing video metadata and user information, I use <strong>AWS DynamoDB</strong> — a fast and flexible <strong>NoSQL</strong> database offering scalability and performance essential for a dynamically growing video-sharing platform. DynamoDB allows for easy management of large volumes of data without the need for dedicating resources to maintain database servers. Thanks to DynamoDB’s automatic scaling, the application can efficiently handle an increase in users as well as a rise in the volume of uploaded and viewed video content. Additionally, for fast access to frequently used data and its caching, I utilize <strong>Redis</strong>, which significantly improves the application’s response time, providing an even better experience for users.</p><h4>Storing Processed Video:</h4><p>After processing, videos are stored in <strong>AWS S3</strong>, which offers reliable and scalable object storage solutions. AWS S3 enables me to securely store processed video files in various formats and resolutions, crucial for delivering optimal content to users across different devices and with varying internet connection speeds. Using S3 facilitates easy management of file access, versioning, and rapid content distribution through integration with CDN (Content Delivery Network) services. This ensures users can enjoy fast and smooth access to videos, regardless of their location.</p><h4>Content Delivery Network and Cache:</h4><p>To optimize the delivery of video content to users worldwide, I utilize a <strong>CloudFront (CDN)</strong>. CDN reduces the distance data travels between the server and the user by storing copies of video content in multiple geographical locations. As a result, when a user requests access to a video, the content is delivered from the nearest server, significantly reducing latency and improving content loading speed.</p><p>Additionally, to further accelerate access to popular content and reduce the load on the main servers, I employ caching mechanisms. <strong>Cache</strong> temporarily stores frequently requested data, allowing for faster retrieval by users. In a video sharing application, caching is especially important for video thumbnails, metadata, and other elements that are often read. Both CDN and caching significantly contribute to enhancing the overall performance of the platform, providing users with a smooth and satisfying viewing experience.</p><h4>Video Searching and Delivery:</h4><p>To enhance the video search process in my application, I integrated <br><strong>AWS CloudSearch</strong>, a cloud search service that enables fast and precise searching across large datasets. CloudSearch indexes video metadata stored in the database, allowing users to easily find content using keywords, categories, or other filters. Thanks to its flexible scaling, CloudSearch efficiently handles an increasing number of search queries, ensuring smooth and rapid delivery of results.</p><p>Meanwhile, when a user selects a specific video to watch, requests to AWS S3, where the video files are stored and managed by <strong>AWS Lambda</strong>. Lambda is a compute service that executes code in response to events, such as a user’s request. This means that each video playback request is quickly processed, and the corresponding file is directly delivered to the user without the need for additional server infrastructure. Utilizing AWS Lambda not only minimizes latency but also optimizes costs, as you pay only for the code execution time in response to actual requests.</p><h4><strong>Horizontal Scalability:</strong></h4><p>In my project, achieving horizontal scalability is crucially supported by the use of <strong>Auto Scaling Groups</strong> in AWS. These groups automatically adjust the number of EC2 instances in response to changing loads, allowing for the efficient handling of an increase in user requests without manual intervention. Thus, the system can dynamically add new instances as the number of users or requests grows, and reduce resources when demand decreases.</p><p>Auto Scaling Groups ensure that the video-sharing application remains performant and available even during peak loads, which is key to maintaining a high-quality user experience. Horizontal scalability, supported by Auto Scaling Groups, is fundamental in the cloud, allowing the application to flexibly adapt to changing conditions without overloading individual servers or exceeding resource limits.</p><p>Additionally, services like S3, DynamoDB, and CloudSearch also offer built-in scalability, meaning they can handle an increase in queries and stored data without direct intervention. The integration with CDN continues to play a significant role in optimizing content delivery to users globally, ensuring content loads quickly and efficiently from various locations.</p><p>As a result, through the use of Auto Scaling Groups and other AWS services, the application’s architecture can scale horizontally in a smooth and efficient manner, responding to the growing needs of users, ensuring continuous and reliable operation of the platform.</p><h4>Security and Optimization:</h4><p>In the design of a video sharing application, security, and optimization are of paramount importance. Utilizing AWS services such as <strong>Identity and Access Management (IAM)</strong> for access and permissions management allows for detailed control over resource and operation access. <strong>AWS Shield</strong> and <strong>AWS WAF (Web Application Firewall)</strong> protect against DDoS attacks and other online threats, ensuring the continuous availability of services and protection against abuses.</p><p>For optimization, monitoring, and logging with <strong>Amazon CloudWatch</strong> enable performance tracking of the application and identification of potential issues. Automated scaling in response to performance metric changes is essential for maintaining optimal operation while managing costs.</p><p><strong>DevOps practices</strong>, including <strong>continuous integration and continuous deployment (CI/CD)</strong>, support the rapid and safe implementation of changes in the application. This allows for regular updates of functionalities and security patches, minimizing the risk of errors.</p><p>Security and optimization are ongoing processes that require constant monitoring and adjustments. By adhering to best practices and leveraging advanced tools offered by AWS, the application is secured and optimized for performance, ensuring readiness for future challenges.</p><h3>Summary:</h3><p>The design of a serverless architecture for a video sharing application demonstrates how the use of modern technologies and cloud services can revolutionize the way we create and scale web applications. Flexibility, security, and optimization are key elements of this project, each essential for building scalable and efficient systems in a dynamically changing digital environment.</p><p>The utilization of services such as AWS Lambda, S3, DynamoDB, Kafka, CloudSearch, along with CDN integration and the adoption of DevOps practices, highlight the significance of the cloud in application development. As a result, the architecture not only meets the demands of contemporary users looking for fast and secure solutions but also provides developers with tools for effective resource and cost management.</p><p>This project proves that with the right approach and use of available technologies, a solid foundation can be created for any web application that is ready to meet future challenges. Introducing serverless into the video sharing application architecture not only simplifies infrastructure management and scaling but also allows focusing on feature development, ultimately translating into a better user experience.</p><p>Thank you for taking the time to read this post. I hope I was able to provide valuable information and inspiration related to designing a serverless architecture for a video sharing application.</p><p>Feel free to leave a comment below. Your feedback is incredibly valuable to me, and I look forward to any discussions. Thank you again, and see you in the next posts!</p><p>Dominik</p><p>Let’s connect via <a href="https://www.linkedin.com/in/dominik-zygarski/"><strong>LinkedIn</strong></a><strong>.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=400e718f77b3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[System Design Applications. Architectures, APIs, and Beyond.]]></title>
            <link>https://medium.com/@dominikzygarski_88070/cloud-engineer-academy-module-3-system-design-applications-architectures-apis-and-beyond-af118496d9dc?source=rss-b563547878e3------2</link>
            <guid isPermaLink="false">https://medium.com/p/af118496d9dc</guid>
            <category><![CDATA[amazon-web-services]]></category>
            <category><![CDATA[api]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[serverless]]></category>
            <dc:creator><![CDATA[Dominik Zygarski]]></dc:creator>
            <pubDate>Mon, 11 Mar 2024 14:22:42 GMT</pubDate>
            <atom:updated>2025-03-17T18:08:21.920Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*w7z66Os5EQWpyphtXTaBjg.png" /></figure><p>I’ve recently immersed myself into the fascinating world of application design models. This exploration has expanded my understanding of the intricate structures and strategies that make robust and scalable applications possible.</p><p>I’ll break down these design approaches using an e-commerce application as an example.</p><p><strong>Three-Tier Architecture: Building Software Like a Layered Cake.</strong></p><p>Three-tier architecture is like a cake with three layers. Each layer has a specific job to do, just like each tier of a cake has a distinct flavor or texture.</p><p>The first layer, called the presentation layer, is like the icing on the cake. It’s what users see and interact with. This layer handles things like displaying web pages or responding to user input.</p><p>The second layer, known as the application layer, is like the filling of the cake. It’s where the “business logic” of the application lives — things like processing data or performing calculations. This layer makes decisions and does the heavy lifting behind the scenes.</p><p>The third layer, called the data layer, is like the cake base. It’s where all the data is stored and managed. This layer handles things like storing information in databases and retrieving it when needed.</p><p>By organizing an application into these three layers, you can keep things neat and organized, making it easier to manage and update. It also allows different parts of the application to be developed and maintained independently, which can improve efficiency and flexibility.</p><p><strong>Monolithic Architecture in Application Design.</strong></p><p>A monolithic architecture is like a big, single block of code. Everything the application needs to do is in this one big block, which makes it straightforward to build and test. It’s simple because you have everything in one place — like having all parts of a toy building set in one box.</p><p>But, if your application grows, this big block can become hard to handle. Imagine trying to find a single piece in a huge toy box — it takes time. This is the problem with scaling; as your application gets bigger, it’s harder to make changes or add new features. And when you do make changes, you often have to stop the whole application to update it, which isn’t always ideal. This can make your app less flexible and updating it can take longer than with other architectures.</p><p>It’s essential to note that many such apps need to transition from older legacy systems to modern cloud environments. This move requires careful integration with cloud services to ensure that the application can scale to meet demand, manage resources efficiently, and maintain cost-effectiveness. Our journey will explore how we can revamp a traditional monolithic application, making it cloud-compatible while keeping an eye on keeping costs down and performance up.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eAufflRk1s_Nq2Yizrbztw.png" /></figure><p>Advantages:<strong> ✅ Simplicity ✅ Uniformity</strong></p><p>Disadvantages:<strong> ❌ Scalability ❌ Flexibility ❌ Deployment</strong></p><p><strong>Microservices Architecture: Flexibility in your Application’s DNA.</strong></p><p>Microservices architecture is like a LEGO set. Instead of building one big model, you create lots of small blocks that can easily connect together. Each piece, or “microservice” does one thing really well, like handling user logins or managing the shopping cart in an online store.</p><p>The cool thing is, because each part is separate, you can change one without messing with the others. This makes adding features or fixing issues a lot faster and easier. It’s great for growing your app because you can just add new blocks as needed.</p><p>But there’s a tricky part too: you have to manage all these little pieces and make sure they work well together, which can get complex. However, if you want to be able to make changes quickly and scale your app as it gets more popular, microservices are a smart way to build it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*G8-uPkSpl-IGM3MEXLXbxw.png" /></figure><p>Advantages:<strong> ✅ Scalability ✅ Flexibility ✅ Resilience ✅ Iteration</strong></p><p>Disadvantages:<strong> ❌ Complexity ❌ Dependency</strong></p><p><strong>The Power of Serverless Architecture.</strong></p><p>Serverless architecture is like having a powerful tool that springs into action whenever you need it. It’s hosted in the cloud, so you don’t have to worry about managing any servers yourself. This means you can focus on building your application without getting bogged down in infrastructure management.</p><p>One of the great things about serverless architecture is that you only pay for the time your code is running. This can save you money compared to traditional hosting models where you pay for servers running all the time, whether you’re using them or not. Additionally, serverless functions are active only when triggered, which further optimizes costs.</p><p>Another benefit is scalability. Because serverless services run in the cloud, they can automatically scale up or down to handle any amount of traffic. This means your application can handle sudden spikes in usage without any manual intervention.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*glXM0vrPLNbYD9tW4UuB_g.png" /></figure><p>Advantages:<strong> ✅ Cloud managed ✅ Scalability ✅ Event triggered</strong></p><p>Disadvantages:<strong> ❌ Debugging</strong> <strong>❌ Security</strong></p><p><strong>Cloud Storage Options.</strong></p><p><strong><em>File Storage:</em></strong></p><ul><li>Characteristics: File storage systems organize data into a hierarchical structure, similar to how files are stored on a personal computer. Users access files through a network, such as a local area network (LAN) or the internet.</li><li>Use Cases: Ideal for storing and sharing files that need to be accessed by multiple users or applications simultaneously. Commonly used for shared documents, media files, and application data.</li><li>Pros: Familiar file system interface, suitable for collaboration and file sharing among users.</li><li>Cons: Limited scalability compared to object storage, may encounter performance issues with a large number of concurrent users or files.</li></ul><p><strong><em>Object Storage:</em></strong></p><ul><li>Characteristics: Object storage organizes data as objects within a flat namespace, rather than in a hierarchical structure like file storage. Each object includes metadata and a unique identifier, allowing for efficient storage and retrieval.</li><li>Use Cases: Ideal for storing unstructured data such as images, videos, backups, and logs. Well-suited for applications requiring massive scalability and durability.</li><li>Pros: Highly scalable, cost-effective, and offers high durability. Provides seamless integration with cloud-native applications and services.</li><li>Cons: May have higher latency for small, random access compared to block storage. Not as suitable for applications requiring frequent updates to stored data.</li></ul><p><strong><em>Block Storage:</em></strong></p><ul><li>Characteristics: Block storage divides data into uniformly sized blocks and stores them as separate volumes. It provides raw storage volumes that can be mounted to servers or virtual machines, allowing for direct access to individual blocks.</li><li>Use Cases: Ideal for applications requiring low-latency, high-performance storage, such as databases, virtual machines, and transactional workloads.</li><li>Pros: Offers high performance, low-latency access to data, making it suitable for I/O-intensive applications. Provides flexibility to allocate and manage storage volumes according to application requirements.</li><li>Cons: May be less cost-effective for storing large amounts of unstructured data compared to object storage. Requires more management overhead for provisioning and managing storage volumes.</li></ul><p>Each storage option has its own strengths and weaknesses, making them suitable for different use cases and application requirements. Understanding these characteristics can help in choosing the most appropriate storage solution for specific workloads and applications.</p><p><strong>Caching.</strong></p><p>Caching is used to speed up data retrieval, especially for frequently accessed information in applications. It’s employed when there’s a need to improve performance, reduce latency, and optimize resource utilization. Caching is particularly beneficial in scenarios where fast response times are crucial, such as web servers, databases, and real-time processing systems, as it helps to minimize the time and resources required to access data, leading to smoother and more efficient operations.</p><p><strong>Unlocking the Power of APIs.</strong></p><p>An API is a powerful tool that enhances the efficiency of software programs by enabling them to communicate seamlessly. It acts as a bridge between different applications, allowing them to exchange data and functionalities swiftly. This enhances flexibility as applications can adapt and evolve independently, knowing they can rely on the API to connect with other systems effortlessly. Integration becomes smoother and more effective, facilitating collaboration between diverse software components and driving innovation in the digital landscape.</p><p><strong>API vs Load Balancer</strong></p><p>API (Application Programming Interface) is a set of rules and protocols that allow different software applications to communicate with each other by defining how software components should interact.</p><p>A load balancer, on the other hand, is a device or software that distributes incoming network traffic across multiple servers or resources to ensure optimal resource utilization, improve reliability, and maintain high availability.</p><p>In summary, while APIs facilitate communication and data exchange between software components, load balancers manage incoming network traffic to ensure efficient resource utilization and high availability.</p><p><strong>Server vs Serverless Scaling.</strong></p><p><strong><em>Server Scaling:</em></strong></p><ul><li>Server scaling involves increasing or decreasing the capacity of physical or virtual servers to handle varying levels of workload.</li><li>In server scaling, additional resources such as CPU, memory, or storage are added to existing servers or new servers are provisioned to accommodate increased demand.</li><li>Scaling servers typically requires manual intervention and configuration to adjust resource allocation and ensure optimal performance.</li><li>This approach is suitable for applications with predictable or steady traffic patterns, where resource requirements can be estimated in advance.</li></ul><p><strong><em>Serverless Scaling:</em></strong></p><ul><li>Serverless scaling involves automatically adjusting the resources allocated to functions or services based on demand.</li><li>In serverless scaling, cloud providers manage the underlying infrastructure and dynamically allocate resources as needed to handle incoming requests.</li><li>Scaling in a serverless environment is automatic and transparent to the developer, with no need for manual intervention or configuration.</li><li>This approach is well-suited for applications with unpredictable or fluctuating traffic patterns, as resources are allocated on-demand and only for the duration of each function execution.</li></ul><p>In summary, server scaling involves manual adjustment of server resources to accommodate changes in workload, while serverless scaling relies on automatic resource allocation managed by cloud providers to handle varying levels of demand.</p><p>Thank you for reading my blog! I hope you found this post helpful. Stay tuned for more articles on various topics related to technology and computer science.</p><p>Your feedback is always appreciated. See you in the next post!</p><p>Dominik</p><p>Let’s connect via <a href="https://www.linkedin.com/in/dominik-zygarski/"><strong>LinkedIn</strong></a><strong>.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=af118496d9dc" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>