<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Sanjal S Eralil on Medium]]></title>
        <description><![CDATA[Stories by Sanjal S Eralil on Medium]]></description>
        <link>https://medium.com/@sanjal-eralil?source=rss-9e2186f06b63------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*LWS5nzTWIHbiKdRIi58BDQ.jpeg</url>
            <title>Stories by Sanjal S Eralil on Medium</title>
            <link>https://medium.com/@sanjal-eralil?source=rss-9e2186f06b63------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 23:20:59 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@sanjal-eralil/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Complete Guide to AWS VPC Peering: Connecting Private Networks Across AWS]]></title>
            <link>https://aws.plainenglish.io/complete-guide-to-aws-vpc-peering-connecting-private-networks-across-aws-18746372c850?source=rss-9e2186f06b63------2</link>
            <guid isPermaLink="false">https://medium.com/p/18746372c850</guid>
            <category><![CDATA[sanjal-s-eralil]]></category>
            <category><![CDATA[vpc-peering]]></category>
            <category><![CDATA[aws-vpc-peering]]></category>
            <dc:creator><![CDATA[Sanjal S Eralil]]></dc:creator>
            <pubDate>Mon, 16 Feb 2026 08:07:23 GMT</pubDate>
            <atom:updated>2026-02-26T17:07:11.523Z</atom:updated>
            <content:encoded><![CDATA[<p>In modern cloud architectures, connecting private networks securely is a fundamental challenge that every DevOps engineer encounters. Whether you’re building a multi-tier application spread across different VPCs, enabling shared services like centralized monitoring and logging, or integrating with third-party services that require private connectivity — the need to establish secure communication between isolated networks is inevitable.</p><p>Recently, I faced a real-world scenario where applications running across multiple development and testing environments needed to communicate with a license validation server hosted in a separate AWS account and region. The server was only accessible via private IP addresses, and we needed a solution that was secure, low-latency, and didn’t expose any services to the public internet. With over a dozen environments spanning multiple AWS regions — from US East to Europe — each requiring connectivity to the same central service, the challenge was both interesting and complex.</p><p><strong>The solution? AWS VPC Peering.</strong></p><p>VPC Peering emerged as the most effective approach for this use case. It provided direct, encrypted communication over AWS’s private backbone network — eliminating the need to route traffic through the public internet, configure VPNs, or expose endpoints publicly. The implementation taught me valuable lessons about network architecture, CIDR planning, DNS resolution in distributed systems, and the importance of understanding routing fundamentals.</p><p>In this comprehensive guide, I’ll share everything I learned about VPC Peering</p><h3>Table of Contents</h3><ol><li>Understanding VPCs and Subnets</li><li>What is VPC Peering?</li><li>When to Use VPC Peering</li><li>Setting Up VPC Peering: A Complete Walkthrough</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/909/1*zmBcKUDDNEvYRiXdaqI7QA.png" /><figcaption>Visual representation of the example explained in the article</figcaption></figure><h3>1. Understanding VPCs and Subnets</h3><h4>What is a VPC (Virtual Private Cloud)?</h4><p>A VPC is your <strong>isolated private network in AWS</strong>. Think of it as your own virtual data center in the cloud, completely separated from other AWS customers and even your own other VPCs.</p><p><strong>Key Components:</strong></p><ul><li><strong>CIDR Block</strong>: The IP address range for your network (e.g., 10.100.0.0/16)</li><li><strong>Subnets</strong>: Smaller network segments within the VPC</li><li><strong>Route Tables</strong>: Rules determining where network traffic goes</li><li><strong>Internet Gateway (IGW)</strong>: Enables direct internet access</li><li><strong>NAT Gateway</strong>: Enables outbound internet access for private resources</li></ul><h4>VPC Example: Application VPC</h4><pre>Application VPC<br>├─ VPC ID: vpc-app123456789<br>├─ CIDR: 10.100.0.0/16<br>├─ Region: us-east-1<br>├─ Account: 123456789012<br>└─ Contains: Web servers, databases, application pods</pre><h3>Public vs Private Subnets</h3><h4>Public Subnets</h4><p><strong>Purpose</strong>: Resources that need to be directly accessible from the internet or need direct outbound internet access.</p><p><strong>Characteristics:</strong></p><ul><li>Route to internet via <strong>Internet Gateway (IGW)</strong></li><li>Can receive traffic from the internet (if security groups allow)</li><li>Resources typically get public IP addresses</li><li>Common uses: Load balancers, bastion hosts, NAT gateways</li></ul><p><strong>Route Table Example:</strong></p><pre>Destination         Target              Purpose<br>-----------         ------              -------<br>10.100.0.0/16   →   local               VPC internal traffic<br>0.0.0.0/0       →   igw-12345678        Internet traffic (direct)</pre><h4>Private Subnets</h4><p><strong>Purpose</strong>: Resources that should NOT be directly accessible from the internet but need outbound internet access.</p><p><strong>Characteristics:</strong></p><ul><li>Route to internet via <strong>NAT Gateway</strong> (in a public subnet)</li><li>Cannot receive unsolicited traffic from internet</li><li>Resources use private IP addresses only</li><li>Common uses: Application servers, databases, container pods</li></ul><p><strong>Route Table Example:</strong></p><pre>Destination         Target              Purpose<br>-----------         ------              -------<br>10.100.0.0/16   →   local               VPC internal traffic<br>0.0.0.0/0       →   nat-87654321        Internet traffic (via NAT)</pre><h4>Complete VPC Architecture Example</h4><pre>Application VPC (10.100.0.0/16) - us-east-1<br>│<br>├── Public Subnets (2 for high availability)<br>│   ├── Public Subnet AZ1: 10.100.1.0/24 (us-east-1a)<br>│   │   ├── Application Load Balancer<br>│   │   ├── NAT Gateway #1<br>│   │   └── Route Table: rtb-public123<br>│   │       ├── 10.100.0.0/16 → local<br>│   │       └── 0.0.0.0/0 → igw-12345678<br>│   │<br>│   └── Public Subnet AZ2: 10.100.2.0/24 (us-east-1b)<br>│       ├── NAT Gateway #2<br>│       └── Route Table: rtb-public123 (shared with AZ1)<br>│<br>└── Private Subnets (2 for high availability)<br>    ├── Private Subnet AZ1: 10.100.50.0/24 (us-east-1a)<br>    │   ├── Application Pods (EKS)<br>    │   ├── Database Instances<br>    │   └── Route Table: rtb-private-az1<br>    │       ├── 10.100.0.0/16 → local<br>    │       └── 0.0.0.0/0 → nat-gateway-1<br>    │<br>    └── Private Subnet AZ2: 10.100.60.0/24 (us-east-1b)<br>        ├── Application Pods (EKS)<br>        ├── Database Instances<br>        └── Route Table: rtb-private-az2<br>            ├── 10.100.0.0/16 → local<br>            └── 0.0.0.0/0 → nat-gateway-2</pre><p><strong>Why 2 Availability Zones?</strong></p><ul><li>High availability: If one AZ fails, workloads continue in the other</li><li>Separate NAT Gateways: If NAT in AZ1 fails, AZ2 pods still have internet access</li></ul><h3>2. What is VPC Peering?</h3><h4>Definition</h4><p>VPC Peering creates a <strong>private network connection</strong> between two VPCs using AWS’s internal network infrastructure. Once established, resources in peered VPCs can communicate using private IP addresses as if they were in the same network.</p><h4>Key Characteristics</h4><p><strong>Private Connection:</strong></p><ul><li>Traffic never traverses the public internet</li><li>Uses AWS’s private global network backbone</li><li>Encrypted in transit automatically</li></ul><p><strong>Non-Transitive:</strong></p><ul><li>If VPC-A peers with VPC-B, and VPC-B peers with VPC-C</li><li>VPC-A CANNOT reach VPC-C through VPC-B</li><li>Each peering relationship must be explicit</li></ul><p><strong>One-to-One:</strong></p><ul><li>Each peering connection links exactly two VPCs</li><li>To connect 3 VPCs, you need 3 peering connections (A↔B, B↔C, A↔C)</li></ul><p><strong>Flexible:</strong></p><ul><li>Works across regions (cross-region peering)</li><li>Works across AWS accounts (cross-account peering)</li><li>Works within the same region (intra-region peering)</li></ul><h4>Visual: Without VPC Peering</h4><pre>┌─────────────────────────┐              ┌──────────────────────────┐<br>│  Application VPC        │              │  Service VPC             │<br>│  10.100.0.0/16          │              │  10.200.0.0/16           │<br>│  us-east-1              │              │  us-west-2               │<br>│                         │              │                          │<br>│  ┌──────────────┐       │    ❌ No     │  ┌─────────────┐         │<br>│  │ Web App      │       │   Direct    │  │ Database    │         │<br>│  │ 10.100.50.10 │──────►│    Path     │  │ 10.200.10.5 │         │<br>│  └──────────────┘       │              │  └─────────────┘         │<br>│                         │              │                          │<br>└─────────────────────────┘              └──────────────────────────┘</pre><pre>Must use: VPN, public internet, or no connectivity</pre><h4>Visual: With VPC Peering</h4><pre>┌─────────────────────────┐              ┌──────────────────────────┐<br>│  Application VPC        │   Peering    │  Service VPC             │<br>│  10.100.0.0/16          │◄───────────► │  10.200.0.0/16           │<br>│  us-east-1              │ pcx-abc123   │  us-west-2               │<br>│                         │              │                          │<br>│  ┌──────────────┐       │    ✅        │  ┌─────────────┐         │<br>│  │ Web App      │       │   Private    │  │ Database    │         │<br>│  │ 10.100.50.10 │──────►│    Path      │►─│ 10.200.10.5 │         │<br>│  └──────────────┘       │              │  └─────────────┘         │<br>│                         │              │                          │<br>└─────────────────────────┘              └──────────────────────────┘</pre><pre>Direct private communication via AWS backbone</pre><h3>3. When to Use VPC Peering</h3><h4>Ideal Use Cases</h4><p>✅ <strong>Connecting application tiers across VPCs</strong></p><ul><li>Front-end VPC needs to access back-end database VPC</li><li>Microservices in different VPCs need to communicate</li></ul><p>✅ <strong>Shared services architecture</strong></p><ul><li>Multiple application VPCs accessing a central monitoring/logging VPC</li><li>Multiple environments (dev, test, prod) accessing shared services</li></ul><p>✅ <strong>Cross-account access</strong></p><ul><li>Development account VPCs accessing production account resources</li><li>Customer VPCs accessing vendor/partner VPCs (your scenario!)</li></ul><p>✅ <strong>Cross-region replication</strong></p><ul><li>Database replication between regions</li><li>Disaster recovery architectures</li></ul><h4>When NOT to Use VPC Peering</h4><p>❌ <strong>Overlapping IP ranges</strong> (use PrivateLink instead)</p><p>❌ <strong>Transitive routing needed</strong> (use Transit Gateway instead)</p><p>❌ <strong>Connecting many VPCs</strong> (use Transit Gateway for hub-and-spoke)</p><p>❌ <strong>Public service endpoints</strong> (use PrivateLink or public load balancers)</p><h3>4. Setting Up VPC Peering: A Complete Walkthrough</h3><p>Let’s walk through a real-world example connecting two VPCs.</p><h4>Scenario</h4><p><strong>Application VPC</strong> (Requester):</p><ul><li>VPC ID: vpc-app123456789</li><li>CIDR: 10.100.0.0/16</li><li>Region: us-east-1</li><li>Account: 123456789012</li><li>Contains: Web application that needs to access a database</li></ul><p><strong>Database VPC</strong> (Accepter):</p><ul><li>VPC ID: vpc-db987654321</li><li>CIDR: 10.200.0.0/16</li><li>Region: us-west-2</li><li>Account: 987654321098</li><li>Contains: PostgreSQL database server</li></ul><p><strong>Goal</strong>: Enable the web application to securely connect to the database over private IPs.</p><h4>Phase 1: Pre-Flight Checks</h4><h4>Step 1.1: Verify No CIDR Overlap</h4><p>bash</p><pre># Check Application VPC CIDR<br>aws ec2 describe-vpcs \<br>  --vpc-ids vpc-app123456789 \<br>  --region us-east-1 \<br>  --query &#39;Vpcs[0].CidrBlock&#39; \<br>  --output text</pre><pre># Output: 10.100.0.0/16</pre><pre># Check Database VPC CIDR<br>aws ec2 describe-vpcs \<br>  --vpc-ids vpc-db987654321 \<br>  --region us-west-2 \<br>  --query &#39;Vpcs[0].CidrBlock&#39; \<br>  --output text</pre><pre># Output: 10.200.0.0/16</pre><p><strong>Verification</strong>: 10.100.0.0/16 and 10.200.0.0/16 do NOT overlap ✅</p><h4>Step 1.2: Identify Route Tables (Application VPC)</h4><p>bash</p><pre># List all route tables in Application VPC<br>aws ec2 describe-route-tables \<br>  --filters &quot;Name=vpc-id,Values=vpc-app123456789&quot; \<br>  --region us-east-1 \<br>  --query &#39;RouteTables[*].[RouteTableId,Tags[?Key==`Name`].Value|[0],Associations[0].Main]&#39; \<br>  --output table<br>```</pre><pre>**Example Output:**<br>```<br>RouteTableId          Name                      Main<br>rtb-pub123           app-public-subnets        False<br>rtb-priv-az1         app-private-subnet-az1    False<br>rtb-priv-az2         app-private-subnet-az2    False<br>rtb-main456          (none)                    True</pre><p><strong>Identify which route tables to update</strong>: All non-main tables (3 in this example)</p><h4>Phase 2: Create and Accept Peering Connection</h4><h4>Step 2.1: Create Peering Request (From Application VPC Account)</h4><p>bash</p><pre># Create VPC peering connection<br>aws ec2 create-vpc-peering-connection \<br>  --vpc-id vpc-app123456789 \<br>  --peer-vpc-id vpc-db987654321 \<br>  --peer-owner-id 987654321098 \<br>  --peer-region us-west-2 \<br>  --region us-east-1 \<br>  --tag-specifications &#39;ResourceType=vpc-peering-connection,Tags=[{Key=Name,Value=app-to-database-peering}]&#39;</pre><p><strong>Output:</strong></p><pre>{<br>    &quot;VpcPeeringConnection&quot;: {<br>        &quot;VpcPeeringConnectionId&quot;: &quot;pcx-example12345&quot;,<br>        &quot;Status&quot;: {<br>            &quot;Code&quot;: &quot;initiating-request&quot;,<br>            &quot;Message&quot;: &quot;Initiating Request to 987654321098&quot;<br>        }<br>    }<br>}</pre><p><strong>Save the Peering Connection ID</strong>: pcx-example12345</p><p><strong>What just happened:</strong></p><ul><li>AWS created a peering connection object</li><li>Sent notification to the accepter account</li><li>Status: pending-acceptance</li><li>No traffic flows yet</li></ul><h4>Step 2.2: Accept Peering Request (From Database VPC Account)</h4><p><strong>Switch to the Database VPC account</strong>, then:</p><pre># Accept the peering connection<br>aws ec2 accept-vpc-peering-connection \<br>  --vpc-peering-connection-id pcx-example12345 \<br>  --region us-west-2</pre><p><strong>Output:</strong></p><pre>{<br>    &quot;VpcPeeringConnection&quot;: {<br>        &quot;Status&quot;: {<br>            &quot;Code&quot;: &quot;provisioning&quot;,  // Changes to &quot;active&quot; in seconds<br>            &quot;Message&quot;: &quot;Provisioning&quot;<br>        }<br>    }<br>}</pre><p><strong>Verify it’s active:</strong></p><pre>aws ec2 describe-vpc-peering-connections \<br>  --vpc-peering-connection-ids pcx-example12345 \<br>  --region us-west-2 \<br>  --query &#39;VpcPeeringConnections[0].Status.Code&#39; \<br>  --output text</pre><pre># Output: active</pre><h4>Phase 3: Configure Routing (Both Sides)</h4><h4>Step 3.1: Add Routes in Application VPC (Requester Side)</h4><p><strong>Add route to Database VPC CIDR in each route table:</strong></p><p><strong>Public Subnets Route Table:</strong></p><p>bash</p><pre>aws ec2 create-route \<br>  --route-table-id rtb-pub123 \<br>  --destination-cidr-block 10.200.0.0/16 \<br>  --vpc-peering-connection-id pcx-example12345 \<br>  --region us-east-1</pre><p><strong>Private Subnet AZ1 Route Table:</strong></p><p>bash</p><pre>aws ec2 create-route \<br>  --route-table-id rtb-priv-az1 \<br>  --destination-cidr-block 10.200.0.0/16 \<br>  --vpc-peering-connection-id pcx-example12345 \<br>  --region us-east-1</pre><p><strong>Private Subnet AZ2 Route Table:</strong></p><pre>aws ec2 create-route \<br>  --route-table-id rtb-priv-az2 \<br>  --destination-cidr-block 10.200.0.0/16 \<br>  --vpc-peering-connection-id pcx-example12345 \<br>  --region us-east-1</pre><p><strong>Verify routes were added:</strong></p><pre>aws ec2 describe-route-tables \<br>  --filters &quot;Name=vpc-id,Values=vpc-app123456789&quot; \<br>          &quot;Name=route.destination-cidr-block,Values=10.200.0.0/16&quot; \<br>  --region us-east-1 \<br>  --query &#39;RouteTables[*].[RouteTableId,Routes[?DestinationCidrBlock==`10.200.0.0/16`].State|[0]]&#39; \<br>  --output table<br>```</pre><pre>**Expected Output:**<br>```<br>RouteTableId      State<br>rtb-pub123        active<br>rtb-priv-az1      active<br>rtb-priv-az2      active</pre><h4>Step 3.2: Add Routes in Database VPC (Accepter Side)</h4><p><strong>Switch to Database VPC account</strong>, then identify route tables:</p><p>bash</p><pre>aws ec2 describe-route-tables \<br>  --filters &quot;Name=vpc-id,Values=vpc-db987654321&quot; \<br>  --region us-west-2 \<br>  --query &#39;RouteTables[*].[RouteTableId,Tags[?Key==`Name`].Value|[0],Associations[0].Main]&#39; \<br>  --output table</pre><p><strong>Add routes to Application VPC CIDR:</strong></p><p>bash</p><pre># Example: Database public subnet route table<br>aws ec2 create-route \<br>  --route-table-id rtb-db-pub456 \<br>  --destination-cidr-block 10.100.0.0/16 \<br>  --vpc-peering-connection-id pcx-example12345 \<br>  --region us-west-2</pre><pre># Database private subnet AZ1<br>aws ec2 create-route \<br>  --route-table-id rtb-db-priv-az1 \<br>  --destination-cidr-block 10.100.0.0/16 \<br>  --vpc-peering-connection-id pcx-example12345 \<br>  --region us-west-2</pre><pre># Database private subnet AZ2<br>aws ec2 create-route \<br>  --route-table-id rtb-db-priv-az2 \<br>  --destination-cidr-block 10.100.0.0/16 \<br>  --vpc-peering-connection-id pcx-example12345 \<br>  --region us-west-2</pre><h4>Phase 4: Configure Security Groups</h4><p>Peering provides the network path, but <strong>security groups</strong> control the firewall rules.</p><h4>Step 4.1: Application VPC Security Group (Outbound)</h4><p>bash</p><pre># Allow outbound to Database VPC on PostgreSQL port<br>aws ec2 authorize-security-group-egress \<br>  --group-id sg-app123 \<br>  --ip-permissions IpProtocol=tcp,FromPort=5432,ToPort=5432,IpRanges=&#39;[{CidrIp=10.200.0.0/16,Description=&quot;Database VPC PostgreSQL&quot;}]&#39; \<br>  --region us-east-1</pre><h4>Step 4.2: Database VPC Security Group (Inbound)</h4><p>bash</p><pre># Allow inbound from Application VPC on PostgreSQL port<br>aws ec2 authorize-security-group-ingress \<br>  --group-id sg-db456 \<br>  --ip-permissions IpProtocol=tcp,FromPort=5432,ToPort=5432,IpRanges=&#39;[{CidrIp=10.100.0.0/16,Description=&quot;Application VPC&quot;}]&#39; \<br>  --region us-west-2</pre><h4>Phase 5: Test Connectivity</h4><h4>Step 5.1: Test from Application to Database</h4><p>bash</p><pre># From an EC2 instance or pod in Application VPC<br># Test if you can reach the database IP<br>telnet 10.200.10.5 5432</pre><pre># Or use nc (netcat)<br>nc -zv 10.200.10.5 5432</pre><pre># Or test with PostgreSQL client<br>psql -h 10.200.10.5 -U appuser -d mydb</pre><h4>Step 5.2: Verify with Ping</h4><p>bash</p><pre># Ping the database server (if ICMP is allowed in security groups)<br>ping 10.200.10.5</pre><h3>Advanced: Cross-Account, Cross-Region Peering</h3><h3>Additional Considerations</h3><p><strong>Cross-Account:</strong></p><ul><li>Accepter must explicitly accept the peering request</li><li>Both accounts incur data transfer charges</li><li>IAM permissions required to create/accept peering</li></ul><p><strong>Cross-Region:</strong></p><ul><li>Higher latency (typically 20–80ms vs &lt;1ms same region)</li><li>Higher data transfer costs</li><li>Still uses AWS private backbone (not internet)</li><li>Both regions must support the VPC peering feature</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=18746372c850" width="1" height="1" alt=""><hr><p><a href="https://aws.plainenglish.io/complete-guide-to-aws-vpc-peering-connecting-private-networks-across-aws-18746372c850">Complete Guide to AWS VPC Peering: Connecting Private Networks Across AWS</a> was originally published in <a href="https://aws.plainenglish.io/">AWS in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[DNS Resolution in K8s :From Service Discovery to Pod Routing- Complete Flow]]></title>
            <link>https://blog.devops.dev/dns-resolution-in-k8s-from-service-discovery-to-pod-routing-complete-flow-fb5feea97f7d?source=rss-9e2186f06b63------2</link>
            <guid isPermaLink="false">https://medium.com/p/fb5feea97f7d</guid>
            <category><![CDATA[coredns]]></category>
            <category><![CDATA[sanjal-s-eralil]]></category>
            <category><![CDATA[k8s-dns]]></category>
            <category><![CDATA[dns-resolution]]></category>
            <category><![CDATA[k8s]]></category>
            <dc:creator><![CDATA[Sanjal S Eralil]]></dc:creator>
            <pubDate>Wed, 04 Feb 2026 18:31:56 GMT</pubDate>
            <atom:updated>2026-02-05T08:51:13.327Z</atom:updated>
            <content:encoded><![CDATA[<h3>DNS Resolution in K8s :From Service Discovery to Pod Routing- Complete Flow</h3><p>When I first started working with Kubernetes, one thing puzzled me: containers could magically find each other by name, regardless of where they were running or how many times they restarted. Kubernetes seemed to handle everything automatically.</p><p>But how does a frontend Pod running on Node 1 find a database Pod on Node 3 just by querying “postgres”? How does traffic actually reach the right Pod when there are dozens of replicas? And what’s really happening when we create a Service?</p><p>In this deep dive, we’ll explore:</p><ul><li>How CoreDNS provides cluster-wide service discovery</li><li>The role of label selectors in routing traffic to Pods</li><li>How Services bridge the gap between names and IPs</li><li>What changes (and what doesn’t) with different Service types</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qOeIh0U8exjjYyjkZorEBA.png" /></figure><h3>Part 1: The Foundation — Understanding K8s Networking</h3><h4>The Pod Networking Challenge</h4><p>In Kubernetes, Pods are <strong>ephemeral</strong>. They get created, destroyed, and rescheduled constantly:</p><pre>Time T0:<br>┌─────────────────────────────────────────────────┐<br>│ api-pod-1: 10.244.1.5  (Node 1)                │<br>│ api-pod-2: 10.244.2.8  (Node 2)                │<br>│ api-pod-3: 10.244.3.12 (Node 3)                │<br>└─────────────────────────────────────────────────┘</pre><pre>Time T1: api-pod-2 crashes, gets replaced<br>┌─────────────────────────────────────────────────┐<br>│ api-pod-1: 10.244.1.5  (Node 1)                │<br>│ api-pod-4: 10.244.1.20 (Node 1) ← New IP!      │<br>│ api-pod-3: 10.244.3.12 (Node 3)                │<br>└─────────────────────────────────────────────────┘</pre><p><strong>The Problem:</strong> If your frontend is configured to connect to 10.244.2.8, it breaks when that Pod is replaced. You can&#39;t hardcode Pod IPs!</p><p><strong>The Solution:</strong> Kubernetes uses a two-layer approach:</p><ol><li><strong>Services</strong> — Provide stable virtual IPs (ClusterIPs)</li><li><strong>DNS</strong> — Maps service names to those stable IPs</li></ol><p>This is the foundation of service discovery in Kubernetes.</p><h3>Part 2: CoreDNS — The Cluster DNS Server</h3><h4>What is CoreDNS?</h4><p>CoreDNS is a flexible, plugin-based DNS server that runs as a Deployment inside your cluster. Think of it as your cluster’s internal phone book.</p><pre>┌────────────────────────────────────────────────────────┐<br>│  kube-system Namespace                                  │<br>│                                                          │<br>│  ┌─────────────┐      ┌─────────────┐                 │<br>│  │  CoreDNS    │      │  CoreDNS    │                 │<br>│  │  Pod #1     │      │  Pod #2     │                 │<br>│  └─────────────┘      └─────────────┘                 │<br>│         ↑                     ↑                         │<br>│         └──────────┬──────────┘                         │<br>│                    │                                    │<br>│         ┌──────────▼────────────┐                      │<br>│         │  kube-dns Service     │                      │<br>│         │  ClusterIP: 10.96.0.10│                      │<br>│         └──────────┬────────────┘                      │<br>└────────────────────┼───────────────────────────────────┘<br>                     │<br>      All Pods query this IP for DNS</pre><h4>Checking CoreDNS in Your Cluster</h4><pre># View CoreDNS pods<br>kubectl get pods -n kube-system -l k8s-app=kube-dns</pre><pre># Output:<br>NAME                       READY   STATUS    RESTARTS   AGE<br>coredns-74ff55c5b-7mqxj   1/1     Running   0          45d<br>coredns-74ff55c5b-n8lkw   1/1     Running   0          45d</pre><pre># View the kube-dns Service<br>kubectl get svc -n kube-system kube-dns</pre><pre># Output:<br>NAME       TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)         AGE<br>kube-dns   ClusterIP   10.96.0.10   &lt;none&gt;        53/UDP,53/TCP   90d</pre><p>Every Pod in your cluster is automatically configured to use 10.96.0.10 for DNS queries!</p><h4>Inside a Pod’s DNS Configuration</h4><p>Let’s see what a Pod actually sees:</p><pre># Create a test pod<br>kubectl run test-pod --image=busybox --restart=Never -- sleep 3600</pre><pre># Check its DNS config<br>kubectl exec test-pod -- cat /etc/resolv.conf<br>```</pre><pre>Output:<br>```<br>nameserver 10.96.0.10<br>search default.svc.cluster.local svc.cluster.local cluster.local<br>options ndots:5</pre><p><strong>Breaking this down:</strong></p><ol><li><strong>nameserver 10.96.0.10</strong> — All DNS queries go to CoreDNS</li><li><strong>search domains</strong> — Automatic completion for short names</li></ol><ul><li>default.svc.cluster.local - Services in same namespace</li><li>svc.cluster.local - Services in any namespace</li><li>cluster.local - Cluster domain</li></ul><p>3.<strong> ndots:5</strong> — Query behavior control (we’ll explore this later)</p><h3>Part 3: Services — The Stable Endpoint Layer</h3><p>Before DNS can work, we need something stable to resolve to. Enter: <strong>Services</strong>.</p><h4>Creating Pods with Labels</h4><p>First, let’s create some Pods. The key here is <strong>labels</strong> — they’re how Services find Pods:</p><p>yaml</p><pre># api-pods.yaml<br>apiVersion: v1<br>kind: Pod<br>metadata:<br>  name: api-pod-1<br>  labels:<br>    app: api           # Label 1<br>    version: v1        # Label 2<br>    tier: backend      # Label 3<br>spec:<br>  containers:<br>  - name: api<br>    image: hashicorp/http-echo<br>    args:<br>    - &quot;-text=Hello from Pod 1&quot;<br>    - &quot;-listen=:8080&quot;<br>    ports:<br>    - containerPort: 8080<br>---<br>apiVersion: v1<br>kind: Pod<br>metadata:<br>  name: api-pod-2<br>  labels:<br>    app: api<br>    version: v1<br>    tier: backend<br>spec:<br>  containers:<br>  - name: api<br>    image: hashicorp/http-echo<br>    args:<br>    - &quot;-text=Hello from Pod 2&quot;<br>    - &quot;-listen=:8080&quot;<br>    ports:<br>    - containerPort: 8080<br>---<br>apiVersion: v1<br>kind: Pod<br>metadata:<br>  name: api-pod-3<br>  labels:<br>    app: api<br>    version: v1<br>    tier: backend<br>spec:<br>  containers:<br>  - name: api<br>    image: hashicorp/http-echo<br>    args:<br>    - &quot;-text=Hello from Pod 3&quot;<br>    - &quot;-listen=:8080&quot;<br>    ports:<br>    - containerPort: 8080</pre><p>Apply it:</p><pre>kubectl apply -f api-pods.yaml</pre><pre># Check the pods<br>kubectl get pods -o wide<br>```</pre><pre>Output:<br>```<br>NAME        READY   STATUS    RESTARTS   AGE   IP            NODE<br>api-pod-1   1/1     Running   0          10s   10.244.1.5    node-1<br>api-pod-2   1/1     Running   0          10s   10.244.2.8    node-2<br>api-pod-3   1/1     Running   0          10s   10.244.3.12   node-3</pre><h4>Creating a Service with Label Selectors</h4><p>Now let’s create a Service that uses <strong>label selectors</strong> to find these Pods:</p><pre># api-service.yaml<br>apiVersion: v1<br>kind: Service<br>metadata:<br>  name: api-service<br>spec:<br>  selector:<br>    app: api       # Match Pods with app=api<br>    version: v1    # AND version=v1<br>  ports:<br>  - protocol: TCP<br>    port: 80         # Service port (what clients connect to)<br>    targetPort: 8080 # Pod port (where traffic goes)<br>  type: ClusterIP</pre><p>Apply it:</p><pre>kubectl apply -f api-service.yaml</pre><pre># Check the service<br>kubectl get svc api-service<br>```</pre><pre>Output:<br>```<br>NAME          TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE<br>api-service   ClusterIP   10.100.50.10   &lt;none&gt;        80/TCP    5s</pre><h3>Part 4: The Magic — How Label Selectors Route to Pods</h3><h4>The Endpoints Object</h4><p>When you create a Service with a selector, Kubernetes automatically creates an <strong>Endpoints</strong> object with the same name:</p><pre>kubectl get endpoints api-service<br>```</pre><pre>Output:<br>```<br>NAME          ENDPOINTS                                      AGE<br>api-service   10.244.1.5:8080,10.244.2.8:8080,10.244.3.12:8080   30s</pre><p>Let’s look at the full object:</p><pre>kubectl get endpoints api-service -o yaml</pre><pre>apiVersion: v1<br>kind: Endpoints<br>metadata:<br>  name: api-service  # Same name as the Service!<br>subsets:<br>- addresses:<br>  - ip: 10.244.1.5<br>    targetRef:<br>      kind: Pod<br>      name: api-pod-1<br>  - ip: 10.244.2.8<br>    targetRef:<br>      kind: Pod<br>      name: api-pod-2<br>  - ip: 10.244.3.12<br>    targetRef:<br>      kind: Pod<br>      name: api-pod-3<br>  ports:<br>  - port: 8080<br>    protocol: TCP</pre><blockquote>this is the bridge between the service and the pod</blockquote><p><strong>The Endpoint Controller</strong> (running in kube-controller-manager) continuously:</p><ol><li><strong>Watch</strong> all Services with selectors</li><li><strong>For each Service</strong>:</li></ol><ul><li><strong>Query</strong>: “Find all Pods matching these labels”</li><li><strong>Filter</strong>: Only include <strong>READY</strong> Pods</li><li><strong>Extract</strong>: Get their IP addresses</li><li><strong>Update</strong>: Write to <strong>Endpoints</strong> object</li></ul><ol><li><strong>kube-proxy</strong> watches <strong>Endpoints</strong></li><li><strong>kube-proxy</strong> updates <strong>iptables/IPVS</strong> rules</li></ol><h4>Tracing the process of endpoint creation</h4><pre># Watch the endpoints in real-time</pre><pre><br>kubectl get endpoints api-service -w</pre><p>In another terminal:</p><pre># Delete a pod<br>kubectl delete pod api-pod-2<br>```</pre><pre>Back in the watch terminal, you&#39;ll see:<br>```<br>NAME          ENDPOINTS                                AGE<br>api-service   10.244.1.5:8080,10.244.3.12:8080        1m<br>               ↑<br>               └── Pod 2 removed immediately!</pre><p>The Endpoint Controller detected that api-pod-2 is gone and updated the Endpoints object instantly.</p><h4>Testing Label Selector Matching</h4><pre># See which Pods match the Service selector<br>kubectl get pods -l app=api,version=v1</pre><pre># Output:<br>NAME        READY   STATUS    RESTARTS   AGE<br>api-pod-1   1/1     Running   0          5m<br>api-pod-2   1/1     Running   0          5m<br>api-pod-3   1/1     Running   0          5m</pre><pre>apiVersion: v1<br>kind: Pod<br>metadata:<br>  name: api-pod-v2<br>  labels:<br>    app: api<br>    version: v2      # Different version!<br>    tier: backend<br>spec:<br>  containers:<br>  - name: api<br>    image: hashicorp/http-echo<br>    args:<br>    - &quot;-text=Hello from v2&quot;<br>    - &quot;-listen=:8080&quot;<br>    ports:<br>    - containerPort: 8080</pre><pre>kubectl apply -f api-pod-v2.yaml</pre><pre># Check if it&#39;s in the endpoints<br>kubectl get endpoints api-service<br>```</pre><pre>Output:<br>```<br>NAME          ENDPOINTS                                      AGE<br>api-service   10.244.1.5:8080,10.244.2.8:8080,10.244.3.12:8080   10m<br>```</pre><pre>Notice: The v2 Pod is NOT included! The selector requires `version: v1`.</pre><h3>Part 5: DNS Resolution — Putting It All Together</h3><pre><br><br>Now let&#39;s see how DNS resolves service names to ClusterIPs.<br><br>### The Complete DNS Resolution Flow<br>```<br>┌──────────────────────────────────────────────────────────┐<br>│  STEP 1: Application Makes Request                       │<br>│  ┌────────────────────────────────┐                      │<br>│  │  Client Pod                     │                      │<br>│  │  curl http://api-service        │                      │<br>│  └────────────┬───────────────────┘                      │<br>└───────────────┼──────────────────────────────────────────┘<br>                │<br>                ▼<br>┌──────────────────────────────────────────────────────────┐<br>│  STEP 2: DNS Lookup Triggered                            │<br>│  ┌────────────────────────────────┐                      │<br>│  │  /etc/resolv.conf says:        │                      │<br>│  │  nameserver 10.96.0.10         │                      │<br>│  │  search default.svc.cluster... │                      │<br>│  └────────────┬───────────────────┘                      │<br>└───────────────┼──────────────────────────────────────────┘<br>                │<br>                ▼<br>┌──────────────────────────────────────────────────────────┐<br>│  STEP 3: Apply Search Domains                            │<br>│  &#39;api-service&#39; has 1 dot (&lt; 5)                           │<br>│  Try: api-service.default.svc.cluster.local ✓            │<br>└───────────────┼──────────────────────────────────────────┘<br>                │<br>                ▼<br>┌──────────────────────────────────────────────────────────┐<br>│  STEP 4: Query CoreDNS                                   │<br>│  DNS query to 10.96.0.10:53                              │<br>│  Question: IP of api-service.default.svc.cluster.local?  │<br>└───────────────┼──────────────────────────────────────────┘<br>                │<br>                ▼<br>┌──────────────────────────────────────────────────────────┐<br>│  STEP 5: CoreDNS Processes Request                       │<br>│  ┌────────────────────────────────┐                      │<br>│  │  1. Parse query                │                      │<br>│  │  2. Query Kubernetes API:      │                      │<br>│  │     &quot;Get Service &#39;api-service&#39; │                      │<br>│  │      in namespace &#39;default&#39;&quot;   │                      │<br>│  │  3. K8s returns ClusterIP      │                      │<br>│  │  4. Build DNS response         │                      │<br>│  └────────────┬───────────────────┘                      │<br>└───────────────┼──────────────────────────────────────────┘<br>                │<br>                ▼<br>┌──────────────────────────────────────────────────────────┐<br>│  STEP 6: DNS Response                                    │<br>│  Answer: api-service = 10.100.50.10                      │<br>│  TTL: 30 seconds                                         │<br>└───────────────┼──────────────────────────────────────────┘<br>                │<br>                ▼<br>┌──────────────────────────────────────────────────────────┐<br>│  STEP 7: Connect to Service ClusterIP                   │<br>│  Client connects to 10.100.50.10:80                      │<br>└───────────────┼──────────────────────────────────────────┘<br>                │<br>                ▼<br>┌──────────────────────────────────────────────────────────┐<br>│  STEP 8: kube-proxy Routes to Pod                       │<br>│  iptables rules route to one of:                         │<br>│  - 10.244.1.5:8080  (Pod 1)                             │<br>│  - 10.244.2.8:8080  (Pod 2)                             │<br>│  - 10.244.3.12:8080 (Pod 3)                             │<br>│  ✅ Request succeeds!                                    │<br>└──────────────────────────────────────────────────────────┘</pre><h4>Testing DNS Resolution</h4><pre># Test DNS from our test pod<br>kubectl exec test-pod -- nslookup api-service<br>```<br><br>Output:<br>```<br>Server:    10.96.0.10<br>Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local<br><br>Name:      api-service<br>Address 1: 10.100.50.10 api-service.default.svc.cluster.local</pre><p>Perfect! DNS returned the Service&#39;s ClusterIP.</p><h4>Making a Request</h4><pre># Make an HTTP request using the service name<br>kubectl exec test-pod -- wget -qO- http://api-service<br>```<br><br>Output:<br>```<br>Hello from Pod 2</pre><p>Run it again:</p><pre>kubectl exec test-pod -- wget -qO- http://api-service<br>```<br><br>Output:<br>```<br>Hello from Pod 1</pre><p><strong>Load balancing in action!</strong> Each request hits a different Pod.</p><h3>Part 6: How Traffic Actually Reaches Pods (kube-proxy)</h3><h4>The Missing Link</h4><p>We know DNS gives us the ClusterIP (10.100.50.10), but ClusterIP is just a <strong>virtual IP</strong> - it doesn&#39;t actually exist on any network interface! So how does traffic reach the real Pods?</p><p><strong>Answer: kube-proxy creates iptables rules that intercept traffic destined for ClusterIPs and redirects it to actual Pod IPs.</strong></p><pre>┌─────────────────────────────────────────────────────────────────┐<br>│                                                                 │<br>│  CLIENT POD                                                     │<br>│  &quot;I want to connect to 10.100.50.10:80&quot;                        │<br>│                                                                 │<br>└────────────────────────────┬────────────────────────────────────┘<br>                             │<br>                             ▼<br>                    <br>         Packet leaves pod with destination:<br>                  10.100.50.10:80<br>                             │<br>                             ▼<br>                    <br>╔═════════════════════════════════════════════════════════════════╗<br>║                                                                 ║<br>║  NODE&#39;S NETWORK STACK                                          ║<br>║  (iptables rules created by kube-proxy)                        ║<br>║                                                                 ║<br>║  ┌───────────────────────────────────────────────────────────┐ ║<br>║  │ Rule: If destination = 10.100.50.10:80                    │ ║<br>║  │       Then randomly pick a Pod:                           │ ║<br>║  │                                                            │ ║<br>║  │       • 33% → Change to 10.244.1.5:8080                  │ ║<br>║  │       • 33% → Change to 10.244.2.8:8080                  │ ║<br>║  │       • 33% → Change to 10.244.3.12:8080                 │ ║<br>║  └───────────────────────────────────────────────────────────┘ ║<br>║                                                                 ║<br>╚═════════════════════════════════════════════════════════════════╝<br>                             │<br>                             ▼<br>                    <br>         Packet now has destination:<br>                  10.244.2.8:8080<br>                             │<br>                             ▼<br>                    <br>┌─────────────────────────────────────────────────────────────────┐<br>│                                                                 │<br>│  DESTINATION POD                                                │<br>│  Routed to actual Pod on Node 2                                │<br>│  ✅ Connection established!                                     │<br>│                                                                 │<br>└─────────────────────────────────────────────────────────────────┘</pre><h3>Conclusion</h3><p>Understanding Kubernetes DNS reveals the elegant simplicity behind what initially seems like magic. When you type curl http://api-service, a precisely orchestrated chain unfolds: CoreDNS resolves the name to a stable ClusterIP, label selectors connect the Service to matching Pods through Endpoints, and kube-proxy&#39;s iptables rules route traffic to a healthy Pod — all in milliseconds.</p><p>The beauty lies in the abstraction. Pods crash, scale, and move between nodes, yet your application code never changes. DNS always returns the same ClusterIP, while the Endpoint Controller quietly updates which Pods receive traffic behind the scenes.</p><p>This knowledge transforms how you debug: slow DNS? Check ndots. No endpoints? Verify label selectors. Connection refused? Inspect Pod readiness probes. You&#39;re no longer guessing — you understand the system.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fb5feea97f7d" width="1" height="1" alt=""><hr><p><a href="https://blog.devops.dev/dns-resolution-in-k8s-from-service-discovery-to-pod-routing-complete-flow-fb5feea97f7d">DNS Resolution in K8s :From Service Discovery to Pod Routing- Complete Flow</a> was originally published in <a href="https://blog.devops.dev/">DevOps.dev</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How DNS resolution works in custom docker bridge network and not on default docker0 bridge network?]]></title>
            <link>https://blog.devops.dev/how-dns-resolution-works-in-custom-docker-bridge-network-and-not-on-default-docker0-bridge-network-adab783cdb49?source=rss-9e2186f06b63------2</link>
            <guid isPermaLink="false">https://medium.com/p/adab783cdb49</guid>
            <category><![CDATA[dns-resolution]]></category>
            <category><![CDATA[dns]]></category>
            <category><![CDATA[sanjal-s-eralil]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[docker-bridge-network]]></category>
            <dc:creator><![CDATA[Sanjal S Eralil]]></dc:creator>
            <pubDate>Mon, 26 Jan 2026 09:21:48 GMT</pubDate>
            <atom:updated>2026-01-27T08:04:00.703Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction</h3><p>When you start working with Docker, one of the first surprises you encounter is that containers can’t find each other by name on the default network. You try ping db from your app container, and it fails with &quot;Name or service not known.&quot; But create a custom network, and suddenly everything works! Let&#39;s explore what&#39;s happening under the hood.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/883/1*WRpInVoBoLmBZMtr_qF0AQ.png" /></figure><h3>Part 1: The Default Bridge Network (docker0)</h3><h4>What is the Default Bridge Network?</h4><p>When Docker is installed, it automatically creates a bridge network called docker0. Any container you run without specifying a network connects to this default bridge.</p><p>bash</p><pre># Check the default bridge<br>docker network ls</pre><pre># Output shows:<br>NETWORK ID     NAME      DRIVER    SCOPE<br>abc123def456   bridge    bridge    local    ← This is docker0</pre><h4>Setting Up a Simple Example</h4><p>Let’s create two containers and see how they communicate:</p><pre># Start a PostgreSQL database container<br>docker run -d --name database postgres:14</pre><pre># Start an Nginx container<br>docker run -d --name webserver nginx</pre><pre># Check if they&#39;re on the same network<br>docker network inspect bridge</pre><p>Both containers are now connected to the default bridge network and assigned IPs from the 172.17.0.0/16 subnet.</p><h4>Finding Container IPs</h4><pre># Get database container&#39;s IP<br>docker inspect database -f &#39;{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}&#39;<br># Output: 172.17.0.2</pre><h4>Testing Communication: By IP Works! ✅</h4><pre># Ping database from webserver using IP address<br>docker exec webserver ping 172.17.0.2</pre><pre># Output:<br>PING 172.17.0.2 (172.17.0.2) 56(84) bytes of data.<br>64 bytes from 172.17.0.2: icmp_seq=1 ttl=64 time=0.103 ms<br>64 bytes from 172.17.0.2: icmp_seq=2 ttl=64 time=0.075 ms</pre><p><strong>Success!</strong> Containers can communicate using IP addresses.</p><h4>Testing Communication: By Name Fails! ❌</h4><pre># Try to ping database by container name<br>docker exec webserver ping database</pre><pre># Output:<br>ping: database: Name or service not known</pre><p><strong>It fails!</strong> But why?</p><h4>The Problem: No DNS Resolution</h4><p>Let’s check how DNS is configured in our container:</p><pre>docker exec webserver cat /etc/resolv.conf</pre><pre># Output:<br>nameserver 8.8.8.8<br>nameserver 8.8.4.4</pre><p>The container is using <strong>Google’s public DNS servers</strong> (or your host’s DNS servers), which have no idea what “database” is. These DNS servers can resolve google.com or github.com, but they don&#39;t know anything about your local Docker containers.</p><h4>The Default Bridge Limitation</h4><p>On the default bridge network:</p><ul><li>✅ Containers <strong>can</strong> communicate using IP addresses</li><li>❌ Containers <strong>cannot</strong> communicate using container names</li><li>❌ No built-in DNS resolution for container names</li><li>📋 Must manually manage IPs or use legacy --link (deprecated)</li></ul><p><strong>This is a major pain point for applications!</strong> Imagine hardcoding IPs in your database connection strings — nightmare for maintenance!</p><h3>Part 2: Custom Bridge Networks — The Solution</h3><h3>Creating a Custom Bridge Network</h3><p>Docker introduced custom networks in version 1.9 to solve this exact problem:</p><pre># Create a custom bridge network<br>docker network create app-network</pre><pre># Verify it was created<br>docker network ls</pre><pre># Output:<br>NETWORK ID     NAME           DRIVER    SCOPE<br>xyz789abc012   app-network    bridge    local    ← Our custom network</pre><h4>Running Containers on Custom Network</h4><pre># Remove old containers<br>docker rm -f database webserver</pre><pre># Start containers on our custom network<br>docker run -d --name database --network app-network postgres:14<br>docker run -d --name webserver --network app-network nginx</pre><h4>Testing Communication: By Name Works! ✅</h4><pre># Ping database by name<br>docker exec webserver ping database</pre><pre># Output:<br>PING database (172.18.0.2) 56(84) bytes of data.<br>64 bytes from database.app-network (172.18.0.2): icmp_seq=1 ttl=64 time=0.089 ms<br>64 bytes from database.app-network (172.18.0.2): icmp_seq=2 ttl=64 time=0.073 ms</pre><p><strong>It works!</strong> Notice it automatically resolved database to 172.18.0.2.</p><h4>The Magic: Built-in DNS Server</h4><p>Let’s see what changed:</p><pre>docker exec webserver cat /etc/resolv.conf</pre><pre># Output:<br>nameserver 127.0.0.11<br>options ndots:0<br>```</pre><pre>Now the container is using `127.0.0.11` - **Docker&#39;s embedded DNS server!**</pre><pre>---</pre><pre>## **Part 3: Under the Hood - How Docker DNS Works**</pre><pre>### **The DNS Architecture**</pre><pre>When you create a custom bridge network, Docker sets up a sophisticated DNS resolution system:<br>```<br>┌─────────────────────────────────────────────────────────┐<br>│  Container &quot;webserver&quot;                                   │<br>│                                                          │<br>│  1. Application: ping database                           │<br>│         ↓                                                │<br>│  2. Checks /etc/resolv.conf → nameserver 127.0.0.11     │<br>│         ↓                                                │<br>│  3. Sends DNS query to 127.0.0.11:53                    │<br>│         ↓                                                │<br>└─────────┼────────────────────────────────────────────────┘<br>          │<br>          │ iptables intercepts and redirects<br>          ↓<br>┌─────────┼────────────────────────────────────────────────┐<br>│  HOST   │                                                 │<br>│         ↓                                                 │<br>│  ┌──────────────────────────────────┐                   │<br>│  │  Docker Daemon (DNS Resolver)    │                   │<br>│  │                                   │                   │<br>│  │  Internal DNS Table:              │                   │<br>│  │  ┌────────────────────────────┐  │                   │<br>│  │  │ Network: app-network       │  │                   │<br>│  │  │ database  → 172.18.0.2     │  │                   │<br>│  │  │ webserver → 172.18.0.3     │  │                   │<br>│  │  └────────────────────────────┘  │                   │<br>│  │                                   │                   │<br>│  │  Returns: 172.18.0.2              │                   │<br>│  └──────────────────────────────────┘                   │<br>│         ↓                                                 │<br>└─────────┼────────────────────────────────────────────────┘<br>          │<br>          ↓<br>┌─────────┼────────────────────────────────────────────────┐<br>│  Container receives: database = 172.18.0.2               │<br>│  Connection established! ✅                               │<br>└──────────────────────────────────────────────────────────┘</pre><h3>Key Components:</h3><ol><li><strong>127.0.0.11 — The Magic IP</strong></li></ol><ul><li>Not a real network interface</li><li>Intercepted by iptables rules</li><li>Redirected to Docker daemon’s DNS resolver</li></ul><p><strong>2. Docker Daemon’s DNS Resolver</strong></p><ul><li>Maintains internal mapping of container names to IPs</li><li>Separate mappings per network</li><li>Updates automatically when containers start/stop</li></ul><p><strong>3. iptables Rules</strong></p><ul><li>Redirect DNS queries from 127.0.0.11:53 to Docker daemon</li><li>Transparent to the container</li></ul><h4>Verification: The DNS Query</h4><p>Let’s see the DNS resolution in action:</p><pre># Install DNS tools in container<br>docker exec -it webserver bash<br>apt-get update &amp;&amp; apt-get install -y dnsutils</pre><pre># Query Docker&#39;s DNS<br>nslookup database 127.0.0.11</pre><pre># Output:<br>Server:     127.0.0.11<br>Address:    127.0.0.11#53</pre><pre>Name:   database<br>Address: 172.18.0.2</pre><p>Perfect! Docker’s DNS server resolved database to its IP address.</p><h3>Part 4: Practical Real-World Example</h3><h4>Building a Multi-Container Application</h4><p>Let’s build a typical web application stack:</p><p>bash</p><pre># Create application network<br>docker network create webapp-net</pre><pre># PostgreSQL database<br>docker run -d \<br>  --name postgres-db \<br>  --network webapp-net \<br>  -e POSTGRES_PASSWORD=secret \<br>  -e POSTGRES_DB=myapp \<br>  postgres:14</pre><pre># Backend API (Node.js/Python/etc)<br>docker run -d \<br>  --name api-server \<br>  --network webapp-net \<br>  -e DATABASE_URL=postgresql://postgres:secret@postgres-db:5432/myapp \<br>  my-api-image</pre><pre># Frontend web server<br>docker run -d \<br>  --name frontend \<br>  --network webapp-net \<br>  -p 3000:3000 \<br>  -e API_URL=http://api-server:8080 \<br>  my-frontend-image</pre><p><strong>Notice the connection strings:</strong></p><ul><li>@postgres-db:5432 - using <strong>container name</strong>, not IP!</li><li>http://api-server:8080 - using <strong>container name</strong>, not IP!</li></ul><p>This is <strong>much cleaner</strong> and <strong>maintainable</strong> than hardcoding IPs.</p><h4>Testing the Stack</h4><pre># Check if API can reach database<br>docker exec api-server ping postgres-db<br># ✅ Works!</pre><pre># Check if frontend can reach API<br>docker exec frontend curl <a href="http://api-server:8080/health">http://api-server:8080/health</a><br># ✅ Works!</pre><h3>Part 5: Why This Matters</h3><h4>Benefits of Custom Bridge Networks:</h4><ol><li><strong>Automatic Service Discovery</strong></li></ol><ul><li>No hardcoded IPs</li><li>Containers find each other by name</li><li>Works even if containers restart with different IPs</li></ul><p><strong>2. Better Isolation</strong></p><ul><li>Each custom network is isolated</li><li>Containers on different networks can’t communicate by default</li><li>Improved security</li></ul><p><strong>3. Clean Configuration</strong></p><ul><li>Connection strings use logical names</li><li>Easier to read and maintain</li><li>Self-documenting</li></ul><p><strong>4. Dynamic Updates</strong></p><ul><li>DNS mappings update automatically</li><li>No manual intervention needed</li><li>Container restarts handled transparently</li></ul><h3>Why Default Bridge Doesn’t Have DNS:</h3><p><strong>Historical Reasons:</strong></p><ul><li>Default bridge existed since Docker 1.0 (2013)</li><li>DNS feature added in Docker 1.9 (2015)</li><li>Backward compatibility — couldn’t change default behavior</li><li>Encourages best practice (use custom networks)</li></ul><h3>Conclusion</h3><p>Docker’s DNS resolution on custom bridge networks is a powerful feature that makes container orchestration much simpler. By understanding the difference between the default bridge (no DNS) and custom bridges (built-in DNS), you can build more robust and maintainable containerized applications.</p><p>The key takeaway: <strong>Use custom bridge networks for automatic service discovery through Docker’s embedded DNS server at 127.0.0.11.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=adab783cdb49" width="1" height="1" alt=""><hr><p><a href="https://blog.devops.dev/how-dns-resolution-works-in-custom-docker-bridge-network-and-not-on-default-docker0-bridge-network-adab783cdb49">How DNS resolution works in custom docker bridge network and not on default docker0 bridge network?</a> was originally published in <a href="https://blog.devops.dev/">DevOps.dev</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Contributing to Infrastructure: Working with Remote Terraform State in AWS S3]]></title>
            <link>https://aws.plainenglish.io/contributing-to-infrastructure-working-with-remote-terraform-state-in-aws-s3-0848a0f25c7a?source=rss-9e2186f06b63------2</link>
            <guid isPermaLink="false">https://medium.com/p/0848a0f25c7a</guid>
            <category><![CDATA[remote-infrastructure]]></category>
            <category><![CDATA[sanjal-s-eralil]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[terraform-remote-state]]></category>
            <dc:creator><![CDATA[Sanjal S Eralil]]></dc:creator>
            <pubDate>Tue, 16 Dec 2025 13:26:53 GMT</pubDate>
            <atom:updated>2025-12-18T06:58:38.034Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/855/1*PWddaAw7sBy7FQR8dc6FVQ.png" /></figure><h3>Introduction</h3><p>When joining a team that manages AWS infrastructure with Terraform, you’ll often need to connect to an existing remote state stored in S3. This guide walks you through the process of connecting to your team’s Terraform state and safely contributing infrastructure changes.</p><h3>Prerequisites</h3><ul><li>AWS CLI installed and configured</li><li>Terraform installed (check with your team for the required version)</li><li>AWS access credentials provided by your team</li><li>S3 bucket name and state file path from your team’s documentation</li></ul><h3>Why Remote State Matters</h3><p>Remote state allows multiple team members to work on the same infrastructure safely. By storing state in S3, everyone works from the same source of truth, preventing conflicts and ensuring consistency across your infrastructure deployments.</p><h3>Step 1: Clone the Terraform Repository</h3><p>Start by cloning your team’s infrastructure repository:</p><p>bash</p><pre>git clone &lt;your-team-terraform-repo&gt;<br>cd &lt;infrastructure-directory&gt;</pre><h3>Step 2: Configure AWS CLI</h3><p>Before any Terraform operations, authenticate to AWS using the credentials provided by your team. This ensures you’re working in the correct AWS account and region:</p><p>bash</p><pre>aws configure</pre><p>You’ll be prompted to enter:</p><ul><li>AWS Access Key ID</li><li>AWS Secret Access Key</li><li>Default region (e.g., us-east-1, us-west-2, eu-west-1)</li><li>Default output format (recommended: json)</li></ul><p><strong>Verify Your Configuration:</strong></p><p>bash</p><pre>aws sts get-caller-identity<br>aws s3 ls s3://&lt;your-team-bucket&gt;/</pre><h3>Step 3: Set Up Backend Configuration</h3><p>There are two approaches to configure the S3 backend. Check with your team to see which method they use.</p><h3>Option A: Static Backend Configuration</h3><p>If your team uses static configuration, you’ll find a backend.tf file in the repository:</p><p>hcl</p><pre>terraform {<br>  backend &quot;s3&quot; {<br>    bucket  = &quot;your-team-state-bucket&quot;<br>    key     = &quot;environment/component/terraform.tfstate&quot;<br>    region  = &quot;us-east-1&quot;<br>    encrypt = true<br>  }<br>}</pre><h3>Option B: Dynamic Backend Configuration (Multi-Environment)</h3><p>For teams managing multiple environments, create an empty backend.tf file:</p><p>hcl</p><pre>terraform {<br>  backend &quot;s3&quot; {<br>    # Backend configuration will be provided via -backend-config during terraform init<br>  }<br>}</pre><p>This allows different team members to work on different environments by specifying configuration at runtime.</p><h3>Step 4: Initialize Terraform and Connect to Remote State</h3><h3>For Static Backend Configuration</h3><p>Simply run:</p><p>bash</p><pre>terraform init</pre><h3>For Dynamic Backend Configuration</h3><p>Provide the backend details at initialization. Your team should document these values:</p><p>bash</p><pre>terraform init \<br>  -backend-config=&quot;bucket=your-team-state-bucket&quot; \<br>  -backend-config=&quot;key=environment/component/terraform.tfstate&quot; \<br>  -backend-config=&quot;region=us-east-1&quot; \<br>  -backend-config=&quot;encrypt=true&quot;</pre><p><strong>Examples for Different Environments:</strong></p><p>bash</p><pre># Production Environment<br>terraform init \<br>  -backend-config=&quot;bucket=company-terraform-states&quot; \<br>  -backend-config=&quot;key=production/infrastructure/terraform.tfstate&quot; \<br>  -backend-config=&quot;region=us-east-1&quot; \<br>  -backend-config=&quot;encrypt=true&quot;</pre><pre># Development Environment<br>terraform init \<br>  -backend-config=&quot;bucket=company-terraform-states&quot; \<br>  -backend-config=&quot;key=development/infrastructure/terraform.tfstate&quot; \<br>  -backend-config=&quot;region=us-east-1&quot; \<br>  -backend-config=&quot;encrypt=true&quot;</pre><h3>Step 5: Verify Your Connection</h3><p>Confirm that you’ve successfully connected to the remote state:</p><p>bash</p><pre>terraform state list</pre><p>This displays all resources currently managed in the state file. You should see a list of existing infrastructure resources.</p><h3>Step 6: Making Changes to Infrastructure</h3><p>Once connected, you can start contributing to the infrastructure.</p><h3>Review Planned Changes</h3><p>Always preview changes before applying them:</p><p>bash</p><pre>terraform plan \<br>  -var=&quot;region=us-east-1&quot; \<br>  -var=&quot;environment=production&quot; \<br>  -var=&quot;application_name=web-app&quot;</pre><p><strong>Note:</strong> Replace these variables with the ones your team uses. Check your repository’s variables.tf file or team documentation.</p><h3>Apply Changes</h3><p>After reviewing the plan and getting necessary approvals:</p><p>bash</p><pre>terraform apply \<br>  -var=&quot;region=us-east-1&quot; \<br>  -var=&quot;environment=production&quot; \<br>  -var=&quot;application_name=web-app&quot;</pre><h3>Refresh State Without Changes</h3><p>To sync the state with actual AWS resources without modifying anything:</p><p>bash</p><pre>terraform plan -refresh-only \<br>  -var=&quot;region=us-east-1&quot; \<br>  -var=&quot;environment=production&quot;</pre><pre>terraform apply -refresh-only \<br>  -var=&quot;region=us-east-1&quot; \<br>  -var=&quot;environment=production&quot; \<br>  -auto-approve</pre><h3>Switching Between Environments or State Files</h3><p>When you need to work on a different environment or component:</p><h3>Step 1: Clean Local Terraform Cache</h3><p>This prevents conflicts between different state configurations.</p><p><strong>On Linux/macOS:</strong></p><p>bash</p><pre>rm -rf .terraform<br>rm .terraform.lock.hcl  # Optional but recommended</pre><p><strong>On Windows (PowerShell):</strong></p><p>powershell</p><pre>Remove-Item -Recurse -Force .terraform<br>Remove-Item .terraform.lock.hcl  # Optional</pre><h3>Step 2: Update AWS Configuration (If Needed)</h3><p>If switching to a different region or AWS account:</p><p>bash</p><pre>aws configure</pre><h3>Step 3: Re-initialize with New Backend</h3><p>bash</p><pre>terraform init \<br>  -backend-config=&quot;bucket=your-team-state-bucket&quot; \<br>  -backend-config=&quot;key=new-environment/component/terraform.tfstate&quot; \<br>  -backend-config=&quot;region=us-east-1&quot; \<br>  -backend-config=&quot;encrypt=true&quot;</pre><h3>Understanding State Operations</h3><h3>Viewing Specific Resources</h3><p>To see the current state of a specific resource:</p><p>bash</p><pre>terraform state show aws_instance.web_server</pre><h3>Removing Resources from State</h3><p>Sometimes you need to remove a resource from state tracking (consult your team first):</p><p>bash</p><pre>terraform state rm aws_instance.web_server</pre><p><strong>Important</strong>: This only removes tracking from Terraform; the actual AWS resource remains unchanged.</p><h3>Importing Existing Resources</h3><p>To bring existing AWS resources under Terraform management:</p><p>bash</p><pre>terraform import \<br>  -var=&quot;region=us-east-1&quot; \<br>  -var=&quot;environment=production&quot; \<br>  aws_instance.web_server \<br>  i-1234567890abcdef0</pre><h3>Team Collaboration Best Practices</h3><h3>1. Always Pull Latest Code</h3><p>Before making changes, ensure you have the latest infrastructure code:</p><p>bash</p><pre>git pull origin main</pre><h3>2. Use Feature Branches</h3><p>Create a branch for your infrastructure changes:</p><p>bash</p><pre>git checkout -b feature/add-new-security-group</pre><h3>3. Run Plan Before Apply</h3><p>Always run terraform plan and review output before applying changes. Share the plan output with your team for review if making significant changes.</p><h3>4. Coordinate with Team</h3><p>For production changes:</p><ul><li>Notify your team before applying changes</li><li>Use change management processes if your organization requires them</li><li>Consider maintenance windows for critical infrastructure</li></ul><h3>5. Document Your Changes</h3><p>Update documentation and commit messages to explain infrastructure changes:</p><p>bash</p><pre>git commit -m &quot;Add security group for new database cluster&quot;</pre><h3>Security Best Practices</h3><ol><li><strong>Never Commit Credentials</strong>: Keep AWS credentials out of version control. Use environment variables or AWS credential files.</li><li><strong>Enable State Encryption</strong>: Always use encrypt = true in backend configuration.</li><li><strong>Use State Locking</strong>: Implement DynamoDB state locking to prevent concurrent modifications:</li></ol><p>hcl</p><pre>terraform {<br>     backend &quot;s3&quot; {<br>       bucket         = &quot;your-team-state-bucket&quot;<br>       key            = &quot;environment/terraform.tfstate&quot;<br>       region         = &quot;us-east-1&quot;<br>       encrypt        = true<br>       dynamodb_table = &quot;terraform-state-lock&quot;<br>     }<br>   }</pre><ol><li><strong>Limit State Access</strong>: Ensure only authorized team members have access to the S3 bucket containing state files.</li><li><strong>Review Permissions</strong>: Understand your AWS IAM permissions before making infrastructure changes.</li></ol><h3>Common Workflows</h3><h3>Daily Development Workflow</h3><p>bash</p><pre># 1. Update code<br>git pull origin main</pre><pre># 2. Switch to your branch<br>git checkout feature/your-feature</pre><pre># 3. Plan changes<br>terraform plan -var-file=&quot;environments/dev.tfvars&quot;</pre><pre># 4. Apply if everything looks good<br>terraform apply -var-file=&quot;environments/dev.tfvars&quot;</pre><pre># 5. Commit and push<br>git add .<br>git commit -m &quot;Description of infrastructure changes&quot;<br>git push origin feature/your-feature</pre><h3>Starting Work on New Environment</h3><p>bash</p><pre># 1. Clean previous state<br>rm -rf .terraform</pre><pre># 2. Configure AWS for new environment<br>aws configure</pre><pre># 3. Initialize with new state<br>terraform init \<br>  -backend-config=&quot;bucket=state-bucket&quot; \<br>  -backend-config=&quot;key=new-env/terraform.tfstate&quot; \<br>  -backend-config=&quot;region=us-west-2&quot;</pre><pre># 4. Verify connection<br>terraform state list</pre><h3>Troubleshooting Common Issues</h3><h3>Issue: “Backend initialization required”</h3><p><strong>Solution</strong>: You need to run terraform init with the correct backend configuration.</p><h3>Issue: “Error acquiring the state lock”</h3><p><strong>Solution</strong>: Another team member may be running Terraform. Wait for them to complete, or if the lock is stale, check your team’s process for releasing locks.</p><h3>Issue: “No configuration files found”</h3><p><strong>Solution</strong>: Ensure you’re in the correct directory containing .tf files.</p><h3>Issue: “Provider version mismatch”</h3><p><strong>Solution</strong>: Your local provider versions differ from those in the remote state. Run terraform init -upgrade or match the versions specified in the state.</p><h3>Issue: Terraform shows resources will be created that already exist</h3><p><strong>Solution</strong>: The state is out of sync with actual infrastructure. These resources may need to be imported or the state refreshed.</p><h3>Issue: “Access Denied” when accessing S3 bucket</h3><p><strong>Solution</strong>: Verify your AWS credentials and check with your team that you have proper IAM permissions for the state bucket.</p><h3>Getting Help</h3><p>When encountering issues:</p><ol><li><strong>Check Team Documentation</strong>: Your team likely has specific guidelines for your infrastructure</li><li><strong>Review State File</strong>: Use terraform state list and terraform state show to understand current state</li><li><strong>Ask Team Members</strong>: Senior team members can help troubleshoot complex state issues</li><li><strong>Use Version Control History</strong>: Review previous commits to understand infrastructure changes</li></ol><h3>Conclusion</h3><p>Connecting to and contributing to remote Terraform state is a fundamental skill for infrastructure teams. By following this guide, you can safely work with your team’s infrastructure, make changes confidently, and collaborate effectively. Remember to always authenticate properly, verify your changes with terraform plan, and coordinate with your team on significant infrastructure modifications.</p><h3>Quick Reference</h3><p>bash</p><pre># Initial Setup<br>aws configure<br>terraform init -backend-config=&quot;bucket=&lt;bucket&gt;&quot; \<br>  -backend-config=&quot;key=&lt;path&gt;&quot; \<br>  -backend-config=&quot;region=&lt;region&gt;&quot;</pre><pre># Daily Operations<br>terraform state list                    # View managed resources<br>terraform plan -var-file=&quot;vars.tfvars&quot;  # Preview changes<br>terraform apply -var-file=&quot;vars.tfvars&quot; # Apply changes</pre><pre># Switching State Files<br>rm -rf .terraform                       # Clean cache<br>terraform init -backend-config=...      # Reinitialize</pre><pre># State Management<br>terraform state show &lt;resource&gt;         # View resource details<br>terraform refresh                       # Sync state with AWS<br>terraform import &lt;resource&gt; &lt;aws-id&gt;    # Import existing resource</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0848a0f25c7a" width="1" height="1" alt=""><hr><p><a href="https://aws.plainenglish.io/contributing-to-infrastructure-working-with-remote-terraform-state-in-aws-s3-0848a0f25c7a">Contributing to Infrastructure: Working with Remote Terraform State in AWS S3</a> was originally published in <a href="https://aws.plainenglish.io/">AWS in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Role of OIDC Provider in EKS Clusters]]></title>
            <link>https://aws.plainenglish.io/the-role-of-oidc-provider-in-eks-clusters-b26d61bd5d69?source=rss-9e2186f06b63------2</link>
            <guid isPermaLink="false">https://medium.com/p/b26d61bd5d69</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[oidc]]></category>
            <category><![CDATA[oidc-provider]]></category>
            <category><![CDATA[sanjal-s-eralil]]></category>
            <category><![CDATA[ek]]></category>
            <dc:creator><![CDATA[Sanjal S Eralil]]></dc:creator>
            <pubDate>Tue, 25 Nov 2025 04:38:07 GMT</pubDate>
            <atom:updated>2025-11-27T20:12:50.372Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Youtube : <a href="https://www.youtube.com/@SanjalSEralil">https://www.youtube.com/@SanjalSEralil</a></blockquote><blockquote><strong>OIDC provider is an essential component in implementing IRSA in EKS clusters</strong></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uyBa7BnPiRelnPLyDYxjfQ.png" /><figcaption>FLOW DIAGRAM</figcaption></figure><h3>Introduction</h3><p>The OIDC (OpenID Connect) provider in Amazon EKS serves as a trust bridge between Kubernetes service accounts and AWS IAM. It enables pods to securely access AWS services without storing static credentials, fundamentally transforming how applications authenticate with AWS.</p><h3>The Core Role: Establishing Trust</h3><p>The OIDC provider’s primary role is to establish trust between two separate identity systems. Kubernetes manages pod identities through service accounts, while AWS IAM controls access to AWS resources. Without the OIDC provider, these systems can’t communicate securely.</p><p>When you create an EKS cluster, AWS generates a unique OIDC issuer URL for that cluster. However, AWS IAM doesn’t automatically trust this issuer. By creating an IAM OIDC identity provider, you explicitly tell AWS: “Trust identity tokens from this specific Kubernetes cluster.”</p><p>This trust is non-negotiable. AWS will reject any authentication attempts from your cluster until the OIDC provider exists in your IAM account.</p><h3>How It Works</h3><p><strong>Token Generation</strong>: Kubernetes injects a signed JWT token into each pod. This token contains the pod’s identity: service account name, namespace, and cluster issuer.</p><p><strong>Token Validation</strong>: When a pod requests AWS access, AWS STS validates the token by contacting the OIDC provider endpoint to retrieve public signing keys and verify the token’s authenticity.</p><p><strong>Credential Issuance</strong>: After successful validation and trust policy evaluation, STS issues temporary AWS credentials (valid for one hour) to the pod.</p><p>The OIDC provider is the critical component that makes token validation possible. Without it, STS has no way to verify tokens from your cluster.</p><h3>Why OIDC Provider is Mandatory</h3><p>You cannot skip the OIDC provider for IAM roles with service accounts (IRSA). Attempting to use IRSA without an OIDC provider results in immediate authentication failures:</p><pre>No OpenIDConnect provider found in your account for <br>https://oidc.eks.region.amazonaws.com/id/YOUR_CLUSTER_ID</pre><p>The alternatives are all inferior:</p><ul><li><strong>Static credentials in secrets</strong>: Security nightmare with long-lived keys</li><li><strong>Instance profiles</strong>: Too broad, all pods inherit the same permissions</li><li><strong>Third-party tools</strong>: Added complexity, mostly deprecated</li></ul><p>The OIDC provider is AWS’s recommended approach because it’s secure, scalable, and requires minimal management.</p><h3>Security Benefits</h3><p><strong>No Static Credentials</strong>: The OIDC provider eliminates the need for AWS access keys stored in Kubernetes secrets. Only short-lived tokens exist, automatically rotated every hour.</p><p><strong>Fine-Grained Access Control</strong>: Each service account can assume different IAM roles with specific permissions. You achieve true least-privilege access at the pod level.</p><p><strong>Complete Audit Trail</strong>: Every authentication attempt is logged in CloudTrail, providing visibility into which pods access which AWS resources.</p><p><strong>Automatic Token Rotation</strong>: The AWS SDK handles token refresh transparently, with no application code changes needed.</p><h3>Setup Overview</h3><p><strong>One-Time Setup Per Cluster</strong>: Create the OIDC provider once:</p><p>bash</p><pre># Using eksctl (simplest method)<br>eksctl utils associate-iam-oidc-provider --cluster your-cluster-name --approve</pre><p><strong>Repeatable Pattern</strong>: After the OIDC provider exists, follow this pattern for each application:</p><ol><li><strong>Create IAM role</strong> with trust policy referencing the OIDC provider (specify which service account can assume it)</li><li><strong>Attach AWS permissions</strong> to the role (S3, RDS, DynamoDB, etc.)</li><li><strong>Create/annotate service account</strong> in Kubernetes with the role ARN</li><li><strong>Deploy pods</strong> using that service account — they automatically get AWS access</li></ol><p><strong>Key Point</strong>: You create the OIDC provider once, but you create multiple IAM roles (one per application or permission set) and multiple service accounts (one per pod workload). All service accounts in the cluster use the same OIDC provider for authentication.</p><h3>Key Characteristics</h3><p><strong>One OIDC Provider, Many Service Accounts</strong>: Each cluster has exactly one OIDC provider, but you can create unlimited service accounts. Each service account can be annotated with a different IAM role ARN, giving different pods different AWS permissions.</p><pre>1 Cluster = 1 OIDC Provider = Many Service Accounts = Many IAM Roles</pre><p><strong>One Provider Per Cluster</strong>: Each EKS cluster needs its own OIDC provider. This ensures isolation — tokens from one cluster cannot assume roles in another.</p><p><strong>Immutable Issuer URL</strong>: The OIDC issuer URL never changes during the cluster’s lifetime, ensuring stable trust relationships.</p><p><strong>Minimal Management</strong>: Once configured, the OIDC provider requires no ongoing maintenance. It silently validates every authentication request for all service accounts in the cluster.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/956/1*y_lh40TLugGYDmlKeqtorg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/953/1*yPqZ_JphqNK9HZuMt89CYQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/955/1*R3lXDJDRv8ye5vNjwmU_fg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*xnn46YXrh3WPXoJYxte4QA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*5czdsXTLauJq4ePhREcEgg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/953/1*_9kS31IlKuQQZqzZuQa_0g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/955/1*bBWk1h3lLGV4HnaGUr4PPQ.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b26d61bd5d69" width="1" height="1" alt=""><hr><p><a href="https://aws.plainenglish.io/the-role-of-oidc-provider-in-eks-clusters-b26d61bd5d69">The Role of OIDC Provider in EKS Clusters</a> was originally published in <a href="https://aws.plainenglish.io/">AWS in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Getting Started with Services in Pagerduty]]></title>
            <link>https://blog.devops.dev/getting-started-with-services-in-pagerduty-8f1f0a9acbfb?source=rss-9e2186f06b63------2</link>
            <guid isPermaLink="false">https://medium.com/p/8f1f0a9acbfb</guid>
            <category><![CDATA[pagerduty]]></category>
            <category><![CDATA[pagerduty-alert]]></category>
            <category><![CDATA[datadog]]></category>
            <category><![CDATA[pagerduty-service]]></category>
            <category><![CDATA[sanjal-s-eralil]]></category>
            <dc:creator><![CDATA[Sanjal S Eralil]]></dc:creator>
            <pubDate>Mon, 03 Nov 2025 10:06:19 GMT</pubDate>
            <atom:updated>2025-11-17T10:17:07.950Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GmjUFW2vhss3tqP1AMcSLA.png" /></figure><h3>What is a Service?</h3><p>A Service represents any application, system, or component that needs monitoring. Examples:</p><ul><li>Web application</li><li>Database</li><li>API service</li><li>Mobile app</li></ul><p>Each service connects to an Escalation Policy that defines who gets notified when something goes wrong.</p><h3>What is an Escalation Policy?</h3><p>An Escalation Policy is a set of rules that determines:</p><ul><li>Who gets notified first</li><li>How long to wait before notifying the next person</li><li>What notification methods to use (email, SMS, phone call)</li></ul><p><strong>Simple Escalation Flow:</strong></p><ol><li>Level 1: Primary on-call person gets notified immediately</li><li>Wait 15 minutes — if no response, move to Level 2</li><li>Level 2: Secondary person or team lead gets notified</li><li>Wait 30 minutes — if no response, move to Level 3</li><li>Level 3: Manager gets notified</li><li>Repeat until someone responds</li></ol><h3>How They Work Together</h3><p><strong>Monitoring Tool → Service → Escalation Policy → Team Members</strong></p><ol><li>Your monitoring tool detects an issue</li><li>Sends alert to PagerDuty Service</li><li>Service triggers its Escalation Policy</li><li>Team members get notified based on the policy rules</li><li>Global Orchestration helps filter and prioritize alerts</li></ol><h3>Common Setup Examples</h3><p><strong>Single Team Setup:</strong></p><ul><li>1 Service → 1 Escalation Policy → 1 Team</li></ul><p><strong>Multiple Services, One Team:</strong></p><ul><li>Service A (Web App) → Escalation Policy 1 → Web Team</li><li>Service B (Database) → Escalation Policy 1 → Web Team</li></ul><p><strong>Multiple Teams:</strong></p><ul><li>Service A (Web App) → Escalation Policy 1 → Frontend Team</li><li>Service B (Database) → Escalation Policy 2 → Backend Team</li></ul><h3>Quick Start Steps</h3><ol><li>Create your first Service</li><li>Set up an Escalation Policy with your team members</li><li>Connect your monitoring tools to the Service</li><li>Test the escalation flow</li><li>Add Global Orchestration features as needed</li></ol><h3>Key Takeaways</h3><ul><li>Service = The thing you’re monitoring</li><li>Escalation Policy = Who gets notified and when</li></ul><h3>How to Connect Datadog with PagerDuty Service</h3><h3>Step 1: Create PagerDuty Service and Escalation Policy</h3><p><strong>Create an Escalation Policy First:</strong></p><ol><li>In PagerDuty, go to People → Escalation Policies</li><li>Click New Escalation Policy</li><li>Give it a name (e.g., “Web Team Escalation”)</li><li>Add escalation levels:</li></ol><ul><li>Level 1: Select primary on-call person, set timeout (15 minutes)</li><li>Level 2: Select secondary person, set timeout (30 minutes)</li><li>Level 3: Select manager or team lead</li></ul><p>5. Click Save</p><p><strong>Create a Service:</strong></p><ol><li>Go to Services → Service Directory</li><li>Click New Service</li><li>Fill in the details:</li></ol><ul><li>Name: Your service name (e.g., “Web Application”)</li><li>Description: Brief description of what this service monitors</li></ul><p>4. In Integrations section:</p><ul><li>Select Use our API directly</li><li>Choose Events API v2</li></ul><p>5. In Assign section:</p><ul><li>Select the escalation policy you created above</li></ul><p>6. Click Create Service</p><blockquote>Important: Copy the Integration Key — you’ll need this for Datadog</blockquote><h3>Step 2: Configure Datadog Integration</h3><p><strong>In Datadog:</strong></p><ol><li>Go to Integrations → Integrations</li><li>Search for PagerDuty and click on it</li><li>Click Install Integration (if not already installed)</li><li>Go to Configuration tab</li><li>Click Add Account</li><li>Fill in the details:</li></ol><ul><li>Account Name: Your PagerDuty account name</li><li>Service Name: Name for this integration (e.g., “Web App Alerts”)</li><li>Integration Key: Paste the key you copied from PagerDuty service</li></ul><p>7. Click Save</p><p><strong>Test the Integration:</strong></p><p>Considering that the cluster has datadog installed using the org api key</p><ol><li>In Datadog, go to Monitors → Manage Monitors</li><li>Create a test monitor or edit an existing one</li><li>In the Say what’s happening section:</li></ol><ul><li>Add @pagerduty-[service-name] in the message</li><li>Example: @pagerduty-web-app-alerts</li></ul><p>4. Save the monitor</p><p>5. Test by triggering the monitor condition</p><h3>Step 3: Verify the Connection</h3><p><strong>Check if it’s working:</strong></p><ol><li>Trigger a test alert in Datadog</li><li>Check PagerDuty Incidents page</li><li>Verify the incident appears with correct details</li><li>Confirm team members receive notifications</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vj3P_3h0KRGqHeZ9D26FLQ.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8f1f0a9acbfb" width="1" height="1" alt=""><hr><p><a href="https://blog.devops.dev/getting-started-with-services-in-pagerduty-8f1f0a9acbfb">Getting Started with Services in Pagerduty</a> was originally published in <a href="https://blog.devops.dev/">DevOps.dev</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Secure CI/CD Pipeline Architecture with AWS Systems Manager(SSM)]]></title>
            <link>https://aws.plainenglish.io/secure-ci-cd-pipeline-architecture-with-aws-systems-manager-ssm-0d6ff14c7671?source=rss-9e2186f06b63------2</link>
            <guid isPermaLink="false">https://medium.com/p/0d6ff14c7671</guid>
            <category><![CDATA[aws-system-manager]]></category>
            <category><![CDATA[sanjal-s-eralil]]></category>
            <category><![CDATA[ssm-agent]]></category>
            <category><![CDATA[sms]]></category>
            <category><![CDATA[ci-cd-pipeline]]></category>
            <dc:creator><![CDATA[Sanjal S Eralil]]></dc:creator>
            <pubDate>Wed, 01 Oct 2025 05:09:27 GMT</pubDate>
            <atom:updated>2025-10-01T05:09:27.116Z</atom:updated>
            <content:encoded><![CDATA[<p>AWS Systems Manager (SSM) revolutionizes infrastructure management by replacing traditional SSH-based access with a secure, polling-based architecture that eliminates inbound connections while providing comprehensive operational control.</p><h3>Core Concept: The Paradigm Shift</h3><h3>Traditional SSH Model (What We’re Replacing)</h3><pre>┌─────────────┐    SSH (Port 22)    ┌─────────────┐<br>│   Client    │ ──────────────────► │   Server    │<br>│ (Admin PC)  │   Direct Connection │ (EC2 Instance)│<br>└─────────────┘                     └─────────────┘</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/583/1*ACpqiDC_Hq-VEe2DvhI_oQ.png" /><figcaption>PORT 22 must be open in inbound rules</figcaption></figure><p><strong>SSH Model Challenges:</strong></p><ul><li>Requires inbound port 22 open to internet</li><li>SSH key management complexity</li><li>Direct attack surface exposure</li><li>Manual firewall rule management</li><li>VPN dependencies for private instances</li></ul><h3>SSM Architecture (The Modern Approach)</h3><pre>┌─────────────┐   HTTPS (443)   ┌─────────────┐   Message Queue   ┌─────────────┐   HTTPS Polling   ┌─────────────┐<br>│   Client    │ ───────────────► │ AWS SSM API │ ─────────────────► │ SSM Agent   │ ─────────────────► │   Server    │<br>│ (GitHub CI) │                  │   Service   │                    │ (Outbound)  │                    │(EC2/Bastion)│<br>└─────────────┘                  └─────────────┘                    └─────────────┘                    └─────────────┘</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/568/1*u5J4WhMEHwkPTzoqqoihiA.png" /></figure><blockquote>The SSM command flow when involved with a bastion(ec2) and a CI/CD runner works like this:</blockquote><blockquote>The CI/CD runner (e.g., GitHub runner) sends the command to the AWS SSM API over HTTPS (port 443). This command is addressed to a target instance (either the bastion host or a regular EC2 instance).</blockquote><blockquote>The AWS SSM service stores this command in its queue for the target instance.</blockquote><blockquote>The target instance (could be a bastion host or any EC2 with SSM Agent) continuously polls the AWS SSM service over outbound HTTPS (port 443) to see if any commands are awaiting execution.</blockquote><blockquote>Once the command is detected by the polling SSM Agent on the target instance, it executes the command locally.</blockquote><blockquote>The execution result is sent back to AWS SSM service over HTTPS (port 443).</blockquote><blockquote>The CI/CD runner can query the SSM service for command status and output.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/574/1*C1c465lATOydtfXPv7BF3A.png" /><figcaption>no inbound ports open</figcaption></figure><p><strong>SSM Model Benefits:</strong></p><ul><li><strong>Zero inbound ports</strong> required on server</li><li><strong>Outbound-only</strong> connections from server</li><li><strong>IAM-based authentication</strong> (no SSH keys)</li><li><strong>Comprehensive audit trail</strong> in CloudTrail</li><li><strong>Works with private subnets</strong> seamlessly</li></ul><h3>Technical Architecture Deep Dive</h3><h3>The Polling Revolution</h3><p>The fundamental difference is <strong>connection direction</strong>:</p><p><strong>SSH Approach:</strong></p><pre>Client ──────────► Server (Listening on Port 22)<br>       Inbound Connection</pre><p><strong>SSM Approach:</strong></p><pre>Server ──────────► AWS SSM API ──────────► Client<br>       Outbound Polling    Command Queue</pre><h3>Network Flow Analysis</h3><h4>Traditional SSH Security Groups</h4><pre># Bastion Host Security Group (SSH Model)<br>Inbound Rules:<br>  - Type: SSH<br>    Protocol: TCP<br>    Port: 22<br>    Source: 0.0.0.0/0  # ⚠️ Internet exposed<br>    Description: &quot;SSH access&quot;</pre><pre>Outbound Rules:<br>  - Type: All Traffic<br>    Protocol: All<br>    Port: All<br>    Destination: 0.0.0.0/0</pre><h4>SSM Security Groups</h4><pre># Bastion Host Security Group (SSM Model)<br>Inbound Rules: []  # ✅ No inbound ports needed</pre><pre>Outbound Rules:<br>  - Type: HTTPS<br>    Protocol: TCP<br>    Port: 443<br>    Destination: 0.0.0.0/0<br>    Description: &quot;AWS API access only&quot;</pre><h3>SSM Agent Architecture</h3><pre>┌─────────────────────────────────────────────────────────────┐<br>│                    EC2 Instance / Bastion Host              │<br>├─────────────────────────────────────────────────────────────┤<br>│  ┌─────────────┐    ┌─────────────┐    ┌─────────────┐     │<br>│  │ SSM Agent   │    │   Command    │    │   Results   │     │<br>│  │ (Daemon)    │───►│  Execution   │───►│ Collection  │     │<br>│  └─────────────┘    │   Engine     │    │   Module    │     │<br>│         │            └─────────────┘    └─────────────┘     │<br>│         ▼                                       │            │<br>│  Polling Loop                                   ▼            │<br>│  Every 15-30s                            Send Results        │<br>└─────────────────────────────────────────────────────────────┘<br>         │                                       ▲<br>         ▼ HTTPS (443) Outbound                  │ HTTPS (443)<br>┌─────────────────────────────────────────────────────────────┐<br>│                     AWS SSM Service                         │<br>├─────────────────────────────────────────────────────────────┤<br>│  ┌─────────────┐    ┌─────────────┐    ┌─────────────┐     │<br>│  │   Message   │    │   Command    │    │   Result    │     │<br>│  │   Broker    │    │    Queue     │    │   Storage   │     │<br>│  └─────────────┘    └─────────────┘    └─────────────┘     │<br>└─────────────────────────────────────────────────────────────┘<br>         ▲<br>         │ HTTPS (443)<br>         ▼<br>┌─────────────────────────────────────────────────────────────┐<br>│                    Client (GitHub Actions)                  │<br>└─────────────────────────────────────────────────────────────┘</pre><h3>How SSM Agent Works</h3><h3>Step-by-Step Workflow</h3><h4>1. Initial Registration</h4><pre># On instance boot, SSM Agent registers itself<br>POST /register HTTP/1.1<br>Host: ssm.us-west-2.amazonaws.com</pre><pre>{<br>  &quot;InstanceId&quot;: &quot;i-1234567890abcdef0&quot;,<br>  &quot;Region&quot;: &quot;us-west-2&quot;, <br>  &quot;Platform&quot;: &quot;Amazon Linux&quot;,<br>  &quot;AgentVersion&quot;: &quot;3.2.582.0&quot;<br>}</pre><h4>2. Continuous Polling Loop</h4><pre># Simplified SSM Agent polling logic<br>while True:<br>    try:<br>        # Agent makes outbound HTTPS call<br>        response = call_ssm_api(&quot;GetMessages&quot;, instance_id)<br>        <br>        if response.has_commands():<br>            for command in response.commands:<br>                result = execute_command(command)<br>                send_result(command.id, result)<br>        <br>        time.sleep(15)  # Default polling interval<br>        <br>    except NetworkError:<br>        exponential_backoff()<br>    except AuthError:<br>        refresh_credentials()</pre><h4>3. Command Execution Flow</h4><pre>Client Command               │ SSM Service              │ EC2 Instance<br>                            │                          │<br>aws ssm send-command ──────►│ Store in queue          │<br>                            │                          │<br>                            │                          │◄──── Agent polls every 15s<br>                            │ Return pending command ──►│<br>                            │                          │ Execute locally:<br>                            │                          │ /bin/bash -c &quot;ls -la&quot;<br>                            │                          │<br>                            │◄─── Send results back ───│<br>Return results to client ◄──│                          │</pre><h3>Network Security Analysis</h3><h4>Connection Patterns</h4><pre>Traditional SSH:<br>Internet ──► Bastion Host (Port 22 Open) ──► Private Network<br>           ⚠️ Inbound Attack Surface</pre><pre>SSM Architecture:<br>Bastion Host ──► AWS SSM API (Port 443) ◄── Client<br>              ✅ Outbound Only, No Attack Surface</pre><h4>Authentication Comparison</h4><p><strong>SSH Authentication:</strong></p><ul><li>SSH private/public key pairs</li><li>Key distribution and rotation</li><li>Potential for key compromise</li><li>Manual access management</li></ul><p><strong>SSM Authentication:</strong></p><ul><li>AWS IAM roles and policies</li><li>Temporary credentials via STS</li><li>Fine-grained permissions</li><li>Centralized access control</li></ul><h3>Implementation Guide</h3><h3>Prerequisites and Setup</h3><h4>Modern AMI Support (Pre-installed)</h4><p>SSM Agent comes pre-installed on most modern AWS AMIs:</p><ul><li>✅ Amazon Linux 2023, Amazon Linux 2</li><li>✅ Ubuntu 18.04, 20.04, 22.04</li><li>✅ Windows Server 2016, 2019, 2022</li><li>✅ SUSE Linux Enterprise Server 15</li><li>✅ Red Hat Enterprise Linux 8, 9</li></ul><p><em>Note: For older AMIs or on-premises servers, manual installation may be required.</em></p><h4>Essential IAM Role Configuration</h4><p><strong>Trust Policy:</strong></p><pre>{<br>  &quot;Version&quot;: &quot;2012-10-17&quot;,<br>  &quot;Statement&quot;: [<br>    {<br>      &quot;Effect&quot;: &quot;Allow&quot;,<br>      &quot;Principal&quot;: {&quot;Service&quot;: &quot;ec2.amazonaws.com&quot;},<br>      &quot;Action&quot;: &quot;sts:AssumeRole&quot;<br>    }<br>  ]<br>}</pre><p><strong>Required Permissions:</strong></p><pre>{<br>  &quot;Version&quot;: &quot;2012-10-17&quot;,<br>  &quot;Statement&quot;: [<br>    {<br>      &quot;Effect&quot;: &quot;Allow&quot;,<br>      &quot;Action&quot;: [<br>        &quot;ssm:UpdateInstanceInformation&quot;,<br>        &quot;ssm:SendCommand&quot;,<br>        &quot;ssm:GetCommandInvocation&quot;,<br>        &quot;ssm:GetDocument&quot;<br>      ],<br>      &quot;Resource&quot;: &quot;*&quot;<br>    },<br>    {<br>      &quot;Effect&quot;: &quot;Allow&quot;, <br>      &quot;Action&quot;: [<br>        &quot;ec2messages:*&quot;<br>      ],<br>      &quot;Resource&quot;: &quot;*&quot;<br>    }<br>  ]<br>}</pre><h3>Network Configuration</h3><h4>For Private Subnets (Recommended)</h4><pre>┌─────────────────────────────────────────────────────────────┐<br>│                        VPC                                   │<br>├─────────────────────────────────────────────────────────────┤<br>│  ┌─────────────┐                 ┌─────────────┐            │<br>│  │   Public    │                 │   Private   │            │<br>│  │   Subnet    │                 │   Subnet    │            │<br>│  │             │                 │             │            │<br>│  │ NAT Gateway │◄────────────────│ Bastion +   │            │<br>│  │             │                 │ SSM Agent   │            │<br>│  └─────────────┘                 └─────────────┘            │<br>│         │                                                   │<br>│         ▼                                                   │<br>│  Internet Gateway                                           │<br>└─────────────────────────────────────────────────────────────┘<br>         │<br>         ▼<br>    Internet (AWS APIs)</pre><p><strong>Route Table:</strong></p><pre>Destination    │ Target<br>0.0.0.0/0     │ NAT Gateway (for outbound)<br>10.0.0.0/16   │ Local (for VPC traffic)</pre><h4>Alternative: VPC Endpoints (Enhanced Security)</h4><pre># Create VPC Endpoints for full private communication<br>aws ec2 create-vpc-endpoint \<br>    --vpc-id vpc-12345678 \<br>    --service-name com.amazonaws.region.ssm \<br>    --vpc-endpoint-type Interface</pre><h3>Practical Comparison: SSH vs SSM</h3><h3>SSH-based Bastion Access</h3><pre># Traditional SSH workflow<br>ssh-keygen -t rsa -b 4096                    # Generate keys<br>scp script.sh user@bastion:/tmp/              # Copy files<br>ssh user@bastion                              # Connect<br>sudo /tmp/script.sh                           # Execute<br>exit                                          # Disconnect</pre><p><strong>Issues with SSH Approach:</strong></p><ul><li>Exposed port 22 to internet</li><li>SSH key management overhead</li><li>No audit trail for commands</li><li>VPN required for private instances</li><li>Manual file transfer required</li></ul><h3>SSM-based Management</h3><pre># Modern SSM workflow<br>aws s3 cp script.sh s3://bucket/scripts/      # Upload script<br>aws ssm send-command \<br>    --document-name &quot;AWS-RunShellScript&quot; \<br>    --parameters &#39;commands=[&quot;aws s3 cp s3://bucket/scripts/script.sh /tmp/&quot;,&quot;chmod +x /tmp/script.sh&quot;,&quot;/tmp/script.sh&quot;]&#39; \<br>    --targets &quot;Key=instanceids,Values=i-1234567890abcdef0&quot;</pre><p><strong>Advantages of SSM Approach:</strong></p><ul><li>✅ No inbound ports on bastion</li><li>✅ Automatic audit logging</li><li>✅ Works with private instances</li><li>✅ No SSH key management</li><li>✅ Scriptable and automatable</li></ul><h3>Real-World Implementation Example</h3><h3>GitHub Actions + SSM Pipeline</h3><pre>name: Automate deployment via SSM<br>jobs:<br>  deployment:<br>    runs-on: ubuntu-latest<br>    steps:<br>    - name: Copy deployment scripts to S3 bucket<br>      run: aws s3 sync ./scripts/ s3://deployment-bucket/scripts/<br>      <br>    - name: Run deployment script through SSM<br>      run: |<br>        aws ssm send-command \<br>          --document-name &quot;AWS-RunShellScript&quot; \<br>          --parameters &#39;commands=[<br>            &quot;mkdir -p /opt/deployment&quot;,<br>            &quot;aws s3 cp s3://deployment-bucket/scripts/deploy.sh /opt/deployment/&quot;,<br>            &quot;chmod u+x /opt/deployment/deploy.sh&quot;,<br>            &quot;/opt/deployment/deploy.sh&quot;<br>          ]&#39; \<br>          --targets &quot;Key=tag:Environment,Values=production&quot;</pre><p>This approach provides:</p><ul><li><strong>Zero SSH exposure</strong> on production systems</li><li><strong>Complete automation</strong> through CI/CD</li><li><strong>Full audit trail</strong> of all deployments</li><li><strong>Secure file distribution</strong> via S3</li><li><strong>No VPN dependencies</strong> for private infrastructure</li></ul><h3>Key Takeaways</h3><h3>Why SSM is Superior</h3><ol><li><strong>Security by Design</strong>: Eliminates inbound attack surface entirely</li><li><strong>Zero Trust Architecture</strong>: Every connection authenticated via IAM</li><li><strong>Operational Excellence</strong>: Complete audit trail and automation support</li><li><strong>Scalability</strong>: Works identically across thousands of instances</li><li><strong>Compliance</strong>: Built-in logging meets regulatory requirements</li></ol><h3>When to Use SSM vs SSH</h3><p><strong>Use SSM When:</strong></p><ul><li>Managing production infrastructure</li><li>Requiring audit compliance</li><li>Working with private subnets</li><li>Implementing CI/CD automation</li><li>Managing at scale (10+ instances)</li></ul><p><strong>SSH May Still Be Appropriate For:</strong></p><ul><li>Development environments</li><li>One-off debugging sessions</li><li>Legacy systems without SSM support</li><li>Direct file transfer requirements</li></ul><p>AWS Systems Manager represents the future of secure infrastructure management, eliminating traditional remote access vulnerabilities while providing superior operational capabilities for modern DevOps practices.</p><h3>A message from our Founder</h3><p><strong>Hey, </strong><a href="https://linkedin.com/in/sunilsandhu"><strong>Sunil</strong></a><strong> here.</strong> I wanted to take a moment to thank you for reading until the end and for being a part of this community.</p><p>Did you know that our team run these publications as a volunteer effort to over 3.5m monthly readers? <strong>We don’t receive any funding, we do this to support the community. ❤️</strong></p><p>If you want to show some love, please take a moment to <strong>follow me on </strong><a href="https://linkedin.com/in/sunilsandhu"><strong>LinkedIn</strong></a><strong>, </strong><a href="https://tiktok.com/@messyfounder"><strong>TikTok</strong></a>, <a href="https://instagram.com/sunilsandhu"><strong>Instagram</strong></a>. You can also subscribe to our <a href="https://newsletter.plainenglish.io/"><strong>weekly newsletter</strong></a>.</p><p>And before you go, don’t forget to <strong>clap</strong> and <strong>follow</strong> the writer️!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0d6ff14c7671" width="1" height="1" alt=""><hr><p><a href="https://aws.plainenglish.io/secure-ci-cd-pipeline-architecture-with-aws-systems-manager-ssm-0d6ff14c7671">Secure CI/CD Pipeline Architecture with AWS Systems Manager(SSM)</a> was originally published in <a href="https://aws.plainenglish.io/">AWS in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Monolithic vs Microservices Architecture: Which Should You Choose?]]></title>
            <link>https://sanjal-eralil.medium.com/monolithic-vs-microservices-architecture-which-should-you-choose-1bf948917ac9?source=rss-9e2186f06b63------2</link>
            <guid isPermaLink="false">https://medium.com/p/1bf948917ac9</guid>
            <category><![CDATA[sanjal-s-eralil]]></category>
            <category><![CDATA[microservice-architecture]]></category>
            <category><![CDATA[monolithic-architecture]]></category>
            <category><![CDATA[design-systems]]></category>
            <dc:creator><![CDATA[Sanjal S Eralil]]></dc:creator>
            <pubDate>Tue, 30 Sep 2025 18:27:46 GMT</pubDate>
            <atom:updated>2025-09-30T18:27:46.875Z</atom:updated>
            <content:encoded><![CDATA[<p>When building modern applications, one of the most critical architectural decisions you’ll face is choosing between monolithic and microservices architecture. Each approach has distinct advantages and trade-offs that can significantly impact your development process, deployment strategy, and long-term maintenance. Let’s explore both architectures to help you make an informed decision for your next project.</p><h3>Understanding Monolithic Architecture</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/329/1*2jayenV6QaSvm4vYcHTW1w.png" /></figure><p>A monolithic application is built as a single, unified codebase where all components work together as one piece of software. This architecture was the standard approach before microservices emerged and remains a popular starting point for new applications. In a monolith, your entire application — whether it’s an API or a frontend — typically lives in one repository and produces a single executable. The entire application is developed, deployed, and scaled as one cohesive unit.</p><h3>The Advantages of Monoliths</h3><p><strong>Simplicity in Development</strong></p><p>Monolithic architecture shines in its straightforward development process. All your code exists in one place, eliminating the need to juggle multiple repositories. Testing becomes more manageable since you’re working with a single application. For small to medium-sized projects, this simplicity translates directly into faster time-to-market, which is why most applications naturally begin as monoliths.</p><p><strong>Streamlined Deployment</strong></p><p>With only one application to deploy, your release process remains uncomplicated. You maintain a single CI/CD pipeline and infrastructure setup, reducing operational overhead and the potential for deployment-related issues.</p><p><strong>Easier Debugging</strong></p><p>When problems arise, you have just one application to investigate. The real benefit becomes apparent during local development, where stepping through code in a single application is far more straightforward than coordinating multiple services running on different ports.</p><p><strong>Performance Considerations</strong></p><p>While not necessarily more performant overall, monoliths eliminate network latency between components. When different parts of your code need to communicate, they do so with negligible overhead since everything runs in the same process.</p><h3>The Limitations of Monoliths</h3><p>Despite these benefits, monolithic architecture presents several challenges that often drive teams toward microservices as applications mature.</p><p><strong>Growing Complexity</strong></p><p>Monoliths tend to become bloated over time. Even with the best coding practices and clean architecture principles, applications eventually reach a size where they become difficult to understand and maintain. If onboarding a new developer takes over a month just to explain the application’s structure, you’re likely dealing with an oversized monolith.</p><p><strong>Slow Release Cycles</strong></p><p>As your application accumulates features, the testing burden before each release grows proportionally. Since the application is a single component, even minor changes require releasing the entire system. This can lead to lengthy deployment processes, especially with mature CI/CD pipelines that include traffic switching, monitoring, and validation steps.</p><p><strong>Scaling Limitations</strong></p><p>Perhaps the most significant drawback is the inability to scale specific components independently. When one feature experiences high traffic, you must scale the entire application — either by spinning up additional instances or upgrading server resources. This approach becomes increasingly expensive as your application grows.</p><h3>Embracing Microservices Architecture</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/632/1*kLAhOWIacazRgC7ccY82eQ.png" /></figure><p>Microservices architecture addresses monolithic limitations by decomposing large applications into smaller, independent components. Each microservice has a single responsibility and typically handles one specific business function. For example, Netflix might have separate services for search, video streaming, and recommendations. Each service is self-contained with its own infrastructure and database, versioned independently, and released separately from others.</p><h3>Communication Between Microservices</h3><p>Microservices need effective communication mechanisms to function as a cohesive system. Three primary approaches exist:</p><p><strong>API Communication</strong> provides synchronous interaction through HTTP endpoints. Each microservice exposes its own API that others can call, typically including health check endpoints for monitoring. While this introduces some latency, technologies like gRPC and compression can optimize performance.</p><p><strong>Message Brokers</strong> enable asynchronous communication for non-time-sensitive operations. For instance, sending emails doesn’t require immediate responses, so placing messages on a queue allows your application to continue processing while a dedicated microservice handles email delivery. Popular options include RabbitMQ and AWS SQS.</p><p><strong>Service Mesh</strong> handles communication infrastructure, managing service discovery and reliability across your entire system.</p><h3>The Benefits of Microservices</h3><p><strong>Enhanced Maintainability</strong></p><p>Well-designed microservices focus on single responsibilities, making them easier to maintain. Some services may require no changes at all once deployed. An email service, for example, likely has stable requirements and can run indefinitely without updates. Compare this to monoliths, where every deployment risks affecting unrelated functionality.</p><p><strong>Improved Reliability</strong></p><p>Deploying individual microservices leaves others unaffected, reducing the risk of system-wide failures. This is why major platforms like GitHub rarely experience complete outages — typically, only specific features are impacted while the rest of the system continues functioning normally.</p><p><strong>Technology Flexibility</strong></p><p>Teams can select appropriate technology stacks for specific services. While some languages excel at particular tasks — Python for machine learning, for instance — it’s generally wise to maintain consistency across services and deviate only when clearly beneficial.</p><p><strong>Independent Team Workflows</strong></p><p>Different teams can work on separate microservices simultaneously without stepping on each other’s toes, enabling parallel development at scale.</p><p><strong>Targeted Scalability</strong></p><p>When specific services experience increased load, you can scale only those components rather than the entire application, potentially saving significant infrastructure costs.</p><p><strong>Rapid Deployment</strong></p><p>Microservices facilitate multiple daily releases by limiting changes to small, easily testable components that can be deployed independently without risking the broader system.</p><h3>The Challenges of Microservices</h3><p><strong>Local Development Complexity</strong></p><p>Running highly interdependent microservices locally can be challenging. Developers must either mock external service endpoints or spin up multiple Docker containers, adding complexity to the development environment.</p><p><strong>Debugging Difficulties</strong></p><p>Production issues require investigating multiple services to trace problems to their source. This necessitates robust monitoring systems to track service health and behavior, which is where service mesh solutions become particularly valuable.</p><p><strong>Infrastructure Costs</strong></p><p>While microservices can reduce costs at scale through targeted scaling, small applications may see increased infrastructure expenses. The cost benefits typically emerge only when dealing with large applications with uneven traffic distribution.</p><p><strong>Increased System Complexity</strong></p><p>Microservices introduce complexity across your entire system — multiple CI/CD pipelines, inter-service communication management, and comprehensive monitoring requirements. With just a few services this is manageable, but hundreds of microservices can become an operational challenge.</p><h3>Making the Right Choice</h3><p>For most new applications without existing infrastructure, <strong>starting with a monolith is the recommended approach</strong>. There’s no point solving scaling problems you don’t yet have. Startups especially should prioritize rapid market entry, addressing scaling concerns only when they materialize with actual paying customers. Even giants like Netflix began as monoliths, transitioning to microservices only when scaling demands required it.</p><p><strong>Consider starting with microservices</strong> if you’re integrating into existing workflows or expect high traffic from day one. In such cases, beginning with microservices or a semi-monolithic approach — where key components are separated — makes more sense. Once live, you can evaluate whether further decomposition is beneficial.</p><p>When transitioning from monolith to microservices, analyze which components could function independently. Identify scaling bottlenecks and determine which operations can run asynchronously versus synchronously. This measured approach allows you to decompose your monolith strategically rather than all at once.</p><h3>Conclusion</h3><blockquote>Both monolithic and microservices architectures have their place in modern software development. Monoliths offer simplicity, rapid development, and straightforward operations — ideal for new projects and small to medium applications. Microservices provide scalability, reliability, and flexibility — essential for large, complex applications with diverse scaling needs. The key is understanding your current requirements, anticipated growth, and team capabilities, then choosing the architecture that best serves your specific context rather than following trends or assumptions about what “modern” architecture should look like.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*owqcdLGIt-RTHtxX4MAIjw.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1bf948917ac9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[EKS Access Entries:Policies:IAM Roles: A Complete Guide]]></title>
            <link>https://aws.plainenglish.io/eks-access-entries-policies-iam-roles-a-complete-guide-48e666688dba?source=rss-9e2186f06b63------2</link>
            <guid isPermaLink="false">https://medium.com/p/48e666688dba</guid>
            <category><![CDATA[kubernetes-rbac]]></category>
            <category><![CDATA[aws-eks]]></category>
            <category><![CDATA[sanjal-s-eralil]]></category>
            <category><![CDATA[eks-access-entry]]></category>
            <category><![CDATA[aws-iam-role]]></category>
            <dc:creator><![CDATA[Sanjal S Eralil]]></dc:creator>
            <pubDate>Mon, 29 Sep 2025 17:03:00 GMT</pubDate>
            <atom:updated>2025-10-01T04:31:27.791Z</atom:updated>
            <content:encoded><![CDATA[<p>When working with Amazon EKS (Elastic Kubernetes Service), one of the most confusing aspects is understanding how IAM Roles, IAM Policies, and EKS Access Entries work together. If you’ve ever wondered why you need both IAM policies AND EKS access policies, or how these components interact to control permissions, this guide will clear up the confusion.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/585/1*wfsWHVaLmWYavsoZAxCnaA.png" /></figure><h3>The Three Core Components</h3><h3>1. IAM Role: The Identity Layer</h3><p>An <strong>IAM Role</strong> is fundamentally about identity — it defines <strong>WHO</strong> can do something in AWS. Think of it as a set of permissions that AWS services, users, or applications can temporarily “assume” or take on.</p><p><strong>Key Characteristics:</strong></p><ul><li>Acts as an identity that can be assumed</li><li>Works across all AWS services</li><li>Defines trust relationships (who can assume the role)</li></ul><p>Example IAM Role Trust Policy:</p><pre>{<br>  &quot;Version&quot;: &quot;2012-10-17&quot;,<br>  &quot;Statement&quot;: [<br>    {<br>      &quot;Effect&quot;: &quot;Allow&quot;,<br>      &quot;Principal&quot;: {<br>        &quot;Service&quot;: &quot;ec2.amazonaws.com&quot;<br>      },<br>      &quot;Action&quot;: &quot;sts:AssumeRole&quot;<br>    }<br>  ]<br>}</pre><h3>2. IAM Policy: The Permission Layer</h3><p>An <strong>IAM Policy</strong> is a document that defines permissions — it specifies <strong>WHAT</strong> actions are allowed or denied on which resources. These policies get attached to roles, users, or groups to grant them specific permissions.</p><p><strong>Types of IAM Policies:</strong></p><ul><li><strong>AWS Managed:</strong> Created and maintained by AWS</li><li><strong>Customer Managed:</strong> Created and maintained by you</li><li><strong>Inline:</strong> Attached directly to a specific role/user</li></ul><p><strong>Example IAM Policy:</strong></p><pre>{<br>  &quot;Version&quot;: &quot;2012-10-17&quot;,<br>  &quot;Statement&quot;: [<br>    {<br>      &quot;Effect&quot;: &quot;Allow&quot;,<br>      &quot;Action&quot;: &quot;s3:GetObject&quot;,<br>      &quot;Resource&quot;: &quot;arn:aws:s3:::my-bucket/*&quot;<br>    }<br>  ]<br>}</pre><h3>3. EKS Access Entry: The Bridge Layer</h3><p><strong>EKS Access Entries</strong> are the newest addition, introduced in 2023 as a modern replacement for the traditional aws-auth ConfigMap. They act as a bridge that connects AWS IAM identities to Kubernetes RBAC (Role-Based Access Control) within an EKS cluster.</p><p><strong>Key Characteristics:</strong></p><ul><li>EKS cluster-specific (not AWS-wide)</li><li>Maps IAM principals to Kubernetes permissions</li><li>Managed through native AWS APIs</li><li>Provides better security and audit trail than ConfigMaps</li></ul><h4>How They Work Together</h4><p>The relationship between these components follows a clear flow:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/626/1*aW2vwg8OgR7uwWj7EHh7ug.png" /></figure><p><strong>Scope and Purpose Comparison</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*MYnFS32FC0cdwiaMZ9AHbQ.png" /></figure><h3><strong>Traditional vs Modern Approach</strong></h3><p><strong>The Old Way: aws-auth ConfigMap</strong></p><pre>apiVersion: v1<br>kind: ConfigMap<br>metadata:<br>  name: aws-auth<br>  namespace: kube-system<br>data:<br>  mapRoles: |<br>    - rolearn: arn:aws:iam::123456789012:role/MyRole<br>      username: my-user<br>      groups:<br>      - system:masters</pre><p><strong>The New Way: Access Entries</strong></p><pre>resource &quot;aws_eks_access_entry&quot; &quot;example&quot; {<br>  cluster_name      = aws_eks_cluster.example.name<br>  principal_arn     = aws_iam_role.example.arn<br>  kubernetes_groups = [&quot;system:masters&quot;]<br>  type             = &quot;STANDARD&quot;<br>}</pre><h3>Why Access Entries Are Better</h3><p><strong>Native AWS API Integration:</strong></p><ul><li>Managed through standard AWS APIs instead of Kubernetes ConfigMaps</li><li>Better integration with Infrastructure as Code tools</li></ul><p><strong>Enhanced Security:</strong></p><ul><li>No need to manage sensitive ConfigMaps in the kube-system namespace</li><li>Reduced risk of accidental misconfigurations</li></ul><p><strong>Improved Management:</strong></p><ul><li>Easier to manage with Terraform, CloudFormation, or AWS CLI</li><li>Better version control and change tracking</li></ul><p><strong>Superior Audit Trail:</strong></p><ul><li>Complete logging and tracking through AWS CloudTrail</li><li>Better visibility into access changes</li></ul><h3><strong>The Two-Policy System: A Common Confusion</strong></h3><h3>IAM Policies (AWS Level)</h3><p><strong>Attached to:</strong> IAM Role<br> <strong>Purpose:</strong> Control what the role can do in AWS services<br> <strong>Examples:</strong></p><ul><li>AmazonEKSWorkerNodePolicy</li><li>AmazonS3ReadOnlyAccess</li><li>EC2InstanceProfileForImageBuilder</li></ul><h3>EKS Access Policies (Kubernetes Level)</h3><p><strong>Attached to:</strong> EKS Access Entry<br> <strong>Purpose:</strong> Control what the role can do inside the EKS cluster<br> <strong>Examples:</strong></p><ul><li>AmazonEKSClusterAdminPolicy</li><li>AmazonEKSAdminPolicy</li><li>AmazonEKSEditPolicy</li><li>AmazonEKSViewPolicy</li></ul><h3>Visual Flow of the Two-Policy System</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/795/1*zlYTdA7QU1UXUFWAecgyXw.png" /></figure><h3>Implementation</h3><h4>AWS Management Console</h4><p>Navigate to your EKS cluster in the AWS Console and follow these steps:</p><ol><li>Select your cluster and click the <strong>“Access”</strong> tab</li><li>Choose <strong>“Create Access Entry”</strong></li><li>Select the IAM principal (role or user) you want to grant access</li><li>Map the principal to appropriate Kubernetes groups</li><li>Associate relevant EKS access policies</li></ol><p>This approach provides a user-friendly interface for managing access without requiring command-line expertise.</p><h4>AWS CLI</h4><p>For programmatic access management, use the AWS CLI:</p><p>bash</p><pre>aws eks create-access-entry \<br>  --cluster-name &lt;cluster-name&gt; \<br>  --principal-arn &lt;IAM-role-or-user-ARN&gt; \<br>  --type STANDARD \<br>  --kubernetes-groups system:masters</pre><p>The command supports different access types:</p><ul><li>STANDARD: General purpose access</li><li>EC2_LINUX: For EC2 Linux worker nodes</li><li>EC2_WINDOWS: For EC2 Windows worker nodes</li></ul><h4>Terraform Implementation</h4><p>Infrastructure as Code practitioners can use the aws_eks_access_entry resource:</p><p>hcl</p><pre>resource &quot;aws_eks_access_entry&quot; &quot;example&quot; {<br>  cluster_name       = aws_eks_cluster.example.name<br>  principal_arn      = aws_iam_role.example.arn<br>  kubernetes_groups  = [&quot;system:masters&quot;]<br>  type              = &quot;STANDARD&quot;<br>}</pre><p>This declarative approach ensures consistent access management across environments and provides version control for access configurations.</p><h3>Traditional Approach: aws-auth ConfigMap</h3><p>Before Access Entries, EKS clusters used a special Kubernetes ConfigMap called aws-auth in the kube-system namespace to manage IAM-to-Kubernetes authentication mapping.</p><h3>How It Works</h3><p>The aws-auth ConfigMap contains YAML data that maps IAM roles and users to Kubernetes users and groups. When someone attempts to access the cluster using AWS credentials, the EKS service consults this ConfigMap to determine their Kubernetes permissions.</p><h3>Implementation Process</h3><h4>Step 1: Retrieve Current Configuration</h4><p>bash</p><pre>kubectl -n kube-system get configmap/aws-auth -o yaml &gt; aws-auth.yaml</pre><h4>Step 2: Edit the Configuration</h4><p>Modify the aws-auth.yaml file to include your IAM principal mappings:</p><p>yaml</p><pre>apiVersion: v1<br>kind: ConfigMap<br>metadata:<br>  name: aws-auth<br>  namespace: kube-system<br>data:<br>  mapRoles: |<br>    - rolearn: arn:aws:iam::&lt;account-id&gt;:role/MyRole<br>      username: my-user<br>      groups:<br>        - system:masters<br>  mapUsers: |<br>    - userarn: arn:aws:iam::&lt;account-id&gt;:user/MyUser<br>      username: my-user<br>      groups:<br>        - system:masters</pre><h4>Step 3: Apply Changes</h4><p>bash</p><pre>kubectl apply -f aws-auth.yaml</pre><h3>A message from our Founder</h3><p><strong>Hey, </strong><a href="https://linkedin.com/in/sunilsandhu"><strong>Sunil</strong></a><strong> here.</strong> I wanted to take a moment to thank you for reading until the end and for being a part of this community.</p><p>Did you know that our team run these publications as a volunteer effort to over 3.5m monthly readers? <strong>We don’t receive any funding, we do this to support the community. ❤️</strong></p><p>If you want to show some love, please take a moment to <strong>follow me on </strong><a href="https://linkedin.com/in/sunilsandhu"><strong>LinkedIn</strong></a><strong>, </strong><a href="https://tiktok.com/@messyfounder"><strong>TikTok</strong></a>, <a href="https://instagram.com/sunilsandhu"><strong>Instagram</strong></a>. You can also subscribe to our <a href="https://newsletter.plainenglish.io/"><strong>weekly newsletter</strong></a>.</p><p>And before you go, don’t forget to <strong>clap</strong> and <strong>follow</strong> the writer️!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=48e666688dba" width="1" height="1" alt=""><hr><p><a href="https://aws.plainenglish.io/eks-access-entries-policies-iam-roles-a-complete-guide-48e666688dba">EKS Access Entries:Policies:IAM Roles: A Complete Guide</a> was originally published in <a href="https://aws.plainenglish.io/">AWS in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Application Load Balancer with AWS LoadBalancer Controller and Network LoadBalancer with non ALB…]]></title>
            <link>https://aws.plainenglish.io/application-load-balancer-with-aws-loadbalancer-controller-and-network-loadbalancer-with-non-alb-9e1a35b1e326?source=rss-9e2186f06b63------2</link>
            <guid isPermaLink="false">https://medium.com/p/9e1a35b1e326</guid>
            <category><![CDATA[application-load-balancer]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[sanjal-s-eralil]]></category>
            <category><![CDATA[alb]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Sanjal S Eralil]]></dc:creator>
            <pubDate>Mon, 08 Sep 2025 03:43:13 GMT</pubDate>
            <atom:updated>2025-09-18T01:52:47.539Z</atom:updated>
            <content:encoded><![CDATA[<h3>Application Load Balancer with AWS LoadBalancer Controller and Network LoadBalancer with non ALB Ingress</h3><blockquote>If you have deployed services for your application — such as a Java application or any other application running inside a Kubernetes cluster — and need to expose them to external traffic, an Ingress resource becomes essential. An Ingress manages external access by routing HTTP and HTTPS requests from outside the cluster to the appropriate internal services.</blockquote><blockquote>On AWS EKS, the common cloud-native approach is to use the AWS Load Balancer Controller, which provisions and manages an AWS Application Load Balancer (ALB) to handle ingress traffic efficiently with deep AWS integration. However, you also have other options like the widely adopted open-source NGINX Ingress Controller with Network Load Balancer, which runs entirely inside your Kubernetes cluster and offers powerful, flexible routing capabilities. Depending on your architecture, operational preferences, and cloud provider, you can choose the ingress solution that best fits your needs.</blockquote><blockquote>The below article comprehensively explain both approaches</blockquote><p>In an AWS EKS (Elastic Kubernetes Service) or Kubernetes cluster-based deployment, understanding the difference between ALB (Application Load Balancer) and the ALB Ingress Controller (now called AWS Load Balancer Controller) is essential because both can coexist but serve distinct roles in your architecture.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/586/1*t9g5Vgjgwntw69DA0w-tKg.png" /></figure><h3>ALB (Application Load Balancer)</h3><ul><li><strong>AWS Managed Service</strong>: The ALB itself is provisioned and managed by AWS outside of your Kubernetes cluster. It provides Layer 7 (HTTP/HTTPS) load balancing.</li><li><strong>Runs on Physical AWS Infrastructure</strong>: The ALB exists on AWS’s EC2 infrastructure, managed by AWS. You do not control the underlying hosts.</li><li><strong>Traffic Management</strong>: It is the entry point for external HTTP(S) traffic into your workloads. It forwards traffic to registered targets, which are typically the ENIs (Elastic Network Interfaces) or NodePorts associated with your Kubernetes cluster nodes.</li><li><strong>Access Logs</strong>: ALB can generate detailed access logs (client IP, request path, target, response code, etc.) and ship them to S3 or CloudWatch for analysis or compliance.</li><li>Datadog Visibility: The ALB is an AWS-managed resource and not a Kubernetes pod. Therefore:<br>It is not directly visible to the Datadog Agent running inside your cluster.</li><li>Metrics and logs about the ALB are fetched from AWS APIs (if you have AWS integration in Datadog), not from within your cluster agent.</li></ul><h3>ALB Ingress Controller (AWS Load Balancer Controller)</h3><ul><li><strong>Kubernetes Pod</strong>: This is a controller that runs inside your Kubernetes cluster, typically in the kube-system namespace. It watches for Kubernetes Ingress resources and manages the lifecycle of AWS ALBs accordingly.</li><li><strong>Controller Application</strong>: The controller interprets Ingress resources (and optionally Service resources with annotation) and automatically provisions/de-provisions/configures ALBs in AWS to route traffic to your cluster’s workloads.</li><li><strong>Application Logs</strong>: The controller pod generates its own logs (such as controller activity, error messages, reconciliation results), which represent the health and activities of the controller — not the end-user traffic</li><li>Datadog Visibility: Since the controller runs as a Kubernetes pod:</li><li>Its logs and metrics ARE visible to the Datadog Agent (if you configure Datadog logging/metrics collection for that namespace or pod).</li><li>You can get controller-level diagnostics, not traffic-level analytics.</li></ul><h3>How Both Work Together</h3><ul><li>Provisioning: The ALB Ingress Controller (AWS Load Balancer Controller) runs in your cluster. When you create or change Kubernetes Ingress resources, the controller reacts by making AWS API calls to create, modify, or delete ALBs in your AWS account as needed.</li><li>Traffic Flow: Once provisioned, the ALB (AWS resource) receives external HTTP(S) requests and routes them (according to your Ingress rules) to your Kubernetes service endpoints.</li></ul><p>Monitoring:</p><ul><li>The actual traffic data (hits, response times, logs) at the ALB is only visible from AWS logs/metrics or via cloud integrations with monitoring tools.</li><li>The ALB Ingress Controller’s own logs are about the AWS resource management operations and any reconciliation issues, and these are available to your cluster observability tools (like the Datadog agent running as a DaemonSet or Deployment).</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/921/1*Z7WcXzaN6s5rYXB_uFBT0g.png" /></figure><h3>Coexistence</h3><p>Yes, both exist simultaneously in a typical Kubernetes-on-AWS deployment using cloud-native load balancing:</p><ul><li>The controller is responsible for managing ALBs based on your declared Ingress resources.</li><li>The ALB is the actual infrastructure piece handling live production traffic from outside AWS to your Kubernetes workloads</li></ul><blockquote>The actual Ingress in Kubernetes is a resource — a set of rules that define how external HTTP(S) traffic should be routed to services inside the cluster.</blockquote><blockquote>The AWS Load Balancer Controller (formerly ALB Ingress Controller) is the Ingress Controller — a Kubernetes component (running as a pod inside the cluster) that watches Ingress resources and manages the provisioning and configuration of AWS Application Load Balancers (ALBs) accordingly.</blockquote><blockquote>The ALB (Application Load Balancer) itself is not the ingress controller; it is the AWS-managed load balancer resource created and managed by the AWS Load Balancer Controller based on the Ingress rules.</blockquote><blockquote>In short:</blockquote><blockquote>Ingress: Kubernetes config/resource defining routing rules.</blockquote><blockquote>Ingress Controller (AWS Load Balancer Controller): Software inside the cluster that interprets Ingress resources and manages ALB creation.</blockquote><blockquote>ALB: Actual AWS load balancer hardware/software managing external traffic.</blockquote><h3>Network Load Balancer (Alternate approach)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/568/1*NDtRMLjyROUwxoaDdzqRDg.png" /></figure><p>The Network Load Balancer (NLB) often appears in discussions about Kubernetes ingress and service exposure on AWS because it is one of the types of AWS Load Balancers used to expose services externally when not using the AWS Application Load Balancer (ALB).</p><p>Here’s why it comes into the picture:</p><ul><li>The <strong>AWS ALB operates at Layer 7 (application level)</strong> and is commonly used with the AWS Load Balancer Controller to handle Kubernetes Ingress resources.</li><li>When using an ingress controller like NGINX (which runs inside the cluster as pods), the ingress controller itself manages HTTP traffic but needs an external load balancer to expose it outside the cluster.</li><li>On AWS, that external load balancer is typically a Network Load Balancer (NLB) or sometimes a Classic Load Balancer.</li><li>The NLB operates at<strong> Layer 4 (transport level)</strong> forwarding TCP/UDP traffic to the NGINX ingress controller pods via a Kubernetes Service of type LoadBalancer.</li><li>So, with NGINX Ingress, the flow for external traffic often looks like this:</li><li>External client →<strong> NLB (AWS Network Load Balancer)</strong> → NGINX Ingress controller pods inside Kubernetes → backend services/pods.</li><li>The NLB is the entry point for traffic at the infrastructure (AWS) level, while NGINX handles HTTP routing within the cluster.</li></ul><p>In contrast, when using AWS ALB + AWS Load Balancer Controller:</p><ul><li>The ALB itself is the ingress point and handles Layer 7 routing directly, so there is no need for a separate NLB in front of the ingress controller.</li></ul><p>In summary:</p><ul><li>NLB often appears when you use <strong>NGINX </strong>(or other non-ALB ingress controllers) on AWS to expose the ingress controller pods externally.</li><li>It is an AWS load balancer type optimized for high performance and low latency at the network layer.</li><li>It is different from ALB in terms of layering and features but essential for exposing ingress pods without ALB.</li></ul><h3><strong>CONCLUSION</strong></h3><blockquote>The Application Load Balancer (ALB) and Network Load Balancer (NLB) are both AWS networking services that distribute incoming traffic, but they operate at different layers of the OSI model. ALB works at the application layer (Layer 7), providing advanced routing based on HTTP/S attributes, while NLB operates at the transport layer (Layer 4), handling raw TCP/UDP traffic with high performance and low latency</blockquote><h3>A message from our Founder</h3><p><strong>Hey, </strong><a href="https://linkedin.com/in/sunilsandhu"><strong>Sunil</strong></a><strong> here.</strong> I wanted to take a moment to thank you for reading until the end and for being a part of this community.</p><p>Did you know that our team run these publications as a volunteer effort to over 3.5m monthly readers? <strong>We don’t receive any funding, we do this to support the community. ❤️</strong></p><p>If you want to show some love, please take a moment to <strong>follow me on </strong><a href="https://linkedin.com/in/sunilsandhu"><strong>LinkedIn</strong></a><strong>, </strong><a href="https://tiktok.com/@messyfounder"><strong>TikTok</strong></a>, <a href="https://instagram.com/sunilsandhu"><strong>Instagram</strong></a>. You can also subscribe to our <a href="https://newsletter.plainenglish.io/"><strong>weekly newsletter</strong></a>.</p><p>And before you go, don’t forget to <strong>clap</strong> and <strong>follow</strong> the writer️!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9e1a35b1e326" width="1" height="1" alt=""><hr><p><a href="https://aws.plainenglish.io/application-load-balancer-with-aws-loadbalancer-controller-and-network-loadbalancer-with-non-alb-9e1a35b1e326">Application Load Balancer with AWS LoadBalancer Controller and Network LoadBalancer with non ALB…</a> was originally published in <a href="https://aws.plainenglish.io/">AWS in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>