<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by KELVI MANAVADARIA on Medium]]></title>
        <description><![CDATA[Stories by KELVI MANAVADARIA on Medium]]></description>
        <link>https://medium.com/@kelvi030117?source=rss-6a8f83281443------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*1kneJ4PzUH8JWpmQ</url>
            <title>Stories by KELVI MANAVADARIA on Medium</title>
            <link>https://medium.com/@kelvi030117?source=rss-6a8f83281443------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 17:11:10 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@kelvi030117/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Inside the Cluster: Securing Pod Communication with Kubernetes Network Policies]]></title>
            <link>https://medium.com/@kelvi030117/inside-the-cluster-securing-pod-communication-with-kubernetes-network-policies-3f3c345b7384?source=rss-6a8f83281443------2</link>
            <guid isPermaLink="false">https://medium.com/p/3f3c345b7384</guid>
            <category><![CDATA[networking]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <dc:creator><![CDATA[KELVI MANAVADARIA]]></dc:creator>
            <pubDate>Wed, 23 Apr 2025 18:36:37 GMT</pubDate>
            <atom:updated>2025-04-23T18:36:37.700Z</atom:updated>
            <content:encoded><![CDATA[<p>In the default configuration of most Kubernetes clusters, all pods can communicate with each other without restrictions. This flat network topology is simple but presents significant security risks in production environments. Kubernetes Network Policies address this challenge by providing a mechanism to control traffic flow between pods, enforcing the principle of least privilege at the network level.</p><p>This blog will provide you overview of Kubernetes Network Policies and how it secure pod-to-pod communication in your clusters.</p><h3>Default Network Behavior</h3><p>Without Network Policies, Kubernetes follows an “allow all” model:</p><ul><li>Every pod can communicate with every other pod within the cluster</li><li>Every pod can communicate with external resources</li><li>All ingress and egress traffic is allowed</li></ul><h3>Our SuperHero — Network Policy</h3><p>Network Policies are Kubernetes API resources that specify how groups of pods are allowed to communicate with each other and with other network endpoints. They act as a firewall within your cluster, controlling both ingress (incoming) and egress (outgoing) traffic at the pod level.</p><h3>How Network Policies Work</h3><h4>Policy Enforcement</h4><p>Network Policies are implemented by the Container Network Interface (CNI) plugin in your cluster. When a Network Policy is applied, the CNI plugin translates the policy into rules for the underlying network infrastructure (such as iptables rules).</p><h4>Key Concepts</h4><p>Network Policies use the following key concepts to define traffic rules:</p><ol><li><strong>Pod Selectors</strong>: Labels that identify which pods the policy applies to.</li><li><strong>Namespace Selectors</strong>: Labels that identify which namespaces the policy applies to.</li><li><strong>CIDR Blocks</strong>: IP address ranges for defining external endpoints.</li><li><strong>Port Specifications</strong>: TCP/UDP ports and protocols to allow or deny.</li><li><strong>ingress</strong>: Defines rules for incoming connections to selected pods.</li></ol><ul><li><strong>from</strong>: Sources allowed to connect (pod selectors, namespace selectors, or IP blocks)</li><li><strong>ports</strong>: Specific ports and protocols allowed</li></ul><p><strong>6. egress</strong>: Defines rules for outgoing connections from selected pods.</p><ul><li><strong>to</strong>: Destinations pods can connect to (pod selectors, namespace selectors, or IP blocks)</li><li><strong>ports</strong>: Specific ports and protocols allowed</li></ul><h4>Policy Evaluation</h4><p>When multiple Network Policies select the same pod, the resulting behavior follows these rules:</p><ul><li>Policies are additive — if any policy allows a connection, it’s allowed</li><li>If no policies match a particular connection, it’s denied (for targeted pods)</li><li>Pods not selected by any Network Policy remain unrestricted</li></ul><h4>Prerequisites for Network Policies</h4><p><strong>Supported CNI Plugins</strong></p><p>Not all Kubernetes installations support Network Policies out of the box. You need a CNI plugin that implements them, such as:</p><ul><li>Calico</li><li>Cilium</li><li>Weave Net</li><li>Antrea</li><li>Kube-router</li></ul><p>Common CNI plugins that <strong>do not</strong> support Network Policies include:</p><ul><li>Flannel (unless used with an additional plugin)</li><li>Basic kubenet</li></ul><h3>Common Network Policy Patterns</h3><h4>Default Deny All Ingress Traffic</h4><p>This policy prevents all ingress traffic to pods in a namespace:</p><pre>apiVersion: networking.k8s.io/v1<br>kind: NetworkPolicy<br>metadata:<br>  name: default-deny-ingress<br>  namespace: default<br>spec:<br>  podSelector: {}<br>  policyTypes:<br>  - Ingress</pre><h4>Default Deny All Egress Traffic</h4><p>This policy prevents all egress traffic from pods in a namespace:</p><pre>apiVersion: networking.k8s.io/v1<br>kind: NetworkPolicy<br>metadata:<br>  name: default-deny-egress<br>  namespace: default<br>spec:<br>  podSelector: {}<br>  policyTypes:<br>  - Egress</pre><h4>Allow Traffic from Specific Namespace</h4><p>This policy allows traffic from pods in a specific namespace:</p><pre>apiVersion: networking.k8s.io/v1<br>kind: NetworkPolicy<br>metadata:<br>  name: allow-from-monitoring<br>  namespace: default<br>spec:<br>  podSelector: {}<br>  policyTypes:<br>  - Ingress<br>  ingress:<br>  - from:<br>    - namespaceSelector:<br>        matchLabels:<br>          purpose: monitoring</pre><h4>Allow External Traffic to Specific Pods</h4><p>This policy allows external traffic to pods with specific labels:</p><pre>apiVersion: networking.k8s.io/v1<br>kind: NetworkPolicy<br>metadata:<br>  name: allow-external-to-web<br>  namespace: default<br>spec:<br>  podSelector:<br>    matchLabels:<br>      app: web<br>  policyTypes:<br>  - Ingress<br>  ingress:<br>  - from:<br>    - ipBlock:<br>        cidr: 203.0.113.0/24<br>    ports:<br>    - protocol: TCP<br>      port: 80</pre><h3>Three-Tier Web Application Example</h3><p>Consider a typical three-tier application with frontend, backend, and database layers. Here’s how to secure it with Network Policies:</p><p>Network Policy for frontend-tier to allow traffic from external network -</p><pre># Allow internet traffic to frontend<br>apiVersion: networking.k8s.io/v1<br>kind: NetworkPolicy<br>metadata:<br>  name: frontend-policy<br>  namespace: app<br>spec:<br>  podSelector:<br>    matchLabels:<br>      tier: frontend<br>  policyTypes:<br>  - Ingress<br>  - Egress<br>  ingress:<br>  - ports:<br>    - protocol: TCP<br>      port: 80<br>  egress:<br>  - to:<br>    - podSelector:<br>        matchLabels:<br>          tier: backend<br>    ports:<br>    - protocol: TCP<br>      port: 8080</pre><p>Network Policy for backend-tier to allow ingress traffic from frontend-tier and egress traffic to backend-tier -</p><pre># Allow frontend to backend communication<br>apiVersion: networking.k8s.io/v1<br>kind: NetworkPolicy<br>metadata:<br>  name: backend-policy<br>  namespace: app<br>spec:<br>  podSelector:<br>    matchLabels:<br>      tier: backend<br>  policyTypes:<br>  - Ingress<br>  - Egress<br>  ingress:<br>  - from:<br>    - podSelector:<br>        matchLabels:<br>          tier: frontend<br>    ports:<br>    - protocol: TCP<br>      port: 8080<br>  egress:<br>  - to:<br>    - podSelector:<br>        matchLabels:<br>          tier: database<br>    ports:<br>    - protocol: TCP<br>      port: 5432</pre><p>Network Policy for database-tier to allow traffic from backend-tier-</p><pre># Allow backend to database communication<br>apiVersion: networking.k8s.io/v1<br>kind: NetworkPolicy<br>metadata:<br>  name: database-policy<br>  namespace: app<br>spec:<br>  podSelector:<br>    matchLabels:<br>      tier: database<br>  policyTypes:<br>  - Ingress<br>  ingress:<br>  - from:<br>    - podSelector:<br>        matchLabels:<br>          tier: backend<br>    ports:<br>    - protocol: TCP<br>      port: 5432</pre><h3>Conclusion</h3><p>Kubernetes Network Policies provide a powerful mechanism for controlling pod-to-pod communication within your clusters. By implementing fine-grained network controls, you can significantly improve your security posture, meet compliance requirements, and enforce the principle of least privilege.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3f3c345b7384" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Inside the Cluster: Ingress in Focus — How External Traffic Reaches Your Services]]></title>
            <link>https://medium.com/@kelvi030117/inside-the-cluster-ingress-in-focus-how-external-traffic-reaches-your-services-95006bd8cdb4?source=rss-6a8f83281443------2</link>
            <guid isPermaLink="false">https://medium.com/p/95006bd8cdb4</guid>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[ingresscontroller]]></category>
            <category><![CDATA[kuberenetes]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[KELVI MANAVADARIA]]></dc:creator>
            <pubDate>Wed, 23 Apr 2025 06:15:10 GMT</pubDate>
            <atom:updated>2025-04-23T06:15:10.502Z</atom:updated>
            <content:encoded><![CDATA[<h3>Inside the Cluster: Ingress in Focus — How External Traffic Reaches Your Services</h3><p>In the dynamic landscape of Kubernetes, one of the most critical aspects is managing how external traffic enters your cluster and reaches the appropriate services. At the heart of this process is Kubernetes Ingress — an elegant solution that acts as the gateway between the outside world and your containerized applications. Let’s dive deep into how Ingress works, why it’s essential, and how traffic flows from a user’s browser all the way to your application pods.</p><h3>The Challenge: Getting Traffic Into Your Cluster</h3><p>Before we explore Ingress, let’s understand the fundamental challenge it solves. In a Kubernetes environment, pods are ephemeral — they come and go, change IP addresses, and scale up or down based on demand. This dynamic nature creates a significant challenge: how do you reliably route external traffic to these constantly changing pods?</p><p>The naive approach would be to use Kubernetes Service resources with type LoadBalancer for every application that needs external access. However, this introduces several problems:</p><ol><li><strong>Cost inefficiency</strong>: Each LoadBalancer typically provisions an actual cloud load balancer, which adds to your cloud bill.</li><li><strong>Limited routing capabilities</strong>: Basic load balancers don’t support advanced HTTP routing based on paths or hostnames.</li><li><strong>Certificate management complexity</strong>: SSL/TLS certificates would need to be managed separately for each service.</li><li><strong>No unified control point</strong>: Security policies, rate limiting, and authentication would be scattered across different resources.</li></ol><p>Enter Ingress — Kubernetes’ solution to these challenges.</p><h3>Ingress: The Traffic Orchestrator</h3><p>Kubernetes Ingress is not just another resource; it’s an architectural pattern that centralizes and standardizes how external HTTP/HTTPS traffic enters your cluster. It consists of two key components:</p><ol><li><strong>Ingress Resources</strong>: Kubernetes API objects that define routing rules.</li><li><strong>Ingress Controllers</strong>: The actual implementation that fulfills these rules.</li></ol><p>This separation of concerns is powerful — it allows you to define what you want (the rules) separately from how it’s implemented (the controller).</p><h3>The Anatomy of an Ingress Resource</h3><p>An Ingress resource is essentially a collection of rules that tell Kubernetes how to route external HTTP/HTTPS traffic to internal services. Here’s a simple example of nginx ingress:</p><pre>apiVersion: networking.k8s.io/v1<br>kind: Ingress<br>metadata:<br>  name: example-ingress<br>  annotations:<br>    nginx.ingress.kubernetes.io/rewrite-target: /<br>spec:<br>  ingressClassName: nginx<br>  rules:<br>  - host: myapp.example.com<br>    http:<br>      paths:<br>      - path: /api<br>        pathType: Prefix<br>        backend:<br>          service:<br>            name: api-service<br>            port:<br>              number: 80<br>      - path: /<br>        pathType: Prefix<br>        backend:<br>          service:<br>            name: frontend-service<br>            port:<br>              number: 80<br>  tls:<br>  - hosts:<br>    - myapp.example.com<br>    secretName: myapp-tls-secret</pre><p>This resource tells Kubernetes:</p><ul><li>Route traffic for myapp.example.com/api/* to the api-service</li><li>Route all other traffic for myapp.example.com to the frontend-service</li><li>Use the TLS certificate stored in the myapp-tls-secret for HTTPS</li></ul><p>The beauty of this approach is that all routing logic is centralized in one resource, making it easy to understand and manage.</p><h3>Ingress Controllers: Where the Magic Happens</h3><p>While the Ingress resource defines rules, it’s the Ingress Controller that actually implements them. An Ingress Controller is essentially a specialized application running inside your cluster that:</p><ol><li>Watches for Ingress resources</li><li>Configures itself based on those resources</li><li>Routes traffic accordingly</li></ol><p>Popular Ingress Controllers include:</p><ul><li><strong>NGINX Ingress Controller</strong>: Based on the powerful NGINX web server</li><li><strong>Traefik</strong>: A modern HTTP reverse proxy with automatic HTTPS</li><li><strong>HAProxy Ingress</strong>: Leveraging the high-performance HAProxy load balancer</li><li><strong>AWS ALB Ingress Controller</strong>: Specifically designed for AWS environments</li><li><strong>Istio Ingress Gateway</strong>: Part of the Istio service mesh ecosystem</li></ul><p>Each controller has its own strengths, ecosystem, and configuration options, but they all serve the same fundamental purpose: implementing your Ingress rules.</p><h3>The Traffic Journey: From Browser to Pod</h3><p>Now, let’s follow a request from a user’s browser all the way to your application pod:</p><ol><li><strong>DNS Resolution</strong>: The user types myapp.example.com in their browser, which resolves to the IP address of your Ingress Controller&#39;s load balancer.</li><li><strong>Load Balancer</strong>: The cloud provider’s load balancer (created when you deployed the Ingress Controller) receives the request and forwards it to one of the Ingress Controller pods.</li><li><strong>Ingress Controller</strong>: The controller pod:</li></ol><ul><li>Terminates TLS (if configured)</li><li>Examines the HTTP request to determine the host and path</li><li>Matches the request against Ingress rules</li><li>Selects the appropriate backend service</li></ul><p><strong>4. Service</strong>: The Kubernetes Service acts as an internal load balancer, selecting one of the available pods based on its selection algorithm (typically round-robin).</p><p><strong>5.Pod</strong>: Finally, the request reaches an application pod, which processes it and generates a response.</p><p><strong>6. Return Journey</strong>: The response follows the same path in reverse: Pod → Service → Ingress Controller → Load Balancer → User’s Browser.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*hI2D-NhW3SxzZatHkdByhw.png" /></figure><p>This entire journey happens in milliseconds, and the user experiences it as a seamless interaction with your application.</p><h3>IngressClass: The Traffic Director</h3><p>As Kubernetes clusters grow in complexity, organizations often need multiple Ingress Controllers for different purposes. This is where IngressClass comes into play.</p><p>IngressClass solves the “which controller should handle this Ingress?” problem by explicitly binding Ingress resources to specific controllers. Here’s how it works:</p><ol><li><strong>Define IngressClass resources</strong> for each controller type:</li></ol><pre>apiVersion: networking.k8s.io/v1<br>kind: IngressClass<br>metadata:<br>  name: external-lb<br>spec:<br>  controller: example.com/ingress-controller-external</pre><ol><li><strong>Reference the class in your Ingress resources</strong>:</li></ol><pre>spec:<br>  ingressClassName: external-lb<br>  # other specifications...</pre><p>This explicit binding ensures that:</p><ul><li>Each Ingress resource is handled by the appropriate controller</li><li>Multiple teams can use different controllers without conflicts</li><li>Specialized controllers can be used for specific purposes (e.g., external vs. internal traffic)</li></ul><h3>Conclusion</h3><p>Kubernetes Ingress transforms how we manage external access to services in a cluster. By centralizing routing logic, simplifying certificate management, and enabling advanced traffic patterns, Ingress has become an indispensable part of Kubernetes architecture.</p><blockquote><strong>Note: </strong>Refer to my another blog if you want to implement AWS ALB Ingress Controller and Resource — <a href="https://medium.com/@kelvi030117/kubernetes-ingress-with-aws-alb-ingress-controller-a-complete-setup-guide-bdc5c7c9961b">Kubernetes Ingress with AWS ALB Ingress Controller: A Complete Setup Guide</a></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=95006bd8cdb4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Inside the Cluster: Deep Dive into Kubernetes DNS and Service Discovery]]></title>
            <link>https://medium.com/@kelvi030117/inside-the-cluster-deep-dive-into-kubernetes-dns-and-service-discovery-8a21e3bc4699?source=rss-6a8f83281443------2</link>
            <guid isPermaLink="false">https://medium.com/p/8a21e3bc4699</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[coredns]]></category>
            <category><![CDATA[cloud]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[KELVI MANAVADARIA]]></dc:creator>
            <pubDate>Tue, 22 Apr 2025 18:55:30 GMT</pubDate>
            <atom:updated>2025-04-22T18:55:30.606Z</atom:updated>
            <content:encoded><![CDATA[<p>In a Kubernetes cluster, applications need to communicate with each other, but pod IP addresses are ephemeral — they change when pods are rescheduled, scaled, or updated. This creates a fundamental challenge: how can services reliably discover and connect to each other?</p><p>DNS (Domain Name System) solves this problem by providing stable naming and service discovery. Instead of hardcoding IP addresses that will change, applications can use consistent service names that automatically resolve to the correct destinations.</p><p>Without DNS in Kubernetes:</p><ul><li>Applications would need to track changing IP addresses manually</li><li>Service discovery would require custom implementation for each application</li><li>Configuration would be brittle and require frequent updates</li><li>Scaled deployments would be nearly impossible to manage</li></ul><h3>Evolution from kube-dns to CoreDNS</h3><h4>kube-dns Architecture</h4><p>The original Kubernetes DNS solution (kube-dns) consisted of:</p><ul><li><strong>SkyDNS</strong>: For DNS resolution</li><li><strong>dnsmasq</strong>: For caching and stub domains</li><li><strong>Sidecar containers</strong>: For health checking and metrics</li></ul><p>Limitations included:</p><ul><li>Security vulnerabilities in dnsmasq</li><li>Complex multi-container deployment</li><li>Limited customization options</li><li>Performance issues at scale</li></ul><h4>CoreDNS Introduction</h4><p>CoreDNS became the default DNS provider in Kubernetes v1.13 (December 2018), offering:</p><ul><li>Single Golang binary (more secure, less resource-intensive)</li><li>Plugin-based architecture for extensibility</li><li>Improved performance and caching</li><li>Better compatibility with DNS standards</li><li>Simplified deployment and maintenance</li></ul><h4>Migration Path</h4><p>The migration from kube-dns to CoreDNS was designed to be seamless:</p><ul><li>CoreDNS uses the same Service IP as kube-dns (for backward compatibility)</li><li>The Service name remains kube-dns in the kube-system namespace</li><li>DNS queries and responses maintain the same format</li><li>Existing applications don’t need changes to work with CoreDNS</li></ul><h3>How CoreDNS Works in Kubernetes</h3><h4>Deployment Architecture</h4><p>CoreDNS runs as:</p><ul><li>A Deployment in the kube-system namespace</li><li>Multiple replicas for high availability</li><li>Exposed via a ClusterIP Service named kube-dns</li><li>Typically allocated a static cluster IP</li></ul><pre>kubectl get deployment coredns -n kube-system<br>kubectl get service kube-dns -n kube-system</pre><h4>Core Components</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sUC4yoDZBUyQQhDy4QUf5Q.jpeg" /></figure><ol><li><strong>Pods</strong>: CoreDNS runs as a set of pods managed by a Deployment</li><li><strong>ConfigMap</strong>: Contains the Corefile configuration</li><li><strong>Service</strong>: The kube-dns service that exposes CoreDNS to the cluster</li><li><strong>ServiceAccount</strong>: Provides necessary permissions to access the Kubernetes API</li></ol><h4>The Corefile: CoreDNS Configuration</h4><p>The Corefile defines how CoreDNS processes DNS queries using a plugin chain:</p><pre>.:53 {<br>    errors<br>    health {<br>       lameduck 5s<br>    }<br>    ready<br>    kubernetes cluster.local in-addr.arpa ip6.arpa {<br>        pods insecure<br>        fallthrough in-addr.arpa ip6.arpa<br>        ttl 30<br>    }<br>    prometheus :9153<br>    forward . /etc/resolv.conf<br>    cache 30<br>    loop<br>    reload<br>    loadbalance<br>}</pre><p>This configuration is stored in the coredns ConfigMap in the kube-system namespace:</p><pre>kubectl get configmap coredns -n kube-system -o yaml</pre><h4>Plugin Chain Explained</h4><p>When a DNS query reaches CoreDNS, it flows through plugins in order:</p><ol><li><strong>errors</strong>: Logs errors to stdout</li><li><strong>health</strong>: Provides a health check endpoint (:8080/health)</li><li><strong>ready</strong>: Provides a readiness check endpoint (:8181/ready)</li><li><strong>kubernetes</strong>: Responds to DNS queries based on Kubernetes service/pod records</li><li><strong>prometheus</strong>: Exposes metrics in Prometheus format (:9153/metrics)</li><li><strong>forward</strong>: Forwards queries that can’t be answered locally to upstream DNS</li><li><strong>cache</strong>: Caches responses for performance</li><li><strong>loop</strong>: Detects forwarding loops</li><li><strong>reload</strong>: Allows automatic reloading of the Corefile when it changes</li><li><strong>loadbalance</strong>: Implements round-robin DNS for multiple endpoints</li></ol><h3>Query Flow Diagram</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*C6DM3dXgiMW3n3xCko6pqg.jpeg" /></figure><h3>DNS Records in Kubernetes</h3><h4>Service DNS Records</h4><p><strong>1. Normal Services (ClusterIP)</strong></p><p><strong>Format</strong>: &lt;service-name&gt;.&lt;namespace&gt;.svc.cluster.local<br> <strong>Example</strong>: web-service.default.svc.cluster.local</p><p>When resolved, returns the ClusterIP of the service.</p><p><strong>2. Headless Services (ClusterIP: None)</strong></p><p><strong>Format</strong>: &lt;service-name&gt;.&lt;namespace&gt;.svc.cluster.local<br> <strong>Example</strong>: mongodb.default.svc.cluster.local</p><p>When resolved, returns the IPs of all pods backing the service (not a single ClusterIP).</p><h4>3. StatefulSet Pod DNS</h4><p><strong>Format</strong>: &lt;pod-name&gt;.&lt;service-name&gt;.&lt;namespace&gt;.svc.cluster.local<br> <strong>Example</strong>: mongodb-0.mongodb.default.svc.cluster.local</p><p>StatefulSets provide stable network identities for each pod via DNS.</p><h4>Pod DNS Records</h4><p><strong>1. Standard Pod DNS</strong></p><p><strong>Format</strong>: &lt;ip-with-dashes&gt;.&lt;namespace&gt;.pod.cluster.local<br> <strong>Example</strong>: 10-244-1-25.default.pod.cluster.local</p><p>Where 10.244.1.25 is the pod IP with dots replaced by dashes.</p><p><strong>2. Pods with Hostname and Subdomain</strong></p><p><strong>Format</strong>: &lt;hostname&gt;.&lt;subdomain&gt;.&lt;namespace&gt;.svc.cluster.local</p><p>Set via pod spec:</p><pre>apiVersion: v1<br>kind: Pod<br>metadata:<br>  name: busybox<br>spec:<br>  hostname: busybox-1<br>  subdomain: custom-subdomain<br>  containers:<br>  - name: busybox<br>    image: busybox</pre><p>This creates: busybox-1.custom-subdomain.default.svc.cluster.local</p><h3>DNS Record Types</h3><p>CoreDNS creates different record types:</p><ol><li><strong>A records</strong>: For IPv4 addresses</li><li><strong>AAAA records</strong>: For IPv6 addresses</li><li><strong>SRV records</strong>: For named ports</li></ol><ul><li>Format: _&lt;port-name&gt;._&lt;port-protocol&gt;.&lt;service&gt;.&lt;namespace&gt;.svc.cluster.local</li><li>Example: _http._tcp.web-service.default.svc.cluster.local</li></ul><h3>DNS Resolution Shortcuts</h3><p>Kubernetes supports DNS resolution shortcuts for convenience:</p><ol><li><strong>Same namespace</strong>: Just use &lt;service-name&gt;</li><li><strong>Different namespace</strong>: Use &lt;service-name&gt;.&lt;namespace&gt;</li><li><strong>Any context</strong>: Use the full &lt;service-name&gt;.&lt;namespace&gt;.svc.cluster.local</li></ol><h3>DNS Policies in Kubernetes</h3><p>Kubernetes supports four DNS policies for pods:</p><h4>1. ClusterFirst (Default)</h4><p>Queries matching cluster domain are sent to CoreDNS, others to upstream DNS.</p><pre>spec:<br>  dnsPolicy: &quot;ClusterFirst&quot;</pre><h4>2. Default</h4><p>Uses the node’s DNS resolution configuration from /etc/resolv.conf.</p><pre>spec:<br>  dnsPolicy: &quot;Default&quot;</pre><h4>3. ClusterFirstWithHostNet</h4><p>For pods using host network namespace that still need cluster DNS.</p><pre>spec:<br>  hostNetwork: true<br>  dnsPolicy: &quot;ClusterFirstWithHostNet&quot;</pre><h4>4. None</h4><p>Allows completely custom DNS with the dnsConfig field.</p><pre>spec:<br>  dnsPolicy: &quot;None&quot;<br>  dnsConfig:<br>    nameservers:<br>      - 1.1.1.1<br>      - 8.8.8.8<br>    searches:<br>      - ns1.svc.cluster.local<br>      - my.dns.search.suffix<br>    options:<br>      - name: ndots<br>        value: &quot;2&quot;<br>      - name: edns0</pre><h3>Conclusion</h3><p>Kubernetes DNS and service discovery, powered by CoreDNS, provide the critical foundation for service-to-service communication in the cluster.</p><blockquote>P. S. — Again I have used AI to create images, still lazy !!🙃</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8a21e3bc4699" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Inside the Cluster: How Kubernetes Assigns IPs to Pods and Services]]></title>
            <link>https://medium.com/@kelvi030117/inside-the-cluster-how-kubernetes-assigns-ips-to-pods-and-services-48c24354188c?source=rss-6a8f83281443------2</link>
            <guid isPermaLink="false">https://medium.com/p/48c24354188c</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[networking]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[ip]]></category>
            <dc:creator><![CDATA[KELVI MANAVADARIA]]></dc:creator>
            <pubDate>Tue, 22 Apr 2025 16:08:06 GMT</pubDate>
            <atom:updated>2025-04-22T16:08:06.110Z</atom:updated>
            <content:encoded><![CDATA[<p>Networking in Kubernetes can seem like a black box when you’re getting started — IPs flying around, services just <em>magically</em> working, and pods talking across nodes without any visible wiring. Curious about what’s really happening under the hood, I decided to break it all down and understand how IP allocation works in Kubernetes for both pods and services.</p><p>This blog marks the beginning of a deep-dive series on Kubernetes networking — starting from the basics of IP assignments.</p><h3>Starting Point: How Are IPs Assigned to Pods and Nodes?</h3><h4><strong>Pod IP Allocation — The Early Days</strong></h4><p>When Kubernetes was first designed, it faced a fundamental challenge: how to give each container (pod) its own IP address and enable networking between them.</p><h4>The Simplest Case: Single-Node Cluster</h4><p>In a single-node setup without CNI:</p><ol><li>The node has its own IP (e.g., 192.168.1.10)</li><li>Each pod needs its own unique IP</li><li>The kubelet (Kubernetes agent) creates a virtual bridge network (called cbr0)</li><li>This bridge assigns IPs to pods from a subnet (e.g., 10.10.0.0/24)</li></ol><p><strong>Example:</strong></p><ul><li>Node: 192.168.1.10</li><li>Pod 1: 10.10.0.2</li><li>Pod 2: 10.10.0.3</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OJftPse7W6fuijpVrDs4EA.jpeg" /></figure><p>These pods can talk to each other through the bridge, but there’s a problem: how do pods communicate across different nodes?</p><h4>The Multi-Node Problem</h4><p>When you add more nodes:</p><ul><li>Node 1 has pod subnet 10.10.0.0/24</li><li>Node 2 has pod subnet 10.10.1.0/24</li></ul><p>Pod A on Node 1 (10.10.0.2) wants to talk to Pod B on Node 2 (10.10.1.3), but:</p><ol><li>The packet leaves Pod A</li><li>It hits the bridge on Node 1</li><li>Node 1 doesn’t know how to route to 10.10.1.3</li><li>The packet is dropped</li></ol><p>This is where the basic Kubernetes networking model breaks down.</p><h4>Enter Kubenet: The First Fix</h4><p>Kubernetes has a simple built-in network plugin called “kubenet” that tries to address this:</p><ol><li>Each node is assigned a pod CIDR (e.g., Node 1: 10.10.0.0/24, Node 2: 10.10.1.0/24)</li><li>Kubenet creates routes on each node</li><li>The Kubernetes controller manager updates these routes</li></ol><p><strong>Example:</strong></p><pre># On Node 1&#39;s routing table:<br>10.10.1.0/24 via 192.168.1.11  # Route to Node 2&#39;s pod subnet</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WW_OPOBnCBKyGDaTdQuPYA.jpeg" /></figure><p>This works, but with significant limitations:</p><ul><li>Requires manual route configuration in the underlying network</li><li>No support for network policies (security)</li><li>Limited scalability</li><li>No overlay network capabilities for cloud environments</li></ul><h4>The Real Game-Changer: CNI</h4><p>The <strong>Container Network Interface (CNI)</strong> came in to save the day. It standardized networking in Kubernetes by providing:</p><ul><li>A consistent API for networking plugins</li><li>Pluggable architecture (choose your networking tool)</li><li>Automatic IP management cluster-wide</li><li>Support for network policies and overlays</li></ul><h4>IP Allocation With CNI in Action</h4><p>When using a CNI plugin, the IP allocation process becomes more sophisticated:</p><ol><li>The CNI plugin is configured with a large CIDR range (e.g., 10.10.0.0/16)</li><li>When a new pod is created, Kubernetes asks the CNI plugin for an IP</li><li>The CNI plugin has an IPAM (IP Address Management) component that tracks all IPs</li><li>The CNI configures the network interfaces and routing for the pod</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AfvWw7ajxNT2D1ZUnWywmQ.jpeg" /></figure><p><strong>Example with Calico CNI:</strong></p><ol><li>You create a new pod on Node 2</li><li>Kubelet calls the Calico CNI plugin</li><li>Calico checks its IPAM database and allocates 10.10.5.23</li><li>Calico sets up the pod’s network interface with this IP</li><li>Calico updates its distributed routing information</li><li>Other nodes now know that 10.10.5.23 is reachable via Node 2</li></ol><h3>What About Services? The Service IP Story</h3><p>When you create a <strong>Kubernetes Service</strong>, it gets a virtual IP. Here’s how that works:</p><ol><li>During cluster setup, a CIDR range is reserved for services (e.g., 10.96.0.0/12)</li><li>The <strong>Service Controller</strong> (inside kube-controller-manager) tracks used and free IPs</li><li>When a new service is created:</li></ol><ul><li>The API server receives the request</li><li>The Service Controller picks the next free IP</li><li>It assigns the IP and stores it in etcd</li></ul><p>4. IPs are virtual — they don’t live on any real interface but are accessible cluster-wide</p><p>5. These IPs remain stable throughout the service’s lifetime</p><h4>Making It All Work: Kube-Proxy Magic</h4><p>The true magic happens with kube-proxy, which runs on every node and makes Service IPs actually work:</p><p><strong>What Kube-Proxy Does:</strong></p><p>Kube-proxy watches the API server for changes to Services and Endpoints and configures the local node to implement the Service abstraction:</p><ol><li><strong>Service Detection</strong>: Continuously monitors the Kubernetes API for Service and Endpoints changes</li><li><strong>IP Rules Creation</strong>: Sets up routing/redirection rules on each node</li><li><strong>Load Balancing</strong>: Distributes traffic among all pods that match the Service selector</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AI0CizQFGhK11b4RWe4pxA.jpeg" /></figure><h3>Wrapping Up</h3><ul><li><strong>Pod IPs</strong> → Managed via <strong>CNI plugins</strong> like Calico or Flannel</li><li><strong>Service IPs</strong> → Managed by <strong>kube-proxy</strong> using virtual routing and redirection</li></ul><blockquote>P.S. — I am too lazy to create this daigrams manually so I have used AI. 😅</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=48c24354188c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kubernetes Ingress with AWS ALB Ingress Controller: A Complete Setup Guide]]></title>
            <link>https://medium.com/@kelvi030117/kubernetes-ingress-with-aws-alb-ingress-controller-a-complete-setup-guide-bdc5c7c9961b?source=rss-6a8f83281443------2</link>
            <guid isPermaLink="false">https://medium.com/p/bdc5c7c9961b</guid>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[aws-eks]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[KELVI MANAVADARIA]]></dc:creator>
            <pubDate>Wed, 05 Feb 2025 20:29:27 GMT</pubDate>
            <atom:updated>2025-02-05T20:31:20.227Z</atom:updated>
            <content:encoded><![CDATA[<p><a href="https://medium.com/@kelvi030117/containerizing-a-node-js-79f6b36cb3dd">Missed Part 1? Read it here: Containerizing a Node.js Login and Register App with Docker</a></p><p><a href="https://medium.com/@kelvi030117/deploying-microservices-with-aws-eks-scalable-reliable-and-self-healing-e62412e8815d">Missed Part 2? Read it here: Deploying Microservices with AWS EKS — Scalable, Reliable, and Self-Healing</a></p><p>Congratulations! Your <strong>Node.js login and register microservices</strong> are now running on Kubernetes. But there’s one key piece missing — <strong>how do users access them from the outside world?</strong> Without a proper gateway, users wouldn’t be able to interact with your microservices.</p><p>This is where A<strong>WS Application Load Balancer (ALB) Ingress</strong> comes in. It acts as a traffic manager, routing external requests to the correct service within our Kubernetes cluster. With <strong>microservices</strong> exposed behind an ALB, users can access specific parts of the application (like the login or register page) without worrying about the underlying architecture.</p><p>In this final blog of our series, we’ll: <br>✅ Set up an <strong>Ingress Controller</strong> in EKS <br>✅ Configure <strong>AWS ALB Ingress</strong> to route external traffic to the correct microservice <br>✅ Make our <strong>login and register microservices</strong> publicly accessible</p><p>Let’s dive in !!!</p><h3>Final Architecture !!</h3><p>In this final architecture, we will configure the <strong>Ingress Controller</strong> in EKS and set up <strong>AWS ALB Ingress</strong> to route external HTTP(S) traffic to the appropriate microservices, such as login.kelvimanavadaria.site and register.kelvimanavadaria.site. The ALB will manage routing to Kubernetes services behind <strong>ClusterIP</strong>, ensuring secure and scalable access while efficiently handling traffic within the cluster. This completes the deployment of a robust and accessible microservices architecture on AWS EKS.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/831/1*Z8i43niMxN8v8y8oTa69lA.png" /></figure><h3><strong>1. Create an IAM OIDC Provider for Your Cluster:</strong></h3><p>First, set your cluster name:</p><pre>cluster_name=my-cluster</pre><p>Obtain the OIDC ID of your cluster:</p><pre>oidc_id=$(aws eks describe-cluster --name $cluster_name --query &quot;cluster.identity.oidc.issuer&quot; --output text | cut -d &#39;/&#39; -f 5) echo $oidc_id</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/568/1*1etG_hwR_v9SiY9r1SCOOw.png" /></figure><p>Associate an IAM OIDC identity provider with your cluster using the following command:</p><pre>eksctl utils associate-iam-oidc-provider --cluster $cluster_name --approve</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ki8YghWUg-e9PfUzcPNPOw.png" /></figure><h3>2. <strong>Create an IAM Role Using eksctl:</strong></h3><p>Download the IAM policy for the AWS Load Balancer Controller that allows it to interact with AWS APIs.</p><pre>curl -O https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.7.2/docs/install/iam_policy.json</pre><p>Create an IAM policy using the downloaded file:</p><pre>aws iam create-policy \<br>    --policy-name AWSLoadBalancerControllerIAMPolicy \<br>    --policy-document file://iam_policy.json</pre><p>Create a Service Account in the kube-system namespace with the IAM policy you just created:</p><pre>eksctl create iamserviceaccount   --cluster=new-cluster   --namespace=kube-system   --name=aws-load-balancer-controller   --role-name AmazonEKSLoadBalancerControllerRole   --attach-policy-arn=arn:aws:iam::933316127405:policy/AWSLoadBalancerControllerIAMPolicy --approve</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-U5F_Ji438Buq6KW6WzL4A.png" /></figure><h3>3. Install AWS Load Balancer Controller</h3><p>Install Helm</p><pre>curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3<br>chmod 700 get_helm.sh<br>./get_helm.sh</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qqByIhYAkwC8JQTuED6jeA.png" /></figure><p>Add the eks-chart Helm repository:</p><pre>helm repo add eks https://aws.github.io/eks-charts</pre><p>Update your local Helm repo to ensure you have the latest charts:</p><pre>helm repo update eks</pre><p>Install the AWS Load Balancer Controller using Helm:</p><pre>$ helm install aws-load-balancer-controller eks/aws-load-balancer-controller \<br>  -n kube-system \<br>  --set clusterName=kelvi-eks \<br>  --set serviceAccount.create=false \<br>  --set serviceAccount.name=aws-load-balancer-controller</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FVKfs78S7ByKzn6AgHr75g.png" /></figure><h3><strong>4. Create the Ingress Resource:</strong></h3><p>Create an Ingress resource to expose your microservices (login-service and register-service) through the AWS Application Load Balancer. The following YAML defines the Ingress configuration:</p><pre>apiVersion: networking.k8s.io/v1<br>kind: Ingress<br>metadata:<br>  name: login-register-ingress<br>  namespace: default<br>  annotations:<br>    alb.ingress.kubernetes.io/scheme: internet-facing<br>    alb.ingress.kubernetes.io/target-type: ip<br>    alb.ingress.kubernetes.io/listen-ports: &#39;[{&quot;HTTP&quot;: 80}]&#39;<br>spec:<br>  ingressClassName: alb<br>  rules:<br>  rules:<br>    - host: login.kelvimanavadaria.site<br>      http:<br>        paths:<br>          - path: &quot;/&quot;<br>            pathType: Prefix<br>            backend:<br>              service:<br>                name: login-service<br>                port:<br>                  number: 3000<br>    - host: register.kelvimanavadaria.site<br>      http:<br>        paths:<br>          - path: &quot;/&quot;<br>            pathType: Prefix<br>            backend:<br>              service:<br>                name: register-service<br>                port:<br>                  number: 3001</pre><p>Apply the Ingress configuration to your cluster:</p><pre>kubectl apply -f &lt;path-to-ingress-yaml-file&gt;</pre><h3>5. Verify the Setup by Pinging the URL:</h3><p>After applying the Ingress resource, you can verify the configuration by pinging the URLs login.kelvimanavadaria.site and register.kelvimanavadaria.site.</p><p>If everything is set up correctly, you should be able to access login and register page as below given image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LsQ6oU6xwYydilmET0ZSVQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9IzOIBJ6rDLGT5CFl39k5Q.png" /></figure><p>And just like that, with the AWS ALB Ingress in place, your Node.js login and register services are now ready to greet the world! <br>From local development to cloud deployment, your app is now scaled, accessible, and prepared to handle all the real-world traffic — like a seasoned pro at a rock concert!</p><p>Stay tuned, there’s more cloud magic coming your way! 🌟</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bdc5c7c9961b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploying Microservices with AWS EKS — Scalable, Reliable, and Self-Healing]]></title>
            <link>https://medium.com/@kelvi030117/deploying-microservices-with-aws-eks-scalable-reliable-and-self-healing-e62412e8815d?source=rss-6a8f83281443------2</link>
            <guid isPermaLink="false">https://medium.com/p/e62412e8815d</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[aws-eks]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[ingress]]></category>
            <dc:creator><![CDATA[KELVI MANAVADARIA]]></dc:creator>
            <pubDate>Wed, 05 Feb 2025 19:39:03 GMT</pubDate>
            <atom:updated>2025-02-05T19:39:03.483Z</atom:updated>
            <content:encoded><![CDATA[<h3>Deploying Microservices with AWS EKS — Scalable, Reliable, and Self-Healing</h3><p><a href="https://medium.com/@kelvi030117/containerizing-a-node-js-79f6b36cb3dd">Missed Part 1? Read it here: Containerizing a Node.js Login and Register App with Docker</a></p><p>Now that we’ve containerized our microservices, the next step is deploying them in a way that can handle growth, ensure reliability, and automatically recover from failures. With <strong>microservices architecture</strong>, each component is independent and can be scaled as needed, but managing this at scale requires robust infrastructure. This is where <strong>Amazon Elastic Kubernetes Service (EKS)</strong> comes into play.</p><p>EKS provides an environment that simplifies the orchestration of containers while offering powerful features for managing and scaling microservices.</p><p>In this blog, we’ll focus on:<br>✅ Deploying our <strong>microservices</strong> to EKS using <strong>Kubernetes Deployments</strong><br>✅ Exposing the microservices through <strong>Kubernetes Services</strong> to enable communication between them<br>✅ Ensuring each service is <strong>scalable</strong> and <strong>resilient</strong></p><p>By the end of this post, your microservices will be running on AWS EKS.</p><h3>Architecture</h3><p>This architecture deploys a <strong>MySQL database</strong>, <strong>Login service</strong>, and <strong>Register service</strong> on <strong>AWS EKS</strong> using Kubernetes. Each component runs as a <strong>Deployment</strong> with three replicas for scalability and fault tolerance. A <strong>ClusterIP service</strong> is used for internal communication, allowing the Login and Register services to connect to the MySQL database securely. This setup ensures efficient load distribution and high availability within the cluster.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/831/1*988NCvPOg9W5OozvMkyjQA.png" /></figure><p>Before proceeding, ensure you have an AWS EKS cluster deployed. Follow this blog for a step-by-step guide on creating an EKS cluster:<br><a href="https://medium.com/@vipul.chaudhary2609/curious-about-kubernetes-on-aws-lets-build-an-eks-cluster-40151626802a">Curious About Kubernetes on AWS? Let’s Build an EKS Cluster!</a></p><h3>1. Deploying Mysql Components</h3><p><strong>MySQL Deployment: </strong>Create a deployment with 3 replicas for high availability, configure the MySQL container with necessary environment variables.<br><strong>MySQL Service:</strong> Expose MySQL with a ClusterIP service.</p><pre>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: mysql-deployment<br>spec:<br>  replicas: 3<br>  selector:<br>    matchLabels:<br>      app: mysql<br>  template:<br>    metadata:<br>      labels:<br>        app: mysql<br>    spec:<br>      containers:<br>      - name: mysql-database<br>        image: mysql:5.7.5<br>        env:<br>        - name: MYSQL_ROOT_PASSWORD<br>          value: &quot;Admin@2003&quot;<br>        - name: MYSQL_DATABASE<br>          value: &quot;user_auth&quot;<br>        ports:<br>        - containerPort: 3306<br>---<br>apiVersion: v1<br>kind: Service<br>metadata:<br>  name: mysql-service<br>spec:<br>  selector:<br>    app: mysql<br>  ports:<br>    - protocol: TCP<br>      port: 3306<br>      targetPort: 3306<br>  type: ClusterIP</pre><p>Deploy the database components file using below given :</p><pre>kubectl create -f &lt;path-to-database-component-file&gt;</pre><h3>2. Deploying Login Components</h3><p><strong>Login App Deployment:</strong> Deploy the Login App with 3 replicas using the login image created in the previous blog. Configure the environment variables to connect to the database, where the DB_HOST should reference the MySQL service name. the MySQL service.<br><strong>Login App Service:</strong> Expose Login App with a ClusterIP service.</p><pre>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: login-deployment<br>spec:<br>  replicas: 3<br>  selector:<br>    matchLabels:<br>      app: login-app<br>  template:<br>    metadata:<br>      labels:<br>        app: login-app<br>    spec:<br>      containers:<br>      - name: login-app<br>        image: kelvi17/login-register:login<br>        ports:<br>        - containerPort: 3000<br>        env:<br>          - name: DB_HOST<br>            value: mysql-service.default.svc.cluster.local<br>          - name: DB_USER<br>            value: root<br>          - name: DB_PASSWORD<br>            value: Admin@2003<br>          - name: DB_NAME<br>            value: user_auth<br>---<br>apiVersion: v1<br>kind: Service<br>metadata:<br>  name: login-service<br>spec:<br>  selector:<br>    app: login-app<br>  ports:<br>    - protocol: TCP<br>      port: 3000<br>      targetPort: 3000<br>  type: ClusterIP</pre><p>Deploy the login components file using below given :</p><pre>kubectl create -f &lt;path-to-login-component-file&gt;</pre><h3>3. Deploying Register Components</h3><p><strong>Register App Deployment:</strong> Deploy the Login App with 3 replicas using the register image created in the previous blog. Configure the environment variables to connect to the database, where the DB_HOST should reference the MySQL service name. the MySQL service.<br><strong>Register App Service:</strong> Expose Register App with a ClusterIP service.</p><pre>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: register-deployment<br>spec:<br>  replicas: 3<br>  selector:<br>    matchLabels:<br>      app: register-app<br>  template:<br>    metadata:<br>      labels:<br>        app: register-app<br>    spec:<br>      containers:<br>      - name: register-app<br>        image: kelvi17/login-register:register<br>        ports:<br>        - containerPort: 3001<br>        env:<br>          - name: DB_HOST<br>            value: mysql-service.default.svc.cluster.local<br>          - name: DB_USER<br>            value: root<br>          - name: DB_PASSWORD<br>            value: Admin@2003<br>          - name: DB_NAME<br>            value: user_auth<br>---<br>apiVersion: v1<br>kind: Service<br>metadata:<br>  name: register-service<br>spec:<br>  selector:<br>    app: register-app<br>  ports:<br>    - protocol: TCP<br>      port: 3001<br>      targetPort: 3001<br>  type: ClusterIP</pre><p>Deploy the register components file using below given :</p><pre>kubectl create -f &lt;path-to-register-component-file&gt;</pre><p>To verify that all deployments and services are running successfully, execute the following command:-</p><pre>kubectl get all</pre><p>It should display the expected output.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xA48kWfb5RqhiYeK5M-kwQ.png" /></figure><p>Stay tuned for the next blog, where we’ll explore how to <strong>expose these services to external users using AWS ALB Ingress</strong>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e62412e8815d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Containerizing a Node.js]]></title>
            <link>https://medium.com/@kelvi030117/containerizing-a-node-js-79f6b36cb3dd?source=rss-6a8f83281443------2</link>
            <guid isPermaLink="false">https://medium.com/p/79f6b36cb3dd</guid>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[aws-eks]]></category>
            <category><![CDATA[microservices]]></category>
            <dc:creator><![CDATA[KELVI MANAVADARIA]]></dc:creator>
            <pubDate>Wed, 05 Feb 2025 17:05:53 GMT</pubDate>
            <atom:updated>2025-02-05T17:05:53.336Z</atom:updated>
            <content:encoded><![CDATA[<h3>Containerizing a Node.js Login and Register App with Docker — The First Step in Microservices Deployment</h3><p>Picture this — you’ve built a <strong>login and register system</strong> using Node.js. Everything works perfectly on your local machine, but now you face a new challenge: <strong>How do you scale it?</strong> What happens if you need to add more services, like user profile management, or a payment system in the future?</p><p>That’s where the <strong>microservices architecture</strong> comes in. Instead of having a single, monolithic application, you break your app into smaller, independent services that can be developed, deployed, and scaled separately. For instance, we’ll treat the login and register components as <strong>two microservices</strong> within the larger application ecosystem.</p><p><strong>Welcome to the first part of our series, where we containerize our Node.js microservices</strong> — login and register — and prepare them for deployment on Kubernetes. By the end of this blog, you’ll have the Docker images of your microservices ready to scale in the cloud.</p><p>In this post, we’ll focus on<br>✅ Writing Dockerfiles for each service and containerizing each <strong>microservice</strong> (login and register) using Docker.</p><p>In the next blog, we’ll take these Dockerized microservices and deploy them to <strong>AWS EKS</strong> for scalable management. Let’s dive into building our first microservice!</p><h3>Steps:</h3><h3>1. Create a Login App</h3><blockquote><strong>Note: </strong>Follow the below given all steps in a specific folder use <strong>mkdir Login</strong> to create folder and <strong>cd Login</strong> to use that folder</blockquote><h4>1. Create a index.html file</h4><p>a. Create a index.html file using <strong>vi index.html </strong>in /public folder<strong>.<br> </strong>b. Add the below given Code in that file.</p><pre>&lt;!DOCTYPE html&gt;<br>&lt;html lang=&quot;en&quot;&gt;<br>&lt;head&gt;<br>  &lt;meta charset=&quot;UTF-8&quot;&gt;<br>  &lt;meta name=&quot;viewport&quot; content=&quot;width=device-width, initial-scale=1.0&quot;&gt;<br>  &lt;title&gt;Login&lt;/title&gt;<br>  &lt;style&gt;<br>    body {<br>      font-family: Arial, sans-serif;<br>      background-color: #f4f4f9;<br>      display: flex;<br>      justify-content: center;<br>      align-items: center;<br>      height: 100vh;<br>      margin: 0;<br>    }<br><br>    .login-container {<br>      background-color: #fff;<br>      padding: 30px;<br>      border-radius: 10px;<br>      box-shadow: 0px 4px 10px rgba(0, 0, 0, 0.1);<br>      width: 400px;<br>    }<br><br>    .login-container h2 {<br>      text-align: center;<br>      margin-bottom: 20px;<br>      color: #333;<br>    }<br><br>    .form-group {<br>      margin-bottom: 15px;<br>    }<br><br>    label {<br>      display: block;<br>      font-size: 14px;<br>      color: #555;<br>      margin-bottom: 5px;<br>    }<br><br>    input[type=&quot;text&quot;],<br>    input[type=&quot;password&quot;] {<br>      width: 100%;<br>      padding: 10px;<br>      font-size: 16px;<br>      border-radius: 5px;<br>      border: 1px solid #ccc;<br>      box-sizing: border-box;<br>    }<br><br>    input[type=&quot;text&quot;]:focus,<br>    input[type=&quot;password&quot;]:focus {<br>      outline: none;<br>      border-color: #4a90e2;<br>    }<br><br>    button {<br>      width: 100%;<br>      padding: 10px;<br>      background-color: #4a90e2;<br>      color: white;<br>      border: none;<br>      border-radius: 5px;<br>      font-size: 16px;<br>      cursor: pointer;<br>    }<br><br>    button:hover {<br>      background-color: #357abd;<br>    }<br><br>    .form-group p {<br>      margin-top: 10px;<br>      font-size: 12px;<br>      color: #999;<br>    }<br>  &lt;/style&gt;<br>&lt;/head&gt;<br>&lt;body&gt;<br>  &lt;div class=&quot;login-container&quot;&gt;<br>    &lt;h2&gt;Login&lt;/h2&gt;<br>    &lt;form action=&quot;/login&quot; method=&quot;POST&quot;&gt;<br>      &lt;div class=&quot;form-group&quot;&gt;<br>        &lt;label for=&quot;username&quot;&gt;Username:&lt;/label&gt;<br>        &lt;input type=&quot;text&quot; id=&quot;username&quot; name=&quot;username&quot; required&gt;<br>      &lt;/div&gt;<br><br>      &lt;div class=&quot;form-group&quot;&gt;<br>        &lt;label for=&quot;password&quot;&gt;Password:&lt;/label&gt;<br>        &lt;input type=&quot;password&quot; id=&quot;password&quot; name=&quot;password&quot; required&gt;<br>      &lt;/div&gt;<br><br>      &lt;button type=&quot;submit&quot;&gt;Login&lt;/button&gt;<br><br>      &lt;div class=&quot;form-group&quot;&gt;<br>        &lt;p&gt;Don&#39;t have an account? &lt;a href=&quot;${REGISTER_URL}&quot;&gt;Sign up here&lt;/a&gt;.&lt;/p&gt;<br>      &lt;/div&gt;<br>    &lt;/form&gt;<br>  &lt;/div&gt;<br>&lt;/body&gt;<br>&lt;/html&gt;</pre><h4>2. Create an app.js file</h4><p>a. Create a app.js file using <strong>vi app.js.<br></strong>b. Add the below given code in the app.js file</p><pre>require(&#39;dotenv&#39;).config();<br>const express = require(&#39;express&#39;);<br>const path = require(&#39;path&#39;);<br>const bodyParser = require(&#39;body-parser&#39;);<br>const mysql = require(&#39;mysql&#39;);<br>const bcrypt = require(&#39;bcrypt&#39;);<br>const app = express();<br><br>// Middleware to serve static files<br>app.use(express.static(path.join(__dirname)));<br><br>// MySQL database connection using environment variables<br>const db = mysql.createConnection({<br>  host: process.env.DB_HOST,<br>  user: process.env.DB_USER,<br>  password: process.env.DB_PASSWORD,<br>  database: process.env.DB_NAME<br>});<br><br>db.connect((err) =&gt; {<br>  if (err) throw err;<br>  console.log(&#39;Connected to MySQL&#39;);<br>});<br><br>// Serve the static HTML file<br>app.get(&#39;/&#39;, (req, res) =&gt; {<br>  res.sendFile(path.join(__dirname, &#39;index.html&#39;));<br>});<br><br>// Handle login request<br>app.post(&#39;/login&#39;, (req, res) =&gt; {<br>  const { username, password } = req.body;<br><br>  db.query(&#39;SELECT * FROM users WHERE username = ?&#39;, [username], async (err, results) =&gt; {<br>    if (err) throw err;<br><br>    if (results.length === 0) {<br>      return res.send(&#39;User not found&#39;);<br>    }<br><br>    const user = results[0];<br><br>    const match = await bcrypt.compare(password, user.password);<br>    if (match) {<br>      res.send(&#39;Login successful!&#39;);<br>    } else {<br>      res.send(&#39;Incorrect password&#39;);<br>    }<br>  });<br>});<br><br>// Start the server<br>app.listen(3000, () =&gt; {<br>  console.log(&#39;Server running on http://localhost:3000&#39;);<br>});</pre><h4>3. Create Dockerfile for Login App</h4><p>a. Create a Dockerfile using <strong>vi Dockerfile<br></strong>b. Add the below given code in Dockerfile.</p><pre>FROM node:14<br>WORKDIR /usr/src/app<br>COPY package*.json ./<br>RUN npm install<br>COPY . .<br>EXPOSE 3000<br>CMD [ &quot;node&quot;, &quot;app.js&quot; ]</pre><h4>4. Build Image using Dockerfile</h4><p>a. Build the Docker image using the below given command.</p><pre>docker build -t login-app .</pre><p>b. Use docker image to see that the image is created.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/707/1*xZzoRwclPspt3tmigSorIQ.png" /></figure><h3>2. Create a Register App</h3><blockquote><strong>Note: </strong>Follow the below given all steps in a specific folder use <strong>mkdir Register </strong>to create folder and <strong>cd Register </strong>to use that folder</blockquote><h4>1. Create a index.html file</h4><p>a. Create a index.html file using <strong>vi index.html </strong>in /public folder<strong>.<br></strong>b. Add the below given Code in that file.</p><pre>&lt;!DOCTYPE html&gt;<br>&lt;html lang=&quot;en&quot;&gt;<br>&lt;head&gt;<br>  &lt;meta charset=&quot;UTF-8&quot;&gt;<br>  &lt;meta name=&quot;viewport&quot; content=&quot;width=device-width, initial-scale=1.0&quot;&gt;<br>  &lt;title&gt;Register&lt;/title&gt;<br>  &lt;style&gt;<br>    body {<br>      font-family: Arial, sans-serif;<br>      background-color: #f4f4f9;<br>      display: flex;<br>      justify-content: center;<br>      align-items: center;<br>      height: 100vh;<br>      margin: 0;<br>    }<br><br>    .register-container {<br>      background-color: #fff;<br>      padding: 30px;<br>      border-radius: 10px;<br>      box-shadow: 0px 4px 10px rgba(0, 0, 0, 0.1);<br>      width: 400px;<br>    }<br><br>    .register-container h2 {<br>      text-align: center;<br>      margin-bottom: 20px;<br>      color: #333;<br>    }<br><br>    .form-group {<br>      margin-bottom: 15px;<br>    }<br><br>    label {<br>      display: block;<br>      font-size: 14px;<br>      color: #555;<br>      margin-bottom: 5px;<br>    }<br><br>    input[type=&quot;text&quot;],<br>    input[type=&quot;password&quot;],<br>    input[type=&quot;email&quot;] {<br>      width: 100%;<br>      padding: 10px;<br>      font-size: 16px;<br>      border-radius: 5px;<br>      border: 1px solid #ccc;<br>      box-sizing: border-box;<br>    }<br><br>    input[type=&quot;text&quot;]:focus,<br>    input[type=&quot;password&quot;]:focus,<br>    input[type=&quot;email&quot;]:focus {<br>      outline: none;<br>      border-color: #4a90e2;<br>    }<br><br>    button {<br>      width: 100%;<br>      padding: 10px;<br>      background-color: #4a90e2;<br>      color: white;<br>      border: none;<br>      border-radius: 5px;<br>      font-size: 16px;<br>      cursor: pointer;<br>    }<br><br>    button:hover {<br>      background-color: #357abd;<br>    }<br><br>    .form-group p {<br>      margin-top: 10px;<br>      font-size: 12px;<br>      color: #999;<br>    }<br><br>    .form-group a {<br>      color: #4a90e2;<br>      text-decoration: none;<br>    }<br><br>    .form-group a:hover {<br>      text-decoration: underline;<br>    }<br>  &lt;/style&gt;<br>&lt;/head&gt;<br>&lt;body&gt;<br>  &lt;div class=&quot;register-container&quot;&gt;<br>    &lt;h2&gt;Register&lt;/h2&gt;<br>    &lt;form action=&quot;/register&quot; method=&quot;POST&quot;&gt;<br>      &lt;div class=&quot;form-group&quot;&gt;<br>        &lt;label for=&quot;username&quot;&gt;Username:&lt;/label&gt;<br>        &lt;input type=&quot;text&quot; id=&quot;username&quot; name=&quot;username&quot; required&gt;<br>      &lt;/div&gt;<br><br>      &lt;div class=&quot;form-group&quot;&gt;<br>        &lt;label for=&quot;email&quot;&gt;Email:&lt;/label&gt;<br>        &lt;input type=&quot;email&quot; id=&quot;email&quot; name=&quot;email&quot; required&gt;<br>      &lt;/div&gt;<br><br>      &lt;div class=&quot;form-group&quot;&gt;<br>        &lt;label for=&quot;password&quot;&gt;Password:&lt;/label&gt;<br>        &lt;input type=&quot;password&quot; id=&quot;password&quot; name=&quot;password&quot; required&gt;<br>      &lt;/div&gt;<br><br>      &lt;button type=&quot;submit&quot;&gt;Register&lt;/button&gt;<br><br>      &lt;div class=&quot;form-group&quot;&gt;<br>        &lt;p&gt;Already have an account? &lt;a href=&quot;${LOGIN_URL}&quot;&gt;Login here&lt;/a&gt;.&lt;/p&gt;<br>      &lt;/div&gt;<br>    &lt;/form&gt;<br>  &lt;/div&gt;<br>&lt;/body&gt;<br>&lt;/html&gt;</pre><h4>2. Create an app.js file</h4><p>a. Create a app.js file using <strong>vi app.js.<br></strong>b. Add the below given code in the app.js file</p><pre>require(&#39;dotenv&#39;).config();<br>const express = require(&#39;express&#39;);<br>const path = require(&#39;path&#39;);<br>const bodyParser = require(&#39;body-parser&#39;);<br>const mysql = require(&#39;mysql&#39;);<br>const bcrypt = require(&#39;bcrypt&#39;);<br>const app = express();<br><br>// Middleware to serve static files<br>app.use(express.static(path.join(__dirname)));<br><br>// Middleware to parse JSON bodies<br>app.use(bodyParser.urlencoded({ extended: true }));<br><br>// MySQL database connection using environment variables<br>const db = mysql.createConnection({<br>  host: process.env.DB_HOST,<br>  user: process.env.DB_USER,<br>  password: process.env.DB_PASSWORD,<br>  database: process.env.DB_NAME<br>});<br><br>db.connect((err) =&gt; {<br>  if (err) throw err;<br>  console.log(&#39;Connected to MySQL&#39;);<br>});<br><br>// Serve the static HTML file<br>app.get(&#39;/&#39;, (req, res) =&gt; {<br>  res.sendFile(path.join(__dirname,  &#39;index.html&#39;));<br>});<br><br>// Handle registration request<br>app.post(&#39;/register&#39;, async (req, res) =&gt; {<br>  const { username, email, password } = req.body;<br><br>  // Check if user already exists<br>  db.query(&#39;SELECT * FROM users WHERE username = ?&#39;, [username], async (err, results) =&gt; {<br>    if (err) throw err;<br><br>    if (results.length &gt; 0) {<br>      return res.send(&#39;Username already taken&#39;);<br>    }<br><br>    // Hash the password<br>    const hashedPassword = await bcrypt.hash(password, 10);<br><br>    // Insert the new user into the database<br>    db.query(&#39;INSERT INTO users (username, email, password) VALUES (?, ?, ?)&#39;, [username, email, hashedPassword], (err, result) =&gt; {<br>      if (err) throw err;<br>      res.send(&#39;Registration successful!&#39;);<br>    });<br>  });<br>});<br><br><br>// Start the server<br>app.listen(3001, () =&gt; {<br>  console.log(&#39;Server running on http://localhost:3001&#39;);<br>});</pre><h4>3. Create Dockerfile for Login App</h4><p>a. Create a Dockerfile using <strong>vi Dockerfile<br></strong>b. Add the below given code in Dockerfile.</p><pre>FROM node:18<br>WORKDIR /usr/src/app<br>COPY package*.json ./<br>RUN npm install<br>COPY . .<br>EXPOSE 3001<br>CMD [&quot;node&quot;, &quot;app.js&quot;]</pre><h4>4. Build Image using Dockerfile</h4><p>a. Build the Docker image using the below given command.</p><pre>docker build -t register-app .</pre><p>b. Use docker image to see that the image is created.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/830/1*KwTksbBpsDylA115LkCNpw.png" /></figure><h3>3. Push the images to the Docker Hub</h3><p>Write the below given commands to tag the Docker.</p><pre>docker tag login-app {username}/{repo-name}:login-app<br>docker tag register-app {username}/{repo-name}:register-app</pre><p>Push the tagged images to your Docker Hub repository.</p><pre>docker push {username}/{repo-name}:login-app <br>docker push {username}/{repo-name}:register-app</pre><p>Stay tuned for the next part of this series where we will deploy this microservices on <strong>AWS EKS </strong>!!!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=79f6b36cb3dd" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Elevating Security Standards: Zero Trust Network Access in AWS]]></title>
            <link>https://medium.com/@kelvi030117/elevating-security-standards-zero-trust-network-access-in-aws-f1ccfaf0791e?source=rss-6a8f83281443------2</link>
            <guid isPermaLink="false">https://medium.com/p/f1ccfaf0791e</guid>
            <dc:creator><![CDATA[KELVI MANAVADARIA]]></dc:creator>
            <pubDate>Thu, 23 Jan 2025 16:41:00 GMT</pubDate>
            <atom:updated>2025-01-23T16:41:00.517Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>TABLE OF CONTENT: -</strong></p><ol><li>Introduction</li><li>Perspective on Zero Trust</li><li>The Triad of Zero Trust</li><li>Human-to-application &amp; Server-to-Server Dynamics</li><li>Zero Trust Network Access</li><li>AWS Services for Zero Trust Network Access</li><li>Implementing ZTNA Architecture with AWS</li><li>Supplementary Security Measures</li><li>Conclusion</li></ol><h4><strong>1. Introduction</strong></h4><p>This blog is all about Zero Trust Network Access (ZTNA) in AWS, part of the broader Zero Trust Model. We’ll cover everything from the basics of the Zero Trust Model to ZTNA and which AWS services to use for it. The goal is to make your architecture secure using AWS services and implementing ZTNA for added protection.</p><h4><strong>2. Perspective on Zero Trust</strong></h4><p>Definition of Zero Trust varies from place to place but by AWS Zero trust is essentially a security model and associated set of mechanisms that focus on providing security controls around digital assets that don’t solely or fundamentally depend on traditional network controls or network perimeters.</p><p>Zero Trust is a security framework requiring all users, whether in or outside the organization’s network, to be authenticated, authorized, and continuously validated for security configuration and posture before being granted or keeping access to applications and data.</p><h4><strong>3. The Triad of Zero Trust</strong></h4><p>Zero Trust is majorly an integration of networking, identity and security.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AP7DCWOf1Bsw0s_0PIHQdA.png" /></figure><p>About these 3 sectors on which Zero Trust relies, two major sole sectors are network centric and identity centric viewpoint of architecture. Though all 3 sectors are dependent on each other, but the security cannot be defined as a separate perspective because security is something which is very crucial and important and needs to be proactively integrated at each and every stage of architecture building rather than discussed afterwards.</p><p>Network centric and identity centric comes under the security and in many places, it comes under the name of “Types of security controls”.</p><p>Let’s discuss network centric and identity centric views in architecture in detail with the help of an example of a simple AWS architecture. <br> <br>In this setup, a Virtual Private Cloud (VPC) hosts a Load Balancer distributing traffic to multiple EC2 instances managed by an Auto Scaling Group. Users access the application via the Load Balancer, which ensures even load distribution and fault tolerance. The Auto Scaling Group dynamically adjusts the number of EC2 instances based on demand, ensuring scalability and high availability.</p><ol><li><strong>NETWORK CENTRIC</strong></li></ol><p>In a network-centric view within Zero Trust, the focus is on securing the network and controlling access based on network-related parameters.</p><p>From the above example a network centric will be like securing the VPC, and applying firewalls, adding security groups, adding NACLs,etc.</p><p><strong>2. IDENTITY CENTRIC</strong></p><p>In an identity-centric view within Zero Trust, the emphasis shifts from network location to individual user authentication and authorization</p><p>Using the above example, the AWS IAM roles, User authentication, Role based access to the services all come under the identity centric.</p><blockquote><strong>IMP: </strong>— In a zero-trust architecture, networking and identity-based controls aren’t just simultaneously present and configured, they’re actually aware of one another.</blockquote><p>An illustrative example of this is an Amazon Virtual Private Cloud (VPC) endpoint. VPC endpoints provide private network connectivity to AWS services from your own virtual private cloud and allow you to specify access control policies. These policies and their associated enforcement engine understand not only the network, but also the identities and resources that are flowing across this border network control. They can make authorization decisions that consider this converged context.</p><h4><strong>4. Human-to-application &amp; Server-to-Server Dynamics</strong></h4><p>One can categorize any architectural design into two main components: human-to-application and server-to-server.</p><p><strong>4.1 Human-to-Application</strong> refers to the interaction between users and software applications. It encompasses the user interface, authentication, user experience, and presentation layer, focusing on how users interact with and consume applications.</p><p>From the above architecture example, the external user interacting with load balancer for accessing the website on the EC2 is a part of human-to-application architecture.</p><p><strong>4.2 Server-to-Server</strong> pertains to the communication and interactions between backend systems, servers, or services within an architecture.</p><p>From the above architecture example, the load balancer communicating with auto scaling group and EC2 instance is part of the server-to-server architecture.</p><p>From an external viewpoint, there exists an association between the dimensions of human-to-application and server-to-server, which can be correlated with the paradigms of network-centric and identity-centric, as demonstrated in the provided table:</p><p>User interactions (human-to-application) involve authentication and authorization, while server communications (server-to-server) entail server identities and verification for secure and trusted interaction within the architecture.</p><p>So further discussion is about the human-to-application part which is also called Zero Trust Network Access (ZTNA).</p><h4><strong>5. Zero Trust Network Access</strong></h4><p>Zero Trust Network Access (ZTNA) is a security service that verifies users and grants access to specific applications based on identity and context policies.</p><p>ZTNA removes implicit trust to restrict network movement and reduce attack surfaces.</p><h4><strong>6. AWS Services for Zero Trust Network Access</strong></h4><p>This is a list of services that we can use to create a secure ZTNA biased architecture.</p><p>6.1 AWS Cognito</p><p>6.2 AWS Verified Access</p><p>6.3 AWS Verified Permission</p><p>6.4 OCID Configurations (OpenID Connect Identity Provider)</p><p>6.5 AWS Identity and Access Management (IAM)</p><p>6.6 AWS Single Sign-On (SSO)</p><p>6.7 AWS Web Application Firewall (WAF)</p><p>6.8 AWS Firewall Manager</p><p>6.9 AWS Security Hub</p><p>6.10 AWS Network firewall</p><p>6.11 AWS Shield</p><p>6.12 AWS Route 53</p><p>6.13 AWS Certificate Manager</p><h4><strong>7. Implementing ZTNA Architecture with AWS</strong></h4><p>First let’s see the human-to-application side of architecture. For the human side I have created the below given architecture.</p><p>In the outlined architecture, user identities are stored either in AWS Cognito user pools or in other identity providers. To authenticate and authorize these users, the system leverages AWS Verified Access (AVA), which utilizes Cedar language policies for granting user access. However, prior to AVA, user authentication is necessary. AWS Cognito user pools handle the authentication for users stored within them, whereas a different identity provider uses OpenID Connect Identity Provider (OCID) for authentication, with the help of authentication endpoints such as the Auth Endpoint, Token Endpoint, and User-info Endpoint. Once users complete the authentication and authorization processes, they gain access to the application based on the specified policies.</p><p>Secondly, to secure applications or the services interacting with humans I have drafted the below given architecture.</p><p>So, after users are authenticated and before they access the application services, they need to pass through the layer of a security which contains different services like AWS Firewall Manager, AWS Network Firewall, AWS Shield, AWS Web Application Firewall, AWS Route53 and many other services too. There is no need to use all these services or only these services we can modify accordingly. We also have a security hub which is a centralized security service and automates security practices. Then the application services need to be secured by the Security Groups, IAM and other services available.</p><h4><strong>8. Supplementary Security Measures</strong></h4><p>There are some additional things that we can use to make our application more secure: -</p><ol><li>AWS IAM</li><li>AWS SSO</li><li>AWS trust provider integration</li><li>Mutual TLS</li><li>Lambda Authorizer</li><li>HTTPS</li><li>JWT authorizer</li></ol><h4><strong>9. Conclusion</strong></h4><p>The blog above introduces an architecture designed for implementing Zero Trust Network Access (ZTNA) in AWS. It reflects a personal viewpoint on using AWS services for ZTNA, allowing for modifications based on customer preferences and specific needs.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f1ccfaf0791e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Querying Unstructured JSON Logs with AWS Glue ETL Job and Athena: A Step-by-Step Guide]]></title>
            <link>https://medium.com/@kelvi030117/querying-unstructured-json-logs-with-aws-glue-etl-job-and-athena-a-step-by-step-guide-56fdc993ef98?source=rss-6a8f83281443------2</link>
            <guid isPermaLink="false">https://medium.com/p/56fdc993ef98</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[aws-community]]></category>
            <category><![CDATA[etl]]></category>
            <category><![CDATA[aws-glue]]></category>
            <category><![CDATA[aws-athena]]></category>
            <dc:creator><![CDATA[KELVI MANAVADARIA]]></dc:creator>
            <pubDate>Tue, 10 Sep 2024 17:50:22 GMT</pubDate>
            <atom:updated>2024-09-10T17:50:22.974Z</atom:updated>
            <content:encoded><![CDATA[<p>In the previous guide, we explored the ease of querying structured JSON logs with AWS Glue Crawlers and Athena. It was all neat and organized, like opening a book where every chapter was perfectly titled. But what happens when the story becomes a little chaotic? Unstructured logs enter the picture, with unpredictable fields, shifting formats, and missing values. These logs, often generated by observability tools like New Relic &amp; Datadog, present an entirely new challenge.<br> <br> You can’t simply throw these unstructured logs into a Glue Crawler and expect to get a clean table for querying. No — these logs need to be tamed, transformed, and molded into something manageable before you can extract value from them. Enter <strong>AWS Glue ETL</strong>, a powerful tool designed to handle the most unruly of data sets.<br> <br> Let’s dive deeper into the steps required to transform and query unstructured logs effectively.</p><h3><strong>Architecture:</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*VMay_HTxDJs85pp4OvXqtw.png" /></figure><h3><strong>Steps:</strong></h3><h4><strong>1.</strong> <strong>Store the Unstructed Json file in S3.</strong></h4><p>In this guide, we’re working with New Relic infrastructure logs, which are generated by the New Relic Infrastructure agent running on an EC2 instance. These logs are unstructured, lacking a consistent format, as shown in the image below. They contain various fields, making it impractical to use a Glue Crawler to process them effectively.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*LlwsUXfnPKsNZUx05HKHvg.png" /></figure><p>Upload you Unstructured JSON Log in the Bucket.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*-0g8f4FCuYvLSK4yT9ql8w.png" /></figure><h4>2. <strong>Create a Glue Database</strong></h4><p><strong>1.</strong> Next, create a database in AWS Glue to store the data catalog for your logs.</p><p>a. Go to <strong>AWS Glue → Databases</strong>.</p><p>b. Click <strong>Add Database</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*T0UbDjo-VybN30WKd3nexg.png" /></figure><p>c. Provide a name for the database and click <strong>Create Database</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*W6LsNoTJ-MZkt0A0xwPa1g.png" /></figure><h4>3. <strong>Create a IAM Role</strong></h4><p>To run the AWS Glue Crawler, create an IAM Role with the necessary permissions to access your S3 bucket.<br>Attach<strong> </strong>the managed policy AWSGlueServiceRole to grant necessary permissions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*WFaWEY6WsYKYf3qmHNPDXg.png" /></figure><p>Define an inline policy with permissions to access your S3 bucket like below image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*1u8mi0NFWc6BYVv6lkSlPA.png" /></figure><h4>4. <strong>Create Glue ETL Job</strong></h4><p>Since the JSON logs are unstructured, we need to use an ETL job to transform the data and extract key fields.</p><p>If you examine the log file, there are four key fields that appear frequently: context, msg, timestamp, and level. These fields vary in structure, so we can use regular expressions in the ETL job to extract them and store the results in the Glue Data Catalog table.</p><p><strong>Regular expressions:</strong></p><p>context_pattern = r’”context”:(\{[^}]+\})’</p><p>msg_pattern = r’”msg”:”([^”]+)”’</p><p>time_pattern = r’”timestamp”:”([^”]+)”’</p><p>level_pattern = r’”level”:”([^”]+)”’</p><p><strong>Steps to create a Glue ETL job:</strong></p><ol><li>Navigate to the <strong>AWS Glue</strong> dashboard and select <strong>ETL Jobs</strong>.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*vU0jrD1GbW9HTMUCgARoNA.png" /></figure><p>2. Click <strong>Create job</strong> and choose the <strong>Blank graph</strong> option.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*MFBTXPJ6pxJ_UcIp86lzEA.png" /></figure><p>3. In the <strong>Script</strong> section, add the provided ETL script. Make sure to specify the S3 log file URI, output bucket, database name, and table name.</p><pre>import sys<br>from awsglue.utils import getResolvedOptions<br>from pyspark.context import SparkContext<br>from awsglue.context import GlueContext<br>from awsglue.job import Job<br>from pyspark.sql.functions import regexp_extract<br>from awsglue.dynamicframe import DynamicFrame<br>from pyspark.sql.functions import col, trim<br><br># @params: [JOB_NAME, S3_URI, TABLE_NAME]<br>args = getResolvedOptions(sys.argv, [&#39;JOB_NAME&#39;])<br><br>sc = SparkContext()<br>glueContext = GlueContext(sc)<br>spark = glueContext.spark_session<br>job = Job(glueContext)<br>job.init(args[&#39;JOB_NAME&#39;], args)<br><br># Read the JSON log file from S3 using the provided S3 URI<br>df = spark.read.text(&quot;s3://logquerybucketblog/Dynamic_Logs.logs&quot;)<br>print(df.show())<br><br>context_pattern = r&#39;&quot;context&quot;:(\{[^}]+\})&#39;<br>msg_pattern = r&#39;&quot;msg&quot;:&quot;([^&quot;]+)&quot;&#39;<br>time_pattern = r&#39;timestamp&quot;:&quot;([^&quot;]+)&quot;&#39;<br>level_pattern = r&#39;&quot;level&quot;:&quot;([^&quot;]+)&quot;&#39;<br><br># Extract fields into separate columns<br>final_df = df.withColumn(&quot;Message&quot;, regexp_extract(&quot;value&quot;, msg_pattern, 1)) \<br>             .withColumn(&quot;Timestamp&quot;, regexp_extract(&quot;value&quot;, time_pattern, 1)) \<br>             .withColumn(&quot;Level&quot;, regexp_extract(&quot;value&quot;, level_pattern, 1)) \<br>             .withColumn(&quot;Context&quot;, regexp_extract(&quot;value&quot;, context_pattern, 1))<br><br>final_df = final_df.drop(&#39;value&#39;)<br><br>filtered_df = final_df.filter(<br>    (trim(col(&quot;Message&quot;)) != &quot;&quot;) |<br>    (trim(col(&quot;Timestamp&quot;)) != &quot;&quot;) |<br>    (trim(col(&quot;Level&quot;)) != &quot;&quot;) |<br>    (trim(col(&quot;Context&quot;)) != &quot;&quot;)<br>)<br><br>print(filtered_df.show())<br><br># Convert to DynamicFrame<br>dynamic_frame = DynamicFrame.fromDF(filtered_df, glueContext, &quot;dynamic_frame&quot;)<br><br>sink = glueContext.getSink(<br>    path=&quot;s3://output-logqueryblog/&quot;,<br>    connection_type=&quot;s3&quot;,<br>    updateBehavior=&quot;LOG&quot;,<br>    partitionKeys=[],<br>    compression=&quot;snappy&quot;,<br>    enableUpdateCatalog=True,<br>    transformation_ctx=&quot;sink&quot;<br>)<br>sink.setCatalogInfo(catalogDatabase=&quot;log_query_database&quot;, catalogTableName=&quot;dynamic_log_query_database&quot;)<br>sink.setFormat(&quot;json&quot;)<br>sink.writeFrame(dynamic_frame)<br><br>job.commit()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*S8nmEpwTZlzDnGVsRTlxEQ.png" /></figure><p>4. In the <strong>Job details</strong> tab, add a job name and assign the IAM role that was created earlier.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*TT3_TmycbVIbXmPsF3iBHA.png" /></figure><p>5. Save the job.</p><p>6. Ensure that the output bucket’s policy allows access by the IAM role.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*jBUwd_XB6QQHovtN2hKGsw.png" /></figure><h4>5. <strong>Run the AWS Glue ETL Job</strong></h4><p>To run the job, click the Run button.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*LwrvKGPqHnuF4FVyVAiufA.png" /></figure><p>You can track its progress in the Run tab, and it will take a few minutes to complete.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*0jb0wg6MJmx8u4dDSp2aTA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*lqFz_c3DO5LpYA3iPfw3Hg.png" /></figure><p>Once the job is finished, a new table will be generated in the Glue Data Catalog using the database name you provided in the code.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*7DMb_VKIBgxYpwLJPqWETA.png" /></figure><h4>6. <strong>Use Athena to Query the Data</strong></h4><p><strong>1.</strong> To query the data</p><p>a. In the AWS Glue console, go to <strong>Database</strong> → <strong>Table</strong> and select <strong>View Data</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*7DMb_VKIBgxYpwLJPqWETA.png" /></figure><p>b. Click <strong>Proceed</strong> on the pop-up that appears.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*4IfT6d8AtxAnKxi2Jae4kg.png" /></figure><p>c. You will be redirected to the Athena dashboard.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*mdCLdhPVJLsW4vpkydf5Ig.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*_mGXJczMyj7BcOY73nSb4A.png" /></figure><p>From there, you can run your SQL queries and view the results displayed in a table format.</p><p>And there you have it — unstructured logs transformed into a clean, queryable format using AWS Glue ETL. With Athena, you can now easily sift through the data and extract valuable insights. From chaos to clarity, this process shows the power of turning even the messiest logs into something meaningful.</p><p>Happy learning !!!!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=56fdc993ef98" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Querying Static JSON Logs with AWS Glue Crawler and Athena: A Step-by-Step Guide]]></title>
            <link>https://medium.com/@kelvi030117/querying-static-json-logs-with-aws-glue-crawler-and-athena-a-step-by-step-guide-c926ddad6b1a?source=rss-6a8f83281443------2</link>
            <guid isPermaLink="false">https://medium.com/p/c926ddad6b1a</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[aws-community]]></category>
            <category><![CDATA[aws-athena]]></category>
            <category><![CDATA[aws-glue-crawler]]></category>
            <category><![CDATA[aws-glue]]></category>
            <dc:creator><![CDATA[KELVI MANAVADARIA]]></dc:creator>
            <pubDate>Sat, 07 Sep 2024 18:24:26 GMT</pubDate>
            <atom:updated>2024-09-07T18:24:26.016Z</atom:updated>
            <content:encoded><![CDATA[<p>As a cloud engineer managing AWS applications, you’re swamped with hundreds of structured JSON logs daily. Manually sifting through them is cumbersome, so you turn to AWS Athena and AWS Glue.</p><p>You use AWS Glue Crawler to organize your logs into a structured format, then leverage Athena to run SQL queries on this data effortlessly. What once took hours is now streamlined, making log analysis quick and efficient.</p><h3>Let’s dive into the steps to achieve this:<br> <br> Architecture:</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/558/1*5sr7rbbJXBLGj3KXNF2ymA.png" /></figure><h3>Steps:</h3><h4>1. Store the Structed Json file in S3.</h4><p>We have a structured JSON file with a predefined schema, where field names and types are consistent across all instances. <br> <br>For example, in this JSON, the structure includes the following fixed fields: v, level, name, hostname, pid, time, msg, and component.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*_1urZS1iZuxGlxVa9ySufA.png" /></figure><p>Upload you Structed JSON Log in the Bucket.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*m5dZ-6jgSMJ1y6vVwIiDiQ.png" /></figure><p><strong>2.</strong> <strong>Create a Glue Database<br> </strong>Next, create a database in AWS Glue to store the data catalog for your logs.</p><p>a. Go to <strong>AWS Glue → Databases</strong>.</p><p>b. Click <strong>Add Database</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*T0UbDjo-VybN30WKd3nexg.png" /></figure><p>Provide a name for the database and click <strong>Create Database</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*W6LsNoTJ-MZkt0A0xwPa1g.png" /></figure><p><strong>3.</strong> <strong>Create a IAM Role<br> </strong>To run the AWS Glue Crawler, create an IAM Role with the necessary permissions to access your S3 bucket.</p><p>Attach<strong> </strong>the managed policy AWSGlueServiceRole to grant necessary permissions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*WFaWEY6WsYKYf3qmHNPDXg.png" /></figure><p>Define an inline policy with permissions to access your S3 bucket like below image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*1u8mi0NFWc6BYVv6lkSlPA.png" /></figure><h4>4. Set Up AWS Glue Crawler</h4><p>To set up the AWS Glue Crawler and organize your structured data into a table, follow these steps:</p><ol><li>Go to <strong>AWS Glue</strong> → <strong>Crawlers</strong> → <strong>Add Crawler</strong>.</li><li>Enter a name for your Glue Crawler and click <strong>Next</strong>.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*ekoorGSTNHDjeY9VjQ04eQ.png" /></figure><p>3. Select <strong>Not yet mapped to a Glue Table</strong> and click <strong>Add a data source</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*968taDpnKo4h5ouTDQfSJA.png" /></figure><p>4. Choose the S3 bucket containing your JSON file and click <strong>Next</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*gyXoNotUMA1V1a5yY647yw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*efwnJ4NeV5LIb-TFJc3Jmg.png" /></figure><p>5. Select the IAM role you created earlier and click <strong>Next</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*9dxM_tqL4S5arCJQ59wHUA.png" /></figure><p>6. Choose the Glue database you set up and click <strong>Next</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*-GiLfg1i7pfUTrVdGyhOKQ.png" /></figure><p>7. Review your settings and click <strong>Create Crawler</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*1aH0XaNOaZwsrnHrLjvFlw.png" /></figure><h4>5. Run the AWS Glue Crawler</h4><p>Once the crawler is created, you need to run it to process the data from your log file into the AWS Glue Data Catalog.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*V4f8ygyl0DVZ6DBNjYKbAA.png" /></figure><p>Click <strong>Run Crawler</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*ruQlpFovsz9j2qAadgkdIw.png" /></figure><p>After a short period, the crawler will complete its task. You can then go to the Glue database you created and check it to see the new table that has been added.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*xCjkVzweSinG7mLJggrb8w.png" /></figure><h4>6. Use Athena to Query the Data</h4><p>To query the data</p><p>a. In the AWS Glue console, go to <strong>Database</strong> → <strong>Table</strong> and select <strong>View Data</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*xCjkVzweSinG7mLJggrb8w.png" /></figure><p>b. Click <strong>Proceed</strong> on the pop-up that appears.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*4IfT6d8AtxAnKxi2Jae4kg.png" /></figure><p>c. You will be redirected to the Athena dashboard.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*Gu-UZNHTiy80SIzJtAvU3A.png" /></figure><p>d. From there, you can run your SQL queries and view the results displayed in a table format.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*CxTsHyupKDlAPYCuCkTBlg.png" /></figure><p>With AWS Glue and Athena, tasks that once took hours are now completed in minutes. You can even automate the process by scheduling Glue Crawlers, ensuring your logs are always up-to-date and ready for instant analysis. Just quick and easy SQL queries to access the insights you need.</p><p>Stay tuned for the next part, where we’ll dive into querying <strong>dynamic JSON logs</strong> and handling more complex data structures with the same ease!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c926ddad6b1a" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>