<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ali Abbas Jaffri on Medium]]></title>
        <description><![CDATA[Stories by Ali Abbas Jaffri on Medium]]></description>
        <link>https://medium.com/@aliabbasjaffri_?source=rss-3914e337751e------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Zu-GdfXhWxYhLgeIg3waEQ.jpeg</url>
            <title>Stories by Ali Abbas Jaffri on Medium</title>
            <link>https://medium.com/@aliabbasjaffri_?source=rss-3914e337751e------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 11:45:02 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@aliabbasjaffri_/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Introducing mlinfra — a hassle free way to deploy ML Infrastructure]]></title>
            <link>https://itnext.io/introducing-mlinfra-a-hassle-free-way-to-deploy-ml-infrastructure-13f77de8bcf7?source=rss-3914e337751e------2</link>
            <guid isPermaLink="false">https://medium.com/p/13f77de8bcf7</guid>
            <category><![CDATA[mlops-platform]]></category>
            <category><![CDATA[mlops-tool]]></category>
            <category><![CDATA[mlops]]></category>
            <category><![CDATA[infrastructure]]></category>
            <category><![CDATA[infrastructure-as-code]]></category>
            <dc:creator><![CDATA[Ali Abbas Jaffri]]></dc:creator>
            <pubDate>Wed, 31 Jan 2024 23:06:42 GMT</pubDate>
            <atom:updated>2024-01-31T23:06:42.425Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MWA5Tj0X-xem2_c8e5hD_w.png" /></figure><h3>Introducing mlinfra — a hassle free way to deploy ML Infrastructure</h3><p>I’m really excited to be launching <a href="https://mlinfra.io/">mlinfra</a>, a python package designed to streamline the deployment of various MLOps tools within an MLOps stack, quickly and with best practices. The core philosophy behind mlinfra is to simplify and expedite the deployment of MLOps infrastructure. This approach enables <em>ML Engineers</em> or <em>ML Platform engineers</em> to concentrate on delivering business value, by significantly reducing the time and resources typically required for deploying MLOps tools on the cloud.</p><p><a href="https://mlinfra.io/">mlinfra</a></p><p>As an ML Engineer, I well understand the complexities and challenges we face in deploying and configuring machine learning infrastructure. The traditional approach to deploying ML infrastructure often involves tedious, repetitive tasks such as writing complex terraform logic, leaving less time for the critical aspects of ML development. This is where mlinfra steps in. It’s designed to automate and streamline the deployment of various MLOps tools within a highly configurable MLOps stack, ensuring rapid deployment aligned with best practices.</p><p>The fundamental concept behind <strong>mlinfra</strong> is to establish a universal Infrastructure as Code (IaC) framework that expedites the creation and deployment of MLOps stacks. This package empowers <em>MLOps Engineers</em> and <em>Platform Engineers</em> to deploy a variety of MLOps tools across different stages of the machine learning lifecycle. The essence of <strong>mlinfra</strong> lies in its Python layer, which interprets the deployment configuration and utilizes Terraform modules to deploy these tools. It further enhances the process by dynamically generating a suite of inputs, roles, and permissions, thereby simplifying and streamlining the deployment process.</p><p>The heart of <strong>mlinfra</strong> lies in its core philosophy: simplify and accelerate the deployment of MLOps infrastructure to the cloud. This means you, as an engineer, can focus more on delivering robust business logic and less on the nuances of deployment. Developed from a personal need and countless hours of dedication, <strong>mlinfra</strong> offers a sophisticated, yet incredibly user-friendly approach to deploying MLOps stacks in the cloud.</p><p>✨ Features that set <strong>mlinfra</strong> apart:</p><ul><li>A lightweight Python package with intrinsic logic for cloud infrastructure deployment.</li><li>built on top of battle hardened renowned terraform modules like <a href="https://registry.terraform.io/namespaces/terraform-aws-modules">terraform-aws-modules</a> by <a href="https://medium.com/u/5312b1f1bba0">Anton Babenko</a>.</li><li>Cloud-agnostic design, enabling the effortless deployment of any MLOps lifecycle tool across various cloud platforms.</li></ul><p>Whether you’re an experienced ML engineer or a tech enthusiast, your input and support can significantly impact the development of <strong>mlinfra</strong>.</p><p>Using <strong>mlinfra</strong> is really straightforward as well; create a Python virtual environment, install the tool, and explore the <a href="https://mlinfra.io/code/aws/cloud_infra/">example stacks available</a> on our website.</p><pre>python -m pip install mlinfra</pre><p>For more information, dive into our <a href="https://mlinfra.io/">documentation</a>. Feel free to reach out directly for any queries or suggestions.</p><p><em>PS: </em>Do give the project some support by sharing it with people working in MLOps domain and giving it a star!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=13f77de8bcf7" width="1" height="1" alt=""><hr><p><a href="https://itnext.io/introducing-mlinfra-a-hassle-free-way-to-deploy-ml-infrastructure-13f77de8bcf7">Introducing mlinfra — a hassle free way to deploy ML Infrastructure</a> was originally published in <a href="https://itnext.io/">ITNEXT</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kubernetes V — Networking in kubernetes clusters — I]]></title>
            <link>https://itnext.io/kubernetes-v-networking-in-kubernetes-clusters-i-424cb9aed3e1?source=rss-3914e337751e------2</link>
            <guid isPermaLink="false">https://medium.com/p/424cb9aed3e1</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[control-plane]]></category>
            <category><![CDATA[cni]]></category>
            <category><![CDATA[networking]]></category>
            <category><![CDATA[worker-node]]></category>
            <dc:creator><![CDATA[Ali Abbas Jaffri]]></dc:creator>
            <pubDate>Mon, 16 Oct 2023 21:23:06 GMT</pubDate>
            <atom:updated>2023-10-16T21:23:06.187Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*y7xv3GfuQwr9jPlK" /><figcaption>Photo by <a href="https://unsplash.com/@carrier_lost?utm_source=medium&amp;utm_medium=referral">Ian Taylor</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Kubernetes V — Networking in kubernetes clusters — I</h3><p>A key component of Kubernetes is <em>networking</em>, which can be a bit confusing for users new to the technology. It is the communication layer that allows components on multiple levels to pass information to each other. There are many layers to Kubernetes networking, and we’ll cover these all in the coming posts. These can be grouped into three major parts:</p><p><strong>Part 1:</strong></p><ul><li>Control Plane &amp; worker node communication</li><li>Control plane in high availability</li><li>Worker node to worker node communication</li><li>Worker node to control plane communication</li></ul><p><strong>Part2</strong>: Accessing applications running within the Kubernetes cluster (services and ingresses)</p><p><strong>Part 3</strong>: Scaling to thousands of services that intercommunicate with each other (<em>service mesh architecture</em>)</p><p>Kubernetes cluster relies on worker nodes to orchestrate different services and workloads. During cluster creation, these worker nodes are created along the control plane and are usually based on top of virtual machines; or EC2 instances in AWS world. In the case of a managed kubernetes cluster, the internals and management of the control plane are hidden from the user; however, the worker nodes are visible in the EC2 console. These worker nodes need to discover and register themselves to their respective control plane after creation to become a part of the kubernetes cluster.</p><ul><li><a href="https://kubernetes.io/docs/concepts/cluster-administration/networking/">Cluster Networking</a></li><li><a href="https://www.tigera.io/learn/guides/kubernetes-networking/">Kubernetes Networking</a></li></ul><h3>Control Plane &amp; worker node communication</h3><h4>Worker node to control plane during cluster creation</h4><p>The kubelet is an essential component that runs on each worker node and is responsible for managing communications and containers on the node and between the node and the control plane. During a worker node startup, the kubelet communicates with the control plane, through the kubernetes apiserver, to register the node. It uses the information in its kubeconfig file (passed to the worker node during node initialization) to authenticate and connect to the apiserver. The kubelet sends a Node object to the API server, providing information about the node’s capacity and resources. Once the API server receives this information and verifies the node’s credentials, it adds the worker node to the cluster.</p><pre>apiVersion: v1<br>kind: Node<br>metadata:<br>  name: my-worker-node<br>  labels:<br>    environment: production<br>    location: datacenter-1<br>spec:<br>  taints:<br>    - key: node-role<br>      value: worker<br>      effect: NoSchedule<br>  unschedulable: false<br>status:<br>  addresses:<br>    - type: InternalIP<br>      address: 192.168.1.100<br>    - type: Hostname<br>      address: my-worker-node<br>  capacity:<br>    cpu: &quot;4&quot;<br>    memory: 8Gi<br>  allocatable:<br>    cpu: &quot;2&quot;<br>    memory: 4Gi<br>  conditions:<br>    - type: Ready<br>      status: &quot;True&quot;<br>      lastHeartbeatTime: &quot;2023-10-15T12:00:00Z&quot;<br>      lastTransitionTime: &quot;2023-10-15T11:58:00Z&quot;</pre><p><a href="https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.27/#node-v1-core">Kubernetes API Reference Docs</a></p><p>The control plane components and services, including the kubernetes apiserver, typically have DNS names associated with them. Worker nodes use DNS resolution to find the API server’s address. The DNS name for the Kubernetes API server is usually kubernetes.default.svc.cluster.local. The worker node resolves this DNS name to an IP address and connects to the API server using the resolved address.</p><ul><li><a href="https://kubernetes.io/docs/tasks/administer-cluster/dns-debugging-resolution/">Debugging DNS Resolution</a></li><li><a href="https://www.nslookup.io/learning/the-life-of-a-dns-query-in-kubernetes/">The life of a DNS query in Kubernetes</a></li></ul><h4>Control plane to worker nodes after cluster creation</h4><p>Now that the worker nodes have found the control plane and registered with it, control plane needs to interact with its worker nodes for orchestrating and management of services and workloads. All communications between control plane nodes and worker nodes are secured via TLS and the control plane nodes and worker nodes use TLS certificates for authentication and to establish secure communication channels. The primary communication channel from the control plane to worker nodes is through the kubelet running on each worker node. The kubelet is an agent that runs on every node in the cluster and is responsible for managing containers on that node.</p><ul><li>The control plane components, including the API server, scheduler, and controller manager, communicate with the kubelet on each worker node to request the creation, deletion, or inspection of containers.</li><li>The kubelet exposes an HTTP API (by default on port 10255) that the control plane components use to send commands to the kubelet on each node.</li><li>The kubelet also periodically reports the node’s status, including the status of running containers, back to the control plane.</li></ul><p>Kube Proxy is another component that runs on each worker node. It is responsible for maintaining network rules on the node to ensure that network traffic is correctly routed to containers. Kube Proxy communicates with the control plane to sync network rules based on changes to service definitions in Kubernetes. It uses the Kubernetes API server to watch for changes and update iptables rules accordingly.</p><p>While not a direct communication channel between the control plane and worker nodes, the control plane indirectly interacts with the container runtime through the kubelet. The kubelet communicates with the container runtime to start, stop, and manage containers as directed by the control plane. The control plane indirectly influences communication between pods running on different worker nodes through the assignment of IP addresses, routing, and network policies. This is referred to as the Container Networking Interface or CNI. Network plugins and solutions (e.g., <a href="https://www.tigera.io/project-calico/">Calico</a>, <a href="https://github.com/flannel-io/flannel">Flannel</a>, <a href="https://cilium.io/">Cilium</a>) are responsible for configuring networking rules and ensuring pod-to-pod communication across worker nodes.</p><ul><li><a href="https://kubernetes.io/docs/concepts/architecture/control-plane-node-communication/">Communication between Nodes and the Control Plane</a></li><li><a href="https://www.suse.com/c/rancher_blog/comparing-kubernetes-cni-providers-flannel-calico-canal-and-weave/">Comparing Kubernetes CNI Providers: Flannel, Calico, Cana...</a></li></ul><h3>Control plane nodes and components in high availability</h3><p>High availability setup in a kubernetes cluster relies heavily on the underlying communication layer between components to ensure that the cluster and its services are available even in case when some of the nodes or node components are unhealthy. In HA setup, the election of a leader node is crucial to ensure that control plane and its components (such as the API server, controller manager, and scheduler) continue to function even if one of the control plane nodes fails. The leader election process in Kubernetes is based on distributed consensus algorithms like Raft which is the election mechanisms used by the underlying etcd key-value store.</p><p>In an HA setup, etcd, which is kubernetes distributed key-value store to store cluster configuration and state is typically deployed as a cluster across multiple nodes. Control plane components like the API server, controller manager, and scheduler are designed to operate in a leader/follower mode for fault tolerance.</p><p>When these components start, they attempt to acquire a lease (lock) in etcd to become the leader. The lease typically has a time-to-live (TTL) associated with it. If a component successfully acquires the lease, it becomes the leader and can perform its tasks. The leader retains its position in the cluster by maintaining connectivity to etcd and updating it regarding its health. This connectivity to etcd is maintained via liveness probes and health checks to determine the health and availability of components. If a component becomes unhealthy or unresponsive, it may be evicted from the leader position, triggering a new leader election.</p><p>Etcd uses a leader election algorithm to ensure that only one control plane node or component is the leader at any given time. When a leader fails or loses connectivity, etcd initiates a new leader election. Control plane nodes or component instances contend for leadership by attempting to acquire the lease in etcd. The first one to successfully acquire the lease becomes the leader. To maintain consistency and fault tolerance, an odd number of etcd nodes is required in an etcd cluster (e.g., 3, 5, 7 nodes). This ensures that a majority of nodes must agree on the leader, and in the event of network partitions or node failures, the cluster can still function without risking data inconsistency.</p><ul><li><a href="https://etcd.io/docs/v3.5/faq/">FAQ</a></li><li><a href="https://raft.github.io/">Raft Consensus Algorithm</a></li></ul><h3>Worker node to worker node communication</h3><p>Worker-to-worker communication in Kubernetes primarily occurs when pods running on different worker nodes need to communicate with each other over the network. Kubernetes provides a network overlay solution to enable this communication, and the specifics of how it works depend on the chosen container networking solution. Here’s a general overview of how worker-to-worker communication takes place:</p><ol><li><strong>Pod Communication Abstraction</strong>: Kubernetes abstracts network communication between pods by assigning each pod a unique IP address. Pods can communicate with each other using these IP addresses, regardless of the worker node they are running on.</li><li><strong>Container Network Interface (CNI)</strong>: Kubernetes uses a Container Network Interface (CNI) plugin to manage the networking configuration for pods. The CNI plugin is responsible for setting up the network stack, IP allocation, routing, and firewall rules.</li><li><strong>Network Overlay</strong>: Most CNI plugins implement a network overlay solution. An overlay network abstracts the underlying physical or virtual network infrastructure, allowing pods on different worker nodes to communicate as if they were on the same network.</li><li><strong>Routing and Tunneling</strong>: When a pod on one worker node needs to communicate with a pod on another worker node, the overlay network uses techniques like VXLAN, Geneve, or other encapsulation methods to create tunnels between nodes. These tunnels encapsulate the network packets and route them to the destination node.</li><li><strong>Network Policy</strong>: Kubernetes Network Policies allow administrators to define rules for network traffic between pods. Network Policies can control which pods are allowed to communicate with each other, helping to enforce security and segmentation.</li><li><strong>Service Load Balancing</strong>: When accessing services in Kubernetes (e.g., a Service object with multiple pods as endpoints), kube-proxy on each worker node is responsible for load balancing the traffic to the appropriate pod. This load balancing ensures that client pods on different nodes can access the service without needing to know the specific IP address of individual pods.</li><li><strong>DNS Resolution</strong>: Kubernetes provides DNS-based service discovery for pods. When a pod needs to communicate with another pod, it can use the DNS name of the service, and Kubernetes DNS resolves that name to the appropriate pod’s IP address.</li><li><strong>Firewall Rules</strong>: Network policies, firewall rules, and security group configurations (depending on the underlying infrastructure) play a role in controlling and securing network traffic between pods on different worker nodes. These rules help enforce the desired network policies and security posture.</li><li><strong>Pod IP Address Assignment</strong>: Each pod running in the cluster is assigned a unique IP address from the CIDR block of the overlay network. The CNI plugin manages IP address allocation and ensures that there are no IP conflicts.</li></ol><p>Kubernetes’ container networking solutions abstract the underlying network infrastructure complexities and allow pods on different worker nodes to communicate seamlessly, making it easier to build distributed applications. The specific implementation details and configuration options may vary depending on the chosen CNI plugin and network policy settings.</p><ul><li><a href="https://aws.amazon.com/blogs/containers/de-mystifying-cluster-networking-for-amazon-eks-worker-nodes/">De-mystifying cluster networking for Amazon EKS worker nodes | Amazon Web Services</a></li><li><a href="https://komodor.com/learn/kubernetes-nodes-complete-guide/">Kubernetes Nodes - The Complete Guide</a></li></ul><h3>Worker node to control plane for status checks and heart beats</h3><p>Worker nodes in a Kubernetes cluster have a heartbeat mechanism as part of their communication with the control plane components. The heartbeat mechanism is used to detect the health and availability of worker nodes and ensure that they are responsive and operational. The primary component responsible for the heartbeat mechanism on worker nodes is the kubelet. The kubelet runs on each worker node and is responsible for managing containers and reporting the node’s status to the control plane. The kubelet sends periodic “heartbeats” or probes to the control plane components, such as the Kubernetes API server, to let them know that the node is still alive and functioning. These probes are typically sent at regular intervals. Along with the heartbeats, the kubelet also sends updates about the node’s status, including information about the containers running on the node, available resources, and any issues it has detected (e.g., container failures).</p><p>The control plane components, including the Kubernetes API server, continuously monitor the heartbeats and status updates from the kubelet. If the control plane detects that a node has not sent a heartbeat within a certain timeout period or if it receives a specific signal from the kubelet indicating a problem, it considers the node as “unhealthy.” When a node is deemed unhealthy due to a lack of heartbeats or other issues, the control plane may take actions such as rescheduling the pods running on the unhealthy node to other healthy nodes. This helps maintain the desired level of availability and reliability for applications running in the cluster. In addition to kubelet-to-control-plane heartbeats, Kubernetes also supports liveness probes, which are defined at the pod level. Liveness probes allow Kubernetes to check the health of individual containers within pods. If a container fails its liveness probe, Kubernetes may restart the container or, in some cases, take more drastic actions like restarting the entire pod.</p><ul><li><a href="https://kubernetes.io/docs/concepts/architecture/nodes/">Nodes</a></li><li><a href="https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/589-efficient-node-heartbeats/README.md">enhancements/keps/sig-node/589-efficient-node-heartbeats/README.md at master · kubernetes/enhancements</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=424cb9aed3e1" width="1" height="1" alt=""><hr><p><a href="https://itnext.io/kubernetes-v-networking-in-kubernetes-clusters-i-424cb9aed3e1">Kubernetes V — Networking in kubernetes clusters — I</a> was originally published in <a href="https://itnext.io/">ITNEXT</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kubernetes IV — Composition of a Kubernetes node — II]]></title>
            <link>https://itnext.io/kubernetes-iv-composition-of-a-kubernetes-node-ii-e241511db100?source=rss-3914e337751e------2</link>
            <guid isPermaLink="false">https://medium.com/p/e241511db100</guid>
            <category><![CDATA[kube-proxy]]></category>
            <category><![CDATA[data-plane]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[container-runtime]]></category>
            <category><![CDATA[kubelet]]></category>
            <dc:creator><![CDATA[Ali Abbas Jaffri]]></dc:creator>
            <pubDate>Sun, 08 Oct 2023 12:34:36 GMT</pubDate>
            <atom:updated>2023-10-08T13:02:17.433Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*70cX7Ra4TvsX3e6We-whUg.png" /><figcaption>source: <a href="https://www.appviewx.com/education-center/what-is-kubernetes/">appviewx.com</a></figcaption></figure><h3>Kubernetes IV: Composition of a Kubernetes node — II</h3><p>We already discussed the <a href="https://itnext.io/kubernetes-iii-composition-of-a-kubernetes-node-i-cf766178a0ac">control plane at length in a previous post</a> and saw different control plane components and their roles and responsibilities in managing a Kubernetes cluster. While the control plane manages the state and outsources workloads to the data plane, the data plane runs these workloads, which are in the form of containerized applications. The data plane or the worker nodes are the workhorse of a Kubernetes cluster.</p><p><a href="https://itnext.io/kubernetes-iii-composition-of-a-kubernetes-node-i-cf766178a0ac">Kubernetes III — Composition of a Kubernetes node — I</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2d6KCf0rV6r_lgbUB4mKqw.png" /><figcaption>Source: <a href="https://kubernetes.io/docs/concepts/overview/components/">Kubernetes documentation</a></figcaption></figure><h3>Node Components</h3><p>Worker nodes, also known as data plane or just nodes, are an essential part of a Kubernetes cluster responsible for running containerized applications. They perform various tasks to ensure that containers are scheduled, deployed, and managed effectively. Node components run on every node and communicate with the control plane to maintain the desired state of the cluster.</p><h4>kubelet</h4><p>Kubelet is a fundamental component of a Kubernetes worker node, and its role is to ensure that Pods are running correctly and conform to the desired state as defined in the Kubernetes manifests on that node. Kubelet is the node-level agent that bridges the gap between the Kubernetes control plane and the container runtime on a worker node. It plays a critical role in maintaining the health and performance of containers on a Kubernetes cluster’s worker nodes. Some main responsibilities of kubelet include:</p><ol><li><strong>Pod Lifecycle Management:</strong> Kubelet is responsible for the creation, management, and monitoring of Pods on a worker node. It watches the Kubernetes API server for Pod specifications assigned to the node and ensures that the containers defined in those Pods are running. It starts and stops containers as needed to match the desired state described in the Pod manifests.</li><li><strong>Container Execution:</strong> Kubelet communicates with the container runtime (e.g., <a href="https://www.docker.com/">Docker</a>, <a href="https://containerd.io/">containerd</a>) running on the node. It instructs the container runtime to create and manage containers based on the specifications provided in the Pod manifest. Kubelet ensures that the containers are properly initialized, running, and healthy.</li><li><strong>Health Monitoring:</strong> Kubelet continually monitors the health of containers within its jurisdiction. It checks the status of containers and restarts them if they fail or become unhealthy, helping to maintain the desired state of the Pod.</li><li><strong>Resource Management:</strong> Kubelet is responsible for managing resources (CPU, memory, etc.) allocated to containers. It enforces resource limits and allocates resources based on the Pod specifications, ensuring fair resource sharing among containers on the same node.</li><li><strong>Node Status Reporting:</strong> Kubelet reports the node’s status and resource utilization back to the Kubernetes control plane. This information is crucial for the scheduler to make informed decisions about where to place new Pods based on resource availability.</li><li><strong>Eviction and Cleanup:</strong> Kubelet also performs eviction and cleanup tasks when the node becomes resource-constrained. It terminates containers that exceed their resource limits or remove old and unused containers and images to free up resources.</li></ol><p>Kubelet also manages images by pulling them from container registries (e.g., <a href="https://hub.docker.com/">Docker Hub</a>) when needed and caches them on the node to avoid redundant downloads. It also controls node-specific configurations, such as applying node labels and taints to the node as directed by the Kubernetes API server. These configurations help in node selection and pod placement.</p><ul><li><a href="https://github.com/kubernetes/kubelet">GitHub - kubernetes/kubelet: kubelet component configs</a></li><li><a href="https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/">kubelet</a></li><li><a href="https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/">Resource Management for Pods and Containers</a></li><li><a href="https://kube.academy/courses/the-kubernetes-machine/lessons/kubelet">Kubelet - The Kubernetes Machine - KubeAcademy</a></li></ul><h4>kube-proxy</h4><p>kube-proxy, short for kubernetes proxy, manages network traffic and enables network communication between pods, both within the worker node and across the entire Kubernetes cluster. Its role is crucial for ensuring that applications can communicate with each other, that traffic is load-balanced effectively, and that network policies are enforced consistently within the cluster. Some important tasks of kube-proxy include:</p><ol><li><strong>Network Proxying:</strong> kube-proxy serves as a network proxy for the Pods on the worker node. It routes network traffic to the appropriate destination based on the rules and policies defined in the Kubernetes Services and Endpoints resources. This enables communication between Pods running on different nodes and maintains network connectivity within the cluster.</li><li><strong>Network Policies:</strong> kube-proxy also enforces network policies defined within the cluster. Network policies specify rules for controlling the flow of traffic between Pods based on labels, allowing administrators to define fine-grained network access controls.</li><li><strong>Service Discovery:</strong> kube-proxy helps with service discovery within the cluster. It allows Pods to access services by their DNS names, which are automatically resolved to the corresponding IP addresses of the Pods behind the service. This abstracts the underlying network complexity for the application.</li><li><strong>Load Balancing:</strong> kube-proxy distributes incoming network requests or connections to these Pods in a balanced manner. This load balancing ensures that traffic is evenly distributed among the available Pods, improving application availability and scalability.</li><li><strong>Connection Tracking:</strong> kube-proxy keeps track of established network connections to ensure that packets are forwarded correctly. It manages Network Address Translation (NAT) for outgoing connections from Pods to external destinations, making it appear as if traffic originates from the worker node.</li><li><strong>Endpoint Updates:</strong> kube-proxy monitors the Kubernetes API server for changes in Service and Endpoint resources. When a Service or Pod is added or removed, Kube Proxy updates its rules and configurations dynamically to reflect these changes, ensuring that traffic is correctly routed to the relevant Pods.</li><li><strong>IP Tables or IPVS Rules:</strong> kube-proxy typically relies on one of two modes to implement network proxying and load balancing: IP tables or IP Virtual Server (IPVS). The choice of mode depends on the cluster configuration and the user’s preferences. IP tables mode uses Linux kernel’s IP tables rules, while IPVS mode leverages the IPVS kernel module for efficient load balancing.</li></ol><ul><li><a href="https://github.com/kubernetes/kube-proxy">GitHub - kubernetes/kube-proxy: kube-proxy component configs</a></li><li><a href="https://kubernetes.io/docs/reference/command-line-tools-reference/kube-proxy/">kube-proxy</a></li><li><a href="https://kodekloud.com/blog/kube-proxy/">Kube-Proxy: What Is It and How It Works</a></li><li><a href="https://mayankshah.dev/blog/demystifying-kube-proxy/">Demystifying kube-proxy</a></li></ul><h4>Container runtime</h4><p>The container runtime executes and manages containers in a node. Its primary role is to create and run containerized applications efficiently and securely, such as Docker containers, and provide the necessary runtime isolation and environment for these containers. It ensures that container applications adhere to resource constraints, are able to communicate andinteract with the host system as well as with other containers as needed. Common roles of a container runtime environment include:</p><ol><li><strong>Container Execution:</strong> The core responsibility of the container runtime is to run containers based on container images. It takes container images and creates instances of these containers on the worker node. These containers can be based on Docker images or other container formats, depending on the runtime in use.</li><li><strong>Isolation:</strong> Container runtimes provide strong process and filesystem isolation for containers. Each container runs in its own isolated environment, ensuring that processes within one container do not interfere with or affect processes in other containers or on the host system. This isolation enhances security and reliability.</li><li><strong>Resource Management:</strong> Container runtimes help manage resources such as CPU, memory, and storage for containers. They enforce resource limits and usage constraints specified in the Pod or container configuration, ensuring that containers do not consume more resources than allocated.</li><li><strong>Networking:</strong> Container runtimes set up networking for containers, allowing them to communicate with each other, the host system, and external networks. They manage network namespaces and may provide features like network address translation (NAT) to allow containers to share the host’s network stack securely.</li><li><strong>Storage:</strong> Container runtimes handle container storage, including creating writable layers for containers, managing volumes, and providing a file system for the container’s use. They ensure that container data is isolated and preserved across container restarts.</li><li><strong>Image Management:</strong> Container runtimes fetch, store, and manage container images from container registries as needed. They handle image caching, pulling new images when required, and storing image layers efficiently to minimize storage space.</li><li><strong>Security:</strong> Container runtimes implement security features like user namespaces, seccomp profiles, and AppArmor/SELinux policies to enhance the security of containers. These features help prevent privilege escalation and limit container capabilities.</li><li><strong>Lifecycle Management:</strong> Container runtimes manage the entire container lifecycle, including container creation, execution, monitoring, and termination. They ensure that containers are started and stopped according to the desired state defined in Kubernetes Pod specifications.</li><li><strong>Interoperability:</strong> Kubernetes interacts with container runtimes through the Container Runtime Interface (CRI), which provides a standardized interface for communication between Kubernetes components (like Kubelet) and the container runtime. This allows for flexibility in choosing different container runtimes while maintaining compatibility with Kubernetes.</li></ol><ul><li><a href="https://kubernetes.io/docs/concepts/containers/#container-runtimes">Containers</a></li><li><a href="https://kubernetes.io/docs/concepts/architecture/cri/">Container Runtime Interface (CRI)</a></li><li><a href="https://containerd.io/">containerd</a></li><li><a href="https://cri-o.io/">cri-o</a></li></ul><h4>Replacing containerd for Docker as CRI</h4><p>Kubernetes 1.20 announcement came with the news of deprecation of docker shim as a CRI. Since then, containerd has gained popularity as a replacement for Docker as the Container Runtime Interface (CRI) in Kubernetes. Containerd adheres to the Open Container Initiative (OCI) standards, making it a well-established and standardized container runtime. This aligns with Kubernetes’ goal of standardization and interoperability, reducing vendor lock-in. Containerd is designed to be a minimalist runtime focused solely on container management. This simplicity means it has a smaller codebase, which can lead to fewer potential security vulnerabilities and a reduced attack surface compared to Docker.</p><p>containerd also offers improved container startup times and reduced resource overhead compared to Docker, as it eliminates some of the higher-level features and abstractions, such as user friendly UX, that is useful for human users but unnecessary for Kubernetes. I would encourage you to read all about the details behind the move, especially how the Kubernetes maintainers were rather caught up in the middle of a management of tools crisis.</p><ul><li><a href="https://kubernetes.io/blog/2020/12/08/kubernetes-1-20-release-announcement/#dockershim-deprecation">Kubernetes 1.20: The Raddest Release</a></li><li><a href="https://kubernetes.io/blog/2020/12/02/dont-panic-kubernetes-and-docker/">Don&#39;t Panic: Kubernetes and Docker</a></li><li><a href="https://opencontainers.org/">Open Container Initiative</a></li><li><a href="https://kubernetespodcast.com/episode/208-containerd/">Kubernetes Podcast from Google: Episode 208 - History of containerd, with Phil Estes</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e241511db100" width="1" height="1" alt=""><hr><p><a href="https://itnext.io/kubernetes-iv-composition-of-a-kubernetes-node-ii-e241511db100">Kubernetes IV — Composition of a Kubernetes node — II</a> was originally published in <a href="https://itnext.io/">ITNEXT</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kubernetes III — Composition of a Kubernetes node — I]]></title>
            <link>https://itnext.io/kubernetes-iii-composition-of-a-kubernetes-node-i-cf766178a0ac?source=rss-3914e337751e------2</link>
            <guid isPermaLink="false">https://medium.com/p/cf766178a0ac</guid>
            <category><![CDATA[kube-controller-manager]]></category>
            <category><![CDATA[kubernetes-scheduler]]></category>
            <category><![CDATA[kube-apiserver]]></category>
            <category><![CDATA[control-plane]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Ali Abbas Jaffri]]></dc:creator>
            <pubDate>Tue, 03 Oct 2023 07:02:39 GMT</pubDate>
            <atom:updated>2023-10-08T13:01:32.666Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*70cX7Ra4TvsX3e6We-whUg.png" /><figcaption>source: <a href="https://www.appviewx.com/education-center/what-is-kubernetes/">appviewx.com</a></figcaption></figure><h3>Kubernetes III: Composition of a Kubernetes node — I</h3><p>I’ve been away for a while from contributing because of some challenging ordeals I had undertaken in the past year. I’ll share those in future posts and shed some light on a new journey I have set off on.</p><p>We’ve seen in previous posts a bit about deploying Kubernetes the right way. The idea was to understand how to architect a Kubernetes cluster for scalable usecases using terraform and terragrunt. However, I didn’t cover the architectural details of the different components of a Kubernetes cluster and their purposes. This post describes the roles and responsibilities of Kubernetes components in reasonable depth.</p><ul><li><a href="https://itnext.io/kubernetes-i-multi-node-deployment-using-terraform-and-terragrunt-30c40a1238e8">Kubernetes — I: multi-node deployment using terraform and terragrunt</a></li><li><a href="https://itnext.io/kubernetes-ii-deploying-applications-on-dedicated-nodes-using-terraform-terragrunt-9fef9ea96d8d">Kubernetes — II: Deploying applications on dedicated nodes using terraform &amp; terragrunt</a></li></ul><p>Kubernetes consists of two main components: the control-plane and the data-plane. The control plane is responsible for managing and maintaining the state of Kubernetes cluster and ensuring the health and integrity of data plane, more commonly referred as worker nodes or simply nodes, and the workloads running on top of them. While control plane manages the state and outsources workloads to data plane, data plane runs / executes these workloads, which are in the form of containerised applications. Every Kubernetes cluster has at least one control plane and one data plane. Borrowed from Kubernetes documentation page, the following image outlines different components of a Kubernetes cluster. We will be using this as the reference to explain different components.</p><figure><img alt="different components of control plane and data plane of kubernetes cluster" src="https://cdn-images-1.medium.com/max/1024/1*2d6KCf0rV6r_lgbUB4mKqw.png" /><figcaption>Source: <a href="https://kubernetes.io/docs/concepts/overview/components/">Kubernetes documentation</a></figcaption></figure><p><em>We’ll cover control plane components for now, as there is a lot of ground to cover. Data plane components are coming up in a post later.</em></p><p><a href="https://kubernetes.io/docs/concepts/overview/components/">Kubernetes Components</a></p><h3>Control Plane Components</h3><p>Control plane is the brain of a Kubernetes cluster. It is responsible for managing and maintaining the state of applications and those of the worker nodes, ensure order across the cluster, manage scheduling of workloads and nodes and detecting and responding to cluster events (for example, starting up a new <a href="https://kubernetes.io/docs/concepts/workloads/pods/">pod</a> when a deployment’s replicas field is unsatisfied). Control plane components are as follows:</p><h3>apiserver</h3><p>In Kubernetes, apiserver is the central component that manages the configuration, state, and interactions within a Kubernetes cluster. It serves as the gateway for users and applications to interact with the cluster, enforces security policies, and orchestrates actions on cluster resources. Its critical role in Kubernetes makes it a fundamental component of cluster management and orchestration. API server has following primary responsibilities:</p><ol><li><strong>API Endpoint for Kubernetes:</strong> The API server provides a unified and consistent API endpoint that clients, including administrators, developers, and automation tools, use to interact with the Kubernetes cluster. This API is the primary interface through which users and applications communicate with the cluster.</li><li><strong>Cluster Configuration and State:</strong> The API server stores the configuration and state of the entire Kubernetes cluster. It keeps track of all the resources and objects in the cluster, such as pods, services, nodes, deployments, and more. This includes information about the desired state (as declared by users or defined in configuration files) and the current observed state of these resources.</li><li><strong>Resource Management:</strong> The API server handles the lifecycle management of Kubernetes resources. Users and applications can create, update, delete, and query resources by making API requests to the API server. It enforces access control and authorization policies to ensure that only authorized users can perform specific actions on resources.</li><li><strong>Admission Control:</strong> The API server is responsible for admission control, which involves validating and mutating requests to ensure they conform to cluster policies and constraints. Admission controllers can enforce policies related to security, resource quotas, and more.</li><li><strong>Authentication and Authorization:</strong> The API server manages authentication and authorization. It supports various authentication mechanisms, including client certificates, bearer tokens, and more. It verifies the identity of users and ensures they have the necessary permissions to perform requested actions on resources.</li><li><strong>Kubelet Communication:</strong> The API server communicates with kubelets running on worker nodes. It sends instructions to kubelets to create, update, or delete pods on worker nodes. Kubelets, in turn, report the status and health of nodes and pods back to the API server.</li></ol><p>Apart from these, apiserver also generates events that capture changes and activities within the cluster. These events can be monitored and used for cluster troubleshooting, auditing, and logging purposes. The API server supports admission webhooks (we will cover these later), which allow users to extend and customize the validation and mutation of requests and resources in a pluggable way. In an HA (high availability) configuration, the API server can be replicated and load-balanced to ensure that the control plane remains available even if one or more API server instances fail.</p><ul><li><a href="https://github.com/kubernetes/apiserver">GitHub - kubernetes/apiserver: Library for writing a Kubernetes-style API server.</a></li><li><a href="https://kubernetes.io/docs/reference/command-line-tools-reference/kube-apiserver/">kube-apiserver</a></li></ul><h3>etcd</h3><p><a href="https://etcd.io/docs/">etcd</a> is a critical component of the Kubernetes control plane that serves as a distributed and consistent key-value store. It stores cluster configuration, resource state, and other essential data, ensuring that the control plane components can operate reliably, even in complex and dynamic environments. etcd’s design principles of <a href="https://etcd.io/docs/v3.2/faq/#should-i-add-a-member-before-removing-an-unhealthy-member">distributed consensus</a> and <a href="https://etcd.io/docs/v3.5/op-guide/clustering/">high availability</a> are essential for the stability and robustness of Kubernetes clusters.</p><ol><li><strong>Data Storage and Consistency:</strong> etcd is responsible for storing and managing all of the configuration data and the current state of the Kubernetes cluster. This includes information about nodes, pods, services, configuration settings, and other cluster-related data. etcd provides a reliable and consistent storage layer that ensures that the cluster’s data remains accurate and consistent.</li><li><strong>Configuration Settings:</strong> Kubernetes itself relies on etcd to store its configuration settings. This includes cluster-level configuration, such as API server flags, admission control policies, and network settings.</li><li><strong>State of Cluster Resources:</strong> etcd maintains the state of all cluster resources. For example, when you create a Kubernetes Deployment or a Service, etcd stores information about these resources, including their desired state and current observed state.</li><li><strong>Event Streaming:</strong> etcd can be configured to provide event streaming, allowing components of the control plane and external systems to watch for changes to cluster resources. This feature is crucial for components like the API server, which need to react to changes in resource states.</li><li><strong>Coordination and Locking:</strong> etcd can be used for coordination and distributed locking purposes. Kubernetes components use etcd to coordinate actions and ensure that they don’t conflict with one another.</li><li><strong>Dynamic Configuration:</strong> Kubernetes allows for dynamic configuration updates. When configuration changes are made (e.g., through <strong>kubectl</strong> or by an administrator), etcd is updated, and this change is immediately reflected in the cluster&#39;s behavior.</li></ol><p>etcd is designed to be distributed and highly available. It can be deployed as a cluster across multiple nodes, and it uses distributed consensus algorithms (such as <a href="https://raft.github.io/">Raft</a>) to ensure that data remains available and consistent, even in the presence of node failures or network partitions. As a distributed data store, etcd can be scaled horizontally by adding more etcd nodes to the cluster. This can improve read and write performance and enhance the overall reliability of the control plane.</p><p>etcd supports backup and restore mechanisms, allowing administrators to take snapshots of the cluster’s state. These snapshots can be used for disaster recovery or for migrating clusters. It is rather unclear whether EKS has etcd backup mechanism in place for restoring etcd state for disaster recovery, but there are external tools, such as <a href="https://velero.io/docs/v1.8/backup-reference/">valero by vmware-tanzo</a> that can be used to backup an etcd cluster. As a <em>brownie point</em> to your etcd knowledge, i would recommend spending some time to <a href="https://aws.amazon.com/blogs/containers/managing-etcd-database-size-on-amazon-eks-clusters/">understand management of etcd database size</a> and <a href="https://aws.amazon.com/blogs/containers/explore-etcd-defragmentation-in-amazon-eks">its defragmentation</a>. These have an impact on the decision making process in Kubernetes and you should keep an eye on these in a production cluster.</p><ul><li><a href="https://etcd.io/docs/">Documentation versions</a></li><li><a href="https://kubernetes.io/docs/tasks/administer-cluster/configure-upgrade-etcd/">Operating etcd clusters for Kubernetes</a></li><li><a href="https://aws.amazon.com/blogs/containers/backup-and-restore-your-amazon-eks-cluster-resources-using-velero/">Backup and restore your Amazon EKS cluster resources using Velero | Amazon Web Services</a></li></ul><h3>kube-scheduler</h3><p>kube-scheduler plays an important role in Kubernetes by making intelligent decisions about where to place pods in the cluster based on resource requirements, constraints, and other policies. This ensures efficient resource utilization, high availability, and optimal workload distribution, contributing to the overall reliability and performance of Kubernetes applications. Some important tasks that kube-scheduler undertakes are:</p><ol><li><strong>Pod Scheduling:</strong> When a user or application creates a new pod resource in the cluster, the kube-scheduler is responsible for determining which worker node should run that pod. It takes into account a variety of factors to make this decision.</li><li><strong>Resource Requirements:</strong> The kube-scheduler considers the resource requests and limit requirements and constraints specified for the pod, such as CPU and memory. It attempts to place pods on nodes that have sufficient available resources to meet these requirements.</li><li><strong>Node Affinity and Anti-affinity:</strong> Node affinity and anti-affinity rules allow users to specify preferences or restrictions for where pods should be scheduled based on node labels and other attributes. The kube-scheduler evaluates these rules when making placement decisions.</li><li><strong>Node Taints and Tolerations:</strong> Nodes can be “tainted” with specific characteristics that pods must “tolerate” to be scheduled on them. The kube-scheduler ensures that pods with matching tolerations are placed on suitable nodes.</li><li><strong>Pod Priority and Preemption:</strong> Pods can be assigned priorities, and the kube-scheduler considers these priorities when making scheduling decisions. It will try to schedule higher-priority pods first and may preempt lower-priority pods if necessary to make room for higher-priority ones.</li><li><strong>Continuous Monitoring:</strong> The kube-scheduler continuously monitors the cluster’s state, including node health and resource availability. If a node becomes unavailable or if the resource constraints change, it can reschedule pods to ensure their continued operation.</li></ol><p>The kube-scheduler strives to evenly distribute pods across available nodes to balance the workload and prevent resource contention. It avoids packing pods onto a single node if other nodes have capacity. Although kube-scheduler is the default scheduler in Kubernetes, it is possible to configure and use custom schedulers that tailor scheduling decisions to specific requirements or policies.</p><p>The kube-scheduler is extensible through the use of scheduling plugins, which can implement custom logic for node selection based on specific criteria. Multiple schedulers can also be configured in a cluster and can be assigned different workloads based on labels or other meta identifiers. Once the kube-scheduler determines the best node for a pod, it updates the pod’s status and binds it to the chosen node. The pod then gets scheduled and runs on that node. An example is the volcano scheduler, most commonly used to managing ML workloads.</p><ul><li><a href="https://github.com/kubernetes/kube-scheduler">GitHub - kubernetes/kube-scheduler: kube-scheduler component configs</a></li><li><a href="https://kubernetes.io/docs/tasks/extend-kubernetes/configure-multiple-schedulers/">Configure Multiple Schedulers</a></li><li><a href="https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/">Resource Management for Pods and Containers</a></li><li><a href="https://github.com/volcano-sh/volcano">GitHub - volcano-sh/volcano: A Cloud Native Batch System (Project under CNCF)</a></li></ul><h3>kube controller manager</h3><p>The kube controller manager is responsible for maintaining the desired state of various resources and controllers within the Kubernetes cluster. It manages several built-in controllers, each designed to ensure that specific resources and their corresponding desired states are maintained within the cluster. There are various different controllers in Kubernetes and each controller is responsible for a specific aspect of the cluster’s functionality. Some of the common controllers managed by kube controller manager include:</p><ol><li><strong>Node Controller:</strong> This controller is responsible for monitoring the state of worker nodes in the cluster. It detects when nodes become unreachable and marks them as unhealthy. It also manages the addition and removal of nodes from the cluster.</li><li><strong>Replication / ReplicaSet Controller:</strong> These controllers ensure that the specified number of pod replicas are running and maintained according to the desired configuration. They create or delete pods as needed to meet the desired replica count.</li><li><strong>Namespace Controller:</strong> The Namespace controller is responsible for creating and deleting namespaces based on user requests. It ensures that namespace resources are created or cleaned up as needed.</li><li><strong>DaemonSet Controller:</strong> The DaemonSet controller ensures that a specified set of pods, called daemons, runs on every node in the cluster. It is often used for cluster-level add-ons and system agents.</li><li><strong>StatefulSet Controller:</strong> The StatefulSet controller manages stateful applications by ensuring that pods are consistently and sequentially created, updated, or deleted. It is commonly used for databases and other stateful workloads.</li><li><strong>Job and CronJob Controller:</strong> These controllers manage batch workloads. The Job controller ensures that a specified number of pod instances complete successfully, while the CronJob controller schedules jobs at specified intervals using cron-like expressions.</li><li><strong>Persistent Volume (PV) Controller and Persistent Volume Claim (PVC) Controller:</strong> These controllers manage the lifecycle of persistent volumes and persistent volume claims. They ensure that claims are bound to available volumes and that volumes are provisioned and released as needed.</li><li><strong>Endpoints Controller:</strong> The Endpoints controller populates the Endpoints resource with the addresses of all pods that match a particular Service. This allows network traffic to be directed to the appropriate pods behind a Service.</li><li><strong>Service Account and Token Controller:</strong> These controllers manage the lifecycle of service accounts and the associated API tokens used for authentication and authorization of pods.</li></ol><p>Kubernetes also allows users to create custom controllers to manage custom resources and implement custom behavior. The kube controller manager supports the registration of custom controllers through the Kubernetes API. Each controller operates on a reconciliation loop, continuously comparing the desired state (as declared by users or defined in configuration) with the observed state in the cluster. If any discrepancies are found, the controller takes corrective actions to bring the cluster state in line with the desired state.</p><ul><li><a href="https://github.com/kubernetes/kube-controller-manager">GitHub - kubernetes/kube-controller-manager: kube-controller-manager component configs</a></li><li><a href="https://kubernetes.io/docs/concepts/architecture/controller/">Controllers</a></li><li><a href="https://kubernetes.io/docs/reference/command-line-tools-reference/kube-controller-manager/">kube-controller-manager</a></li></ul><h3>cloud-controller-manager</h3><p><a href="https://kubernetes.io/docs/concepts/architecture/cloud-controller/">Cloud Controller Manager</a> (CCM) interacts with cloud provider APIs and services to manage resources and functionality that are specific to the underlying cloud infrastructure. Kubernetes is designed to be agnostic to the underlying cloud infrastructure. However, cloud-specific operations require knowledge of the specific cloud provider’s APIs and services. The CCM abstracts these cloud-specific operations from the core Kubernetes code. Here’s an explanation of the Cloud Controller Manager’s role and purpose in Kubernetes:</p><ol><li><strong>Cloud Resource Management:</strong> The CCM manages various cloud resources and services, such as:</li><li><strong>Load Balancers:</strong> It creates, configures, and manages cloud-specific load balancers that distribute traffic to Kubernetes service endpoints.</li><li><strong>Block Storage:</strong> It provisions and manages block storage volumes for Persistent Volume Claims (PVCs) in the cluster, integrating with cloud-specific storage services.</li><li><strong>Node Management:</strong> The CCM may interact with cloud provider APIs to perform node-related operations, such as node scaling and instance management.</li><li><strong>Network Configuration:</strong> It manages network configurations, including virtual networks, subnets, and routes, to ensure connectivity and isolation within the cluster.</li><li><strong>Customization and Optimization:</strong> The CCM allows cloud-specific optimizations and customization for Kubernetes clusters. It can tailor operations to take advantage of cloud provider features and optimize performance and resource utilization.</li><li><strong>Extensibility:</strong> The CCM architecture is extensible, allowing cloud providers to develop and maintain their own CCM implementations. This enables support for various cloud platforms and APIs, including AWS, Azure, Google Cloud, and others.</li><li><strong>Cloud-Specific Authentication:</strong> The CCM handles cloud-specific authentication and credential management, ensuring that Kubernetes components can securely access cloud provider resources and services. This is usually managed in the form of service accounts talking with access management api of the underlying cloud provider.</li><li><strong>Node Termination Handling:</strong> When nodes are terminated or scaled down in a cloud environment, the CCM may be responsible for performing necessary cleanup actions, such as detaching storage volumes and releasing cloud resources.</li></ol><p>By separating cloud-specific code into the CCM, Kubernetes maintains a cleaner and more modular codebase. It allows the core Kubernetes code to focus on container orchestration and cluster management, while the CCM handles interactions with the cloud provider. Cloud Controller Manager bridges the gap between Kubernetes and cloud providers by abstracting and managing cloud-specific operations. On managed cloud providers, this means improving portability, flexibility, and maintainability of Kubernetes clusters across while enabling the utilization of cloud-native features and resources.</p><ul><li><a href="https://kubernetes.io/docs/concepts/architecture/cloud-controller/">Cloud Controller Manager</a></li><li><a href="https://github.com/kubernetes/cloud-provider">GitHub - kubernetes/cloud-provider: cloud-provider defines the shared interfaces which Kubernetes cloud providers implement. These interfaces allow various controllers to integrate with any cloud provider in a pluggable fashion. Also serves as an issue tracker for SIG Cloud Provider.</a></li><li><a href="https://github.com/kubernetes/cloud-provider-aws">GitHub - kubernetes/cloud-provider-aws</a></li><li><a href="https://github.com/kubernetes/community/tree/master/sig-cloud-provider">community/sig-cloud-provider at master · kubernetes/community</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cf766178a0ac" width="1" height="1" alt=""><hr><p><a href="https://itnext.io/kubernetes-iii-composition-of-a-kubernetes-node-i-cf766178a0ac">Kubernetes III — Composition of a Kubernetes node — I</a> was originally published in <a href="https://itnext.io/">ITNEXT</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kubernetes — II: Deploying applications on dedicated nodes using terraform & terragrunt]]></title>
            <link>https://itnext.io/kubernetes-ii-deploying-applications-on-dedicated-nodes-using-terraform-terragrunt-9fef9ea96d8d?source=rss-3914e337751e------2</link>
            <guid isPermaLink="false">https://medium.com/p/9fef9ea96d8d</guid>
            <category><![CDATA[helm]]></category>
            <category><![CDATA[aws-eks]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[terragrunt]]></category>
            <dc:creator><![CDATA[Ali Abbas Jaffri]]></dc:creator>
            <pubDate>Tue, 29 Nov 2022 23:57:25 GMT</pubDate>
            <atom:updated>2023-10-08T13:00:13.473Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JLniprC-2MPZ8doVDXkuKA.png" /><figcaption>image credits: <a href="https://www.macstadium.com/blog/how-to-k8s-getting-started-with-helm-charts">macstadium</a></figcaption></figure><h3>Kubernetes II: Deploying applications on dedicated nodes using terraform &amp; terragrunt</h3><p>In part I of this series, we went through different architectural patterns of Kubernetes cluster and created a multi-node Kubernetes cluster on EKS using terraform &amp; terragrunt. In part II, we would be exploring how to deterministically deploy helm charts on Kubernetes for application lifecycle management using terraform &amp; terragrunt.</p><p><a href="https://itnext.io/kubernetes-i-multi-node-deployment-using-terraform-and-terragrunt-30c40a1238e8">Kubernetes — I: multi-node deployment using terraform and terragrunt</a></p><p>Operations applications, or OpsApps, are packages that we schedule atop Kubernetes to set up the base for our applications to run. These packages augment the capabilities of the applications and allow users make the most of the services being provided to them. Such packages are installed on Kubernetes using a popular and widely used package manager called the Helm and the packages are referred to as Helm Charts (packages on Helm are <a href="https://helm.sh/docs/topics/charts/">packaged in a special format called <em>charts</em></a>, a collection of Kubernetes resources).</p><p><a href="https://helm.sh/">Helm</a></p><p>Helm makes it easy to deploy application packages on top of Kubernetes. You can create a helm deployment for any application; think of its as an application packaging tool for deployment on Kubernetes. These packages usually come with a massive list of configurations, usually in the form of a yaml file called values.yaml. This file provides a deterministic deployment environment for your OpsApps, as this file is checked into git after its configurations have been applied. We will see near the end of this post how to automate the deployment of a Helm chart using terraform.</p><p>Helm charts form the foundation of a solid, and efficient functioning structure for a microservice-oriented architecture running on Kubernetes. Most notable of these charts are ingress-nginx, hashicorp vault, cert-manager, rabbitmq and linkerd to name a few. Each chart specializes in its specific functionality and allows application developers to focus on developing the business logic of their application rather than creating the supporting infrastructure to function at scale.</p><p>Let’s understand the example of <a href="https://kubernetes.github.io/ingress-nginx">ingress-nginx</a> chart. A controller in Kubernetes tracks at least a single resource type and makes sure to <a href="https://kubernetes.io/docs/concepts/architecture/controller">bring the cluster closer to the desired state</a>. An <a href="https://kubernetes.io/docs/concepts/services-networking/ingress/">ingress controller</a>, such as ingress-nginx, exposes http and https routes from outside the Kubernetes cluster and acts as a load balancer to the services that request data within the cluster. Using this helm chart, developers can leverage load management of services, TLS/SSL termination of public endpoints as well as limited caching services, rather than having to set up all of these services on their own.</p><ul><li><a href="https://kubernetes.github.io/ingress-nginx/">Welcome - Ingress-Nginx Controller</a></li><li><a href="https://kubernetes.io/docs/concepts/architecture/controller/">Controllers</a></li></ul><p>Helm chart can be configured on a Kubernetes cluster using a cli tool, that points to the <a href="https://kubernetes.io/docs/tasks/access-application-cluster/configure-access-multiple-clusters/">current context</a> in the kubeconfig in the .kubeconfig folder. That is the easiest-to-use method, but it might not be a perfect fit for applying changes on multiple clusters or clusters which are in production. Maintainers would need to track the configuration of different Helm charts that go into their Kubernetes cluster as well as compare any new changes that they would want to introduce into the system. This is where terraform &amp; terragrunt come into the picture. With the help of <a href="https://registry.terraform.io/providers/hashicorp/helm/latest/docs">Helm provider from Hashicorp</a>, users can make deterministic and reproducible changes to their clusters. The only requirement is to download the values file for the Helm chart to correctly configure and apply the chart. This would take care of the first half of the problem; to install OpsApps on the Kubernetes cluster, but we still need to figure out how to schedule these applications on top of node groups that are dedicated to these OpsApps. That is because these are operation-critical applications and need to have dedicated access to resources, such as CPU, RAM, disk space, etc.</p><p><a href="https://registry.terraform.io/providers/hashicorp/helm/latest/docs">Terraform Registry</a></p><p>This segregation of application execution on different Kubernetes node groups is managed by built-in tools such as NodeSelector, Taints &amp; Tolerations, and Node Affinity &amp; Anti-Affinity mechanisms. These techniques make OpsApps application pods get attached to specific nodes and repel other applications from that node group. This is done by creating labels and attaching them on nodes and applications that help Kubernetes identify which applications are to be scheduled on which nodes. Let’s briefly see how these mechanisms allow Kubernetes to achieve that.</p><ul><li>NodeSelector: Node Selector works quite like Node Affinity, with the difference that it matches directly against node labels which are defined during node creation.</li><li>NodeAffinity and anti-affinity: Node affinity and anti-affinity allow you to highly customize the rules and logic for pod scheduling. It is a lot more expressive than NodeSelector and allows to schedule pods on nodes that match a certain set of rules or constraints.</li><li>Taints &amp; Tolerations: <em>Node affinity</em> is a property of Pods that attracts them to a set of nodes (either as a preference or a hard requirement). <em>Taints</em> are the opposite; its a property of nodes and allows a node to repel a set of pods. Tolerations are applied to pods, and allow (but do not require) the pods to schedule onto nodes with matching taints.</li></ul><p><a href="https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/">Assigning Pods to Nodes</a></p><p>Continuing on from where i last left on my first post of the series on Kubernetes, i’d be adding three Helm charts to the Kubernetes cluster and scheduling them on respective OpsApps node group. I took an alternative strategy, and refactored the code to make it concise, easy to understand by leverage the power of modules in terraform. I utilised the terragrunt-aws-eks module from <a href="https://github.com/antonbabenko">Anton Babenko</a>, in order to create the eks cluster, node groups and IRSA for eks. IRSA, or IAM Roles for Service Accounts, is a feature of AWS which allows you to make use of IAM roles at the pod level by combining an <a href="https://openid.net/connect/">OpenID Connect</a> (OIDC) identity provider and Kubernetes service account annotations.</p><ul><li><a href="https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html">IAM roles for service accounts</a></li><li><a href="https://github.com/terraform-aws-modules/terraform-aws-eks">GitHub - terraform-aws-modules/terraform-aws-eks: Terraform module to create Amazon Elastic Kubernetes (EKS) resources 🇺🇦</a></li></ul><p>Lets start with creating an eks cluster using this module.</p><pre>################################################################################<br># Supporting Resources<br>################################################################################<br><br>resource &quot;aws_kms_key&quot; &quot;eks&quot; {<br>  description             = var.aws_kms_key.description<br>  deletion_window_in_days = var.aws_kms_key.deletion_window_in_days<br>  enable_key_rotation     = var.aws_kms_key.enable_key_rotation<br>  tags                    = var.aws_kms_key.tags<br>}<br><br>resource &quot;aws_security_group&quot; &quot;additional_security_group&quot; {<br>  name_prefix = var.aws_security_group.name_prefix<br>  vpc_id      = var.aws_security_group.vpc_id<br><br>  dynamic &quot;ingress&quot; {<br>    for_each = var.aws_security_group.ingresses<br>    content {<br>      from_port   = ingress.value.from_port<br>      to_port     = ingress.value.to_port<br>      protocol    = ingress.value.protocol<br>      cidr_blocks = ingress.value.cidr_blocks<br>    }<br>  }<br>  tags = var.aws_security_group.tags<br>}<br><br>################################################################################<br># Supporting Resources<br>################################################################################<br><br><br>################################################################################<br># EKS Module<br>################################################################################<br><br>module &quot;eks&quot; {<br>  source  = &quot;terraform-aws-modules/eks/aws&quot;<br>  version = &quot;18.30.3&quot;<br><br>  cluster_name                    = var.aws_eks_cluster.cluster_name<br>  cluster_version                 = var.aws_eks_cluster.cluster_version<br>  cluster_endpoint_private_access = var.aws_eks_cluster.cluster_endpoint_private_access<br>  cluster_endpoint_public_access  = var.aws_eks_cluster.cluster_endpoint_public_access<br><br>  cluster_addons = {<br>    coredns = {<br>      resolve_conflicts = &quot;OVERWRITE&quot;<br>    }<br>    kube-proxy = {}<br>    # vpc-cni = {<br>    #   resolve_conflicts        = &quot;OVERWRITE&quot;<br>    #   service_account_role_arn = module.vpc_cni_irsa.iam_role_arn<br>    # }<br>  }<br><br>  cluster_encryption_config = [{<br>    provider_key_arn = aws_kms_key.eks.arn<br>    resources        = [&quot;secrets&quot;]<br>  }]<br><br>  vpc_id     = var.aws_eks_cluster.vpc_id<br>  subnet_ids = var.aws_eks_cluster.subnets<br><br>  # Extend cluster security group rules<br>  cluster_security_group_additional_rules = var.aws_eks_cluster.cluster_security_group_additional_rules<br><br>  # Extend node-to-node security group rules<br>  node_security_group_additional_rules = var.aws_eks_cluster.node_security_group_additional_rules<br><br>  # eks_managed_node_group_defaults = {<br>  #   iam_role_attach_cni_policy = true<br>  # }<br><br>  eks_managed_node_groups = var.aws_eks_cluster.eks_managed_node_groups<br><br>  tags = var.aws_eks_cluster.tags<br>}<br><br>################################################################################<br># EKS Module<br>################################################################################<br><br>module &quot;vpc_cni_irsa&quot; {<br>  source  = &quot;terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks&quot;<br>  version = &quot;~&gt; 4.12&quot;<br><br>  role_name_prefix      = var.vpc_cni_irsa.role_name_prefix<br>  attach_vpc_cni_policy = var.vpc_cni_irsa.attach_vpc_cni_policy<br>  vpc_cni_enable_ipv4   = var.vpc_cni_irsa.vpc_cni_enable_ipv4<br><br>  oidc_providers = {<br>    main = {<br>      provider_arn               = module.eks.oidc_provider_arn<br>      namespace_service_accounts = [&quot;kube-system:aws-node&quot;]<br>    }<br>  }<br><br>  tags = var.aws_eks_cluster.tags<br>}</pre><p>The respective terragrunt configuration for these would be follows. Notice the labels and taints in the eks_managed_node_groups block.</p><pre>inputs = {<br>  aws_kms_key = {<br>    description             = &quot;EKS Secret Encryption Key&quot;<br>    deletion_window_in_days = 7<br>    enable_key_rotation     = true<br>    tags                    = local.tags<br>  }<br><br>  aws_security_group = {<br>    name_prefix = &quot;eks-cluster-additional-sg&quot;<br>    vpc_id      = dependency.vpc.outputs.vpc_id<br>    ingresses = [{<br>      from_port = 22<br>      to_port   = 22<br>      protocol  = &quot;tcp&quot;<br>      cidr_blocks = [<br>        &quot;10.0.0.0/8&quot;,<br>        &quot;172.16.0.0/12&quot;,<br>        &quot;192.168.0.0/16&quot;,<br>      ]<br>    }]<br>    tags = local.tags<br>  }<br><br>  aws_eks_cluster = {<br>    cluster_name                    = &quot;eks-cluster&quot;<br>    cluster_version                 = &quot;1.24&quot;<br>    cluster_endpoint_private_access = true<br>    cluster_endpoint_public_access  = true<br>    vpc_id                          = dependency.vpc.outputs.vpc_id<br>    subnets                         = dependency.vpc.outputs.vpc_public_subnets_ids<br>    cluster_security_group_additional_rules = {<br>      egress_nodes_ephemeral_ports_tcp = {<br>        description                = &quot;To node 1025-65535&quot;<br>        protocol                   = &quot;tcp&quot;<br>        from_port                  = 1025<br>        to_port                    = 65535<br>        type                       = &quot;egress&quot;<br>        source_node_security_group = true<br>      }<br>    }<br>    node_security_group_additional_rules = {<br>      ingress_self_all = {<br>        description = &quot;Node to node all ports/protocols&quot;<br>        protocol    = &quot;-1&quot;<br>        from_port   = 0<br>        to_port     = 0<br>        type        = &quot;ingress&quot;<br>        self        = true<br>      }<br>      egress_all = {<br>        description      = &quot;Node all egress&quot;<br>        protocol         = &quot;-1&quot;<br>        from_port        = 0<br>        to_port          = 0<br>        type             = &quot;egress&quot;<br>        cidr_blocks      = [&quot;0.0.0.0/0&quot;]<br>        ipv6_cidr_blocks = [&quot;::/0&quot;]<br>      }<br>    }<br>    eks_managed_node_groups = {<br>      default_node_group_1 = {<br>        create_launch_template = false<br>        launch_template_name   = &quot;&quot;<br><br>        disk_size = 50<br><br>        min_size     = 1<br>        max_size     = 7<br>        desired_size = 1<br><br>        capacity_type        = &quot;SPOT&quot;<br>        force_update_version = true<br>        instance_types       = [&quot;t3.small&quot;]<br>        taints               = []<br>      }<br>      default_node_group_2 = {<br>        create_launch_template = false<br>        launch_template_name   = &quot;&quot;<br><br>        disk_size = 50<br><br>        min_size     = 1<br>        max_size     = 7<br>        desired_size = 1<br><br>        capacity_type        = &quot;SPOT&quot;<br>        force_update_version = true<br>        instance_types       = [&quot;t3.small&quot;]<br><br>        labels = {<br>          NodeTypeClass = &quot;appops&quot;<br>        }<br><br>        taints = [{<br>          key    = &quot;dedicated&quot;<br>          value  = &quot;appops&quot;<br>          effect = &quot;NO_SCHEDULE&quot;<br>          }<br>        ]<br>      }<br>    }<br>    tags = local.tags<br>  }<br>  vpc_cni_irsa = {<br>    role_name_prefix      = &quot;VPC-CNI-IRSA&quot;<br>    attach_vpc_cni_policy = true<br>    vpc_cni_enable_ipv4   = true<br>    tags                  = local.tags<br>  }<br>}</pre><p>Create a module for Helm release as it will be reused multiple times</p><pre>resource &quot;helm_release&quot; &quot;helm_chart&quot; {<br>  name             = var.helm_chart.name<br>  namespace        = var.helm_chart.namespace<br>  create_namespace = var.helm_chart.create_namespace<br>  repository       = var.helm_chart.repository<br>  chart            = var.helm_chart.chart<br>  version          = var.helm_chart.chart_version<br>  cleanup_on_fail  = true<br>  values           = [var.helm_chart.values]<br><br>  dynamic &quot;set&quot; {<br>    for_each = var.helm_chart.set<br>    content {<br>      name  = set.value.name<br>      value = set.value.value<br>      type  = set.value.type<br>    }<br>  }<br>}</pre><p>Add the label in NodeSelector in the values.yaml of Helm charts and set Node Affinity accordingly. I have been wanting to add complex hierarchical types to set function, but it seems to be an impossible task with very limited support from terragrunt. I decided to better resort to updating the values file directly for this task.</p><pre># Note the labels and taints in the eks_managed_node_groups block<br>labels = {<br>  NodeTypeClass = &quot;appops&quot;<br>}<br><br>taints = [{<br>  key    = &quot;dedicated&quot;<br>  value  = &quot;appops&quot;<br>  effect = &quot;NO_SCHEDULE&quot;<br>  }<br>]</pre><p>Because of our configured node labels and taints, we would have the following configuration for our Helm charts:</p><pre># node selector<br>nodeSelector:<br>  kubernetes.io/os: linux<br>  NodeTypeClass: appops<br><br><br>## <br># tolerations<br>tolerations:<br>  - key: &quot;dedicated&quot;<br>    operator: &quot;Equal&quot;<br>    value: &quot;appops&quot;<br>    effect: &quot;NoSchedule&quot;<br><br>##<br># node affinity<br>affinity:<br>  nodeAffinity:<br>    requiredDuringSchedulingIgnoredDuringExecution:<br>      nodeSelectorTerms:<br>        - matchExpressions:<br>            - key: NodeTypeClass<br>              operator: In<br>              values:<br>                - appops</pre><p>Best part about using terragrunt is that you can execute terragrunt run-all apply in the root directory and go enjoy a coffee and by the time you’re back, all the infrastructure is done getting deployed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fMfBoU7FFyOE-e2qiETwBA.png" /><figcaption>Eureka, all pods areup and healthy! Note how all the opsapps, ingress-nginx and cert-manager are scheduled on the same node.</figcaption></figure><p>I could see my OpsApps pods getting scheduled on the respective nodes. But i really wanted to see autoscaling in action, so i set the instance type of my node from t3.small to t3.micro. When i created the cluster again with that configuration, i could see that the nodes got created, but the pods are running into the following error</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Pp2g-u8JtQAF-tax_ysHMw.png" /><figcaption>FailedScheduling — 5 minutes — default-scheduler0/3 nodes are available: 1 Too many pods, 1 node(s) had untolerated taint {node.kubernetes.io/not-ready: }, 1 node(s) had untolerated taint {node.kubernetes.io/unschedulable: }, 1 node(s) were unschedulable, 2 node(s) didn’t match Pod’s node affinity/selector. preemption: 0/3 nodes are available: 1 No preemption victims found for incoming pod, 2 Preemption is not helpful for scheduling.</figcaption></figure><p>I followed all the tricks in the book, but it seemed like i definitely missed something because my nodes were not autoscaling as expected. It was then when i figured out that i managed to create only the node groups, but there needs to be a signal from within the cluster to the control plane that the capacity requirements are not being matched and more resources are required to cater this. We already created Autoscaling Group as a part of cluster creation and we just needed to install a component that automatically adjusts the size of the cluster; this component was <a href="https://github.com/kubernetes/autoscaler">cluster-autoscaler</a>. Great part was that it was available as a Helm chart and i just needed to use the same Helm release component that i had already created.</p><ul><li><a href="https://aws.github.io/aws-eks-best-practices/cluster-autoscaling/">Cluster-Autoscaler - EKS Best Practices Guides</a></li><li><a href="https://docs.aws.amazon.com/eks/latest/userguide/autoscaling.html">Scale cluster compute with Karpenter and Cluster Autoscaler</a></li></ul><p>Once done, lets apply these changes, and see if our Helm charts are scheduled on our respective nodes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NIV-9OXE6Yh_vFVuBQ12GQ.png" /><figcaption>More nodes this time. It shows that our nodes did get created of the instance types t3.micro</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qASbOTdk647T96KTKNXsfg.png" /><figcaption>Sneak peak to EKS cluster validates that.</figcaption></figure><p><em>PS. I would really appreciate a star on the github repo and sharing this post if you find this useful, as it allows me to create more relevant content in future.</em></p><p><a href="https://github.com/aliabbasjaffri/multinode-terraform-terragrunt">GitHub - aliabbasjaffri/multinode-terraform-terragrunt: Repository for multi-node terraform code with terragrunt on AWS cloud</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9fef9ea96d8d" width="1" height="1" alt=""><hr><p><a href="https://itnext.io/kubernetes-ii-deploying-applications-on-dedicated-nodes-using-terraform-terragrunt-9fef9ea96d8d">Kubernetes — II: Deploying applications on dedicated nodes using terraform &amp; terragrunt</a> was originally published in <a href="https://itnext.io/">ITNEXT</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kubernetes — I: multi-node deployment using terraform and terragrunt]]></title>
            <link>https://itnext.io/kubernetes-i-multi-node-deployment-using-terraform-and-terragrunt-30c40a1238e8?source=rss-3914e337751e------2</link>
            <guid isPermaLink="false">https://medium.com/p/30c40a1238e8</guid>
            <category><![CDATA[kubernetes-cluster]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[terragrunt]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[high-availability]]></category>
            <dc:creator><![CDATA[Ali Abbas Jaffri]]></dc:creator>
            <pubDate>Tue, 25 Oct 2022 06:35:36 GMT</pubDate>
            <atom:updated>2023-10-08T13:00:46.850Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-a7LLjP06CgISFfzIoF3Sg.png" /></figure><h3>Kubernetes I: multi-node deployment using terraform and terragrunt</h3><p>While gathering ideas and information for this post, I realized there is so much to the world of Kubernetes that is not commonly known. I decided that instead of writing just a single post, I will be writing a multi-part series on Kubernetes, educating you all regarding all that I know and there is to know about Kubernetes in a structured manner. It took me a while to organize all my knowledge and experiences while writing this post to explain Kubernetes from an ordinary perspective. Some blog series, such as ones from <a href="https://iximiuz.com/en/categories/?category=Kubernetes">Ivan Velichko</a> that I religiously follow, tend to build up a basic understanding of Kubernetes and its intricacies. However, there is a general lack of understanding regarding how the components, apart from the <a href="https://kubernetes.io/docs/concepts/overview/components/">architectural components</a>, logically stack up on top of each other such as the components of the Kubernetes-API, custom resource definitions, operators, CNI, and so on. I will be writing about those concepts in upcoming posts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Uk1p4lh9SiXOlv3QvVHrBg.png" /><figcaption>Architectural components of Kubetnetes. source: kubernetes.io</figcaption></figure><p><a href="https://kubernetes.io/docs/concepts/overview/components/">Kubernetes Components</a></p><p>In my previous roles, I had the opportunity to deploy Kubernetes from scratch at various scales. These roles and experiences made me categorize Kubernetes cluster distros into essentially two basic types:</p><ul><li>User-managed open-source Kubernetes clusters, such as <a href="https://www.rancher.com/">Rancher</a>, <a href="https://kind.sigs.k8s.io/">kind</a> or <a href="https://www.redhat.com/en/technologies/cloud-computing/openshift">OpenShift</a>: these Kubernetes clusters are usually available as both open-source and maintained solutions from their providers. Users can deploy variations of architectures suitable for their use cases on their underlying infrastructure or rely on the <a href="https://www.suse.com/products/suse-rancher-hosted/">managed services from the maintainers</a>.</li><li>Cloud-managed Kubernetes clusters, such as GKE, EKS, and AKS: these are the publicly available, easy-to-set-up single-click deployment variants of Kubernetes that are maintained by the cloud providers. Management of the control plane, its scalability, and security is solely the responsibility of the cloud providers whereas that of the data plane is the responsibility of the users.</li></ul><ul><li><a href="https://cloud.google.com/kubernetes-engine">Google Kubernetes Engine (GKE)</a></li><li><a href="https://aws.amazon.com/eks/">Amazon EKS Customers | Managed Kubernetes Service | Amazon Web Services</a></li><li><a href="https://azure.microsoft.com/en-us/products/kubernetes-service/#overview">Managed Kubernetes Service (AKS) | Microsoft Azure</a></li></ul><p>The main difference between these two types of cluster distros is that in the user-managed clusters, the user manages the security, redundancy, and scalability of the control plane, as well as that of the data plane and all the components that exist in between. This is in comparison with cloud-managed clusters, where the scalability, redundancy, and security of the control plane are managed by the cloud service providers and the user manages the affairs of the data plane. Setting up a user-managed cluster effectively is a demanding task, and from experience, it takes a lot of time and effort to manage all the loose ends together. These can be in the form of <a href="https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/">managing Kubernetes version upgrades</a>, <a href="https://docs.ranchermanager.rancher.io/how-to-guides/new-user-guides/kubernetes-clusters-in-rancher-setup/launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/vsphere/create-a-vm-template">correct configuration of underlying VMs</a>, <a href="https://docs.ranchermanager.rancher.io/how-to-guides/new-user-guides/backup-restore-and-disaster-recovery/back-up-rancher-launched-kubernetes-clusters">safely snap shotting a running cluster</a>, <a href="https://etcd.io/docs/v3.3/op-guide/recovery/">restoration of etcd database</a>, <a href="https://docs.ranchermanager.rancher.io/how-to-guides/new-user-guides/kubernetes-resources-setup/encrypt-http-communication">encryption of communication between nodes</a> to name a few. However, setting up a user-managed Kubernetes cluster from scratch is a tremendous learning experience and the journey teaches a lot of “patience” and respect for all the managed Kubernetes providers. Setting up a cloud-managed Kubernetes cluster is rather simple and requires only a few button clicks to provision. However, depending on the use case, it can be a bit tricky how to set up your managed cluster effectively to address scalability and security as you grow.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7qPWiTB3aKaRuMlvnnLrIQ.png" /><figcaption>Kubernetes and all the different corners of its abstractions! credits: <a href="https://www.pionative.com/">https://www.pionative.com/</a></figcaption></figure><p>There are different deployment patterns for Kubernetes to address the scalability and redundancy needs of its users. These are:</p><ul><li>Single instance deployment: such as <a href="https://docs.ranchermanager.rancher.io/pages-for-subheaders/rancher-on-a-single-node-with-docker">Rancher docker deployment</a> or <a href="https://kind.sigs.k8s.io/">kind</a> clusters. In this deployment, Kubernetes is installed on top of docker which is installed on bare metal or similar compute instance. It is usually good for learning, local development, and debugging scenarios, but not recommended for production use cases due to scalability and redundancy limitations.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3tWHZpBcrNlxEp_rYK2rkg.png" /><figcaption>credits: kind.sigs.k8s.io</figcaption></figure><ul><li>Multi-worker-nodes deployment: In this deployment, there are multiple worker nodes or node groups that are communicating with a single instance control plane. The worker nodes take tasks and other responsibilities from the control plane and can be set to scale in or out depending on requirements. But if the number of worker nodes grow rapidly, then there can be bottlenecking and/or latency problems, as a single control plane can come under throttle or may not be available.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/752/1*Kx801wGAQ5IpRzPecP6zSw.png" /><figcaption>multi-node deployment on Kubernetes. Worker nodes can range from 1 to n, depending upon your requirements.</figcaption></figure><ul><li>Multi-worker-nodes HA deployment: HA is more commonly known as High Availability deployment. In this deployment, multiple redundant control planes run in quorum with each other and may or may not have multiple instances of the <a href="https://kubernetes.io/docs/concepts/overview/components/#etcd">etcd database</a> running in a HA cluster format in parallel. Each control plane is usually deployed to a distinct Availability Zone and this setup ensures that in cases of downtime in one or more Availability Zones, one instance is guaranteed availability. This set up is common for multi-node groups that run similar or related applications or workloads. This configuration is useful when you have specific application groups that you want to schedule or bundle together on Kubernetes along with their resources and is supported by all types of Kubernetes clusters. We will discuss more on this later in this post.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3pPyfzz0P_7PG9Mqx7Du6Q.png" /><figcaption>HA Cluster with multinode strategy. Note the individual etcd clusters which <a href="https://learnk8s.io/etcd-kubernetes">share state amongst each other</a> credits: kubernetes.io</figcaption></figure><ul><li>HA redundant control plane deployment with external etcd nodes and multi-worker-nodes deployment: In this mode, your Kubernetes cluster is on steroids! This is probably the most redundant and highly available form of Kubernetes cluster and is entirely distributed in nature regarding all of its important working components. It is also the recommended way of running Kubernetes in production environments.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TlJYo4NUwtGPxmMhxzQW4Q.png" /><figcaption>credits: kubernetes.io</figcaption></figure><p><a href="https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/ha-topology/">Options for Highly Available Topology</a></p><p>I previously had the opportunity to deploy a single instance docker deployment of Kubernetes, which we used primarily for our dev environment, as well as high availability Kubernetes clusters from Rancher and multi-node deployments of EKS and AKS clusters. I found it way easier to manage the clusters and add components to them if the resources were defined in a declarative manner, using IaC. I have used terraform and terragrunt, together, in the past and have also wrote previously about how to best structure your projects with them. The deployment of multi-node clusters via terraform and terragrunt has made the process trivial and easier to understand.</p><p><a href="https://itnext.io/structuring-terraform-project-using-terragrunt-part-i-4c6e936c4858">Structuring terraform project using terragrunt — Part I</a></p><p>The reason for discussing these all different types of deployment patterns is to understand why I chose to write about multi-node deployment. Single-node deployments are nice for experimental or learning purposes but have very little real-world practical value and are seldom used in production. As more and more people are moving from the world of on-prem and self-hosted to managed cloud services, they benefit from out-of-the-box high availability and redundancy features of the control plane which are readily available by any new Kubernetes cluster that you create. However, the data plane remains to be the user’s responsibility and it is important to be aware of the different ways in which you can scale, isolate and distribute your workloads.</p><p><a href="https://www.forbes.com/sites/forbestechcouncil/2021/03/12/why-migrate-to-the-cloud-the-basics-benefits-and-real-life-examples/">Council Post: Why Migrate To The Cloud: The Basics, Benefits And Real-Life Examples</a></p><p>It is important to understand the <a href="https://cloud.google.com/blog/products/gcp/exploring-container-security-isolation-at-different-layers-of-the-kubernetes-stack">concept of isolation</a> in the world of Kubernetes. Containerizing applications is the first step in providing an isolated environment for your applications and workloads. Kubernetes provides an initial layer of isolation by running these containers in pods which provide isolation to the application containers running inside it; including the <a href="https://kubernetes.io/docs/concepts/services-networking/network-policies/">NetworkPolicy</a> resources (I have written about such <a href="https://itnext.io/network-policies-in-kubernetes-c983c1b5eee9">network policies</a> in the past and would encourage you to take a look at it and try it out). Related / relevant pods are then weakly isolated and boxed within namespaces in Kubernetes that scope all the required resources (secrets, config maps, roles, bindings, policies, etc.) within themselves which are required by pods to run successfully. A <a href="https://kubernetes.io/docs/concepts/architecture/nodes/">node in Kubernetes</a> can be a physical or a virtual machine that runs pods and typically multiple nodes are running in a Kubernetes cluster, managed by a control plane with a specific strategy to schedule related workloads. This can be further complemented by scheduling those applications on Kubernetes worker nodes (or just nodes when referring to worker nodes in managed clusters) that are purposefully created to host similar applications. Nodes typically form the data plane of a Kubernetes cluster and are responsible for providing resources (CPU, memory, volume, etc.) and runtime (container execution engine such as <a href="https://containerd.io/">containerd</a>) to containerized applications. This strategy ensures that related workloads share the same computational resources and reside on the same underlying node instance/s.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xcfYydxFALYhfU3IRw7ozw.png" /><figcaption>Isolation layers in Kubernetes. credits: cloud.google.com</figcaption></figure><p>You would usually need Kubernetes to run certain base-level application packages that form an important constituent of its operations for your applications to run effectively. These applications are usually installed and configured on Kubernetes with the help of a Kubernetes package manager, such as <a href="https://helm.sh/">Helm</a>, which is by far the most famous and used package manager for Kubernetes. Some examples of these applications which are installed via helm charts are <a href="https://cert-manager.io/">cert-manager</a>, <a href="https://github.com/kubernetes/ingress-nginx">ingress-nginx</a>, <a href="https://linkerd.io/">linkerd</a>, etc. which add tremendous features to your Kubernetes cluster and help your applications achieve their use cases. However, in the view of Kubernetes, these are just pods scheduled on top of it to perform a specific job. We can refer to these applications as operation-critical applications or OpsApps for short. They need to be isolated from other client-facing applications so that they have their dedicated resources and are not stalled by any high resource-consuming client application. This strategy breaks down worker node groups into two base categories:</p><ul><li>OpsApps (Operations Critical Applications) Nodes</li><li>Application Nodes</li></ul><p>We will be creating a highly available Kubernetes cluster, which will be running multiple worker nodes, each with its own purpose and group of applications to schedule. This will provide resource isolation and better management of your applications on your infrastructure. To achieve this, I created a fresh VPC via the famous <a href="https://registry.terraform.io/modules/terraform-aws-modules/vpc/aws/latest">module from Anton Babenkov</a>, which allowed me to configure VPC, create multi-AZ subnets, configure gateways and set up routing tables almost instantly. This saves time and helps set up a decomposable environment.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f902b9a6cc4c56648899731f3b347b56/href">https://medium.com/media/f902b9a6cc4c56648899731f3b347b56/href</a></iframe><p>The next step was to set up <a href="https://docs.aws.amazon.com/eks/latest/userguide/service_IAM_role.html">IAM roles</a>, <a href="https://docs.aws.amazon.com/eks/latest/userguide/security-iam-awsmanpol.html">IAM policies</a>, and <a href="https://aws.github.io/aws-eks-best-practices/security/docs/">security groups</a> for Kubernetes to tighten the security around the cluster. We need to create these different roles as they are responsible for managing scalability and security of our cluster and assign them respective policies.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d074c3819f8e97dcea4affaa35dfdd68/href">https://medium.com/media/d074c3819f8e97dcea4affaa35dfdd68/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ade88539a896b425cee80afcb5f5d057/href">https://medium.com/media/ade88539a896b425cee80afcb5f5d057/href</a></iframe><p>Next, we create security groups and security group rules, to limit the access available to our cluster.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ca8434eb19be5a638bb8aebc59fe4e1f/href">https://medium.com/media/ca8434eb19be5a638bb8aebc59fe4e1f/href</a></iframe><p>The last and final step; creating the Kubernetes cluster, creating nodes, and attaching all the components together.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f9ac60ebaeeb98ec049ba85e91107272/href">https://medium.com/media/f9ac60ebaeeb98ec049ba85e91107272/href</a></iframe><p>After creating the cluster, we use different techniques available from Kubernetes to make OpsApps pods get attached to specific nodes and repel other applications from that node group. These are NodeAffinity, Taints &amp; Tolerations, and NodeSelector mechanisms. We configure the labels and selectors associated with different nodes with helm charts of these applications to schedule them on appropriate nodes.</p><ul><li>NodeSelector: Node Selector works quite like Node Affinity, with the difference that it matches directly against node labels.</li><li>NodeAffinity and anti-affinity: Node affinity and anti-affinity allow you to highly customize the rules and logic for pod scheduling. It is a lot more expressive than NodeSelector and allows to schedule pods on nodes that match a certain set of rules or constraints.</li><li>Taints &amp; Tolerations: <em>Node affinity</em> is a property of Pods that attracts them to a set of nodes (either as a preference or a hard requirement). <em>Taints</em> are the opposite; they allow a node to repel a set of pods. Tolerations are applied to pods, and allow (but do not require) the pods to schedule onto nodes with matching taints.</li></ul><p><a href="https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/">Assigning Pods to Nodes</a></p><p>I’ll be posting an example on how to do that soon!</p><p><em>PS. I would really appreciate a star on the github repo if you find this useful, as it allows me to create more relevant content in future.</em></p><p><a href="https://github.com/aliabbasjaffri/multinode-terraform-terragrunt">GitHub - aliabbasjaffri/multinode-terraform-terragrunt: Repository for multi-node terraform code with terragrunt on AWS cloud</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=30c40a1238e8" width="1" height="1" alt=""><hr><p><a href="https://itnext.io/kubernetes-i-multi-node-deployment-using-terraform-and-terragrunt-30c40a1238e8">Kubernetes — I: multi-node deployment using terraform and terragrunt</a> was originally published in <a href="https://itnext.io/">ITNEXT</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Structuring terraform project using terragrunt — Part II]]></title>
            <link>https://itnext.io/structuring-terraform-project-using-terragrunt-part-ii-ab0dc08b2bc6?source=rss-3914e337751e------2</link>
            <guid isPermaLink="false">https://medium.com/p/ab0dc08b2bc6</guid>
            <category><![CDATA[iac]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[terragrunt]]></category>
            <category><![CDATA[azure]]></category>
            <dc:creator><![CDATA[Ali Abbas Jaffri]]></dc:creator>
            <pubDate>Mon, 12 Sep 2022 14:18:22 GMT</pubDate>
            <atom:updated>2022-09-13T09:08:51.600Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CFxiFLgY4vK3CltkRe8tqg.png" /><figcaption>image credits: <a href="https://github.com/aaratn/terraenv">terraenv</a></figcaption></figure><h3>Structuring terraform project using terragrunt — Part II</h3><p><em>This is the second part of my series on structuring terraform project using terragrunt. You’d be able to follow along my thought process well if you could go through the first post of the series.</em></p><p><a href="https://itnext.io/structuring-terraform-project-using-terragrunt-part-i-4c6e936c4858">Structuring terraform project using terragrunt — Part I</a></p><p>We already talked about structuring small and medium-scale projects using terraform and terragrunt. The setup I already described in the previous post is valid for a single to a handful of services, across multiple deployment environments, but it might get a bit too clunky to manage and scale if more services are packed in there. The approach I took previously was the mono repo approach, where I package all my terraform and terragrunt code in a single repo. Mono repo works nice as a single source of truth and can be quite helpful to understand the intricacies and dependencies of the entire infrastructure, but it adds a significant overhead from a maintenance and extensibility perspective.</p><p><a href="https://www.hashicorp.com/blog/terraform-mono-repo-vs-multi-repo-the-great-debate">Terraform Mono Repo vs. Multi Repo: The Great Debate</a></p><p>The multi-repo approach jumps right in the middle in such cases. The idea of breaking down the mono repo applies similarly to IaC projects as it does to application projects. It reduces the maintenance build-up on a single repository and allows you to configure each project individually. But just like applications, you cannot create multi Repos for IaC from day one; you always start small from the mono repo and take the multi repo path when needed.</p><h4><strong>Mono repo</strong></h4><h4>advantages</h4><ul><li>single source of truth for entire infrastructure</li><li>easier to understand all the services</li></ul><h4>disadvantages</h4><ul><li>a lot of base infrastructure code is not useful for everyone</li><li>it might be a bit hard to navigate around too much code!</li><li>hard to extend and build new features in without breaking anything else</li><li>leads to a single state file, which can be a problem in cases where you get blocked by changes from different parts / versions of the terraform code</li></ul><h4>Multi repo</h4><h4>advantages</h4><ul><li>a good way to abstract away the underlying details, e.g. different layers, from users that do not concern them</li><li>less chance of lower level configuration issues</li><li>easier to version different modules and providers for different requirements; useful when an update on a module or a provider has an impact on other components</li><li>allows development teams to be independent of each other</li></ul><h4>disadvantages</h4><ul><li>learning curve associated with understanding the project structure</li></ul><p><a href="https://adinermie.com/mono-vs-multi-which-repo-structure-is-right-for-you/">Mono vs Multi - Which Repo Structure Is Right For You? - AdinErmie.com</a></p><p>There are of course tradeoffs in all approaches but in my experience, a multi-repo approach was easier to scale and manage compared to remaining with the same mono repo we started with. For an IaC multi-repo approach, a project has to be realized in terms of layers, with each of those layers composing their mono repo for a multi-repo approach. We can use the application example from the previous post, in terms of identifying layers and extracting a mono repo for them. Those layers would culminate in the form of a repo for each:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a7920f9b27ae0e8da9003f258b88d825/href">https://medium.com/media/a7920f9b27ae0e8da9003f258b88d825/href</a></iframe><p>There are a few benefits of breaking down your project structure into layers:</p><ul><li>identification of base components in your infrastructure which would mostly remain static and wouldn’t be altered</li><li>easy import and reference of dependency modules from base modules to modules higher up in the hierarchy without getting the base modules changed</li><li>easy to identify new requirements/bug fixes and limit the blast radius of errors</li></ul><p>We deliberated initially on understanding how to structure our terraform project in a manner that would sustain scalability as the modules start to grow in the repo. This culminated in the form of all modules packaged in one single terraform repo with the infrastructure modules having their folders and the configuration of these modules in subfolders. The idea was to let the modules grow in their mono repo and once the management scale grows, separate components in the form of base layers and adopt a multi-layer approach.</p><p><a href="https://azure.microsoft.com/en-us/">Cloud Computing Services | Microsoft Azure</a></p><p>In terraform repo, a hierarchy for deploying base_infrastructure modules would allow the code structure to remain organized and easy to maintain and debug. We discussed at length a scalable and clean way to make our terraform code reflect the actual state of hierarchy that would exist in the cloud and made it easier for us to configure our terragrunt configuration in the same order:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d4d46aafa0c936b8f7227251f8b6babc/href">https://medium.com/media/d4d46aafa0c936b8f7227251f8b6babc/href</a></iframe><p>The terraform code for base_infrastructure never grew to the proportions that we would have to break it down in multi-repo even though we are running a wide variety of services on Azure. For application_infrastructure code, we had to cater a new repo as the code would be generating new environments for our users so we would want it to be completely distinct from base_infrastructure. A basic example from this setup would be our deployment of Kubernetes for the base_infrastructure. We had a multi-node deployment of Kubernetes to run the applications for our clients and to interact with multiple other services (<a href="https://azure.microsoft.com/en-us/services/storage/blobs/">Azure Blob</a>, <a href="https://azure.microsoft.com/en-us/services/key-vault/#product-overview">Azure KeyVault</a>, <a href="https://azure.microsoft.com/en-us/products/devops/">Azure DevOps</a>, <a href="https://azure.microsoft.com/en-us/products/cosmos-db/">Azure CosmosDb</a>, etc.). For all this to be scalable and maintainable, we had to set up things in a way that some basic connectivity and deployment of Kubernetes with other Azure services is always there, and the rest is provisioned as per requirements basis.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b7d1563d78f2b639438ac05000ef66c1/href">https://medium.com/media/b7d1563d78f2b639438ac05000ef66c1/href</a></iframe><p>The corresponding folder structure for terragrunt project for base_infrastructure would subsequently look like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7e108f219495a50689b5e94012b73e1d/href">https://medium.com/media/7e108f219495a50689b5e94012b73e1d/href</a></iframe><p>A similar setup would be done for the applications where instead of setting up infrastructure components, users would import the existing infrastructure and use it to deploy their applications. The import part can be easily managed via a two-step approach:</p><ul><li>setup data sources module in terraform repo that would import the existing infrastructure in your applications repo</li><li>setup dependency in target application terragrunt to import required data and variables from the data sources</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3356b5e7cd954981bd9268319ce3f669/href">https://medium.com/media/3356b5e7cd954981bd9268319ce3f669/href</a></iframe><p>Setting up a multi repo in terragrunt is a three-step process:</p><ul><li>Setting up a terraform repo with all the terraform code</li><li>Breaking down different ‘layered’ terragrunt components in their repos</li><li>Setting up an import scheme by which importing dependencies from other repos to the existing one</li></ul><p><a href="https://cloudcasts.io/course/terraform/organizing-with-multiple-states">Organizing With Multiple States</a></p><p>The first part is the easiest to set up. Keeping the code and the configuration away from each other allows faster troubleshooting and keeps the areas of concern distinct from each other while keeping the terraform code dry. We configured a git reference scheme in terragrunt which would provide flexibility in referencing terraform code from a specific branch or tagged release. That proved to be very useful during development sprints, hotfixes, and debugging where you can make changes to terraform code and refer to the branch of your changes. This can be established in the following manner for a sample terraform module called ingress-nginx helm-chart installed on Kubernetes:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5941b5d30beea78045f64a38e217b2ae/href">https://medium.com/media/5941b5d30beea78045f64a38e217b2ae/href</a></iframe><p>There could be an even more complicated requirement of <a href="https://cloud.google.com/kubernetes-engine/docs/concepts/types-of-clusters">setting up</a> and <a href="https://helm.sh/">configuring</a> multiple clusters of Kubernetes in multiple regions across the globe or managed data bricks deployment across multiple teams in the same organization. The good thing about the project setup defined above is that it would easily map to a significantly complex infrastructure setup, as described above while making it comfortable for the development team to identify regions where to contribute or debug.</p><ul><li><a href="https://devops.stackexchange.com/questions/14470/terraform-organising-modules-code-for-multiple-projects">Terraform organising modules/code for multiple projects</a></li><li><a href="https://stackoverflow.com/questions/44288602/terraform-multiple-state-files-best-practice-examples">Terraform Multiple State Files Best Practice Examples</a></li></ul><p>I realized it would be nicer with a running example of a base_infrastructure component such as Kubernetes to understand the topic better and value the insight and hours that went into setting up the mentioned structure, which would be part of my future posts. Let me know how you set up infrastructure code for your organizations. It would be interesting to hear how others are doing it and how it could be done better.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ab0dc08b2bc6" width="1" height="1" alt=""><hr><p><a href="https://itnext.io/structuring-terraform-project-using-terragrunt-part-ii-ab0dc08b2bc6">Structuring terraform project using terragrunt — Part II</a> was originally published in <a href="https://itnext.io/">ITNEXT</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[aws-vault to authenticate your aws accounts]]></title>
            <link>https://itnext.io/aws-vault-to-authenticate-your-aws-accounts-696d08446106?source=rss-3914e337751e------2</link>
            <guid isPermaLink="false">https://medium.com/p/696d08446106</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[security]]></category>
            <category><![CDATA[aws-vault]]></category>
            <category><![CDATA[aws-cli]]></category>
            <dc:creator><![CDATA[Ali Abbas Jaffri]]></dc:creator>
            <pubDate>Fri, 02 Sep 2022 14:14:05 GMT</pubDate>
            <atom:updated>2022-09-05T06:57:12.551Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7K8nKBb2Fmc44_oTvqj-3w.png" /><figcaption>image credits: spot.io</figcaption></figure><p>I had the opportunity to use aws-vault with the team at my current project. I had previously just relied upon my configured aws cli tool to log into an account, but realised that it can be quite a head scratcher if you want to access or execute commands on multiple different aws sub-accounts from the cli. aws-vault solves this problem; it allows you to configure multiple aws sub accounts to be used accessed via the cli. The best part is to configure it once and you can easily integrate with any other cli tool that might need access to aws services.</p><p><a href="https://github.com/99designs/aws-vault">GitHub - 99designs/aws-vault: A vault for securely storing and accessing AWS credentials in development environments</a></p><p>We had three different aws sub-accounts, dev, staging and prod, in our aws account to rollout our changes on. I was accessing these accounts most commonly for infrastructure development using terraform. I had to configure the aws/config file everytime i wanted to switch to another sub-account prior to using aws-vault. And setting it up also takes just 5 minutes!</p><p>The prerequisite to configuring aws-vault is to have aws cli already installed followed by installing aws-vault via brew install --cask aws-vault for mac users. In an ideal aws setup, the user would be created in the root account and would have a mfa associated with it. In every aws sub-account, an admin or a high privilege role would allow the user to access different services. You would need to create access key and secret for the user to be able to access the account via the cli. After creating that, you’d need to update your aws/config file to have all the accounts configured.</p><p><a href="https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html">Installing or updating to the latest version of the AWS CLI</a></p><p>A typical setup with multiple sub-accounts would look like this:</p><pre>[default]<br>region=eu-central-1<br>output=yaml</pre><pre># the role_arn is the admin or similar role<br># that you need to create in every sub-account</pre><pre>[profile dev]<br>mfa_serial=arn:aws:iam::&lt;root-account-id&gt;:mfa/ali<br>role_arn=arn:aws:iam::&lt;dev-account-id&gt;:role/admin</pre><pre>[profile staging]<br>mfa_serial=arn:aws:iam::&lt;root-account-id&gt;:mfa/ali<br>role_arn=arn:aws:iam::&lt;staging-account-id&gt;:role/admin</pre><pre>[profile prod]<br>mfa_serial=arn:aws:iam::&lt;root-account-id&gt;:mfa/ali<br>role_arn=arn:aws:iam::&lt;prod-account-id&gt;:role/admin</pre><p>Once you’ve set up the config file, you can start adding the accounts for dev, staging and prod. That would be as simple as executing aws-vault add dev. The cli would prompt you for ACCESS_KEY_ID and SECRET_ACCESS_KEY which you already created for your user. You can verify if the configuration was successful or not by executing aws-vault exec dev -- aws sts get-caller-identity and verifying if you can see your user credentials. You can continue to create aws-vault config for the rest of the accounts and you’d have safely setup your aws cli access. An advantage of using aws-vault is that you do not need to store your access credentials in plain text format in aws/credentials file.</p><p>Setting up aws-vault eased up my deployment process to quite an extent. I do not need to modify my aws/config file to do cross account deployments anymore. I just need to select the respective account and execute my required command.</p><pre>aws-vault exec dev -- terraform plan<br>aws-vault exec staging -- terraform plan<br>aws-vault exec prod -- terraform plan</pre><p>You can inspect all the information stored in your aws-vault account for specific sub-account via aws-vault exec &lt;sub-account&gt; -- env | grep AWS. Let me know how do you find the usage of this utility.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=696d08446106" width="1" height="1" alt=""><hr><p><a href="https://itnext.io/aws-vault-to-authenticate-your-aws-accounts-696d08446106">aws-vault to authenticate your aws accounts</a> was originally published in <a href="https://itnext.io/">ITNEXT</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Structuring terraform project using terragrunt — Part I]]></title>
            <link>https://itnext.io/structuring-terraform-project-using-terragrunt-part-i-4c6e936c4858?source=rss-3914e337751e------2</link>
            <guid isPermaLink="false">https://medium.com/p/4c6e936c4858</guid>
            <category><![CDATA[terragrunt]]></category>
            <category><![CDATA[aws-ecs-fargate]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[infrastructure-as-code]]></category>
            <dc:creator><![CDATA[Ali Abbas Jaffri]]></dc:creator>
            <pubDate>Thu, 01 Sep 2022 15:03:49 GMT</pubDate>
            <atom:updated>2022-09-13T13:23:01.755Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iNHbjIaDKw72YOD52kn-zQ.png" /><figcaption>image credits: <a href="https://github.com/aaratn/terraenv">terraenv</a></figcaption></figure><h3>Structuring terraform projects using terragrunt — Part I</h3><p>I’ve been noticing that there is a reasonable gap regarding the wisdom on how to configure terraform projects. I decided to write a two part series on how to configure your terraform projects; ranging from small and medium sized projects, all the way to significantly large scale projects. I’ll be discussing pros and cons of using terragrunt and some tips on how to get it working for your projects.</p><p>I recently read a very nice write up by founder of Gruntwork, Yevgeniy Brikman, in which he compares using terraform workspaces, multiple branches and terragrunt to configure different environments. It is a 4 part series and even though it might seem like a biased perspective coming from the founder himself, the author was quite on point with a lot of comparative aspects that he mentioned in his posts.</p><p><a href="https://blog.gruntwork.io/how-to-manage-multiple-environments-with-terraform-32c7bc5d692">How to manage multiple environments with Terraform</a></p><p>I would highly encourage you to take a moment and go through those series and understand why just relying on different branching structure or using variables using .tfvars file is not enough. Yevgeniy however didn’t cater terraspace in his write up, which in fact is a direct competitor of terragrunt, with more opinions on how to configure your terraform project. Another emerging tool in this space is pulumi which is removing the need to learn .hcl language and bringing the infrastructure and cloud configuration to every developer. Pulumi allows its users code in the language of their choosing to generate infrastructure for multiple cloud environments.</p><ul><li><a href="https://terraspace.cloud/">Terraspace | The Terraform Framework</a></li><li><a href="https://www.pulumi.com/">Pulumi - Infrastructure as Code, Secrets Management, and AI</a></li></ul><p>The structuring process is quite straight forward for small to medium sized terraform projects. I would take the example of a recent post that i published; creating AWS ECS Fargate cluster for containerized application on AWS using terraform and terragrunt.</p><p><a href="https://itnext.io/creating-an-ecs-fargate-service-for-container-applications-using-terraform-and-terragrunt-2af5db3b35c0">Creating an ECS Fargate service for container applications using terraform and terragrunt</a></p><p>In my experience, Terraform projects need to be structured correct right from the beginning. It gets relatively harder to adopt drastic changes as the project develops as you cannot make subtle changes without messing up a significant chunk of infrastructure. Its is also important to think about the different environments you’d be creating in your cloud so that the terragrunt configuration can well reflect that. A subtle point to note here is that this needs to be well thought through since the beginning so that you can seamlessly progress as the project matures.</p><p>The very first ingredient of effective structuring is to understand the layering of your infrastructure and how all the structures are going to build on top of each other. As I was setting a clean environment for the ECS project, i identified that there are few components, such as <a href="https://aws.amazon.com/vpc/">AWS VPC</a>, <a href="https://aws.amazon.com/ecs/">AWS ECS cluster</a> and <a href="https://aws.amazon.com/elasticloadbalancing/application-load-balancer/">AWS Application Load Balancer</a>, which would be set up in the beginning and everything else would be built on top on them. I created terraform code for those components and started to configure the terragrunt configuration in a way i would want my infrastructure to layer on top of each other. The base layer components, in this case VPC, ECS cluster and ALB, were part of base_infrastructure which was to be set up in the beginning, and the containerized application would then be layered on top of the base_infrastructure. Extra care should be put in when identifying and configuring components in the base_infrastructure; base layer has to be stable and well configured right from the beginning so that we don’t introduce lots of changes later, which can possibly lead to recreation of these components, which, most of the times, would not be possible if layered components are dependent on them.</p><p>Once the requirements have been broken down in base_infrastructure and applications components, you can start with writing some code. Start with creating two base folders; terraform and terragrunt. All .tf code goes in terraform folder whereas all .hcl configuration goes in terragrunt folder. Modules in terraform folder can have a flat hierarchy, where all the components exist at the same level, ideally in their own folders. The terragrunt configuration exists in hierarchy in terragrunt folder, which is further split into base_infrastructure and applications sub-hierarchy. Another thing to cater would be the different environments that you’d be creating your infrastructure for; namely development, staging or production environments. You’d need introduce further sub-hierarchy within the folders under terragrunt to cater for dev , staging and prod environments. Create the modules that belong to that hierarchy of the infrastructure once those environment folders are in place.</p><p>There is no official guideline on this, but over the past years with multiple infrastructure projects, i’ve realised with experience that this is quite logical way so far to distribute your infrastructure components so that there is less impact of changes on other components in the architecture. This setup definitely has its own pros and cons; this would be an ideal setup in case your project is not very extensive, and it doesn’t take 15 minutes to go through the entire folder structure to find the terragrunt file you need to configure or edit. The separation of configuration for modules allow you to define <a href="https://terragrunt.gruntwork.io/docs/reference/config-blocks-and-attributes/#dependency">dependencies</a> so that the base modules are configured before the dependent modules. The project structure is extensibility friendly; if you need to add a new module to your infrastructure, you just need to create a new terraform module and add the terragrunt configuration in the respective folder.</p><p><a href="https://terragrunt.gruntwork.io/docs/reference/config-blocks-and-attributes">Configuration Blocks and Attributes</a></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/962309e35c6d35572dbe53e309393abf/href">https://medium.com/media/962309e35c6d35572dbe53e309393abf/href</a></iframe><p>Let me know if you have any comments, suggestions or improvements regarding this setup. Also if you’re using an alternate setup for your terraform projects, please let me know that in the comments as well. It would be interesting to hear how others are tackling this.</p><p><em>You can find the Part II to these series here</em>:</p><p><a href="https://itnext.io/structuring-terraform-project-using-terragrunt-part-ii-ab0dc08b2bc6">Structuring terraform project using terragrunt — Part II</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4c6e936c4858" width="1" height="1" alt=""><hr><p><a href="https://itnext.io/structuring-terraform-project-using-terragrunt-part-i-4c6e936c4858">Structuring terraform project using terragrunt — Part I</a> was originally published in <a href="https://itnext.io/">ITNEXT</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Creating an ECS Fargate service for container applications using terraform and terragrunt]]></title>
            <link>https://itnext.io/creating-an-ecs-fargate-service-for-container-applications-using-terraform-and-terragrunt-2af5db3b35c0?source=rss-3914e337751e------2</link>
            <guid isPermaLink="false">https://medium.com/p/2af5db3b35c0</guid>
            <category><![CDATA[aws-fargate]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[terragrunt]]></category>
            <category><![CDATA[ecs-fargate]]></category>
            <category><![CDATA[terraform]]></category>
            <dc:creator><![CDATA[Ali Abbas Jaffri]]></dc:creator>
            <pubDate>Tue, 23 Aug 2022 10:17:19 GMT</pubDate>
            <atom:updated>2022-09-08T12:58:08.558Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/741/1*8ffPI15bxUMli4MYD9ETzg.png" /></figure><h3>Creating an ECS Fargate service for containers using terraform and terragrunt</h3><p>I came across an interesting challenge during my job-hunt-mania of 2022 for a DevOps engineer role. The task required me to deploy and expose a docker container, which in this case was the famous <a href="https://hub.docker.com/_/ghost">ghost application</a>, using <a href="https://aws.amazon.com/fargate/">AWS ECS Fargate</a>. I had previously worked on <a href="https://kubernetes.io/">Kubernetes</a> as a container orchestrator and was quite unaware of how to get around ECS Fargate. I realised it to be a wonderful opportunity to learn ECS Fargate cluster as well as deploy an application on top of it, all while using terraform.</p><p><a href="https://aws.amazon.com/fargate/">Serverless Compute - AWS Fargate - AWS</a></p><p>The challenge was to setup a single click IaC deployment for the ECS Fargate cluster which would be able to autoscale given some thresholds; all this to be done in a span of mere 6 hours! The task, however being straight forward, was effort consuming as i had to setup a terraform project and decided to configure it using terragrunt. My first intuition was to search for a blog post from someone that allows to achieve this functionality, and i was able to find <a href="https://section411.com/2019/07/hello-world/">one such blog</a> which did a good enough job, but didn’t cover everything.</p><p>I divided my task into two parts; creating base infrastructure which comprised of all networking and necessary base applications to support my use case and of application infrastructure to provide a generic template of how to deploy such an application. My process was as follows:</p><ul><li>Create a VPC, subnets and setup all required routing tables</li><li>Create ECS Fargate components</li><li>- ECS Cluster</li><li>- ECS Service</li><li>- ECS Task definition</li><li>Create AWS application load balancer; ALB</li><li>Create security groups to tighten the security around my application</li><li>Create appautoscaling policy and threshold to autoscale my application based on defined metrics.</li></ul><p>I created a fresh VPC via the famous <a href="https://registry.terraform.io/modules/terraform-aws-modules/vpc/aws/latest">module from Anton Babenkov</a>, which allowed me to configure VPC, create multi-AZ subnets, configure gateways and set up routing tables almost instantly. This saved me time and helped me set up a decomposable environment for my application. Next step was to identify the rest of base infrastructure components and the application infrastructure components, and isolate them accordingly. My base infrastructure components culminated in the form of all networking related components, ECS cluster and an application load balancer to access my application; all other components went to application infrastructure. My project structure looked as follows:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/962309e35c6d35572dbe53e309393abf/href">https://medium.com/media/962309e35c6d35572dbe53e309393abf/href</a></iframe><p>I needed to first understand what is an ECS Fargate cluster and what are the components necessary to deploy an application on it. I found some good resources that helped me <a href="https://www.freecodecamp.org/news/amazon-ecs-terms-and-architecture-807d8c4960fd/">understand an ECS Cluster</a> and <a href="https://stackoverflow.com/a/42961623/4158615">differentiate between ECS services and tasks</a>. I am writing those out here for everyone to follow along; an ECS task is a container running the code from the user, whereas an ECS service runs and makes sure that the required number of tasks are executing in the ECS cluster.</p><p><a href="https://stackoverflow.com/a/42961623/4158615">What is the difference between a task and a service in AWS ECS?</a></p><p>I created three different availability zones and configured public subnets for each of those. The VPC module is quite flexible in that you can select the number of availability zones that you want and configure public / private subnets in them accordingly. For the sake of this project, i chose to deploy to all three zones to have a high availability deployment.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/37d24c105258de34d0b825ac083a06ad/href">https://medium.com/media/37d24c105258de34d0b825ac083a06ad/href</a></iframe><p>After the VPC is created, we create an ECS cluster which is a rather simple task; you just need to create a ECS cluster resource and name it.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a014e795586c9477dc92fa94d81af00d/href">https://medium.com/media/a014e795586c9477dc92fa94d81af00d/href</a></iframe><p>Now that we have an ECS cluster setup, we would need to create an application load balancer that would be taking requests for us and distributing it to the multiple tasks that the service will be creating and maintaining.</p><p><em>Please note that the security group configuration is intentionally left open for the world for the sake of ease of understanding but it is a bad practise.</em></p><p><em>Please adhere to your respective VPC and Subnet setup and adjust the ingress and egress on your security groups to restrict unnecessary access.</em></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6f24eb4e1456cb8a2de0baa3ffd3c553/href">https://medium.com/media/6f24eb4e1456cb8a2de0baa3ffd3c553/href</a></iframe><p>Next, we configure the ECS service and task for our application. We would need two service roles in order to accomplish the cluster setup; one as execution role for the ECS service and the other as autoscaling role for ECS to scale out on demand. I module-d the service role and created two subsequent role modules.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ca70cd4e16269cc4e4096e4279104d60/href">https://medium.com/media/ca70cd4e16269cc4e4096e4279104d60/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/281b06cedd98aa78647755ff0aa1b653/href">https://medium.com/media/281b06cedd98aa78647755ff0aa1b653/href</a></iframe><p>With the service roles in place, we can finally create the ECS service and task for the ghost container and expose it via the ALB.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6d01f1b7301e60ca9b7bf76302c5342d/href">https://medium.com/media/6d01f1b7301e60ca9b7bf76302c5342d/href</a></iframe><p>The last task for me was to understand <a href="https://stackoverflow.com/questions/65774071/aws-ecs-cluster-auto-scaling-vs-service-auto-scaling/65782253#65782253">how the ECS cluster auto-scaling worked</a>. To set up autoscaling, i needed to define only CPU and Memory based metrics and set a good estimated threshold value, that would allow the service to scale out. The autoscaling can be achieved by creating an aws_appautoscaling_target and creating respective aws_appautoscaling_policy to create a metric to scale out from.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/61efc25766095c08421559cd3c3e1429/href">https://medium.com/media/61efc25766095c08421559cd3c3e1429/href</a></iframe><p>During my research afterwards, i also realised that there could be <a href="https://faun.pub/deploy-a-containerized-application-on-aws-with-terraform-bf929bb3bb6b">ECS clusters comprising of EC2 instances</a>, where the user can specify the kind and type of EC2 instances to spawn up. ECS sure had a lot of tricks under its sleeve that i failed to notice during my initial encounter with it.</p><p>This was one of the harder and longer tutorials i’ve shared, and frankly i was quite surprised when i received this as an interview challenge, and really proud of myself for having accomplished it within the given time frame. I realise that there is a lot packed in this tutorial and it might not be possible for everyone to aptly follow along what i have been doing, and therefore i would be posting the github url for the repo along with this post, so that the curious can take a look and be able to join the dots themselves.</p><p>Personally, i believe hosting and exposing a container on ECS was harder than doing the same on a Kubernetes cluster. ECS might be a good service but setting it up and connecting everything together to get it to work was not at all a trivial task in terraform. I can imagine the friction in setting it up and managing at scale for larger teams and the simplicity and ease of execution that Kubernetes bring in comparison to it, with clearer APIs and resources.</p><p><em>PS. I would really appreciate a star on the github repo if you find this useful, as it allows me to create more relevant content in future.</em></p><p><a href="https://github.com/aliabbasjaffri/ecs-terraform-terragrunt">GitHub - aliabbasjaffri/ecs-terraform-terragrunt: This repository contains terraform code to deploy a simple ghost api container on ECS Fargate cluster with an application load balancer and autoscaling group</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2af5db3b35c0" width="1" height="1" alt=""><hr><p><a href="https://itnext.io/creating-an-ecs-fargate-service-for-container-applications-using-terraform-and-terragrunt-2af5db3b35c0">Creating an ECS Fargate service for container applications using terraform and terragrunt</a> was originally published in <a href="https://itnext.io/">ITNEXT</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>