<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Santosh Gupta on Medium]]></title>
        <description><![CDATA[Stories by Santosh Gupta on Medium]]></description>
        <link>https://medium.com/@virtualcloud?source=rss-e96c43f34ad6------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*l8sR00Qva5T_HlcOys3Kmw.jpeg</url>
            <title>Stories by Santosh Gupta on Medium</title>
            <link>https://medium.com/@virtualcloud?source=rss-e96c43f34ad6------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 05:29:01 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@virtualcloud/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[GPU Time Slicing on AWS EKS with Bottlerocket and Karpenter]]></title>
            <link>https://virtualcloud.medium.com/gpu-time-slicing-on-aws-eks-with-bottlerocket-and-karpenter-edec45c535ed?source=rss-e96c43f34ad6------2</link>
            <guid isPermaLink="false">https://medium.com/p/edec45c535ed</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[gpu]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[aws-eks]]></category>
            <category><![CDATA[nvidia]]></category>
            <dc:creator><![CDATA[Santosh Gupta]]></dc:creator>
            <pubDate>Thu, 03 Apr 2025 14:23:08 GMT</pubDate>
            <atom:updated>2025-04-03T14:23:08.111Z</atom:updated>
            <content:encoded><![CDATA[<p>Ever looked at your AWS bill after spinning up some GPU instances for your EKS cluster and thought, “Ouch! That GPU looks awfully lonely and underutilized.”? You’re not alone. GPUs are powerful beasts, but often, especially in development, testing, or light inference scenarios, our workloads barely tickle their massive potential. It’s like buying a Formula 1 car just to drive to the corner store for milk.</p><p>Enter <strong>GPU Time Slicing</strong>, a nifty way to make that expensive silicon work a bit harder (and smarter!) for its keep. Today, we’ll dive into what it is, why you might need it, its quirks, and most importantly, how to set it up on your EKS cluster running the lean, mean Bottlerocket OS, provisioned by the ever-helpful Karpenter.</p><h4>What in the CUDA is GPU Time Slicing?</h4><p>GPU Time Slicing is like a time-sharing system for GPUs. Instead of a single workload monopolizing the GPU, multiple workloads can use the same GPU by sharing it in slices of time. Imagine your GPU is a delicious, high-performance pizza. Normally, Kubernetes gives the <em>entire</em> pizza to one pod, even if that pod only nibbles on a single pepperoni. <strong>GPU Time Slicing</strong> is like saying, “Hey, let’s cut this pizza into smaller slices!” It allows multiple pods to <em>share</em> a single physical GPU. The GPU rapidly switches context between these different containers, giving each a “slice” of its processing time.</p><p><strong>Crucially, this is NOT the same as NVIDIA MIG (Multi-Instance GPU).</strong> MIG physically partitions the GPU’s resources (memory, compute units) into smaller, fully isolated GPU instances. Time Slicing is more like cooperative multitasking — everyone gets a turn on the <em>same</em> hardware, sharing <em>all</em> the resources, including the entire GPU memory. Think of it as roommates sharing one TV versus everyone having their own smaller TV in their room (MIG). <em>More on this in future blogs.</em></p><p>So, why go through the hassle of configuring this?</p><ol><li><strong>Cost Efficiency:</strong> This is the big one. Instead of needing a separate GPU instance for every little task that needs <em>some</em> GPU access, you can pack several onto a single instance.</li><li><strong>Increased Utilization:</strong> Get more out of your GPU by letting multiple workloads share the resource, especially for tasks that don’t need 100% GPU dedication.</li><li><strong>Simpler Dev/Test Environments:</strong> Perfect for teams where multiple developers need occasional GPU access for experiments, debugging, or running smaller models without needing dedicated, high-power hardware each time.</li><li><strong>Light Inference Workloads:</strong> If you have multiple inference endpoints that have low QPS or don’t require millisecond latency, time slicing can be a cost-effective way to serve them.</li></ol><h3>The Good, The Bad, and The Slightly Janky (Pros &amp; Cons)</h3><p>Like any sharing arrangement, time slicing has its ups and downs:</p><p><strong>Pros:</strong></p><ul><li><strong>Cost Savings:</strong> Did we mention this? Worth saying twice.</li><li><strong>Higher Utilization:</strong> Makes that GPU earn its keep.</li><li><strong>Relatively Simple Setup:</strong> Compared to MIG, the configuration (as we’ll see) is less complex.</li><li><strong>Good for Burst Loads:</strong> Handles multiple tasks that need <em>occasional</em> GPU power well.</li></ul><p><strong>Cons:</strong></p><ul><li><strong>Performance Overhead:</strong> Context switching isn’t free. There’s a performance hit compared to a dedicated GPU. Don’t expect peak performance for heavy training.</li><li><strong>No Memory Isolation:</strong> All containers sharing the GPU share its memory pool. One greedy container can cause Out-Of-Memory (OOM) errors for <em>everyone</em> sharing that GPU. It’s like that roommate who uses all the hot water.</li><li><strong>No Fault Isolation:</strong> If one container crashes the GPU driver or causes instability, it can affect <em>all</em> other containers on that GPU. Shared fate, shared pain.</li><li><strong>Not Ideal for Heavy Compute/Training:</strong> Performance-sensitive, long-running, high-throughput tasks are generally better off with dedicated GPUs or MIG.</li><li><strong>Monitoring Complexity:</strong> Understanding individual container GPU usage can be trickier than with dedicated resources. nvidia-smi might show combined usage.</li></ul><p><strong>Think of it this way:</strong> Time slicing is great for casual GPU users who can tolerate slightly lower performance and understand the risks of sharing. It’s <em>not</em> for demanding workloads that need guaranteed resources and isolation.</p><h3>Let’s Get Our Hands Dirty</h3><p>Alright, enough talk! Let’s configure this on an AWS EKS cluster with Bottlerocket and Karpenter.</p><p><strong>Prerequisites:</strong></p><ol><li>An EKS cluster (duh!)</li><li>Karpenter is installed and configured to manage nodes. (If not, check out the <a href="https://karpenter.sh/">Karpenter docs</a>)</li><li>kubectl is configured to talk to your cluster.</li><li>AWS CLI configured.</li><li>You’re using or plan to use Bottlerocket OS for your GPU nodes.</li></ol><h4><strong>Configure Your Karpenter NodePool &amp; EC2NodeClass</strong></h4><p>Assuming we already have the latest version of Karpenter up and running, first, let’s deploy a Karpenter N<em>odePool </em>and <em>EC2NodeClass </em>resource <strong><em>without GPU time-split configuration</em></strong> and validate if everything is woking as expected before jumping to time slicing.</p><p>Below are the manifest files which I used. You can updated with required details and use the same in your environment.</p><pre>apiVersion: karpenter.k8s.aws/v1<br>kind: EC2NodeClass<br>metadata:<br>  name: gpu-ec2nodeclass-without-time-slicing<br>spec:<br>  tags:<br>    Name: gpu-ec2nodeclass-without-time-slicing<br>  amiSelectorTerms:<br>    - alias: bottlerocket@latest<br>  instanceProfile: &quot;instanceprofile-to-be-used-by-EC2-provisioned-by-Karpenter&quot;<br>  subnetSelectorTerms:<br>    - id: &quot;private-subnet-id, for ex. subnet-xxxxxxxxxxxxx&quot;<br>    - id: &quot;private-subnet-id, for ex. subnet-xxxxxxxxxxxxx&quot;<br>  securityGroupSelectorTerms:<br>    - id: &quot;security group ID to be attached to Ec2, for ex. sg-xxxxxxxxxxxx&quot;<br>  metadataOptions:<br>    httpEndpoint: enabled<br>    httpProtocolIPv6: disabled<br>    httpPutResponseHopLimit: 2<br>    httpTokens: required<br>  blockDeviceMappings:<br>    - deviceName: /dev/xvda<br>      ebs:<br>        volumeSize: 50Gi<br>        volumeType: gp3<br>        encrypted: true<br>    - deviceName: /dev/xvdb<br>      ebs:<br>        volumeSize: 100Gi<br>        volumeType: gp3<br>        encrypted: true</pre><pre>apiVersion: karpenter.sh/v1<br>kind: NodePool<br>metadata:<br>  name: gpu-nodepool-without-time-slicing<br>spec:<br>  template:<br>    metadata:<br>      labels:<br>        nodegroup: gpu-app-without-time-slicing<br>    spec:<br>      nodeClassRef:<br>        group: karpenter.k8s.aws<br>        kind: EC2NodeClass<br>        name: gpu-ec2nodeclass-without-time-slicing<br>      requirements:<br>        - key: kubernetes.io/arch<br>          operator: In<br>          values: [&quot;amd64&quot;]<br>        - key: kubernetes.io/os<br>          operator: In<br>          values: [&quot;linux&quot;]<br>        - key: karpenter.sh/capacity-type<br>          operator: In<br>          values: [&quot;spot&quot;]<br>        - key: &quot;topology.kubernetes.io/zone&quot;<br>          operator: In<br>          values: [&quot;us-east-1a&quot;, &quot;us-east-1b&quot;]<br>        - key: node.kubernetes.io/instance-type<br>          operator: In<br>          values: [&quot;g4dn.xlarge&quot;]<br>      expireAfter: 720h  <br>  limits:<br>    cpu: 10<br>  disruption:<br>    consolidationPolicy: WhenEmptyOrUnderutilized<br>    consolidateAfter: 5m</pre><p>In the above NodePool config, I’m using “<em>g4dn.xlarge”</em> instance, and the instance type is “<em>spot” </em>to keep the cost less.</p><p>Now, let&#39;s save these yaml files. We can named those as <strong><em>“gpu-ec2nodeclass-without-time-slicing.yaml”</em></strong>and <strong><em>“gpu-nodepool-without-time-slicing.yaml”</em></strong> deploy it using kubectl</p><pre>kubectl apply -f gpu-ec2nodeclass-without-time-slicing.yaml<br>kubectl apply -f gpu-nodepool-without-time-slicing.yaml</pre><p>Once deployed, we can check these with the command below</p><pre>kubectl get nodepool<br>kubectl get ec2nodeclass</pre><p>Below is a screenshot for the same. In the <strong>Ready c</strong>olumn, you should see <em>True</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P4PT0GSwPn9IgUD9bLV0Eg.png" /></figure><p>Now, let’s deploy a sample app requesting a GPU. Below is the manifest</p><pre>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: gpu-app-without-time-slicing<br>  labels:<br>    app: gpu-app-without-time-slicing<br>spec:<br>  replicas: 1<br>  selector:<br>    matchLabels:<br>      app: gpu-app-without-time-slicing<br>  template:<br>    metadata:<br>      labels:<br>        app: gpu-app-without-time-slicing<br>    spec:<br>      nodeSelector:<br>        nodegroup: gpu-app-without-time-slicing<br>      containers:<br>        - name: nvidia-smi<br>          image: nvidia/cuda:12.3.2-runtime-ubuntu22.04<br>          command: [&quot;/bin/bash&quot;, &quot;-c&quot;]<br>          args:<br>            - |<br>              nvidia-smi;<br>              echo &quot;GPU Check Complete. Keeping container alive...&quot;;<br>              sleep infinity<br>          resources:<br>            limits:<br>              nvidia.com/gpu: 1</pre><p>We can save this yaml as “<em>gpu-deployment-without-time-slicing.yaml</em>” and deploy it using the command below -</p><pre>kubectl apply -f gpu-deployment-without-time-slicing.yaml</pre><p>Once deployed successfully, it will create a pod which will go into the pending state as there are no GPU instances available in the Cluster to schedule this pod.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/976/1*_Eu4DtUrhUkG_EUN3x6Spw.png" /></figure><p>Karpenter will check the existing nodepools based on the <strong><em>nodeselector </em></strong>and provision a new spot GPU instance. You can see that the selected OS is based on the bottlerocket Nvidia base image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*W6yZaYKN45UonyX1GOrjtQ.png" /></figure><p>Once the instance is up and running, we can see that the GPU pod, which was in Unscheduled state, is now running on this instance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PHWp0hSJm5d7N6-YBJZ8GA.png" /></figure><p>When we describe this new GPU instance with below command, we can see that it has only 1 GPU i.e. “nvidia.com/gpu”: “1”.</p><pre>kubectl get nodes -o json | jq -r &#39;.items[] | select(.status.capacity.&quot;nvidia.com/gpu&quot; != null) | {name: .metadata.name, capacity: .status.capacity}&#39;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Uv1g3kTMzaiRlAK1uX2WKA.png" /></figure><p>Now, since we have validated everything without time slicing. Let&#39;s try to deploy using the GPU time slicing config.</p><p>Below is the Karpenter EC2NodeClass and NodePool manifests.</p><pre>apiVersion: karpenter.k8s.aws/v1<br>kind: EC2NodeClass<br>metadata:<br>  name: gpu-ec2nodeclass-with-time-slicing<br>spec:<br>  tags:<br>    Name: gpu-ec2nodeclass-with-time-slicing<br>  amiSelectorTerms:<br>    - alias: bottlerocket@latest<br>  instanceProfile: &quot;instanceprofile-to-be-used-by-EC2-provisioned-by-Karpenter&quot;<br>  subnetSelectorTerms:<br>    - id: &quot;private-subnet-id, for ex. subnet-xxxxxxxxxxxxx&quot;<br>    - id: &quot;private-subnet-id, for ex. subnet-xxxxxxxxxxxxx&quot;<br>  securityGroupSelectorTerms:<br>    - id: &quot;security group ID to be attached to Ec2, for ex. sg-xxxxxxxxxxxx&quot;<br>  metadataOptions:<br>    httpEndpoint: enabled<br>    httpProtocolIPv6: disabled<br>    httpPutResponseHopLimit: 2<br>    httpTokens: required<br>  blockDeviceMappings:<br>    - deviceName: /dev/xvda<br>      ebs:<br>        volumeSize: 50Gi<br>        volumeType: gp3<br>        encrypted: true<br>    - deviceName: /dev/xvdb<br>      ebs:<br>        volumeSize: 100Gi<br>        volumeType: gp3<br>        encrypted: true<br>  userData: |<br>    [settings.kubelet-device-plugins.nvidia]<br>    device-sharing-strategy = &quot;time-slicing&quot;<br>    [settings.kubelet-device-plugins.nvidia.time-slicing]<br>    replicas = 10</pre><p>Here, I’m using the UserData section of <em>EC2NodeClass </em>to <strong><em>enable the GPU slicing and creating 10 slices using replicas option.</em></strong></p><p>Now, we can save these yaml files as <strong><em>“gpu-ec2nodeclass-with-time-slicing.yaml”</em></strong>and <strong><em>“gpu-nodepool-with-time-slicing.yaml”</em></strong> deploy it using kubectl</p><pre>kubectl apply -f gpu-ec2nodeclass-without-time-slicing.yaml<br>kubectl apply -f gpu-nodepool-without-time-slicing.yaml</pre><p>Now, when we check the <em>nodepool </em>and <em>ec2nodeclass </em>resources, we can see both has Ready status as True.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NmAJ8yQoI9UypqWCGqdMPg.png" /></figure><p>Now, let’s deploy the sample app requesting for GPU.</p><pre>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: gpu-app-with-time-slicing<br>  labels:<br>    app: gpu-app-with-time-slicing<br>spec:<br>  replicas: 1<br>  selector:<br>    matchLabels:<br>      app: gpu-app-with-time-slicing<br>  template:<br>    metadata:<br>      labels:<br>        app: gpu-app-with-time-slicing<br>    spec:<br>      nodeSelector:<br>        nodegroup: gpu-app-with-time-slicing<br>      containers:<br>        - name: nvidia-smi<br>          image: nvidia/cuda:12.3.2-runtime-ubuntu22.04<br>          command: [&quot;/bin/bash&quot;, &quot;-c&quot;]<br>          args:<br>            - |<br>              nvidia-smi;<br>              echo &quot;GPU Check Complete. Keeping container alive...&quot;;<br>              sleep infinity<br>          resources:<br>            limits:<br>              nvidia.com/gpu: 1</pre><p>For now, I’m deploying only 1 replica and then will increase the same after validating that the time slicing has been configured. As soon as we deploy this, Karpenter will create a new GPU instance ip-10.xx.xx.30.ec2.internal.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/864/1*VyOWdmhDDk13l7BhTir2JA.png" /></figure><p>Now, if we describe this new instance, we can see that it has splitted the GPU and renamed it as <em>“nvidia.com/gpu.shared”: “10”</em></p><pre>kubectl get nodes -o json | jq -r &#39;.items[] | select(.status.capacity.&quot;nvidia.com/gpu.shared&quot; != null) | {name: .metadata.name, capacity: .status.capacity}&#39;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UYoOJfeWITX0EBAD0o8zyQ.png" /></figure><p>However, when I checked my pod, it was still not scheduled.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/797/1*B66IzKnE5Arpa5KGGP1IUA.png" /></figure><p>After checking further, I see that the node claim which is created by karpenter for this node is in unknown state.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JPfTLDntKfZp6alGPwPQWA.png" /></figure><p>When I described the nodeclaim, I can see an error —<em> Resource “nvidia.com/gpu” was requested but not registered</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/629/1*pLHCvXb-WjFXUIs5-_Ya3A.png" /></figure><p>This is because in the deployment.yaml file, we are requsting for <em>nvidia.com/gpu</em> resource, however, when the EC2 is spinning up, the GPU is getting splitted and getting renamed as we saw earlier.</p><p>After checking further, I can see an option to not rename the GPU after splitting, you can check <a href="https://bottlerocket.dev/en/os/1.34.x/api/settings/kubelet-device-plugins/#nvidia-time-slicing-rename-by-default">here</a>. We need to add the “<em>rename-by-default = false</em>” option in the Ec2NodeClass Userdata. Below is the updated manifest, lets update the same in our EC2NodeClass.</p><pre>apiVersion: karpenter.k8s.aws/v1<br>kind: EC2NodeClass<br>metadata:<br>  name: gpu-ec2nodeclass-with-time-slicing<br>spec:<br>  tags:<br>    Name: gpu-ec2nodeclass-with-time-slicing<br>  amiSelectorTerms:<br>    - alias: bottlerocket@latest<br>  instanceProfile: &quot;instanceprofile-to-be-used-by-EC2-provisioned-by-Karpenter&quot;<br>  subnetSelectorTerms:<br>    - id: &quot;private-subnet-id, for ex. subnet-xxxxxxxxxxxxx&quot;<br>    - id: &quot;private-subnet-id, for ex. subnet-xxxxxxxxxxxxx&quot;<br>  securityGroupSelectorTerms:<br>    - id: &quot;security group ID to be attached to Ec2, for ex. sg-xxxxxxxxxxxx&quot;<br>  metadataOptions:<br>    httpEndpoint: enabled<br>    httpProtocolIPv6: disabled<br>    httpPutResponseHopLimit: 2<br>    httpTokens: required<br>  blockDeviceMappings:<br>    - deviceName: /dev/xvda<br>      ebs:<br>        volumeSize: 50Gi<br>        volumeType: gp3<br>        encrypted: true<br>    - deviceName: /dev/xvdb<br>      ebs:<br>        volumeSize: 100Gi<br>        volumeType: gp3<br>        encrypted: true<br>  userData: |<br>    [settings.kubelet-device-plugins.nvidia]<br>    device-sharing-strategy = &quot;time-slicing&quot;<br>    [settings.kubelet-device-plugins.nvidia.time-slicing]<br>    replicas = 10<br>    rename-by-default = false</pre><p>I deleted the unknown status nodeclain and then scale down and scale up the deplyment. Karpenter created new GPU Instance to schedule the pod, we can see in below screenshot.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xivvGoGUxvLAMSvwwL8yXQ.png" /></figure><p>Now, let’s check if the GPU name was renamed or not.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7iI7YlA9ERrVzAxJ1dkpQg.png" /></figure><p><strong>NOTE</strong>: Since GPU is not renamed, we have to update the query to find the correct information</p><pre>kubectl get nodes -o json | jq -r &#39;.items[] | select(.status.capacity.&quot;nvidia.com/gpu.shared&quot; != null) | {name: .metadata.name, capacity: .status.capacity}&#39;</pre><p>We can see that a single GPU has been splitted to 10. Also, the pod has been scheduled successfully this time without any issue and karpenter nodeclaim is also in ready state. Now, let’s increse the no. of relicase of the deployment.</p><pre>kubectl scale deployment gpu-app-with-time-slicing --replicas=5 -n default</pre><p>We can see all the 5 pods has been scheduled on the same instance. This validate that the GPU time slicing is woking as expcted.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dAUnTB94XOQOMcasrOmx5w.png" /></figure><h3>Conclusion: Slice Smarter, Not Harder!</h3><p>GPU time slicing on EKS with Bottlerocket and Karpenter offers a fantastic way to improve resource utilization and cut costs, especially for development, testing, and lighter inference tasks. By leveraging Bottlerocket’s streamlined configuration and Karpenter’s intelligent provisioning, setting it up is quite manageable.</p><p>Just remember the trade-offs: you sacrifice performance isolation and guaranteed memory for cost savings and density. It’s not a silver bullet, but it’s a very useful tool in your Kubernetes arsenal. So go ahead, slice up those pricey GPU pizzas and make them serve more guests!</p><p>Hit the clap button 10 tims (:D) if you find this informative.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=edec45c535ed" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Helm Hooks: The Unsung Heroes of Your Kubernetes Deployments]]></title>
            <link>https://virtualcloud.medium.com/helm-hooks-the-unsung-heroes-of-your-kubernetes-deployments-5c485af8f1f3?source=rss-e96c43f34ad6------2</link>
            <guid isPermaLink="false">https://medium.com/p/5c485af8f1f3</guid>
            <category><![CDATA[kubernetes-deployment]]></category>
            <category><![CDATA[helm]]></category>
            <category><![CDATA[helm-hooks]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[aws-eks]]></category>
            <dc:creator><![CDATA[Santosh Gupta]]></dc:creator>
            <pubDate>Wed, 12 Mar 2025 07:25:37 GMT</pubDate>
            <atom:updated>2025-03-12T07:25:37.491Z</atom:updated>
            <content:encoded><![CDATA[<p>Imagine you’re hosting a grand party (i.e., deploying your application) and you’ve hired the best caterer (Helm) in town. But wait — what about the behind-the-scenes crew that ensures every glass is filled and every dish is served at just the right moment? Enter <strong>Helm hooks</strong>: the unsung heroes that manage the pre-party setup, mid-party magic, and post-party cleanup in your Kubernetes deployments.</p><p>In this blog post, we’ll dive deep into the world of Helm hooks, exploring how they work, what problems they solve, and how you can wield them to streamline your deployment process. And don’t worry — we’ll sprinkle in a bit of humor to keep things light, even if your deployment logs sometimes read like horror stories.</p><h4>What Are Helm Hooks?</h4><p>Helm hooks are annotations added to Kubernetes resource manifests within your Helm charts. They tell Helm, <em>“Hey, run this resource at a specific point in the lifecycle of a release!”</em> Think of them as special event triggers that execute at key moments, such as before an installation, after an upgrade, or even during a rollback.</p><p>A simple example of a Helm hook might look like this:</p><pre>apiVersion: batch/v1<br>kind: Job<br>metadata:<br>  name: pre-install-job<br>  annotations:<br>    &quot;helm.sh/hook&quot;: &quot;pre-install&quot;<br>spec:<br>  template:<br>    spec:<br>      containers:<br>        - name: setup-container<br>          image: busybox<br>          command: [&quot;sh&quot;, &quot;-c&quot;, &quot;echo Preparing the party...&quot;]<br>      restartPolicy: Never</pre><p>Here, the <em>&quot;helm.sh/hook&quot;: &quot;pre-install&quot;</em> annotation instructs Helm to run this job <strong>before</strong> creating the rest of your resources. It’s like making sure the tables are set before guests arrive.</p><h4>When Do Hooks Run?</h4><p>Helm hooks run at different stages of the Helm release lifecycle. Here’s a quick rundown:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/734/1*cZkam8O6e8FinRzbcqjfMg.png" /></figure><p>Hooks are like your party’s event coordinators — each with a specific role to ensure that everything from setting up the snack table to cleaning up afterwards happens right on schedule.</p><h4>What Can You Do With Helm Hooks?</h4><p>Helm hooks open up a world of possibilities to automate and safeguard your deployments. Here are some common use cases:</p><ol><li><strong>Database Migrations</strong></li></ol><p>Ever rolled out a new version of your app only to find your database schema didn’t get the memo? A <strong>pre-upgrade</strong> hook can trigger a job to migrate your database before your app starts using it, ensuring a smooth transition without data disasters.</p><pre>annotations:<br>  &quot;helm.sh/hook&quot;: &quot;pre-upgrade&quot;<br>  &quot;helm.sh/hook-delete-policy&quot;: &quot;before-hook-creation&quot;</pre><p>2. <strong>Service Readiness Checks</strong></p><p>Before deploying your application, you might need to ensure that external services (like databases or caches) are up and running. A <strong>pre-install</strong> hook can run a script that pings these services, giving you peace of mind that the stage is set for your main event.</p><pre>annotations:<br>  &quot;helm.sh/hook&quot;: &quot;pre-install&quot;<br>  &quot;helm.sh/hook-weight&quot;: &quot;-5&quot; # Runs before other hooks</pre><p>3. <strong>Cleanup and Tear-Down</strong></p><p>When you uninstall a Helm release, some resources might not get cleaned up automatically — think persistent volumes or temporary config maps. A <strong>post-delete</strong> hook can run a cleanup job to ensure there’s no digital debris left behind.</p><pre>annotations:<br>  &quot;helm.sh/hook&quot;: &quot;post-delete&quot;<br>  &quot;helm.sh/hook-delete-policy&quot;: &quot;hook-succeeded&quot;</pre><p>4. <strong>Automated Testing</strong></p><p>Want to know if your deployment was a success without breaking a sweat? Use a <strong>test</strong> hook to run integration tests post-deployment. This way, you can catch issues early and avoid those embarrassing “Oops, something went wrong” moments.</p><pre>annotations:<br>  &quot;helm.sh/hook&quot;: &quot;test&quot;</pre><p>You can run your tests with:</p><pre>helm test my-release</pre><p>There are numerous use cases for Helm hooks, each tailored to the specific requirements and logic of an application. Helm hooks provide a flexible way to automate and manage deployment workflows seamlessly.</p><p>If you’ve been following along, you may have noticed that we used <strong><em>hook policies</em></strong> and <strong><em>hook weights</em></strong> in the examples above. Let’s take a closer look at what they are and how they work.</p><h4>Hook Delete Policy: Keeping Your Cluster Clean</h4><p>By default, Helm doesn’t automatically clean up hook resources, which can lead to a lot of clutter in your cluster. To prevent this, Helm provides the <strong>helm.sh/hook-delete-policy</strong> annotation, which tells Helm when to delete hook resources after execution.</p><p><strong>Available Policies:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/872/1*DbImpxtoqUohhlVf44b1KA.png" /></figure><p><strong>NOTE:</strong> Just to clarify, the <strong>helm.sh/hook-delete-policy</strong> annotation controls the deletion of the <strong>Kubernetes resource</strong> (e.g., a Job, Pod, ConfigMap, etc.) created by the Helm hook—not the hook definition itself in the Helm chart.</p><h4>Hook Weights: Controlling Execution Order</h4><p>By default, Helm executes hooks <strong>in alphabetical order</strong> based on their names. However, sometimes you need precise control over execution order, especially when multiple hooks run at the same lifecycle stage (e.g., multiple pre-install hooks).</p><p><strong>How Does It Work?</strong></p><ul><li>You can assign <strong>weights</strong> to hooks using the <strong>helm.sh/hook-weight</strong> annotation.</li><li>Hooks with <strong>lower weights run first</strong> and hooks with <strong>higher weights run later</strong>.</li></ul><p><strong>Ordering Multiple Hooks:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/841/1*Q8Yy9OQcFJ3Y4Jrd33HBbw.png" /></figure><p>This is <strong>super useful</strong> when you have dependencies between hooks (e.g., ensuring a database is initialized before applying application configs).</p><h4>Best Practices for Using Helm Hooks</h4><p>While Helm hooks are powerful, they come with responsibilities. Here are some tips to keep in mind:</p><ul><li><strong>Use Hooks for Ancillary Tasks:</strong> Keep your critical business logic out of hooks. They’re great for setup, cleanup, and tests, but not for running your main application code.</li><li><strong>Control Execution Order with Weights:</strong> Use the helm.sh/hook-weight annotation to dictate the order in which hooks run. Higher weights run later, so plan accordingly.</li><li><strong>Set Deletion Policies:</strong> Prevent resource clutter by using helm.sh/hook-delete-policy (e.g., &quot;hook-succeeded&quot; or &quot;before-hook-creation&quot;) to clean up hooks after they’ve done their job.</li><li><strong>Monitor Hook Logs:</strong> Always review logs from hook executions. They can offer vital clues if something goes awry during your deployment process.</li><li><strong>Test in a Staging Environment:</strong> Before using hooks in production, try them out in a controlled environment to avoid any unexpected hiccups. After all, you wouldn’t want your party to turn into a disaster on opening night!</li></ul><h3>Final Thoughts</h3><p>Helm hooks might not be the most glamorous feature of Kubernetes deployments, but they’re indispensable for creating robust, automated, and resilient workflows. Whether you’re preparing the ground with pre-install checks, safeguarding your data with migrations, or tidying up after a deletion, Helm hooks are your backstage pass to a smooth deployment show.</p><p>So next time you’re planning your deployment, remember: it’s not just about the main event. It’s the little things — the hooks — that make all the difference. Happy deploying, and may your Helm charts always be error-free (or at least, gracefully recoverable)!</p><p>Have a favorite use case or a humorous story about a deployment gone sideways? Drop it in the comments below — let’s share a laugh (and some lessons) together!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5c485af8f1f3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Sync secrets from AWS Secrets Manager to Kubernetes Secrets using External Secrets Operator]]></title>
            <link>https://virtualcloud.medium.com/sync-secrets-from-aws-secrets-manager-to-kubernetes-secrets-using-external-secrets-operator-29be58d3beea?source=rss-e96c43f34ad6------2</link>
            <guid isPermaLink="false">https://medium.com/p/29be58d3beea</guid>
            <category><![CDATA[aws-secret-manager]]></category>
            <category><![CDATA[kubernetes-secret]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[external-secrets-operator]]></category>
            <dc:creator><![CDATA[Santosh Gupta]]></dc:creator>
            <pubDate>Mon, 17 Jun 2024 13:03:45 GMT</pubDate>
            <atom:updated>2024-06-17T13:14:38.060Z</atom:updated>
            <content:encoded><![CDATA[<p>In modern cloud-native applications, managing secrets &amp; credentials efficiently and securely is critical for maintaining a robust production environment. AWS Secret Manager provides a secure and centralized way to manage sensitive information such as API keys, database credentials, etc. Similarly, Kubernetes secrets offer a reliable way to handle secrets inside the Kubernetes cluster. Integrating these two can streamline secret management in a production-grade EKS cluster.</p><p>The External Secrets Operator simplifies this process by syncing secrets from external sources (like AWS Secrets Manager) to Kubernetes secrets. Here’s a step-by-step guide to achieving this in a production-grade EKS cluster.</p><h4>Prerequisites</h4><ul><li>An AWS account with access to create and manage secrets in AWS Secrets Manager.</li><li>An EKS cluster to which you have access.</li><li>kubectl installed and configured to access your EKS cluster.</li><li>Helm installed on your local machine.</li></ul><p><strong>Step 1: Set Up AWS Secrets Manager</strong></p><p>First, create a secret in AWS Secrets Manager. In the later part of this blog, this secret will be synced to the Kubernetes secret. Below are the steps to create the secrets if you don’t know already.</p><ol><li>Log in to the AWS Management Console.</li><li>Navigate to <strong>Secrets Manager</strong>.</li><li>Click <strong>Store a new secret</strong>.</li><li>Choose <strong>Other type of secret</strong> and enter key-value pairs for your secret.</li><li>Name your secret, in my case, I’m using the name<strong> </strong><strong>my-app-secret.</strong></li></ol><p>Here is a screenshot from the AWS console.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1023/0*N35hE8_gcLU9DMH3" /></figure><p>Once the secret has been created, you can search the secret name in AWS secrets managers and validate the details as shown below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*4ZaEypC_CWlz7qfc" /></figure><p><strong>Step 2: Install External Secrets Operator</strong></p><p>I have an EKS cluster running with a single EC2 node on which we will install the External Secrets Operator. I have configured kubectl to interact with the EKS cluster, below is a screenshot showing the details of the EKS cluster node and all the pods running on it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1023/0*TIxinoxsE491fjTC" /></figure><p>Now, we will install External Secrets Operator using the Helm chart. First, let’s add the Helm repository for External Secrets Operator:</p><pre>helm repo add external-secrets https://charts.external-secrets.io<br>helm repo update</pre><p>Once the repo is added, we can run the below command to create a new namespace “<strong><em>external-secrets</em></strong>” and Install the External Secrets Operator:</p><pre>kubectl create namespace external-secrets<br>helm install external-secrets external-secrets/external-secrets -n external-secrets</pre><p>Once it’s installed, we can see the external secrets pods are up and running.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1022/0*cMvi3YsFQpHvj4_7" /></figure><p><strong>Step 3: Configure AWS Credentials for External Secrets Operator</strong></p><p>To allow the External Secrets Operator to access AWS Secrets Manager, we need to set up the proper AWS IAM role and service account with the necessary policies. Let’s first create an IAM policy that grants access to our secret in AWS Secrets Manager. Below is the policy that I used, please add the AWS region and account ID as per your infra--</p><pre>{<br>    &quot;Version&quot;: &quot;2012-10-17&quot;,<br>    &quot;Statement&quot;: [<br>        {<br>            &quot;Effect&quot;: &quot;Allow&quot;,<br>            &quot;Action&quot;: [<br>                &quot;secretsmanager:GetResourcePolicy&quot;,<br>                &quot;secretsmanager:GetSecretValue&quot;,<br>                &quot;secretsmanager:DescribeSecret&quot;,<br>                &quot;secretsmanager:ListSecretVersionIds&quot;<br>            ],<br>            &quot;Resource&quot;: [<br>                &quot;arn:aws:secretsmanager:&lt;aws-region&gt;:&lt;account-id&gt;:secret:my-app-secret-d2XlqT&quot;<br>            ]<br>        }<br>    ]<br>}</pre><p>Let’s name this policy “ <em>SecretsManagerAccessPolicy</em> “. Now, create an IAM role and attach the policy created above. Before creating the IAM role, as a pre-requisite, we need to associate the EKS cluster with an IAM OIDC provider, I have already done that. if you haven’t already done so, below is the command for the same -</p><pre>eksctl utils associate-iam-oidc-provider --region &lt;region&gt; --cluster &lt;cluster-name&gt; --approve</pre><p>Once you have the OIDC provider arn, we need to add those details in the trust relationships while creating the IAM role which will allow the EKS cluster to assume this role, here is a sample trust policy that I’m using with this IAM role, please fill the “ <em>oidc provider arn</em>” and “ <em>oidc provider name</em> “ along with service account and namespace. This will make sure that only the service account in this namespace can assume the IAM role.</p><pre>{<br>    &quot;Version&quot;: &quot;2012-10-17&quot;,<br>    &quot;Statement&quot;: [<br>        {<br>            &quot;Effect&quot;: &quot;Allow&quot;,<br>            &quot;Principal&quot;: {<br>                &quot;Federated&quot;: &quot;&lt;oidc provider arn&gt;&quot;<br>            },<br>            &quot;Action&quot;: &quot;sts:AssumeRoleWithWebIdentity&quot;,<br>            &quot;Condition&quot;: {<br>                &quot;StringEquals&quot;: {<br>                    &quot;&lt;oidc provider name&gt;:sub&quot;: &quot;system:serviceaccount:&lt;namespace&gt;:&lt;service-account-name&gt;&quot;<br>                }<br>            }<br>        }<br>    ]<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*LHx5YIf8os1Syr5g" /></figure><p>Now, we need to create a Kubernetes service account and add the above IAM role as an annotation. Below is the command which we will use to do this. In our case, the service account name is “ <em>external-secrets-sa</em> “.</p><pre>eksctl create iamserviceaccount --name external-secrets-sa --namespace external-secrets--cluster &lt;cluster-name&gt; --attach-role-arn arn:aws:iam::&lt;account-id&gt;:role/EKSSecretsRole --approve</pre><p>Once I applied the above command, it created the service account in the <strong><em>external-secrets</em></strong> namespace and annotated the IAM role.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*XGaX9xY6iIz4FuvB" /></figure><p><strong>Step 4: Create Kubernetes Secret Store</strong></p><p>Let’s create a Custom Resource Definition (CRD) that defines the external secret. This CRD tells the operator which secret to fetch from AWS Secrets Manager and how to map it to Kubernetes secrets. I have created a file named secretstore.yaml, which will define a SecretStore resource, below are manifest file details. As you can see, we are referencing the service account here which has access to the aws secret.</p><pre>apiVersion: external-secrets.io/v1beta1<br>kind: SecretStore<br>metadata:<br>  name: aws-secret-store<br>  namespace: external-secrets<br>spec:<br>  provider:<br>    aws:<br>      service: SecretsManager<br>      region: us-east-1<br>      auth:<br>        jwt:<br>          serviceAccountRef:<br>            name: external-secrets-sa</pre><p>Let’s apply this configuration via the below command.</p><pre>kubectl apply -f secretstore.yaml</pre><p>Once it’s applied successfully, we can describe this resource to check the details -</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/966/0*My91aUFc-6G1FHdf" /></figure><p><strong>Step 5: Create an External Secret</strong></p><p>We need to create an ExternalSecret resource to specify which secret from AWS Secrets Manager should be synced. I have created a file named externalsecret.yaml. We have defined the target as &quot; <em>my-secret-from-external-secret</em>&quot; which will be the name of the Kubernetes secret and the key name here is the name of the AWS secret which we need to sync, in our case it&#39;s &quot; <em>my-app-secret</em> &quot;</p><pre>apiVersion: external-secrets.io/v1beta1<br>kind: ExternalSecret<br>metadata:<br>  name: my-secret-from-external-secret<br>  namespace: external-secrets<br>spec:<br>  refreshInterval: 1h<br>  secretStoreRef:<br>    name: aws-secret-store<br>    kind: SecretStore<br>  target:<br>    name: my-secret-from-external-secret<br>  dataFrom:<br>  - extract:<br>      key: my-app-secret</pre><p>now, apply this configuration from the kubectl command.</p><pre>kubectl apply -f externalsecret.yaml</pre><p>Once the external secret resource has been created, it will create a Kubernetes secret and sync it with the aws secrets manager via the secretstore resource. In my case, it has created a secret name “ <em>my-secret-from-external-secret</em>” and then synced all the content from the AWS secret <em>“my-app-secret”</em>, here is a screenshot from my testing. As you can see, this secret is controlled by ExternalSecret.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/747/0*iONRRxCga5-5DvRE" /></figure><h3>Conclusion</h3><p>By following these steps, we have successfully synced secrets from AWS Secrets Manager to Kubernetes secrets in our EKS cluster using the External Secrets Operator. This setup ensures that your secrets are managed securely and efficiently, leveraging AWS’s robust secret management capabilities and Kubernetes’s seamless integration. This approach is scalable and production-ready, ensuring that your applications remain secure and maintain best practices for secret management.</p><p>I hope this has been informative for you, please let me know in the comments if you have any queries.</p><p><em>Originally published at </em><a href="https://virtualhackey.wordpress.com/2024/06/17/sync-secrets-from-aws-secrets-manager-to-kubernetes-secrets-using-external-secrets-operator/"><em>http://virtualhackey.wordpress.com</em></a><em> on June 17, 2024.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=29be58d3beea" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Securely Access Kubernetes Secrets in k8s Pods as Environment Variables]]></title>
            <link>https://virtualcloud.medium.com/securely-access-kubernetes-secrets-in-k8s-pods-as-environment-variables-c3ece7b2eb40?source=rss-e96c43f34ad6------2</link>
            <guid isPermaLink="false">https://medium.com/p/c3ece7b2eb40</guid>
            <category><![CDATA[aws-secrets-manager]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[kubernetes-security]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Santosh Gupta]]></dc:creator>
            <pubDate>Sun, 16 Jun 2024 09:56:54 GMT</pubDate>
            <atom:updated>2024-06-17T13:55:08.063Z</atom:updated>
            <content:encoded><![CDATA[<p>In Kubernetes, managing sensitive data such as API keys, passwords, and certificates is a critical aspect of application security. Kubernetes secrets provide a way to store and manage sensitive information securely. One of the common practices is to mount these secrets to pods and access them as environment variables. In this blog post, we’ll walk you through the steps to achieve this, ensuring your sensitive data remains secure and easily accessible by your applications.</p><h3>Prerequisites</h3><ol><li>A running Kubernetes cluster. For this demo, I have created a cluster using Kind, for more details you can check my blog <a href="https://medium.com/@virtualcloud/the-easiest-way-to-create-a-kubernetes-cluster-without-kubeadm-1a8b719d3b86">here.</a></li><li>kubectl installed and configured to interact with your cluster.</li></ol><h3>Creating Kubernetes Secrets</h3><p>First, let’s create a Kubernetes secret. We can do this using a YAML file or the kubectl command. For this demo, I&#39;ll show you both approaches.</p><p>Using kubectl to Create Secrets</p><pre>kubectl create secret generic my-secret --from-literal=username=my-username --from-literal=password=my-password</pre><p>This command creates a secret named my-secret with two key-value pairs: username and password.</p><p>Using YAML to Create Secrets</p><p>Alternatively, you can define the secret in a YAML file:</p><pre>apiVersion: v1<br>kind: Secret<br>metadata:<br>  name: my-secret<br>type: Opaque<br>data:<br>  username: bXktdXNlcm5hbWU=<br>  password: bXktcGFzc3dvcmQ=</pre><p>In the YAML file, the values for username and password are base64 encoded. You can encode these values using an online tool or the base64 command in Linux:</p><pre>echo -n &#39;my-username&#39; | base64 echo -n &#39;my-password&#39; | base64</pre><p>Apply the secret using kubectl:</p><pre>kubectl apply -f secret.yaml</pre><p>Here is a screenshot of the secrets that I have created from the above YAML file. We can see the secret is created in the default namespace and the username and password is in base64 encoded format.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/903/0*qa8qB_JL4xogK6r5" /></figure><h3>Why Secrets as Environment Variables?</h3><p>As a best practice, secrets or passwords should not be hardcoded in Kubernetes manifests, application code, or Dockerfiles. Instead, it is recommended to define environment variables in your code and make those variables available inside the pod. However, these secrets should not be defined directly in the Kubernetes manifests. The optimal approach is to store secrets in a secure vault, such as AWS Secrets Manager, and synchronize them with Kubernetes secrets. Detailed instructions on how to accomplish this can be found in my blog post <a href="https://virtualcloud.medium.com/sync-secrets-from-aws-secrets-manager-to-kubernetes-secrets-using-external-secrets-operator-29be58d3beea">here</a>.</p><p>Once the Kubernetes secrets are available, they can be referenced from the pod manifest, using them as environment variables. Below is an example YAML file for the pod, referencing the secret:</p><pre>apiVersion: v1<br>kind: Pod<br>metadata:<br>  name: my-pod<br>spec:<br>  containers:<br>  - name: my-container<br>    image: nginx<br>    env:<br>      - name: USERNAME<br>        valueFrom:<br>          secretKeyRef:<br>            name: my-secret<br>            key: username<br>      - name: PASSWORD<br>        valueFrom:<br>          secretKeyRef:<br>            name: my-secret<br>            key: password</pre><p>In this specification:</p><ul><li>We define a pod named my-pod with a single container running the nginx image.</li><li>We set two environment variables (USERNAME and PASSWORD) by referencing the my-secret secret and the specific keys within that secret.</li></ul><p>Now, let’s apply the YAML file to create the pod:</p><pre>kubectl apply -f pod.yaml</pre><p>Below is a screenshot of the same when I applied the pod YAML file:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/891/0*79a0Tt8T3-K-6Kz_" /></figure><p>Now the pod is running, we can verify that the secrets are correctly mounted as environment variables. we will use kubectl exec to run a command in the pod and print the environment variables:</p><pre>kubectl exec -it my-pod -- /bin/sh -c &#39;echo $USERNAME &amp;&amp; echo $PASSWORD&#39;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*7yWTQQKK8aBKwo9v" /></figure><p>This command should output the values of USERNAME and PASSWORD as specified in the secret. We can also exec inside the pod and verify if the environment variables are available inside the pod:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/643/0*S0Xa9SVWS4w_zk1p" /></figure><h3>Best Practices</h3><p>Below are some of the best practices that we should follow to secure the environment further.</p><p><strong>1. Limit Access to Secrets</strong></p><p>Ensure that only authorized pods and users have access to secrets. Use Kubernetes RBAC to enforce this.</p><p><strong>2. Use Dedicated Secrets for Different Applications</strong></p><p>Avoid sharing secrets across multiple applications. This minimizes the risk of unintended access.</p><p><strong>3. Regularly Rotate Secrets</strong></p><p>Regularly update and rotate secrets to reduce the risk of exposure. Kubernetes does not automatically update secrets in running pods, so you may need to recreate pods or use rolling updates.</p><p><strong>4. Monitor and Audit Access</strong></p><p>Monitor and audit access to secrets. Use Kubernetes audit logs to keep track of access patterns and detect any unauthorized access.</p><h3>Conclusion</h3><p>Mounting Kubernetes secrets as environment variables in pods is a straightforward yet powerful way to manage sensitive data in your applications. By following the steps outlined in this guide, you can securely and efficiently manage secrets, ensuring that your applications have access to the data they need without compromising security. Remember to follow best practices for secret management to maintain a secure and resilient Kubernetes environment.</p><p><em>Originally published at </em><a href="https://virtualhackey.wordpress.com/2024/06/16/securely-access-kubernetes-secrets-in-k8s-pods-as-environment-variables/"><em>http://virtualhackey.wordpress.com</em></a><em> on June 16, 2024.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c3ece7b2eb40" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Integrate VSCode with Ollama and LLMs to Setup Personal Copilot]]></title>
            <link>https://virtualcloud.medium.com/integrate-vscode-with-ollama-and-llms-to-setup-personal-copilot-25325b66b7d1?source=rss-e96c43f34ad6------2</link>
            <guid isPermaLink="false">https://medium.com/p/25325b66b7d1</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[genai]]></category>
            <category><![CDATA[github-copilot]]></category>
            <dc:creator><![CDATA[Santosh Gupta]]></dc:creator>
            <pubDate>Thu, 04 Apr 2024 20:19:35 GMT</pubDate>
            <atom:updated>2024-04-04T20:22:11.861Z</atom:updated>
            <content:encoded><![CDATA[<p>Are you tired of searching for code snippets, struggling with complex configurations, or feeling overwhelmed during development? Look no further! In this blog post, we’ll explore how to set up your personal <strong>code assistant</strong> using <strong>Large Language Models (LLMs)</strong> via <strong>Ollama</strong>. Whether you’re a seasoned developer or just starting, having an AI-powered companion can significantly boost your productivity and streamline your workflow.</p><p>While ChatGPT can be useful for generating code, it can become cumbersome when you’re constantly switching between it and your code editor like VS Code. Additionally, providing prompts and generating code without the full context of the project can be challenging and time-consuming to generate the expected results.</p><p>We already have tools like GitHub Copilot, which is an AI-powered code completion tool developed by GitHub in collaboration with OpenAI. It is designed to assist developers by providing contextual code suggestions and completions as they write code within their integrated development environment (IDE).</p><p>“Why build your own copilot when there’s already one available?” you might ask. Well, for starters, many existing solutions are paid, putting them out of reach for some developers. Moreover, concerns about security and privacy often arise. Unlike cloud-based alternatives, Ollama operates entirely on your local machine. There’s no need to send your source code to external servers. If you prioritize data privacy or prefer working offline, Ollama offers an exceptional self-hosted solution. Additionally, Ollama harnesses open-source LLMs, freeing you from dependency on a single vendor or platform. This independence ensures flexibility and adaptability to your specific needs and preferences.</p><p>Refer to my earlier post for guidance on installing Ollama <a href="https://medium.com/@virtualcloud/ai-at-your-service-how-to-setupyour-personalized-chat-assistance-using-ollama-and-llm-models-5e1d2813a8ea">here</a>. In this article, we’ll delve into integrating Ollama with VS Code to transform it into your personal code assistant.”</p><p>First, launch your VS Code and navigate to the extensions marketplace. Search for ‘ <em>Llama Coder</em> ‘ and proceed to install it. Here’s a screenshot of what it looks like in my VS Code console:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*uRRRVmK8LjVs5WBE" /></figure><p>After installing the Llama Coder extension, you’ll notice its icon displayed at the bottom right corner of your VS Code window. Here’s a screenshot for reference:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*8Fhi4Z2G9Cn5duWp" /></figure><p>Clicking on this icon will open the settings of the Llama Coder extension, allowing you to select your preferred LLM model for your code assistant. For this demonstration, I’ll stick with the default model. Here’s a screenshot illustrating the process:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ycVDdeVEb3WAq3Wu" /></figure><p>Now, let’s create our test Python file named ‘test.py’. As soon as you start typing in the file, ‘Llama Coder’ will attempt to generate code or provide auto-completion suggestions. Since this is the first time using Llama Coder, the base model is not installed yet, and it will prompt you to download it. Here’s a screenshot illustrating the prompt:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*fiiTGXoJHnEkbx9H" /></figure><p>Click ‘Yes’ to initiate the download of this model. The download time may vary depending on your internet speed and the size of the selected model.</p><p>Once the LLM model has been successfully downloaded, you’ll witness the magic of code generation unfold. For this demonstration, I experimented with a straightforward Python function to add two integers, and Llama Coder promptly began providing suggestions as soon as I defined the function. Here’s a screenshot showcasing the same:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/808/0*dvAggCgB8poWV_tv" /></figure><p>In conclusion, the integration of VS Code with Ollama and LLMs opens up a world of possibilities for developers seeking enhanced productivity and code assistance. As demonstrated, this setup allows for seamless code generation and autocomplete features directly within the familiar environment of VS Code. Furthermore, the flexibility to choose from a variety of powerful models ensures that developers can tailor their code assistance experience to suit their specific requirements.</p><p>I hope this has been informative for you. Please comment in case of any queries, Happy coding!</p><p><em>Originally published at </em><a href="https://virtualhackey.wordpress.com/2024/04/04/integrate-vscode-with-ollama-and-llms-to-setup-personal-copilot/"><em>http://virtualhackey.wordpress.com</em></a><em> on April 4, 2024.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=25325b66b7d1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI at Your Service: How to SetupYour Personalized Chat Assistance Using Ollama and LLM Models]]></title>
            <link>https://virtualcloud.medium.com/ai-at-your-service-how-to-setupyour-personalized-chat-assistance-using-ollama-and-llm-models-5e1d2813a8ea?source=rss-e96c43f34ad6------2</link>
            <guid isPermaLink="false">https://medium.com/p/5e1d2813a8ea</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ollama]]></category>
            <category><![CDATA[genai]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Santosh Gupta]]></dc:creator>
            <pubDate>Thu, 04 Apr 2024 18:38:07 GMT</pubDate>
            <atom:updated>2024-04-04T18:43:56.406Z</atom:updated>
            <content:encoded><![CDATA[<p>In today’s fast-paced world, where information is at our fingertips, leveraging the power of AI has become essential to streamline tasks and enhance productivity. Imagine having your <strong>personal chat assistant</strong> that is running locally on your computer instead of someone else’s computer and can scrap internet sites, answer your queries, and assist you in real time. In this guide, we’ll explore how to set up your very own chat companion using <strong>Large Language Models (LLMs)</strong> and <strong>Ollama</strong>. Let’s dive in!</p><h3>What is OLLAMA?</h3><p><strong>Ollama</strong> is a lightweight, extensible framework for building and running language models on your local machine. It provides a simple API for creating, running, and managing models, as well as a library of pre-built models that can be easily used in various applications. Ollama allows you to work with large language models locally, without relying on external servers or cloud services. It offers a library of pre-trained language models that cover a wide range of use cases. These models can be customized and fine-tuned for specific tasks. Some example models available in the Ollama library include:</p><ul><li><strong>Llama 2</strong>: A 7B parameter model.</li><li><strong>Mistral</strong>: Another 7B parameter model.</li><li><strong>Dolphin Phi</strong>: A 2.7B parameter model.</li><li><strong>Neural Chat</strong>: A 7B parameter model.</li><li><strong>Starling</strong>: A 7B parameter model.</li><li><strong>Code Llama</strong>: A 7B parameter model.</li><li>And more!</li></ul><h3>The Power of LLM Models</h3><p>The heart of Ollama lies in its utilization of LLM models, these models have demonstrated remarkable capabilities in understanding and generating human-like text, making them ideal for various natural language processing tasks, including conversational agents, content generation, and more. LLM models leverage deep learning techniques to analyze vast amounts of text data, learning patterns, semantics, and linguistic nuances. This enables them to generate coherent and contextually relevant responses to user queries, mimicking human conversation with impressive accuracy.</p><h3>Getting Started with Ollama</h3><p>First, we need to download the Ollama. As of now, we have these agents available in preview. I’ll be installing it on my Windows PC, however it’s available for Linux and Mac as well. Here is the official <a href="https://ollama.com/download">Link</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/625/0*6c93VeivPgVGx5QC" /></figure><p>Once downloaded, we need to install it. It’s pretty straightforward, just double-click on the installer and click install, here is a screenshot.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*VS4micHTCC30XsoN" /></figure><p>Once installed, we can open the Powershell/CMD console and type <em>ollama</em>. If Ollama is installed successfully on your computer, you will be able to see the available commands and flags with Ollama:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Z4Eoa95mEch3Aq7n" /></figure><p>Now, let’s download our first LLM model. We have a list of models available on the Ollama platform, you can check the models library <a href="https://ollama.com/library">here</a>.</p><p>For this demo purpose, we will install two different LLM models llama2 and mistral as there are two ways to fetch the models. You can either pull them first using the below command:</p><p>Here is a screenshot for the same:</p><pre> <br>ollama pull mistral</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*TxkXRYwK3JJADbSH" /></figure><p>OR we can directly download and run the model using the below command:</p><pre> <br>ollama run llama2</pre><p>It can take a couple of minutes depending on your internet speed and the size of the model you are trying to pull.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Z-BNWpqBK_6txohX" /></figure><p>Once it’s been successfully downloaded, it will give you a prompt to send a message. You can ask your queries with the selected model which you are running. In my case, I asked “Who is Virat Kohli?”, below is a screenshot of the reply:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*JkeONb_0dNZOuK4R" /></figure><p>The speed of generation of these texts will depend on the spec and configuration of your PC as the LLM model is running locally.</p><p>I hope this has been informative for you. Please comment below in case of any queries.</p><p><em>Originally published at </em><a href="https://virtualhackey.wordpress.com/2024/04/04/ai-at-your-service-how-to-build-your-personalized-chat-assistance-using-ollama-and-llm-models/"><em>http://virtualhackey.wordpress.com</em></a><em> on April 4, 2024.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5e1d2813a8ea" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Unveiling Docker Images in EKS Cluster with Bottlerocket nodes]]></title>
            <link>https://virtualcloud.medium.com/unveiling-docker-images-in-eks-cluster-with-bottlerocket-nodes-9d16ec9b2850?source=rss-e96c43f34ad6------2</link>
            <guid isPermaLink="false">https://medium.com/p/9d16ec9b2850</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[gitops]]></category>
            <category><![CDATA[cloud]]></category>
            <category><![CDATA[bottle-rocket]]></category>
            <dc:creator><![CDATA[Santosh Gupta]]></dc:creator>
            <pubDate>Fri, 01 Mar 2024 10:48:10 GMT</pubDate>
            <atom:updated>2024-03-01T10:50:37.631Z</atom:updated>
            <content:encoded><![CDATA[<p>In the dynamic world of cloud computing, Kubernetes has emerged as the go-to platform for container orchestration, and Amazon EKS is a popular choice for managing Kubernetes clusters. However, what adds an extra layer of efficiency and security to an EKS cluster is the use of Bottlerocket OS.</p><p><strong>What is Bottlerocket OS?</strong></p><p>Bottlerocket OS is a purpose-built, minimalistic Linux distribution designed specifically for running containers at scale. Developed by AWS, it’s built to provide an optimized, secure, and reliable platform for managing containerized applications within Kubernetes clusters. Here’s why Bottlerocket OS stands out:</p><ol><li><strong>Security:</strong> Bottlerocket employs a minimal attack surface by using a read-only file system, reducing the risk of security vulnerabilities. It also supports automatic updates, ensuring that clusters are always up to date with the latest security patches.</li><li><strong>Minimalist Design</strong>: Bottlerocket OS comes with a minimalist design, focusing solely on the requirements of container orchestration. This streamlined approach eliminates unnecessary components, reducing the attack surface and enhancing security.</li><li><strong>Immutable Infrastructure</strong>: Bottlerocket OS operates on an immutable infrastructure model, meaning once deployed, the OS image cannot be modified. This approach ensures consistency across nodes and simplifies management by allowing for easy rollbacks and updates.</li></ol><p>Now that we have a good understanding of Bottlerocket OS and its advantages, let’s explore how to use the <em>crictl</em> tool to interact with the Bottlerocket OS and perform operations, for example listing the docker images being used in the cluster.</p><p>Imagine encountering a minor hiccup — a Docker image refusing to be pulled. The solution seems simple: log into the node and take a peek at the existing images. But hold your horses! If your EKS cluster nodes are powered by Bottlerocket OS, get ready for a detour through its robust security guardrails. As they say, with great power comes great responsibility — and a bit of extra legwork! 😉</p><p>Getting access to your node running Bottlerocket OS can sometimes feel like traversing a labyrinth of IAM roles and security protocols. Thankfully, with proper IAM roles attached to my nodes, initiating a session using the AWS Session Manager was a breeze.</p><p>Once you logged into the console, to interact with the Bottlerocket host filesystem, our gateway was the admin container. We can use the below command to enter into admin container:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/954/0*iNPmB1xX_oaOn4ck" /></figure><p>Now, the next step is to switch to the root shell of the Bottlerocket OS so that we can access the root filesystem. Execute the following command:</p><p>Once you are in the root shell, you need to navigate to the exact location of the crictl tool (<strong><em>/run/host-containerd/io.containerd.runtime.v2.task/default/admin/rootfs/usr/local/bin/crictl</em></strong> ) to interact with the OS. For example, if you want to check the crictl tool version or list down the docker images available, run the below commands:</p><pre>/run/host-containerd/io.containerd.runtime.v2.task/default/admin/rootfs/usr/local/bin/crictl --version</pre><pre>/run/host-containerd/io.containerd.runtime.v2.task/default/admin/rootfs/usr/local/bin/crictl images</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*z-Pp6EqLsqhWl1ht" /></figure><p>I hope this has been informative for you. Please comment in case of any queries.</p><p><em>Originally published at </em><a href="https://virtualhackey.wordpress.com/2024/03/01/unveiling-docker-images-in-eks-cluster-with-bottlerocket-nodes/"><em>http://virtualhackey.wordpress.com</em></a><em> on March 1, 2024.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9d16ec9b2850" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[ArgoCD Application Deployment with Helmfile plugin]]></title>
            <link>https://virtualcloud.medium.com/argocd-application-deployment-with-helmfile-plugin-292d840ac673?source=rss-e96c43f34ad6------2</link>
            <guid isPermaLink="false">https://medium.com/p/292d840ac673</guid>
            <category><![CDATA[helmfile]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[argo-cd]]></category>
            <category><![CDATA[helm]]></category>
            <category><![CDATA[gitops]]></category>
            <dc:creator><![CDATA[Santosh Gupta]]></dc:creator>
            <pubDate>Sat, 24 Feb 2024 16:46:27 GMT</pubDate>
            <atom:updated>2024-02-24T16:51:34.876Z</atom:updated>
            <content:encoded><![CDATA[<p>In the realm of continuous deployment and management of Kubernetes applications, ArgoCD stands out as a powerful tool. Its declarative approach to managing Kubernetes manifests and its GitOps principles make it a favorite among platform engineers. One of the standout features of Argo CD is its extensibility through plugins, which allows users to integrate additional functionalities seamlessly. In this blog post, we’ll explore ArgoCD’s Config Management Plugin capabilities by installing the Helmfile plugin and deploying applications using Helmfile configuration.</p><h3>What is Helmfile?</h3><p>Before we delve into integrating Helmfile with Argo CD, let’s take a moment to understand what Helmfile is. Helmfile serves as a declarative configuration management tool designed specifically for deploying Helm charts to Kubernetes clusters. It simplifies the management of Helm releases by enabling users to define releases and their corresponding values within a YAML file. Helmfile provides features such as templating, environment-specific values, and dependency management, making it a powerful tool for managing Kubernetes applications. This approach facilitates a more organized and version-controlled method for deploying Helm charts, ultimately streamlining the Helm deployment process.</p><p>Here is a sample configuration, let’s call it <em>helmfile.yaml </em>, which we will use in the demo later:</p><pre>repositories:<br>  - name: prometheus-community<br>    url: https://prometheus-community.github.io/helm-charts<br>  - name: bitnami<br>    url: https://charts.bitnami.com/bitnami<br> <br>releases:<br>- name: prometheus<br>  namespace: prometheus<br>  chart: prometheus-community/prometheus<br>  set:<br>  - name: rbac.create<br>    value: false<br>- name: postgresql<br>  namespace: postgresql<br>  chart: bitnami/postgresql<br>  values:<br>    - auth:<br>        username: secret-user<br>        password: secret-password</pre><p>In the above configuration file, we have added two different repositories and then we are calling these repos in the releases block where we can customize the different parameters of each repository. For more details, please check the official <a href="https://helmfile.readthedocs.io/en/latest/">helmfile </a>documentation.</p><h3>Introducing ArgoCD Configuration Management Plugin</h3><p>ArgoCD offers a robust set of configuration management tools out of the box such as Helm, Jsonnet, and Kustomize. However, there may be scenarios where you either prefer to use a different tool or require features beyond the capabilities of Argo CD’s native tools. In such cases, integrating a Config Management Plugin (CMP) becomes essential through which you can deploy the tools of your choice.</p><p>The pivotal component within ArgoCD responsible for generating Kubernetes manifests is the “ <em>repo server</em> “ It accomplishes this task by parsing source files retrieved from Helm, OCI, or Git repositories. When a Config Management Plugin is appropriately configured, the repo server can offload the manifest-building process to the plugin, offering flexibility in tooling and workflow customization.</p><p>To install a plugin in ArgoCD, the first step is to create a ConfigMap of CMP configuration. Plugins will be configured via a ConfigManagementPlugin manifest, while the ConfigManagementPluginlooks like a Kubernetes object, it is not actually a custom resource. It&#39;s just something that Argo CD &quot;understands&quot;. Below is a sample config map file:</p><pre>apiVersion: v1<br>kind: ConfigMap<br>metadata:<br>  name: cmp-helmfile<br>  namespace: argocd<br>data:<br>  plugin.yaml: |<br>    apiVersion: argoproj.io/v1alpha1<br>    kind: ConfigManagementPlugin<br>    metadata:<br>      name: cmp-helmfile<br>    spec:<br>      version: v1.0<br>      init:<br>        command: [sh, -c, &#39;echo &quot;Initializing...&quot;&#39;]<br>      generate:<br>        command: [sh, -c]<br>        args: [&quot;helmfile template --quiet --namespace $ARGOCD_APP_NAMESPACE&quot;]<br>      discover:<br>        fileName: &quot;helmfile.yaml&quot;</pre><p>In the above example configuration for the Config Management Plugin (CMP) for helmfile, we’ve defined three crucial sections under the spec:</p><ul><li>init - The &#39;init&#39; section specifies the command to be executed at the beginning of each manifest generation process. Here, we use a shell command (sh) to echo &quot;Initializing...&quot;. The init step is commonly used for setting up dependencies or performing any necessary initialization tasks. Failure to execute this step with a non-zero status code will result in the failure of manifest generation.</li><li>generate - The generate section is where the actual processing occurs. It runs within the Argo CD Application source directory and is responsible for generating Kubernetes manifests. In this example, we execute the helmfile template command, which generates Kubernetes manifests based on Helmfile configuration. We pass the --quiet flag to suppress unnecessary output, and we specify the target namespace for the Argo CD Application using the $ARGOCD_APP_NAMESPACE environment variable. It&#39;s important to note that the output of the generate step must be valid Kubernetes objects and must be written to stdout. You can read more about what env vars are available to you on the official <a href="https://argo-cd.readthedocs.io/en/stable/user-guide/build-environment/">Argo CD Build Environment</a> documentation page.</li><li>discover - The discover section is a flag to indicate if the sidecar for your CMP needs to run. In this case, we&#39;re looking for a file named helmfile.yaml.</li></ul><p>If you need more details, check out the official documentation of ArgoCD <a href="https://argo-cd.readthedocs.io/en/stable/operator-manual/config-management-plugins/">config management plugins</a>.</p><p>Let’s save the above config map as “ <em>cmp-helmfile.yaml</em> “ and deploy it to the K8s cluster:</p><pre>kubectl apply -f cmp-helmfile.yaml -n argocd</pre><p>We can see that the config map has been deployed in the argocd namespace where ArgoCD application is already deployed:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/871/0*3KKEKjbzpifyTswZ" /></figure><p>To ensure seamless integration of Helmfile with ArgoCD, the next crucial step involves patching the ArgoCD’s Repo Controller Deployment to include the necessary sidecar components. For Helmfile to function properly, we require the Helmfile binary, the Helm binary, and the Helm Diff plugin to be installed within the container. Fortunately, leveraging the existing image hosted in the Google Container Registry at ghcr.io/helmfile/helmfile, we can easily fulfill these requirements. Let&#39;s proceed by creating the patch file for this operation:</p><pre>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: argocd-repo-server<br>  namespace: argocd<br>spec:<br>  template:<br>    spec:<br>      containers:<br>        - name: cmp-helmfile<br>          securityContext:<br>            runAsNonRoot: true<br>            runAsUser: 999<br>          image: ghcr.io/helmfile/helmfile:v0.157.0<br>          imagePullPolicy: IfNotPresent<br># Entrypoint should be Argo CD lightweight CMP server i.e. argocd-cmp-server<br>          command: [/var/run/argocd/argocd-cmp-server]<br>          volumeMounts:<br>            - mountPath: /var/run/argocd<br>              name: var-files<br>            - mountPath: /home/argocd/cmp-server/plugins<br>              name: plugins<br>            - mountPath: /home/argocd/cmp-server/config/plugin.yaml<br>              subPath: plugin.yaml<br>              name: cmp-helmfile<br>            - mountPath: /tmp<br>              name: cmp-tmp<br>      volumes:<br>        - name: cmp-helmfile<br>          configMap:<br>            name: cmp-helmfile<br>        - emptyDir: {}<br>          name: cmp-tmp</pre><p>Here are some key considerations to take into account:</p><ol><li><strong>User Permissions</strong>: The sidecar container must run as user 999 to access the files within the cloned repository securely.</li><li><strong>Container Image Selection</strong>: Ensure the container image chosen either includes the necessary tools or select one known to have them. Alternatively, you can embed your plugin.yaml file, provided it can be accessed at /home/argocd/cmp-server/config/.</li><li><strong>Mounting Configuration</strong>: In this example, we’re mounting the plugin.yaml file using the ConfigMap we just created. Additionally, it&#39;s essential to include other mounts from Argo CD, such as /var/run/argocd (containing the argocd-cmp-server command) and /home/argocd/cmp-server/plugins.</li></ol><p>Once the patch file is prepared, let’s save it as “ <em>argocd-repo-server-patch.yaml</em>” and apply it to ArgoCD by patching the argocd-repo-server Deployment:</p><pre>kubectl -n argocd patch deployments/argocd-repo-server --patch-file .\argocd-repo-server-patch.yaml</pre><p>Once the patch is applied, we can see a new container is up and running inside the argocd-repo-server pod:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/908/0*RmXNTB1OpH-Ve3Lc" /></figure><p>Now, let’s create an ArgoCD application that will point to the <em>helmfile.yaml </em>(one which we defined at the start of this blog, I kept that file at path <em>helmfiles/my-helmfiles-app/helmfile.yaml</em> in my bitbucket repo). Here is the sample application manifest:</p><pre>piVersion: argoproj.io/v1alpha1<br>kind: Application<br>metadata:<br>  name: my-helmfiles-app<br>  namespace: argocd<br>spec:<br>  project: default<br>  source:<br>#mention your own repo which include the helmfile.yaml and define the path<br>    repoURL: ssh://git@bitbucket.org/your_repo_location.git<br>    targetRevision: HEAD<br>    path: helmfiles/my-helmfiles-app<br>  destination:<br>    server: https://kubernetes.default.svc<br>    namespace: my-helmfiles-app<br>  syncPolicy:<br>    automated:<br>      prune: true<br>      selfHeal: true<br>    syncOptions:<br>      - CreateNamespace=true<br>    retry:<br>      limit: 2</pre><p>Let’s save this manifest as <em>application.yaml</em> and deploy using the below command:</p><pre>kubectl apply -f .\application.yaml</pre><p>Once it’s deployed, we can see a new tile in the ArgoCD console:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/883/0*HIqN14VNMog4Ae8D" /></figure><p>We can see both helm charts have been deployed and the status is healthy and synced:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*TZK0DL3FV7kb8tzd" /></figure><p>We can see that all the pods are up and running in the “ <em>my-helmfiles-app</em> “ namespace which is mentioned in the ArgoCD application manifest:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*inSPquu7xe8lqdvd" /></figure><h3>Conclusion</h3><p>Integrating the Helmfile plugin with ArgoCD extends its capabilities, allowing for the seamless deployment of Helm charts alongside Kubernetes manifests. By leveraging the Configuration Management Plugin (CMP) support in ArgoCD, you can achieve a unified deployment experience, managing both Helm charts and other Kubernetes resources through a single interface. By following the steps outlined in this guide, you can enhance your deployment workflows, ensuring consistency, repeatability, and efficiency across your Kubernetes clusters. Embrace the power of ArgoCD and Helmfile integration to elevate your GitOps practices to new heights.</p><p>I hope this has been informative for you. Please let me know in case of any queries.</p><p><em>Originally published at </em><a href="https://virtualhackey.wordpress.com/2024/02/24/argocd-deployment-with-helmfile-plugin/"><em>http://virtualhackey.wordpress.com</em></a><em> on February 24, 2024.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=292d840ac673" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mount AWS S3 on Kubernetes pods without Access Key & Secret Key]]></title>
            <link>https://virtualcloud.medium.com/mount-aws-s3-on-kubernetes-pods-without-access-key-secret-key-b4f20440970e?source=rss-e96c43f34ad6------2</link>
            <guid isPermaLink="false">https://medium.com/p/b4f20440970e</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[aws-s3]]></category>
            <category><![CDATA[pod-mount]]></category>
            <dc:creator><![CDATA[Santosh Gupta]]></dc:creator>
            <pubDate>Sat, 14 Oct 2023 10:30:09 GMT</pubDate>
            <atom:updated>2023-10-14T21:44:25.687Z</atom:updated>
            <content:encoded><![CDATA[<p>AWS EKS is a powerful managed Kubernetes service that simplifies the deployment and management of containerized applications. It’s a popular choice for running container workloads in the cloud. AWS S3 is a highly scalable, cost-effective, durable, and secure object storage service.</p><p>As we know, containerized applications may require access to external storage for data persistence, combining the power of EKS and S3 can be a game-changer, as it allows you to seamlessly access and store data. This would be really helpful for GenAI use cases, where you need to access the large LLM models which you would like to have inside your container. Instead of copying these large models into the docker images, we can have it on s3 and mount the s3 during the container startup which would make the data available as a network share. In this blog post, we’ll explore how we can mount AWS S3 buckets on pods running on AWS EKS, without using any access key and secret key.</p><p>First of all, we need to make sure that the EKS cluster has an existing IAM OIDC provider. In order to check that, first we need to extract the cluster OIDC provider ID, we can use the below command for the same.</p><pre>aws eks describe-cluster --name cluster_name --query &quot;cluster.identity.oidc.issuer&quot; --output text</pre><p>Once you have the cluster OIDC ID extracted from the above command, run the below command to check if the IAM OIDC provider is available for the cluster.</p><pre>aws iam list-open-id-connect-providers | grep EXAMPLED539D4633E53DE1B716D3041E</pre><p>If we get an output from the above command, then you already have an IAM OIDC provider for your cluster. If no output is returned, then you must create an IAM OIDC provider for your cluster. We can use the below command to create the IAM OIDC provider.</p><pre>eksctl utils associate-iam-oidc-provider --cluster cluster_name --approve</pre><p>Once we have validated the IAM OIDC provider for the EKS cluster, we need to create an s3 bucket. I already have an s3 bucket created for this demo. Now we need to create an IAM policy that will have access to your s3 bucket. Below is a sample policy that I used.<br>NOTE: Just for demo purposes, I’m using the “s3:*” and “Resource:<strong>*”</strong> permissions, however, for actual production use cases, you should limit the permissions to resource-specific.</p><pre>{<br>    &quot;Version&quot;: &quot;2012-10-17&quot;,<br>    &quot;Statement&quot;: [<br>        {<br>            &quot;Sid&quot;: &quot;VisualEditor0&quot;,<br>            &quot;Effect&quot;: &quot;Allow&quot;,<br>            &quot;Action&quot;: [<br>                &quot;s3:*&quot;<br>            ],<br>            &quot;Resource&quot;: &quot;*&quot;<br>        }<br>    ]<br>}</pre><p>Let’s save the above policy as <strong><em>“iam-policy.json”</em></strong>, now we can use the below command to create the IAM policy -</p><pre>aws iam create-policy --policy-name &quot;aws-s3-mountpoint-policy&quot; --policy-document file://iam-policy.json</pre><p>Once the policy has been created, we need to create an IAM role and attach this policy. After that, we need to create a Kubernetes service account and annotate the IAM role arn to the service account. The easiest way to do all these is via eksctl command line. For this demo, we will be using a Kubernetes namespace <strong><em>“s3-mount”</em></strong>, we can use the below command to create the k8s namespace</p><pre>kubectl create ns s3-mount</pre><p>Once the namespace has been created, we can use the below command to create the k8s service account “<strong><em>s3-mount-sa</em></strong>”, IAM role and then annotate the IAM role arn to the service account.</p><pre>eksctl create iamserviceaccount --name s3-mount-sa --namespace s3-mount --cluster &lt;cluster_name&gt; --role-name &lt;s3 role name&gt; --attach-policy-arn &lt;arn of aws-s3-mountpoint-policy&gt; --approve</pre><p>As we can see in the below screenshot, the service account <strong><em>“s3-mount-sa”</em></strong> has been created in “<strong><em>s3-mount</em></strong>” namespace, and the IAM role “<strong><em>aws-s3-mountpoint-role</em></strong>” has been annotated.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/656/0*6GE9UH6eTc32XvHk" /></figure><p>We can validate the IAM role and policy from aws console as well.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/656/0*PATVG15wdYpQtF-a" /></figure><p><strong>NOTE</strong>: We could have attached the s3 access policy to the EKS node group role as by default same permission gets propagated to all the pods running inside that node group. However, as a best practice, we should restrict the permission to a specific pod, therefore we are using IRSA(IAM role for service account) here.</p><p>Now, we can create a pod that will use the above-mentioned service account, we can use the below manifest to create a pod that will run in a loop and won’t exit.</p><pre>apiVersion: v1<br>kind: Pod<br>metadata:<br>  name: s3-mount-pod-02<br>  namespace: s3-mount<br>spec:<br>  serviceAccountName: s3-mount-sa<br>  hostNetwork: true<br>  containers:<br>    - name: app<br>      image: amazonlinux<br>      command: [&quot;/bin/sh&quot;]<br>      args: [&quot;-c&quot;, &quot;while true; do echo $(date -u) &gt;&gt; /example/out.txt; sleep 5; done&quot;]<br>      securityContext:<br>        privileged: true</pre><p>Let’s save the above manifest as <strong><em>“s3-mount-pod.yaml”</em></strong> and use the below command to create the pod.</p><pre>kubectl apply -f .\s3-mount-pod.yaml</pre><p>Now, we can see that the pod has been created and is in a running state.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*6OyxsWSlhVI_ODaA" /></figure><p>Let’s exec into the pod in order to mount the s3 bucket, we can use the below command to do the same.</p><pre>kubectl exec -it -n s3-mount s3-mount-pod-02 -- sh -c &quot;clear; (bash || ash || sh)&quot;</pre><p>Once we are inside the pod, let’s try to create a directory, where we will mount the s3 bucket. For this demo, I’m creating a directory named “<strong><em>s3-mountpoint</em></strong>”. We can run the below command for the same -</p><p>We can validate if the folder has been created by the “<strong><em>ls -lah</em></strong>” command.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Y8QNn90r1ScYkvKt" /></figure><p>Now, we need to install a couple of tools along with the s3-mount before we can mount the s3 bucket on this pod. We installed unzip, wget, util-linux-ng, awscliv2, and mount-s3 tools.</p><pre>yum install unzip -y<br>yum install wget -y<br>yum install util-linux-ng -y<br>curl &quot;https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip&quot; -o &quot;awscliv2.zip&quot;<br>unzip awscliv2.zip<br>./aws/install<br>wget https://s3.amazonaws.com/mountpoint-s3-release/latest/x86_64/mount-s3.rpm<br>yum install -y ./mount-s3.rpm</pre><p>Once the installation is completed successfully, we are all set to mount the s3 bucket. For this demo, we have a s3 bucket “<strong><em>aws-s3-mountpoint-eks-01</em></strong>” which consists of two files. We can use the below command to mount the s3 bucket inside the pod</p><pre>mount-s3 aws-s3-mountpoint-eks-01 /s3-mountpoint</pre><p>Once the bucket is mounted, we can validate if the same s3 files are available.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*vPGfbs4cW19FCpjh" /></figure><p>These are the same files that we can see on the AWS console of s3.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1020/0*_a388JpUwsxfPlGB" /></figure><p><strong>Note</strong>: If you are using custom CNI with AWS EKS, the instance metadata service may not be reachable and you will observe a timeout issue. I have written a blog post on how to fix that, check <a href="https://medium.com/@virtualcloud/ec2-instance-metadata-is-not-accessible-from-pods-on-aws-eks-with-cilium-cni-imdsv2-1a04103dda1f">here</a></p><p>Now, let’s try to do some unzip operation and see if that works on this s3 mount. For this demo, I have uploaded a zip file “<strong><em>logs.zip</em></strong>” to s3. <br>I used the unzip command to unzip the log file and it worked as expected. You can see below it’s unzipped the log files.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1021/0*x2PL1O-vpQYxIgpE" /></figure><p>Now, If I go to the AWS S3 console, you can see the same log files are available.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1023/0*fyrY8Ti05RsKMEea" /></figure><p>This confirms that after mounting the s3 bucket, we can do some file operations as well. This will be very useful and handy for many scenarios in k8s.</p><p>Now, for an actual use case, we can create a <strong>dockerfile</strong> that will install all the required tools and mount the s3 bucket at the container startup. We can control the bucker name etc. with the env variable in order to have a generic docker image.</p><p>I hope this has been informative for you. Please let me know in case of any queries.</p><p><em>Originally published at </em><a href="https://virtualhackey.wordpress.com/2023/10/14/mount-aws-s3-on-kubernetes-pods-without-access-key-secret-key/"><em>http://virtualhackey.wordpress.com</em></a><em> on October 14, 2023.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b4f20440970e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[EC2 instance metadata is not accessible from pods on AWS EKS with Cilium CNI | IMDSv2]]></title>
            <link>https://virtualcloud.medium.com/ec2-instance-metadata-is-not-accessible-from-pods-on-aws-eks-with-cilium-cni-imdsv2-1a04103dda1f?source=rss-e96c43f34ad6------2</link>
            <guid isPermaLink="false">https://medium.com/p/1a04103dda1f</guid>
            <category><![CDATA[cilium]]></category>
            <category><![CDATA[vpc]]></category>
            <category><![CDATA[imdsv2]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[ec2]]></category>
            <dc:creator><![CDATA[Santosh Gupta]]></dc:creator>
            <pubDate>Thu, 21 Sep 2023 08:12:18 GMT</pubDate>
            <atom:updated>2023-09-21T08:12:18.729Z</atom:updated>
            <content:encoded><![CDATA[<p>AWS EKS is a managed Kubernetes service that makes it easier to deploy, manage, and scale containerized applications in AWS. It provides a highly available and secure environment for your applications. However, when running containers within an EKS cluster, you may encounter certain limitations and challenges. When we use the default VPC CNI, there is a limitation on how many pods you can run on a given instance type or how many IPs would be available to use. For some huge clusters, you may end up exhausting the IPs of the entire subnet. To overcome some of these limitations and better security and encryption, we decided to use Cilium. Cilium is designed to enhance the networking and security features of containerized environments and is often used in conjunction with Kubernetes. I will write another blog post on how to use cilium as a primary CNI instead of VPC CNI.</p><p>I was working on a project when I encountered an issue where my pod was not able to access the EC2 instance metadata service. I was using the Instance Metadata service v2 (IMDSv2) for my EC2 instances, and in order to access the instance metadata, the HopLimit should be 2. I validated my configuration, and it was set up correctly. You can check it with the below command and look for “<strong><em>MetadataOptions</em></strong>” (you will find it at the end of the command output).</p><pre>aws ec2 describe-instances --instance-id &lt;instance id name&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/643/1*XW8yQx2Op7P0YzjBr7CmAg.png" /></figure><p>If <strong><em>HttpPutResponseHopLimit</em></strong> is set to 1, you have to change it to 2 in order to access instance metadata from a pod, you can do the same with the below command –</p><pre> <br>aws ec2 modify-instance-metadata-options --instance-id &lt;instance id name&gt; --http-put-response-hop-limit 2 --http-endpoint enabled</pre><p>In my case, the HopLimit was already set to 2 even though I was not able to access the metadata. To troubleshoot it further, I exec into the pod and ran the below commands.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*njfUf8XtfYjXnvnmMd3IJg.png" /></figure><p>However, I was getting the error “<strong><em>curl: (56) Recv failure: Connection reset by peer</em></strong>” and “<strong><em>HTTP/1.1 401 Unauthorized</em></strong>“. Below a a screenshot for reference.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*y-WPEaH1utz6t8FTWePqUg.png" /></figure><p>Since I have CIlium installed with Hubble UI, I opened Hubble UI (using kubectl port-forward via the below command. I have cilium installed in the kube-system namespace, therefore, i have used the same)</p><pre>kubectl port-forward svc/hubble-ui 8081:80 -n kube-system</pre><p>From the Hubble UI, we can see that the connection is getting <strong><em>dropped from the world to the pod</em></strong>, whereas traffic was allowed from the pod to the metadata IP (169.254.169.254). Below is a screenshot for the same</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QPaTgyGxwGAKpeLYN007oQ.png" /></figure><p>Since we have not defined any Cilium policies to block the ingress traffic, I was wondering why the traffic from the EC2 metadata service is not reaching the Pods.</p><p>It seems like, since we are using different subnets and IP ranges for pods via Cilium, the metadata traffic is not routable to pods. Now, to make the pods available over the same network, I enabled the <strong><em>hostNetwork </em></strong>true for my pods. Now, when I tried accessing the metadata service from the pod, it was successful. Below is a screenshot of the same.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wgHZtXLNKmn77tEbStmoVg.png" /></figure><p>If you want to test this scenario, you can use the below pod manifest file. Once the pod is running, you can exec and check.</p><pre>apiVersion: v1<br>kind: Pod<br>metadata:<br>  name: pod-01<br>spec:<br>  hostNetwork: true<br>  containers:<br>    - name: app01<br>      image: amazonlinux<br>      command: [&quot;/bin/sh&quot;]<br>      args: [&quot;-c&quot;, &quot;while true; do echo $(date -u) &gt;&gt; /example/out.txt; sleep 5; done&quot;]<br>      securityContext:<br>        privileged: true</pre><p>I hope this has been informative for you. Please let me know in case of any queries in comment.</p><p><em>Originally published at </em><a href="https://virtualhackey.wordpress.com/2023/09/13/argocd-create-credentials-template-via-declarative-configuration/"><em>http://virtualhackey.wordpress.com</em></a><em> on September 21, 2023.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1a04103dda1f" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>