<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Re Alvarez Parmar on Medium]]></title>
        <description><![CDATA[Stories by Re Alvarez Parmar on Medium]]></description>
        <link>https://medium.com/@realz?source=rss-13f619270e3f------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*D0I_PAjiV08e1NzVxtFJDA.jpeg</url>
            <title>Stories by Re Alvarez Parmar on Medium</title>
            <link>https://medium.com/@realz?source=rss-13f619270e3f------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 01:26:57 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@realz/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[An introduction to Envoy AI Gateway]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://realz.medium.com/an-introduction-to-envoy-ai-gateway-8553b6202bbf?source=rss-13f619270e3f------2"><img src="https://cdn-images-1.medium.com/max/766/1*RF_prPzqy3Rv7bnMUjBIOA.png" width="766"></a></p><p class="medium-feed-snippet">Original article: https://blog.realvarez.com/blog/envoy-ai-gateway/</p><p class="medium-feed-link"><a href="https://realz.medium.com/an-introduction-to-envoy-ai-gateway-8553b6202bbf?source=rss-13f619270e3f------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://realz.medium.com/an-introduction-to-envoy-ai-gateway-8553b6202bbf?source=rss-13f619270e3f------2</link>
            <guid isPermaLink="false">https://medium.com/p/8553b6202bbf</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[envoy-proxy]]></category>
            <category><![CDATA[ai-gateway]]></category>
            <dc:creator><![CDATA[Re Alvarez Parmar]]></dc:creator>
            <pubDate>Wed, 08 Oct 2025 21:18:42 GMT</pubDate>
            <atom:updated>2025-10-09T17:56:45.324Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Writing made me a better architect]]></title>
            <link>https://realz.medium.com/writing-made-me-a-better-architect-79d299a24a43?source=rss-13f619270e3f------2</link>
            <guid isPermaLink="false">https://medium.com/p/79d299a24a43</guid>
            <category><![CDATA[writing]]></category>
            <category><![CDATA[software]]></category>
            <dc:creator><![CDATA[Re Alvarez Parmar]]></dc:creator>
            <pubDate>Thu, 29 May 2025 17:18:01 GMT</pubDate>
            <atom:updated>2025-05-29T17:18:01.679Z</atom:updated>
            <content:encoded><![CDATA[<p>Early in my career as a software architect, I thought the job was about having answers. Sitting in meetings, sketching on whiteboards, and confidently explaining trade-offs. Being good at thinking on my feet and making quick decisions seemed like the key skills.</p><p>But there was something missing.</p><p>Everything changed when I started writing things down. Not just meeting notes or bullet points in Confluence, but real writing. Sitting with a blank page and working through problems from first principles.</p><p>Writing revealed something uncomfortable: many ideas that seemed clear in conversation weren’t actually clear at all.</p><h3>Your Job Is to Listen and Build</h3><p>As an architect, your job is to listen to people and help them build things that work and last. Engineers come with half-formed ideas. Product managers describe features that don’t quite make sense yet. Executives paint visions that sound compelling but need structure.</p><p>It’s easy to think listening means nodding along and then providing solutions. But real listening often means writing down what you heard, then writing down what you think, then seeing where the gaps are.</p><h3>Breaking Out of Autopilot</h3><p>A few years ago, I came across something that changed how I approach complex problems. Charles Duhigg writes about “disfluency” in “Smarter Faster Better”, the counterintuitive idea that making things slightly harder can improve thinking.</p><p>When everything flows smoothly, your brain coasts. You rely on familiar patterns and miss important details. But add some friction (change your environment, use a different tool, force yourself to slow down) and something clicks. Your mind engages more deeply.</p><p>Using Obsidian creates this kind of productive friction. Instead of jumping straight into shared documents where it’s easy to fall into familiar writing patterns, stepping into a clean, distraction-free space first signals your brain: time to think differently.</p><h3>Writing Reveals What You Don’t Know</h3><p>Here’s what happens when you write about an architecture decision:</p><p>You start with what feels like a clear idea. You know why service A should call service B. The trade-offs seem obvious.</p><p>Then you try to explain it in a clean document. Suddenly you realize you’re making assumptions you haven’t validated. You discover edge cases you hadn’t considered. You catch yourself using vague words like “handles” without explaining what that actually means.</p><p>Writing doesn’t just capture thinking, it improves it.</p><h3>Better Decisions, Better Systems</h3><p>When you write regularly, architecture decisions get better. Not because you become smarter, but because writing forces you to examine reasoning before committing to it.</p><p>You catch flawed assumptions early. You spot missing requirements before they become problems. You understand the true constraints instead of the ones you assumed.</p><p>The systems that emerge are more thoughtful because the thinking behind them is more thorough.</p><h3>Words Outlast Conversations</h3><p>Perhaps the best part is that everything written down becomes knowledge that persists. When that engineer who chose the database leaves the company, or when the product manager who defined requirements switches teams, the reasoning survives.</p><p>Future maintainers don’t just inherit code, they inherit understanding.</p><p>Writing makes you a better architect because it makes you a better thinker. The friction of putting thoughts into structured notes forces clarity that conversations never quite achieve.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=79d299a24a43" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Running Android on Kubernetes]]></title>
            <link>https://realz.medium.com/running-android-on-kubernetes-be73b940833f?source=rss-13f619270e3f------2</link>
            <guid isPermaLink="false">https://medium.com/p/be73b940833f</guid>
            <category><![CDATA[android]]></category>
            <category><![CDATA[mobile-app-development]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Re Alvarez Parmar]]></dc:creator>
            <pubDate>Sat, 02 Nov 2024 00:04:00 GMT</pubDate>
            <atom:updated>2025-05-06T17:40:18.651Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TS4hFTmDBHk6qRvLmwkKlg.png" /></figure><h3>Running Android on Amazon EKS</h3><p>Teams that develop software meant to run on Android often need virtual Android devices for development and testing. These virtual devices, aka Android Emulators, enable development and testing workflows without requiring a physical Android device. While they are flexible and more cost-effective, managing multiple instances of emulators at scale can become cumbersome. This post shows how to run virtual Android devices using <a href="https://github.com/google/android-cuttlefish">Cuttlefish</a> and Amazon EKS.</p><p><em>Cuttlefish</em> is a configurable virtual Android device that runs on Linux x86 and ARM hardware. Unlike traditional Android emulators, which require dedicated hardware or virtual machines, Cuttlefish runs in a container. This is a significant advantage as it means that you can create multiple virtual Android devices on the same hardware. It allows you to maximize hardware utilization.</p><p>And, you’d definitely want to maximize hardware utilization. Cuttlefish (or any other Android emulator) requires that the underlying host machine has Kernel-based Virtual Machine (KVM). In AWS, KVM is only available on bare metal instances, which aren’t cheap. At the time of writing, the cheapest bare metal instance is C6g.metal. Running one C6g will set you back $2.176 per hour in the US East (N. Virginia) AWS Region.</p><p>How can you maximize hardware utilization? The answer is Kubernetes. Couple Cuttlefish with Kubernetes and you can orchestrate workloads that require virtual Android devices easily. This combination:</p><ul><li>Simplifies creation and termination of virtual Android devices</li><li>Provides a consistent environment for development and testing</li><li>Enforces resource allocation (using Kubernetes requests and limits)</li></ul><p>I won’t recommend this solution to anyone that’s looking to run just one virtual device. It will be too expensive to operate and there may be cheaper SaaS-based options available (although I don’t have any experience with their pricing model). The viability of implementation increases as the number of required virtual devices grows.</p><p>This solution becomes attractive when:</p><ul><li>Scale is required</li><li>SaaS solutions don’t meet security needs</li><li>Low latency connection is crucial for performance (especially in CI environments)</li></ul><h3>What Do You Need?</h3><p>To run Cuttlefish on EKS, you’ll need the following:</p><ul><li>An EKS cluster</li><li>A managed node group or Karpenter with a node pool (with bare metal instances, such as, c6g.metal)</li><li>Cuttlefish container image</li></ul><h3>Cuttlefish Container Image</h3><p>Prebuilt Cuttlefish container mages are available on <a href="https://github.com/google/android-cuttlefish/actions/workflows/artifacts.yaml?query=event%3Apush">GitHub</a>. For my setup, I chose to build the container image instead of using a prebuilt one. The process is straightforward:</p><pre>git clone https://github.com/google/android-cuttlefish.git<br>cd android-cuttlefish/docker<br>./image-builder.sh</pre><p>Once the image builds, you’ll have a cuttlefish-orchestration image on your machine. Keep in mind that the architecture of your image will be the same as the host machine. That is, if you run the image-builder.sh script on an ARM machine, you&#39;ll get an ARM image. If you want to run Cuttlefish on x86, use an x86 machine to build the image. Alternatively, you can use <a href="https://docs.docker.com/build/building/multi-platform/">Docker Multi-platform builds</a> to build x86 and ARM images on the same machine.</p><p>Besides the container image, you need two additional files to run a virtual Android device using Cuttlefish: the device image and the associated host package. The detailed instructions to get these files are available in <a href="https://source.android.com/docs/devices/cuttlefish/get-started">Cuttlefish documentation</a>. Here’s a summary:</p><ol><li>Go to <a href="http://ci.android.com/">http://ci.android.com/</a></li><li>Enter a branch name, if required (the default is aosp-main)</li><li>Under aosp_arm for ARM or aosp_cf_x86_64_phone for x86, click on the download button</li><li>In the Details panel that appears when you click the download button, switch to the “Artifacts” tab</li><li>On the artifacts panel, download these two files:</li><li>The device image file: aosp_cf_x86_64_phone-img-xxxxxx.zip</li><li>The host package: cvd-host_package.tar.gz</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HKqC5m91SV6n2s70fyOjgQ.png" /><figcaption>Figure 1. The Android Continuous Integration site <a href="http://ci.android.com/">http://ci.android.com/</a>.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*C52Lpkb2uKEhS6e3f_Su_Q.png" /><figcaption>Figure 2. The “Artifacts” tab in the Details panel.</figcaption></figure><p>Now, we must create a custom Cuttlefish container image that includes the device image and host package files.</p><p>Place the two files in a clean directory. In that directory, create a Dockerfile:</p><pre>FROM cuttlefish-orchestration<br><br>WORKDIR /opt/cuttlefish-android<br><br>COPY aosp_cf_arm64_only_phone-img-*.zip .<br><br>ADD cvd-host_package.tar.gz .<br><br>RUN apt-get install -y zip<br><br>RUN unzip ./aosp_cf_arm64_only_phone-img-*.zip<br><br>RUN rm aosp_cf_arm64_only_phone-img-*.zip</pre><p>Finally, build the image and push it to a container registry:</p><pre>docker build . -t cuttlefish-custom:v1</pre><h3>Configuring the Node</h3><p>Cuttlefish requires that the host machine has vhost_vsock and vhost_net kernel modules loaded. These kernel modules are part of the virtio framework and enable efficient I/O and network communication between the virtual Android device and the host machine.</p><p>To load these modules on an existing node, run:</p><pre>sudo modprobe vhost_vsock vhost_net</pre><p>You can enable these kernel modules at node startup. If you use EKS managed node groups, you’ll have to create a launch template. The instructions are <a href="https://docs.aws.amazon.com/eks/latest/userguide/launch-templates.html-template-user-data">here</a>.</p><p>For Karpenter, you’ll include the commands to load these kernel modules in the EC2NodeClass configuration. Here’s an example of an ARM NodePool and EC2NodeClass :</p><pre>apiVersion: karpenter.sh/v1<br>kind: NodePool<br>metadata:<br>  name: default<br>spec:<br>  template:<br>    spec:<br>      requirements:<br>        - key: kubernetes.io/arch<br>          operator: In<br>          values: [&quot;arm64&quot;]<br>        - key: kubernetes.io/os<br>          operator: In<br>          values: [&quot;linux&quot;]<br>        - key: karpenter.sh/capacity-type<br>          operator: In<br>          values: [&quot;on-demand&quot;]<br>        - key: karpenter.k8s.aws/instance-category<br>          operator: In<br>          values: [&quot;c6g.metal&quot;]<br>      nodeClassRef:<br>        name: cuttlefish<br>---<br>apiVersion: karpenter.k8s.aws/v1<br>kind: EC2NodeClass<br>metadata:<br>  name: cuttlefish<br>spec:<br>  amiFamily: AL2<br>  blockDeviceMappings:<br>    - deviceName: /dev/xvda<br>      ebs:<br>        volumeSize: 100Gi # Recommended <br>        volumeType: gp3<br>  role: &quot;KarpenterNodeRole&quot; # replace with your cluster name<br>  subnetSelectorTerms:<br>    - tags:<br>        karpenter.sh/discovery: &quot;EKS_CLUSTER_NAME&quot; # replace with your cluster name<br>  securityGroupSelectorTerms:<br>    - tags:<br>        karpenter.sh/discovery: &quot;EKS_CLUSTER_NAME&quot; # replace with your cluster name<br>  userData: |<br>  sudo modprobe vhost_vsock vhost_net</pre><h3>Deploying Cuttlefish</h3><p>With the infrastructure setup, now you can run Cuttlefish Pods. Instead of running individual Pods, I chose to create a Deployment, which can be scaled up and down on demand.</p><p>Cuttlefish containers need access to /dev/kvmto access the host’s KVM functionality. These Pods also need to run in privileged mode to be able to access /dev/kvm. While this provides the necessary capabilities, it also poses security risks and should be used cautiously in production environments. Please implement security measures to limit access to these Pods.</p><p>Here’s the manifest with a nodeSelector, which schedules the Pod on a c6g.metal instance:</p><pre>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: android-arm<br>  labels:<br>    app: android<br>spec:<br>  replicas: 1<br>  selector:<br>    matchLabels:<br>      app: android<br>  template:<br>    metadata:<br>      labels:<br>        app: android<br>    spec:<br>      nodeSelector:<br>        node.kubernetes.io/instance-type: c6g.metal<br>      containers:<br>      - name: android<br>        image: 12345678910.dkr.ecr.us-west-2.amazonaws.com/cuttlefish:graviton-v2 # Replace this with your image and tag<br>        command: [&quot;/opt/cuttlefish-android/bin/launch_cvd&quot;]<br>        args:<br>        - &quot;--cpus&quot;<br>        - &quot;4&quot;<br>        - &quot;--memory_mb&quot;<br>        - &quot;4096&quot;<br>        securityContext:<br>          privileged: true<br>        env:<br>        - name: HOME<br>          value: &quot;/opt/cuttlefish-android&quot;<br>        volumeMounts:<br>        - name: kvm<br>          mountPath: /dev/kvm<br>      volumes:<br>      - name: kvm<br>        hostPath:<br>          path: /dev/kvm</pre><p>Once you deploy this manifest, you’ll have an instance of Cuttlefish Android running in your cluster.</p><h3>Connecting to a Virtual Device</h3><p>Once a Cuttlefish Pod is running, you can connect to it using adb, Android Studio, or WebRTC.</p><p>There are two ways to connect to this Pod:</p><ol><li>Using Kubernetes port forwarding</li><li>Directly connecting to the Pod</li></ol><p>While connecting to the virtual device using Kubernetes port forwarding is easy and simple, it’s not going to be a smooth experience from a performance standpoint. Connecting to a remote virtual device is sensitive to network latency. The farther you are from the Pod, the worse will be your experience. In my experience, if the latency between the machine running Android Studio and the Pod is more than 30ms, stepping through breakpoints can take up to 30 seconds.</p><p>Nonetheless, port forwarding is still useful for troubleshooting and when the dev machine and Pod are on different networks. To create a port forwarding, run:</p><pre>kubectl port-forward deployments/android-arm 6520</pre><p>In another terminal window, connect to the virtual device using adb:</p><pre>adb connect localhost:6520</pre><p>Now, you can interact with the device. For instance, to open a shell connection, run:</p><pre>adb shell</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DnvrCKZHFKOLGynHyITy_A.png" /><figcaption>Figure 3. Accessing Android shell using `adb shell`</figcaption></figure><h3>Direct Connection</h3><p>The connection provided by kubectl port forwarding will not be fast enough for debugging. It’s best to have a direct connection from your machine to the Pod. Since all Pods in EKS get a VPC IP address, you can connect directly to the Pod as long as you can reach a Pod’s IP address.</p><p>For example, I created a Windows EC2 instance in the same VPC as my Kubernetes cluster. From this instance, I can directly access the Pod. My Android Pod’s IP is 192.168.178.175. I can go to https://192.168.178.175:8443 to connect to the device over WebRTC.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/609/1*Rp5jm02GBR_VT_U1BLF8mg.png" /><figcaption>Figure 4. Connecting to the virtual device using WebRTC.</figcaption></figure><p>Having direct connectivity from your machine to the Pod is a luxury not every environment can afford. In most organizations, developer machines and Kubernetes clusters are in separate networks. In these scenarios, you’ll have to utilize a Transit Gateway to be able to connect to a Pod directly. Another option is to create EC2-based development machines that are on the same VPC.</p><h3>Scaling Virtual Devices</h3><p>The benefit of this setup is that you can easily go from one virtual device to hundreds of virtual devices without worrying about the underlying infrastructure. For instance, I can create another virtual device by scaling the deployment:</p><pre>kubectl scale deployment android-arm --replicas 2</pre><h3>Determining Pod addresses is a problem yet to be solved</h3><p>A downside of this implementation is that developers must know the IP address of the virtual device’s Pod. This problem is more accentuated in environments where developers don’t have access to the Kubernetes cluster.</p><p>Since, Pod IP addresses are not static, there are several challenges that are yet to be solved:</p><ul><li>Pod IP addresses can change when Pods are rescheduled or restarted, making it difficult for developers to maintain consistent connections to the virtual devices.</li><li>Developers may need to update their configurations with new IP addresses, which is time-consuming and error-prone.</li></ul><p>I am still exploring options like a VPN, <a href="https://mirrord.dev/">mirrord</a>, an Ingress (for WebRTC only.) If you have a suggestion, I’d love to know.</p><p>You can always create an internal load balancer (NLB) for every developer Pod. However, it comes with additional cost. In this implementation, every user has their own virtual device (backed by a Deployment with replicas=1). For instance, developerA.myorg.com routes to a Pod dedicated to DeveloperA. Should DeveloperA’s Pod fail and get restarted, she won’t need to determine the IP address of the new Pod.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/621/1*Rp9-NT1M6fVH0DEpDJB-gw.png" /><figcaption>Figure 5. Using an internal NLB to route traffic to user Pods</figcaption></figure><p>Another option: build a layer of abstraction (perhaps using Backstage and ArgoCD) that gives users the ability to create Cuttlefish Pods and provides IP addresses without the user needing access to the cluster.</p><h3>Turn Servers</h3><p><strong>Update: </strong><a href="https://github.com/google/android-emulator-container-scripts/blob/master/js/turn/README.MD">Turn servers</a> are the perfect solution for exposing devices on private networks. Here’s a <a href="https://medium.com/@lemonchoismarceau/running-concurent-android-cuttlefish-jobs-webrtc-connection-08d79a1303dc">Medium post</a> that explains this.</p><h3>Conclusion</h3><p>Using Cuttlefish with Kubernetes offers a scalable implementation for Android development and testing workflows. It allows you to maximize hardware utilization, especially when there’s a need to run multiple virtual Android devices. This architecture also gives you full control over your virtual devices. You can customize the device to suit your specific needs.</p><p>However, there are still challenges to address, particularly in terms of Pod IP address management and developer access. There’s still room for improvement when it comes to device discovery.</p><p>In spite of these challenges, the combination presents a promising approach for teams looking to streamline Android development, especially for non-interactive needs.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=be73b940833f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Add topology label to your Kubernetes Pods]]></title>
            <link>https://itnext.io/add-topology-label-to-your-kubernetes-pods-8c6fb4c1f891?source=rss-13f619270e3f------2</link>
            <guid isPermaLink="false">https://medium.com/p/8c6fb4c1f891</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[kyverno]]></category>
            <category><![CDATA[topology]]></category>
            <category><![CDATA[controller]]></category>
            <dc:creator><![CDATA[Re Alvarez Parmar]]></dc:creator>
            <pubDate>Tue, 20 Feb 2024 21:02:03 GMT</pubDate>
            <atom:updated>2024-02-23T15:13:55.854Z</atom:updated>
            <content:encoded><![CDATA[<h3>Add topology labels to your Kubernetes Pods</h3><p>I had a customer ask me today: <em>How can I add Availability Zone and Region labels to my Pods?</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ciEGM3AQIMKc_ND7" /><figcaption>Photo by <a href="https://unsplash.com/@onice?utm_source=medium&amp;utm_medium=referral">Joey Huang</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>In most managed Kubernetes services, like Amazon EKS, Kubernetes nodes have a label topology.kubernetes.io/zone. This label contains the name of the Availability Zone (AZ) in which the node is located. One possible way of adding an AZ label to Pods would be to get the topology label from the associated node.</p><p>Kubernetes doesn’t add topology labels to Pods currently, so I started exploring if there’s a controller that could add the labels. We already had Kyverno installed in the cluster, so if we could use Kyverno, it would save everyone a lot of time. Luckily, the solution was easier than we thought.</p><p><a href="https://kyverno.io/">Kyverno</a> is a policy engine designed for Kubernetes. It simplifies writing Kubernetes policies. Its distinct advantage lies in its user-friendly approach. Unlike other Kubernetes policy like OPA, you don’t have to learn Rego or another language to author policies.</p><h3>Kyverno Mutate Rules</h3><p>Kyverno finds its primary utility in performing typical operations such as <em>mutation</em> or <em>validation</em> upon the creation of resources like Pods, Services, or Deployments. For example, <a href="https://github.com/kyverno/policies/blob/main/other/add-labels/add-labels.yaml">this policy</a> lets you automatically add a label when you create a Pod.</p><p>The problem is that the AZ in which the Pod will land is unknown until the Pod gets scheduled and a node is assigned. To address this, we needed a mutating rule that adds label Pods after they get scheduled.</p><p>We solved this issue by using <a href="https://kyverno.io/docs/high-availability/#background-controller">Kyverno Background Controller</a> with Pod/binding as a resource for Kyverno to act upon, instead of the Pod itself. Then, we could use Kyverno’s mutateExistingOnPolicyUpdate attribute, which enabled us to update an existing resource.</p><p>Let’s see this in action.</p><h3>Configure Kyverno</h3><p>There are two things you need to make this policy work:</p><ol><li>Configure Kyverno to monitor Pod/binding resource</li><li>Give Kyverno backgroundController the permissions to update Pods</li></ol><p>If you’ve installed Kyverno with default configuration, chances are your installation excludes Pod/binding resource type. You can check if this resource type exists in the resourceFilter list in kyverno configMap in your cluster.</p><p>Here’s a screenshot of kyverno configMap in my cluster after installing Kyverno using the official Helm chart:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZJwKWVccDKvUJnt6vvZtsQ.png" /><figcaption>Kyverno default ConfigMap</figcaption></figure><p>Notice that [Pod/binding, *.*] is included in the resourceFilters. We need to remove the Pod/binding resource from the list.</p><p>You can edit the configMap to remove or generate a values.yaml to use Helm to reconfigure Kyverno. The command below generates a values.yaml:</p><pre>curl &quot;https://raw.githubusercontent.com/kyverno/kyverno/main/charts/kyverno/values.yaml&quot; | sed  &#39;s/Pod\/binding//g&#39; &gt; values.yaml</pre><h3>Update Kyverno backgroundController:core role</h3><p>Next, we need to give the Kyverno the permission to update Pods. Kyverno Background Controller uses kyverno-background-controller service account. This service account is bound to kyverno:background-controller ClusterRole.</p><p>By default, this role doesn’t have the permission to update or patch Pods. We have to patch the role to add this permission. You can add the permission in-place by using:</p><pre>kubectl \<br>  --namespace kyverno \<br>  edit clusterrole kyverno:background-controller:core</pre><p>I patched kyverno:background-controller:core ClusterRole to add permissions. Be careful as running the command below may override any custom permissions you&#39;ve added over Kyverno&#39;s default installation:</p><pre>kubectl patch clusterrole kyverno:background-controller:core \<br>  --type=&#39;json&#39; -p=&#39;[{&quot;op&quot;: &quot;add&quot;, &quot;path&quot;: &quot;/rules/6/resources&quot;, &quot;value&quot;:[&quot;pod&quot;, &quot;configmaps&quot;, &quot;secrets&quot;, &quot;resourcequotas&quot;, &quot;limitranges&quot;]}]&#39;</pre><p>This is how the ClusterRole looks like:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*Z0MiocU2wI3AYVHVw2qgBA.png" /><figcaption>Permit Kyverno to update or patch Pods</figcaption></figure><p>With the permissions setup, the last thing you need is the Kyverno policy that copies topology labels from nodes and adds them to Pods.</p><h3>Kyverno Policy</h3><p>This policy below copies the topology.kubernetes.io/zone and topology.kubernetes.io/region labels from the node to the Pod. Create a policy manifest:</p><pre>cat &lt;&lt;EOF &gt; add-topology-labels-to-pods.yaml<br>apiVersion: kyverno.io/v1<br>kind: ClusterPolicy<br>metadata:<br>  annotations:<br>    pod-policies.kyverno.io/autogen-controllers: none<br>    policies.kyverno.io/subject: Pod<br>    policies.kyverno.io/title: Add scheduled Node&#39;s topology labels to a Pod<br>    policies.kyverno.io/description: &gt;-<br>      Containers running in Pods may sometimes need topology labels. This policy watches for then mutates<br>      the /binding subresource of a Pod to add topology.kubernetes.io/zone and topology.kubernetes.io/region labels on Pods. <br>  name: add-topology-labels-to-pod<br>spec:<br>  admission: true<br>  background: false<br>  failurePolicy: Fail<br>  mutateExistingOnPolicyUpdate: true<br>  rules:<br>  - context:<br>    - name: node<br>      variable:<br>        default: &quot;&quot;<br>        jmesPath: request.object.target.name<br>    - apiCall:<br>        jmesPath: metadata.labels.&quot;topology.kubernetes.io/zone&quot; || &#39;empty&#39;<br>        method: GET<br>        urlPath: /api/v1/nodes/{{node}}<br>      name: ZoneLabel<br>    - apiCall:<br>        jmesPath: metadata.labels.&quot;topology.kubernetes.io/region&quot; || &#39;empty&#39;<br>        method: GET<br>        urlPath: /api/v1/nodes/{{node}}<br>      name: RegionLabel<br>    match:<br>      any:<br>      - resources:<br>          kinds:<br>          - Pod/binding<br>    mutate:<br>      patchStrategicMerge:<br>        metadata:<br>          labels:<br>            topology.kubernetes.io/zone: &#39;{{ ZoneLabel }}&#39;<br>            topology.kubernetes.io/region: &#39;{{ RegionLabel }}&#39;<br>      targets:<br>      - apiVersion: v1<br>        kind: Pod<br>        name: &#39;{{ request.object.metadata.name }}&#39;<br>        namespace: &#39;{{ request.object.metadata.namespace }}&#39;<br>    name: add-topology-labels-to-pod<br>    skipBackgroundRequests: true<br>  validationFailureAction: Audit<br>EOF</pre><p>Apply the policy:</p><pre>kubectl apply -f add-topology-labels-to-pods.yaml</pre><p>This policy first gets the region and zone labels from the associated node using Kubernetes API. Then it adds the labels to Pods. Please see <a href="https://kyverno.io/docs/writing-policies/external-data-sources/#variables-from-kubernetes-api-server-calls">Kyverno documentation</a> to learn more about getting data from Kubernetes API Server calls.</p><h3>Validate labels</h3><p>Create a new Pod and you should see the new labels.</p><pre>kubectl run test --image=nginx<br>kubectl get pods test --show-labels</pre><p>You should see topology labels in the output:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2XT6B07lpnAylLJHTB7T0g.png" /><figcaption>Topology labels in a Pod</figcaption></figure><h3>Conclusion</h3><p>Adding Availability Zone and Region labels to Pods can be a crucial requirement in many scenarios. You can use labels to limit the Pods that Prometheus scrapes metrics from, which can result in cost savings.</p><p>By utilizing Kyverno’s mutate rules and background controller, we could dynamically add topology labels to Pods after they were scheduled. This approach streamlined the process, eliminating the need to rely on a complex solution.</p><p>Reference: <a href="https://github.com/kyverno/policies/issues/887">https://github.com/kyverno/policies/issues/887</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8c6fb4c1f891" width="1" height="1" alt=""><hr><p><a href="https://itnext.io/add-topology-label-to-your-kubernetes-pods-8c6fb4c1f891">Add topology label to your Kubernetes Pods</a> was originally published in <a href="https://itnext.io/">ITNEXT</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using Docker to build images in Kubernetes 1.24 and above]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://blog.devops.dev/using-docker-to-build-images-in-kubernetes-1-24-and-above-249196e05d34?source=rss-13f619270e3f------2"><img src="https://cdn-images-1.medium.com/max/2600/1*LYGT0ib5Mj4VWpEW2fdv0A.jpeg" width="4128"></a></p><p class="medium-feed-link"><a href="https://blog.devops.dev/using-docker-to-build-images-in-kubernetes-1-24-and-above-249196e05d34?source=rss-13f619270e3f------2">Continue reading on DevOps.dev »</a></p></div>]]></description>
            <link>https://blog.devops.dev/using-docker-to-build-images-in-kubernetes-1-24-and-above-249196e05d34?source=rss-13f619270e3f------2</link>
            <guid isPermaLink="false">https://medium.com/p/249196e05d34</guid>
            <category><![CDATA[containerd]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[platform-engineering]]></category>
            <dc:creator><![CDATA[Re Alvarez Parmar]]></dc:creator>
            <pubDate>Fri, 08 Dec 2023 00:36:01 GMT</pubDate>
            <atom:updated>2024-03-03T17:12:35.431Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Cert-Manager: Encrypt Kubernetes service to service communication with self-signed certificates]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://itnext.io/cert-manager-encrypt-kubernetes-service-to-service-communication-with-self-signed-certificates-e5bf276a14fe?source=rss-13f619270e3f------2"><img src="https://cdn-images-1.medium.com/max/1920/1*Hgyi_r6R3ly5v8-cDFtK_g.png" width="1920"></a></p><p class="medium-feed-snippet">A simple setup to secure in-cluster traffic in Kubernetes </p><p class="medium-feed-link"><a href="https://itnext.io/cert-manager-encrypt-kubernetes-service-to-service-communication-with-self-signed-certificates-e5bf276a14fe?source=rss-13f619270e3f------2">Continue reading on ITNEXT »</a></p></div>]]></description>
            <link>https://itnext.io/cert-manager-encrypt-kubernetes-service-to-service-communication-with-self-signed-certificates-e5bf276a14fe?source=rss-13f619270e3f------2</link>
            <guid isPermaLink="false">https://medium.com/p/e5bf276a14fe</guid>
            <category><![CDATA[encryption]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[cybersecurity]]></category>
            <dc:creator><![CDATA[Re Alvarez Parmar]]></dc:creator>
            <pubDate>Fri, 23 Jun 2023 23:44:56 GMT</pubDate>
            <atom:updated>2023-06-27T06:42:48.398Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Run more pods per GPU with NVIDIA Multi-Instance GPU]]></title>
            <link>https://itnext.io/run-more-pods-per-gpu-with-nvidia-multi-instance-gpu-d4f7fb07c9b5?source=rss-13f619270e3f------2</link>
            <guid isPermaLink="false">https://medium.com/p/d4f7fb07c9b5</guid>
            <category><![CDATA[amazon-eks]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[gpu]]></category>
            <category><![CDATA[nvidia]]></category>
            <dc:creator><![CDATA[Re Alvarez Parmar]]></dc:creator>
            <pubDate>Tue, 23 May 2023 23:35:57 GMT</pubDate>
            <atom:updated>2023-08-03T06:29:22.051Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/960/0*wtZEzMz_c85WJyez" /></figure><p>Photo by <a href="https://unsplash.com/es/@pawel_czerwinski?utm_source=ghost&amp;utm_medium=referral&amp;utm_campaign=api-credit">Pawel Czerwinski</a> / <a href="https://unsplash.com/?utm_source=ghost&amp;utm_medium=referral&amp;utm_campaign=api-credit">Unsplash</a></p><p>Machine learning (ML) workloads require tremendous amounts of computing power. Of all the infrastructure components that ML applications require, GPUs are the most critical. With their parallel processing capabilities, GPUs have revolutionized domains like deep learning, scientific simulations, and high-performance computing. But not all ML workloads require the same amount of resources. Traditionally, ML scientists have had to pay for a full GPU regardless of whether they needed it.</p><p>In 2020, NVIDIA introduced <a href="https://www.nvidia.com/en-us/technologies/multi-instance-gpu/">Multi-Instance GPU (MIG)</a>. This feature partitions a GPU into multiple, smaller, fully isolated GPU instances. It is particularly beneficial for workloads that do not fully saturate the GPU’s compute capacity. It allows users to run multiple workloads in parallel on a single GPU to maximize resource utilization. This post shows how to use MIG on <a href="https://aws.amazon.com/eks/">Amazon EKS</a>.</p><h3>NVIDIA Multi-Instance GPU</h3><p>MIG is a feature of NVIDIA GPUs based on <a href="https://www.nvidia.com/en-us/data-center/ampere-architecture/">NVIDIA Ampere architecture</a>. It allows you to maximize the value of NVIDIA GPUs and reduce resource wastage. Using MIG, you can partition a GPU into smaller GPU instances, called MIG devices. Each MIG device is fully isolated with its own high-bandwidth memory, cache, and compute cores. You can create <em>slices</em> to control the amount of memory and number of compute resources per MIG device.</p><p>MIG gives you the ability to fine tune the amount of GPU resources your workloads get. This feature provides guaranteed quality of service (QoS) with deterministic latency and throughput to ensure workloads can safely share GPU resources without interference.</p><p>NVIDIA has extensive <a href="https://docs.nvidia.com/datacenter/tesla/mig-user-guide/?ref=blog.realvarez.com">documentation explaining the inner workings of MIG</a>, so I won’t repeat the information here.</p><h3>Using MIG with Kubernetes</h3><p>Many customers I work with choose Kubernetes to operate their ML workloads. Kubernetes provides a powerful and scalable scheduling mechanism, making it easier to orchestrate workloads on a cluster of virtual machines. Kubernetes also has a vibrant community building tools like <a href="https://www.kubeflow.org/?ref=blog.realvarez.com">Kubeflow</a> that make it easier to build, deploy, and manage ML pipelines.</p><p>MIG on Kubernetes is still an underutilized feature due its complexity. NVIDIA documentation is partly to be blamed here. While NVIDIA’s documentation explains how MIG works extensively (albeit with a lot of repetition), it is lacking when it comes to providing resources like tutorials and example for MIG deployments and configurations on Kubernetes. What makes matters worse is that to use MIG on Kubernetes, you have to install a bunch of resources such as the NVIDIA driver, NVIDIA container runtime, and device plugins.</p><p>Thankfully, <a href="https://github.com/NVIDIA/gpu-operator?ref=blog.realvarez.com">NVIDIA GPU Operator</a> automates the deployment, configuration, and monitoring GPU resources in Kubernetes. It simplifies installing the components necessary for using MIG on Kubernetes. Its key features are:</p><ul><li>Automatic GPU driver installation and management</li><li>Automatic GPU resource allocation and scheduling</li><li>Automatic GPU monitoring and alerting</li><li>Support for NVIDIA Container Runtime</li><li>Support for NVIDIA Multi-Instance GPU (MIG)</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Fow0WvcYCnlsI5gzARaojA.png" /><figcaption>NVIDIA GPU Operator</figcaption></figure><p>The operator installs the following components:</p><ul><li><strong>NVIDIA device driver</strong></li><li><a href="https://github.com/kubernetes-sigs/node-feature-discovery?ref=blog.realvarez.com"><strong>Node Feature Discovery</strong></a>. Detects hardware features on the node</li><li><a href="https://github.com/NVIDIA/gpu-feature-discovery?ref=blog.realvarez.com"><strong>GPU Feature Discovery</strong></a>. Automatically generates labels for the set of GPUs available on a node</li><li><a href="https://github.com/NVIDIA/dcgm-exporter?ref=blog.realvarez.com"><strong>NVIDIA DCGM</strong></a> Exporter. Exposes GPU metrics exporter for <a href="https://prometheus.io/?ref=blog.realvarez.com">Prometheus</a> leveraging <a href="https://developer.nvidia.com/dcgm?ref=blog.realvarez.com">NVIDIA DCGM</a></li><li><a href="https://github.com/NVIDIA/k8s-device-plugin?ref=blog.realvarez.com"><strong>Device Plugin</strong></a>. Exposes the number of GPUs on each nodes of your cluster, keeps track of the health of your GPUs, and runs GPU enabled containers in your Kubernetes cluster</li><li><strong>Device Plugin Validator</strong>. Runs a series of validations via InitContainers for each component and writes out results under /run/nvidia/validations</li><li><a href="https://github.com/NVIDIA/nvidia-container-toolkit?ref=blog.realvarez.com"><strong>NVIDIA Container Toolkit</strong></a></li><li><strong>NVIDIA CUDA Validator</strong></li><li><strong>NVIDIA Operator Validator</strong>.Validates driver, toolkit, CDA, and NVIDIA Device Plugin</li><li><a href="https://github.com/NVIDIA/mig-parted?ref=blog.realvarez.com"><strong>NVIDIA MIG Manager</strong></a>. <a href="https://github.com/NVIDIA/mig-parted/tree/main?ref=blog.realvarez.com">MIG Partition Editor</a> for NVIDIA GPUs in Kubernetes clusters</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*iFIB2vy5uoWe2fau.png" /></figure><h3>NVIDIA GPU Operator on Amazon EKS</h3><p>While NVIDIA GPU Operator makes it easy to use GPUs in Kubernetes, some of its components require newer versions of the Linux kernel and operating system. Amazon EKS provides a Linux AMI for GPU workloads that pre-installs NVIDIA drivers and container runtime. At the time of writing, this AMI provides Linux kernel 5.4. However, NVIDIA GPU Operator Helm Charts default are configured for Ubuntu or Centos 8. Therefore, making NVIDIA GPU Operator work on Amazon EKS is not as simple as executing:</p><pre>helm install gpu-operator nvidia/gpu-operator</pre><h3>Walkthrough</h3><p>Let’s start the walkthrough by installing NVIDIA GPU Operator. You’d need an EKS cluster with a node group made up of EC2 instances that come with NVIDIA GPUs (P4 or P5 instances). Here’s an <a href="https://eksctl.io/?ref=blog.realvarez.com">eksctl</a> manifest if you’d like to create a new cluster for this walkthrough:</p><pre>apiVersion: eksctl.io/v1alpha5<br>kind: ClusterConfig<br>metadata:<br>  name: p4d-cluster<br>  region: eu-west-1<br>managedNodeGroups:<br>  - name: demo-gpu-workers<br>    instanceType: p4d.24xlarge<br>    minSize: 1<br>    desiredCapacity: 1<br>    maxSize: 1<br>    volumeSize: 200</pre><p>I am going to use a P4d.24XL instance for this demo. Each P4d.24XL EC2 instance has 8 NVIDIA A100 Tensor core GPUs. Each A100 GPU has 40GB memory. By default, you can only run one GPU workload per GPU with each pod getting a 40GB GPU memory slice. This means you are limited to running 8 pods per instance.</p><p>Using MIG, you can partition each GPU to run multiple pods per GPU. On a P4d.24XL node with 8 A100 GPUs, you can create 7 5GB A100 slices per GPU. As a result, you can run 7*8 = 56 pods concurrently. Alternatively, you can create 24 pods with 10GB slices, or 16 pods with 20GB slices, or 8 pods with 40GB slices.</p><p>Since the latest versions of the components that the operator installs are incompatible with the current version of Amazon EKS optimized accelerated Amazon Linux AMI, I have manually set the versions of incompatible components to a version that works with the AMI.</p><p>Install NVIDIA GPU Operator:</p><pre>helm repo add nvidia https://helm.ngc.nvidia.com/nvidia \<br>   &amp;&amp; helm repo update<br><br>helm upgrade gpuo \<br>    nvidia/gpu-operator \<br>    --set driver.enabled=true \<br>    --set mig.strategy=mixed \<br>    --set devicePlugin.enabled=true \<br>    --set migManager.enabled=true \<br>    --set migManager.WITH_REBOOT=true \<br>    --set toolkit.version=v1.13.1-centos7 \<br>    --set operator.defaultRuntime=containerd \<br>    --set gfd.version=v0.8.0 \<br>    --set devicePlugin.version=v0.13.0 \<br>    --set migManager.default=all-balanced</pre><p>View the resources created by GPU Operator:</p><pre>$ kubectl get pods<br>NAME                                                  READY   STATUS      RESTARTS   AGE<br>gpu-feature-discovery-529vf                           1/1     Running     0          20m<br>gpu-operator-9558bc48-z4wlh                           1/1     Running     0          3d20h<br>gpuo-node-feature-discovery-master-7f8995bd8b-d6jdj   1/1     Running     0          3d20h<br>gpuo-node-feature-discovery-worker-wbtxc              1/1     Running     0          20m<br>nvidia-container-toolkit-daemonset-lmpz8              1/1     Running     0          20m<br>nvidia-cuda-validator-bxmhj                           0/1     Completed   1          19m<br>nvidia-dcgm-exporter-v8p8f                            1/1     Running     0          20m<br>nvidia-device-plugin-daemonset-7ftt4                  1/1     Running     0          20m<br>nvidia-device-plugin-validator-pf6kk                  0/1     Completed   0          18m<br>nvidia-mig-manager-82772                              1/1     Running     0          18m<br>nvidia-operator-validator-5fh59                       1/1     Running     0          20m</pre><p>GPU Feature Discovery adds labels to the node that help Kubernetes schedule workloads that require a GPU. You can see the label by describing the node:</p><pre>$ kubectl describe node <br>...<br>Allocatable:<br>  attachable-volumes-aws-ebs:  39<br>  cpu:                         95690m<br>  ephemeral-storage:           18242267924<br>  hugepages-1Gi:               0<br>  hugepages-2Mi:               0<br>  memory:                      1167644256Ki<br>  nvidia.com/gpu:              8<br>  pods:                        250<br>...</pre><p>Pods can request a GPU by specifying GPU in resources. Here’s a sample pod manifest:</p><pre>kind: Pod<br>metadata:<br>  name: dcgmproftester-1<br>spec:<br>  restartPolicy: &quot;Never&quot;<br>  containers:<br>  - name: dcgmproftester11<br>    image: nvidia/samples:dcgmproftester-2.0.10-cuda11.0-ubuntu18.04<br>    args: [&quot;--no-dcgm-validation&quot;, &quot;-t 1004&quot;, &quot;-d 30&quot;]  <br>    resources:<br>      limits:<br>         nvidia.com/gpu: 1    <br>    securityContext:<br>      capabilities:<br>        add: [&quot;SYS_ADMIN&quot;]</pre><p>We won’t create a pod that uses a full GPU because we already know that it will work out of the box. Instead, we’ll create pods that use partial GPUs.</p><h3>Creating MIG partitions on Kubernetes</h3><p>NVIDIA provides two strategies for exposing MIG partitioned devices on a Kubernetes node. In <strong>single strategy</strong>, a node only exposes a single type of MIG devices across all GPUs. Whereas, <strong>Mixed strategy</strong> allows you to create multiple different sized MIG devices across all of a node’s GPUs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*ysr5eVxl0IknZ4Nw7MLp-Q.png" /><figcaption>MIG device naming</figcaption></figure><p>Using MIG single strategy, you can create similar sized MIG devices. On a P4d.24XL, you can create 56 1g.5gb slices, or 24 2g.10gb slices, or 16 3g.20gb slices, or a 1 4g.40gb or 7g.40gb slice.</p><p>Mixed strategy will allow you to create a few 1g.5gb along with a few 2g.10gb and 3g.20gb slices. It is useful when your cluster has workloads with varying GPU resource requirements.</p><h3>Create MIG devices with single strategy</h3><p>Let’s create a single strategy and see how to use it with Kubernetes. NVIDIA GPU Operator makes it easy to create MIG partitions. To configure partitions, all you have to do is label the node. MIG manager runs as daemonset on all nodes. When it detects node labels, it will use to create MIG devices.</p><p>Label a node to create 1g.5gb MIG devices across all GPUs (replace $NODE with a node in your cluster):</p><pre>kubectl label nodes $NODE nvidia.com/mig.config=all-1g.5gb --overwrite</pre><p>Two things will happen once you label the node this way. First, the node will no longer advertise any full GPUs and the nvidia.com/gpu label will be set to 0. Second, your node will advertise 56 1g.5gb MIG devices.</p><pre>$ kubectl describe node $NODE<br>...<br>  nvidia.com/gpu:              0<br>  nvidia.com/mig-1g.5gb:       56<br>...</pre><p><em>Please note that it may take a few seconds for the change to take effect. The node will have a label </em><em>nvidia.com/mig.config.state=pending when the change is still in progress. Once MIG manager completes partitioning, the label will be set to </em><em>nvidia.com/mig.config.state=success</em>.</p><p>We can now create a deployment that uses MIG devices.</p><p>Create a deployment:</p><pre>cat &lt;&lt; EOF &gt; mig-1g-5gb-deployment.yaml<br>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: mig1.5<br>spec:<br>  replicas: 1<br>  selector:<br>    matchLabels:<br>      app: mig1-5<br>  template:<br>    metadata:<br>      labels:<br>        app: mig1-5<br>    spec:<br>      containers:<br>      - name: vectoradd<br>        image: nvidia/cuda:8.0-runtime<br>        command: [&quot;/bin/sh&quot;, &quot;-c&quot;]<br>        args: [&quot;nvidia-smi &amp;&amp; tail -f /dev/null&quot;]<br>        resources:<br>          limits:<br>            nvidia.com/mig-1g.5gb: 1<br>EOF</pre><p>You should now have a pod running that consumes 1x 1g.5gb MIG device.</p><pre>$ kubectl get deployments.apps mig1.5<br>NAME     READY   UP-TO-DATE   AVAILABLE   AGE<br>mig1.5   1/1     1            1           1h</pre><p>Let’s scale the deployment to 100 replicas. Only 56 pods will get created because the node can only accommodate 56 1g.5gb MIG devices (8 GPUs * 7 MIG slices per GPU) .</p><p>Scale the deployment:</p><pre>kubectl scale deployment mig1.5 --replicas=100</pre><p>Notice that only 56 pods become available:</p><pre>$ kubectl get deployments.apps mig1.5<br>NAME     READY    UP-TO-DATE   AVAILABLE   AGE<br>mig1.5   56/100   100          56          1h</pre><p>Exec into one of the containers and run nvidia-smi to view allocated GPU resources.</p><pre>kubectl exec &lt;YOUR MIG1.5 POD&gt; -ti -- nvidia-smi</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*r8IdITogcSI6JM9d.png" /></figure><p>As you can see, this pod has only 5gb GPU memory.</p><p>Let’s scale the deployment down to 0:</p><pre>kubectl scale deployment mig1.5 --replicas=0</pre><h3>Create MIG devices with mixed strategy</h3><p>In single strategy, all MIG devices were 1g.5gb devices. Now let’s slice the GPUs so that each node supports multiple MIG device configurations. MIG manager uses a configmap to store MIG configuration. When we labeled the node with all-1g.5gb, MIG partition editor uses the configmap to determine the partition scheme.</p><pre>$ kubectl describe configmaps default-mig-parted-config<br>...<br><br>  all-1g.5gb:<br>    - devices: all<br>      mig-enabled: true<br>      mig-devices:<br>        &quot;1g.5gb&quot;: 7<br><br>...</pre><p>This configmap also includes other profiles like all-balanced. The all-balanced profile creates 2x 1g.10gb, 1x 2g.20gb, and 1x 3g.40gb MIG devices per GPU. You can create your own custom profile by editing the configmap.</p><p>all-balanced MIG profile:</p><pre>$ kubectl describe configmaps default-mig-parted-config<br><br>...<br>  all-balanced:<br>    - device-filter: [&quot;0x20B010DE&quot;, &quot;0x20B110DE&quot;, &quot;0x20F110DE&quot;, &quot;0x20F610DE&quot;]<br>      devices: all<br>      mig-enabled: true<br>      mig-devices:<br>        &quot;1g.5gb&quot;: 2<br>        &quot;2g.10gb&quot;: 1<br>        &quot;3g.20gb&quot;: 1<br>...</pre><p>Let’s label the node to use all-balanced MIG profile:</p><pre>kubectl label nodes $NODE nvidia.com/mig.config=all-balanced --overwrite</pre><p>Once the node has nvidia.com/mig.config.state=success label, describe the node and you&#39;ll see multiple MIG devices listed in the node:</p><pre>$ kubectl describe node $NODE<br><br>...<br><br>  nvidia.com/mig-1g.5gb:       16<br>  nvidia.com/mig-2g.10gb:      8<br>  nvidia.com/mig-3g.20gb:      8<br><br>...</pre><p>With all-balanced profile, this P4d.24XL node can run 16x 1g.5gb, 8x 2g.20gb, and 8x 3g.20gb pods.</p><p>Let’s test this out by creating two additional deployments. One with pods that use one 2g.10gb MIG device and another using 3g.10gb MIG device.</p><p>Create deployments:</p><pre>cat &lt;&lt; EOF &gt; mig-2g-10gb-and-3g.20gb-deployments.yaml<br>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: mig2-10<br>spec:<br>  replicas: 1<br>  selector:<br>    matchLabels:<br>      app: mig2-10<br>  template:<br>    metadata:<br>      labels:<br>        app: mig2-10<br>    spec:<br>      containers:<br>      - name: vectoradd<br>        image: nvidia/cuda:8.0-runtime<br>        command: [&quot;/bin/sh&quot;, &quot;-c&quot;]<br>        args: [&quot;nvidia-smi &amp;&amp; tail -f /dev/null&quot;]<br>        resources:<br>          limits:<br>            nvidia.com/mig-2g.10gb: 1<br>---<br><br>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: mig3-20<br>spec:<br>  replicas: 1<br>  selector:<br>    matchLabels:<br>      app: mig3-20<br>  template:<br>    metadata:<br>      labels:<br>        app: mig3-20<br>    spec:<br>      containers:<br>      - name: vectoradd<br>        image: nvidia/cuda:8.0-runtime<br>        command: [&quot;/bin/sh&quot;, &quot;-c&quot;]<br>        args: [&quot;nvidia-smi &amp;&amp; tail -f /dev/null&quot;]<br>        resources:<br>          limits:<br>            nvidia.com/mig-3g.20gb: 1<br>EOF<br><br>kubectl apply -f mig-2g-10gb-and-3g.20gb-deployments.yaml</pre><p>Once pods from these deployments are running, scale all three deployments to 20 replicas:</p><pre>kubectl scale deployments mig1.5 mig2-10 mig3-20 --replicas=20</pre><p>Let’s see how many of these replicas start running:</p><pre>kubectl get deployments</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*daUtNSl6UpR-fgNS.png" /></figure><p>Let’s see how much GPU memory a 3g.20gb pod receives:</p><pre>kubectl exec mig3-20-&lt;pod-id&gt; -ti -- nvidia-smi</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FUkLDoWs2BEcRl1FC3aQ3g.png" /></figure><p>As expected, this pod has 20GB GPU memory allocated.</p><h3>Cleanup</h3><p>Delete the cluster and the node group:</p><pre>eksctl delete cluster &lt;CLUSTER_NAME&gt;</pre><h3>Conclusion</h3><p>This post shows how to partition GPUs using NVIDIA Multi-Instance GPU and using it with Amazon EKS. Using MIG on Kubernetes can be complex, but NVIDIA GPU Operator simplifies the process of installing MIG dependencies and partitioning.</p><p>By leveraging the capabilities of MIG and the automation provided by the NVIDIA GPU Operator, ML scientists can optimize their GPU usage, run more workloads per GPU, and achieve better resource utilization in their scalable ML applications. With the ability to run multiple applications per GPU and tailor the allocation of resources, you can optimize your ML workloads to achieve higher scalability and performance in your applications.</p><h3>Resources</h3><ul><li><a href="https://docs.nvidia.com/datacenter/cloud-native/mig/mig.html">Multi-Instance GPU — NVIDIA Cloud Native Technologies documentation</a></li><li><a href="https://docs.nvidia.com/datacenter/tesla/mig-user-guide/">NVIDIA Multi-Instance GPU User Guide :: NVIDIA Tesla Documentation</a></li><li><a href="https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/overview.html">NVIDIA GPU Operator Overview — NVIDIA Cloud Native Technologies documentation</a></li><li><a href="https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/gpu-operator-mig.html">GPU Operator with MIG — NVIDIA Cloud Native Technologies documentation</a></li><li><a href="https://docs.google.com/document/d/1mdgMQ8g7WmaI_XVVRrCvHPFPOMCm5LQD5JefgAh6N8g/edit">Supporting MIG in Kubernetes</a></li><li><a href="https://docs.google.com/document/d/1Dxx5MwG_GiBeKOuMNwv4QbO8OqA7XFdzn7fzzI7AQDg/edit#">Challenges Supporting MIG in Kubernetes</a></li><li><a href="https://docs.google.com/document/d/1bshSIcWNYRZGfywgwRHa07C0qRyOYKxWYxClbeJM-WM/edit#">Steps to Enable MIG Support in Kubernetes</a></li></ul><p><em>Originally published at </em><a href="https://blog.realvarez.com/get-more-out-of-gpus-with-nvidia-multi-instance-gpu/"><em>https://blog.realvarez.com</em></a><em> on May 23, 2023.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d4f7fb07c9b5" width="1" height="1" alt=""><hr><p><a href="https://itnext.io/run-more-pods-per-gpu-with-nvidia-multi-instance-gpu-d4f7fb07c9b5">Run more pods per GPU with NVIDIA Multi-Instance GPU</a> was originally published in <a href="https://itnext.io/">ITNEXT</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Use containerd to handle k8s.gcr.io deprecation]]></title>
            <link>https://itnext.io/use-containerd-to-handle-k8s-gcr-io-deprecation-dba870b9df5?source=rss-13f619270e3f------2</link>
            <guid isPermaLink="false">https://medium.com/p/dba870b9df5</guid>
            <category><![CDATA[solutions]]></category>
            <category><![CDATA[containerd]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[docker]]></category>
            <dc:creator><![CDATA[Re Alvarez Parmar]]></dc:creator>
            <pubDate>Wed, 05 Apr 2023 06:53:01 GMT</pubDate>
            <atom:updated>2023-04-16T15:00:40.196Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/960/0*PCwmttjWVXh5dzwT" /></figure><p>Photo by <a href="https://unsplash.com/@mullins?utm_source=ghost&amp;utm_medium=referral&amp;utm_campaign=api-credit">David Mullins</a> / <a href="https://unsplash.com/?utm_source=ghost&amp;utm_medium=referral&amp;utm_campaign=api-credit">Unsplash</a></p><p>The Kubernetes community is getting ready for yet another major change. Until the fall 2022, k8s.gcr.io container registry hosted many Kubernetes community-managed container images like Cluster Autoscaler, metrics-server, cluster-proportional-autoscaler. The &quot;gcr.io&quot; is Google Cloud Registry. In order to be vendor neutral, the community is moving away from using Google Cloud Registry to host container images.</p><p>As a result, starting March 20th, traffic from the old k8s.gcr.io registry is being redirected to registry.k8s.io. The older k8s.gcr.io will remain functioning for sometime but it is eventually getting deprecated.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*1uwNdHx0hUOO4CWw.jpeg" /></figure><h3>What’s the impact?</h3><p>The change that occurred six days after Pi day is unlikely to cause major problems. There are some edge cases. But unless you operate in an airgapped or highly restrictive environment that applies strict domain name access controls, you won’t notice the change.</p><p>This doesn’t mean that there’s no action required. Now is the time to scan code repositories and clusters for usage of the old registry. Failing to act will result in cluster components failing.</p><p>Once the old registry goes away, <strong>Kubernetes will not be able to create new Pods</strong> (unless image is cached) if the container uses an image hosted on k8s.gcr.io.</p><h3>What do you need to change?</h3><p>Cluster owners and development teams have to ensure they are not using any images stored in the old registry. The change is fairly simple. It’s pretty simple, really.</p><p>You need to change your manifests to use registry.k8s.io container registry.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*JXbRGEMMTQ5fSeu_.png" /></figure><p>You can find out which Pods use the old registry using kubectl:</p><pre>kubectl get pods --all-namespaces -o jsonpath=&quot;{.items[*].spec.containers[*].image}&quot; |\0;31;37M0;31;38m<br>tr -s &#39;[[:space:]]&#39; &#39;\n&#39; |\<br>sort |\<br>uniq -c | grep -i gcr.io</pre><p>Here are the Pods in my test cluster that use the old registry:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/942/0*jAGX-AKVJoTNvGBI.jpeg" /></figure><p>These are the at-risk Pods. I’ll have to update the container registry used in the Pods.</p><p>When hunting for references to old registry, be sure to include containers that may not be currently running in your cluster. Don’t forget to scan code repositories.</p><h3>What if I don’t control the workloads?</h3><p>One of my colleagues raised an intriguing question. Is there’s a way to handle this change at a cluster level? He had a valid concern. Many large enterprises might not be able to implement this change in time before the community sunsets k8s.gcr.io.</p><p>I work with many customers that manage large Kubernetes clusters, but have little control over the workloads that get deployed into the cluster. Some of these clusters are shared by hundreds of development teams. The burden is on Central Platform Engineering teams to disperse this information to individual dev teams (who are busy writing code and not checking Kubernetes news!).</p><p>So, what can these teams do to make sure when the old registry finally croaks, they don’t get paged for in the middle of the night for ErrImagePull and ImagePullBackOff errors?</p><p>Turns out you can use containerd to handle this redirection at node level. Let’s find out how.</p><h3>Using mirrors in containerd</h3><p>Ever since Dockerhub started rate limiting image pulls, many have opted to store images in local registries. Mirrors save network bandwidth, reduce image pull time, and don’t rate-limit.</p><p>You can configure registry.k8s.io as a mirror to k8s.gcr.io in containerd. This configuration will <strong>automatically pull images from </strong><strong>registry.k8s.io</strong> whenever a Pod uses an image stored in k8s.gcr.io.</p><p>On your worker node, append these lines in the containerd config file at /etc/containerd/config.toml:</p><pre>[plugins.&quot;io.containerd.grpc.v1.cri&quot;.registry]<br> config_path = &quot;/etc/containerd/certs.d&quot;</pre><p>The final file on an Amazon EKS cluster looks like this:</p><pre>version = 2<br>root = &quot;/var/lib/containerd&quot;<br>state = &quot;/run/containerd&quot;<br><br>[grpc]<br>address = &quot;/run/containerd/containerd.sock&quot;<br><br>[plugins.&quot;io.containerd.grpc.v1.cri&quot;.containerd]<br>default_runtime_name = &quot;runc&quot;<br><br>[plugins.&quot;io.containerd.grpc.v1.cri&quot;]<br>sandbox_image = &quot;602401143452.dkr.ecr.us-west-2.amazonaws.com/eks/pause:3.5&quot;<br><br>[plugins.&quot;io.containerd.grpc.v1.cri&quot;.containerd.runtimes.runc]<br>runtime_type = &quot;io.containerd.runc.v2&quot;<br><br>[plugins.&quot;io.containerd.grpc.v1.cri&quot;.containerd.runtimes.runc.options]<br>SystemdCgroup = true<br><br>[plugins.&quot;io.containerd.grpc.v1.cri&quot;.cni]<br>bin_dir = &quot;/opt/cni/bin&quot;<br>conf_dir = &quot;/etc/cni/net.d&quot;<br><br>[plugins.&quot;io.containerd.grpc.v1.cri&quot;.registry]<br>config_path = &quot;/etc/containerd/certs.d&quot;</pre><p>Next, create a directory called k8s.gcr.io and a hosts.toml file inside it:</p><pre>mkdir -p /etc/containerd/certs.d/k8s.gcr.io<br><br>cat &lt;&lt; EOF &gt; /etc/containerd/certs.d/k8s.gcr.io/hosts.toml<br>server = &quot;https://k8s.gcr.io&quot;<br><br>[host.&quot;https://registry.k8s.io&quot;]<br>capabilities = [&quot;pull&quot;, &quot;resolve&quot;]<br>EOF</pre><p>Image pull requests to k8s.gcr.io will now be sent to registry.k8s.io.</p><p>Restart containerd and kubelet for the change to take effect.</p><pre>systemctl restart containerd kubelet</pre><p>Let’s validate that images are indeed getting pulled from the new registry. I added an entry to my /etc/hosts file to break k8s.gcr.io:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*miTMzj40m8E9IMzX.png" /></figure><p>Containerd can no longer pull an image from k8s.gcr.io</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Hg2nIpA1GDonG22C.jpeg" /></figure><p>I can tell ctr to use the mirror by specifying the -hosts-dir parameter:</p><pre>ctr images pull --hosts-dir &quot;/etc/containerd/certs.d&quot; k8s.gcr.io/pause:latest</pre><p>This time the operation succeeds.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*b2RFDYKYWH_BSVcz.jpeg" /></figure><p>Any Pods I create now onwards will use the new registry even though the manifests reference old registry. Here’s a test using a pause container.</p><pre>kubectl create deployment pause --image k8s.gcr.io/pause</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*YAJeKVwWmNzXp9fI.jpeg" /></figure><p>Perfect! Kubernetes could create Pods even though I blocked k8s.gcr.io on the node.</p><h3>What’s the best way to implement this in production?</h3><p>In my little demo, I changed a single node in the cluster. What about the rest of the nodes?</p><p>There are three ways you can use to implement this change on every node in your cluster:</p><ol><li>The easiest way is to use a daemonset to change to containerd config.toml and add hosts.toml file. IBM cloud has shared <a href="https://raw.githubusercontent.com/IBM-Cloud/kube-samples/master/containerd-registry-daemonset-example?ref=re-alvarez-parmars-blog">this</a> on Github</li><li>You can use <a href="https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html?ref=re-alvarez-parmars-blog">EC2 user data</a> or <a href="https://aws.amazon.com/systems-manager/?ref=re-alvarez-parmars-blog">AWS Systems Manager</a> to make this change when a node gets created</li><li>You can <a href="https://docs.aws.amazon.com/eks/latest/userguide/eks-ami-build-scripts.html?ref=re-alvarez-parmars-blog">create your own AM</a>I</li></ol><h3>What if I use Docker as runtime?</h3><p>Starting Kubernetes version 1.24, containerd is the only runtime available in Amazon EKS AMIs. If you have an edge case that requires using Docker, there&#39;s still hope.</p><p>Docker also has support for registry mirrors. <a href="https://docs.docker.com/registry/recipes/mirror/?ref=re-alvarez-parmars-blog#configure-the-docker-daemon">Here’s</a> the documentation page you need.</p><h3>Don’t rely on stop gaps</h3><p>While the solution included in this post works, I recommend only using as a safety measure. The main reason is that you’ll need to customize the Amazon EKS AMI or create your own AMI to use it.</p><p>You’ll have less operational overhead if you can simply use EKS AMIs as is. The best way to handle this registry deprecation is to update manifests.</p><p>Oh, and by the way, you can also use mirrors to enforce pull through cache.</p><p><em>Originally published at </em><a href="https://blog.realvarez.com/use-containerd-to-handle-k8s-gcr-io-deprecation/"><em>https://blog.realvarez.com</em></a><em> on April 5, 2023.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dba870b9df5" width="1" height="1" alt=""><hr><p><a href="https://itnext.io/use-containerd-to-handle-k8s-gcr-io-deprecation-dba870b9df5">Use containerd to handle k8s.gcr.io deprecation</a> was originally published in <a href="https://itnext.io/">ITNEXT</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Goodreads Review]]></title>
            <link>https://realz.medium.com/goodreads-review-ef5b5c892019?source=rss-13f619270e3f------2</link>
            <guid isPermaLink="false">https://medium.com/p/ef5b5c892019</guid>
            <category><![CDATA[tech-book-review]]></category>
            <category><![CDATA[book-review]]></category>
            <dc:creator><![CDATA[Re Alvarez Parmar]]></dc:creator>
            <pubDate>Thu, 24 Nov 2022 22:53:08 GMT</pubDate>
            <atom:updated>2022-11-24T22:56:44.622Z</atom:updated>
            <content:encoded><![CDATA[<h3>Book Review: Chip War — Chris Miller</h3><p>Chris Miller’s biography of semiconductors appears on the bookshelves amid wars in Ukrain and Syria, tensions in South China Sea, and a period of political turmoil with price of bread skyrocketing in Europe. Earlier in 2022, the Biden government committed $52 billion in subsidies to build new chip factories in the U.S. However, by Halloween the momentum had faded. Between the slowdown in consumer demand for electronic gadgets, new U.S. restrictions on sales to China and rising inflation, chip makers were left uncertain about their future production plans.</p><p>The timing for this book release couldn’t be any better. Chip war tells the history of the invention without which we might still as well be living in dark ages. It’s relevance in modern warfare. The fragility of its global supply chain. Miller thesis is about the confluence of geopolitics and technology.</p><p>Chip war starts at the end of WWII. The world was still recovering from the massive shock and headed into a battle of nations to outcompete with innovation. Miller paints a picture of the early days of semiconductors, which mostly revolved around replacing vaccum tubes with transistors. Shortly after the war ended in 1945, Bell Labs brought a group of scientists to find solid state replacement for the fragile glass vaccum tubes amplifiers. One of them was William Shockley who’d be a key contributor to the discovery of transistors aka semiconductors.</p><p><em>A state-of-the-art computer called ENIAC, built for the U.S. Army at the University of Pennsylvania in 1945 to calculate artillery trajectories, had eighteen thousand vacuum tubes. It took up an entire room</em></p><p>Following Shockley’s invention of the semiconductors at Bell Labs, Texas Instruments, a company focused on seismic industry, purchased the patent license for $25,000. Transistors soon began replacing vaccum tubes in computers. In 1958, Jack Kilby joined TI and figured out a way to string different transistors together to create an <em>integrated circuit.</em> Kilby will be awarded Nobel Prize in Physics.</p><p>Meanwhile Shockley’s opened his own laboratory in Palo Alto making it the first silicon focused company in what is now known as Silicon Valley. While Shockley attracted brilliant scientists, his employee retention skills needed improvement. A group of eight former Shockley employees created Fairchild Semiconductor. Among which was Gordon Moore, who’s known for <a href="https://en.wikipedia.org/wiki/Moore&#39;s_law">Moore’s law</a> and who’d form Intel along with Robert Noyce, another of the “<a href="https://en.wikipedia.org/wiki/Traitorous_eight">traitorous eight</a>”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/354/1*gcZp4BmiTyf-EZlaKIZYuw.png" /><figcaption>The traitorous eight</figcaption></figure><p>In 1962, the MIT lab chose Fairchild chips for the Apollo program.</p><p><em>“The computer that eventually took Apollo 11 to the moon weighed seventy pounds and took up about one cubic foot of space, a thousand times less than the University of Pennsylvania’s ENIAC computer that had calculated artillery trajectories during World War II.”</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JDsc-CEh7eUHLgpL7P_aYA.png" /></figure><p>The success of Fairchild chips pushed Texas Instruments to find applications of their semiconductors in defense departments. TI had figured out ways to produce smaller and smaller circuits using photolithography (printing with light).</p><p>In the 1960s, as computer sales grew, TI and Fairchild started turning chips into a mass market product.</p><p>The USSR meanwhile severely lagged behind the US in semiconductors know how. The Kremlin chose the patch of copying American innovations instead of developing their own technology. This strategy put Russia on a path of failure. They never gained significant expertise in chip designing and manufacturing. Using espionage and KGB they continued to infiltrate American knowledge but their chips could never compete with Fairchild or TI’s products.</p><p>America’s lead in semiconductor expertise gave the country a massive leg up over Russia. While US army built precise missiles in 1970s, Russian missiles were missing targets consistently in Syria even in the 2010s. Soviet strategy of copying chips ultimately resulted in American dominance worldwide.</p><p>US strategy was to encourage chip fabrication in Japan, Singapore, Taiwan, and Hong Kong. Japan’s Sony saw huge success in building electronics using chips developed in Silicon Valley and Texas. America sold chips to Japan, and Japan sold goods to Americans. To save on labor costs, Fairchild started sending its semiconductors to Hong Kong for final assembly. Within a decade every American semiconductor including TI and Motorola had foreign assembly facilities. American investment pulled these countries away from communism. The millions of dollars of investment transformed economies and bolstered political stability across south-east Asia.</p><p><em>“In Hong Kong, electronics manufacturing created more jobs than any sector except textiles. In Malaysia, semiconductor production boomed in Penang, Kuala Lumpur, and Melaka, with new manufacturing jobs providing work for many of the 15 percent of Malaysian workers who had left farms and moved to cities between 1970 and 1980.”</em></p><p>In 1968, Noyce and Moore left Fairchild and founded Intel. Their first product was DRAM, the first generalized computer chip. Intel made a fortune selling chips for mass market calculators and mainframe computers while mostly ignoring the government. By 1972, Pentagon was pouring money into new weapon systems that capitalized on America’s advantage in microelectronics.</p><p><em>In the early 1960s, it had been possible to claim the Pentagon had created Silicon Valley. In the decade since, the tables had turned. The U.S. military lost the war in Vietnam, but the chip industry won the peace that followed, binding the rest of Asia, from Singapore to Taiwan to Japan, more closely to the U.S. via rapidly expanding investment links and supply chains.</em></p><p>Offshoring chip fabrication helped Japanese and Taiwanese companies develop crucial expertise. By 1986, Japan had overtaken America in the number of chips produced. Japan was also supplying 70 percent of the world’s lithography equipment. American companies started becoming increasingly dependent on overseas manufacturers. As Japanese firms grabbed market share, CEOs of America’s biggest chip firms spent more and more time in Washington, lobbying Congress and the Pentagon. Most American companies fled the DRAM market by the late 70s.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/1*KVhLJkmQmkwlAP7bUTQ3TQ.png" /></figure><p>After a decade of pain, Micron outcompeted Japanese Toshiba and Fujitsu by aggressively cutting costs and improving manufacturing practices. Intel meanwhile abandoned the DRAM business and established a stronghold on the market for IBM PC chips. None of the Japanese chip giants could replicate Intel’s pivot to microprocessors or its mastery of the PC ecosystem. Only one Japanese firm, NEC, really tried, but it never won more than a tiny share of the microprocessor market.</p><figure><img alt="Image" src="https://cdn-images-1.medium.com/max/1024/0*kIadl4zXvt-7AjkZ.jpeg" /></figure><p>The brutal DRAM competition provided an opening for Samsung. With strong financial support from the South Korean government and banks, Samsung bet its future on semiconductors. US chipmakers hoped the Koreans would undercut Japanese, who were dumping chips in the US. The Americans didn’t intend for the deal to benefit Korean firms, but they were happy to see anyone but Japan producing the chips they needed.</p><p>In 1998 South Korean firms had overtaken Japan as the world’s largest producers of DRAM, while Japan’s market share fell from 90 percent in the late 1980s to 20 percent by 1998.</p><p>The late 80s brought a new revolution to chip manufacturing. Taiwanese government provided funds for the founding of TSMC, which will go on to become the most valuable semiconductor company. It served <a href="https://en.wikipedia.org/wiki/Fabless_manufacturing">fabless</a> chip companies like NVIDIA, AMD, and later Apple. Its customers designed chip that TSMC manufactured.</p><p>Miller writes that initially, the Chinese chip industry suffered from the lack of engineers and China’s lack of participation in the global economy. China’s new leader Deng Xiaoping put semiconductors at the center of his agenda, hoping that China could use advances in semiconductors to help develop new weapons systems, consumer electronics, and computers. In 2000, China replicated TSMC model and founded SMIC attracting $1.5 Billion from Motorola, Goldman Sachs, and Toshiba. Chinas leadership wanted to be the ones developing core technologies.</p><p><em>China’s problem isn’t only in chip fabrication. In nearly every step of the process of producing semiconductors, China is staggeringly dependent on foreign technology, almost all of which is controlled by China’s geopolitical rivals — Taiwan, Japan, South Korea, or the United States.</em></p><p>In spite of millions of dollars SIMC was a distant competitor to TSMC. Huawei gave China government their first success story. Huawei followed Samsung strategy of producing quality versions at lower cost, and selling them to the world, grabbing international market share from international rivals.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*Cdr7dzXNMayC-I7Pe6IysA.png" /></figure><p><em>A Wall Street Journal review of total subsidies provided by the Chinese government reached a figure of $75 billion, in the form of subsidized land, state-backed credit, and tax deductions at a scale far above what most Western companies get from their governments.</em></p><p>Huawei started as a phone company and later added telecom equipments to its portfolio. The company spread rapidly across the world. As it grew, incumbent Western firms selling telecom equipment were forced to merge or pushed out of the market. Canada’s Nortel went bankrupt. Alcatel-Lucent sold its operations to Finland’s Nokia.</p><p>Huawei was poised for success with the emergence of 5G, even though it heavily relied on US-made chips.</p><p><em>With Huawei’s design arm proving itself world-class, it wasn’t hard to imagine a future in which Chinese chip design firms were as important customers of TSMC as Silicon Valley giants. If the trends of the late 2010s were projected forward, by 2030 China’s chip industry might rival Silicon Valley for influence. This wouldn’t simply disrupt tech firms and trade flows. It would also reset the balance of military power.</em></p><p>Miller’s excellent analysis shows the great strides China has made towards gaining technical equivalency with the US. Major players like IBM have had to please the Chinese government to have access to a billion customers. In 2018, the Wall Steet Journal reported that Intel first disclosed the Spectre and Meltdown flaws to customers, including Chinese tech companies, before notifying the U.S. government, a fact that only intensified Pentagon officials’ concern about their declining influence over the chip industry. While China is pouring billions of dollars into its chip industry, many believe that America has fallen decisively behind.</p><p>The US government has tried to slow down the worldwide deployment of telecom equipment produced by Huawei and ZTE using sanctions as a weapon. The US, it turns out, has escalation dominance when it comes to severing supply chains.</p><p>Thanks to TSMC, Samsung, and other companies willing to produce their chips, companies like Qualcomm, NVIDIA, Apple, and others could focus on their core strengths in managing spectrum and in semiconductor design.</p><p>NVIDIA found a huge market in artificial intelligence and machine learning as cloud computing gained ground. Intel, who had dominated the PC and server market until then, suffered massive setbacks when it lost the opportunity to put an Intel chip on Apple devices and cloud companies began creating chips based on Arm architecture. Intel’s 7nm and 10nm manufacturing process is now delayed by at least half a decade, even as Samsung and TSMC have charged ahead.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ef5b5c892019" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Reduce Amazon EKS cost by scaling node groups to zero]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://blog.devops.dev/reduce-amazon-eks-cost-by-scaling-node-groups-to-zero-41dce9db50ef?source=rss-13f619270e3f------2"><img src="https://cdn-images-1.medium.com/max/1920/1*hSdvS85abfbMjRC-WdpACA.jpeg" width="1920"></a></p><p class="medium-feed-snippet">Amazon EKS just released the support for Kubernetes version 1.24. This version supports scaling managed node groups to zero.</p><p class="medium-feed-link"><a href="https://blog.devops.dev/reduce-amazon-eks-cost-by-scaling-node-groups-to-zero-41dce9db50ef?source=rss-13f619270e3f------2">Continue reading on DevOps.dev »</a></p></div>]]></description>
            <link>https://blog.devops.dev/reduce-amazon-eks-cost-by-scaling-node-groups-to-zero-41dce9db50ef?source=rss-13f619270e3f------2</link>
            <guid isPermaLink="false">https://medium.com/p/41dce9db50ef</guid>
            <category><![CDATA[platform-engineering]]></category>
            <category><![CDATA[amazon-eks]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[cost-optimization]]></category>
            <dc:creator><![CDATA[Re Alvarez Parmar]]></dc:creator>
            <pubDate>Tue, 22 Nov 2022 22:00:17 GMT</pubDate>
            <atom:updated>2022-12-09T11:22:34.395Z</atom:updated>
        </item>
    </channel>
</rss>