<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ajay Tripathy on Medium]]></title>
        <description><![CDATA[Stories by Ajay Tripathy on Medium]]></description>
        <link>https://medium.com/@ajaytripathy?source=rss-f640f2f76fb9------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*tVtIPTTSWkkD1y8U.jpg</url>
            <title>Stories by Ajay Tripathy on Medium</title>
            <link>https://medium.com/@ajaytripathy?source=rss-f640f2f76fb9------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 09:22:18 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ajaytripathy/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Find an optimal set of nodes for a Kubernetes cluster]]></title>
            <link>https://medium.com/kubecost/find-an-optimal-set-of-nodes-for-a-kubernetes-cluster-751e4c3b40b7?source=rss-f640f2f76fb9------2</link>
            <guid isPermaLink="false">https://medium.com/p/751e4c3b40b7</guid>
            <category><![CDATA[optimization]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <dc:creator><![CDATA[Ajay Tripathy]]></dc:creator>
            <pubDate>Wed, 19 Feb 2020 12:34:30 GMT</pubDate>
            <atom:updated>2020-02-19T18:42:15.936Z</atom:updated>
            <content:encoded><![CDATA[<p>One of the most impactful ways to reduce spend on Kubernetes infrastructure is to make sure your clusters are optimally sized for the workloads they run. Working with many teams over the past year we have seen that it’s not so obvious how to arrive at an optimally-sized cluster, even in autoscaling environments. In response, we are publicly launching a new tool that we’ve used to help teams recently! While implementing this solution with users so far we’ve seen <strong>savings in the range of 25–60%</strong>, even having a major impact in autoscaling environments.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*K1RPYKZ_AgCc93nj.png" /></figure><h3>How it works</h3><p>This new tool generates context-aware cluster sizing recommendations after analyzing Kubernetes metrics on historical workloads and cloud billing data. As a user, you only need to provide the context of the cluster, i.e. whether it is being used for development, production, or high-availability workloads. We then recommend cluster configurations, which optimize for the respective balance of cost, simplicity, and performance/headroom. Because this solution is Kubernetes native, we consider the following when providing recommendations:</p><ul><li>Analysis of historical pod/container resource consumption to predict future requirements</li><li>Consideration of both resource requests and usage</li><li>Ensuring largest workloads can be scheduled successfully</li><li>Accounting for DaemonSet/kube-system replication requirements for each node</li><li>Allocating headroom based on cluster context, e.g. dev, prod, high availability</li><li>Avoiding shared core machines for production/HA environments</li></ul><h3>How to use</h3><p>This tool is free to use and can be <a href="http://kubecost.com/install">installed</a> in minutes with Helm or a flat manifest. After installing Kubecost, then navigate to the <strong>Savings</strong> page and look for <strong>cluster right-sizing</strong>. You will see initial recommendations based on a “development” cluster context, but you can select a different cluster type to receive recommendations more customized to your situation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*KCRq2t5s_ET4u3Mg.png" /></figure><p>The product supports clusters on both <strong>AWS and GCP</strong> today and begins collecting workload data as soon as it’s installed. <strong>Note:</strong> no data is egressed from your cluster with this product and everything runs with your infrastructure.</p><h3>Working Example</h3><p>The following example presents two potential strategies in a Production cluster. There is a tradeoff between cost and complexity in this scenario. The simpler of the two configurations requires only a single node pool, but the multi-pool solution offers an additional 14% savings per month by bin packing resources more efficiently than otherwise possible with only one node type. Advanced metrics are available directly in the UI to better understand the rationale for these recommendations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1010/0*1NOn3wfmK9p0OFnm.png" /></figure><h3>Our Approach</h3><p>To generate accurate recommendations, a potential cluster must be adequately-sized such that it is possible to schedule all workloads on the cluster. To assess this requirement, workloads scheduled as DaemonSets must appear on each node, making total resource allocation a function of node count. Furthermore, one large workload may not be able to fit on a fleet of small nodes, even if the aggregate amount of resources would appear adequate. We are able to use metrics generated by the core <a href="https://github.com/kubecost/cost-model">Kubecost allocation model</a> (open source) to calculate nth-percentile total resource allocation over time, the percentage attributed to DaemonSets, and the largest single workload.</p><p>As a function of the cluster context, e.g. dev or production, we then analyze the shape of historical resource consumption and apply the appropriate headroom budget. For development clusters, we look at the trending 85th percentile consumption, for production 98th, and HA 99.9th.</p><p>Beyond these requirements, we apply a heuristic to manage the tradeoff between cost and complexity. For example, many teams would agree that, given the choice between saving 30% with a simple option and saving 31% with an utterly complex one, the simple option sounds more attractive. Therefore, we’ve chosen to start with two implementation strategies instead of one, hence the recommendations in the aforementioned examples.</p><p>You might be wondering why cluster sizing can not be solved by autoscaling. In the time we’ve spent helping teams of all kinds optimize their Kubernetes environments, we’ve found that cluster autoscaling, while useful, does not always resolve right-sizing concerns. Consider that autoscaling, even when it is working, does not allow for the possibility of adding different node pools to a cluster. It’s still easy, therefore, to end up over-provisioning if the autoscaler’s single node type is inappropriate for the resource proportions of the given workload. Over-provisioning by an autoscaler can be a painful experience, whereas using the cluster sizing recommendation puts the control and information in your hands to right-size your cluster when the time is right, without fear of runaway costs due to a single resource spike, memory leak, or ill-fitting node type.</p><h3>About us</h3><p><a href="https://kubecost.com/">Kubecost</a> provides real-time visibility into cost and capacity management decisions on Kubernetes. Our products provide showback/chargeback, resource optimization insights, and more for Kubernetes-centric environments.</p><p>This new tool is only the tip of the iceberg with respect to cluster right-sizing. We hope you’ll find it useful! If you have any questions or feedback, please reach out to us at any time on <a href="https://join.slack.com/t/kubecost/shared_invite/enQtNTA2MjQ1NDUyODE5LWFjYzIzNWE4MDkzMmUyZGU4NjkwMzMyMjIyM2E0NGNmYjExZjBiNjk1YzY5ZDI0ZTNhZDg4NjlkMGRkYzFlZTU">Slack</a> or at <a href="/service/mailto:team@kubecost.com">team@kubecost.com</a>.</p><p><em>Originally published at </em><a href="http://blog.kubecost.com/blog/cluster-right-sizing/"><em>http://blog.kubecost.com</em></a><em> on February 19, 2020.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=751e4c3b40b7" width="1" height="1" alt=""><hr><p><a href="https://medium.com/kubecost/find-an-optimal-set-of-nodes-for-a-kubernetes-cluster-751e4c3b40b7">Find an optimal set of nodes for a Kubernetes cluster</a> was originally published in <a href="https://medium.com/kubecost">kubecost</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Practical Guide to Setting Kubernetes Requests and Limits]]></title>
            <link>https://medium.com/kubecost/a-practical-guide-to-setting-kubernetes-requests-and-limits-8bf7c1b61ed8?source=rss-f640f2f76fb9------2</link>
            <guid isPermaLink="false">https://medium.com/p/8bf7c1b61ed8</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[docker]]></category>
            <dc:creator><![CDATA[Ajay Tripathy]]></dc:creator>
            <pubDate>Wed, 25 Sep 2019 12:34:30 GMT</pubDate>
            <atom:updated>2019-09-26T19:00:40.808Z</atom:updated>
            <content:encoded><![CDATA[<p>Setting Kubernetes requests and limits effectively has a major impact on application performance, stability, and cost. And yet working with many teams over the past year has shown us that determining the right values for these parameters is hard. For this reason, we have created this short guide and are launching a new product to help teams more accurately set Kubernetes requests and limits for their applications.</p><h3>The Basics</h3><p>Resource requests and limits are optional parameters specified at the container level. Kubernetes computes a Pod’s request and limit as the sum of requests and limits across all of its containers. Kubernetes then uses these parameters for scheduling and resource allocation decisions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*eofzBQ6iCCe4tMvE.png" /></figure><p><strong>Requests</strong></p><p>Pods will get the amount of <em>memory</em> they request. If they exceed their memory request, they could be killed if another pod happens to need this memory. Pods are only ever killed when using less memory than requested if critical system or high priority workloads need the memory.</p><p>Similarly, each container in a Pod is allocated the amount of <em>CPU</em> it requests, if available. It may be allocated additional CPU cycles if available resources are not needed by other running Pods/Jobs.</p><p>Note: if a Pod’s total requests are not available on a single node, then the Pod will remain in a Pending state (i.e. not running) until these resources become available.</p><p><strong>Limits</strong></p><p>Resource limits help the Kubernetes scheduler better handle resource contention. When a Pod uses more memory than its limit, its processes will be killed by the kernel to protect other applications in the cluster. Pods will be CPU throttled when they exceed their CPU limit.</p><p>If no limit is set, then the pods can use excess memory and CPU when available.</p><p>Here’s a quick example of how you set requests and limits on a container spec:</p><pre>apiVersion: v1<br>kind: Pod<br>metadata:<br>  name: hello-app<br>spec:<br>  containers:<br>  - name: wp<br>    image: wordpress<br>    resources:<br>      requests:<br>        memory: &quot;64Mi&quot;<br>        cpu: &quot;250m&quot;<br>      limits:<br>        memory: &quot;128Mi&quot;<br>        cpu: &quot;500m&quot;</pre><p>CPU requests are set in cpu units where 1000 millicpu (“m”) equals 1 vCPU or 1 Core. So 250m CPU equals ¼ of a CPU. Memory can be set with Ti, Gi, Mi, or Ki units. For more advanced technical info on the mechanics of these parameters, we suggest these articles:</p><ul><li><a href="https://github.com/kubernetes/community/blob/master/contributors/design-proposals/node/resource-qos.md">https://github.com/kubernetes/community/blob/master/contributors/design-proposals/node/resource-qos.md</a></li><li><a href="https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/">https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/</a></li></ul><h3>The Tradeoffs</h3><p>Determining the right level for requests and limits is about managing trade-offs, as shown in the following tables. When setting requests, there is inherently a tradeoff between the cost of running an application and the performance/outage risk for this application. Balancing these risks depends on the relative cost of extra CPU/RAM compared to the expected cost of an application throttle or outage event. For example, if allocating another 1 Gb of RAM ($5 cost) reduces the risk of an application outage event ($10,000 cost) by 1% then it would be worth the additional cost of these compute resources.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/803/1*buoCfRmkAY_Ue-qFRkswsw.png" /></figure><p>When setting limits, the tradeoffs are similar but not quite the same. The tradeoff here is the relative performance of individual applications on your shared infrastructure vs the total cost of running these applications. For example, setting the aggregated amount of CPU limits higher than the allocated number of CPUs exposes applications to potential throttling risk. Provisioning additional CPUs (i.e. increase spend) is one potential answer while reducing CPU limits for certain applications (i.e. increase throttling risk) is another.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tGeoPU_3DqaPra6S6HFZ-g.png" /></figure><p>In the following section, we present a framework for managing these tradeoffs more effectively.</p><h3>Determining the right values</h3><p>When setting requests, start by determining the acceptable probability of a container’s usage exceeding its request in a specific time window, e.g. 24 hours. To predict this in future periods, we can analyze historical resource usage. As an example, allowing usage to be above a request threshold with 0.01 probability (i.e. three nines) means that it will, on average, face increased risk of throttling or being killed 1.44 minutes a day.</p><p>You can classify applications into different availability tiers and apply these rules of thumb for targeting the appropriate level of availability:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*a-6HTOe53rHo5htE_dB3Fg.png" /></figure><p>This approach of analyzing historical usage patterns typically provides both a good representation of the future and is easy to understand/introspect. Applying extra headroom allows for fluctuations that may have been missed by your historical sampling. We recommend measuring usage over 1 week at a minimum and setting thresholds based on the specific availability requirements of your pod.</p><p><em>*Developers sharing experimental clusters may rely on broader protection from </em><a href="https://kubernetes.io/docs/concepts/policy/resource-quotas/"><em>resource quotas</em></a><em>. Quotas set aggregate caps at the namespace level and can help protect tasks like long-running batch or ML jobs from getting killed because someone improperly specified resources in another namespace.</em></p><h3>Our Solution</h3><p>Seeing the difficulty of setting these parameters correctly and managing them over time motivated us to create a solution in the Kubecost product to directly generate recommendations for your applications. Our recommendations are based on a configurable Availability Tiers (e.g. Production or Dev), which is easily tracked by namespace or other concepts directly in the Kubecost product.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/950/0*CQS93bnhPuVRNc-K.png" /></figure><p>In addition to providing request recommendations, this solution also proactively detects out of memory and CPU throttle risks. The full Kubecost product is available via a single Helm command ( <a href="http://docs.kubecost.com/install">install options</a>) and these recommendations can easily be viewed for each container in the Namespace view. Our commercial product is free for small clusters, comes with a free trial for larger clusters, and is based on the <a href="https://github.com/kubecost/cost-model">Kubecost open source project</a>.</p><p>Also, here are sample Prometheus queries if you want to calculate these metrics yourself!</p><p><strong>Memory Request (Production Tier)</strong></p><pre>1.5 * avg(quantile_over_time(.99,container_memory_working_set_bytes{container_name!=&quot;POD&quot;,container_name!=&quot;&quot;}[7d])) by (container_name,pod_name,namespace)</pre><p>We recommend container_memory_working_set_bytes because this metric excludes cached data and is what Kubernetes uses for OOM/scheduling decisions. More info in this <a href="https://medium.com/faun/how-much-is-too-much-the-linux-oomkiller-and-used-memory-d32186f29c9d">article</a>.</p><p><strong>CPU Request (Production Tier)</strong></p><p>First, create a recording rule with this expression. Note we recommend using irate to capture short-term spike in resource needs.</p><pre>avg( irate(container_cpu_usage_seconds_total{container_name!=&quot;POD&quot;, container_name!=&quot;&quot;}[5m]) ) by (container_name,pod_name,namespace)</pre><p>Then run this query:</p><pre>1.5 * quantile_over_time(.99,container_cpu_usage_irate[7d])</pre><h3>Vertical pod autoscaling</h3><p>The goal of vertical pod autoscaling (VPA) is to remove the need to worry about specifying values for a container’s CPU and memory requests. It can be a great solution in certain situations, often with stateless workloads, but you should note that this tool is still in beta as of September 2019. Here are some of the practical limitations to be aware of:</p><ul><li>Pods are evicted and need to restart when the VerticalPodAutoscaler needs to change the Pod’s resource requests.</li><li>VPA can cause performance risk and outages if not configured correctly and adds observability complexity.</li><li>To appropriately handle scale up events, it’s recommended that Cluster Autoscaler also be enabled to handle the increased resource requirements sizes of your workloads.</li><li>The VPA requires careful tuning to implement a tier-based solution with different parameters for highly available apps, dev, prod, staging etc.</li></ul><p>We advise teams to be cautious when using VPA for critical production workloads. It introduces complexity to your infrastructure and you should adequately ensure that your deployments and VPA itself are configured correctly. Risks aside, it can be a great solution when applied correctly.</p><p>More info on VPA is available here:</p><ul><li><a href="https://github.com/kubernetes/autoscaler/tree/master/vertical-pod-autoscaler">https://github.com/kubernetes/autoscaler/tree/master/vertical-pod-autoscaler</a></li><li><a href="https://github.com/kubernetes/community/blob/master/contributors/design-proposals/autoscaling/vertical-pod-autoscaler.md">https://github.com/kubernetes/community/blob/master/contributors/design-proposals/autoscaling/vertical-pod-autoscaler.md</a></li></ul><h3>Conclusion</h3><p>Setting requests and limits effectively can make or break application performance and reliability in Kubernetes. This set of guidelines and this new Kubecost tool can help you manage the inherent risks and tradeoffs when solution like vertical pod autoscaling are not the right fit. Our recommendations combined cost data and health insights are available in Kubecost today to help you make informed decisions.</p><h3>About Kubecost</h3><p><a href="http://kubecost.com/">Kubecost</a> provides cost and capacity management tools purpose built for Kubernetes. We help teams do Kubernetes chargeback and more. Reach out via email (team@kubecost), <a href="https://join.slack.com/t/kubecost/shared_invite/enQtNTA2MjQ1NDUyODE5LWFjYzIzNWE4MDkzMmUyZGU4NjkwMzMyMjIyM2E0NGNmYjExZjBiNjk1YzY5ZDI0ZTNhZDg4NjlkMGRkYzFlZTU">Slack</a> or visit <a href="http://kubecost.com/">our website</a> if you want to talk shop or learn more!</p><p>A big thanks to all that have given us feedback and made contributions!</p><p><em>Originally published at </em><a href="http://blog.kubecost.com/blog/requests-and-limits/"><em>http://blog.kubecost.com</em></a><em> on September 25, 2019.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8bf7c1b61ed8" width="1" height="1" alt=""><hr><p><a href="https://medium.com/kubecost/a-practical-guide-to-setting-kubernetes-requests-and-limits-8bf7c1b61ed8">A Practical Guide to Setting Kubernetes Requests and Limits</a> was originally published in <a href="https://medium.com/kubecost">kubecost</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[One of the great promises of using Kubernetes is that it has the ability to scale your…]]></title>
            <link>https://medium.com/@ajaytripathy/one-of-the-great-promises-of-using-kubernetes-is-that-it-has-the-ability-to-scale-your-5009fe1101ce?source=rss-f640f2f76fb9------2</link>
            <guid isPermaLink="false">https://medium.com/p/5009fe1101ce</guid>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Ajay Tripathy]]></dc:creator>
            <pubDate>Tue, 13 Aug 2019 00:35:44 GMT</pubDate>
            <atom:updated>2019-08-13T00:35:44.568Z</atom:updated>
            <content:encoded><![CDATA[<p>One of the great promises of using Kubernetes is that it has the ability to scale your infrastructure dynamically based on user demand. But by default it won’t add or remove machines. To do this, you’ll have to enable the Kubernetes Cluster Autoscaler.</p><p>In this article, we’ll explain how the Kubernetes Cluster Autoscaler works to scale hardware based on the needs of your Kubernetes workloads. We’ll go over the autoscaling algorithm. We’re also sharing a tool we built to help teams understand why the autoscaler is or isn’t scaling as expected, and to see how your current infrastructure would react to enabling the autoscaler.</p><h4>The Basics</h4><p>Clusters are how Kubernetes groups machines. They are comprised of Nodes (individual machines, oftentimes virtual) which run Pods. Pods have containers that request resources such as CPU, Memory, and GPU. The Cluster Autoscaler adds or removes Nodes in a Cluster based on resource requests from Pods.</p><p>A high-level overview of a Kubernetes cluster</p><p>When Does The Cluster Autoscaler Add Capacity?</p><p>The cluster autoscaler increases the size of the cluster when there are pods that are not able to be scheduled due to resource shortages. It can be configured to not scale up or down past a certain number of machines. Here is an overview of how it makes scaling decisions:</p><p>When enabled, the cluster autoscaler algorithm checks for pending pods.</p><p>The cluster autoscaler requests a newly provisioned node if: 1) there are pending pods due to not having enough available cluster resources to meet their requests and 2) the cluster or node pool has not reached the user-defined maximum node count.</p><p>Kubernetes detects the new node once it is provision by the underlying infrastructure, e.g. GCP</p><p>The Kubernetes scheduler allocates the pending pods to the new node.</p><p>Go back to Step 1 if there are still pods in a pending state.</p><p>When Do Nodes Get Turned Down?</p><p>Scaling down is a little bit more complex. The process to check whether a node is safe to delete starts when pod requests on that node are lower than a user-defined threshold (default of 50% request utilization). It’s worth noting that this node scaledown check does not consider actual CPU/memory usage, and instead only looks at resource requests. Once a node passes that check, the actual Kubernetes scheduling algorithm is called to determine whether the pods running on that node can be moved somewhere else. The cluster autoscaler also runs the following checks on each pod in the node to minimize the chance of service disruption:</p><p>If the pod is part of a daemonset, the pod is safe to turn down, since daemonsets are supposed to run statelessly on all nodes. Removing the node should not reschedule a pod in a daemonset.</p><p>If the pod is a <a href="https://kubernetes.io/docs/tasks/administer-cluster/static-pod/">mirror pod</a>, (only relevant if you’ve created static pods), it is considered safe to turn down.</p><p>Removing the pod does not bring the number of replicas below the specified minimum replica count, unless you have specified a pod disruption budget and have remaining disruptions to “spend” on moving the pod.</p><p>The pod doesn’t use any local storage on the node; since the node is going away, that local storage will be lost.</p><p>Kube system pods won’t get moved unless they specify a pod disruption budget.</p><p>If any of these node or pod-level checks do not pass, then the node will not be turned down.</p><p>How to Enable Cluster Autoscaling</p><p>The procedure is slightly different for every cloud provider. Here’s a list of steps for common ones:</p><p>GCE <a href="https://kubernetes.io/docs/concepts/cluster-administration/cluster-management/">https://kubernetes.io/docs/concepts/cluster-administration/cluster-management/</a></p><p>GKE</p><p><a href="https://cloud.google.com/container-engine/docs/cluster-autoscaler">https://cloud.google.com/container-engine/docs/cluster-autoscaler</a></p><p>AWS <a href="https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/aws/README.md">https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/aws/README.md</a></p><p>Azure <a href="https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/azure/README.md">https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/azure/README.md</a></p><p>Alibaba Cloud <a href="https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/alicloud/README.md">https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/alicloud/README.md</a></p><p>Limitations</p><p>The cluster autoscaler doesn’t take into account actual CPU/GPU/Memory usage, just resource requests and limits. Most teams overprovision at the pod level, so in practice we see aggressive upscaling and conservative downscaling.</p><p>Scaling up isn’t immediate, so we’ve seen services experience downtime or latency as they get stuck waiting to schedule more nodes to be added to the pool. While the cluster autoscaler should issue a scale-up request within 30-60 seconds to a cloud provider, the actual time the cloud provider takes to create a node can be on the order of minutes.</p><p>To avoid issues where you need to wait to scale up, most teams want to leave some resources idle. While you can <a href="https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md#how-can-i-configure-overprovisioning-with-cluster-autoscaler">overprovision</a> by using “pause pods” with low priority to “reserve” space for pods of higher priority, this requires a fair amount of configuration. And again, the inputs to the scaling algorithm here are resource requests, not actual utilization.</p><p>Scaling down when requested utilization dips below your threshold isn’t guaranteed with the autoscaler, because as mentioned above nodes can have non removable pods. We’ve seen unreplicated workloads without disruption budgets get scheduled on big nodes and waste money.</p><p>Best Practices If You Are Using the Cluster Autoscaler</p><p>Keep your pod requests close to actual utilization</p><p>Consider using the <a href="https://github.com/kubernetes/autoscaler/tree/master/vertical-pod-autoscaler">vertical pod autoscaler</a> (more on this in an upcoming blog post) to set your requests close to actual utilization automatically.</p><p>Set pod disruption budgets <a href="https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md#how-to-set-pdbs-to-enable-ca-to-move-kube-system-pods">where applicable</a> for kube-system pods, to avoid nodes that can’t be turned down</p><p>Avoid using local storage for pods</p><p>Monitor your pods to make sure they don’t make expensive nodes ineligible for scale-down</p><p>Our Tool Today</p><p>To help ensure that your nodes can scale down when needed, we built a tool that statically runs checks from the cluster autoscaler library and displays whether a node is safe to delete at various levels of utilization in your cluster. If the node isn’t safe, we show which pods are causing the node to stick, and which removal tests they are failing.</p><p>You don’t need to use the cluster autoscaler to use this tool -- nodes that pass the checks can be safely removed manually as well.</p><p>We’ll be open-sourcing our tool in the coming weeks, but in the meantime, if you’re interested in giving it a trial run, reach out to <a href="/service/mailto:team@kubecost.com">team@kubecost.com</a> to get access to our pilot program!</p><pre>UPDATE: this tool is now part of the core Kubecost product, available at <a href="https://slack-redir.net/link?url=http%3A%2F%2Fkubecost.com">http://kubecost.com</a>. This product is based on our open source projects at Github.</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5009fe1101ce" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding Kubernetes Cluster Autoscaling]]></title>
            <link>https://medium.com/kubecost/understanding-kubernetes-cluster-autoscaling-675099a1db92?source=rss-f640f2f76fb9------2</link>
            <guid isPermaLink="false">https://medium.com/p/675099a1db92</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[autoscaling]]></category>
            <dc:creator><![CDATA[Ajay Tripathy]]></dc:creator>
            <pubDate>Wed, 30 Jan 2019 13:10:25 GMT</pubDate>
            <atom:updated>2019-08-13T23:05:56.198Z</atom:updated>
            <content:encoded><![CDATA[<p>One of the great promises of using Kubernetes is that it has the ability to scale your infrastructure dynamically based on user demand. But by default it won’t add or remove machines. To do that, you’ll have to enable the Kubernetes Cluster Autoscaler.</p><p>In this article, we’ll explain how the Kubernetes Cluster Autoscaler works to scale hardware based on the needs of your Kubernetes workloads. We’ll go over the autoscaling algorithm. We’re also sharing a tool we built to help teams understand why the autoscaler is or isn’t scaling as they expected, and to see how your current infrastructure would react to enabling the autoscaler.</p><p><strong>The Basics</strong></p><p><em>Clusters</em> are how Kubernetes groups machines. They are comprised of <em>Nodes</em> (individual machines, oftentimes virtual) which run <em>Pods</em>. <em>Pods</em> have containers that request resources such as CPU, Memory, and GPU. The Cluster Autoscaler adds or removes <em>Nodes</em> in a <em>Cluster </em>based on resource requests from <em>Pods</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/786/0*IShO2x_jlRIseZ7e" /><figcaption><em>A high-level overview of a Kubernetes cluster</em></figcaption></figure><p><strong>When Does The Cluster Autoscaler Add Capacity?</strong></p><p>The cluster autoscaler increases the size of the cluster when there are pods that are not able to be scheduled due to resource shortages. It can be configured to not scale up or down past a certain number of machines. Here is an overview of how it makes scaling decisions:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/802/0*CyQVykoco5oiVlGP" /></figure><ol><li>When enabled, the cluster autoscaler algorithm checks for pending pods.</li><li>The cluster autoscaler requests a newly provisioned node if: 1) there are pending pods due to not having enough available cluster resources to meet their requests and 2) the cluster or node pool has not reached the user-defined maximum node count.</li><li>Kubernetes detects the new node once it is provision by the underlying infrastructure, e.g. GCP</li><li>The Kubernetes scheduler allocates the pending pods to the new node.</li><li>Go back to Step 1 if there are still pods in a pending state.</li></ol><p><strong>When Do Nodes Get Turned Down?</strong></p><p>Scaling down is a little bit more complex. The process to check whether a node is safe to delete starts when pod requests on that node are lower than a user-defined threshold (default of 50% request utilization). It’s worth noting that this node scaledown check does not consider actual CPU/memory usage, and instead only looks at resource requests. Once a node passes that check, the actual Kubernetes scheduling algorithm is called to determine whether the pods running on that node can be moved somewhere else. The cluster autoscaler also runs the following checks on each pod in the node to minimize the chance of service disruption:</p><ul><li>If the pod is part of a daemonset, the pod is safe to turn down, since daemonsets are supposed to run statelessly on all nodes. Removing the node should not reschedule a pod in a daemonset.</li><li>If the pod is a <a href="https://kubernetes.io/docs/tasks/administer-cluster/static-pod/">mirror pod</a>, (only relevant if you’ve created static pods), it is considered safe to turn down.</li><li>Removing the pod does not bring the number of replicas below the specified minimum replica count, unless you have specified a pod disruption budget and have remaining disruptions to “spend” on moving the pod.</li><li>The pod doesn’t use any local storage on the node; since the node is going away, that local storage will be lost.</li><li>Kube system pods won’t get moved unless they specify a pod disruption budget.</li></ul><p>If any of these node or pod-level checks do not pass, then the node will not be turned down.</p><p><strong>How to Enable Cluster Autoscaling</strong></p><p>The procedure is slightly different for every cloud provider. Here’s a list of steps for common ones:</p><ul><li><a href="https://kubernetes.io/docs/concepts/cluster-administration/cluster-management/">GCE</a></li><li><a href="https://cloud.google.com/container-engine/docs/cluster-autoscaler">GKE</a></li><li><a href="https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/aws/README.md">AWS</a></li><li><a href="https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/azure/README.md">Azure</a></li><li><a href="https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/alicloud/README.md">Alibaba Cloud</a></li></ul><p><strong>Limitations</strong></p><ul><li>The cluster autoscaler doesn’t take into account actual CPU/GPU/Memory usage, just resource requests and limits. Most teams overprovision at the pod level, so in practice we see aggressive upscaling and conservative downscaling.</li><li>Scaling up isn’t immediate, so we’ve seen services experience downtime or latency as they get stuck waiting to schedule more nodes to be added to the pool. While the cluster autoscaler should issue a scale-up request within 30–60 seconds to a cloud provider, the actual time the cloud provider takes to create a node can be on the order of minutes.</li><li>To avoid issues where you need to wait to scale up, most teams want to leave some resources idle. While you <em>can</em> <a href="https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md#how-can-i-configure-overprovisioning-with-cluster-autoscaler">overprovision</a> by using “pause pods” with low priority to “reserve” space for pods of higher priority, this requires a fair amount of configuration. And again, the inputs to the scaling algorithm here are resource requests, not actual utilization.</li><li>Scaling down when requested utilization dips below your threshold isn’t guaranteed with the autoscaler, because as mentioned above nodes can have non removable pods. We’ve seen unreplicated workloads without disruption budgets get scheduled on big nodes and waste money.</li></ul><p><strong>Best Practices If You Are Using the Cluster Autoscaler</strong></p><ul><li>Keep your pod requests close to actual utilization. Consider using the <a href="https://github.com/kubernetes/autoscaler/tree/master/vertical-pod-autoscaler">vertical pod autoscaler</a> (more on this in an upcoming blog post) to set your requests close to actual utilization automatically.</li><li>Set pod disruption budgets <a href="https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md#how-to-set-pdbs-to-enable-ca-to-move-kube-system-pods">where applicable</a> for kube-system pods, to avoid nodes that can’t be turned down</li><li>Avoid using local storage for pods</li><li>Monitor your pods to make sure they don’t make expensive nodes ineligible for scale-down</li></ul><p><strong>Our Tool Today</strong></p><p>To help ensure that your nodes can scale down when needed, we built a tool that statically runs checks from the cluster autoscaler library and displays whether a node is safe to delete at various levels of utilization in your cluster. If the node isn’t safe, we show which pods are causing the node to stick, and which removal tests they are failing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/894/0*7biFjMyTB4dC807w" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*41D3d2rVWRSUflYG" /></figure><p>You don’t need to use the cluster autoscaler to use this tool — nodes that pass the checks can be safely removed manually as well.</p><p>We’ll be open-sourcing our tool in the coming weeks, but in the meantime, if you’re interested in giving it a trial run, reach out to <a href="/service/mailto:team@kubecost.com">team@kubecost.com</a> to get access to our pilot program!</p><p>Update: this tool is now part of the core Kubecost product, available at <a href="http://kubecost.com/">http://kubecost.com</a>. This product is based on our open source projects at Github.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=675099a1db92" width="1" height="1" alt=""><hr><p><a href="https://medium.com/kubecost/understanding-kubernetes-cluster-autoscaling-675099a1db92">Understanding Kubernetes Cluster Autoscaling</a> was originally published in <a href="https://medium.com/kubecost">kubecost</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>