<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Yash Bhutwala on Medium]]></title>
        <description><![CDATA[Stories by Yash Bhutwala on Medium]]></description>
        <link>https://medium.com/@ymb002?source=rss-9a2110cb1ba1------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*BNaBTa-lNk5-ttTjOliLqQ.jpeg</url>
            <title>Stories by Yash Bhutwala on Medium</title>
            <link>https://medium.com/@ymb002?source=rss-9a2110cb1ba1------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 13:40:21 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ymb002/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Vertical Pod Autoscaling: Right Sizing Your Pods]]></title>
            <link>https://medium.com/@ymb002/vertical-pod-autoscaling-right-sizing-your-pods-18af3a0d5184?source=rss-9a2110cb1ba1------2</link>
            <guid isPermaLink="false">https://medium.com/p/18af3a0d5184</guid>
            <category><![CDATA[autoscaling]]></category>
            <category><![CDATA[containers]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[docker]]></category>
            <dc:creator><![CDATA[Yash Bhutwala]]></dc:creator>
            <pubDate>Wed, 06 May 2020 20:03:32 GMT</pubDate>
            <atom:updated>2020-05-06T20:03:32.140Z</atom:updated>
            <content:encoded><![CDATA[<h3>Background</h3><p>Kubernetes workloads specify their CPU/memory needs via “resource requests”. CPU resources are measured in virtual cores or more commonly in “millicores”, e.g. 500m denoting 50% of a vCPU. Memory resources are measured in Bytes and the usual suffixes can be used, e.g. 500Mi denoting 500 Mebibyte. Resource requests “block” capacity on worker nodes, i.e. a Pod with 1000m CPU requests on a node with 4 vCPUs will leave only 3 vCPUs available for other Pods. Requests is the most important attribute for a workload, as they let the Kubernetes scheduler know where the workload can be placed. Yet, there is also limits, which enforce the maximum resource a workload is allowed to consume, with <em>compressible</em> resources such as CPU resulting in throttling and <em>in-compressible</em> resources such as memory resulting in OOMKill events.</p><h3>Problem</h3><h4>What do you set these requests/limits to?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QP4po_MpQyO5qrtp2RH4nw.png" /></figure><h4>Once you set them, are they correct?</h4><p>Too little</p><ul><li>CPU starvation</li><li>Out of Memory errors</li><li>Workload eviction</li></ul><p>Too much?</p><ul><li>Resource Waste</li></ul><h4>How to get it right?</h4><ul><li>Canarying</li><li>Trial and Error</li></ul><h4>Where to get the numbers from?</h4><ul><li>Time, effort &amp; resources</li><li>Canary may be different than prod</li><li>Resource needs will change over time</li></ul><p>Daily/Weekly traffic patterns</p><p>User base growing over time</p><p>App lifecycle phases with different resource needs</p><blockquote>As you can imagine, coming up with definitive numbers for cpu and memory requirements for different containers is hard, so most people get lazy, pick their best guess, stick it in a yaml file and never change it (even though app requirements change over time).</blockquote><p>Unfortunately, we do this currently for all the products at my company.</p><h4>The goal of VPA/HPA is to replace this human operator based approach with a much more data driven automated approach to right-sizing pods to increase utilization and reduce slack — difference between allocation and usage.</h4><h3>How VPA works</h3><p>Three different loosely coupled parts:</p><ul><li>Recommender</li><li>Updater</li><li>Admission Controller</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3_0J-loTywSxNOWLOKzrzA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*kaWYp8dP_hmkjdoQ" /></figure><p>VPA recommender works by looking at resource usage via the metric server and looking at OOM events as well as evictions from the API Server; it also can be configured to use historical data from Prometheus.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2GQNEIfuCJvocAV6k54SeQ.png" /></figure><h3>How VPA Recommender works internally currently</h3><p>Recommendations are calculated using <a href="https://github.com/kubernetes/autoscaler/blob/vertical-pod-autoscaler-0.8.0/vertical-pod-autoscaler/pkg/recommender/util/decaying_histogram.go#L43">decaying histogram</a> of weighted samples from the metrics server, where the newer samples are assigned higher weights; older samples are decaying and hence affect less and less w.r.t. to the recommendations. <a href="https://github.com/kubernetes/autoscaler/blob/vertical-pod-autoscaler-0.8.0/vertical-pod-autoscaler/pkg/recommender/model/aggregate_container_state.go#L174">CPU</a> is calculated using the <a href="https://github.com/kubernetes/autoscaler/blob/vertical-pod-autoscaler-0.8.0/vertical-pod-autoscaler/pkg/recommender/logic/recommender.go#L102">90th percentile of all cpu samples</a> , and <a href="https://github.com/kubernetes/autoscaler/blob/vertical-pod-autoscaler-0.8.0/vertical-pod-autoscaler/pkg/recommender/model/aggregate_container_state.go#L175">memory</a> is calculated using the <a href="https://github.com/kubernetes/autoscaler/blob/vertical-pod-autoscaler-0.8.0/vertical-pod-autoscaler/pkg/recommender/logic/recommender.go#L106">90th percentile</a> <a href="https://github.com/kubernetes/autoscaler/blob/vertical-pod-autoscaler-0.8.0/vertical-pod-autoscaler/pkg/recommender/model/aggregate_container_state.go#L96"><strong>peak</strong></a> over the 8 day window. <a href="https://github.com/kubernetes/autoscaler/blob/vertical-pod-autoscaler-0.8.0/vertical-pod-autoscaler/pkg/recommender/model/aggregations_config.go#L28">8 days</a> of history is used for recommendation (1 memory usage sample <a href="https://github.com/kubernetes/autoscaler/blob/vertical-pod-autoscaler-0.8.0/vertical-pod-autoscaler/pkg/recommender/model/aggregations_config.go#L35">per day</a> ). Prometheus can be used a history provider in this calculation. By default, vpa is collecting data about all controllers, so when new vpa objects are created, they are already providing stable recommendations (unless you specify memory-save=true). All active vpa recommendations are checkpointed.</p><h4>Output from Recommender</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/802/1*L7hgcIFPcWwFynZsBXYG_w.png" /></figure><p><strong><em>Target is the most important recommendation reported in the VPA custom resource status. This is the recommendation given for you to set the requests to.</em></strong> There is no limit recommendation given by VPA currently. Other recommendations are as follows: <strong>Uncapped Target</strong> gives the recommendation before applying constraints specified in the VPA spec, such as min or max; <strong>Lower and Upper bound</strong> are meant to only be used by the VPA updater to allow pod to be running if the requests are in that range; and not evict them. Lower bounds are calculated as 50th percentile of the histogram, whereas upper bound is calculated as 95th percentile of the histogram. For upper bound, by default to start with it is set to node’s capacity.</p><h3><a href="https://dl.acm.org/doi/pdf/10.1145/3342195.3387524">Snippets from Google’s paper on Autopilot (Google’s internal VPA like system)</a></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*1cvuwYVwyU8NK3BY" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*MZB0P3RvmYTuF-hC" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*rXS6ayosPn_XvrS_" /></figure><ul><li><strong><em>Autopiloted jobs now represent over 48% of our fleet-wide usage.</em></strong></li><li><strong><em>Autopiloted jobs have a slack of just 23%, compared with 46% for manually-managed jobs.</em></strong></li><li><strong><em>Autopilot reduces the number of jobs severely impacted by OOMs by a factor of 10</em></strong></li></ul><h3>Resources</h3><ul><li>KubeCon 2018 VPA <a href="https://www.youtube.com/watch?v=Y4vnYaqhS74">Talk</a> | <a href="https://static.sched.com/hosted_files/kccna18/1f/VPA%20Kubecon%20Seattle.pdf">Slides</a></li><li>KubeCon 2019 Intro to AutoScaling <a href="https://www.youtube.com/watch?v=S_VlztrihzU">Talk</a> | Slides</li><li>KubeCon 2019 EU Deep Dive AutoScaling <a href="https://www.youtube.com/watch?v=3CCfzFqxS-g">Talk</a> | <a href="https://static.sched.com/hosted_files/kccnceu19/5b/SIG%20Autoscaling%20Deep%20Dive%20Kubecon%20EU%202019.pdf">Slides</a></li><li>KubeCon 2019 Cake <a href="https://www.youtube.com/watch?v=58uRFofXUyw">Talk</a> | <a href="https://static.sched.com/hosted_files/kccnceu19/cf/Cake%20presentation.pdf">Slides</a></li><li><a href="https://github.com/kubernetes/autoscaler/issues/2747#issuecomment-616037197">Explanation of how VPA works internally</a></li><li><a href="https://github.com/kubernetes/autoscaler/blob/vertical-pod-autoscaler-0.8.0/vertical-pod-autoscaler/pkg/recommender/logic/recommender.go#L42">VPA recommender code</a></li><li><a href="https://github.com/kubernetes/kube-state-metrics/blob/master/docs/verticalpodautoscaler-metrics.md">VPA metrics in kube-state-metrics</a></li><li><a href="https://cloud.google.com/kubernetes-engine/docs/concepts/verticalpodautoscaler">GKE VPA instructions</a></li><li><a href="https://github.com/kubernetes/community/blob/master/contributors/design-proposals/autoscaling/initial-resources.md">Original 2015 Initial Resources Design Proposal</a></li><li><a href="https://github.com/kubernetes/community/blob/master/contributors/design-proposals/autoscaling/vertical-pod-autoscaler.md">Vertical Pod Autoscaler Design Proposal</a></li><li>In-Place Update of Pod Resources</li><li>currently for dynamic adjusting the requests resource, VPA updater makes a new pod to replace the old one. This feature will make it easier to use VPA, since after this, in-place update of resource requests will be supported without deploying the app</li><li><a href="https://github.com/kubernetes/enhancements/issues/1287">Tracking issue</a></li><li><a href="https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/20181106-in-place-update-of-pod-resources.md">KEP</a></li><li><a href="https://medium.com/infrastructure-adventures/vertical-pod-autoscaler-deep-dive-limitations-and-real-world-examples-9195f8422724">VPA Deep Dive Medium article</a></li><li><a href="https://dl.acm.org/doi/pdf/10.1145/3342195.3387524">Autopilot: workload autoscaling at Google</a></li><li><a href="https://srcco.de/posts/saving-cloud-costs-kubernetes-aws.html">Saving Cloud Costs</a></li><li><a href="https://github.com/hjacobs/kube-resource-report">kube-resource-report</a></li><li><a href="https://srcco.de/posts/saving-cloud-costs-kubernetes-aws.html">Goldilocks</a></li><li><a href="https://mhausenblas.info/right-size-guide/">right-size-guide</a></li><li><a href="https://twitter.com/mhausenblas/status/1225855388584730624">twitter thread</a></li><li><a href="https://github.com/facebookincubator/senpai">senpai</a></li><li><a href="http://www.brendangregg.com/perf.html#CPUstatistics">Linux perf cpu statistics</a></li><li><a href="https://sig-confluence.internal.synopsys.com/pages/viewpage.action?pageId=689101314">other benchmarking tools</a></li><li><a href="https://medium.com/swlh/applications-autoscaling-strategy-in-kubernetes-50535683322f">medium article on application autoscaling strategy</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=18af3a0d5184" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Exploring the Flexibility of Kubernetes]]></title>
            <link>https://medium.com/cloud-heroes/exploring-the-flexibility-of-kubernetes-9f65db2360a0?source=rss-9a2110cb1ba1------2</link>
            <guid isPermaLink="false">https://medium.com/p/9f65db2360a0</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[beginner]]></category>
            <category><![CDATA[containers]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[linux]]></category>
            <dc:creator><![CDATA[Yash Bhutwala]]></dc:creator>
            <pubDate>Sat, 02 Nov 2019 19:12:54 GMT</pubDate>
            <atom:updated>2019-11-04T13:17:26.760Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qVRWoWzdVNIid3wltxUulw.png" /><figcaption>Kubernetes Sequence Diagram</figcaption></figure><p>Pictures are worth a thousand words. The diagram above is the most important diagram in understanding how Kubernetes is designed and works at a higher level. It explains how the different components in the system work with each other to coordinate as a jazz improv.</p><p>Generally, this is the sequential flow of the system:</p><ol><li><strong>Client</strong> sends a request for a Deployment to the <strong>API Server</strong> (for example using kubectl create -f deployment.yml)</li><li><strong>API Server</strong> persists the Deployment to <strong>etcd; etcd</strong> returns 200 to <strong>API Server; API Server </strong>returns 200 to<strong> Client </strong>(obviously, the work isn’t done here, there is a lot of background asynchronous stuff that happens next)</li><li><strong>Controller Manager</strong> has a <strong>watch</strong> against the <strong>API Server; </strong>it sees that a Deployment has been created and it populates that <strong>Informer</strong> to the <strong>SharedCache</strong></li><li><strong>Deployment Controller</strong> sees the <strong>Deployment</strong>, pulls it off the <strong>queue</strong>, creates a<strong> ReplicaSet </strong>and persists the ReplicaSet object back to the<strong> API Server</strong> and back to <strong>etcd</strong></li><li>Repeat Step 3 for ReplicaSet</li><li><strong>ReplicaSet Controller</strong> sees the <strong>ReplicaSet</strong>, pulls it off the queue, creates X number of <strong>Pods </strong>and persists the Pods object back to the<strong> API Server </strong>and back to <strong>etcd</strong></li><li>Repeat Step 3 for Pods (but <strong>Scheduler</strong> does this part now)</li><li><strong>Scheduler</strong> sees the <strong>unscheduled pods</strong>, and executes its<strong> business logic </strong>to<strong> </strong>fill out the <strong>nodeName</strong> field in the pod’s<strong> Spec </strong>with the name of the<strong> Schedulable Node </strong>and persists the scheduled Pods object back to the<strong> API Server </strong>and back to <strong>etcd</strong></li><li>Repeat Step 3 for Pods (but <strong>Kubelet</strong> this time<strong>)</strong></li><li><strong>Kubelet</strong> (on the scheduled node) sees it’s supposed to have a Pod running on its machine and talks with the container runtime (<strong>Docker</strong>, etc.) to make that happen</li></ol><blockquote>You might say that’s an awful lot of steps to create one freaking pod/container. Well, Engineering is about making trade-offs to achieve scale. What this design of Kubernetes allows us is Modularity, Flexibility, and Scalibility.</blockquote><p>To prove the flexibility and pluggabilty of Kubernetes, we’ll show how to run Kubernetes without a controller-manager and scheduler, and still create a Pod and scheduling it yourself.</p><h4>Setup Kubernetes using KinD</h4><p>Create KinD (<a href="https://kind.sigs.k8s.io/">https://kind.sigs.k8s.io/</a>) with this config and this command:</p><p>kind create cluster --config kind-multi-worker-cluster.yml --image kindest/node:v1.16.2</p><p><a href="https://github.com/yashbhutwala/kind-hacks/blob/master/kind-multi-worker-cluster.yml">yashbhutwala/kind-hacks</a></p><h4>Turning off Scheduler and Controller-Manager</h4><p><strong>Let’s go into the control-plane “node”:</strong></p><p>docker exec -it kind-control-plane bash</p><p><strong>Go to where the k8s manifests are:</strong></p><p>cd /etc/kubernetes/manifests/</p><p><strong>Stop running the controller-manager and scheduler:</strong></p><p>mv kube-controller-manager.yaml ../ &amp;&amp; mv kube-scheduler.yaml ../</p><p>Boom! Now when you do <strong>kubectl get po -A; </strong>you will not see any controller-manager or scheduler pods</p><h4>Now, let’s schedule a Pod directly on a node:</h4><p>This will create a Pod manifest with the nginx image. Just FYI, with kubectl run,--restart=Never makes a Pod; if you don’t have that argument, it makes a Deployment.</p><p>kubectl run test --image=nginx -o yaml --replicas=1 --restart=Never --dry-run -o yaml &gt; pod.yaml</p><p><strong>Let’s inspect the pod manifest:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/914/1*AMrqdhVaEIa71shN-3vK0A.png" /></figure><p><strong>Add a nodeName to schedule the Pod to </strong><strong>kind-worker node</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/784/1*JXxTotRu5yiaGdFfxQ542w.png" /></figure><p>Now, all that’s left to do is create it, so go ahead and run kubectl apply -f pod.yaml</p><h4>Boom! The nginx test pod is running without the controller-manager and the scheduler!!</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lYRmrMnIO0L8uhEx4d-3dw.png" /></figure><p>As you can see, we can Steps 3–9 and still be able to run a Pod in our cluster. <strong>This shows the flexibility of Kubernetes and the power of its modular design!</strong></p><h3>Sources</h3><p><a href="https://blog.heptio.com/core-kubernetes-jazz-improv-over-orchestration-a7903ea92ca">Core Kubernetes: Jazz Improv over Orchestration</a></p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F78XQe6FdHac%3Fstart%3D1081%26feature%3Doembed%26start%3D1081&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D78XQe6FdHac&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F78XQe6FdHac%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/d5283d14c4e33eb3d50309dd2816dfdc/href">https://medium.com/media/d5283d14c4e33eb3d50309dd2816dfdc/href</a></iframe><ul><li><a href="https://github.com/yashbhutwala/kind-hacks">yashbhutwala/kind-hacks</a></li><li><a href="https://kind.sigs.k8s.io/">kind</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9f65db2360a0" width="1" height="1" alt=""><hr><p><a href="https://medium.com/cloud-heroes/exploring-the-flexibility-of-kubernetes-9f65db2360a0">Exploring the Flexibility of Kubernetes</a> was originally published in <a href="https://medium.com/cloud-heroes">Cloud Heroes</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Testing Calico locally using Kubernetes in Docker (KinD)]]></title>
            <link>https://medium.com/cloud-heroes/testing-calico-locally-using-kubernetes-in-docker-kind-52981ce20703?source=rss-9a2110cb1ba1------2</link>
            <guid isPermaLink="false">https://medium.com/p/52981ce20703</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[networking]]></category>
            <category><![CDATA[containers]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[calico]]></category>
            <dc:creator><![CDATA[Yash Bhutwala]]></dc:creator>
            <pubDate>Mon, 21 Oct 2019 12:48:06 GMT</pubDate>
            <atom:updated>2019-10-21T12:48:06.505Z</atom:updated>
            <content:encoded><![CDATA[<h3>Create the cluster using this configuration</h3><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f81ac908dff99154758657ad444cd0f0/href">https://medium.com/media/f81ac908dff99154758657ad444cd0f0/href</a></iframe><pre>kind create cluster --config kind-calico.yaml</pre><h3>Set your current KUBECONFIG to the KinD cluster</h3><pre>export KUBECONFIG=&quot;$(kind get kubeconfig-path --name=&quot;kind&quot;)&quot;<br>kubectl get pods -n kube-system</pre><h3>Apply the latest (v3.10 on 2019–10–21) calico manifests:</h3><p>Source: <a href="https://docs.projectcalico.org/v3.10/getting-started/kubernetes/installation/calico#installing-with-the-kubernetes-api-datastore50-nodes-or-less">https://docs.projectcalico.org/v3.10/getting-started/kubernetes/installation/calico#installing-with-the-kubernetes-api-datastore50-nodes-or-less</a></p><pre>kubectl apply -f https://docs.projectcalico.org/v3.10/manifests/calico.yaml</pre><h3>Configure RPF Check Configuration</h3><p>By default, Calico pods fail if the Kernel’s Reverse Path Filtering (RPF) check is not enforced. This is a security measure to prevent endpoints from spoofing their IP address.</p><p>The RPF check is not enforced in Kind nodes at least as of v0.5.1, although <a href="https://github.com/kubernetes-sigs/kind/pull/897">this PR</a> will fix <a href="https://github.com/kubernetes-sigs/kind/issues/891">this issue</a> in the next version. You can either fix this by relaxing Calico’s configuration using:</p><pre>kubectl -n kube-system set env daemonset/calico-node FELIX_IGNORELOOSERPF=true</pre><p>Or you can fix it by manually editing the KinD configuration using:</p><pre>kind get nodes | xargs -n1 -I {} docker exec {} sysctl -w net.ipv4.conf.all.rp_filter=0</pre><h3>Verify Calico Is Up</h3><pre>kubectl -n kube-system get pods | grep calico-node</pre><h3>References:</h3><p><a href="https://github.com/kubernetes-sigs/kind">KinD GitHub</a></p><p><a href="https://kind.sigs.k8s.io/">KinD Official Documentation</a></p><p><a href="https://godoc.org/sigs.k8s.io/kind/pkg/internal/apis/config#Cluster">KinD Cluster Set Up GoDoc</a></p><p><a href="https://github.com/kubernetes-sigs/kind/issues/891">GitHub Issue by Duffie</a></p><p><a href="https://docs.projectcalico.org/v3.10/getting-started/kubernetes/installation/calico#installing-with-the-kubernetes-api-datastore50-nodes-or-less">Installing Calico</a></p><p><a href="https://docs.projectcalico.org/v3.10/reference/architecture/">Calico Architecture</a></p><p><a href="https://docs.projectcalico.org/v3.10/reference/architecture/data-path">Calico Data Path</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=52981ce20703" width="1" height="1" alt=""><hr><p><a href="https://medium.com/cloud-heroes/testing-calico-locally-using-kubernetes-in-docker-kind-52981ce20703">Testing Calico locally using Kubernetes in Docker (KinD)</a> was originally published in <a href="https://medium.com/cloud-heroes">Cloud Heroes</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kubernetes Simplified: RBAC basics]]></title>
            <link>https://medium.com/cloud-heroes/kubernetes-simplified-rbac-basics-216d50c1a6ba?source=rss-9a2110cb1ba1------2</link>
            <guid isPermaLink="false">https://medium.com/p/216d50c1a6ba</guid>
            <category><![CDATA[containers]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[identity]]></category>
            <category><![CDATA[authentication]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Yash Bhutwala]]></dc:creator>
            <pubDate>Wed, 24 Jul 2019 12:53:50 GMT</pubDate>
            <atom:updated>2019-07-24T12:57:30.099Z</atom:updated>
            <content:encoded><![CDATA[<h3>Explain like I’m 5</h3><blockquote>A <strong>Role</strong> is like being a Knight. There can be many Knights. When someone is knighted, the act of knighting is a <strong>role binding</strong>.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*vNjOQZb4CmhC2o9oskmRvA.jpeg" /></figure><h3>Explain like I’m 25</h3><p><strong>R</strong>ole <strong>B</strong>ased <strong>A</strong>ccess <strong>C</strong>ontrol (RBAC), in Kubernetes, is a way to allow the system to answer the question:</p><blockquote>Can <strong>(subject) (verb) (object)</strong>?</blockquote><p><strong>Subject</strong>: this can be a user , group or service-account</p><p><strong>Verb: </strong>this can be create, get, list, update, delete, etc.</p><p><strong>Object:</strong> this is K8s api-resource objects like pod, deployment, namespace, etc.</p><h4>Why do we need it?</h4><p>You have multiple people on your team, and everyone wants to access this one Kubernetes cluster. Each person on the team has different responsibilities and each person needs a level of isolation and security from one another. You do not want one person from accidentally interfering with other person’s work on the cluster. Thinking of about differences between developers, operators, cluster administrators, etc., there is a wide variety of reasons why you want to slice your Kubernetes cluster based on different roles and different capabilities. RBAC is the way you can restrict who can access what within a Kubernetes cluster.</p><p>A Role in RBAC defines a notion of verb (action), i.e.: get, list, etc., and a set of nouns, i.e: pods, secrets, etc.. So, <strong>a Role defines what you can do to a set of resources</strong>. But by itself, the role doesn’t grant any permissions, because we haven’t associated the role with anyone who might make the API calls. And that’s where a binding comes into play. So, a Role Binding is just a mapping between a user or a service account, and a particular role, i.e.: cluster-admin. A user can have multiple roles. And you can also use groups, and often times that’s a better way to manage RBAC is via groups, i.e: dev-team, admin-team. You can associate dev-team with a particular role. And that way if a new developer joins or leaves the team, they can be added or removed from the dev-team group and they will gain or lose to those particular privileges.</p><h4>What is difference between ClusterRole and Role?</h4><p>Within Kubernetes, there is two different types of resources. You can find which resources are what scoped by using kubectl api-resources -o wide. There is namespaced resources, for example a pod, a service. There is also cluster resources, for example, . Therefore, ClusterRoles define verbs on cluster scoped resources, and Roles define verbs for namespaced scoped resources.</p><h4>What are Bindings and what is the difference between ClusterRoleBindings and RoleBindings?</h4><p>In our ELI5 analogy, we explained that the process of getting knighted is analagous to a Role Binding. So, <strong>a Role Binding is grants a Role to a subject.</strong> There is two different types of role bindings you can have. One is ClusterRoleBindings, which provides you with permissions for the entire cluster. It can provide you permissions for cluster resources, as well as it provides you with permissions for resources within any namespace in a cluster. <strong>ClusterRoleBindings are very powerful, and you want to be very careful of how you apply them because they apply not only to any existing namespaces, but to any future namespaces that you might create as well.</strong> There is also RoleBindings, and they provide privileges within the context of a particular namespace. Obviously, ClusterRoleBinding is a cluster level object, and RoleBinding is a namespaced level object.</p><h4>Combinations of Roles and Role Bindings</h4><p>There is three different combinations of Roles and Role Bindings you can have with RBAC</p><p><strong>ClusterRole + RoleBinding</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8bwhzawQdIDqLUkNDzjhIA.png" /><figcaption>Borrowed from Jordan Liggitt’s talk referenced below</figcaption></figure><p><strong>Role + RoleBinding</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6_8LJyryZzbI5vYoZ0xbrg.png" /><figcaption>Borrowed from Jordan Liggitt’s talk referenced below</figcaption></figure><p><strong>ClusterRole + ClusterRoleBinding</strong></p><blockquote>Warning: this should be granted extremely limited, as it is equivalent to God mode in your cluster (depending on the Role defined). Always follow the least-privileges principle, assign only the permissions necessary.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*c23e80YZS2Q0OKXLC8bHSg.png" /><figcaption>Borrowed from Jordan Liggitt’s talk referenced below</figcaption></figure><h3>TLDR</h3><blockquote><strong>A </strong>Role<strong> defines what you can do to a set of resources. A </strong>Role Binding grants a Role to a subject. ClusterRoles are <strong>defined globally</strong>, while <strong>Roles</strong> are <strong>defined locally. ClusterRoles</strong> can be either <strong>granted globally</strong> via <strong>ClusterRoleBindings</strong> or<strong> granted locally</strong> via <strong>RoleBindings</strong>. <strong>Roles</strong> can only be <strong>granted locally</strong> via <strong>RoleBindings</strong>.</blockquote><h3>Heroic Links:</h3><p><a href="https://kubernetes.io/docs/reference/access-authn-authz/rbac/">Kubernetes Docs on RBAC</a></p><p><a href="https://www.youtube.com/watch?v=Nw1ymxcLIDI">Jordan Liggitt KubeCon 2017 Talk on Effective RBAC</a></p><p><a href="https://www.youtube.com/watch?v=G3R24JSlGjY">Brendan Burns Azure Kubernetes Series</a></p><h3>Thanks for reading! In the name of Cloud Justice!</h3><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=216d50c1a6ba" width="1" height="1" alt=""><hr><p><a href="https://medium.com/cloud-heroes/kubernetes-simplified-rbac-basics-216d50c1a6ba">Kubernetes Simplified: RBAC basics</a> was originally published in <a href="https://medium.com/cloud-heroes">Cloud Heroes</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[WTF is happening on my Kubernetes Cluster]]></title>
            <link>https://medium.com/cloud-heroes/wtf-is-happening-on-my-kubernetes-cluster-96aab618213f?source=rss-9a2110cb1ba1------2</link>
            <guid isPermaLink="false">https://medium.com/p/96aab618213f</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[cloud]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[api]]></category>
            <category><![CDATA[containers]]></category>
            <dc:creator><![CDATA[Yash Bhutwala]]></dc:creator>
            <pubDate>Fri, 19 Jul 2019 14:00:22 GMT</pubDate>
            <atom:updated>2019-07-19T14:25:02.852Z</atom:updated>
            <content:encoded><![CDATA[<h3>What’s going on my Kubernetes Cluster?</h3><p>If you want to understand what is happening on your Kubernetes cluster, auditing is the feature for you. With auditing, you can answer the following questions:</p><ul><li>what happened?</li><li>when did it happen?</li><li>who initiated it?</li><li>on what did it happen?</li><li>where was it observed?</li><li>from where was it initiated?</li><li>to where was it going?</li></ul><p>kube-apiserver performs the auditing. It logs requests based on an audit policy. In this article, we are going to create a single node cluster with auditing enabled on the apiserver using kind.</p><h3>Download Kind</h3><p>Kind stands for <strong>K</strong>ubernetes <strong>IN</strong> <strong>D</strong>ocker. So, of course, you’ll need to have Docker and Go. If you don’t have Docker or Go, do you even cloud, bro? Jk, no worries, get it here: <a href="https://docs.docker.com/install/">https://docs.docker.com/install/</a> and <a href="https://golang.org/">https://golang.org/</a>. Next, we’ll install the kind cli. Run GO111MODULE=&quot;on&quot; go get sigs.k8s.io/kind@v0.4.0 . Boom, all set!</p><h3>Kind Config</h3><p>Now that we have kind , let’s make a config for our cluster in order to enable auditing. Run mkdir -p ~/kind and stick this kind-config file in there.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e770dba0e4abee200e0584bb0443a4a2/href">https://medium.com/media/e770dba0e4abee200e0584bb0443a4a2/href</a></iframe><p>This config tells the kind cli to make a single node cluster with specific configuration for the control plane. So, let’s make the audit policy, and audit log on the host machine.</p><h3>Audit Policy</h3><p>Here is a simple audit policy, it logs all requests at the Metadata level. You can learn more about auditing and different configurations for audit policy here: <a href="https://kubernetes.io/docs/tasks/debug-application-cluster/audit/">https://kubernetes.io/docs/tasks/debug-application-cluster/audit/</a></p><p>Copy this yml content into a file named audit-policy.yml.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4531c78a2dba81616b47cb104727d0ca/href">https://medium.com/media/4531c78a2dba81616b47cb104727d0ca/href</a></iframe><h3>Audit Log</h3><p>Next run touch apiserver-audit.log . This will create an empty audit log on the host machine for the api-server to put logs into.</p><p>Now we are ready to create the cluster!! Simply run:</p><p>kind create cluster --name single --config ~/kind/kind-config.yaml --loglevel debug</p><p>Now just do a tail -f ~/kind/apiserver-audit.log and watch all the request’s metadata stream by. This is what a single request log looks like:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d3fb27cfa28d1f51f3c6acf5a1a783de/href">https://medium.com/media/d3fb27cfa28d1f51f3c6acf5a1a783de/href</a></iframe><p>You can see what endpoint each request is hitting, which user or controller is hitting them, the objectRef, and much more. Get playing!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=96aab618213f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/cloud-heroes/wtf-is-happening-on-my-kubernetes-cluster-96aab618213f">WTF is happening on my Kubernetes Cluster</a> was originally published in <a href="https://medium.com/cloud-heroes">Cloud Heroes</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>