<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Alessandro Vozza on Medium]]></title>
        <description><![CDATA[Stories by Alessandro Vozza on Medium]]></description>
        <link>https://medium.com/@ams0?source=rss-92272b78ceea------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*kuGnvlMq0csQvPYfdHEBjA.png</url>
            <title>Stories by Alessandro Vozza on Medium</title>
            <link>https://medium.com/@ams0?source=rss-92272b78ceea------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 08:17:42 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ams0/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Local prototyping of ArgoCD manifests with minikube and Gitea]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://ams0.medium.com/local-prototyping-of-argocd-manifests-with-minikube-and-gitea-a8eb20a0f2d3?source=rss-92272b78ceea------2"><img src="https://cdn-images-1.medium.com/max/637/1*ZsKI2Fzb1-_od2ejSaUlDQ.png" width="637"></a></p><p class="medium-feed-snippet">Have you ever wished you could test your ArgoCD manifests before pushing them to your VCS? GitOps of course it&#x2019;s great, and it&#x2019;s de facto&#x2026;</p><p class="medium-feed-link"><a href="https://ams0.medium.com/local-prototyping-of-argocd-manifests-with-minikube-and-gitea-a8eb20a0f2d3?source=rss-92272b78ceea------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://ams0.medium.com/local-prototyping-of-argocd-manifests-with-minikube-and-gitea-a8eb20a0f2d3?source=rss-92272b78ceea------2</link>
            <guid isPermaLink="false">https://medium.com/p/a8eb20a0f2d3</guid>
            <category><![CDATA[gitea]]></category>
            <category><![CDATA[minikube]]></category>
            <category><![CDATA[argo-cd]]></category>
            <category><![CDATA[gitops]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Alessandro Vozza]]></dc:creator>
            <pubDate>Mon, 14 Nov 2022 21:10:47 GMT</pubDate>
            <atom:updated>2022-11-14T21:10:47.615Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[CKS Exam Prep: Kubeadm VM in Azure]]></title>
            <link>https://medium.com/cooking-with-azure/cks-exam-prep-kubeadm-vm-in-azure-d7ef5dcf47f7?source=rss-92272b78ceea------2</link>
            <guid isPermaLink="false">https://medium.com/p/d7ef5dcf47f7</guid>
            <category><![CDATA[kubeadm]]></category>
            <category><![CDATA[azure]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[container-security]]></category>
            <dc:creator><![CDATA[Alessandro Vozza]]></dc:creator>
            <pubDate>Sun, 10 Jan 2021 21:18:05 GMT</pubDate>
            <atom:updated>2021-01-12T17:14:02.125Z</atom:updated>
            <content:encoded><![CDATA[<h3>CKS Exam Prep: Kubeadm VM</h3><h3>in Azure</h3><p><em>If you want to prepare for the CKS exam, you’ll need a VM to learn on! Here’s I show you how to create a VM in Azure easily and ready to go. This blog is part of a series, check </em><a href="https://medium.com/cooking-with-azure/certified-kubernetes-security-specialist-cks-exam-guide-a8fc2b4c47ea"><em>here</em></a><em> the index.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ga31eAKWEplbJzZrUaxPeg.png" /></figure><p>The Certified Kubernetes Security Specialist (CKS) exam is a recently released certification from the Linux Foundation meant to test your skills to create and secure an hardened Kubernetes cluster. The cluster you’ll have to deal with during the exam are deployed with the standard, official Kubernetes community tool <em>kubeadm</em>; hence, you’ll be better off studying for the exam on the real deal than other distributions (although there are really good ones out there, like <a href="https://kind.sigs.k8s.io/">KIND</a>).</p><p>Since I ❤ Azure, I want to share with you an effortless way to quickly obtain a VM running in Azure with Kubeadm, using <em>containerd</em> as container runtime (that’s what you’ll get during the exam) and <em>canal </em>(calico+flannel) as CNI (so you can use NetworkPolicies). Ready? Let’s start</p><p>First download the script:</p><pre>wget <a href="http://bit.ly/kubeadm-containerd">http://bit.ly/kubeadm-containerd</a></pre><p>Then create a resource group in Azure and deploy the Virtual Machine pointing to the script as <em>custom-data</em>:</p><pre>az group create -n cks</pre><pre>az vm create -g cks -n cks &#39;\<br>--image  UbuntuLTS \<br>--ssh-key-values ~/.ssh/id_rsa.pub \<br>--admin-username cks \<br>--size Standard_B4ms \<br>--custom-data kubeadm-containerd.sh</pre><p><em>Voila’!</em> In a few minutes you’ll be able to SSH into the VM and enjoy a fully functional 1-node kubeadm cluster with containerd and canal. Here’s how you do that:</p><pre>&gt; ssh csk@&lt;VM IP&gt;<br># sudo su<br>$ export KUBECONFIG=/etc/kubernetes/admin.conf<br>$ alias k=kubectl</pre><pre>$ k get no -o wide<br>NAME   STATUS   ROLES                  AGE    VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION     CONTAINER-RUNTIME<br>cks    Ready    control-plane,master   120m   v1.20.1   10.0.0.4      &lt;none&gt;        Ubuntu 18.04.5 LTS   5.4.0-1031-azure   containerd://1.3.3</pre><h3><em>Update!</em></h3><p>I created a Terraform template to deploy a multinode (1 controller + N nodes in VM Scale Set) using Kubeadm+containerd in Azure <a href="https://github.com/ams0/CKS/tree/main/kubeadm-containerd-multinode">here</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d7ef5dcf47f7" width="1" height="1" alt=""><hr><p><a href="https://medium.com/cooking-with-azure/cks-exam-prep-kubeadm-vm-in-azure-d7ef5dcf47f7">CKS Exam Prep: Kubeadm VM in Azure</a> was originally published in <a href="https://medium.com/cooking-with-azure">Cooking with Azure</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[CKS Exam Prep: Setting up Audit Policy in Kubeadm]]></title>
            <link>https://medium.com/cooking-with-azure/cks-exam-prep-setting-up-audit-policy-in-kubeadm-3f1b76bf4bd7?source=rss-92272b78ceea------2</link>
            <guid isPermaLink="false">https://medium.com/p/3f1b76bf4bd7</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[security]]></category>
            <category><![CDATA[container-security]]></category>
            <dc:creator><![CDATA[Alessandro Vozza]]></dc:creator>
            <pubDate>Sun, 10 Jan 2021 21:14:06 GMT</pubDate>
            <atom:updated>2021-01-12T17:14:30.204Z</atom:updated>
            <content:encoded><![CDATA[<p><em>The second part of my CKS series will cover how to enable Audit policy logging for a Kubernetes cluster deployed with Kubeadm. The reference article is </em><a href="https://medium.com/cooking-with-azure/certified-kubernetes-security-specialist-cks-exam-guide-a8fc2b4c47ea"><em>here</em></a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*w7OQJeAPTVIEc84DDzIF_Q.jpeg" /></figure><p>Audit logging (also known as audit trail) refers to the practice of recording events and changes to a system; in the context of Kubernetes security, it’s a <a href="https://kubernetes.io/docs/tasks/debug-application-cluster/audit/">security feature of the API server</a> that will record every (or a subset of your choice) actions and interactions of users with the cluster. As the API server is the only way to interact with the cluster, that makes it a primary target for attackers and naturally the most important service to monitor and secure.</p><p>You can configure <em>what </em>you’re going to log by creating an object of type <em>Policy</em>:</p><pre>cat &gt; /etc/kubernetes/audit-policy.yaml &lt;&lt;EOF<br># Log all requests at the Metadata level.<br>apiVersion: audit.k8s.io/v1<br>kind: Policy<br>rules:<br>- level: Metadata<br>EOF</pre><p>There are four available levels to log every single event/resource granularly:</p><p><strong>None</strong> — don’t log events that match this rule.<br><strong>Metadata</strong> — log request metadata (requesting user, timestamp, resource, verb, etc.) but not request or response body.<br><strong>Request</strong> — log event metadata and request body but not response body. This does not apply for non-resource requests.<br><strong>RequestResponse</strong> — log event metadata, request and response bodies. This does not apply for non-resource requests.</p><p>It’s easier to refer to the <a href="https://kubernetes.io/docs/tasks/debug-application-cluster/audit/">docs</a> to understand how to specify the level per resource audited; during the CKS exam, you’ll be able to access this document (just search for “Audit” in <a href="https://kubernetes.io/docs).">https://kubernetes.io/docs).</a></p><p>The file in itself is not going to do much until we instruct the API server to use it. In most clusters the API server is configured as a <a href="https://kubernetes.io/docs/tasks/configure-pod-container/static-pod/">static pod</a> whose configuration lives in /etc/kubernetes/manifests/kube-apiserver.yaml ; not only we will need to modify the flags of the actual <em>command</em> that starts the API server, but we need to make sure that the server itself can find the policy file we just created, using the pair volumes/volumeMounts:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/11477655e1bdf018805b4b40a6126285/href">https://medium.com/media/11477655e1bdf018805b4b40a6126285/href</a></iframe><p>Note also the options to configure the audit log file, retention (in days) and the maximum size of the audit log (in Mb).</p><h4>Bonus: troubleshoot misconfigured API-server static pod</h4><p>It might happen that due to a typo in editing the static pod definition the api-server won’t come and you’ll be cut off the cluster. Don’t panic! It happens and it’s not difficult to recover. First: <strong>make a backup of the file before editing.</strong> This is really important! Second: since it’s the kubelet process that monitors the folder /etc/kubernetes/manifests for changes, look into its logs:</p><pre>journalctl -u kubelet -f</pre><p>I often restart the kubelet process before looking at the logs to avoid syphoning thru thousands of lines:</p><pre>systemctl restart kubelet</pre><p>Remember that since 1.19 containerd is the standard container runtime for Kubernetes, so you won’t have your familiar docker command line, instead, you can use ctr :</p><pre>ctr -n k8s.io c list | grep api</pre><p>where k8s.io is the namespace where all the Kubernetes containers are running.</p><p><em>I hope it helps! Check out other articles in the CKS series </em><a href="https://medium.com/cooking-with-azure/certified-kubernetes-security-specialist-cks-exam-guide-a8fc2b4c47ea"><em>here</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3f1b76bf4bd7" width="1" height="1" alt=""><hr><p><a href="https://medium.com/cooking-with-azure/cks-exam-prep-setting-up-audit-policy-in-kubeadm-3f1b76bf4bd7">CKS Exam Prep: Setting up Audit Policy in Kubeadm</a> was originally published in <a href="https://medium.com/cooking-with-azure">Cooking with Azure</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Certified Kubernetes Security Specialist (CKS) exam guide]]></title>
            <link>https://medium.com/cooking-with-azure/certified-kubernetes-security-specialist-cks-exam-guide-a8fc2b4c47ea?source=rss-92272b78ceea------2</link>
            <guid isPermaLink="false">https://medium.com/p/a8fc2b4c47ea</guid>
            <category><![CDATA[security]]></category>
            <category><![CDATA[certified-kubernetes]]></category>
            <category><![CDATA[container-security]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Alessandro Vozza]]></dc:creator>
            <pubDate>Sun, 10 Jan 2021 21:11:58 GMT</pubDate>
            <atom:updated>2021-01-10T21:19:36.251Z</atom:updated>
            <content:encoded><![CDATA[<p><em>This post will collect the links to the blog posts that I’m going to write as a preparation guide for CKS. Also, I’m going to cover other security-related topics that not necessarily appear in the exam, but I deem important to talk about when we talk about security in containers in general and Kubernetes in particular.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ner-yFUzwHIG_nkxW9C29g.png" /></figure><p>The new Certified Kubernetes Security Specialist exam (CKS) (curriculum <a href="https://github.com/cncf/curriculum">here</a>) was introduced at the end of 2020 at Kubecon NA and completes the Kubernetes certifications offered by the Cloud Native Computing Foundation (CNCF: disclaimer, I’m one of the <a href="https://www.cncf.io/people/ambassadors/">ambassadors </a>for the foundation), the other two being the Certified Kubernetes Adminstrator (CKA) and the Certified Kubernetes Application Developer (CKAD). It demonstrates how CNCF and the community around Kubernetes is extremely serious about security (check the <a href="https://github.com/cncf/sig-security/blob/master/security-whitepaper/CNCF_cloud-native-security-whitepaper-Nov2020.pdf">security whitepaper</a> made by SIG-Security) and the latest news about hacks like <a href="https://krebsonsecurity.com/2020/12/solarwinds-hack-could-affect-18k-customers/">SUPERNOVA</a> only confirms that it’s high time we include a strong security posture from the start of our container supply chains.</p><p>The exam attests the skills of who successfully pass it to <em>think</em> and <em>act</em> appropriately when architecting and implementing a secure container orchestration architecture, and it’s one of toughest exam I have faced (with the possible exclusion of LFCS/LFCE). I’m going to start here a series of focused blog posts that will deal with specific part of the curriculum, and I’ll try to cover the topic to the best of my knowledge to give you the tools you need to learn and master each topic to the level necessary to pass the certification. But really, it’s up to you to invest enough time and effort in learning and keep learning after getting the certificate: security is a fluid topic and it’s an ever-changing effort to stay up to dated.</p><h3>The exam</h3><p>The exam lasts 2 hours for 17 questions, with different weight for each question, ranging from 14% to 4%; it’s completely browser-based and it’s remotely proctored. You are allowed to open <strong>one</strong> extra tab (I switched to Chrome for the exam, while I use Edge for my daily work, so I can have a clean slate to start) on which you can visit <strong>only </strong>the domain <a href="https://*.kubernetes.io/"><em>https://www.kubernetes.io</em></a> (so you can browse the Kubernetes <a href="https://kubernetes.io/blog/">blog </a>and the <a href="https://kubernetes.io/docs/home/">docs</a>, but <strong>not</strong> other sites like <a href="https://discuss.kubernetes.io./">https://discuss.kubernetes.io.</a> For some questions involving 3rd-party tools (there’s a couple of those) you are allowed to browse other domains like <a href="https://falco.org/docs/"><em>https://falco.org/docs/</em></a> but that’s an exception.</p><h3>Personal take on the exam</h3><p>I honestly had to take the exam twice (with a free retake); it’s a new exam and there’s not a lot of information out there on how to prepare, so I missed the mark the first time, reviewed the topics I knew I had wrong and passed it the second time. Even if the second time I left out the last question; luckily the last question was a “light” one accounting only for few percentage points.</p><p>I really enjoyed studying for the exam! It forced me to both learn new topics and revisit what I <em>thought </em>I knew about Kubernetes security, and also some new cool tool (one above all, <a href="https://github.com/aquasecurity/trivy">Trivy</a> from AquaSecurity). Some really good resources are:</p><ul><li>The exam simulator at <a href="https://killer.sh/"><em>killer.sh</em></a><em> </em>and the accompanying course on <a href="https://www.udemy.com/course/certified-kubernetes-security-specialist/?referralCode=D9329DEE203E7FEBE86B&amp;couponCode=YES-YES-CKS">Udemy</a>: this is really a great resource and helped my a ton! Thanks <a href="https://medium.com/u/7ad6cde74bf7">Kim Wuestkamp</a>! He also has a series of blogs on medium starting <a href="https://itnext.io/cks-exam-series-1-create-cluster-security-best-practices-50e35aaa67ae">here</a>.</li><li>Various gist and github repositories: <a href="https://github.com/abdennour/certified-kubernetes-security-specialist">this</a> from <a href="https://medium.com/u/5727040c50f3">Abdennour Toumi</a>, Scott Walkins’ <a href="https://github.com/echoboomer/k8s-cks-notes"><em>k8s-cks-notes</em></a><em> </em>and Ibrahim Jelliti’s (<a href="https://github.com/ibrahimjelliti/CKSS-Certified-Kubernetes-Security-Specialist">github</a>, <a href="https://www.ibrahimjelliti.com/my-take-towards-cka-ckad-september-curriculum-update-tips">blog</a>).</li><li>The just-released <a href="https://training.linuxfoundation.org/training/kubernetes-security-essentials-lfs260/">LFS260</a>: I just bought it and it looks really good, it’s well written (no videos) and the excercises will keep you busy for a long while.</li></ul><h3>Exam tips &amp; tricks</h3><p>It’s crucial to go fast but to maintain focus: here you really need to read the questions carefully and do <strong>exactly</strong> what the question asks. You can go back and forth thru the questions but make sure you get it right before you complete the exam; also, make sure you are completing the assignment on the right node: you start from a client host machine where you have access to various <em>kubeconfig</em> files (usually at the beginning of the question there’s a kubecttl config use-context command you can easily copy paste into the terminal) and depending on the question you are required to move to either the controller node or one of the worker nodes. You simply get a list of nodes and SSH into the right node (the hostname is the same as the node name); this worked like a charm for me without password authentication.</p><p>One more tip: always copy/save the files provided before modifying them directly! Once I messed up a Network Policy so bad I could not made it work again (you gotta ❤ YAML) so I just start over from the original file.</p><p>It’s worth to add an alias for kubectl ( alias k=kubectl), it will save you precious time and typing (since you’re there add kg=&#39;kubectl get&#39; and kd=&#39;kubectl describe&#39; ); going into the exam I thought I needed fancy stuff like krew plugins (I have made a memorable shortcut to install krew! wget bit.ly/installkrew . There are some good plugins for RBAC visualization out there) but in the end I could get thru the exam without.</p><p>That’s it for now! I’m going to write more about the CKS experience, stay tuned! Here’s the index of the articles I published so far:</p><ul><li><a href="https://medium.com/cooking-with-azure/cks-exam-prep-kubeadm-vm-in-azure-d7ef5dcf47f7">CKS Exam Prep: Kubeadm VM in Azure</a></li><li><a href="https://medium.com/cooking-with-azure/cks-exam-prep-setting-up-audit-policy-in-kubeadm-3f1b76bf4bd7">CKS Exam Prep: Setting up Audit Policy in Kubeadm</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a8fc2b4c47ea" width="1" height="1" alt=""><hr><p><a href="https://medium.com/cooking-with-azure/certified-kubernetes-security-specialist-cks-exam-guide-a8fc2b4c47ea">Certified Kubernetes Security Specialist (CKS) exam guide</a> was originally published in <a href="https://medium.com/cooking-with-azure">Cooking with Azure</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Create AKS clusters with a Helm chart? It’s possible with ClusterAPI!]]></title>
            <link>https://medium.com/cooking-with-azure/create-aks-clusters-with-a-helm-chart-its-possible-with-clusterapi-74f991ee5713?source=rss-92272b78ceea------2</link>
            <guid isPermaLink="false">https://medium.com/p/74f991ee5713</guid>
            <category><![CDATA[azure-kubernetes-service]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[azure]]></category>
            <category><![CDATA[helm]]></category>
            <category><![CDATA[cluster-api]]></category>
            <dc:creator><![CDATA[Alessandro Vozza]]></dc:creator>
            <pubDate>Mon, 07 Dec 2020 16:16:06 GMT</pubDate>
            <atom:updated>2020-12-07T22:37:26.789Z</atom:updated>
            <content:encoded><![CDATA[<h3>Create AKS clusters with a Helm chart? It’s possible with Cluster API!</h3><p><em>Cluster APIs are Kubernetes-native APIs to manage the lifecycle of clusters and nodes as CRDs.</em></p><p><em>Code available at </em><a href="https://github.com/ams0/azure-managed-cluster-capz-helm"><em>https://github.com/ams0/azure-managed-cluster-capz-helm</em></a></p><p>I’ve been following the <a href="https://github.com/kubernetes-sigs/cluster-api">Cluster API</a> project for some time now, but lately my interest was renewed when I find out that the <a href="https://github.com/kubernetes-sigs/cluster-api-provider-azure">cluster-api-provider-azure</a> (sometimes abbreviated in CAPZ) now <a href="https://github.com/kubernetes-sigs/cluster-api-provider-azure/blob/master/docs/book/src/topics/managedcluster.md">supports </a>AKS managed clusters in preview. So I set out to come up with some simple instructions on how to create AKS clusters programmatically. However, since clusters are described by (at least) four templates (as of now), I decided to write a simple Helm chart to easily and repeatably create clusters. Helm allows for an easy packaging and distribution of complex applications, and makes it simple to group all available options either on the command line or in a values file.</p><h4>Preparation</h4><p>First, you’ll need to <a href="https://cluster-api.sigs.k8s.io/user/quick-start.html#install-clusterctl">get </a>the clusterctl command line (I recommend using v0.3.11, the latest at the time of writing). Then, you’ll need to have a Kubernetes cluster to store the CRDs for your AKS clusters. You can use any cluster (another AKS cluster for example) or to keep things simple, you can spin up a local <a href="https://kind.sigs.k8s.io/">KIND </a>cluster, and that’s what we will do here (my preferred way to obtain kind and other tools is to use the gofish <a href="https://gofi.sh/">package manager</a>):</p><pre>kind create cluster --name capi</pre><p>Now clone the repo and edit the values in clusterctl.env :</p><pre>git clone <a href="https://github.com/ams0/azure-managed-cluster-capz-helm.git">https://github.com/ams0/azure-managed-cluster-capz-helm.git</a><br>cd azure-managed-cluster-capz-helm</pre><p>The clustertlc.env contains some environment variable that will influence the installation of the cluster-api-provider-azure (like EXP_AKS=true) which you should use to get the <strong>experimental</strong> support for AKS cluster in CAPZ. Fill the comment values for AZURE_SUBSCRIPTION_ID, AZURE_TENANT_ID and AZURE_LOCATION and source clusterctl.env .</p><p>Next, use clusterctl to install the Cluster API components:</p><pre>clusterctl init --infrastructure azure</pre><p>Wait until all components are deployed in the various capi-*-system namespaces.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/eff76545b130976ca4be4b5e51da70ce/href">https://medium.com/media/eff76545b130976ca4be4b5e51da70ce/href</a></iframe><h4>Deploy a cluster</h4><p>Ready to roll! Deploy your first cluster with Helm:</p><pre>helm install capz1 charts/azure-managed-cluster/  \<br>--set subscriptionID=&lt;subID&gt; \<br>--set cluster.resourceGroupName=aksclusters \<br>--set cluster.nodeResourceGroupName=capz1 \<br>--set cluster.name=aks1 \<br>--set controlplane.sshPublicKey=&quot;$(cat ~/.ssh/id_rsa.pub)&quot; \<br>--set agentpools[0].name=capz1np0 \<br>--set agentpools[0].nodecount=1 \<br>--set agentpools[0].sku=Standard_B4ms \<br>--set agentpools[0].osDiskSizeGB=100 \<br>--set agentpools[1].name=capz1np1 \<br>--set agentpools[1].nodecount=1 \<br>--set agentpools[1].sku=Standard_B4ms \<br>--set agentpools[1].osDiskSizeGB=10</pre><p>If you like you can use a values.yaml file:</p><pre>helm install capz1 charts/azure-managed-cluster/ --set subscriptionID=&lt;subID&gt; --values aks1.yaml</pre><p>The options available are a bit limited at the moment (here’s for example the <a href="https://github.com/kubernetes-sigs/cluster-api-provider-azure/blob/master/config/crd/bases/exp.infrastructure.cluster.x-k8s.io_azuremanagedcontrolplanes.yaml">supported options</a> for the AzureManagedControlPlane CRD), but the Cluster API Azure provider is open to accept suggestions in the form of <a href="https://github.com/kubernetes-sigs/cluster-api-provider-azure/issues">issues </a>or <a href="https://github.com/kubernetes-sigs/cluster-api-provider-azure/pulls">pull requests</a>. Note that the cluster is created with a SystemAssigned-managed identity and that the service principal from cluster.env is the one needed to <em>create </em>the cluster itself, not the one used by the cluster to interact with Azure via the cloud-controller-manager.</p><p>You can check the status of the deployment in several ways; in addition to the obvious az aks list to check the Azure side, you can check the cluster-api object and get the logs of the capz-controller-manager pod:</p><pre>kubectl get cluster-api<br>kubectl  logs -n capz-system -l control-plane=capz-controller-manager -c manager -f</pre><h4>Access the cluster</h4><p>You don’t need to access the Azure API to get the credentials for the cluster, as its kubeconfig config file is stored in the managent cluster (the KIND cluster) as a secret in the namespace where we created the cluster CRDs.</p><pre>kubectl get secret {cluster-name}-kubeconfig -o yaml \<br>-o jsonpath={.data.value} | base64 --decode &gt; aks1.kubeconfig</pre><pre>kubectl --kubeconfig=aks1.kubeconfig cluster-info</pre><p>That’s it!</p><h4>Adding clusters</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/620/0*1VXPMdndEPZHOLMp.jpg" /></figure><p>Let’s go crazy and add a second cluster:</p><pre>helm install capz2 charts/azure-managed-cluster/ --values aks2.yaml</pre><p>And so on! Now for some magic: you can delete the KIND cluster and reapply exactly the same Helm charts/Cluster API CRDs and the controller will just reconcile the desired state (2 clusters) with the current situation in Azure (2 clusters) and happily report that no change is needed. So as you can see this is a great step towards complete automation of AKS infrastructure, from start to finish (you can just add a GitOps configuration repository to the clusters, even via Azure Arc and now you have a full end-to-end automation from clusters to applications). I will expand on the concept in a later article.</p><p>I hope you find Cluster API as exciting as I do, and I’m looking forward to contributing to the project. And I will follow up with another post about day 2 operations (upgrades, scaling and so on) on AKS clusters deployed with Cluster APIs.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=74f991ee5713" width="1" height="1" alt=""><hr><p><a href="https://medium.com/cooking-with-azure/create-aks-clusters-with-a-helm-chart-its-possible-with-clusterapi-74f991ee5713">Create AKS clusters with a Helm chart? It’s possible with ClusterAPI!</a> was originally published in <a href="https://medium.com/cooking-with-azure">Cooking with Azure</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Managing multiple clusters with ArgoCD in Azure/k3s secured w/ Traefik&Let’s Encrypt]]></title>
            <link>https://medium.com/cooking-with-azure/managing-multiple-clusters-with-argocd-in-azure-k3s-secured-w-traefik-lets-encrypt-2de7daabbefa?source=rss-92272b78ceea------2</link>
            <guid isPermaLink="false">https://medium.com/p/2de7daabbefa</guid>
            <category><![CDATA[argoproj]]></category>
            <category><![CDATA[k3d]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[azure]]></category>
            <category><![CDATA[traefik]]></category>
            <dc:creator><![CDATA[Alessandro Vozza]]></dc:creator>
            <pubDate>Thu, 15 Oct 2020 14:36:24 GMT</pubDate>
            <atom:updated>2020-10-22T08:44:30.290Z</atom:updated>
            <content:encoded><![CDATA[<h3>Managing multiple clusters with ArgoCD on k3s running in Azure and secured by Traefik and Let’s Encrypt</h3><p><em>Code available at </em><a href="https://github.com/ams0/argocd-azure-k3s-traefik"><em>https://github.com/ams0/argocd-azure-k3s-traefik</em></a></p><ol><li><a href="#f127">Infrastructure installation</a></li><li><a href="#0d50">Ingress installation</a></li><li><a href="#8fd9">Argo installation</a></li><li><a href="#c905">GitOps flow</a></li><li><a href="#a2a0">Conclusions</a></li></ol><p>I heard about <a href="https://argoproj.github.io/argo-cd/">ArgoCD</a> many times (recently from my friends at <a href="https://www.meetup.com/Cloud-Native-Kubernetes-Netherlands/events/270457327/">Fullstaq</a>) but never tried kicking its tires until now. If you don’t know, ArgoCD is a platform for declarative continuous deployment of Kubernetes applications, and it’s quickly becoming an exceedingly popular choice to deploy and manage applications at scale on multiple clusters. It also recently become an incubated CNCF project.</p><p>Since I want to use it for deploying to a cluster, my plan is to have an ArgoCD instance outside my clusters that can manage them independently from the clusters’ lifecycle; hence, I devised this method of deploying ArgoCD into a VM in Azure that’s running the lightweight distribution of Kubernetes from Rancher Labs called “k3s” installed using the k3s helper tool and exposed via the Traefik ingress controller and secured with Let’s Encrypt certificates. Let’s get to it!</p><h3>Infrastructure installation</h3><p>Prerequisites:</p><ul><li>Azure CLI</li><li>Azure subscription (already logged in)</li><li>kubectl and jq installed</li></ul><p>Start by cloning the <a href="https://github.com/ams0/argocd-azure-k3s-traefik">repo </a>and entering it:</p><pre>git clone <a href="https://github.com/ams0/argocd-azure-k3s-traefik"><em>https://github.com/ams0/argocd-azure-k3s-traefik</em></a><em><br>cd argocd-azure-k3s-traefik</em></pre><p>Let’s create the infrastructure:</p><ul><li>One VM with ports open to access the Kubernetes APIs and ports 80/443 opened for our application):</li></ul><pre>./deploy.sh &lt;rg&gt; &lt;dns_name&gt; &lt;location&gt; &lt;size&gt;</pre><p><em>dns_name</em> should be unique in the region of choice. After a couple of minutes, you’ll have the <em>config</em> file for the k3s cluster (of one VM, with two virtual nodes inside as docker containers).</p><h3>Ingress installation</h3><p>You may have noticed that I skipped the installation of k3s’ built-in Traefik ingress; that’s because it still includes the 1.7 branch and I want to use the newer 2.x branch that introduced the IngressRoute CRD (you can follow the progress on this <a href="https://github.com/rancher/k3s/issues/1141">issue</a>). Let’s now install Traefik with this <a href="https://github.com/ams0/argocd-azure-k3s-traefik/blob/main/install-traefik.sh">script</a> (you’ll need to pass your email for the Let’sEncrypt certificate authority):</p><pre>./install-traefik &lt;email&gt;</pre><h3>Argo installation</h3><p>Finally you can add one or more clusters to be managed by ArgoCD — provided you already have the kubeconfig file available (use the azure-cli to retrieve it) — with this command:</p><pre>./install-argo.sh &lt;dns_name&gt; &lt;region&gt; &lt;password&gt;</pre><p>That’s it! The script will patch argocd-server to run over http (SSL termination is done by Traefik) and will patch the secret with the bcrypt-encoded version of your password. Navigate to <a href="https://dns_name.region.cloudapp.azure.com/"><em>https://dns_name.region.cloudapp.azure.com</em></a> and login to ArgoCD. You can also <a href="https://github.com/argoproj/argo-cd/releases/download">download the CLI</a> and login with:</p><pre>argocd login --username admin \<br>--password Password \<br>dns_name.region.cloudapp.azure.com</pre><p>Finally you can add one or more clusters to be managed by argo with (provided you already have the kubeconfig file available, for example using the azure cli to retrieve it):</p><pre>az aks get-credentials -g rg -n <em>cluster_name</em> -f kubeconfig<br>argocd cluster add --kubeconfig  ./kubeconfig manageme</pre><p>Note that the last option on the above command line must match the context inside the <em>kubeconfig</em> configuration.</p><p>Now, let’s deploy some apps!</p><h3>GitOps flow</h3><p>I’m a big fan of the <a href="https://github.com/fluxcd/helm-controller">Helm-controller</a> project and I wanted to use it with ArgoCD. In a nutshell, the controller lets you create objects of kind <em>HelmRelease</em> in your cluster (representing Helm releases) and manages the lifecycle of those helm releases programmatically (create/update/destroy). So in the <em>manifests/</em> folder in my repository, you’ll find the templates to deploy an helm-controller that will in turn deploy helmreleases also present in the same folder.</p><p>Fork the repository I provided at the top of this post and head over to the Argo UI tocreate a new app (call it to your liking, and choose the default project) pointing to the your fork and to the <em>manifests/</em> path.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*z-ZKpKiNIu85SAKxCZmZSw.png" /></figure><p>Importantly, make sure that directory recurse is on. You can also create an object of type “Application” inside your Argo/k3s cluster to achieve the same result:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f4ed314520fce851567a51bde9fdfb06/href">https://medium.com/media/f4ed314520fce851567a51bde9fdfb06/href</a></iframe><p>The app will start syncing right away, installing the Helm operator first and then an nginx ingress controller and you’ll see the tree of resources being created.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OMbojl19nbrfb1K1yw6Opg.png" /></figure><p>That’s it! Now every change to the github repo will be reflected in your cluster.</p><h3>Conclusion</h3><p>In conclusion, you might be tempted to ask: is this production ready? Absolutely not, there are still some aspects of the deployment I want to improve (AAD authentication for Argo, for instance, and persistence of data as well). However, it’s a quickstart with ArgoCD in Azure and it will help me learn more in the future. I hope you enjoy it too!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bGU7mCw45eS3mdbIuFWQRw.jpeg" /></figure><p><em>thanks to </em><a href="https://medium.com/u/1130dd55b52a"><em>Gino Filicetti</em></a><em> for a draft review of the article.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2de7daabbefa" width="1" height="1" alt=""><hr><p><a href="https://medium.com/cooking-with-azure/managing-multiple-clusters-with-argocd-in-azure-k3s-secured-w-traefik-lets-encrypt-2de7daabbefa">Managing multiple clusters with ArgoCD in Azure/k3s secured w/ Traefik&amp;Let’s Encrypt</a> was originally published in <a href="https://medium.com/cooking-with-azure">Cooking with Azure</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[NFSv4 with Azure Files and Azure Kubernetes Service]]></title>
            <link>https://medium.com/cooking-with-azure/nfsv4-with-azure-files-and-azure-kubernetes-service-141ee8258fd2?source=rss-92272b78ceea------2</link>
            <guid isPermaLink="false">https://medium.com/p/141ee8258fd2</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[azure-kubernetes-service]]></category>
            <category><![CDATA[azure-files]]></category>
            <category><![CDATA[storage]]></category>
            <category><![CDATA[nfs-client-provisioner]]></category>
            <dc:creator><![CDATA[Alessandro Vozza]]></dc:creator>
            <pubDate>Mon, 12 Oct 2020 20:19:15 GMT</pubDate>
            <atom:updated>2020-10-19T13:13:06.084Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Azure is introducing support for NFSv4 for Azure Files shared volumes, we will explore how to use dynamic volume provisioning with Azure Kubernetes Service</em></p><h3>Table of content</h3><ol><li><a href="#3b4d"><strong>Introduction</strong></a></li><li><a href="#d590"><strong>Infrastructure</strong></a></li><li><a href="#fa45"><strong>Azure Kubernetes Service</strong></a></li><li><a href="#9a2d"><strong>Performance testing</strong></a></li><li><a href="#33e9"><strong>Conclusion</strong></a></li></ol><h3>Introduction</h3><p>NFS is a <a href="https://en.wikipedia.org/wiki/Network_File_System">36-years old filesystem</a> but still extremely popular, even in this time and age of Cloud-Native-Everything. It’s simple, reliable and extremely useful when you need an <a href="https://kubernetes.io/docs/concepts/storage/persistent-volumes/#access-modes">RWX (<em>ReadWriteMany</em>)</a> filesystem for your Pods; Microsoft Azure recently a<a href="https://azure.microsoft.com/en-us/updates/azure-files-support-for-nfs-v41-is-now-in-preview/">nnounced its preview support</a> for the latest incarnation of NFS, NFSv4 for Azure Files shared volumes and I wanted to see how easy would be to use it with Azure Kubernetes Service.</p><h3>Infrastructure</h3><p>First, let’s prepare the necessary infrastructure using this handy script, based on this <a href="https://docs.microsoft.com/en-us/azure/storage/files/storage-files-how-to-create-nfs-shares?tabs=azure-portal">document </a>(note that NFSv4 is available only in a handful of <a href="https://docs.microsoft.com/en-us/azure/storage/files/storage-files-compare-protocols#regional-availability">regions</a> at this moment):</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/19eed041b982a29edc8fbb091788839f/href">https://medium.com/media/19eed041b982a29edc8fbb091788839f/href</a></iframe><p>Now we need to create an NFS file share and a Private Endpoint to reach the Storage Account. Since I’m not entirely sure how to do it reliably via the CLI (I think the NFSv4 share is not supported in the CLI yet) I resorted using the trustworthy Azure Portal; follow this <a href="https://docs.microsoft.com/en-us/azure/storage/files/storage-files-how-to-create-nfs-shares?tabs=azure-portal#create-an-nfs-share">guide </a>for the share and this one for the <a href="https://docs.microsoft.com/en-us/azure/storage/files/storage-files-networking-endpoints?tabs=azure-cli#create-a-private-endpoint">Private Endpoint</a> to the storage subnet.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eTy1wpT9moZva_yUPj_ViQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EFjXaBHtUHEfeIYPBon3Tg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/293/1*8GsaLpskD_xwIuIe7Xc9Lw.png" /></figure><p>Note how the chosen capacity for the NFS share will also influence the performance of the share; I did some quick performance test at the end of this article.</p><p>You’ll notice that the the private endpoint will also create a Private DNS zone in your resource group; we will need to add the AKS peered VNET to the list of VNETs that can resolve DNS names in that zone, so the nodes in AKS will be able to mount the NFS share using the DNS name (in our case, nfs4aks.privatelink.file.core.windows.net ). The command is:</p><pre>az network private-dns link vnet create -n dnslink \<br>-g $RG -v aksnet -z privatelink.file.core.windows.net -e false</pre><h3>Azure Kubernetes Service</h3><p>Lastly, let’s create a simple AKS cluster in the peered VNET:</p><pre>az aks create \<br>-g $RG \<br>-s Standard_B4ms -c 1 -n nfs4aks \<br>-l $REGION \<br>--vnet-subnet-id /subscriptions/$SUBSCRIPTION/resourceGroups/$RG/providers/Microsoft.Network/virtualNetworks/aksnet/subnets/aks</pre><pre>az aks get-credentials -g $RG -n nfs4aks</pre><p>The final architecture should look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/727/1*VcLFaU7Wz2F7N6VjklKwEg.png" /></figure><p>Time to provision some persistent storage. Here, we could have used the azure-file <a href="https://github.com/kubernetes-sigs/azurefile-csi-driver/tree/master/deploy/example/nfs">CSI driver</a> but I wanted to keep things very simple and use the nfs-client <a href="https://github.com/kubernetes-retired/external-storage/tree/master/nfs-client">provisioner </a>deployed via Helm (set the nfs.path to the name of the NFS share you created before):</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/decbe4ff41b75c9350f48f3dfaf03cf3/href">https://medium.com/media/decbe4ff41b75c9350f48f3dfaf03cf3/href</a></iframe><h3>Note that NFS doesn’t do any authentication and relies on network security (that is, the presence of the Private Endpoint in the peered vnet).</h3><h3>Performance testing</h3><p>Now we’re ready to benchmark our NFS share! I am going to use <a href="https://github.com/ams0/aks-nvme-ssd-provisioner/blob/master/bench.yaml">this template</a> from my <a href="https://medium.com/cooking-with-azure/using-nvm-e-instances-in-azure-kubernetes-service-40c587dbd67b">previous post on NVMe storage</a>:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4500911f16d41ed00a655d3bb2f833f2/href">https://medium.com/media/4500911f16d41ed00a655d3bb2f833f2/href</a></iframe><p>You can look at the logs of the dbench pod to get the summary of the benchmark:</p><pre>kubectl logs -l job-name=dbench -f</pre><p>After each test you’ll need to clean up, like so:</p><pre>kubectl delete job dbench<br>kubectl delete pvc dbench</pre><p>As you can see it seems that the IOPS scale linearly with increased capacity (while bandwidth seems capped by network throughput):</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/bcb41f2c4d45d9f94be48cd54376eb9e/href">https://medium.com/media/bcb41f2c4d45d9f94be48cd54376eb9e/href</a></iframe><h3>Conclusion</h3><p>I hope this helps and let me know if you plan to use NFSv4!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=141ee8258fd2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/cooking-with-azure/nfsv4-with-azure-files-and-azure-kubernetes-service-141ee8258fd2">NFSv4 with Azure Files and Azure Kubernetes Service</a> was originally published in <a href="https://medium.com/cooking-with-azure">Cooking with Azure</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using KubeVirt in Azure Kubernetes Service — part 3: Windows VM]]></title>
            <link>https://medium.com/cooking-with-azure/using-kubevirt-in-azure-kubernetes-service-part-3-windows-vm-363d6b653d7?source=rss-92272b78ceea------2</link>
            <guid isPermaLink="false">https://medium.com/p/363d6b653d7</guid>
            <category><![CDATA[azure]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[kubevirt]]></category>
            <category><![CDATA[virtualization]]></category>
            <category><![CDATA[azure-kubernetes-service]]></category>
            <dc:creator><![CDATA[Alessandro Vozza]]></dc:creator>
            <pubDate>Fri, 02 Oct 2020 09:46:21 GMT</pubDate>
            <atom:updated>2021-01-16T04:39:47.025Z</atom:updated>
            <content:encoded><![CDATA[<h3>Using KubeVirt in Azure Kubernetes Service — part 3: Windows VM and CSI volume cloning</h3><p><em>We’ll look at how to install a Windows Server 2019 VM inside Azure Kubernetes Service running KubeVirt.</em></p><p>We introduced Kubevirt in <a href="https://medium.com/cooking-with-azure/using-kubevirt-in-azure-kubernetes-service-part-1-8771bfb94d7">part 1</a> and <a href="https://medium.com/cooking-with-azure/using-kubevirt-in-azure-kubernetes-service-part-2-178396939aee?source=collection_detail----9d192c673322-----7-----------------------">2</a>, now it’s time to get a Windows VM up and running.</p><p>Start by downloading the <a href="https://www.microsoft.com/en-US/evalcenter/evaluate-windows-server-2019?filetype=ISO">Windows Server 2019 iso</a> from Microsoft (evaluation version). We will save this iso in a docker image and upload it to the Docker Hub (you can use any other registry of course) by leveraging the container disk image (a regular docker image that contains the iso in the /disk folder).</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/91ab03492077734b8ec38bfdfa8b775b/href">https://medium.com/media/91ab03492077734b8ec38bfdfa8b775b/href</a></iframe><p>Now you can just build and push the image:</p><pre>docker build -t &lt;registry&gt;/&lt;org&gt;/w2k9_iso:sep2020 .<br>docker push &lt;registry&gt;/&lt;org&gt;/w2k9_iso:sep2020</pre><p>The container image will be big and will take some time to upload, but now we have a Windows Server iso available to any AKS cluster that needs it by simply pulling that image. (Note: you could have uploaded the image directly to the cluster using the <a href="https://github.com/kubevirt/containerized-data-importer">Containerized Data Importer</a> project, this will be the subject of another blog post).</p><p>We will now create a PersistentVolumeClaim to hold our installation:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c8a95c07b1efc7c42e4e183b9bc6d9ef/href">https://medium.com/media/c8a95c07b1efc7c42e4e183b9bc6d9ef/href</a></iframe><p>Note the use of the managed-premium storage class for performance.</p><p>Let’s look now at the VM itself:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6f56ac51a163f39a0990b638dbae63a4/href">https://medium.com/media/6f56ac51a163f39a0990b638dbae63a4/href</a></iframe><p>Note that the install VM has 3 disks atttached:</p><ul><li>The PVC winhd as target to install Windows</li><li>The windows iso container disk created before</li><li>The <a href="https://wiki.libvirt.org/page/Virtio#:~:text=Virtio%20is%20a%20virtualization%20standard,and%20cooperates%20with%20the%20hypervisor..">virtio</a> driver disk</li></ul><p>Virtio is the type of devices that libvirt exposes to a VM in Kubevirt; Windows lacks the drivers in the standard iso distribution so we need to provide them at installation time. Creating the VM object will trigger the creation of a VMI (as the status is set to running: true ) which will spin up a virt-launcher Pod to hold the actual libvirt runtime and domain. It will take some time for the VMI object to go from Scheduling to Running because the large iso image must be downloaded by the host first.</p><p>Once running, you can connect to the VNC console for the VM using the command (you’ll need either virtctl or the virt plugin via Krew):</p><pre>kubectl virt vnc vm1</pre><p>(You can use the free <a href="https://sourceforge.net/projects/vcxsrv/">VcXsrv</a> with WSL2 on windows by disabling access control and exporting theDISPLAY environment variable).</p><p>You’ll be prompted with the Windows installation screen, go forward selecting the “Windows Server 2019 Standard with Desktop Experience”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1020/1*fLvBhzgxMmxKFlmgG_0zZg.png" /></figure><p>until you see the disk selection screen which shows a discouraging empty list. At that point, click on Load drivers and browse to the virtio disk. You’ll need to load <strong>both</strong> the network (from \NetKVM\2k19\amd64 folder) and the disk driver from \viostor\2k19\amd64).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EKt5w2QfGseAM7z-ATAOuw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1022/1*WMw7hJPCQ-kw-H9ghUkTzw.png" /></figure><p>Now you’ll be able to see the persistent volume to install Windows to. Complete the installation and when the VM reboots, choose a password and enable RDP in the Local Server Manager.</p><p>You can now delete the VM with:</p><pre>kubectl delete vm vm1</pre><p>The PersistentVolume just created will, well, <em>persist</em> and you can now create a new VM without the ISOs; we are also create a LoadBalancer service to expose the RDP port:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f2012ebeacba79dbff1ca453671756e6/href">https://medium.com/media/f2012ebeacba79dbff1ca453671756e6/href</a></iframe><p>You should now be able to connect to the VM via RDP over the Loadbalancer IP.</p><p>Extra tip: if you are using the new <a href="https://github.com/kubernetes-sigs/azuredisk-csi-driver">CSI volume provisioner</a> for azure-disk, you can clone the volume where you installed Windows before spinning up a new VM. First install the CSI driver with snapshot enabled:</p><pre>kubectl create -f https://raw.githubusercontent.com/kubernetes-sigs/azuredisk-csi-driver/master/deploy/example/storageclass-azuredisk-csi.yaml</pre><p>Then create a new StorageClass:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/df7d77a0c339b9c792b8bfd719b41d70/href">https://medium.com/media/df7d77a0c339b9c792b8bfd719b41d70/href</a></iframe><p>You can choose different disk types too here (the above uses the <a href="https://docs.microsoft.com/bs-latn-ba/azure/virtual-machines/disks-enable-ultra-ssd?tabs=azure-portal">UltraSSD type</a>). Now create the winhad PVC referring to this new StorageClass:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/236381b72d8c7526f04eeeba0c8fe2e6/href">https://medium.com/media/236381b72d8c7526f04eeeba0c8fe2e6/href</a></iframe><p>Now go on and install Windows on the winhd PVC. Delete the VM and before creating a new one, create a new PVC using the first PVC as source:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/540f7d2f3728269115995cdde68a6a0c/href">https://medium.com/media/540f7d2f3728269115995cdde68a6a0c/href</a></iframe><p>and refer to winhd-clone now when you create a new VM.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=363d6b653d7" width="1" height="1" alt=""><hr><p><a href="https://medium.com/cooking-with-azure/using-kubevirt-in-azure-kubernetes-service-part-3-windows-vm-363d6b653d7">Using KubeVirt in Azure Kubernetes Service — part 3: Windows VM</a> was originally published in <a href="https://medium.com/cooking-with-azure">Cooking with Azure</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using KubeVirt in Azure Kubernetes Service — part 2]]></title>
            <link>https://medium.com/cooking-with-azure/using-kubevirt-in-azure-kubernetes-service-part-2-178396939aee?source=rss-92272b78ceea------2</link>
            <guid isPermaLink="false">https://medium.com/p/178396939aee</guid>
            <category><![CDATA[azure]]></category>
            <category><![CDATA[kubevirt]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[azure-kubernetes-service]]></category>
            <dc:creator><![CDATA[Alessandro Vozza]]></dc:creator>
            <pubDate>Sun, 20 Sep 2020 23:44:05 GMT</pubDate>
            <atom:updated>2020-10-02T09:47:08.345Z</atom:updated>
            <content:encoded><![CDATA[<h3>Using KubeVirt in Azure Kubernetes Service — part 2: Installation</h3><p><em>After the introduction to KubeVirt in </em><a href="https://medium.com/cooking-with-azure/using-kubevirt-in-azure-kubernetes-service-part-1-8771bfb94d7"><em>part 1</em></a><em>, we continue in this post to get to know KubeVirt in AKS by installing it and running our first VirtualMachine</em></p><p>We look at how KubeVirt implements the objects necessary to run VirtualMachines workloads in Kubernetes in part 1, and now we’ll get practical deploying those objects to an Azure Kubernetes Cluster. First, let’s create a basic 1-node cluster:</p><pre>az aks create \<br>--resource-group k8s \<br>--network-policy calico \<br>--network-plugin kubenet \<br>--node-vm-size Standard_B4ms \<br>--node-count 1 \<br>--name kubevirt</pre><p>Add a second nodepool with a VM size that has the Intel virtualization extensions (VT-x) enabled (those from the <em>Ds_v3 </em>series all have them):</p><pre>az aks nodepool add \<br>--resource-group k8s \<br>--cluster-name kubevirt \<br>--name nested \<br>--node-vm-size Standard_D4s_v3 \<br>--labels nested=true \<br>--node-count 1</pre><p>Note how we’re attaching a <em>label</em> to the nodepool; this way, scaling the pool will make sure the new instances will all get the same label (which comes in handy later when we’ll want to schedule our VM’s on nodes with native CPU acceleration).</p><p>Retrieve the credentials for the cluster and check the nodes:</p><pre>az aks get-credentials -g k8s -n kubevirt<br>kubectl get nodes</pre><pre>NAME                                STATUS   ROLES   AGE   VERSION<br>aks-nested-72121939-vmss000000      Ready    agent   25m   v1.17.9<br>aks-nodepool1-72121939-vmss000000   Ready    agent   27m   v1.17.9</pre><p>(you can use the --show-labels kubectl option to confirm the label is attached to the second pool).</p><p>Time to install Kubevirt! First let’s install the handy virtctl tool as a <a href="https://github.com/kubernetes-sigs/krew">krew</a> plugin for kubectl:</p><pre>kubectl krew install virt</pre><p>And install the kubevirt operator and a single instance of the KubeVirt object:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/43c241d41cd4457611a4c6a7e76dc0a1/href">https://medium.com/media/43c241d41cd4457611a4c6a7e76dc0a1/href</a></iframe><p>You’ll notice in the kubevirt namespace the <em>virt-operator</em> pod that will in turn create the <em>virt-api </em>and <em>virt-controller</em> (deployments with replica=2) and a DaemonSet for the <em>virt-handler</em>.</p><p>When all pods are up, we can create a VM object that will in turn create a VirtualMachineInstance object (the actual running VM). First, I’d like to save the cloud-config as secret in Kubernetes (in my case, I’m not injecting a password but simply my public SSH key, but it’s a good practice to have the configuration separated from the actual VM object):</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/41391a94ed258cb92be07349c9ee1ea8/href">https://medium.com/media/41391a94ed258cb92be07349c9ee1ea8/href</a></iframe><p>And finally create the VM object:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6ee6d06c046101db1d13d37fc44883b9/href">https://medium.com/media/6ee6d06c046101db1d13d37fc44883b9/href</a></iframe><p>Few things to notice:</p><ul><li>the nodeSelector for making sure the pod gets schedule on a node with nested virtualization</li><li>the request for 1Gb of RAM</li><li>the use of containerDisk to download automatically the latest Fedora image; <em>containerDisks </em>are full cloud images packaged into a Docker image (you can find all available versions on <a href="https://hub.docker.com/r/kubevirt/fedora-cloud-container-disk-demo/tags">DockerHub</a>).</li><li>the cloud-init script referenced from a Kubernetes secret.</li></ul><p>Creating the object of type VirtualMachine will create a VirtualMachineInstance (because the spec.running: true property) which will force the virt-handler on the node to run a pod with libvirt` to actually run the Virtual Machine ( domain in libvirt parlance). You can check:</p><pre>kubectl get po,vm,vmi<br>NAME                          READY   STATUS    RESTARTS   AGE<br>pod/virt-launcher-vm1-g96j6   2/2     Running   0          16m<br><br>NAME                             AGE   VOLUME<br>virtualmachine.kubevirt.io/vm1   19m<br><br>NAME                                     AGE   PHASE     IP           NODENAME<br>virtualmachineinstance.kubevirt.io/vm1   16m   Running   10.244.1.8   aks-nested-72121939-vmss000000</pre><p>And now, for the grand finale, it’s time to expose that VM so we can access it: it’s time to create a service to connect to. It can be done with the <em>virtctl </em>command <em>expose</em> or simply by creating a Service object (note how the same rules for label/selector apply: the service endpoint list will be populated by all VirtualMachines with lables matching the selector, so it’s possible to have access to multiple VM’s from a single Service).</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/629666dc10278a607b19c90c6f507685/href">https://medium.com/media/629666dc10278a607b19c90c6f507685/href</a></iframe><p>Wait for the public IP to show up on the load balancer then SSH to the VM:</p><pre>ssh fedora@`kubectl get svc  vm1ssh -o custom-columns=&quot;:.status.loadBalancer.ingress[0].ip&quot; --no-headers=true`</pre><p>You’re in! It’s a simple VM but feels like a great achievement. Coming up in the next articles: storage, running Windows and more.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=178396939aee" width="1" height="1" alt=""><hr><p><a href="https://medium.com/cooking-with-azure/using-kubevirt-in-azure-kubernetes-service-part-2-178396939aee">Using KubeVirt in Azure Kubernetes Service — part 2</a> was originally published in <a href="https://medium.com/cooking-with-azure">Cooking with Azure</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using KubeVirt in Azure Kubernetes Service — part 1]]></title>
            <link>https://medium.com/cooking-with-azure/using-kubevirt-in-azure-kubernetes-service-part-1-8771bfb94d7?source=rss-92272b78ceea------2</link>
            <guid isPermaLink="false">https://medium.com/p/8771bfb94d7</guid>
            <category><![CDATA[azure]]></category>
            <category><![CDATA[virtualization]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[azure-kubernetes-service]]></category>
            <category><![CDATA[kubevirt]]></category>
            <dc:creator><![CDATA[Alessandro Vozza]]></dc:creator>
            <pubDate>Sun, 20 Sep 2020 23:41:14 GMT</pubDate>
            <atom:updated>2020-10-02T09:46:47.306Z</atom:updated>
            <content:encoded><![CDATA[<h3>Using KubeVirt in Azure Kubernetes Service — part 1: Introduction</h3><p><em>A primer on using kubevirt to run VirtualMachines as container workloads on the Azure Kubernetes service. This first post is intended as an introduction to KubeVirt. You can read the second part </em><a href="https://medium.com/cooking-with-azure/using-kubevirt-in-azure-kubernetes-service-part-2-178396939aee"><em>here</em></a><em>.</em></p><p><a href="https://kubevirt.io/"><em>KubeVirt</em></a> is an open source project whose aim is to help running virtual machine workloads natively within a Kubernetes cluster, by extending the core with components that facilitate the process (using Custom Resource Definitions like VirtualMachines and Disks). It allows for co-existence of mixed workloads and smoother transition from legacy VMs to containers (like a step-wise microservices adoption) unified by the control plane and the familiar tooling (<em>kubectl</em> and the optional <em>virtctl</em>) of Kubernetes.</p><h3><strong>Why kubevirt?</strong></h3><p>For a recent project where we needed to run VMs side-by-side containers, we evaluated a number of options to run VM’s side-by-side with containers:</p><p>- <a href="https://github.com/Azure/azure-service-operator"><strong>Azure Service Operator </strong></a><strong>(ASO)</strong></p><p>The Azure Service Operator is a new project born within Commercial Software Engineering aimed at managing Azure resources as native Kubernetes objects. Altough very promising, we put it aside as we recognize it’s still rough around the edges (see this [<em>issue</em>](https://github.com/Azure/azure-service-operator/issues/1167)).</p><p><em>- </em><a href="https://github.com/hashicorp/terraform-k8s"><strong>Terraform Operator for Kubernetes</strong></a></p><p>A new project from Hashicorp that aims at allowing generic Terraform templates lifecycle management natively in Kubernetes. It could have been a good alternative (after all, we did develop extensive templates for VM/VMSS in the first project phase) but it’s still considered alpha so we will not research it further.</p><p>Both the above projects are interesting and we’ll be considered in the future but we’re focusing on kubevirt to spin up the necessary VMs for the time being.</p><h3>KubeVirt Architecture</h3><p>KubeVirt runs VirtualMachine as embedded in pods running an hypervisor (commonly, `libvirtd`) and exposes them via Kubernetes services:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/939/1*i-hwekMXGejrezA4sHpE9w.png" /></figure><p>Kubevirt is composed of <a href="https://kubernetes.io/blog/2018/05/22/getting-to-know-kubevirt/#kubevirt-components">several components</a>:</p><p>- <em>virt-controller</em> (cluster component, watches for `VirtualMachine` CRDs)</p><p>- <em>virt-api</em> (cluster component, the interface to the controller)</p><p>- <em>virt-handler</em> (DaemonSet)</p><p>- <em>virt-launcher</em> (for every VM pod)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cVoRd4SuwPwnUjhIeUVl1Q.png" /></figure><h3>Implementation</h3><p>Kube Virt (KV from now on) aims to maintain the so-called Application Binary Interface (ABI) stability (a fancy way to say that the aim is to have the same UX as using native VMs) and management capabilities for VMs in Kubernetes. Kube Virt adds a basic CRD to the Kubernetes API called `<em>VirtualMachine</em>` to abstract the concept of a VM; the controller watches for this kind of object being added/modified/deleted and when one is created with `<em>spec.running</em>` set to true it instantiates a new object of kind `<em>VirtualMachineInstance</em>` which represents the actual VM.</p><p>Importantly, the `VirtualMachineInstance` can exposed as a <a href="https://kubevirt.io/user-guide/#/usage/network-service-integration"><strong>regular Kubernetes service</strong></a> of type `<em>LoadBalancer</em>`, `<em>NodePort</em>` or `<em>ClusterIP</em>`. It uses the standard label/selector pair to attach the service to the `<em>VirtualMachineInstance</em>`.</p><h3>VirtualMachine Object</h3><p>The VM object defines the specs for the VirtualMachine and contains the spec for the VirtualMachineInstance (under <em>spec.domain</em>):</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7a5fbac0f1bdcc687dc5a3d2fe63c3bb/href">https://medium.com/media/7a5fbac0f1bdcc687dc5a3d2fe63c3bb/href</a></iframe><p>Notable in this implementation:</p><p>- `<em>spec.running</em>` indicates if the VM should be running or stopped. Can be manipulated by `<em>kubectl patch</em>` or manual editing or directly with `<em>virtctl</em>`:</p><pre>$ virtctl start myvm<br>#is equvalent to<br>kubectl patch virtualmachine myvm --type merge -p <em>&#39;{&quot;spec&quot;:{&quot;running&quot;:false}}&#39;</em></pre><p>- `<em>resources.requests</em>` have the same effects as on containers, allocating resources to the virtual machines.</p><p>- `<em>disks</em>` in the VM are mapped to regular Kubernetes persisten `<em>volumes</em>` of type `<em>containerDisk</em>`, which are prepared beforehand using the <a href="https://github.com/kubevirt/containerized-data-importer"><strong>Containerized Data Importer</strong></a>. The disk can also come directly from an URL using the `dataVolumeTemplates` (see Part 2).</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8771bfb94d7" width="1" height="1" alt=""><hr><p><a href="https://medium.com/cooking-with-azure/using-kubevirt-in-azure-kubernetes-service-part-1-8771bfb94d7">Using KubeVirt in Azure Kubernetes Service — part 1</a> was originally published in <a href="https://medium.com/cooking-with-azure">Cooking with Azure</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>