<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by David Zisky on Medium]]></title>
        <description><![CDATA[Stories by David Zisky on Medium]]></description>
        <link>https://medium.com/@dawid.ziolkowski?source=rss-d3491d6b8e23------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*vnJWWCaFYc6u4dopBwUoeQ.jpeg</url>
            <title>Stories by David Zisky on Medium</title>
            <link>https://medium.com/@dawid.ziolkowski?source=rss-d3491d6b8e23------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 22:59:23 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@dawid.ziolkowski/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[About Azure AKS Default Node Pool]]></title>
            <link>https://itnext.io/about-azure-aks-default-node-pool-2067d03ef1bc?source=rss-d3491d6b8e23------2</link>
            <guid isPermaLink="false">https://medium.com/p/2067d03ef1bc</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[azure]]></category>
            <dc:creator><![CDATA[David Zisky]]></dc:creator>
            <pubDate>Sat, 18 Apr 2020 14:25:44 GMT</pubDate>
            <atom:updated>2020-04-18T18:43:08.504Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*2hxbjBeIgn4J7VFZ" /><figcaption>Photo by <a href="https://unsplash.com/@tvick?utm_source=medium&amp;utm_medium=referral">Taylor Vick</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>If you ever used Azure Managed Kubernetes service you probably needed to change or delete the default node pool at least once — which is the one that is being created at the time of creating the cluster, so essentially — the first node pool in the cluster. If you didn’t, then I’ll tell you — Azure won’t allow you to do it. Let’s dive into Azure AKS node pools and see how to delete that default node pool if you’ll ever need that.</p><p><strong>What is a node pool and why would you need to have more than one:</strong></p><p>A node pool as the name suggests groups x amount of nodes into a logical group. Each of these groups can have a different configuration — and that’s the point of node pools. Nodes in one pool can have different VMs SKUs (different VMs types), different Kubernetes versions, different pod-per-node limits, etc. From the Kubernetes perspective, they all land in the same basket by default — so for example if you’ll create 3 node pools in Azure — you’ll see all the nodes from all 3 node pools in the output of <em>kubectl get nodes.</em> What’s the point then? Well, I mentioned, “by default”. But, by using a few Kubernetes <a href="https://kubernetes.io/docs/concepts/configuration/assign-pod-node/">mechanisms</a>, you can use different node pools for different purposes. Few examples:</p><ul><li>Group your workflows logically, like one node pool for production applications, other node pool for dev/test.</li><li>Group node pools by VMs type — have one node pool with standard machines and one extra node pool with bigger machines (more CPU/RAM) — for example, to keep specific applications which require more computing power on these.</li><li>Group node pools by “importance” — for example, have one node pool which has a lower pods-per-node limit and runs older Kubernetes version to keep the most important microservices there and another node pool, with higher pods-per-node limit and latest Kubernetes version.</li><li>Group node pools by autoscaling limits. For example — have one node pool with autoscaling set to some reasonable number and keep in this node pool all non-critical applications and another node pool with very high autoscaling limit for critical applications. In this way, when you’ll have a spike in the traffic, your main microservices (for example the one running APIs) will scale accordingly, but keep non-critical parts of the system on a node pool which will scale within some range but won’t scale too much to save costs.</li></ul><p>These are only a few examples of why you would like to have multiple node pools for your AKS cluster. It all depends on your specific use case and your needs but the bottom line is, that usually — multiple node pools are useful.</p><p>Now, let’s discuss the “default” node pool. It actually isn’t officially called that way anymore, but you may find references to that name. Basically we are talking about the node pool which is created at the time of creating your AKS cluster. From Azure perspective, that node pool is being created as a “system” node pool and whenever you add a new node pool it’s being created as a “user” node pool. What’s the difference between these two? “System” node pool simply runs pods needed for AKS to work properly (CoreDNS, Kuberentes Dashboard, tunnelfront, metrics-server, omsagentetc), while, as you can probably guess, a user node pool after being created has only kube-proxy pods on nodes, nothing else.</p><p>Unfortunately, in the Azure portal you won’t see which is which. From there both look the same:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/981/1*jLHamkazDxwe_S5Kstf3hw.png" /><figcaption>“agentpool” — default node pool</figcaption></figure><p>However, if you would try to delete the “secondpool” node pool, Azure would delete it just fine, but if you would try to do the same to the “agentpool” then Azure will complain:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/370/1*zAhvum6gzGfkmp89Z84c7A.png" /></figure><p>Since “system” node pool contains pods created by Azure and needed for AKS to function properly then it kinda makes sense to prevent deleting that node pool, right? But sometimes you really need to delete that default node pool (for example to recreate it with a different pods-per-node limit or simply different VMs type). In this case, your only option is to use <a href="https://docs.microsoft.com/en-us/cli/azure/?view=azure-cli-latest">Azure command-line tool</a> “az”.</p><p>I mentioned earlier that within Azure Portal both node pools will look the same. But when listing node pools with “az” command you will see which one is a system and which is in user mode:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/868/1*cVq0wAf2COuHNB5BLyTJUw.png" /></figure><p>Now we have two options:</p><ol><li>Change one of your existing user node pools into a system node pool and then delete the default node pool</li><li>Create a new node pool as a system node pool.</li></ol><p>The first option is as easy as executing the following command:</p><pre>az aks nodepool update --mode System --name [your_node_pool] --resource-group [resource_group] --cluster-name [cluster_name]</pre><p>and similarly, the second option is to pass the same “mode” argument when creating a new node pool:</p><pre>az aks nodepool add --mode System --name [your_node_pool] --resource-group [resource_group] --cluster-name [cluster_name]</pre><p>If you do one of these and list your node pools again, you should now see that we have more than one System node pool:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/877/1*fNR4Pz1mnXP2UVkLPzZNiA.png" /></figure><p>… and that let us delete the default node pool without any issues (both via Azure Portal or CLI):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/778/1*wvxHBzNipjRWBh0jNmfRig.gif" /></figure><p>That’s it. To summarize: in order to delete AKS default node pool, you need to create or nominate another node pool to be in “System” mode and as long as there is at least one system node pool then it doesn’t matter anymore for Azure which node pool you want to delete.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2067d03ef1bc" width="1" height="1" alt=""><hr><p><a href="https://itnext.io/about-azure-aks-default-node-pool-2067d03ef1bc">About Azure AKS Default Node Pool</a> was originally published in <a href="https://itnext.io/">ITNEXT</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kafka-aware security for Kubernetes]]></title>
            <link>https://itnext.io/kafka-aware-security-for-kubernetes-a57b22bd0852?source=rss-d3491d6b8e23------2</link>
            <guid isPermaLink="false">https://medium.com/p/a57b22bd0852</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[kafka]]></category>
            <category><![CDATA[cilium]]></category>
            <category><![CDATA[bpf]]></category>
            <dc:creator><![CDATA[David Zisky]]></dc:creator>
            <pubDate>Mon, 24 Feb 2020 21:32:15 GMT</pubDate>
            <atom:updated>2020-02-26T16:21:20.221Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*xrugjFO3hrBU3ZuS" /><figcaption>Photo by <a href="https://unsplash.com/@oni71?utm_source=medium&amp;utm_medium=referral">Miłosz Klinowski</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>When it comes to Kafka, fine-grained security is very important. Usually, data flowing through Kafka comes from lots of components and often consist of information that shouldn’t be consumed by anyone unprivileged. With the help of <a href="https://cilium.io/">Cilium</a> we can enforce Kafka-aware security policies directly on the Linux kernel level, which means without any changes to the application code or container configuration.</p><p>With a standard Kafka setup, any user or application can write any messages to any topic, as well as read data from any topics.</p><p>A typical approach for securing Kafka is by issuing SSL certificates for each client and forcing Kafka brokers to verify their validity. Another option is to go for SASL which can be applied in a few different ways (classic username/password, Kerberos, etc), but it’s not really that easy to start with. And that’s only Authentication, moving forward to Authorization — things become even more complicated (using kafka-acl command in bigger scale is challenging). Let’s see how can we do it completely differently and independently from Kafka.</p><p>We are going to use a tool called Cilium which uses eBPF technology to secure Kafka on a completely different level. First, let’s talk about Cilium itself. A good explanation is on their own <a href="https://cilium.io/">website</a>:</p><blockquote>Existing Linux network security mechanisms (e.g., iptables) only operate at the network and transport layers (i.e., IP addresses and ports) and lack visibility into the microservices layer.</blockquote><blockquote>Cilium brings API-aware network security filtering to Linux container frameworks like <a href="https://www.docker.com/"><strong>Docker</strong></a> and <a href="https://kubernetes.io/"><strong>Kubernetes</strong></a>. Using a new Linux kernel technology called <strong>eBPF</strong>, Cilium provides a simple and efficient way to define and enforce both network-layer and application-layer security policies based on container/pod identity.</blockquote><p>So what you need to know about Cilium is that it’s a tool which you can install on Kubernetes and it uses eBPF to bring next level or cloud native network filtering. What’s eBPF then? Best explained by <a href="http://www.brendangregg.com/blog/2019-01-01/learn-ebpf-tracing.html">Brendan Gregg</a>:</p><blockquote><strong>eBPF</strong> does to Linux what JavaScript does to HTML. (Sort of.) So instead of a static HTML website, JavaScript lets you define mini programs that run on events like mouse clicks, which are run in a safe virtual machine in the browser. And with eBPF, instead of a fixed kernel, you can now write mini programs that run on events like sending/receiving TCP packet, which are run in a safe virtual machine in the kernel. In reality, eBPF is more like the v8 virtual machine that runs JavaScript, rather than JavaScript itself. eBPF is part of the Linux kernel.</blockquote><p>How all this translates to securing Kafka on Kubernetes then? Cilium, by using eBPF technology can talk to Linux kernel directly and apply security policies to pods like “pod X can only read from Kafka topic Y and pod Z can write only to Kafka topic W”. And all of that is being enforced by applying simple yaml definitions onto our Kubernetes cluster.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*NXqQcloreLXOCpqn" /><figcaption>Photo by <a href="https://unsplash.com/@brazofuerte?utm_source=medium&amp;utm_medium=referral">Collin Armstrong</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>We have enough theory now, so let’s get to the point. Assuming you have Kafka cluster running on your Kubernetes cluster, the first thing you need to do is to install Cilium of course. I won’t cover it here as it’s pretty straight forward by following instructions on <a href="https://docs.cilium.io/en/v1.7/gettingstarted/#installation">Cilium docs</a>. If you have Cilium installed you are ready to secure your Kafka cluster with one (or more) simple YAML file(s). As I mentioned earlier “with a standard Kafka setup, any user or application can write any messages to any topic, as well as read data from any topics”. So if you just installed Kafka it’s wide open. Let’s say we want to have one producer which is only allowed to write to two specific topics, and two consumers which can only read from one of the two topics. A very simple example but just to show you how easy can we go from completely insecure Kafka cluster to fully secure one. The only thing we need to do to achieve above is to apply (<em>kubectl apply -f make-my-kafka-secure.yaml</em>) this simple YAML file:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/059270f0627e0f9d8feb5668101d0219/href">https://medium.com/media/059270f0627e0f9d8feb5668101d0219/href</a></iframe><p>So what happens when we do that? Kubernetes will create a new CiliumNetworkPolicy resource which will be picked up by Cilium which from that point will monitor all Kafka pods (it will find them by labels) and block them from doing anything else than specified in the rules section of YAML file:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b4f4f8c26a3c594b3be68f89ee95e9ec/href">https://medium.com/media/b4f4f8c26a3c594b3be68f89ee95e9ec/href</a></iframe><p>Let’s get back to the whole YAML file and break it down to pieces.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d3f5ff440ffef463622d7e981897314f/href">https://medium.com/media/d3f5ff440ffef463622d7e981897314f/href</a></iframe><p>That’s pretty simple, right? As any Kubernetes resource, we need to specify apiVersion, description, kind and at least the name in the metadata section.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/df2852966e2419fe14ef6fe72752c1b9/href">https://medium.com/media/df2852966e2419fe14ef6fe72752c1b9/href</a></iframe><p>Next, in the spec section, we need to tell Cilium how to find out Kafka broker by providing its pod label in “endpointSelector”.</p><p>And then what rule to apply against that broker. In “ingress” &gt; “fromEndpoint” we specify our clients — so pods which will be connecting to our Kafka cluster (so either produce or consume something to/from Kafka). “toPorts” simply tells on which port is Kafka listening and then in rules we basically tell what the pod specified in “fromEndpoints” will be allowed to do. Full list of possible rules parameters is available here:</p><p><a href="http://docs.cilium.io/en/latest/policy/language/#kafka-beta">Layer 3 Examples - Cilium 1.7.0 documentation</a></p><p>…that’s how simple it is to apply Kafka-aware security policies for Kubernetes. Logic like “allow a pod to only produce on Kafka topic “topicA” and consume on topic “topicB” is easily achievable with Cilium.</p><p>You also need to keep in mind that neither code of the applications in the containers, nor pod configurations have been changed, yet we gained full control over Authentication and Authorization of our Kafka setup by simply applying YAML configuration. Last but not least — when the consumer (or producer) tries to do something that it is not allowed to — it receives an actual Kafka error message, like:</p><blockquote>WARN Error while fetching metadata with correlation id 20 : {example-topic=TOPIC_AUTHORIZATION_FAILED} (org.apache.kafka.clients.NetworkClient)</blockquote><p>And here we come to the point of how Cilium works — it understands the whole traffic flowing within your cluster and can secure it accordingly. It’s not limited to Kafka only, but understands Kafka too. You can use it pretty much the same way for HTTP traffic (think of rules like “pod A can only do a GET call to pod B on /info endpoint”). If you are interested in all of its capabilities, read more on the Cilium website, or wait for my next article ;)</p><p><a href="https://cilium.readthedocs.io/en/stable/">Welcome to Cilium&#39;s documentation! - Cilium 1.7.0 documentation</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a57b22bd0852" width="1" height="1" alt=""><hr><p><a href="https://itnext.io/kafka-aware-security-for-kubernetes-a57b22bd0852">Kafka-aware security for Kubernetes</a> was originally published in <a href="https://itnext.io/">ITNEXT</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Helm 3 in real life]]></title>
            <link>https://itnext.io/helm-3-in-real-life-dfdc1522dd04?source=rss-d3491d6b8e23------2</link>
            <guid isPermaLink="false">https://medium.com/p/dfdc1522dd04</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[helm]]></category>
            <dc:creator><![CDATA[David Zisky]]></dc:creator>
            <pubDate>Sat, 08 Feb 2020 15:22:49 GMT</pubDate>
            <atom:updated>2020-05-27T12:37:08.121Z</atom:updated>
            <content:encoded><![CDATA[<h3>Helm 3 in real life (part 2)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*_j5weh3mWGcRdniS" /><figcaption>Photo by <a href="https://unsplash.com/@ikukevk?utm_source=medium&amp;utm_medium=referral">Kevin Ku</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>A few weeks ago I wrote an article about changes between Helm2 and Helm3:</p><p><a href="https://itnext.io/helm2-vs-helm3-part-1-c76c29106e99">Helm2 vs Helm3 (part 1)</a></p><p>Since then, I’ve been using Helm3 both for private tests and for clients. So here I am again — ready to share with you some issues you will face when using Helm3.</p><h3>Changes you need to make:</h3><p><strong>Rewrite all your CI/CD pipelines</strong>. There are a few changes which require you to rewrite all your CI/CD pipelines. I listed them already in the previous article (removal of helm init, the name needs to be specified, namespaces need to exist, etc), but I need to point it out here again because that was so far the biggest issue for most companies. Check also a nice article by <a href="https://medium.com/@rudolphjacksonm">Jack Morris</a> on this:</p><p><a href="https://itnext.io/breaking-changes-in-helm-3-and-how-to-fix-them-39fea23e06ff">Breaking Changes in Helm 3 (and How to Fix Them)</a></p><p>Pipelines can be very complex and finding all the occurrences of necessary helm parts for change can be trickier than you may think. So please double check that you removed all helm inits, that you create namespaces prior to installing helm3 charts, that you provide a name (or use — generate-name), etc. I’ve seen few cases when most used pipelines were changed properly but those which are used less often (like rollback pipeline for example) weren’t. And that’s a dangerous situation because you may already feel confident with Helm3 and then it hits you when you decide to move to production.</p><h3>Issues you may face:</h3><p><strong>CRD installation — it fails, but it doesn’t</strong></p><p>You may see the following errors when installing helm charts which contain CRDs:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/970/1*C37qDGNKZSca0C6slQIMUw.png" /></figure><p>Stop panicking! It’s actually a fake error and they have been installed properly:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2NqtYd3g9Ur7pibnqKrzNA.png" /></figure><p>You see that error because Helm 3 deprecated the hook for crds and has now a policy that they should be in a crd directory. But it installs them anyway ;) More info here:</p><p><a href="https://v3.helm.sh/docs/topics/charts_hooks/">Chart Hooks</a></p><p><strong>“UPGRADE FAILED: rendered manifests contain a new resource that already exists”</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zGeYtdY-02YJCiCK00ZH_A.png" /></figure><p>Another nasty issue. It may happen sometimes when you try to run “helm upgrade”. But you may be wondering… wait… wasn’t the 3-way merge patch (taking live state into account while doing an upgrade) one of the key features of Helm 3? So how come that my upgrade fails due to “resource already existing” ??? Well, that particular issue is a bit more complicated and unfortunately, the error message doesn’t explain properly what’s happening here. You see — this issue happens only when you change “apiVersion” of the resource and then try to upgrade it. If you would change anything else (metadata, spec) then it would work fine as expected. But if you change this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/772/1*29zQM9s2Spqx1Zg7IrKetA.png" /></figure><p>to this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/656/1*eYkRQfb20soibu-E8j7fbw.png" /></figure><p>…then it will fail because kubernetes treats apiVersion change as a breaking change and does not allow to change it just like this. There is some underground logic required to make that happen which Helm 3 didn’t implement (<a href="https://github.com/helm/helm/issues/6850">yet</a>). There is a workaround for this — you can upgrade specific resource manually (via kubectl) and then remove Helm secret for that particular Chart. Or just don’t upgrade apiVersion until Helm fixes the problem ;)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dfdc1522dd04" width="1" height="1" alt=""><hr><p><a href="https://itnext.io/helm-3-in-real-life-dfdc1522dd04">Helm 3 in real life</a> was originally published in <a href="https://itnext.io/">ITNEXT</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to integrate k3s with the cloud controller]]></title>
            <link>https://itnext.io/how-to-integrate-k3s-with-the-cloud-controller-36bd5020b8f7?source=rss-d3491d6b8e23------2</link>
            <guid isPermaLink="false">https://medium.com/p/36bd5020b8f7</guid>
            <category><![CDATA[digitalocean]]></category>
            <category><![CDATA[k3]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[David Zisky]]></dc:creator>
            <pubDate>Tue, 21 Jan 2020 13:28:15 GMT</pubDate>
            <atom:updated>2022-01-28T20:06:55.660Z</atom:updated>
            <content:encoded><![CDATA[<h3>How to integrate k3s with a cloud controller</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*33KWBeuJYLftuDAb" /><figcaption>Photo by <a href="https://unsplash.com/@dallasreedy?utm_source=medium&amp;utm_medium=referral">Dallas Reedy</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>k3s is great! I already wrote an article on how to create a 4 nodes k3s cluster in under 60 seconds (including VM provisioning — <a href="https://medium.com/@dawid.ziolkowski/how-to-create-a-kubernetes-cluster-in-under-60-seconds-7320a58f5b0">here</a>). The only problem is that it doesn’t come with cloud providers support which means you can’t use LoadBalancers, storage, etc as easily as on managed Kubernetes. But there is a way to do it and that’s what this article is about. So let’s get into it</p><p>Integrating CCM (Cloud Controller Manager) with k3s basically means that your k3s cluster will be able to talk to cloud provider API in order to request and configure things like LoadBalancers (for ingress), apply appropriate labels for nodes, etc. The process differs between cloud providers (and it depends if cloud provider provides CCM in the first place). In this article, we will be installing k3s on DigitalOcean (let me know in the comments if you would like to see an article for different clouds).</p><p>The whole process isn’t very complicated but you need to follow the steps carefully. So, first, the master node. We need to pass the following parameters when installing k3s master node:</p><ul><li>— disable-cloud-controller</li><li>— no-deploy servicelb</li><li>— kubelet-arg=”cloud-provider=external”</li><li>— kubelet-arg=”provider-id=digitalocean://{master_node_id}”</li></ul><p>So, what do they mean? First, as the name suggests disables the default k3s cloud controller. We need to do that otherwise k3s will use its own built-in “dummy” cloud controller. Second — we ask k3s to not deploy servicelb because it would mess up IP addresses (servicelb would overwrite Ingress IPs with nodes IPs — we want to have DigitalOcean LoadBalancer IPs for service type LoadBalancer instead). Third — that’s a requirement for pretty much any CCM. We simply need to instruct kubelet that we will be using an external cloud provider. Last one — that’s a requirement specific for DigitalOcean CCM — if you’ll check the DO CCM GitHub <a href="https://github.com/digitalocean/digitalocean-cloud-controller-manager">repo</a> you’ll see that they simply ask you to pass this parameter to kubelet. [master_node_id] is the droplet id which you can find either in DO Dashboard or by making GET call from the droplet itself to:</p><pre>curl <a href="http://169.254.169.254/metadata/v1/id">http://169.254.169.254/metadata/v1/id</a></pre><p>So in order to install k3s server prepared for CCM we need to do it this way:</p><pre>curl -sfL <a href="https://get.k3s.io/">https://get.k3s.io</a> | sh -s — server \<br>  --disable-cloud-controller \<br>  --no-deploy servicelb \<br>  --kubelet-arg=”cloud-provider=external” \<br>  --kubelet-arg=”provider-id=digitalocean://$master_id”</pre><p>That’s the first part — the preparation. Now after k3s server starts we need to install the CCM itself. In order to do that, first clone the git repository:</p><p><a href="https://github.com/digitalocean/digitalocean-cloud-controller-manager">digitalocean/digitalocean-cloud-controller-manager</a></p><p>Next, we need to create k8s secret containing DigitalOcean API token. To do that you can either use a secret generator from the repo (just follow the instructions) or use this one-liner:</p><pre>kubectl -n kube-system create secret generic digitalocean --from-literal=access-token=[YOUR_DO_API_TOKEN]</pre><p>If the secret is saved then simply apply the yaml manifest from the repository:</p><pre>kubectl apply -f releases/v0.1.21.yml</pre><p>That’s it! Now, whenever you spin up service with type: LoadBalancer the DigitalOcean LoadBalancer will be created and configured to route the traffic to it. Also — since k3s comes by default with Traefik — the DO LB will be automatically created for it as well.</p><p>One more thing. So far we created only k3s master node, for any worker nodes you only need to install k3s with the following parameters:</p><pre>curl -sfL https://get.k3s.io | K3S_TOKEN=${token} sh -s - agent \<br>  --server https://${master_node_ip}:6443 \<br>  --kubelet-arg=&quot;cloud-provider=external&quot; \<br>  --kubelet-arg=&quot;provider-id=digitalocean://$worker_id&quot;</pre><p>And that’s it. All the nodes will have proper labels set (public/private ip, DO zone, etc) and routing via DO LB will happen seamlessly too.</p><p>Last but not least — I’ve created a simple bash script to automate the process. The solution allows you to create k3s cluster (1 master node + x workers)on DigitalOcean with DO CCM in ~2 minutes:</p><p><a href="https://github.com/DavidZisky/kloud3s">GitHub - DavidZisky/kloud3s: Deploys VMs + k3s cluster + cloud controllers</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=36bd5020b8f7" width="1" height="1" alt=""><hr><p><a href="https://itnext.io/how-to-integrate-k3s-with-the-cloud-controller-36bd5020b8f7">How to integrate k3s with the cloud controller</a> was originally published in <a href="https://itnext.io/">ITNEXT</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding (and solving) CORS Issues]]></title>
            <link>https://medium.com/@dawid.ziolkowski/understanding-and-solving-cors-issues-d41b416ab13f?source=rss-d3491d6b8e23------2</link>
            <guid isPermaLink="false">https://medium.com/p/d41b416ab13f</guid>
            <category><![CDATA[frontend]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[cors]]></category>
            <category><![CDATA[backend]]></category>
            <dc:creator><![CDATA[David Zisky]]></dc:creator>
            <pubDate>Sat, 18 Jan 2020 21:17:34 GMT</pubDate>
            <atom:updated>2020-01-18T21:17:34.522Z</atom:updated>
            <content:encoded><![CDATA[<p>Imagine you’re building the UI. You need to connect to remote API to get or send some data. Everything works fine when you test your REST calls with curl, but when you implement them in the UI, they don’t.</p><p>First, you check the code, looking for some typos or other mistakes — but everything seems to be fine. You change the URL to google.com or something and find out that the http function is actually working. The issue appears to happen only when calling that specific API. But it works perfectly fine via command line and Postman. What’s going on then?</p><p>Well, it’s probably the mysterious CORS mechanism blocking you. Rather than making an assumption, though, let’s just check the developer console in the browser. So if you hit right click, select Inspect, and go to Console tab, and then see an error like this one …</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7Dzm3pt9y9H5PtaSHiXVOA.png" /></figure><p>… then indeed, it’s CORS. Let’s find out what it really is, and why you can’t fix it in the UI.</p><h3>What is CORS</h3><p>CORS stands for Cross-Origin Resource Sharing. Doesn’t explain much, huh? Well, it’s really simple to understand, but there are a lot of misconceptions about CORS and plenty ‘solutions’ that don’t work.</p><p>The problem with CORS is that many people just google ‘How to fix CORS in Angular/React/whatever?’, copy-and-paste a few lines of code that addresses something about the headers and move forward. While this may sometimes fix your problem momentarily, it may also create a security hole.</p><p>So… What is CORS?</p><p>CORS is a security mechanism built into (all) modern web-browsers (yes! into your web browser! That’s why your curl calls work just fine). It basically blocks all the http requests from your frontend to any API that is not in the same “Origin” (domain, protocol, and port — which is the case most of the time).</p><p>Now, how does this mechanism work? Let’s say you have an ‘upload’ button in the UI that suppose to upload some form of data to the API. So, in HTML code, you bind that button to some JavaScript function, which does a http POST call:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/713/1*77qQFkTw9DrJFrmtcnwzxw.png" /></figure><p>So when you click that button, you would expect the HTTP POST being sent to the API. But instead, your browser, seemingly ‘hidden’ from you, will send a HTTP OPTIONS request first(but not always! will explain that in a second) to the API. The API will typically reply with a bunch of data that says what browser is allowed to do.</p><p>Now, one thing to mention here: HTTP OPTIONS is sent before your actual request, if that request is considered a ‘non-simple’ request. A “non-simple’ request is one that has Content-type other than application/x-www-form-urlencoded, multipart/form-data, or text-plain (for example, JSON) or when requests include cookies. So, pretty much, most of the time.</p><p>And now, we’re talking CORS. Below, you’ll see an example of the headers sent back by the server (yes, by server — therefore CORS is not something you can fix in the UI code) with a reply to OPTIONS. Look at those Access-Control-* headers and focus on Access-Control-Allow-Origin:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/1*ijjIcfyMmFMyqOnehat53Q.png" /></figure><p>Here’s what’s happening: before sending your requested API call, your browser does a ‘security check’ by asking the API, (via an OPTIONS call) who is allowed to do what. Simple as that.</p><p>An ‘issue with CORS’ occurs when the API does not reply to such request with, ‘Yes, dear browser, you are allowed to do that call’. So, as you can see on the screenshot above, my API responded that my UI: “localhost” is allowed to handle OPTIONS, HEAD, DELETE, POST and GET calls.</p><p>Now that’s the core of all the ‘problems’ with CORS. In order to fix CORS, you need to make sure that the API is sending proper headers (Access-Control-Allow-*). That’s why it’s not something you can fix in the UI, and that’s why it only causes an issue in the browser and not via curl: because it’s the browser that checks and eventually blocks the calls.</p><h3>Solutions and Security</h3><p>So, how to fix it properly without creating a security hole? As I mentioned earlier, people who encounter these errors often just google for a solution and copy-and-paste a few lines of code, which instruct the API to send proper headers. The problem with that is that, in most cases, those ‘solutions’ tell You to use ‘<em>Access-Control-Allow-Origin: *</em>’ — in other words, basically allow anyone to access Your API.</p><p>You might ask, What’s the problem with that? I have authentication on my API anyway. (Which should always be the case, right?)</p><p>Well, if you do, then this solution won’t work for you. Because if your call from the browser contains an Authorization header, then the value of Access-Control-Allow-Origin can’t be ‘ * ‘. The browser will block it. If your API requires authorization then the value of Access-Control-Allow-Origin must specify explicitly allowed origins.</p><p>So the key points to know by now:</p><ol><li>CORS is a mechanism built into web browser. It’s not a UI code issue.</li><li>To fix CORS problems, you need to make changes on the API side.</li></ol><p><em>But… but…,</em> you protest, <em>I don’t have access to that API!</em></p><p>Well, in that case, you have two options:</p><ol><li>Ask whoever manages API to fix/add CORS support</li><li>Create middleware</li></ol><p>Option 1 is clear, right? If the API you are trying to access is your company API, then just go to your backend colleagues and ask them to add CORS support. How they do that will depend on the framework they use. Sometimes it’s as easy as installing some package, sometimes they have to add those headers manually to the API code, but nevertheless — they should know what to do.</p><p>If the API is from some third party, then either you can contact them via their support line, or Github, or some other way. Or, you can use Option 2.</p><p>Option 2: build a middleware. Since CORS is as simple as adding some HTTP headers, and it’s only blocked by the browser, then you can build some proxy-like component that will basically make a call for you, get the response from the desired API, add those headers on top, and then send it back to Your UI. Thus, you will no longer connect directly to that API, but to your middleware. It’s not the best solution, but if really necessary, it will solve the issue.</p><p>There is also another use case, You have some tool installed on one of the servers you manage. You can’t directly change the code of that tool, but you still need to add CORS support to it. In such a case, something you can do is install (can be on the same server), for example, Nginx — configure it as a reverse proxy and add headers in Nginx config, and put that tool as a backend.</p><p>That’s it. I hope you not only got a solution to issues with CORS but, most importantly, you learned how it works. And now you have some ideas for how to fix things when you get those once-mystifying CORS messages.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d41b416ab13f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Helm2 vs Helm3 (part 1)]]></title>
            <link>https://itnext.io/helm2-vs-helm3-part-1-c76c29106e99?source=rss-d3491d6b8e23------2</link>
            <guid isPermaLink="false">https://medium.com/p/c76c29106e99</guid>
            <category><![CDATA[helm-chart]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[helm]]></category>
            <dc:creator><![CDATA[David Zisky]]></dc:creator>
            <pubDate>Thu, 21 Nov 2019 13:57:03 GMT</pubDate>
            <atom:updated>2020-01-18T22:20:05.487Z</atom:updated>
            <content:encoded><![CDATA[<h3>Helm2 vs Helm3</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*pF-ql1YNymlesR8X" /><figcaption>Photo by <a href="https://unsplash.com/@syhussaini?utm_source=medium&amp;utm_medium=referral">Syed Hussaini</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h4>So Helm3 finally has been released. Everybody saying goodbye to Tiller but, that’s not the only change in Helm3. Let’s discuss what else has changed.</h4><h3><strong>Adios Tiller</strong></h3><p>Yes, Tiller has been removed and that’s a good thing but to understand why we need to get a bit of background. Tiller was a server-side (running on Kubernetes cluster) component of Helm and its main purpose was to make possible for multiple different operators to work with the same set of releases. When Helm 2 was developed, Kubernetes did not yet have role-based access control (RBAC) therefore to achieve mentioned goal, Helm had to take care of that itself. It had to keep track of who and where is allowed to installs what. This complexity isn’t needed anymore as from Kubernetes 1.6 RBAC is enabled by default, so there is no need for Helm to do the same job which can be done natively by Kubernetes and that’s why in Helm 3 tiller was removed completely.</p><p>Tiller was also used as a central hub for Helm release information and for maintaining the Helm state. In Helm 3 the same information are fetched directly from Kubernetes API Server and Charts are rendered client-side which makes Helm 3 more “native” and simpler for Kubernetes.</p><p>With Tiller gone, also the security model for Helm has been simplified (with RBAC enabled locking down Tiller for use in a production scenario was quite difficult to manage). Helm permissions are now simply evaluated using kubeconfig file. Cluster administrators can, therefore, restrict user permissions at whatever level they want while releases are still recorded in-cluster, and the rest of Helm functionality remains the same.</p><h3>Ok, let’s forget Tiller. What else has changed?</h3><p><strong>As I mentioned at the beginning — removal of Tiller was a big thing, but not the only major change. Let’s take a look at other:</strong></p><ul><li><strong>Three-way strategic merge patch</strong></li></ul><p>Helm 2 used a two-way strategic merge patch. It means that when you wanted to perform any <em>helm </em>operation, it compared the most recent manifest chart against the proposed chart manifest. It checked for the differences between these two charts to determine what changes needed to be applied to the resources in Kubernetes. Sounds pretty smart, right? The problem is that if changes were applied to the cluster “manually” (for example via <em>kubectl edit</em>), they were not considered. This resulted in resources being unable to roll back to its previous state because Helm2 only checked the last applied chart’s manifest as its current state, and since there were no changes in the chart’s state (we only changed live state on the cluster) Helm simply did not see the need for performing a rollback.</p><p>And that’s were three-way strategic merge patch comes to the rescue. How does Helm3 do that? It simply takes the live state into consideration too (thus 3-way instead of 2-way since now we have the old manifest, its live state, and the new manifest). For example, let’s say you deployed an application with:</p><pre>helm install very_important_app ./very_important_app</pre><p>and for example, this application Chart was set to have 3 replica sets. Now, if someone by mistake will perform <em>kubectl edit</em> or:</p><pre>kubectl scale -replicas=0 deployment/very_important_app</pre><p>and then someone from your team will realise that for some mysterious reason very_important_app is down and will try to execute:</p><pre>helm rollback very_important_app</pre><p>In Helm 2, it would generate a patch, comparing the old manifest against the new manifest. Because this is a rollback and someone only changed live state(so the manifest did not change) Helm would determine that there is nothing to rollback because there is no difference between the old manifest and the new manifest (both expect to have 3 replicas). Rollback is not performed then and the replica count continues to stay at zero. You start panicking now…</p><p>In Helm 3 on the other hand, the patch is generated using the old manifest, the live state, and the new manifest. Helm recognizes that the old state was at three, the live state is at zero and so it determines that the new manifest wishes change it back to three, so it generates a patch to fix that. You stop panicking now…</p><p>A similar process happens with Helm 3 when performing upgrades. Since it takes the live state into account now, then for example if some controller-based application (or something like a service-mesh) inject anything into kubernetes object which was deployed via Helm — it will get removed during <em>helm upgrade</em> process using Helm2. Helm 3 doesn’t do that —again, it takes the live state into consideration. Let’s assume that we want to install for example Istio on our cluster. Istio will start injecting sidecar containers into any deployment, so assuming you deployed something with Helm and your deployment consist of:</p><pre>containers:<br>- name: server<br>  image: my_app:2.0.0</pre><p>Then you installed Istio so your container definition looks like this now instead:</p><pre>containers:<br>- name: server<br>  image: my_app:2.0.0<br>- name: istio-sidecar<br>  image: istio-sidecar-proxy:1.0.0</pre><p>And if you now execute upgrade process with Helm2, you’ll end up with this:</p><pre>containers:<br>- name: server<br>  image: my_app:2.1.0</pre><p>Istio sidecar gets removed since it wasn’t in the Chart. Helm 3 however, generates a patch of the containers object between the old manifest, the live state, and the new manifest. It notices that the new manifest changes the image tag to 2.1.0, but live state contains something extra. So upgrade with Helm 3 will work as you would expect:</p><pre>containers:<br>- name: server<br>  image: my_app:2.1.0<br>- name: istio-sidecar<br>  image: istio-sidecar-proxy:1.0.0</pre><p>Three-way strategic merge patch makes Helming way more predictable and safer. You don’t have to worry anymore that something goes bananas.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*dqCruyxXaxe5PZyN" /><figcaption>Photo by <a href="https://unsplash.com/@giorgiotrovato?utm_source=medium&amp;utm_medium=referral">Giorgio Trovato</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p><strong>Secrets as the default storage driver</strong></p><p>Helm 2 used ConfigMaps to store release information. In Helm 3, Secrets are used instead (with a secret type of <em>helm.sh/release</em>) as the default storage driver. This brings a few advantages and has greatly simplified the functionality of Helm. To get (and apply) the config, Helm2 had to perform quite some operations as config itself was stored encrypted and archived in one of the keys or ConfigMap. Helm3 stores the config directly in Secret so instead of performing multiple operations to get it, it can simply pull out the secret, decrypt and apply. Another advantage is that release name does not have to be unique across the cluster anymore. Secrets containing releases are stored in the namespaces where the release was installed. So you can have multiple releases with the same name as soon as they are in different namespaces.</p><p><strong>JSON Schema Chart Validation</strong></p><p>A JSON Schema validation can now be forced upon chart values. With that functionality, you can ensure that values provided by the user follow the schema created by the chart maintainer. This creates more possibilities for OPS vs DEV cooperation (OPS team can give more freedom to DEVs) and provides better error reporting when the user tries to use an incorrect set of values for a chart.</p><p><strong>Release name is now required</strong></p><p>In Helm 2, if no name was provided, a random name would be generated. Helm 3 will throw an error if no name is provided with helm install (eventually you can use <em>— generate-name</em> flag if you want to still use random names)</p><p><strong><em>Helm serve</em> removed</strong></p><p>Not many people used<em> helm serve (</em>which was used to<em> </em>run a local Chart Repository on your machine for development purposes<em>) </em>but for those who did — it’s removed now. Although you can still use it as a plugin instead.</p><p><strong>Namespaces are not created automatically anymore</strong></p><p>When creating a release in a namespace that does not exist, Helm 2 would create it automatically. Helm 3 follows the behaviour of other Kubernetes tooling and returns an error if the namespace does not exist.</p><p>That’s the most important changes in Helm, there are still a few more. If you are interested check official Helm <a href="https://helm.sh/docs/faq/#changes-since-helm-2">docs</a>. In Part 2 I will show you how to migrate from Helm2 to Helm3.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c76c29106e99" width="1" height="1" alt=""><hr><p><a href="https://itnext.io/helm2-vs-helm3-part-1-c76c29106e99">Helm2 vs Helm3 (part 1)</a> was originally published in <a href="https://itnext.io/">ITNEXT</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to create a Kubernetes cluster in under 60 seconds]]></title>
            <link>https://medium.com/@dawid.ziolkowski/how-to-create-a-kubernetes-cluster-in-under-60-seconds-7320a58f5b0?source=rss-d3491d6b8e23------2</link>
            <guid isPermaLink="false">https://medium.com/p/7320a58f5b0</guid>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[David Zisky]]></dc:creator>
            <pubDate>Mon, 05 Aug 2019 11:33:59 GMT</pubDate>
            <atom:updated>2019-08-05T16:37:23.033Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*STRM6NU396sZo6WY" /><figcaption>Photo by <a href="https://unsplash.com/@chuttersnap?utm_source=medium&amp;utm_medium=referral">chuttersnap</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>In the past few weeks, I have been doing a lot of different tests and deployments on kubernetes. Therefore, I had to create and destroy Kuberentes clusters like a lot (some days — a few times in an hour). I really needed a fresh cluster due to specific things I was testing, so simply deleting all pods/services/secrets/etc to make the cluster “fresh like” wouldn’t help. Another requirement (again due to some of the specific tools I was testing) was to have the most “real like cluster” as possible so all local solutions (Minikube, vagrant, etc) wouldn’t work either. At the beginning, I used DigitalOcean Kubernetes because it’s super simple to deploy and as soon the cluster is up I could just download kubectl config with a click of a button. But there were two problems with it:</p><ol><li>It takes sooooo much time… around 10 minutes to deploy a cluster — if I had to deploy&amp;destroy it few times per day — it becomes a lot of wasted time</li><li>Downloading and loading kubectl config — while it’s simple — it’s still a manual process</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*zkQnkZcKSUM_QmSp" /><figcaption>Photo by <a href="https://unsplash.com/@aronvisuals?utm_source=medium&amp;utm_medium=referral">Aron Visuals</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>So I decided to create something which would allow me to deploy&amp;destroy kubernetes cluster in the cloud faster and easier… and I end up with a simple bash script which creates virtual machines on Google Cloud, deploys 4-node Kuberentes cluster (1 master + 3 workers), downloads kubectl config and loads it into my system… and it does all of it in under 60 seconds! So from nothing (not even having VMs) to being able to do kubectl apply -f any_deployment.yaml in under 1 minute. Let’s talk about how was it possible.</p><p><strong>Requirements:</strong></p><p>One of the important factors for me was to make the solution as portable as possible. So I tried to make as fewer tools required as possible (so no terraform, no ansible, no whatever which require installation and configuration). That’s why I wrote it in bash and the only dependency I kept is to have gcloud installed and configured (preferably set default region and project).</p><p><strong>VMs:</strong></p><p>We start with VMs. Creating VMs normally takes around 45–60 seconds. For example on DigitalOcean VMs are up (meaning ping starts responding) after around 40 seconds, but You need another ~15 seconds for all the systems services to be up (and most importantly — ssh server being able to accept connections). So first we need to make that step faster… like 2 times faster :) Well… we can achieve that by using a slimmer OS image. That’s why currently I stick to Google Cloud because they provide Ubuntu minimal (which is less than 200MB in size). In the meantime, I tried many other minimalistic distributions but some of them were missing necessary kernel modules, some even tho were slim — they boot up quite long. Creating and booting up Ubuntu minimal VMs on Google Cloud takes around 30 seconds (from the gcloud API call till ssh server ready) so we got our first step. Now we have another 30 seconds for…</p><p><strong>Kubernetes:</strong></p><p>How to deploy kubernetes cluster in around 30 seconds… there is one answer — k3s. For those who haven’t heard about k3s — similar to Minimal OS images it’s simply slim k8s solution:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/620/1*7M6FaOpFrGU-xH5DvRbxQg.png" /><figcaption>k3s.io</figcaption></figure><p>By using k3s we don’t actually have to do much to get kubernetes up and running because k3s does everything for you. So my script only downloads k3s installer and executes it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*1n-raVlWxkEwxH5l" /><figcaption>Photo by <a href="https://unsplash.com/@barkiple?utm_source=medium&amp;utm_medium=referral">John Barkiple</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p><strong>Connect the pieces:</strong></p><p>So we found a way to boot up VMs in less than 30 seconds by using slim OS images, we have k3s.io which allows us to run kubernetes in around 20 seconds and now we just need to connect all the pieces together, so — prepare gcloud commands (to deploy VMs) in the script, download and execute k3s installer on master node (simply curl piped to sh), then, get the token (cat command over ssh) generated by k3s which is used for adding nodes to the cluster and download and execute (with token) k3s on worker nodes. The only real challenge was to get kubectl config generated properly — public IP address on google VMs is not visible on the VMs so when k3s generates certificates and then config — it’s not valid for accessing the cluster from the outside. But after a lot of searching, I found the parameter “ — tls-san=” which allows providing additional IP addresses for certificate generation. Then, if k3s is deployed on all nodes, workers are registered properly with the master — our cluster is ready. The only thing left is to simply download the kubectl config (using scp from master node). Everything takes 55–58 seconds usually (Google Cloud isn’t really consistent — sometimes it takes a bit more than 60 seconds, but it doesn’t happen often). So as you can see there is nothing extraordinary about the solution — just a few gcloud and curl commands glued together in one bash script. But it does the job and does it quickly.</p><p><a href="https://github.com/DavidZisky/60sk3s">DavidZisky/60sk3s</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zs2WTUJixaPnt2oiy2K04Q.png" /></figure><p><strong>Room for improvement?</strong></p><p>Definitely! First of all — currently it’s hardcoded to have 4 node cluster (1 master + 3 workers) — to make it configurable it’s rather simple — I just never needed bigger cluster for the tests I did. Secondly — kubectl config is now either being only downloaded (so you can pass it yourself to kubectl command as a parameter) or it overwrites your existing kubectl config (which works just fine for myself — since I don’t have any long-running clusters). But definitely, it would be beneficial to add an option to append the config to your existing config (and then change context). If there is anything else you can think of feel free to let me know in the comments or open GitHub issue.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7320a58f5b0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to build an API without coding]]></title>
            <link>https://itnext.io/how-to-build-an-api-without-coding-350293c58376?source=rss-d3491d6b8e23------2</link>
            <guid isPermaLink="false">https://medium.com/p/350293c58376</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[cloud-native]]></category>
            <category><![CDATA[api-development]]></category>
            <dc:creator><![CDATA[David Zisky]]></dc:creator>
            <pubDate>Sat, 06 Jul 2019 21:07:27 GMT</pubDate>
            <atom:updated>2020-01-30T10:10:06.589Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*IwkHGPD0lOfgiSnf" /><figcaption>Photo by <a href="https://unsplash.com/@casparrubin?utm_source=medium&amp;utm_medium=referral">Caspar Camille Rubin</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>I’ve spent the last 10 years of my life as a System Engineer, which means when I first had to build an API, I had no idea how to do it. I knew python quite a bit since as a SysAdmin I often had to write some smaller or bigger scripts, but building the whole API was a challenge for me. Sure — I could just google the phrase and find dozens of tutorials on how to build an API, but all of them were somehow complicated or created for raw developers (which means — with assumptions that You know much programming already). Fortunately, at work, I got assigned a project for a company which had an API prototype written with Python Eve framework. I’ll explain Python Eve in a second, but it’s not a story about Eve per se. It’s a story about how I made building an API with Python Eve even easier. Shall we?</p><p>As I mentioned above — the story begins with Python Eve and as also mentioned earlier — we’re not focused on Eve today, but I need to give you a bit of a context about it. Eve is a REST-centered API framework built on top of Flask created by <a href="http://nicolaiarocci.com/">Nicola Iarocci</a> . What’s great about it is that it’s effortless to build complete API because all You need to kick-start is a running MongoDB server and two simple python files (app.py which in most basic setup contains 4 lines of code and settings.py which define the endpoints, database connection and extra configuration if you wish). Simplicity is the first selling point of Eve, but make no mistake — it does not limit you at all. It is very feature-rich (it gives most of the things you would expect from an API — a full range of CRUD operations, filtering, sorting, pagination, conditional requests, data validation and so on…). Plus since it’s built on top of Flask — most Flask add-ons should work with Eve too.</p><p>So, let me quickly show You “<em>how to build an API in 5 minutes</em>” and after that, I’ll show you faster method ;)</p><p>To start with Eve you can simply install it with pip:</p><pre><strong>pip install eve</strong></pre><p><strong>NOTE: please use Python 3</strong></p><p>If we have Eve installed, we need to spin up a MongoDB database. It all depends on You platform and Your setup but assuming You have docker installed You can, for example, use official mongo image:</p><pre><strong>docker run --name mongo -d mongo</strong></pre><p>Now, all we need is two small files. One being let’s say app.py (or run.py, or however you prefer to call it) with the following content:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/dbd0786fbb4f5a995e19e93fcf37e550/href">https://medium.com/media/dbd0786fbb4f5a995e19e93fcf37e550/href</a></iframe><p>So we simply imported Eve, created new Eve object and called run function. Now, that’s enough to spin up Eve instance, but we need to define all the endpoints, connection to Mongo and so on. For that, we need a file called settings.py (by default Eve expect that file to be called like this, you can change it if you want, but then you need to point to it in you app.py — for simplicity, we keep to defaults). The content of this file can be customized depending on how many features you want to use, but it must contain the following:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3571a41d4df0e114d1293699d608206b/href">https://medium.com/media/3571a41d4df0e114d1293699d608206b/href</a></iframe><ol><li>MongoDB definition — in the example above I used _URI version but it’s also possible to define MONGO_HOST, MONGO_PORT, MONGO_DBNAME and more options separately</li><li>RESOURCE/ITEMS_METHODS — as the name implies defines resource REST methods (so what is allowed to do when you send a request to <em>http://your_api/example</em> and what’s allowed for particular items like <a href="http://your_api/example/[uuid]"><em>http://your_api/example/[uuid]</em></a></li><li>Schema definition — that’s the core and most time-consuming part of building settings.py file for Eve. Here you need to define a schema for each endpoint. In this example, we have only one schema with only two entries firstname and lastname which are strings. Now, I purposely made it that unrealistically easy because that’s the part we will automate in next step but keep in mind that in real life scenario You would probably have few endpoints with a lot on schema inside each and every single one.</li><li>Endpoints definition — in step 3 we created a schema and in DOMAIN section we have to define mapping endpoint &lt;&gt; schema. In our example, we simply create people endpoint which maps to people variable (which we just defined on line 9)</li></ol><p>That’s everything you need to build fully functional (apart from the fact it’s almost useless but to make it useful we would just have to create more endpoints and schema which is simply defining data names and types (step 3) and map them to endpoints. And here comes the point of this story — let me introduce you EveGenie!</p><p><a href="https://github.com/DavidZisky/evegenie">DavidZisky/evegenie</a></p><p>EveGenie can generate whole settings.py file for You by providing JSON file with the data you expect to get from the API. What it means is that if you have JSON file generated by some UI, some tool, your colleague or anything else you can execute evegenie — it will read it and create schema and endpoint definitions for you (+ MongoDB connection line and RESOURCE/ITEM_METHODS — so the whole file).</p><p>How to use it then? Just like this:</p><pre>python geneve.py sample.json</pre><p><em>NOTE: evegenie is only compatible with Python 3.x</em></p><p>So, forget about building API in 5 minutes… You can do it in 5 seconds — in the GitHub repo you will also find simple run.py file so the only two things (assuming You have eve installed, and evegenie repo cloned) you need to do to create fully functional and running API is:</p><ol><li><strong><em>python geneve.py your_data.json</em></strong></li><li><strong><em>python run.py</em></strong></li></ol><p>Keep in mind that this will create a schema and run the API but it won’t have data inside (the json file you provide for evegenie is only used to create schema — not to load data). But since you have API running you can simply send a POST request with the very same JSON file.</p><p>Having the ability to create API this way is not a solution for developers — while Eve still has a lot of features and you can definitely build very rich production grade API with it — I showed you the most basic usage of Eve because I use it as a System Engineer for quick microservices/networking testing. I don’t use this solution to create an API on its own— I use this to create an API which will help me debug issues with Kubernetes networking, tests of Istio, etc. But the point is — if you always wanted to try something which required running API and you couldn’t do it because you are not a developer, or perhaps you are even frontend developer and want to start moving towards full-stack role — hopefully that was an easy solution for you to achieve it.</p><p>Best regards!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=350293c58376" width="1" height="1" alt=""><hr><p><a href="https://itnext.io/how-to-build-an-api-without-coding-350293c58376">How to build an API without coding</a> was originally published in <a href="https://itnext.io/">ITNEXT</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>