<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Umur Coşkuncan on Medium]]></title>
        <description><![CDATA[Stories by Umur Coşkuncan on Medium]]></description>
        <link>https://medium.com/@mrcskncn?source=rss-e2591123be9f------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*fA6r8wJBqWd4kPqhr0z_Uw.jpeg</url>
            <title>Stories by Umur Coşkuncan on Medium</title>
            <link>https://medium.com/@mrcskncn?source=rss-e2591123be9f------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 05:44:06 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@mrcskncn/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Installing, Configuring and Day 0 Operations on Rancher Server — Series III]]></title>
            <link>https://mrcskncn.medium.com/installing-configuring-and-day-0-operations-on-rancher-server-series-iii-360f75306fc?source=rss-e2591123be9f------2</link>
            <guid isPermaLink="false">https://medium.com/p/360f75306fc</guid>
            <category><![CDATA[rancher]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[clustering]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[sre]]></category>
            <dc:creator><![CDATA[Umur Coşkuncan]]></dc:creator>
            <pubDate>Sun, 13 Jun 2021 23:32:23 GMT</pubDate>
            <atom:updated>2021-06-24T20:22:34.973Z</atom:updated>
            <content:encoded><![CDATA[<h3>Installing, Configuring and Day 0 Operations on Rancher Server — Series III</h3><p><em>We have talked about what Rancher Server is, how it works on the backstage and why it matters for the </em><a href="https://kubernetes.io/"><em>Kubernetes</em></a><em> (K8s) provisioning world at </em><a href="https://mrcskncn.medium.com/what-rancher-server-does-and-how-it-works-series-ii-7d3e72de7359"><em>Series II</em></a><em>. This time we’ll be installing, configuring and making several tunes on it. Difficulty to follow this guide is really easy.</em></p><h3>Preparation</h3><p>In order to run <a href="https://rancher.com/products/rancher/">Rancher Server</a>, we’ll need a <a href="https://en.wikipedia.org/wiki/Linux">Linux</a> VM, preferably with a <a href="https://www.debian.org/">Debian</a> distro. You can obtain the latest Debian Server ISO, then make a regular Linux installation using <a href="https://www.vmware.com/products/vcenter-server.html">vCenter</a>.</p><p><strong>Before the Linux VM provisioning phase; Remember that I’ve mentioned about the Network subnets in </strong><a href="https://mrcskncn.medium.com/kubernetes-on-premises-and-gdpr-does-it-ring-a-bell-74786040a76c"><strong>Series I</strong></a><strong>. The Linux VM that’s going to run Rancher Server must be deployed into Kubernetes VM network, which is: 192.168.100.0/24. It’s up to you to decide but we should at least leave 50 IP addresses for DHCP server to assign on orphan VM’s. So, the DHCP pool could be configured in between: 192.168.100.150 to 192.168.100.200, or any other convention you like.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VdTpwh2PsJ4Mx9AV2iMfmA.png" /><figcaption>Rancher Server specs for our VM</figcaption></figure><p>I’m not going to teach you how to create a Linux VM step by step like you are a toddler. Pull your shit together or follow the guide below to make a simple Debian Server (No GUI) installation with SSH enabled and Networking ready.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FKa3ppfEsyp4%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DKa3ppfEsyp4&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FKa3ppfEsyp4%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/316da23d4182528b99c042cb742a4b26/href">https://medium.com/media/316da23d4182528b99c042cb742a4b26/href</a></iframe><blockquote>There are several discussions about whether you should be running Rancher Server as <a href="https://en.wikipedia.org/wiki/High_availability">High-Available</a> (HA) mode or Standalone. Rancher suggests that you should make all production deployments HA, however I have a different solution to save our resources while enabling <a href="https://en.wikipedia.org/wiki/Cloud_computing#Hybrid_cloud">Hybrid Cloud</a>. Still, no possible downtime for our clusters.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cbjnU7OjIl31bH0A-jNHDQ.png" /><figcaption>You are ready for the next step when you can SSH into the Linux VM you have just provisioned.</figcaption></figure><blockquote>Debian doesn’t include <a href="https://en.wikipedia.org/wiki/Sudo"><strong>sudo</strong></a> by default so, it would be nice if you can install it by switching to root user first.</blockquote><pre>su - root</pre><blockquote>Then, make sure you know your regular Linux user, <em>mine was “cloud” as on the screenshot above — replace “cloud” with your Linux user wherever you see it</em>, thus we can add this user to sudo group.</blockquote><pre>apt install sudo -y<br>usermod -aG sudo cloud<br>reboot</pre><blockquote>We’ll throw some magic in our terminal to update outdated packages if any:</blockquote><pre>sudo apt update<br>sudo apt upgrade -y</pre><blockquote>Make sure we have the necessary drivers installed on our VM.</blockquote><pre>sudo apt install open-vm-tools -y<br>sudo reboot</pre><blockquote>Do you remember this command from <a href="https://mrcskncn.medium.com/kubernetes-on-premises-and-gdpr-does-it-ring-a-bell-74786040a76c">Series I</a>, TL;DR section? It’s time to throw that beauty in, LoL.</blockquote><pre>sudo su -c &quot;curl -sSL https://get.docker.com/ | sh&quot;</pre><p><strong>Now we are ready for some action.</strong></p><h3>Installation</h3><p>We have a great option here, it’s simple, it’s elegant… What more do you want? We can deploy Rancher Server with a containerized way, without having to deal with lots of Linux commands.</p><h4>Let’s start:</h4><p>Get back to your shell and;</p><pre>sudo mkdir -p /opt/rancher<br>sudo chown -R cloud:cloud /opt/rancher</pre><p>—</p><pre>cat &lt;&lt; &#39;EOF&#39; &gt; /opt/rancher/run.sh<br>#!/bin/bash</pre><pre>docker stop rancher-server<br>docker rm -f rancher-server</pre><pre>docker run \<br>--name=rancher-server \<br>--restart=unless-stopped \<br>-p 80:80 \<br>-p 443:443 \<br>-v rancher-data:/var/lib/rancher \<br>--privileged \<br>-d rancher/rancher:stable<br>EOF</pre><p>—</p><pre>chmod +x /opt/rancher/run.sh<br>/opt/rancher/run.sh</pre><p>This way, Rancher Server starts running inside a docker container made by their own. Basically inside that docker container, Rancher runs a <a href="https://k3s.io/">K3s</a> cluster for it’s services and their orchestration. <em>In </em><a href="https://mrcskncn.medium.com/what-rancher-server-does-and-how-it-works-series-ii-7d3e72de7359"><em>Series II,</em></a><em> we didn’t mention about this because it’s out of our control (unless you want to contribute to </em><a href="https://github.com/rancher/rancher"><em>Rancher Server</em></a><em>). What we needed was, understanding how Rancher communicates with vCenter or other peripherals to provision RKE.</em></p><blockquote>Rancher persists it’s data inside the container at /var/lib/rancher path. In order for us to make sure things work if by accident or for an upgrade purpose remove/recreate the rancher-server container, we need the “-v” directive.</blockquote><p>Now let’s try reaching to <a href="https://your-static-vm-ip/">https://your-static-vm-ip/</a></p><p>If it doesn’t work as intended (<em>there is no other reason to unless you are trying to be hero, but…</em>), you can check Rancher Server logs by typing this below and look for errors:</p><pre>docker logs -f rancher-server</pre><p>In the end we should be landing on a page like this;</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-uKenm0-h7KpN5HkA118Ww.png" /><figcaption>Initial landing on Rancher Server container</figcaption></figure><blockquote>Set a password, don’t touch the rest and “Continue”.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nY8_0shGmoz7jkgKQ5MNVw.png" /><figcaption>Clusters page on Rancher Server</figcaption></figure><blockquote>As you can see, there is a cluster named “local” deployed as K3s just for Rancher itself. You can use the Explorer button on the right hand side and sneak peek the workload, namespaces and other stuff.</blockquote><blockquote>You can even use this Rancher local K3s cluster to try out some features in your local machine, just like <a href="https://minikube.sigs.k8s.io/docs/start/">minikube</a>.</blockquote><p>That sums up the installation. Pretty neat, isn’t it?</p><h3>Configuration</h3><p>Actually, when you look into it, there aren’t much to configure when you deploy Rancher Server with the containerized way. However, at this point we’ll configure our Cloud Credentials (vSphere) and Node Templates.</p><h4>Cloud Credentials</h4><p>After logging-in when you land on the Rancher Server dashboard, at the top right corner you can see the user profile we’re on. Click on the drop-down and get to Cloud Credentials item.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IligA58nC5whc34GdoarGQ.png" /><figcaption>Cloud Credentials item on the User Drop-down Menu</figcaption></figure><blockquote>Then, follow the “Add Cloud Credential” button on top again.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cBZsZpRMum__14pYqiWXKQ.png" /><figcaption>Add Cloud Credential from the Cloud Credentials page on the Rancher Server</figcaption></figure><blockquote>Fill in the information according to your vCenter details.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZfPyFhY-DM36flPdIFuEAQ.png" /><figcaption>Add Cloud Credential input boxes</figcaption></figure><blockquote>Hit “Create” to save your cloud credentials into Rancher K3s.<br>Now Rancher can communicate with vCenter API.</blockquote><h4>Node Templates</h4><p>Now let’s hit the Node Templates item from the user drop-down menu at the top right center.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/942/1*q0oHxFEdlEHUTc2lJthVOA.png" /><figcaption>Node Templates item on the User Drop-down Menu</figcaption></figure><blockquote>Click on the “Add Template” button.</blockquote><blockquote>When you are asked, choose the “vSphere” option. We are making this Node Template for vSphere, using the vSphere credentials we have provided before.</blockquote><p>There are 6 steps to configure here and I’ll briefly mention about them how they are supposed to be.</p><ol><li><strong>Account Access:</strong> We will choose which vCenter Cloud Credential to use.</li><li><strong>Scheduling: </strong>Where the VMs made by this template will be created. Like, the vCenter Datacenter, the Datastore, Folder or Host.</li><li><strong>Instance Options: </strong>If this is a template for Master Nodes, then at least 2 Cores and 4 GBs of Memory is required for a health Control Plane. I also want 20 GBs of Disk Space for my Master Nodes to have. As for the Worker Nodes, I’ve set 8 Cores and 16 GBs of Memory with 60 GBs of Disk Space. That’s how I prefer it, you can use lower or higher specs.<br><strong>Don’t ask “Why?”:<br>Make sure: Your “Creation method” is set to “Install from boot2docker ISO (Legacy)” with the value (ISO Link) of:<br></strong><a href="https://releases.rancher.com/os/v1.5.8/vmware/rancheros-autoformat.iso">https://releases.rancher.com/os/v1.5.8/vmware/rancheros-autoformat.iso</a><br><strong>Make sure: You have set your network for the Kubernetes VM network.<br>Make sure: You have “</strong>disk.enableUUID = True” on all your vSphere templates.</li><li><strong>Tags:</strong> You can add tags here. I have no tags.</li><li><strong>Custom attributes (legacy): </strong>You got the gist from the name. I have no custom attributes.</li><li><a href="https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.vm_admin.doc/GUID-2A95EBB8-1779-40FA-B4FB-4D0845750879.html"><strong>vApp</strong></a><strong> Options: </strong>I set as “Do not use vApp”.</li></ol><p>Name your template, don’t touch the Engine Options, Taints or Labels and hit “Create”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*skZ0tV82nnX9KpvKh1Dyug.png" /><figcaption>Node Templates section after we create our templates</figcaption></figure><blockquote>Repeat as many Node Templates as you need.</blockquote><h3>Day 0 Operations</h3><p>So far we have done several things. Filled our Cloud Credentials, created Node Templates we wanted. What we want now is to set a backup retention for our Rancher K3s cluster. At least, that’s what I wanted.</p><p>I’ll show you how to use Rancher Apps &amp; Marketplace below. You can set a backup retention from this section in your K3s cluster. You can also make other Day 0 Operations if you want. Like setting authentication mechanism, ex. <a href="https://en.wikipedia.org/wiki/Active_Directory">Active Directory</a> or <a href="https://en.wikipedia.org/wiki/Lightweight_Directory_Access_Protocol">LDAP</a>.</p><p>In our Rancher Dashboard, hit the “Explorer” button of local (K3s) cluster.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XFuymT-u1G4y839DOljEnw.png" /><figcaption>Rancher Cluster Explorer Menu</figcaption></figure><blockquote>From the top left corner, where the drop-down for Cluster Explorer is, choose “Apps &amp; Marketplace” item.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t8zyMqCCzN1IHb6B36BNNA.png" /><figcaption>Apps &amp; Marketplace menu at Rancher Local K3s Cluster</figcaption></figure><p>At this point, there are lots of <a href="https://helm.sh/">Helm</a> charts we can use via Rancher’s recommendation. I’ll go with “<strong>Rancher Backups</strong>” in my case.</p><p>Don’t touch any options and hit “Install” button at the very bottom right section of the page. You should now see in the terminal that pops up the Helm package has successfully deployed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/596/1*8DBFref7pIrgHP9sD8bVAA.png" /><figcaption>Rancher Cluster Explorer Menu</figcaption></figure><p>After the successful “Rancher Backups” deployment, now you should notice a new item in the Cluster Explorer menu. Before we hit that, we need to create an <a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users.html">AWS IAM user</a> with appropriate <a href="https://aws.amazon.com/s3/">AWS S3</a> permissions. Then, obtain <a href="https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys">programmatic access keys</a> and insert this information to Rancher Server Local K3s Cluster.</p><p><strong>Don’t forget to create an S3 bucket from your AWS Console in the region you desire. The access key you made must have the rights to PutObject into this S3 Bucket.</strong></p><p>Assuming you have this information created, we will encode this information with base64 so that we can create an Opaque secret for Kubernetes.</p><p>Pop your terminal and type these using your credentials:</p><pre>echo -n &#39;AKIAAAAABBBBBCCCCCDD&#39; | base64<br>echo -n &#39;Bv3ekOp7f9Warh9b32nO/A2lgmsMKeG+76zAoNme&#39; | base64</pre><blockquote>Using the outputs from above, click on “<strong>Import YAML</strong>” button on the top-right corner on Rancher Server Cluster Explorer.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/642/1*8TO4H1NTs-kTe7wHM9KGQw.png" /><figcaption>Import YAML button on top-right corner of Rancher Server Cluster Explorer</figcaption></figure><blockquote>You can now feed the information below, leave the namespace as “default” and hit “<strong>Import</strong>”:</blockquote><pre>apiVersion: v1<br>kind: Secret<br>metadata:<br>  name: limon-s3-creds<br>type: Opaque<br>data:<br>  accessKey: QUtJQUFBQUFCQkJCQkNDQ0NDREQ=<br>  secretKey: QnYzZWtPcDdmOVdhcmg5YjMybk8vQTJsZ21zTUtlRys3NnpBb05tZQ==</pre><blockquote>Let’s get back to Rancher Backups from the Cluster Explorer menu and tap “Create” button to make a new backup job.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-kWza7xyIDq44WZCIIMNoQ.png" /><figcaption>Rancher Backups Menu Items</figcaption></figure><blockquote>Name a rule, put up a schedule (I want to have my backups hourly with a 72 hours of retention), set your storage location with the desired region information like below, and hit “<strong>Create</strong>”</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_CKo1N3vu-8QBYymN6RAgg.png" /><figcaption>Rancher Backup Job — Scheduling Page</figcaption></figure><blockquote>With this created, you should now be able to see backup files in your AWS S3 bucket and also within the Rancher Backups dashboard.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ygkhqQR4LknBOEXSxRZQkg.png" /><figcaption>Rancher Backup Job Completed Successfully</figcaption></figure><p>And that’s how it’s done. You can deploy different Helm charts from the Apps &amp; Marketplaces section at your Rancher Server, or use existing <strong>Security</strong> or <strong>Tools </strong>drop-down menus at the <strong>Cluster Manager </strong>view to maintain your Day 0 Operations.</p><p>Up next: We will be deploying a simple, production-ready, Rancher Kubernetes Engine (RKE) using Rancher Server we’ve prepared today.</p><p>Thank you for reading and always, stay tuned!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=360f75306fc" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What Rancher Server does and how it works? — Series II]]></title>
            <link>https://mrcskncn.medium.com/what-rancher-server-does-and-how-it-works-series-ii-7d3e72de7359?source=rss-e2591123be9f------2</link>
            <guid isPermaLink="false">https://medium.com/p/7d3e72de7359</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[on-premise]]></category>
            <category><![CDATA[clustering]]></category>
            <category><![CDATA[rancher]]></category>
            <dc:creator><![CDATA[Umur Coşkuncan]]></dc:creator>
            <pubDate>Sun, 06 Jun 2021 15:30:13 GMT</pubDate>
            <atom:updated>2021-06-13T23:34:07.875Z</atom:updated>
            <content:encoded><![CDATA[<h3>What Rancher Server does and how it works? — Series II</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*klZWcQfkQ1zy5Fw_3oyBow.png" /><figcaption>Rancher Kubernetes Engine (RKE) with K8s v1.20.6 — Dashboard</figcaption></figure><h3>Why?</h3><p><em>I have told you that I’ll be sharing a new story every Sunday and I kept my promise. If you haven’t read the previous post and want to know how we did choose Rancher Server in our scenario, please </em><a href="https://mrcskncn.medium.com/kubernetes-on-premises-and-gdpr-does-it-ring-a-bell-74786040a76c"><em>read this</em></a><em> first and understand the reasons completely.</em></p><p><em>In this Series II, we’ll talk about what Rancher Server does for us, how it handles the management of the underlying infrastructure by automated provisioning and what we gain from it.</em></p><p><a href="https://rancher.com/docs/rancher/v2.5/en/overview/">Rancher Server</a> is a very lightweight and powerful tool to deploy and orchestrate multiple <a href="https://kubernetes.io/">Kubernetes</a> (K8s) clusters, both on-premises and the cloud.</p><p>The overall Rancher Server architecture can be described as below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_4aNT4jOLOWIMnORwkba6A.png" /><figcaption>Managing Kubernetes Clusters through Rancher’s Authentication Proxy</figcaption></figure><p>You should <strong>never</strong> mix Rancher Server and <a href="https://rancher.com/docs/rke/latest/en/">Rancher Kubernetes Engine</a> (RKE) concepts. They are <strong>completely</strong> different things. You can keep using Rancher Server to manage your Kubernetes clusters that provisioned using Rancher Server, without the obligation of using Rancher’s own Kubernetes Platform.</p><h3>Do you know what Docker-Machine is?</h3><p>I wonder if any of the readers have used <a href="https://github.com/docker/machine">Docker-Machine</a> before? It’s a project <a href="https://www.docker.com/">Docker</a> made around 5 years ago, and just like <a href="https://github.com/docker/classicswarm">Docker Swarm</a>, it has f#cked and not maintained anymore.</p><p>In basic terms, Docker-Machine was a way to create/provision Virtual Machine’s (VM) on several platforms with a pre-installed and configured <a href="https://github.com/docker/engine">Docker Engine</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*H2WOg1s6h0ZbzO16JMHbfg.png" /><figcaption>Docker-Machine VM creation with a <a href="https://en.wikipedia.org/wiki/One-liner_program">one-liner</a> for VirtualBox</figcaption></figure><p>It was so simple, yet powerful.</p><h3>How about Rancher-Machine?</h3><p>Well, after Docker-Machine lost it’s maintainability even with all there is to provide, Rancher decided to fork (basically adopted, rather than letting it die) the project 2–3 years ago and continued improving it under a very similar, the <a href="https://github.com/rancher/machine">Rancher-Machine </a>name.</p><p>Additional features included in the Rancher-Machine, is that the upstream support for destination platforms keeps maintained and the Rancher-Machine is embedded inside the Rancher Server.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yGvtkEk4Of2eA4pqmUbnAA.png" /><figcaption>You can see Rancher-Machine running inside Rancher Server</figcaption></figure><h3>vSphere Cluster Lifecycle Management</h3><p>As you might guess from the <a href="https://mrcskncn.medium.com/kubernetes-on-premises-and-gdpr-does-it-ring-a-bell-74786040a76c">Series I</a>, one great, simple solution was needed for VMware vSphere in our case. Both because VMware is a great <a href="https://en.wikipedia.org/wiki/Hypervisor">hypervisor</a> and Rancher is a powerful on-premise cluster provisioner.</p><h4>Node Templates</h4><p>Rancher Server provides us a template option called <a href="https://rancher.com/docs/rancher/v2.5/en/user-settings/node-templates/">Node Templates</a>. In these templates, we provide specific details about how our VMs should be provisioned. This includes how much CPU or Memory we’re going to need or which version of Docker is required to install.</p><p>In my consideration, you should create different Node Templates depending on the group of <a href="https://kubernetes.io/docs/concepts/architecture/nodes/">nodes</a> you think you’ll be provisioning.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aOjWU7Sd3PN4Co11KdAqbQ.png" /><figcaption>Node Templates in Rancher Server</figcaption></figure><blockquote>I decided to create two (2) different templates for my use case, as one for Master nodes and the other one for Worker nodes. Of course, I can expand it to three (3) or four (4) depending on the extension of my cluster.</blockquote><blockquote>To elaborate more, at this point I’m depending on the default vSphere Cloud Provider to provision my <a href="https://kubernetes.io/docs/concepts/storage/persistent-volumes/">Persistent Volume Claims</a> (PVC) natively. But in this way it blocks me to provision an <a href="https://en.wikipedia.org/wiki/Network_File_System">NFS</a> Persistent Volume (PV) as it’s not supported in the driver I use yet. So, I can’t provision any <a href="https://kubernetes.io/docs/concepts/storage/persistent-volumes/#access-modes">ReadWriteMany</a> PVs natively. If I need such feature in the near future, I would have to deploy <a href="https://ceph.io/">Ceph</a> or <a href="https://www.gluster.org/">GlusterFS</a> as a secondary <a href="https://kubernetes.io/docs/concepts/storage/storage-classes/">StorageClass</a>.</blockquote><blockquote>To do that, I would prefer making a separate Node Template called StorageTemplate and deploy any storage related management resources there. Which helps me organize stuff in the end.</blockquote><blockquote>You got the gist I think…</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*v1w2tvTTAMEeuzHkkE8JdA.png" /><figcaption>Rancher-Machine is fed by the Node Templates to provision our VMs</figcaption></figure><p>As much as Rancher-Machine simplifies the provisioning process, Rancher Server still needs some information/credential regarding the provisioned VMs, so that it is able to use this information to manage the VMs or continue the cluster bootstrapping process by installing K8s.</p><p>To do so, Rancher bootstraps an <a href="https://en.wikipedia.org/wiki/Ssh-keygen">SSH Key</a> into these VMs using <a href="https://cloud-init.io/">cloud-init</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zHF2eSeOANdTAyNm_mr30w.png" /><figcaption>Within the fed information, Rancher-Machine provisions the VMs using the vCenter credentials</figcaption></figure><p>So at the end, the way Rancher Server provisions everything as below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*C9nrdde-LDjIBcMqoCZelQ.png" /><figcaption>The 5th step is the cloud-init process, then Rancher Server makes SSH connection to relevant VMs to bootstrap K8s</figcaption></figure><h4>Provisioning with Different Operating Systems</h4><p>So far in the templates, we assumed using the default <a href="https://rancher.com/docs/os/v1.x/en/overview/">RancherOS</a> alone would suffice all our needs. You can confidently run RancherOS for your production workloads and it’s all fine.</p><p>However, if you think that is not enough for you, or you would like to tailor your base operating system, you should make your own <a href="https://github.com/David-VTUK/Rancher-Packer">Golden Image</a> and for that purpose Rancher recommends that you use <a href="https://www.packer.io/">Packer</a>.</p><h3>In-Tree and Out-of-Tree Cloud Provider</h3><p>We have mentioned about <a href="https://kubernetes.io/docs/concepts/architecture/cloud-controller/">Cloud Providers</a> a lot previously. In very simple terms, it’s a module inside K8s that links your cluster into your cloud provider’s API.</p><p>Example native cloud providers built into K8s code are:</p><ul><li><strong>AWS</strong></li><li><strong>Azure</strong></li><li><strong>GCP</strong></li><li><strong>vSphere</strong></li><li><strong>OpenStack, etc.</strong></li></ul><h4>In-Tree (Legacy)</h4><p>Logic and context of each cloud provider is embedded into most of the K8s components. For example, <a href="https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/">kubelet</a> requesting information via a metadata service from the cloud provider.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*bowa8Tnnu2oLHDVugL0JZw.png" /><figcaption>In-Tree Cloud Provider Architecture</figcaption></figure><blockquote>I am using <a href="https://rancher.com/docs/rancher/v2.5/en/cluster-provisioning/rke-clusters/cloud-providers/vsphere/in-tree/">In-Tree</a> method as my current VMware environment is not upstream version to support <a href="https://rancher.com/docs/rancher/v2.5/en/cluster-provisioning/rke-clusters/cloud-providers/vsphere/out-of-tree/">Out-of-Tree</a> cloud provider.</blockquote><h4>Out-of-Tree</h4><p>The logic in this method resides outside the K8s components, typically as deployed workloads. Decoupled from the K8s code, developed and maintained independently.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7aADrYantz369Y2TYf98Mg.png" /><figcaption>Out-of-Tree Cloud Provider Architecture</figcaption></figure><p>So, you can think of this method as the way cloud providers make their own K8s resources to talk with their own APIs.</p><p>If you have some time to spare, you can watch this 1 hour and 26 minutes of Kubernetes Master Class lab video to reinforce your readings.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FozLPpyrqwf8%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DozLPpyrqwf8&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FozLPpyrqwf8%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/afccccf21a44477c94372fb3bf66331e/href">https://medium.com/media/afccccf21a44477c94372fb3bf66331e/href</a></iframe><p>As you can imagine, using Rancher helps a lot to keep management burden at minimum. It helps us provision VMs the way we want and doing all the nasty things, helps us authenticate people in a way to manage different type of clusters, and it is really reliable.</p><p>That’s about right to sum it up. See you in the <a href="https://mrcskncn.medium.com/installing-configuring-and-day-0-operations-on-rancher-server-series-iii-360f75306fc">Series III</a>, which is about Installing and Configuring the Rancher Server, Day 0 and following operations.</p><p>Thank you for reading.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7d3e72de7359" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Production Kubernetes on-premises and GDPR… Does it ring a bell? — Series I]]></title>
            <link>https://mrcskncn.medium.com/kubernetes-on-premises-and-gdpr-does-it-ring-a-bell-74786040a76c?source=rss-e2591123be9f------2</link>
            <guid isPermaLink="false">https://medium.com/p/74786040a76c</guid>
            <category><![CDATA[on-premise]]></category>
            <category><![CDATA[clustering]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[gdpr]]></category>
            <category><![CDATA[site-reliability]]></category>
            <dc:creator><![CDATA[Umur Coşkuncan]]></dc:creator>
            <pubDate>Sun, 30 May 2021 22:31:41 GMT</pubDate>
            <atom:updated>2021-06-06T15:34:33.429Z</atom:updated>
            <content:encoded><![CDATA[<h3>Production Kubernetes on-premises and GDPR… Does it ring a bell? — Series I</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*EiVQM-5pGY6WQWpjpQsrEg.jpeg" /></figure><p><em>It’s been a very long time since I posted something new, and I can feel the excitement already. I was never the person that publishes stuff that you can read from elsewhere, always the edge cases... Man, I’m stuffed like hell. I need to let lots of information get off my chest immediately. Ergo, expect a new story from me every Sunday.</em></p><p><strong><em>Disclaimer: In this story and yet more to come, you’ll be reading lots of swearing and sarcasm against people &amp; organizations whom will mostly get offended once they see this. Once and for all, this is my personal area and my personal experience. You are very welcome to fail for doing the exact opposite and still call me for help. Got used to clean up people’s shit anyways.</em></strong></p><p>Today’s topic considers several big challenges for many of us. Yet, includes simplest solutions possible to tackle them. ’cause there aren’t many great solutions out there anyway. Everybody goes nuts over the GDPR, and I’m telling you; It’s coming, and it’s going to hit harder than you can ever imagine. Although GDPR doesn’t say anything against Kubernetes, moving from a cloud Kubernetes or just cloud itself to an on-prem Kubernetes is a real thing when the reason is only GDPR.</p><p><em>This story and upcoming series are technical reads from a CTO’s perspective, specifically focuses on Kubernetes (K8s) deployment on-premises. But in this Series I, we will only be focusing on our Data Center/Hardware specs and in the end “Why all these matters”.</em></p><blockquote>“I would like to provision a <a href="https://kubernetes.io/">K8s</a> cluster on my on-premises, and I’ll be running production workloads due to <a href="https://gdpr-info.eu/">GDPR</a>. How my organization can do it?”</blockquote><p>The question…</p><p>TL;DR — If you just have one or two containerized services, fuck it. Don’t do it for show business. Spin up half a dozen <a href="https://en.wikipedia.org/wiki/Virtual_machine">virtual machines</a> (VM) and run this:</p><pre>sudo su -c &quot;curl -sSL https://get.docker.com/ | sh&quot;</pre><p>You’ll be a lot more happier, and so do I.</p><p>We used to provision K8s clusters on the cloud and let cloud providers handle shitload of work for us, managing master/<a href="https://etcd.io/">etcd</a> nodes and doing the all things require <a href="https://aviatrix.com/learn-center/cloud-security/egress-and-ingress/">ingress/egress</a>. Whereas provisioning K8s clusters on-premises there with us all the time, we still can’t decide exactly on the reliable solution from many aspects. Nobody in my position whom I spoke with thought the labor for managing master/worker nodes, delivering right packets between networks, making things secure and also scalable as much as possible would be that high. This shit is too much to handle even with a two-pizza team with experienced engineers. Unless — you decide to make the wallet wide open or tell the responsible team that “This is your new life, get used to it.”.</p><p>To be able to make correct choices with on-prem Kubernetes, you need to know what you are doing from ground to top. You can’t just pick a provisioning tool (which most of them suck, like kubespray.), then try doing something, and eventually be shouting across GitHub Issues “why this thing doesn’t work!”. I would tell you my friend but you wouldn’t get it anyhow.</p><p>Yet, we can’t convince the companies to stay on the cloud, understandably. So, due to GDPR/<a href="https://kvkk.gov.tr/">KVKK</a> some of them hugely considers migrating to country grounds (This shit is a crazy trend in Turkey at the moment). While doing this hard operation, why wouldn’t you modernize stuff you haven’t already? Most companies I engaged over-time has lot’s of on-premise resources in several known data centers like Equinix or Sparkle. This story suits the best for these companies.</p><p>Here, my friends, by the power given me from the past 3 years’ experience in this specific scenario, I’ll be telling you not to afraid, even from the “Back-off restarting failed container” errors from a Percona MySQL Galera Cluster or filthiest major cluster upgrade scenarios, all on-premises.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XJNsjLmn5RIqVHW85bdc7w.png" /></figure><h4>Breaking down today&#39;s pillars:</h4><ul><li>Hypervisor</li><li>Storage</li><li>Network</li><li>Provisioner</li></ul><h3>Hypervisor: Let’s talk a bit, just a bit, it’s so deep.</h3><p>Obviously, you haven’t been thinking to deploy on Bare Metal, right..?</p><p>In my consideration and experience, <a href="https://www.vmware.com/">VMware</a> ESXi/vSphere is the most reliable hypervisor out there, and thank god most companies use it. It’s enterprise grade, robust, reliable, fast and fairly priced. In our to-be system, this actually is the only thing you are going to have to pay licensing for. It’s worth and also the sweetest-spot.</p><ul><li>Of course a lot of players are in this area, like <a href="https://xenproject.org/">XEN</a>, <a href="https://www.ovirt.org/">oVirt</a>/<a href="https://www.redhat.com/en/technologies/virtualization/enterprise-virtualization">RHV</a>, <a href="https://www.virtuozzo.com/">Virtuozzo</a>, <a href="https://www.openstack.org/">OpenStack</a>, <a href="https://www.proxmox.com/en/">Proxmox</a>, etc. However, considering from the K8s perspective, cluster provisioning labor is way up high in these, as tools are not that great/natively support machine creation/configuration reliably.</li><li>Talking about the non-K8s perspective, one day on XenServer you can end up hitting disk IO issues (non-licensed installations), or the other day you can suffer ridiculous VDSM/SPM issues on oVirt/RHV that you can’t figure out how.</li></ul><p>I’ve never seen any chronic or catastrophic issues with VMware, so far. It’s proven through the years experience and that&#39;s a huge plus for my organization. We depend on VMware on lots of scenarios without any hesitation, not just K8s and it’s orchestration.</p><h3>Storage: Did you forget why we are doing this?</h3><p>The only thing GDPR actually concerns about is the data, and the way it’s protected. Not the applications god knows where and how they run at, nor the kind of firewall device you initially planned to use. If you are concerned about GDPR, that means you have been collecting/generating/persisting some people’s data, and you will require at the very least one storage solution.</p><p>Specifically, Fibre Channel Flash Arrays is a really modern way to start. Never go with 1U/2U Server snap-on type SSD/HDD Disks, they will fail you in the long run. Also the solution must be working with VMware, could be either VMFS or vSAN. As long as when you have launched a new VM with a disk attached, if it’s provisioned in the <a href="https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.storage.doc/GUID-3CC7078E-9C30-402C-B2E1-2542BEE67E8F.html">Datastore</a> outside ESXi’s, we are good.</p><p>If you don’t have any storage solution and planning to get one, my recommendation would be <a href="https://www.purestorage.com/">Pure Storage</a>. When compared to <a href="https://www.delltechnologies.com/tr-tr/storage/data-storage.htm#scroll=off">EMC</a>, <a href="https://www.netapp.com/">NetApp</a> or other players, you might never have heard of them. But they are astonishingly great, the pricing policies on Pure Storage are really sweet and quality of the disks depend on the <a href="https://en.wikipedia.org/wiki/Original_equipment_manufacturer">OEM</a> itself. Unlike big players, Pure can’t steal from your pocket on failed disk swaps. Depending on your storage space needs and other features, you can find a model for your taste pretty quickly.</p><h3>Network: Have you ever heard of PPS?</h3><p>As the storage works over the network, switching capability comes to mind. You have to use quality switches especially for storage, however you should also have another switch pool for overall networking for ESXi nodes or your firewall. These hardware switching pools (Two devices per pool) must be completely separated to avoid unnecessary bottlenecks. On the storage side, providing at least 10 Gbit/s per port (25 Gbit/s better nowadays) with forwarding throughput of more than 10 billion packets per second (<a href="https://en.wikipedia.org/wiki/Throughput">PPS</a>) for a neat flow.</p><p>My bet here is for <a href="https://www.broadcom.com/products/fibre-channel-networking/switches">Brocade</a>, which is a part of <a href="https://www.broadcom.com/">Broadcom</a> lately. Other quality brands like <a href="https://www.juniper.net/us/en/">Juniper</a> or <a href="https://www.cisco.com/">Cisco</a> are welcome too, as long as they meet specs and you can manage them hassle free. Last time I checked, our Brocade pool has never been rebooted for over 2160 days. I wish you the same outcome.</p><h4>How’s it going to look like when we set our subnets?</h4><ul><li>192.168.0.0/24 — Only for storage devices, must be physical network.</li><li>192.168.1.0/24 — Only for bare metal devices, must be physical network.</li><li>192.168.100.0/24 — Only for Kubernetes VM’s, can be <a href="https://docs.vmware.com/en/VMware-NSX-T-Data-Center/index.html">NSX-T</a> (virtual) network or physical. On this specific network it’s required to deploy a <a href="https://en.wikipedia.org/wiki/Dynamic_Host_Configuration_Protocol">DHCP</a> server. Doesn’t matter if it’s from a hardware switch or via the NSX Edge Service Gateway (<a href="https://docs.vmware.com/en/VMware-NSX-Data-Center-for-vSphere/6.4/com.vmware.nsx.install.doc/GUID-2482B032-F420-432F-A6D0-6CD91506BFCC.html">ESG</a>).</li><li>192.168.254.0/24 — Management network, must be a physical network. We will plug in our <a href="https://www.delltechnologies.com/tr-tr/solutions/openmanage/idrac.htm#scroll=off">iDRAC</a>, <a href="https://www.hpe.com/us/en/servers/integrated-lights-out-ilo.html">iLO</a> or other <a href="https://en.wikipedia.org/wiki/Intelligent_Platform_Management_Interface">IPMI</a> solutions on this network.</li></ul><p>Kubernetes VM’s, or in other terms 192.168.100.0/24 must be able to talk with 192.168.1.0/24 bidirectionally. So that, maintaining disks and other stuff like, machine creation, configuration changes or even machine removal at some point can be done.</p><p>I’ve mentioned this because you will need some sort of router/firewall to egress your way out to internet from these private networks. While setting your policies, you can now consider this. If you are keen on these things, you can get two garbage servers with sufficient CPU/Memory and deploy <a href="https://www.pfsense.org/">pfSense</a> on top to make your own firewall, just for the garbage price.</p><p>Don’t worry about the Hardware Load Balancer at this point, unless you got one like <a href="https://www.f5.com/services/resources/glossary/load-balancer">F5</a>, <a href="https://www.citrix.com/solutions/app-delivery-and-security/load-balancing/what-is-load-balancing.html">Citrix</a>, <a href="https://www.barracuda.com/products/loadbalancer">Barracuda</a>, etc. <strong>Never, ever, use your firewall as a load balancer, even though it supports load balancing! </strong>It’s not meant to do that. They get easily overheated and bottlenecked during the SSL/TLS encryption or decryption phases. Sucking up their average lifetime and locks up your network traffic, so you are dead when this device dies, so the business.</p><h3>Provisioner: The most fun part, Trichotillomania. (and some personal stories)</h3><p>You might have never heard of this disorder. In basic terms, <a href="https://en.wikipedia.org/wiki/Trichotillomania">Trichotillomania</a> is pulling hair out due to stress.</p><p>It’s been 7 years since Kubernetes became globally available. Yet, still we have to deal with shitty on-premise cluster provisioners. They are either trying to <strong>rip off your wallet</strong> or <strong>skull-f#ck your entire architecture</strong> with <strong>meaningless deployment approaches</strong> and eventually create something <strong>unmanageable</strong> and <strong>faulty</strong> from the very beginning. If I’d like to pay for deploying an open-source project, I would’ve gone with <a href="https://tanzu.vmware.com/tanzu">Tanzu</a>, considering my current environment… But why would on EARTH, I would do it? It’s still unstable, yet away from lots of features, and expensive as hell. <strong>Just because paying up means “I’ll get enterprise support over something unmanageable — even for them” — justifies my pure efforts of dealing things on my own by my team with peace alone.</strong></p><p>I tried many things, believe me. I can even make a list named “which one is the shittiest —popularity ordered” and call it a day over the gags.</p><p>After 13 weeks of overtime, hard work and sweat; I’ve found my companion.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*3iQydRCHLaT2oyHMkoxvgQ.jpeg" /><figcaption><a href="https://rancher.com/">https://rancher.com/</a></figcaption></figure><ul><li>Reliable</li><li>Fast</li><li>Robust</li><li>Open-source</li><li>Customizable</li><li>Low pod footprint</li><li>Feels right</li></ul><p>. . .</p><p>The most important of all… <strong>IT WORKS!</strong></p><blockquote>Almost two years ago, I was a consultant, providing DevOps/Cloud/Consultancy services for most of the biggest companies in the country. Back then, I was at OpenShift 4’s (<a href="https://www.openshift.com/">OCP4</a>) launch event and I saw that, every IT manager in those companies were so excited over OCP4 that they completely lost their minds, got hyped as much as they would. So that we had to change our priorities, pioneer the platform, and then spread it around like a religion. Which also affected my position in place.</blockquote><blockquote>We sold a lot, and succeeded. I did my job great, every company I visit at was adoring my efforts to run their nonsensical architectures within a modern approach. During those days, I wasn’t getting why they were paying for OCP4, or Pivotal Kubernetes Service (<a href="https://docs.pivotal.io/pks/1-7/index.html">PKS</a> — became Tanzu), or <a href="https://www.suse.com/products/caas-platform/">SUSE</a> Container as a Service Platform (shittiest of all —LITERALLY — imagine bootstrapping a cluster with <a href="https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/">kubeadm</a> and paying for it to get enterprise support that nobody knows how to help you out, LoL!).</blockquote><blockquote>I thought they had reasons. I thought maybe they wanted the features. What it actually was that they were just illustrated with the wrapped version of K8s. Everything means money when you are an enterprise. They were paying for the tool. They were paying for the consultancy for the tool. They were paying for the educations/trainings for those consultants that they supposed to provide support for the tool. They even paid extra hours for their own extra development efforts on top of <a href="https://en.wikipedia.org/wiki/Vanilla_software">vanilla</a> K8s to adapt OCP4. As an IT guy, I can’t describe my feelings how f#cked up things were.</blockquote><blockquote>Let me tell you what; Several months passed and, technically speaking I still don’t get why they have paid and still are paying for this, LoL! I started to believe I’ll never get it…</blockquote><p>As in all other RedHat projects, OpenShift has an open-source edition as well (following a couple releases behind). If you know OCP4, then you must’ve heard Origin Kubernetes Distribution (<a href="https://www.okd.io/">OKD</a>). They are almost identical in feature-wise, without the obligation to pay for enterprise support or stupid license fees.</p><p>Within my prior experience with OpenShift, I thought I would be more comfortable provisioning K8s with extra benefits. So I did, and provisioning phase was so smooth. Still, better installation experience than Rancher and any other on-premise solution thanks to Installer Provisioned Infrastructure (<a href="https://docs.openshift.com/container-platform/4.7/installing/installing_vsphere/installing-vsphere-installer-provisioned.html">IPI</a>). You don’t even have to worry about load balancing, it deploys <a href="https://github.com/acassen/keepalived">keepalived</a> and <a href="http://www.haproxy.org/">HAProxy</a> cluster (OpenShift Router) for you. Here it comes…</p><p><strong>BUT</strong>, RedHat recommends that master nodes must have 16g memory and 4 cores each, with a 3 node cluster, and another cluster containing 3 worker nodes with minimum of 8g memory and 2 cores, so that OKD pods will be running there. Minimum 120g disk space on all 6 nodes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OA9GPID_iReCWpBvztV4yg.png" /><figcaption>OpenShift Container Platform v4.7.0 — Dashboard</figcaption></figure><p>Can you look at this mess? This is something I forgot during my OCP4 provisioning days. It’s unacceptable. I’m going to sacrifice 18 cores of 2nd Generation Intel Gold processors, 72gb’s of memory and 720g of disk space for what? A fancy cluster? 251 pods are already running, I have just logged-in!</p><p>In the screenshot above, you could say that “C’mooon don’t exaggerate, it’s just using 20gb of memory, not 70gb at all.” Well then my genius friend, I’m challenging you to deploy a <a href="https://www.percona.com/software/mysql-database/percona-xtradb-cluster">PXC</a> cluster on a clean OKD/OCP install, with 600m core and 1g memory on each pxc pod and 150m core and 300m memory on each HAProxy pod.</p><p>See, you won’t be able to deploy second PXC pod, I guarantee you that. This is mainly because the 48g of memory is already reserved for master nodes cluster and they are unschedulable, more than 200 pods are running at the worker node cluster with only 24g memory, there are requests, limits and <a href="https://kubernetes.io/docs/tasks/configure-pod-container/quality-service-pod/">QoS</a> policies in play. We are just getting started with OKD. We don’t have a private registry or any build jobs running, or any other cluster-wide used tool deployed in that matter.</p><p>That wasn’t the only issue but, I pivoted to solutions like <a href="https://kubespray.io/#/">kubespray</a>, <a href="https://docs.conjure-up.io/stable/en/spellbooks/kubernetes">conjure-up</a>, and many more… To have a lightweight approach. Well f#ck me, alright? Never, ever going to walk through that path again, not even for a single customer. Provisioning stages are just <a href="https://www.urbandictionary.com/define.php?term=pita">PITA</a>, and they don’t work as intended. It’s <strong>2021</strong>, why would I want to provision my VM’s first and then make an inventory for them so that our wonderful tools will work their way up from where I or <a href="https://www.terraform.io/">Terraform</a> started. If you make this an option to automatic provisioning approaches, then yeah, good work mate. There are freaks who’d like to over-engineer something they shouldn’t touch, good for them. But as an obligation, I don’t think so.</p><p>Finally landed my feet on Rancher Server.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*klZWcQfkQ1zy5Fw_3oyBow.png" /><figcaption>Rancher Kubernetes Engine (RKE) with K8s v1.20.6 — Dashboard</figcaption></figure><p>Would you look at this beauty? <strong>Impeccable</strong>. Just 13 pods used and we’re alive with 3 masters and 2 workers, <a href="https://kubernetes.github.io/ingress-nginx/">Nginx Ingress Controller</a> and <a href="https://kubernetes.github.io/ingress-nginx/user-guide/default-backend/">Default Backend</a> pods are enabled. <a href="https://vmware.github.io/vsphere-storage-for-kubernetes/documentation/overview.html">vSphere Cloud Provider</a> was also enabled, so now we can provision any volume we want from RKE at any time or scale our cluster within click of a button in about 5 minutes. Feels like Kubernetes on cloud.</p><p>Best part is, I never had to touch any server or provision something from scratch, or spend my days and nights coding Terraform/<a href="https://www.ansible.com/">Ansible</a> scripts.</p><p>Up next: <a href="https://mrcskncn.medium.com/what-rancher-server-does-and-how-it-works-series-ii-7d3e72de7359"><strong>Series II</strong></a>. We’ll dig into lots of details with Rancher and peripherals.</p><p>Stay tuned and thank you for reading.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=74786040a76c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Guide to backup and restore big tables in DynamoDB completely]]></title>
            <link>https://mrcskncn.medium.com/guide-to-backup-and-restore-big-tables-in-dynamodb-completely-44dd1bd73446?source=rss-e2591123be9f------2</link>
            <guid isPermaLink="false">https://medium.com/p/44dd1bd73446</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[amazon]]></category>
            <category><![CDATA[data-backup]]></category>
            <category><![CDATA[amazon-web-services]]></category>
            <category><![CDATA[dynamodb]]></category>
            <dc:creator><![CDATA[Umur Coşkuncan]]></dc:creator>
            <pubDate>Sun, 29 Jan 2017 04:13:53 GMT</pubDate>
            <atom:updated>2017-01-29T04:16:54.463Z</atom:updated>
            <cc:license>https://creativecommons.org/publicdomain/mark/1.0/</cc:license>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/744/1*WNxQLuAMjscBub8DNt9mZg.png" /></figure><p>To be brief, in the company I’m working at, we are depending on real-time data with no tolerance of high-latencies. The lower the latency, the better results we can provide to our customers/clients. After Amazon launched the Frankfurt region (eu-central-1) we decided to move from Ireland (eu-west-1) to Frankfurt. It’s much lower latency when you ping both regions from Istanbul.</p><p>I was looking for a practical and easy solution to the issue of ours but after trying many solutions (like Data Pipeline) and getting tired of their complexity, I tend to lean on some open-source projects to do our task.</p><p>After a couple researches I found this little awesome batch tool working with boto: <a href="https://github.com/bchew/dynamodump">https://github.com/bchew/dynamodump</a></p><p>I gave it a shot and it worked nearly-perfectly. You can use this tool from your local machine or from an EC2, depending on your connectivity and data-size of course. I had issues on tables with high amount of items but, there is a workaround for that. You need to fine-tune the batch script to work with your dataset.</p><p>The default configuration is as follows (inside dynamodump.py):</p><pre>JSON_INDENT = 2<br>AWS_SLEEP_INTERVAL = 10  <em># seconds</em><br>LOCAL_SLEEP_INTERVAL = 1  <em># seconds</em><br>MAX_BATCH_WRITE = 25  <em># DynamoDB limit</em><br>SCHEMA_FILE = &quot;schema.json&quot;<br>DATA_DIR = &quot;data&quot;<br>MAX_RETRY = 6<br>LOCAL_REGION = &quot;local&quot;<br>LOG_LEVEL = &quot;INFO&quot;<br>DATA_DUMP = &quot;dump&quot;<br>RESTORE_WRITE_CAPACITY = 25<br>THREAD_START_DELAY = 1  <em># seconds</em><br>CURRENT_WORKING_DIR = os.getcwd()<br>DEFAULT_PREFIX_SEPARATOR = &quot;-&quot;</pre><p>What I needed to change are those:</p><pre>JSON_INDENT = 4<br>LOCAL_SLEEP_INTERVAL = 3<br>MAX_BATCH_WRITE = 10<br>MAX_RETRY = 10<br>RESTORE_WRITE_CAPACITY = 10</pre><p>Reason I’m changing this is that the script starts <strong>throwing errors</strong> because the <em>provisioned DynamoDB write capacity units/ps is less than the inserted units/ps by the script</em>.</p><p>There are two workarounds here, the first one is the above fine-tunes in the script. Which takes around 30 seconds for 14.000 Items to insert into with this config. But if you have more than 1.000.000 items and you want the restore operation to be completed in less than 5 minutes, then you need to<strong> temporarily scale up the DynamoDB write capacity units per table.</strong> After the restore operation succeeds, you can scale down the write capacity units back to its normal.</p><h3>Back it up!</h3><p>Easy. First, you need to create a user with AWS Access Key (from IAM) with FullAccess permissions to DynamoDB. There is an AWS managed policy called <em>AmazonDynamoDBFullAccess</em> and you can use it for the entire operation.</p><p>To backup every table in Ireland region:</p><pre>python dynamodump.py -m backup -r eu-west-1 -s &quot;*&quot; <em>--accessKey AWS_ACCESS_KEY --secretKey AWS_SECRET_KEY</em></pre><p>To backup single table in Frankfurt region (change <strong>TableName</strong> with your table name):</p><pre><em>python</em> <em>dynamodump</em>.<em>py</em> -<em>m</em> <em>backup</em> -<em>r</em> <em>eu</em>-<em>central</em>-<em>1</em> -<em>s</em> <em>TableName</em> --<em>accessKey</em> <em>AWS_ACCESS_KEY</em> --<em>secretKey</em> <em>AWS_SECRET_KEY</em></pre><h3>Restore any means necessary.</h3><p>One thing your should be careful with is to create table schema (empty table with its structure) before you start inserting data, otherwise script might fail during the sequential operations.</p><p>To restore every table you have, to London (eu-west-2) region:</p><pre><em>python</em> <em>dynamodump</em>.<em>py</em> -<em>m</em> <em>restore</em> -<em>r</em> <em>eu</em>-<em>west</em>-<em>2</em> -<em>s</em> <em>&quot;*&quot;</em> --<em>accessKey</em> <em>AWS_ACCESS_KEY</em> --<em>secretKey</em> <em>AWS_SECRET_KEY</em> --<em>schemaOnly</em><br><em>python</em> <em>dynamodump</em>.<em>py</em> -<em>m</em> <em>restore</em> -<em>r</em> <em>eu</em>-<em>west</em>-<em>2</em> -<em>s</em> <em>&quot;*&quot;</em> --<em>accessKey</em> <em>AWS_ACCESS_KEY</em> --<em>secretKey</em> <em>AWS_SECRET_KEY</em> --<em>dataOnly</em></pre><p>Or if you’d like to insert single table (change TableName with your table name):</p><pre><em>python</em> <em>dynamodump</em>.<em>py</em> -<em>m</em> <em>restore</em> -<em>r</em> <em>eu</em>-<em>west</em>-<em>2</em> -<em>s</em> <em>TableName</em> --<em>accessKey</em> <em>AWS_ACCESS_KEY</em> --<em>secretKey</em> <em>AWS_SECRET_KEY</em> --<em>schemaOnly</em><br><em>python</em> <em>dynamodump</em>.<em>py</em> -<em>m</em> <em>restore</em> -<em>r</em> <em>eu</em>-<em>west</em>-<em>2</em> -<em>s</em> <em>TableName</em> --<em>accessKey</em> <em>AWS_ACCESS_KEY</em> --<em>secretKey</em> <em>AWS_SECRET_KEY</em> --<em>dataOnly</em></pre><h3>Data Location</h3><p>The data you backed-up will be stored in a folder called <strong><em>dump </em></strong>where the dynamodump.py script located. Table schema information is covered in scheme.json and the data per table will be rested as partitions (like 0001.json, 0002.json, etc.) if you have high amount of items per table.</p><p>The operation will take 30 seconds to 30 minutes depending on the data size inside tables. If you are only taking backups, then you can do it on-the-fly but if you are planning to migrate to another region like us, you should consider having a downtime off-hours. To avoid that you may use DynamoDB Streams but this will require a lot of time for research. There are several github projects basing this as well.</p><p>That basically sums everything up.</p><p>Be well.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=44dd1bd73446" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>