<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Thang Nguyen on Medium]]></title>
        <description><![CDATA[Stories by Thang Nguyen on Medium]]></description>
        <link>https://medium.com/@thangnguyen_80?source=rss-1f68c1c2b265------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*zqYjF4lUhrs33PZX.</url>
            <title>Stories by Thang Nguyen on Medium</title>
            <link>https://medium.com/@thangnguyen_80?source=rss-1f68c1c2b265------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 10:19:49 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@thangnguyen_80/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[End-2-End testing Kubernetes infrastructure]]></title>
            <link>https://medium.com/@thangnguyen_80/end-2-end-testing-kubernetes-infrastructure-f27bb2136073?source=rss-1f68c1c2b265------2</link>
            <guid isPermaLink="false">https://medium.com/p/f27bb2136073</guid>
            <category><![CDATA[e2e-testing]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Thang Nguyen]]></dc:creator>
            <pubDate>Tue, 14 Jul 2020 02:07:00 GMT</pubDate>
            <atom:updated>2020-07-14T02:21:39.462Z</atom:updated>
            <content:encoded><![CDATA[<h3>Why having E2E tests</h3><p>It has been great to see DevOps and Infrastructure-As-Code (IAC) are becoming more and more prevalent in the last few years. Kubernetes infrastructures naturally follow IAC, all of the Kubernetes resources use YAML to describe themselves. Most of the Kubernetes projects that I have been exposed to also use some sorts of scripting to glue all the YAMLs together, normally some Python or Bash scripts.</p><p>I am a software engineer by profession, and a strong believer in automated tests. Every piece of code should come with tests. The last time that I worked on a software development project that didn’t have automated tests were many many years ago. However, it is very common to see Kubernetes projects that don’t have any test whatsoever.</p><p>Having adequate tests is extremely important, it does not only make sure the things you created work the way they should, but also making maintenance down the road much more easier.</p><p>For example, if I am to upgrade the Spring Boot framework to a new major version in one of my microservices, if all the tests pass I can be reasonably confident that the microservice still works with the new Springboot version. However, if my Kubernetes cluster has no test, and I need to upgrade my Istio version, I have no way to be sure that it wouldn’t break some of the apps in the cluster. And that supposed-to-be simple task can turn into a very expensive exercise</p><h3>What are the available solutions out there</h3><p>So the idea is to create an ephemeral Kubernetes cluster, apply all the configs of your Kubernetes cluster, deploy the applications that are going to run on it. And then run do assertions to make sure everything is in the state that they are expected to be.</p><p>There are several tools that allow creating Kubernetes cluster for testing</p><ul><li>Minikube</li><li>Kubernetes in Docker (KIND)</li><li>Rancher k3s</li></ul><p>Among those solutions, only KIND is certified by Cloud Native Computing Foundation, and it is the most completed implementation</p><h3>Kubernetes-IN-Docker</h3><p>Having a cluster in a consistent state is very important for testing. This requires create a disposable cluster every time your E2E tests run.</p><p>Kubernetes-IN-Docker (KIND — <a href="https://github.com/kubernetes-sigs/kind">https://github.com/kubernetes-sigs/kind</a>), is a tool for running local Kubernetes clusters using Docker container, which then can be used for local development or CI testing.</p><p>It is much cheaper and faster to use KIND to create a disposable Kubernetes cluster than create one on GCP or AWS EKS. You can create one cluster with KIND in 30 seconds. All that is contained in a Docker container and will go away when the container is deleted.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/856/1*UlVvUk27XKIEnFc8EIPpmg.png" /><figcaption>Create a cluster with KIND</figcaption></figure><p>KIND Github: <a href="https://github.com/kubernetes-sigs/kind">https://github.com/kubernetes-sigs/kind</a></p><h3>Bats-core: Bash Automated Testing System</h3><p>I tend to use Bash scripts for Kubernetes testing due to their simplicity. I use Bats-core as the testing framework to define tests cases and provide assertion capability.</p><blockquote>Bats is a TAP-compliant testing framework for Bash. It provides a simple way to verify that the UNIX programs you write behave as expected.</blockquote><p>Sample test using Bats</p><pre>#!/usr/bin/env bats</pre><pre><a href="http://twitter.com/test">@test</a> &quot;addition using bc&quot; {<br>  result=&quot;$(echo 2+2 | bc)&quot;<br>  [ &quot;$result&quot; -eq 4 ]<br>}</pre><pre><a href="http://twitter.com/test">@test</a> &quot;addition using dc&quot; {<br>  result=&quot;$(echo 2 2+p | dc)&quot;<br>  [ &quot;$result&quot; -eq 4 ]<br>}</pre><p>Bats-core Github: <a href="https://github.com/bats-core/bats-core">https://github.com/bats-core/bats-core</a></p><h3>DETIK: DevOps e2e Testing in Kubernetes</h3><p>DETIK is a subproject of Bats-core, which provides an extra assertion capabilities against Kubernetes clusters. DETIK has a simple syntax, in a natural language to write assertion.</p><p>Sample tests with Bats and DETIK</p><pre>#!/usr/bin/env bats</pre><pre>load &quot;lib/utils&quot;<br>load &quot;lib/detik&quot;</pre><pre>DETIK_CLIENT_NAME=&quot;kubectl&quot;</pre><pre>setup() {<br>  # deploy nginx<br>  kubectl apply -f configmap.nginx.yaml<br>  kubectl apply -f deployment.nginx.yaml</pre><pre>echo &quot;wait for nginx to be ready...&quot;<br>  sleep 10<br>}</pre><pre>teardown() {<br>  # tear down nginx<br>  kubectl delete -f deployment.nginx.yaml<br>  kubectl delete -f configmap.nginx.yaml<br>}</pre><pre><a href="http://twitter.com/test">@test</a> &quot;should deploy nginx with 2 replicas&quot; {<br>  run verify &quot;there are 2 pods named &#39;nginx&#39;&quot;<br> [ &quot;$status&quot; -eq 0 ]<br> <br> run verify &quot;there is 1 service named &#39;nginx&#39;&quot;<br> [ &quot;$status&quot; -eq 0 ]<br>}</pre><p>DETIK Github: <a href="https://github.com/bats-core/bats-detik">https://github.com/bats-core/bats-detik</a></p><h3>Continuous Integration pipeline</h3><p>The end goal of having those tools and tests is to run them on a CI pipeline on every change to your Kubernetes config.</p><p>Here is a sample YAML file for Gitlab CI</p><pre>default:<br>  before_script:<br>    # start up Docker daemon<br>    - export DOCKER_HOST=tcp://127.0.0.1:2375<br>    - nohup /usr/local/bin/dockerd --host=tcp://127.0.0.1:2375 --storage-driver=overlay2&amp;<br>    - timeout 15 sh -c &quot;until docker info; do echo .; sleep 1; done&quot;<br>stages:<br>  - test<br>e2e-test:<br>  image: docker:19.03.11-dind<br>  stage: test<br>  script:<br>    # install curl<br>    - apk update<br>    - apk add bash curl<br>    # install KIND<br>    - echo &quot;&gt;&gt;&gt; Installing kind&quot;<br>    - curl -sSLo kind &quot;<a href="https://github.com/kubernetes-sigs/kind/releases/download/v0.8.1/kind-linux-amd64">https://github.com/kubernetes-sigs/kind/releases/download/v0.8.1/kind-linux-amd64</a>&quot;<br>    - chmod +x kind <br>    - mv kind /usr/local/bin/kind<br>    # install Kubectl<br>    - curl -LO <a href="https://storage.googleapis.com/kubernetes-release/release/%60curl">https://storage.googleapis.com/kubernetes-release/release/`curl</a> -s <a href="https://storage.googleapis.com/kubernetes-release/release/stable.txt%60/bin/linux/amd64/kubectl">https://storage.googleapis.com/kubernetes-release/release/stable.txt`/bin/linux/amd64/kubectl</a><br>    - chmod +x ./kubectl<br>    - mv ./kubectl /usr/local/bin/kubectl<br>    # install Bats<br>    - apk add npm<br>    - npm install -g bats<br>    # start KIND cluster<br>    - ./e2e-kind.sh<br>    # run some tests<br>    - ./e2e-simple.sh</pre><h3>Conclusion</h3><p>Kubernetes-IN-Docker allow creating ephemeral cluster which could be almost identical to your production cluster. Having E2E tests run on KIND will ensure your changes will work in production, greatly improve the quality and confidence when making changes.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f27bb2136073" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>