Part 5 — Using Horizontal Pod AutoScaler

Welcome back, my fellow Jedi apprentice! Now we have a pretty good grasp on the basics of using the Force. Let’s start making our way into some interesting adjacent concepts. In case you need a refresher, here are the links for what we discussed so far:

• Part 1 — What is kubernetes after all?
• Part 2 — A Look under the hood
• Part 3 — Life of a Pod
• Part 4 — Deployments and Services

Now we can start exploring how you can use deployments to implement cool stuff like auto-scaling, auto-healing and graceful degradation; this part will build upon the concepts we learned in part 4. This post is all about auto-scaling using deployments!

Auto-scaling #WTF

This is a simple concept: your workloads can scale up or down according to the load in the cluster or workload resources.

By scale up, I literally mean deploying new pods and scaling down destroying them. To make this possible, Kubernetes has an Object called “Horizontal Pod AutoScaler” (aka HPA) that must be referencing a Deployment with a ReplicaSet (part 4). Fortunately, once you define HPA in the deployment the ReplicaSet becomes implied. Also, there is an HPA controller; it monitors resources and changes the replicas value in the ReplicaSet object. The HPA object tells k8s that we want to auto-scale the deployment so the controller knows it should monitor the Pod and its resources ( Aha! “building blocks” 😉 ).

Now, that is basically how scaling works in k8s, but remember that we need a reference point… the workload scales in reference to something. That something is a Pod Resource. The most basic Pod Resources are CPU and Memory. The cluster automatically monitors their utilization and keeps the related Pod Resource objects up to date; It does this using a Metric Server that aggregates metrics across the nodes by querying the kubelet of each node. The diagram below is for illustrative purposes — but gives you a good sense of how this fits into the picture.

Also, resources can be other parameters too. Anything that can be imported as a Pod Resource can be used as a scaling reference. Kubernetes has a few mechanisms that allow this. Most commonly “Custom metrics” in the Metric Server. It’s also possible to use an object called “Extended Resources”. This effectively allows you to declare resources from outside the cluster that might affect your workload scale (useful for event-driven architecture). Let me know if you want a post on this specifically, we can venture further into it!

Avoid Noise Neighbours #BestPractice

A quick thing we should discuss, yes, neighbours! Essentially, noise neighbours are Pods that consume too many resources to the detriment of other pods in the cluster. So, it’s a good practice to define at least Memory and CPU consumption for your pods (even if you are not using HPA) to avoid the risk of overcrowding your node. Another benefit of doing this is that kubelet will keep track of CPU and Memory utilization in the node, so the control plane can select nodes with enough capacity available for the pod. Also, when you define a minimum and maximum it allows the pod to start warm by having a dedicated amount of resources from the beginning and you prevent rogue workloads from consuming all available resources in case of issues or compromised containers.

Ideally, you should measure your workload resource consumption profile for one Pod. A load test is the best tool to help you understand that. Simply monitor CPU and Memory usage of the Pod while ensuring that the application remains stable and within your reliability parameters (latency, availability, error rate, etc).

There are a few things you should consider; think of this exercise as a Gaussian curve, what you select depends on your traffic pattern.

The graphic above means to illustrate this:

• Minimum load (Yellow): the lowest load you get (this can be idle or with trickle traffic)
• Nominal load (Blue): a chosen percentile in a cycle that represents the most common load for your application (think 99th or 95th in a day for example)
• Peak load (Red): maximum load the application can reliably serve.

You want to configure your Pod Resources to request either your nominal or minimum load at startup and scale-up before reaching the peak load. Which one you pick depends on where your application spends the most time. Of course, this changes over time so make sure to revisit this frequently. Major versions or traffic pattern changes are good indicators that your values need a review.

Now you ask me, this is so much work… can’t I just guesstimate? Well, young Jedi apprentice… no one will stop you! However, be aware of what it means. You may end up scaling too soon or too late, or you may also block more resources than you need or less than you need. So, measuring is better (even if it’s a partial measurement to allow you to make an informed guesstimate approximation). If you can, eliminate the guesswork!!

Ok, back to our deployment! Let’s start by defining our resources in the deployment. Remember our friendly NGINX deployment from part 4? Here is what that object looks like with resources defined for memory and CPU:

As you can see we no longer need to tell the deployment the number of replicas we need (HPA will take care of that), so we remove spec.replicas. We also set the spec.containers.resources fields with the limits of how much CPU and Memory we want. This is where you would reference a custom metric as well.

Sidebar on reading the values of CPU and Memory. They are rather confusing at first sight ! (

• Memory — measured in bytes, it can be represented using binary prefixes (Ki, Mi, Gi) or decimal prefixes (K, M, G).
• CPU — measured in CPU units (which depends on where you are running), for bare-metal it’s 1 hyperthread, but in GCP it’s 1 Core. This metric allows fractions so we can actually request half a CPU (0.5 — read this as 50%). Most workloads won’t actually occupy that much, so kubernetes has the “m” prefix that indicates milli (in decimal prefixes) — so requesting 200m is equivalent to requesting 0.2 units of the CPU (a.k.a.: 20% CPU utilization).

I highly recommend you choose either binary or decimal prefixes. Maintain consistency across all your deployment files, especially for large scale workloads.

I hope this was useful ! )

Now adding HPA is simple…. kind of. The HPA object is still evolving. Pay special attention to the apiVersion you choose (autoscaling/v1 vs autoscaling/v2beta2). On the v2 you can scale in multiple metrics while v1 can’t. Here we are using the v2 flavour.

For auto-scaling purposes, we define the Pod resources. We also define the consumption threshold in which the upscaling should happen. The HPA controller will query the resources every 15s (default) and make adjustments based on the thresholds vs actual consumption. As you can see on the object above that is the spec.metrics[].resource.target field; the way it links back to the deployment is via the spec.scaleTargetRef field.

Let’s see this action — grab the YAML above and follow along! By now you are very familiar with kubectl apply so I’ll skip that step. Let’s focus on the cool stuff!

Time to inspect the deployment! You can see the resource parameters and the replica set; notice it starts with a single pod.

Let’s inspect the actual HPA object, notice the metrics section:

As you can see, it references our nginx-deployment and the current number of pods deployed is one. Now, as a quick test, we can generate some load and see the auto-scaling in action:

Several things go into making an auto-scaling configuration. However, start with what you have. If you can’t do a forecast or don’t have all the required data (new applications I see you), it should not prevent you from starting with good enough numbers. You can fine-tune it over time.

This is a very cool concept that can be explored further, I recommend spending some time playing with this configuration; as an exercise try to forecast what is the TPS you’ll hit 3, 5 and 10 pods with and see if you got it right. Notice the period it takes to scale down after the load is gone? How would that behave in different traffic patterns?

As usual, if you don’t have your cluster yet Katacoda Playground is still an option. You can also use the free tier and credits on GCP to create a GKE cluster that you can play with!

In this post, we reviewed how to implement auto-scaling with deployments, in the next post we will look at auto-healing, graceful degradation which enables us to create more resilient workloads in kubernetes.

Do you like the series so far? Let me know! Also, feel free to follow me on Twitter (@fawix) or LinkedIn so you won’t miss the update! May the Containers be with you!

Kubernetes for dummies: Deployment auto-scaling was originally published in ITNEXT on Medium, where people are continuing the conversation by highlighting and responding to this story.

Part 4 — Making Pods more useful

Welcome back, young Padawan! Now that you have a pretty good idea of the inner workings of the cluster and how a Pod works, let’s check how to use those things more effectively.

As we saw in the last post, a Pod provides our basic building block, but it’s not necessarily production-friendly. We can only access the application from inside the Pod. It would be really inconvenient and risky to expose it to the internet that way. Besides, it’s not permanent. If a Pod dies, it stays dead. There are different ways to solve those two items. We’ll cover the basics and add as we go.

Let’s discover Service and Deployment objects 😎

Deployment #WTF

Deployment adds several goodies to Pods and enables several important adjacent use-cases. In essence, deployments allow k8s to have desired state rules on a Pod or group of Pods. These are things like version rollout, rollback, pause at a basic level.

Kubernetes Deployment also makes use of other k8s objects to further enhance functionality. One super common construct is the “ReplicaSet”, which defines how many copies of the same Pod should be created by this deployment. A copy of a Pod is called a replica. If we say a deployment contains 3 replicas it means it will be running 3 copies of the same Pod. This means that, at any given time, k8s will ensure that the desired amount of Pods are running as part of this deployment! If a pod dies, k8s will start a new one to replace it and match our desired state.

Remember our nginx Pod? What if we wanted to make it a Deployment so in case our Pod dies k8s restarts it automatically?

Here is what that object will look like:

Notice that the object kind is now Deployment. Also, see the spec.template? It contains the definition of the actual Pod. If you compare that with the Pod we created in part 3 you’ll see that it’s the same. Also, take special note of the spec.replicas field. This is part of the ReplicaSet and defines how many copies we want.

Remember we talked about labels in part 3? This is one of its applications. The ReplicaSet Controller selects all the Pods which matches the spec.selector.matchLabels and compare them to the desired state. Those labels should match the information on the actual Pod labels in spec.template.metadata.labels. This concept is used in several places in k8s and groups related objects to build the functionality around them.

Labels are fundamental to organizing large deployments. You can also perform kubectl operations filtering via labels making this construct really powerful. These are also called selector labels and, sometimes, filters (although less common).

Let’s see this in action:

Ok, now that we solved the permanency problem, let’s talk about accessing the application.

Services #WTF

There are multiple ways to expose your application using k8s. Services are one of the simple ways we can do it.

Think of the Service object as a networking abstraction. It provides a way to expose an application running in one or more pods via a single IP. Remember Pods have their own IP. For example, if your deployment has 3 replicas, you don’t want to manage that in your application; you need a consolidated way to access it. Services object will take care of handling that bit.

Here is what a Service object looks like:

One handy feature of services is that it can map any incoming port to a target port on the container as you can see in the example above. The incoming traffic (into the Service) is on spec.ports.port and traffic going from the service to the container is mapped in the spec.ports.targetPort.

At a lower level, a service contains a collection of endpoints. An endpoint is a simple object with the IP of a Pod. Remember labels? That is what k8s uses to group Endpoints as part of a service. Normally, you don’t need to worry about creating endpoints since that is done behind the scene, but you can actually see it in action:

With the labels, k8s will automatically create the endpoints. The endpoint object also allows you to point services to IPs outside the cluster; by manually provisioning the endpoint object. This is useful especially for testing environments or external dependencies that are not deployed in the cluster but needs to be accessed by the pods.

The service object will provide us with an IP and a DNS entry on the cluster DNS server to facilitate service-to-service communications.

Speaking of DNS… kube-dns is a component that is often deployed in the cluster. Although not a mandatory nor a core component, it’s essential to clusters running multiple services. It can be replaced with other DNS services if desired. A popular alternative is coreDNS. It provides name resolution within the cluster, including services domain names.

There are many types of services, each has things they are better for:

• NodePort: this actually blocks a specific port in the node and lets traffic reach the service through the node IP directly.
• ClusterIP: this lets the cluster assign an IP to the service from its services IP pool (more about it in a future post)
• LoadBalancer: this will actually use the cloud controller manager to create a load balancer on the infrastructure provided by the cloud provider (assuming you are running in a hyperscaler environment).

Let’s put it all together!

As you can see, we can access the workload from outside the cluster. The Service object is quite simple and does not allow for complex traffic rules. There are other objects that we can use for that purpose. A popular option is the Ingress object. As the name implies, it offers desired state rules around ingress traffic and routes incoming traffic into the deployment/service. We will talk more about it in a later post.

We just scratched the surface! There is a lot more to explore; grab the sample deployment and service objects from this blog post and fire up your Kubernetes Cluster! If you still don’t have an installation, feel free to use O’Reilly’s Katacoda interactive playground. Be curious! Change the service type. How does it behave? Change the desired number of replicas of an existing deployment. Modify the NGINX welcome page and see if you can route into two different versions of the application. Use the force, my fellow Jedi apprentice!

Now we have a much more robust workload, but we can still make it a lot better! In the next post, we will look at some of the key use-cases this enables. We will look at adjacent concepts, like auto-healing, auto-scaling; and how they play a part in creating resilient workloads in kubernetes.

Follow me on Twitter (@fawix) or LinkedIn so you won’t miss the update! May the Containers be with you!

Kubernetes for dummies: deployments and services was originally published in ITNEXT on Medium, where people are continuing the conversation by highlighting and responding to this story.

Part 3 — Pods, and that other thing?

Welcome back, young Padawan! You now understand the components in a k8s cluster (part 2). Today we take a look at one of our most basic constructs: PODs. We will see how they are involved when we are running our applications.

Pod #WTF

Pods are one of the most fundamental building blocks within Kubernetes. They are the smallest deployable object. Their goal is to provide a runtime environment for your containers. Although Pods need other Objects to make them truly useful, they are the piece that represents your application within Kubernetes. Several other Objects will build upon the capabilities offered by the Pod. Today we will focus on how they run our workloads. In the next post, we will look at those other objects that interact with PODs to make them more robust and versatile.

Maybe you were expecting me to say that the smallest Kubernetes object is the container? Not so, young Padawan! Every running container in a k8s cluster is inside a Pod. Effectively you will never have containers running by themselves.

The Pod provides a running environment for the containers, including networking and storage. A Pod can have multiple containers if necessary. Containers in the same Pod are tightly coupled and share resources (networking and storage, security rules, etc. ).

Conceptually, a Pod looks like the following:

There is only one IP assigned to a Pod, regardless of the number of containers it has. Meaning, inside a Pod, the containers share the same network namespace (IP and Ports). The consequence is that containers within a Pod can communicate through localhost (127.0.0.1). Optionally a Pod can also specify one or more storage methods, referred to as Volumes. All containers in the Pod can access mounted volumes. Pods can share storage as well, but that is for another post.

Fun fact, the name Pod comes from the collective for whales. It alludes to the containers inside it, which predominantly ran with dockerd (Docker logo is a whale!).

Of course, now we’ll look at what an actual object looks like:

These are the mandatory fields required to stand up a Pod. As we advance, we will look at other Objects that use the Pods to build more capabilities and then expand on other fields.

One important thing to notice above is the field spec.containers, which is an array of containers that you can define. The image field is the name of the actual container with its repository (when omitted, it defaults to DockerHub) and a tag. Container tagging is simply a way to manage your versions and releases. With this, you can target a specific version of the container. In the example above, we’re using the nginx container with the stable-alpine tag in DockerHub (because we omitted the repository bit).

Not all containers are created equal

When we talk about containers, the first thing that comes to mind is the containers running our applications. Typically a long-running process that will stay up until we kill it or something goes wrong. However, I quickly want to mention two other types of container in a Pod and what you should use them for:

Init Containers: as the name suggests, init containers are good for initialization and setup. They run before the regular containers in the pod get started. Unlike the regular containers, these should always run to completion and exit successfully or the Pod will enter the Failed state (more on that below).

Ephemeral Containers: it’s a fairly new (alpha in 1.22) type of container. It makes no guarantees for resources or execution, and they are never automatically restarted (don’t use these to build applications). These are useful for troubleshooting and interactive shells for short-lived operations.

For consistency, anytime I talk about “containers” that is your regular workload container. I’ll specifically call out the other two types when we are talking about them.

Life of a Pod

As you can see, Pods provides that fundamental piece we need to run our containers. Now let us look at how a Pod comes to life and eventually dies: The pod Lifecycle.

It’s simple to conceptualize at a high level:

First, we create the Pod. In this stage, Kubernetes fetches the container from the registry. Then k8s ask a node to run the Pod; which causes the creation of necessary constructs around networking and storage. Then it runs the container in that space. If fetching the container fails, the Pod will assume the “UNKNOWN” state.

The process in the container will be in the foreground until completion or until an external command terminates it. This is the Running state. If the Pod fails the state is set to FAILED. If it runs successfully, it assumes the status of TERMINATED.

Ok, let’s create a POD! Grab that example object above and save it as pod.yaml if you want to follow along, young Padawan! PS: if you don’t have a k8s installation yet, feel free to use O’Reilly’s Katacoda interactive playground.

For this, we’ll simply run kubectl create -f pod.yaml

Remember the components we described in part two? You can already yield the force, young Padawan. What do you think is happening under the hood? Think a bit about it before you continue reading for maximum experience points!

Ok, ready!?

Let’s check the details of what is happening under the hood when we ran that command:

1. The user runs a kubectl create command to create the object
2. kubectl makes a call to the API Server to create the object
3. API Server saves the Object in etcd
4. Scheduler detects a new Pod was created and is not assigned to a node
5. The scheduler then updates the Pod object with the chosen node. It takes into account several constraints to define which node gets the Pod. These are labels and taints that define affinity (run groups of pods together) or anti-affinity (avoid scheduling pods together) rules.
6. kube-controller-manager then detects that the Pod desired state doesn’t match the current state and makes the necessary calls to the API Server (it’s a bi-directional flow); at this point, the correct controller will respond to do these changes
7. Which in turn triggers an API call from the API Server to kubelet, in the selected node, to create the actual Pod
8. As part of setting up the Pod, networking kube-proxy does the required changes to the OS via the package filtering layer

With that, your pod is created! Any object creation works like this involving different controllers as needed.

At this point, you can only access the workload via local networking. This means, you need to be connected to the Pod:

If you created our Pod, you should spend some time playing with it. Check what happens when you run kubectl describe pod nginx, or try sending a SIGKILL to the nginx process from inside the Pod. See what happens! Be curious!

Now, you may be thinking: I can’t run my apps with localhost only access!? Correct you are! This is less than ideal for workloads, and we will see how to solve that in part 4 using services. However, at this moment, you effectively have your workload running in kubernetes! One thing to be aware of, once the Pod gets killed, or runs into a failure, it will simply sit in the terminated or failed state and will not be restarted. That is something deployments solves, and we will see that in the next post.

Follow me on Twitter (@fawix) or LinkedIn so you won’t miss the update! May the Containers be with you!

Kubernetes for dummies: Life of a Pod was originally published in ITNEXT on Medium, where people are continuing the conversation by highlighting and responding to this story.

Part 2 — A look under the hood

As promised today we’ll be deep-diving into the architecture of a Kubernetes cluster and see how it all comes together! This is foundational to understanding how k8s runs workloads.

Let’s look inside the Control Plane first! As we already established this is the brain of our cluster. A robust cluster will have multiple Control Plane nodes, preferably in different geographical locations. This is usually taken care of by the cloud provider if you are running in the cloud. For example, In GCP a GKE regional cluster has three control plane nodes in distinct zones to guarantee high availability. The Control Plane is also the entry point when “Interact with the cluster you have” as foretold by the Jedi Master.

Control Plane — Under the Hood

Let’s peek inside! The control plane runs five major components as shown in the diagram below:

API Server: is a process running the Kubernetes API, which follows the REST design pattern. It’s well thought out and implemented. It lets you query and modify the state of objects within Kubernetes. Think of objects as things with a state (normally); entities such as Pods, Nodes, etc (more below). The user documentation is here and is also fully documented using OpenAPI! ~The source, Luke!~

Controller Manager: several things within Kubernetes require comparing the current state with the desired state. If there are any deviations the controller manager will execute a set of actions to correct them. Kubernetes employs the control-loop pattern for various “Controller” implementations (more below). Essentially, this means that the process is running an infinite loop and executing the checks. Pretty much everything in Kubernetes that requires an action is a controller. Kubernetes also supports custom controllers, so you could even write your own if you need to. Important examples of controllers are the Deployment Controller, Job Controller and others.

Cloud Controller Manager: based on the above, you can pretty much guess this one. It’s a controller manager for cloud provider-specific implementations. There are actions in Kubernetes that depend on the environment k8s is running in. For example, say you want to expose your application to the internet. Typically, this requires a Load Balancer which is very platform-dependent and will change whether you are running on GCP, AWS or Azure. Each cloud provider that wants to enable Kubernetes to consume their cloud resources has to provide a basic set of controllers that teach Kubernetes how to use their resources.

Scheduler: also called kube-scheduler and, it has a super important duty: assign Pods to worker nodes. More about it when we talk about Pods, but essentially this is the piece that will guarantee your pods have a home.

etcd: the cluster storage/database — this is where Kubernetes objects are stored. etcd is a distributed key-value store that prioritizes consistency and availability (from CAP theorem). Initially designed by the CoreOS team and then adopted into the broader community, and is currently part of the CNCF landscape.

These five components enable k8s to interact with the system resources and provide a solid platform for your containers. We will explore the new terms and concepts in future posts — for now, focus on understanding each component and its role within the Control Plane.

Kubernetes Objects #WTF

Kubernetes represents its state using objects. It usually describes entities within the cluster (like the worker nodes, pods, etc). Objects contain the entity’s basic information and its desired state. There are several “kinds” of objects and, most objects have a basic set of keys that are always present. That help us Jedi apprentices to navigate the ways of the Containers more easily!

We can interact with the cluster using a handy CLI called kubectl. It mainly talks to the API Server and allows us to do several tasks within the cluster. The API Server accepts JSON only, but kubectl also understands YAML. Ideally, it allows for a simplified human interface (YMMV).

Here is what a basic structure looks like in YAML:

The structure above is pretty much self-explanatory and, we will expand more on these concepts as we advance. The one thing I want to mention is the difference between labels and annotations. Labels are used for organisation and grouping. Annotations add functionality to the object. Typically the latter is provided by applications or tools deployed in the cluster. If you are familiar with the idea of decoration functions this is a very similar concept.

Control Loop #WTF

I like to think of Kubernetes as a framework with a set of actions needed to run workloads in containers. When we create an object, we are using that framework to achieve that goal. For example, we can add a worker node in the cluster to run our containers.

Kubernetes will then use the information about the entity to verify its current state (status) against the desired state (spec) and take corrective action if needed. Something like this:

This construct is called a control loop and is a fundamental principle in how kubernetes operate.

Node — Under the Hood

Now we need a place to run our applications and workloads — that is the main job of the Worker Nodes. It isn’t a good idea to run workloads in the control plane node. In fact, k8s prevents that from happening by default using a mechanism called taint. Fear not padawan, we will explore that later. For now, this means that we need worker nodes in the cluster to run your application.

A worker node is essentially a VM (compute) that has two Kubernetes processes (kubelet and kube-proxy), and a container runtime (see part 1). Before anything can happen, the node has to join the cluster so the control plane knows it can run your workloads. Kubernetes control plane will then manage it and give it instructions using the Kubernetes processes (such as running a workload).

When a node joins the cluster (kubelet says hi to the API Server) their state is tracked by an object that represents the node. The Nodes objects look a bit like the below:

I redacted values for readability, but you get the idea. All the metadata regarding the node and its health is in that object.

Typically new nodes will self-register with the cluster. This communication happens between kubelet in the node and the API Server in the control plane. You also have the option to register nodes manually if necessary, which you might need to do when you are creating your cluster outside of cloud providers. A neat utility for that purpose is the kubeadm. It facilitates the creation and exchange of bootstrap tokens, used by the API Server to know that the call comes from a legit node trying to join the cluster (oversimplification). Let me know if you are interested in a deep dive into the actual bootstrap process!

Using kubectl we can ask the cluster for the state of our nodes. We can do some basic manipulations like drain traffic from a node, make it unschedulable (so scheduler won’t assign workloads to it), get its details etc. Here is what listing the nodes in a cluster looks like:

The more you know… the more there is to learn!

Want to get some hands-on-keyboard time? There are two really easy ways you can get started:

1. Use the free tier and credits on GCP and create a GKE cluster that you can play with: https://cloud.google.com/free
2. Play with the O’Reilly Katacoda platform.

A benefit of option 1 is that you can re-use the same cluster to follow along on future posts. If you choose option 2, I recommend checking out the “Getting Started with Kubeadm” course. If you want something faster check out “Launch a Single Node with Minikube” instead. Spend some time querying the nodes and checking its spec and status. Run a kubectl drain <node name> and see how it’s reflected on the node details. Be curious!

So now you know the inner pieces of a cluster, but how does it work together? The next post will deploy a very simple workload to understand that. We will check what a pod consists of, its lifecycle, how it relates to containers and all that good stuff. Follow me on Twitter (@fawix) or LinkedIn so you won’t miss the update! May the Containers be with you!

PS: Optional hands-on time coming up in next post!

Kubernetes for dummies: introduction

Part 1 — What is kubernetes after all?

Cheers to another successful Kubecon! As I celebrated how far this community has come, I started thinking about when I first met Mr k8s (Kubernetes).

I have been playing with Kubernetes for a while (since 2016) — minikube was in its baby steps. Back then and there weren’t a lot of good resources out there. Today learners are in a much better place with several good resources at hand — even so, I realized that we still have things we could improve after giving a few talks last year.

Fear not, young padawan! With this blog series, I am to give you enough information to start wielding the force. So you can make your way to a Jedi Master by deepening your understanding of the unseen. (Disclaimer: as a fellow disciple, I will share what I know).

Here we will start from zero. In this series, I plan to touch on key concepts for understanding Kubernetes. We will also talk about some of the adjacent technologies and patterns. First, let’s talk about what Kubernetes is and what it’s not.

Kubernetes is:

• Platform (or as people like to say: “it’s an Orchestration Layer”)
• Distributed system framework (when done right)
• Extensible & Resilient
• Declarative
• Open Source

Kubernetes is not:

• Simple ( … goes without swaying … *pun intended*)
• Hypervisor (but it will use them)
• Containers (it allows you to run them)

Also, note that there are several cool things that Kubernetes enables you to do. For example, when combined with the correct adjacent tools and processes using canary releases, auto-scaling, auto-healing, becomes possible. We will talk about these in future posts.

In the realm of IaaS, PaaS, SaaS, Kubernetes classifies as a hybrid. This is because when you are using Kubernetes from hyperscalers (aka “the cloud”) there are some responsibilities that fall under the cloud provider (infrastructure) and others that fall under the customer (workloads). Running a Kubernetes cluster well is difficult since it’s a complex platform with security and operational concerns. If there is interest to explore this topic further let me know.

Containers #WTF

We can’t talk about Kubernetes without talking about containers. Back in the day, we used to run stuff on servers directly. You usually installed the Operating System (OS) then added your libraries and your software et voila. Today we call that running your applications on bare-metal. Sometimes the business needed to cut costs, so multiple applications had to share the same server — wasn’t pretty.

Applications would often clash and compete for resources, to solve that problem (among other things) someone thought about sharing the hardware but enforcing isolation with multiple operating systems. Thus hypervisors became a thing giving rise to virtual machines. Further down the road, we ran into dependency management issues. “But it runs on my machine” was the slogan Docker used to come out into the world making containers more mainstream. These would share the kernel but package their dependencies and application runtime creating a little ~container~ inside the machine. These can be deployed in a VM or Bare-metal, the container won’t know the difference.

Alas, sometimes a picture is worth a thousand words:

Containers use Linux features such as namespaces and cgroups to achieve a certain level of separation in compute, storage and network and to limit resource usage. To run a container you need to build them as defined by the OCI Specification. Then you need a container runtime — this is where dockerd and containerd fit into the picture.

When you build the containers you have to tell the packaging software what to include and any configurations to perform. Normally that is done via a Dockerfile and the image utility that can package and push it to a registry — recently alternatives such as buildpacks are also available but Dockerfiles remain the most widely used option. A Registry is a sort of online catalogue for containers like DockerHub.

After your container is packaged and ready to roll Kubernetes get into the scene.

Microservices #WTF

At the same time that this was all happening with the way we run our applications, people started realizing that separations of concerns are a great thing to have! Heck, in 2008 I was designing systems that would be considered “microservices”, however that wasn’t a term back then.

Conceptually both Monoliths and Microservices are valid systems design patterns and both have their place with pros/cons. The Monolith’s served the bare-metal / VM era well since it was all packaged as a unit and meant to run in the same machine. Microservices on the other hand segment all the logical services into actual packages achieving a certain degree of separation of concerns which serves the container era pretty well. Nonetheless, don’t be fooled, everything can be misused — as monoliths have been historically abused so can microservices… on the other hand microservices solve many of the monoliths issues, but it comes with its own bundle of challenges. Remember, both patterns are equally valid but we will focus on Microservices here.

One such challenge is how do you efficiently and reliably run several containers in a large-scale production environment? Things like resiliency, observability, upgradability are top concerns. Those are the problems Kubernetes set out to solve.

Kubernetes #WTF

If you are thinking of Kubernetes (or k8s for short) it’s likely because you want to provide a production-grade runtime for your containers as we just talked about above. To simplify maintenance and operational responsibilities, etc.

Kubernetes will take care of the shared infrastructure for your containers. For a very high-level view think of a k8s cluster as composed of control plane nodes and worker nodes. For high availability, the control plane runs on multiple nodes.

A control plane is the brain of the cluster and it will manage the worker nodes accordingly; there are key components within the control plane that are responsible for different aspects of the lifecycle of a node and we will talk more about those in a future post.

The nodes are typically responsible for running the workloads and there are several constructs around a container to enable Kubernetes to run it. One such construct is a POD which we will deep dive into soon.

Stay tuned for the next post where we will look under the hood for the control plane and nodes and see how that works. Follow me on Twitter (@fawix) or LinkedIn so you won’t miss the update! May the containers be with you!

Sometime ago we published a few posts about creating a CICD — today we decided to show a bit more about how you make that config more secure by hiding your secrets. If you seen our previous posts on this topic you know how to use HDI to deploy changes to your HANA, and how it can integrate with an existing cloud native workflow all the while ensuring that your changes are properly tested using a CICD pipeline; this is all really cool and will speed up our development cycle, but something is still missing for us to be able to put that beauty into prod!

Hiding sensitive information

If you have been wondering how you can protect your pipeline now that VCAP is right there with all the juicy information to do things to your HDI containers then I have good news for you. We can protect that information while retaining the same functionality.

Secrets … who doesn’t have them? Certainly not HANA (and many other applications for that matter); in the software world secrets are any information you want to keep private — if leaked or obtained by unauthorized parties it could offer them access that can potentially be used to compromise or damage your system.

If in doubt whether something should be a secret or not ask yourself, what would happen if a malicious entity got this information? Well, if that happened to VCAP … you’ll get the idea!

We will use Google Secret Manager to secure the VCAP variable, and then we will change our Cloud Build pipeline to securely consume that secret. The Google Secret Manager is a managed solution that allows us to easily create and consume secrets; but this method would work with other secret engines too.

Creating the secret

First make sure you have the correct information for VCAP services with the details for the user that is dedicated to the CI/CD pipeline, which is the same you created previously using the hanacli tool (“Creating the HDI container”).

If you no longer have the json file with the contents of VCAP services we can recover it from the trigger — it’s there in plain text after all… and that is the problem we are going to solve.

gcloud config set project ${PROJECT_ID} # make sure to use the correct PROJECT ID
cd app_hana/db
touch credentials.json
edit credentials.json

Now, navigate to Cloud Build triggers page, where you have populated the environment variables and recover the value in the _CREDS and then paste it on the file we just created:

If you rather obtain that value from the comfort of your command line do the following:

gcloud components install alpha
gcloud alpha builds triggers describe deploy-app-hana-db --format="value(substitutions)"

This will print the value of _CREDS environment variable so you can just copy and paste in the file (making sure to only get the value) as shown in the screenshot above, if you changed the name of the triggers when you created them make sure to use the correct name above.

Our next step is to create the secret (you’ll need roles/secretmanager.admin IAM permission):

gcloud services enable secretmanager.googleapis.com
gcloud secrets create app-hana-dev-vcap --data-file=./credentials.json --labels env=dev,app=app-hana-db,usage=cicd --replication-policy automatic
gcloud secrets list

Each environment should have a different database and therefore a different VCAP value, for that reason we are not only appending the environment in the name but also adding a few labels to help us catalog and retrieve data in the future. Labels are really useful, so be sure to think about the relevant labels for your use-case.

With that … our secret is born!

Updating the Cloud Build Pipeline

Time to modify our CI/CD pipeline to use our app-hana-dev-vcap secret!

edit cloudbuild.yaml

This change is fairly straightforward, we will add a new step in our YAML file that tells Cloud Build how to get the secret and make it available to other steps as a file and once the pipeline is destroyed the file is destroyed with it.

We will accomplish that using the following gcloud command:

gcloud secrets versions access latest --secret "${_CREDS}" --format='get(payload.data)' | base64 -d >> credentials.json && exit $?

We also need to modify the next step to waitFor the secret step to finish before it starts and to use the file we just created instead to obtain the credentials to connect to the HDI container. Here is what the final cloudbuild.yaml looks like:

Now we need to update the trigger, head to the trigger page on cloud build and edit it, now replace the value on _CREDS with the name of our secret (in this case app-hana-dev-vcap).

Before this can work, however, we still need to tell GCP that the Cloud Build pipeline can access this secret, for that navigate to the IAM & Admin page then find the service account that ends with @cloudbuild.gserviceaccount.com and give it the Secret Manager Secret Accessor role.

Now let’s test our pipeline! Push a change and see the magic happen:

You are done! Enjoy!

If this was useful to you let me or Lucia Subatin know!

SAP on Google Cloud (pt. 4): Secrets in your pipes was originally published in Google Cloud - Community on Medium, where people are continuing the conversation by highlighting and responding to this story.

In this post, we will continue into our CI/CD journey. We have already explored the main concepts (part 1 — including what is CI/CD) and ideas behind our sample application and the design principles for the pipeline when HANA is involved (part 2). Now we will show you what the plumbing looks like for one of the microservices pipelines.

The image above shows the CI/CD flow we will implement. When the app developer commits the code, it will be picked up by a trigger in Cloud Build. It will then build, test and deploy our application. Let’s focus initially on a single environment pipeline since expanding it to multiple environments later is quite simple.

Demo app recap

Before we dive into the pipeline, let’s quickly recap what our application looks like:

As Lucia Subatin explained in part 2, the application has four major components:

1. Frontend: web interface, allows the user access to the application.
2. Backend: translation service
3. DAL: the data access layer
4. Database: SAP HANA — stores our data.

The diagram above shows how these components interact with each other. For more details, check out part 2 of this series.

Where do I even start?

Now you might be thinking: That is great! Where do we start? How do I pick the first microservice to become CI/CD’ed? (yes, I just invented a word… Poetic License FTW!!!)

When first starting a CI/CD pipeline for an existing application, it might be a challenge to know how to begin dissecting the beast. Look for the following traits in a service:

• Fairly independent, meaning it doesn’t have to call a lot of other services within the same application (it’s ok to be a dependency for other services)
• A service that has good test coverage for the unit, functional and integration tests (yes — they are all important)
• Has a relatively low overall complexity

Take a peek at the diagram above again, which service do you think is the most adequate to be the first? Take a guess!

We are starting with the backend service since it’s quite isolated and well tested. When automating deployments, always try to improve your tests to ensure the reliability of your application — your future self will thank you.

Build Containers

The backend depends on the Google Translate API SDK for Golang; this means we would always need to install that SDK with a go get before can build. We also want to run go test and go vet as part of our test process.

Pipeline steps in Cloud Build run inside containers; this means we can simplify our build by creating a custom container with all the tools we need. Let’s go ahead and create a build container for the backend and store it in the container registry for later use within the pipeline.

Why bother, you ask? Eventually, you will want to run 10’s or even 100’s of builds a day — so those few seconds get magnified. This drives the needs to make builds as fast as possible. A build container allows us to skip the download and install step for build dependencies — saving time.

Here is what our backend build container Dockerfile looks like:

# build stage
FROM golang:alpine
RUN apk — no-cache add build-base git bzr gcc
RUN go get -u cloud.google.com/go/translate

ENTRYPOINT [ “/bin/ash” ]

Bonus tip: you may decrease the build time even further by implementing Kaniko Cache.

The Pipeline Configuration

Now that we have the environment, let’s check what the pipeline looks like:

For master branch we will:

1. Run static tests using go vet
2. Run unit tests go test
3. Run functional tests (also go test)
4. Build container (docker)
5. Push container to the registry (docker)
6. Deploy Container to the environment (gcloud)
7. Run integration tests (go test)

The Cloud Build pipeline is configured via a YAML file called cloudbuild.yaml as follows:

Of course, to implement the pipeline for other technologies the details will change, however, the overall flow will remain the same. For example, when testing the DAL layer using NodeJS the command will be npm test instead of go test and so on.

Some important considerations here are:

1. Each microservice has an individual pipeline (cloudbuild.yaml)
2. Each microservice will generate its artefacts independently
3. Each microservice should test the integration with its dependencies.

Remember this image Lucia explained in part 2?

Look at the cloudbuild.yaml closely and you will see a dir: translate on each step that points to the backend microservice and thus this will only run when we make changes in that folder. There are many ways of doing this so pick the version that best suits your release and development needs. Also, notice how the cloudbuild.yaml file sits inside the folder for its relevant microservice.

Creating the Trigger

It’s very straightforward to create a trigger in Cloud Build, once we have the YAML file created:

Notice that we configured the glob pattern to match only changes applied inside our microservice folder in the case of our backend it’s translate/**. This prevents this trigger from going off if we commit in other folders in the repo.

Also see how our regex is matching a branch named master, for non-master branches you can check the “Invert Regex” checkbox. Finally the Cloud Build configuration file also points specifically for our microservice pipeline definition with translate/cloudbuild.yaml

By now you can pretty much visualize how you would create the pipeline for the other microservices too!

To run your pipeline commit to your repository in the correct folder, you also have the option to run it manually. The build logs can be viewed by going to the Console and looking at the build execution like so:

The pipeline for HANA artefacts

The pipeline for deploying the HANA artefacts is mostly the same, but different … but still mostly the same.

As you saw in part 2, this pipeline relies on an HDI container and a special build container with the correct tooling to allow us to connect and deploy to the instance. The actual cloudbuild.yaml file is rather simple:

Notice how it has a single step (for now). It mostly relies on the prepared build environment and tools likehana-cli and HDI connection context via the environment. All that is accomplished by creating a specific build container (much like our backend build container).

This is what the build container Dockerfile looks like:

As a side note, this Dockerfile is not optimized and you may want to tweak it a bit before using.

Notice how we set up the container with the required tooling to connect to HANA via HDI. You can build this container in the same way we created our backend build container. This step is important because it essentially allows us to be ready to connect and deploy our db-things via HDI.

Another important difference is how the npm start command is configured. We added the exit flag, so it won’t wait for a human to press Ctrl+C, without that the pipeline would hang and timeout.

These are the main differences between a regular microservice pipeline and the HANA deployment pipeline using HDI containers.

Now you should be able to build your very own pipeline for your microservices and your HANA artefacts! Enjoy!

Lucia Subatin and Fatima Silveira.

SAP on Google Cloud: Creating the CI/CD Pipeline (pt. 3) was originally published in Google Cloud - Community on Medium, where people are continuing the conversation by highlighting and responding to this story.

Learn how to easily switch contexts with gcloud.

Do you want to know how to work with multiple accounts and organizations in gcloud while keeping your sanity? Let me share with you a little known feature in gcloud that does just that.

gcloud config configurations

There is a sub-module in gcloud config that allows you to group a set of configurations and give it a name. You can then activate it on-demand! This means it becomes super easy to switch between different accounts or organizations whenever you need. In other words, context switching.

This is how you use it:

gcloud config configurations activate config-name

Switching between configurations is very simple and it carries all the information you set when you created it — this means that you don’t have to login again even if you are using different credentials. Simply type the command above et voilà.

Before we get into details on how to create them, let me give you a few examples when this is useful:

• Switching between environments with distinct regions and zones.
• Switching between different credentials, especially for testing.
• Switching between completely different organizations.

Notice on the screenshot above that I have accounts for different organizations, projects, compute zone, and compute region — and these are only a subset of possible settings.

Of course to use a configuration you must first create it. This step requires a bit of typing but it’s a one-time effort, and will save a lot of time in the long run. This is how you create a new configuration:

Give the config a memorable name, then add the project id and the account at a minimum. I typically also set the compute region and zone since I work with these a lot. For development, you may also want to add a service account.

By the way, did you notice I didn’t have to log in when creating the config above? That is because that account was already in use by another configuration. Yes, that easy!

If you do need to add a new account just invoke the following:

gcloud auth login

Using this trick will save you a lot of time and make your experience in gcloud better! It’s well worth to learn and configure if you work with multiple contexts;

Ready to create your first configuration? Let me know how it goes!

How to use multiple accounts with gcloud was originally published in Google Cloud - Community on Medium, where people are continuing the conversation by highlighting and responding to this story.