<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Michael Levan on Medium]]></title>
        <description><![CDATA[Stories by Michael Levan on Medium]]></description>
        <link>https://medium.com/@michael-levan?source=rss-8fa8e88f2ad6------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*2k4oU5MOd8m03lOIvajwfw.jpeg</url>
            <title>Stories by Michael Levan on Medium</title>
            <link>https://medium.com/@michael-levan?source=rss-8fa8e88f2ad6------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 13:03:40 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@michael-levan/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Inside Kagent: Architecting Open Source and Enterprise AI Agent Meshes for 2026]]></title>
            <link>https://michael-levan.medium.com/inside-kagent-architecting-open-source-and-enterprise-ai-agent-meshes-for-2026-4cd42ea6da78?source=rss-8fa8e88f2ad6------2</link>
            <guid isPermaLink="false">https://medium.com/p/4cd42ea6da78</guid>
            <category><![CDATA[mcp-server]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <dc:creator><![CDATA[Michael Levan]]></dc:creator>
            <pubDate>Wed, 19 Nov 2025 16:51:36 GMT</pubDate>
            <atom:updated>2025-11-19T18:22:29.946Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*taeIC-9qFg3oHG2yg3Qjcg.png" /></figure><p>AI has transformed from a buzzword to chatbots that help you figure out the perfect family cooking recipe to agentic infrastructure running in some of the largest organizations in the world. Because of that, AI-driven business value in 2026 will depend on the flexibility, scalability, and resilience of your agent infrastructure, networking, security, and overall observability within your environment.</p><p>With organizations implementing and betting on AI, whether for automation, internal-specific agents to perform a specific task, or to enhance the DevEx/customer experience, mesh-based solutions like kagent are reshaping how enterprise teams build, connect, and secure distributed AI agents.</p><h3>Why kagent?</h3><p>Before jumping into the credibility, deployment scenarios, and enhanced capabilities that kagent gives you, let’s talk about the “why” behind it, because there are some great questions that come out of the “why”.</p><ul><li>Why yet another agentic framework?</li><li>Why run kagent when chatbots and AI consoles exist?</li><li>Why not just use AI that’s built into whatever tool/platform (like a cloud provider) is already being used?</li></ul><p>The answers are flexibility, control, governance, and declarative deployment options.</p><p>Taking a step back, what do the current agentic-based frameworks give you right now? They give you the ability to write an agent in Python or JS (those are the typical general-purpose programming languages used), which involves you writing multiple clients (because you’ll definitely need more than one agent), multiple servers (because those agents need a server so others can connect to them), and connections to MCP servers. Once the clients/servers are built, you now have to create your own security (e.g. libraries/packages within your agent code to secure traffic between A2A, agent to MCP, and to various LLMs), observability (you need to create/expose your own metrics endpoints), and figure out where these clients/servers are even going to run with close to 100% uptime.</p><p>💡It’s important to note that there isn’t anything wrong with agent frameworks. In fact, kagent supports the most popular ones like CrewAI and Agentic Development Kit (ADK) and kagent is built on top of kagent. The biggest question is: Do you want to set up several clients, servers, and roll your own observability, governance and security?</p><p>Kagent with the help of agentgateway and service mesh (Istio Ambient Mesh) resolves all of these concerns for you. Kagent is hosted on Kubernetes and you can either create agents/connections to MCP servers on the UI or do it in a declarative fashion much like any other Kubernetes workload deployment. Kagent interacts directly with agentgateway helping with the networking, observability and security layer (think rate limiting, prompt guards, etc.), and Istio helps secure the traffic (and shows you observability for it) between agents.</p><p>Sidenote: if you want to create MCP servers to connect to, that’s where <a href="https://kagent.dev/docs/kmcp">kmcp</a> comes into play.</p><h3>What is an Agent Mesh?</h3><p>Now that you know the stack (<a href="https://kagent.dev/">kagent</a>, <a href="https://agentgateway.dev/">agentgateway</a>, and <a href="https://ambientmesh.io/">Istio</a>), let’s talk about what it is all together — Agent Mesh.</p><p>An <strong>agent mesh</strong> is a transparent infrastructure layer that manages how autonomous agents (Chatbots, agents, microservices, ML workloads, etc.) communicate, discover each other, and enforce security within complex, distributed environments.</p><p>💡 Think about true distributed systems — the ability to connect, secure, implement HA, and observe across any cluster. Now take that same concept and embed it into Agent protocols.</p><p>Unlike traditional point-to-point APIs or message queues, an Agent Mesh abstracts away the operational burden which helps with handling connectivity, observability, policy, and reliability so engineers in all specialties (Programming, DevOps, Platform Engineering, Security Engineer, etc.) can focus on business logic and AI innovation.</p><p>The key thing to remember is Agent Mesh isn’t a concept that hasn’t been vetted out already in the cloud-native space. It’s inspired by<a href="https://www.solo.io/resources/resource-kit/service-mesh"> service mesh architecture</a>, which has become standard for microservices and Kubernetes workloads.</p><p>Key pieces to the puzzle include:</p><ul><li><strong><em>Discovery and Registration:</em></strong>* Agents can join, leave, or move without manual reconfiguration.</li><li><strong><em>Traffic Management:</em></strong>* Intelligent routing, load balancing, and failover for agent-to-agent calls (L4-L8)</li><li><strong><em>Security:</em></strong>* Automated mTLS encryption, authentication, and fine-grained authorization.</li><li><strong><em>Observability:</em></strong>* Unified metrics, logs, and tracing across agent interactions.</li></ul><p>This approach empowers both open source and enterprise teams to compose, scale, and govern agent-driven systems — without the complexity and risk of DIY networking, security, and discovery.</p><p>One last thing to leave you with before moving on is you may have noticed in the <strong>Traffic Management</strong> bullet point that it says “L8”. With Agent Mesh, an important realization came out — what about the context layer? Agentic Workloads are all about semantic reasoning, which is different form the traditional/syntactic approach. With semantic workflows, a person/agent/bot is asking multiple questions to an Agent and getting a responses in a collected format. What that translates to is any time you ask an LLM several questions via an Agent or an AI dashboard (think Antropics UI, ChatGPT, Geminis UI, etc.), it’s looking at the questions and saying to itself “Are these related?”, and the response you get is based on the decision that was made (e.g — are all of these, in fact, related?). The way that the whole process is done from a semantic perspective is that it converts the data to prompt embeddings (e.g — Vector). The embeddings are how LLMs understand the information being passed in. It’s like how a computer understands binary.</p><h3>Why It Matters: Algorithmic and Market Shifts</h3><p>The landscape is changing at a drastic rate, probably faster than ever before. Think about it — the MCP concept hasn’t even been out 1 year yet and very big organizations are already implementing it into production. That’s VERY different in comparison to what it was before (the typical scenario was a new piece of tech would come out and enterprises would adopt it in 2–5 years). For 10+ years now, the cloud-native solutions were around microservices, stateless workloads, and network/API traffic based on headers. Now, with Agentic Workloads, we’re moving toward a stateful workflow with network traffic that’s based on the body of a call, not the headers. This is a very different concept than what engineers implementing cloud solutions are used to.</p><p>The move to distributed AI architectures powered by stateful/agentic workloads, Kubernetes, and clusters of intelligent agents is gathering pace. Gartner predicts that by 2026, 70% of enterprises will deploy AI agents at scale, up from just 10% in 2022. The challenge? Exactly what kagent fixes — Connecting, scaling, and securing these agents across disparate environments without stifling innovation or exposing the business to risk.</p><p>💡A recent<a href="https://www.solo.io/resources/resource-kit/service-mesh"> Solo.io executive summary</a> lays out the stakes: “Modern enterprises increasingly depend on microservices and cloud-native architectures to drive speed, scalability, and innovation. The efficiency and resilience of these architectures hinge on selecting the right communication patterns.” As Agents become both more numerous and mission-critical, the Agentic Mesh approach is already proven for service-to-service networking emerges as the backbone for next-generation AI infrastructure.</p><p>Here are a few big transitions and needs we’re seeing at Solo when working with customers:</p><ul><li><strong><em>Explosion of AI workloads:</em></strong>* As the cost of inference drops and AI capabilities grow, organizations are running thousands, or millions, of agents.</li><li><strong><em>Hybrid/Multicloud is the norm:</em></strong>* Agents span on-premises, cloud, and edge environments. Network complexity and policy drift threaten reliability.</li><li><strong><em>Agents On Kubernetes:</em></strong><em>:</em>* Engineers don’t want to change the way they’re deploying workloads already. They want an orchestrator that’s declarative and works the same way they’ve been deploying cloud-native workloads, which is Kubernetes (and why kagent exists).</li><li><strong><em>Security and compliance pressure:</em></strong>* AI systems process sensitive data; zero trust, auditability, and encryption are non-negotiable.</li><li><strong><em>Operational agility:</em></strong>* Teams want to deploy, upgrade, and secure agents rapidly — without bottlenecks or manual toil.</li><li><strong><em>Token/LLM Cost:</em></strong><em>:</em>* FinOps is being bigger and bigger within the LLM community as organizations are excited about Agents, but they have zero idea what it’s going to cost them (agentgateway exposes metrics to see token usage).</li></ul><p>Agent meshes, such as those enabled by service mesh technologies like Istio or<a href="https://www.solo.io/products/gloo-mesh"> Gloo Mesh</a>, address these shifts by centralizing control, automating security, and providing observability at scale.</p><h3>Implementation Of Kagent-Based Agent Mesh</h3><p>When implementing Agent Mesh, there will be a few key pieces to the puzzle:</p><ol><li>Service Mesh: In this case, it will be Istio Ambient Mesh to handle all of the L4 and L7 traffic.</li><li>Gateway: agentgateway enterprise to handle all things network traffic to LLMs/Agents/MCP Servers/A2A, observability, and security.</li><li>LLM key: An API key from your favorite AI provider (OpenAI, Anthropic, etc.).</li><li>Operators (Controllers and CRDs): Much like any other Kubernetes workload, CRDs extend the Kubernetes API and Controllers ensure that workloads stay running with reconcillation loops underneath the hood. The same needs will exist for Agentic workloads.</li></ol><p>Once the above configurations are accounted for, you’ll want to Define Your Agent Network Topology (e.g — map out what agents need to communicate, what MCP Servers need to be accessed, if Agents are running in other clusters), Onboard AI Agents, implement security and Zero Trust, and deploy observability (traces, logs, metrics).</p><p>Much like any other application stack/environment, engineering teams won’t have the ability to know what’s going on underneath the hood with observability. Agentgateway exposes several metrics out of the box including LLM and token usage. How agents communicate with each other or to MCP Servers will require network implementations, especially if Agents are running in different clusters or on-prem. For example, if you’re in an air-gapped environment, would the Agents that you’re deploying have internet connectivity to the LLM of your choosing? Last but certainly not least is the security to Agents, for Agents, and the connectivity to MCP Servers. With kagent and agentgateway, implementations like Prompt Guards (like policy enforcement, but for keywords/phrases in a prompt), Rate Limiting (e.g — only 1 token to an LLM every 100 seconds), and authentication to Agents and MCP Servers via system-level authentication like JWT.</p><p>With Agent Mesh, you not only have network, observability, and security at the L4, L7, and L8 layers, but you also have a full fault tolerant environment for Agentic Infrastructure. It’s the definition of managing your Agents just like every good enterprise would an application stack.</p><h3>Benchmarks &amp; Case Studies: AI Mesh in Action</h3><ul><li><strong><em>Case Study: Multi-Cluster AI Resilience in Production</em></strong>*</li></ul><p>A global fintech scaled its AI fraud detection mesh across 12 Kubernetes clusters using Gloo Mesh. Result: downtime fell by 70%, with automated failover and<a href="https://rafay.co/ai-and-cloud-native-blog/top-7-reasons-why-a-centralized-management-platform-is-key-to-overcoming-day-2-kubernetes-challenges"> centralized multi-cluster control</a> eliminating configuration drift and human error.</p><ul><li><strong><em>Case Study: Open Source Acceleration, Enterprise Guardrails</em></strong>*</li></ul><p>A healthcare AI provider adopted open source mesh for initial agent connectivity, then upgraded to enterprise controls for compliance (HIPAA, FIPS). Automation of<a href="https://docs.solo.io/gloo-mesh-enterprise/main/setup/prod/certs/istio/managed/cert-rotation/"> mTLS across clusters</a> and policy-driven governance cut audit prep time by 60% and enabled safe scaling into new markets.</p><ul><li><strong><em>Benchmark: Cost Optimization</em></strong>*</li></ul><p>By moving from per-agent proxies to a sidecarless (ambient) mesh, one e-commerce innovator<a href="https://www.solo.io/blog/how-ambient-mesh-delivers-advanced-resource-and-cost-savings"> reduced infrastructure spend by up to 90%</a> and improved developer velocity — onboarding new AI agents in minutes, not weeks.</p><h3>Pitfalls &amp; Avoidance Tips When Implementing Agentic Mesh</h3><ul><li><strong><em>Overcomplicating the Mesh:</em></strong>* Start with a minimal deployment; add advanced policies (e.g., traffic splitting, L7 filtering) only as needed.</li><li><strong><em>Ignoring Observability:</em></strong>* Without end-to-end tracing (<a href="https://www.solo.io/blog/opentelemetry-gloo-platform-graph">see OpenTelemetry integration</a>), debugging AI agent interactions is nearly impossible.</li><li><strong><em>Manual Certificate Management:</em></strong>* Automate<a href="https://docs.solo.io/gloo-mesh-enterprise/main/setup/prod/certs/istio/managed/cert-rotation/"> certificate rotation and mTLS enforcement</a> to avoid outages and security gaps.</li><li><strong><em>Neglecting Multi-Cluster Planning:</em></strong>* If you anticipate growth, set up<a href="https://docs.solo.io/gloo-mesh-enterprise/main/getting_started/multi/"> mesh federation</a> early to avoid migration pain.</li><li><strong><em>One-Size-Fits-All Policies:</em></strong>* Fine-tune resiliency and security settings for each workload; over-aggressive defaults can harm performance.</li></ul><h3>Conclusion</h3><p>As AI agents become the building blocks of digital transformation, the infrastructure for connecting, securing, and managing them determines business success. Kagent’s Agent Mesh approach, rooted in open source innovation and enterprise-grade control, unlocks the operational agility, resilience, and cost optimization required for 2026 and beyond.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4cd42ea6da78" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kubernetes, EKS, and AWS ECS — Is It All The Same Thing?]]></title>
            <link>https://michael-levan.medium.com/kubernetes-eks-and-aws-ecs-is-it-all-the-same-thing-4af0d34e9a55?source=rss-8fa8e88f2ad6------2</link>
            <guid isPermaLink="false">https://medium.com/p/4af0d34e9a55</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Michael Levan]]></dc:creator>
            <pubDate>Wed, 16 Nov 2022 12:53:32 GMT</pubDate>
            <atom:updated>2022-11-16T12:53:32.687Z</atom:updated>
            <content:encoded><![CDATA[<h3>Kubernetes, EKS, and AWS ECS — Is It All The Same Thing?</h3><p>The orchestration wars, which were between Kubernetes, Mesos, and Docker Swarm was a real thing. The whole idea behind orchestration and choosing how to manage containerized workloads was at the top of everyone’s mind. With how fast containerization is being adopted now, it was a necessary topic for every engineer to truly consider.</p><p>In the end, Kubernetes came out on top (which doesn’t mean the other solutions weren’t good because they very much are).</p><p>However, that doesn’t mean the war is officially over.</p><p>In this blog post, you’ll learn from a theoretical and hands-on perspective what Elastic Container Service is, what orchestration is, and how to implement both AWS EKS and ECS.</p><h3>What’s Orchestration?</h3><p>When most engineers or technical leaders think about orchestration, what do they usually think about? Kubernetes.</p><p>With that, it’s important to take a step back and actually think about what orchestration is.</p><p>Orchestration gives you the ability to:</p><ul><li>Scale containerized workloads.</li><li>Automatically schedule where containerized workloads should run.</li><li>Features like self-healing.</li></ul><p>Essentially, orchestration manages the “infrastructure” side of containers for you so you don’t have to.</p><p>Before orchestration, you could use containers, but they couldn’t really do much other than run one workload and exit. The reason why is that there was no guarantee that a container could run for a long time. It was very good at testing an application or running a workload and then shutting down the container, but that was about it. Then, once orchestration came along, you could have long-running containers running workloads. If a container went down, a new container would get created automatically.</p><p>Thinking about the above, you must ask yourself — is Kubernetes the only way to “orchestrate” containers?</p><p>Before wrapping up this section, let’s not forget that orchestration also means orchestrating workloads that aren’t containerized. For example, with HashiCorp Nomad, you can orchestrate workloads that are containerized and that aren’t containerized.</p><h3>What’s ECS?</h3><p>Elastic Container Service (ECS) is a way to automatically scale and optimize containerized workloads. If you want a simpler solution other than taking on the beast that is Kubernetes, ECS is a great middle-ground with very similar functionality, capabilities, and outcomes.</p><p>ECS scales based on load, performance, and ensures that containers are always up and running. You can either go the Serverless or EC2 route, which you’ll learn about coming up.</p><p>When creating an ECS cluster, you have three options:</p><ul><li>Network only</li><li>Linux-based</li><li>Windows-based</li></ul><p>The Network Only option deploys ECS using an AWS Fargate profile, which is essentially “Serverless”. It works for both Linux and Windows depending on your workload. What this means is that there are no EC2 instances running Linux or Windows that you have to manage.</p><p>Linux-based is ECS that has Linux EC2 instances running the ECS workloads. Windows-based is EC2 that has Windows Servers running the ECS workloads. Luckily, for both of these options, auto-scaling is enabled so you don’t have to worry about scaling up or scaling down the EC2 instances.</p><p>The two biggest differences are that Fargate can be a bit more expensive depending on your workload and Fargate does not currently support DaemonSets, which is a Pod that runs on each worker node.</p><h3>What’s EKS?</h3><p>Before jumping into EKS, let’s answer the question — What is Kubernetes?</p><p>Well, this answer can fill up several books in itself, so let’s keep it brief. As mentioned in the <strong>What’s Orchestration?</strong> section, Kubernetes is a way to schedule, self-heal, and manage containerized workloads. Yes, way more goes into it, but that’s the overall gist.</p><p>EKS is simply a way to run Kubernetes as a Managed Service. When you bootstrap Kubernetes with, for example, Kubeadm on-prem, there’s a lot that goes into it. The overall networking, infrastructure, operating systems, licenses, and management. With EKS, a lot of that is abstracted away from you. You don’t have to worry about managing the Control Planes. Instead, you just worry about the worker nodes which can run as EC2 instances or with Fargate.</p><p>EKS is a great solution if you’re in AWS already and want to run workloads in Kubernetes without having a Kubernetes cluster on-prem.</p><p>As with all cloud-based Managed Kubernetes Services, you do lose access to the Control Planes. Although that may not be a big deal for some, other organizations may want that control. It’s rare, but something to keep in mind.</p><h3>The Difference?</h3><p>As you read through the sections above, you probably thought to yourself <em>ECS and EKS, along with Kubernetes running in every other cloud appear to be doing the same thing.</em></p><p>The answer is yes. Functionally, orchestration is orchestration and at the end of the day, it’s doing one job — orchestrating containers. However, there are differences.</p><p>Let’s think about three key differences:</p><ul><li>How you can run workloads</li><li>Where you can run workloads</li><li>Overall support</li></ul><p>First, let’s talk about how you can run workloads. Although ECS is great for orchestration, at the time of writing this it only supports the Docker runtime. Although this is fine for almost all organizations, it’s still a hurdle that you’ll have to think about. If you want to use something like Podman, you won’t be able to. Because this is a specific AWS service, you will be limited to its capabilities.</p><p>Where you can run these workloads also matters a lot. ECS is dedicated to AWS. That means if you decide you don’t want to use it anymore, you have to think about moving to another solution. If you run Kubernetes in EKS and want to, for example, switch to AKS, it’s not a huge deal. Some Infrastructure-as-Code refactoring and perhaps where you’re pulling the container image from, but that’s about it. The good news is that if you use ECS, the workloads are already containerized. You just have to build for Kubernetes. It’s not like you’re going from applications running on bare metal to Kubernetes. Utilizing ECS, you’re halfway there. ECS is also a better solution for smaller teams because Kubernetes is a beast in itself and may not be needed right away.</p><p>Last but not least is the overall support. Kubernetes has an extremely large ecosystem with a lot of organizations utilizing it and a lot of engineers using it. Because of that, finding an answer to a problem with Kubernetes is probably going to be easier than finding an answer to a problem with ECS. At the same time, ECS is very popular in the AWS space and it’s heavily utilized, so finding solutions won’t be impossible. It’s just not as heavily utilized as Kubernetes overall.</p><h3>Configuring ECS and EKS</h3><p>Now that you’ve went through the theory, let’s take a look at how to configure ECS and EKS.</p><p>There’s a pro and a con to configuring EKS and ECS. The pro is that there are multiple ways to configure them. The con is that there are multiple ways to configure them (see what I did there?).</p><p>Because of that, let’s go the Terraform route.</p><p>If you don’t use Terraform, that’s okay. Do a quick Google search. Something like “how to configure EKS with X tool”. For example — “how to configure EKS with the AWS CLI”.</p><p>There’s a ton of information out there about different configuration methods.</p><h3>ECS</h3><p>First, specify the ECS server that you want to create. This will be considered a “cluster”.</p><pre>resource &quot;aws_ecs_cluster&quot; &quot;levancluster&quot; {<br>  name = &quot;levanecscluster&quot;</pre><pre>  setting {<br>    name  = &quot;containerInsights&quot;<br>    value = &quot;enabled&quot;<br>  }<br>}</pre><p>Next, specify the task definition. A task definition is the containerized workload that you want to run in ECS.</p><pre>resource &quot;aws_ecs_task_definition&quot; &quot;nginxapp&quot; {<br>  family                   = &quot;nginxapptask&quot;<br>  cpu                      = 1024<br>  memory                   = 2048</pre><pre>  container_definitions = &lt;&lt;DEFINITION<br>[<br>  {<br>    &quot;image&quot;: &quot;nginx:latest&quot;,<br>    &quot;name&quot;: &quot;nginx&quot;,<br>    &quot;networkMode&quot;: &quot;awsvpc&quot;,<br>    &quot;portMappings&quot;: [<br>      {<br>        &quot;containerPort&quot;: 80,<br>        &quot;hostPort&quot;: 80<br>      }<br>    ]<br>  }<br>]<br>DEFINITION<br>}</pre><p>The last step is to specify an ECS Service. An ECS Service lets you specify how many copies of your task definition (the containerized workload) you wish to run in the ECS cluster. It also attaches the task definition to the ECS service that you created.</p><pre>resource &quot;aws_ecs_service&quot; &quot;ecsservice&quot; {<br>  name            = &quot;nginxservice&quot;<br>  cluster         = &quot;levanecscluster&quot;<br>  task_definition = aws_ecs_task_definition.nginxapp.arn<br>  desired_count   = 2</pre><pre>  depends_on = [<br>    aws_ecs_task_definition.nginxapp<br>  ]<br>}</pre><h3>EKS</h3><p>First, configure the Terraform backend config and provider.</p><pre>terraform {<br>  backend &quot;s3&quot; {<br>    bucket = &quot;name_of_bucket&quot;<br>    key    = &quot;eks-terraform.tfstate&quot;<br>    region = &quot;us-east-1&quot;<br>  }<br>  required_providers {<br>    aws = {<br>      source = &quot;hashicorp/aws&quot;<br>    }<br>  }<br>}</pre><p>Next, create the IAM Role for EKS to have access to the appropriate resources.</p><pre>resource &quot;aws_iam_role&quot; &quot;eks-iam-role&quot; {<br>  name = &quot;eks-iam-role&quot;</pre><pre>  path = &quot;/&quot;</pre><pre>  assume_role_policy = &lt;&lt;EOF<br>{<br>  &quot;Version&quot;: &quot;2012-10-17&quot;,<br>  &quot;Statement&quot;: [<br>    {<br>      &quot;Effect&quot;: &quot;Allow&quot;,<br>      &quot;Principal&quot;: {<br>        &quot;Service&quot;: &quot;eks.amazonaws.com&quot;<br>      },<br>      &quot;Action&quot;: &quot;sts:AssumeRole&quot;<br>    }<br>  ]<br>}<br>EOF<br>}</pre><p>Attach the IAM policies to the IAM role for EKS and container registry.</p><pre>resource &quot;aws_iam_role_policy_attachment&quot; &quot;AmazonEKSClusterPolicy&quot; {<br>  policy_arn = &quot;arn:aws:iam::aws:policy/AmazonEKSClusterPolicy&quot;<br>  role       = aws_iam_role.eks-iam-role.name<br>}<br>resource &quot;aws_iam_role_policy_attachment&quot; &quot;AmazonEC2ContainerRegistryReadOnly-EKS&quot; {<br>  policy_arn = &quot;arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly&quot;<br>  role       = aws_iam_role.eks-iam-role.name<br>}</pre><pre>## Create the EKS cluster<br>resource &quot;aws_eks_cluster&quot; &quot;k8squickstart-eks&quot; {<br>  name = &quot;k8squickstart-cluster&quot;<br>  role_arn = aws_iam_role.eks-iam-role.arn</pre><pre>  #enabled_cluster_log_types = [&quot;api&quot;, &quot;audit&quot;, &quot;scheduler&quot;, &quot;controllerManager&quot;]</pre><pre>  vpc_config {<br>    subnet_ids = [var.subnet_id_1, var.subnet_id_2]<br>  }</pre><pre>  depends_on = [<br>    aws_iam_role.eks-iam-role,<br>  ]<br>}</pre><p>Next, configure the Worker Nodes.</p><pre>resource &quot;aws_iam_role&quot; &quot;workernodes&quot; {<br>  name = &quot;eks-node-group-example&quot;</pre><pre>  assume_role_policy = jsonencode({<br>    Statement = [{<br>      Action = &quot;sts:AssumeRole&quot;<br>      Effect = &quot;Allow&quot;<br>      Principal = {<br>        Service = &quot;ec2.amazonaws.com&quot;<br>      }<br>    }]<br>    Version = &quot;2012-10-17&quot;<br>  })<br>}</pre><p>Attach the appropriate policies to the worker nodes. The reason why you have to do this is because the policies needed for EKS are different than the policies needed for the worker nodes, which are EC2 instances.</p><pre>resource &quot;aws_iam_role_policy_attachment&quot; &quot;AmazonEKSWorkerNodePolicy&quot; {<br>  policy_arn = &quot;arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy&quot;<br>  role       = aws_iam_role.workernodes.name<br>}</pre><pre>resource &quot;aws_iam_role_policy_attachment&quot; &quot;AmazonEKS_CNI_Policy&quot; {<br>  policy_arn = &quot;arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy&quot;<br>  role       = aws_iam_role.workernodes.name<br>}</pre><pre>resource &quot;aws_iam_role_policy_attachment&quot; &quot;EC2InstanceProfileForImageBuilderECRContainerBuilds&quot; {<br>  policy_arn = &quot;arn:aws:iam::aws:policy/EC2InstanceProfileForImageBuilderECRContainerBuilds&quot;<br>  role       = aws_iam_role.workernodes.name<br>}</pre><pre>resource &quot;aws_iam_role_policy_attachment&quot; &quot;AmazonEC2ContainerRegistryReadOnly&quot; {<br>  policy_arn = &quot;arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly&quot;<br>  role       = aws_iam_role.workernodes.name<br>}</pre><pre>resource &quot;aws_iam_role_policy_attachment&quot; &quot;CloudWatchAgentServerPolicy-eks&quot; {<br>  policy_arn = &quot;arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy&quot;<br>  role       = aws_iam_role.workernodes.name<br>}</pre><p>Lastly, create the resource for the worker nodes.</p><pre>resource &quot;aws_eks_node_group&quot; &quot;worker-node-group&quot; {<br>  cluster_name    = aws_eks_cluster.k8squickstart-eks.name<br>  node_group_name = &quot;k8squickstart-workernodes&quot;<br>  node_role_arn   = aws_iam_role.workernodes.arn<br>  subnet_ids      = [var.subnet_id_1, var.subnet_id_2]<br>  instance_types = [&quot;t3.xlarge&quot;]</pre><pre>  scaling_config {<br>    desired_size = var.desired_size<br>    max_size     = var.max_size<br>    min_size     = var.min_size<br>  }</pre><pre>  depends_on = [<br>    aws_iam_role_policy_attachment.AmazonEKSWorkerNodePolicy,<br>    aws_iam_role_policy_attachment.AmazonEKS_CNI_Policy,<br>    #aws_iam_role_policy_attachment.AmazonEC2ContainerRegistryReadOnly,<br>  ]<br>}</pre><h3>Conclusion</h3><p>At the end of the day, Kubernetes/EKS and ECS are both great options. You can’t really go wrong either way from an orchestration perspective. They both do the same thing 95% of the way. The biggest thing you need to think about is how large your workloads are going to become and if you’re comfortable being locked into a specific AWS service.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4af0d34e9a55" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cilium Network Configuration For Kubernetes (enter eBPF)]]></title>
            <link>https://michael-levan.medium.com/cilium-network-configuration-for-kubernetes-enter-ebpf-5ebd0cc6f1f?source=rss-8fa8e88f2ad6------2</link>
            <guid isPermaLink="false">https://medium.com/p/5ebd0cc6f1f</guid>
            <category><![CDATA[engineering]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[docker]]></category>
            <dc:creator><![CDATA[Michael Levan]]></dc:creator>
            <pubDate>Wed, 02 Nov 2022 09:48:21 GMT</pubDate>
            <atom:updated>2022-11-02T09:48:21.155Z</atom:updated>
            <content:encoded><![CDATA[<p>Throughout the world of Linux, and ultimately, Kubernetes, there have been a lot of use cases that haven’t changed in the underlying infrastructure. One of those use cases that have been around since Kubernetes inception is the use of IP Tables. However, as with all technologies, new innovation needs to happen to make environments more efficient.</p><p>In this blog post, you’ll learn about what eBPF is doing for Kubernetes and how to implement it with Cilium.</p><h3>What Is eBPF?</h3><p>In short, eBPF is a way to remove the need to update Linux kernel code for certain programs to run and from a Kubernetes perspective, takes over kube-proxy’s responsibilities.</p><p>The topics around eBPF are vast. They can probably take up a whole book in itself. Because of that, this section can’t go through every piece of eBPF. However, let’s focus on a few key parts when it comes to Kubernetes.</p><ul><li>Removal of kube-proxy</li><li>Easier scaling</li><li>Security</li></ul><p>kube-proxy has served Kubernetes well, but there’s a problem — it uses IP Tables. Although IP Tables have been in Linux for a long time, it doesn’t scale very well from a Kubernetes perspective. IP Table rules are stored in a list, and when Pods establish a new connection to a Kubernetes Service, it has to go through every single IP Table rule until it reaches the specific rule that it’s looking for. Although that doesn’t seem like a lot for a couple of rules, if you have thousands (which you most likely will), it’s a big performance concern.</p><p>From a scalability perspective, as the number of Kubernetes Services grows, the connection gets worse. One of the biggest reasons is that IP Table rules are not incremental when you create them, which means that kube-proxy writes out the whole table for every single update.</p><p>From a security perspective, Kubernetes NetworkPolicy doesn’t have to be handled at the IP Tables layer anymore. This speeds up policy management a bit because it allows you to have a finer-tuned approach without having to worry about several IP Table rules.</p><p>eBPF gives the ability to remove IP Tables by injecting directly into the Pod network. It improves the performance of routing and scalability by not having to round robin through IP Table rules.</p><h3>Cilium</h3><p>There are a ton of different Container Network Interfaces (CNI) for Kubernetes. With so many different options, you have to imagine that there are many different functionalities. Some are more for “getting started quickly” from an ease-of-use perspective, and there are also CNI’s that are more security-focused or more “advanced”.</p><p>Cilium is a CNI that’s built around eBPF. It focuses heavily on networking, observability, and security for Kubernetes networking that uses eBPF to get the job done.</p><p>What it does from a networking perspective on the top level isn’t any different. It allows you to have networking in your Kubernetes cluster, have specific CIDR’s for Pods, ensure that Pods can communicate, and all of the rest of the networking pieces of a CNI. The biggest difference is that it does it all with an eBPF backend utilizing hash tables instead of having to use kube-proxy and IP Tables.</p><h3>Cilium Implementation</h3><p>For the Cilium install, you will need a Kubernetes cluster. For the purposes of this blog post, it shows how to install Cilium on Kubeadm. However, please note that the installation method should be the same across Kubernetes clusters minus the command below for kubeadm init.</p><p>If you’re planning to run Kubeadm, the following command is an example of what you should use. Even if you don’t use all of the flags, ensure that you use the --skip-phases=addon/kube-proxy flag as this is needed so kube-proxy doesn’t get installed on the control plane.</p><pre>sudo kubeadm init --skip-phases=addon/kube-proxy --control-plane-endpoint $publicIP --apiserver-advertise-address $ip_address --pod-network-cidr=$cidr --upload-certs</pre><p>Next, ensure that Helm is installed.</p><pre>curl -fsSL -o get_helm.sh &lt;https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3&gt;<br>chmod 700 get_helm.sh<br>./get_helm.sh</pre><p>Add the Cilium Helm repo.</p><pre>helm repo add cilium &lt;https://helm.cilium.io/&gt;</pre><p>Install Cilium with Helm. The below Helm installation will do the following:</p><ul><li>Put Cilium in the kube-system Namespace</li><li>Ensure that the kube-proxy replacement is set</li><li>Specify the host of your control plane</li><li>Specify the port of the control plane</li></ul><pre>helm install cilium cilium/cilium \\<br>    --namespace kube-system \\<br>    --set kubeProxyReplacement=strict \\<br>    --set k8sServiceHost=ip_address_of_control_plane \\<br>    --set k8sServicePort=6443</pre><p>After a few minutes, ensure that the Cilium Pods are running successfully.</p><pre>kube get pods -n kube-system</pre><p>Congrats! You have successfully set up Cilium for eBPF and removed the need for IP Tables and kube-proxy.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5ebd0cc6f1f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Implementing Kubernetes Resource Limits and Quotas]]></title>
            <link>https://michael-levan.medium.com/implementing-kubernetes-resource-limits-and-quotas-1aadae50b0e0?source=rss-8fa8e88f2ad6------2</link>
            <guid isPermaLink="false">https://medium.com/p/1aadae50b0e0</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[engineering]]></category>
            <category><![CDATA[docker]]></category>
            <dc:creator><![CDATA[Michael Levan]]></dc:creator>
            <pubDate>Thu, 20 Oct 2022 19:21:15 GMT</pubDate>
            <atom:updated>2022-10-20T19:21:15.899Z</atom:updated>
            <content:encoded><![CDATA[<p>Although there are several areas of tech that are abstracted away from us and most pieces are software-defined when it comes to Kubernetes, there’s still an underlying infrastructure. Whether that’s an infrastructure that you’re managing or a Kubernetes cloud-based service like AKS, EKS, or GKE. Servers still exist and they’re what’s running the Kubernetes platform.</p><p>Because of that, resource consumption for things like CPU, memory, and storage very much exists and needs to be accounted for.</p><p>In this blog post, you’ll learn about implementing resource limits, why it’s important, and the need for rate limits.</p><h3>Why Resource Limits and Quotas</h3><p>When you’re working with a team, there could be:</p><ul><li>multi-tenancy or single-tenancy when it comes to Kubernetes clusters</li><li>The servers could be extremely large or extremely small</li><li>There could be 10 apps or 1,000 apps running on the Kubernetes cluster</li><li>Fixed nodes or auto-scaling nodes</li></ul><p>Thinking about the above, there’s a lot that can go wrong with a cluster that runs out of resources (CPU, memory, storage, etc.). Pods can crash, users can be booted off of the application, servers can shut down and go into a panic, alerts triggered, and worst of all, a long night for the DevOps team.</p><p>The goal with resource limits and quotas is that you want each application to have its “fair share” of resources. No more than it needs and no less.</p><h3>Resource Limits for Pods</h3><p>When you attempt to create a Pod, the kube-scheduler looks at what worker nodes on the Kubernetes cluster have enough resources to host the Pod(s). Once the kube-scheduler finds a suitable node, the Pod gets deployed. When you enforce rate limits, the kubelet enforces the rate limits and doesn’t allow your Pod(s) to go above the limit.</p><p>Below is an example Kubernetes Manifest for resource limits. Notice how this is a standard Manifest, but there’s a resources map that contains requests and limits around memory and CPU. requests are for literally requesting those resources for the Pod, and limits are telling the Pod “you can’t go above this amount of memory and CPU”.</p><pre>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: nginx-deployment<br>spec:<br>  selector:<br>    matchLabels:<br>      app: nginxdeployment<br>  replicas: 2<br>  template:<br>    metadata:<br>      labels:<br>        app: nginxdeployment<br>    spec:<br>      containers:<br>      - name: nginxdeployment<br>        image: nginx:latest<br>        resources:<br>          requests:<br>            memory: &quot;64Mi&quot;<br>            cpu: &quot;250m&quot;<br>          limits:<br>            memory: &quot;128Mi&quot;<br>            cpu: &quot;500m&quot;<br>        ports:<br>        - containerPort: 80</pre><p>When setting up resource limits for Pods, remember to take into account horizontal and vertical auto-scaling. Horizontal auto-scaling is for creating more of the same Pod. Vertical auto-scaling is for increasing resources (memory, CPU, etc.) on the Pod. Both of these are important to keep in mind when architecting the way auto-scaling will work for your Kubernetes environment as you don’t want to be blocked by resource limits.</p><h3>Applying Quotas at the Namespace Level</h3><p>As you learned in the previous section, you can set up resource limits for Pods. Although that can be practical with a few applications, it doesn’t really scale well when you have multiple applications to manage. Because of that, setting quotas at the Namespace level may make a bit more sense for organizations. You do this by utilizing the ResourceQuota object/spec in a Kubernetes Manifest.</p><p>Common scenarios for applying quotas at the namespace level with the ResourceQuota object/spec are:</p><ul><li>Control the amount of compute resources that are available to a specific namespace</li><li>Limit the number of API objects that can be created within a namespace, like Pods and Services</li><li>Control the limit of LoadBalancer objects that can be created in the cloud for a Kubernetes Service as load balancers in the cloud can get expensive</li></ul><p>Applying quotas is great for protecting your environment as well. If you have an application that has some sort of memory leak, that application would just keep attempting to consume RAM until the system crashes. With quotas, you don’t have to worry about that and you can get ahead of the problem much faster.</p><p>Below is an example Kubernetes Manifest that you can use to create a ResourceQuota in a Namespace. The Namespace in this example is called <strong>golangapp</strong>.</p><pre>apiVersion: v1<br>kind: ResourceQuota<br>metadata:<br>  name: memorylimit<br>  namespace: golangapp<br>spec:<br>  hard:<br>    requests.memory: 512Mi<br>    limits.memory: 512Mi</pre><p>When setting up Resource Quotas in a Namespace, remember that there should be a fair amount of architecture design and decision-making around the process because Pods won’t deploy if the Quota has been met. For example, if you limit the memory to 512Mi, but you try to give a Pod 2GB of memory, there will be an error that looks similar to the below.</p><pre>Error from server (Forbidden):<br>  error when creating &quot;name_of_manifest.yaml&quot;: pods<br>&quot;app name&quot; is forbidden:<br>  exceeded quota: memorylimit</pre><h3>The Need For CPU Rate Limits</h3><p>In the last section of this blog post, I wanted to bring up a controversial topic; CPU rate limits.</p><p>There are two primary reasons to think about CPU rate limits.</p><p>The first is for benchmarks. If you’re benchmarking an application, the resources are going to sky-rocket. Because of that, you want to ensure that you don’t give your Pods unlimited resources.</p><p>The second is for language-specific needs. For example, if you’re using an older version of Java (somewhere around version 8u131 or below), JVM is unaware of the limit of resources and attempts to more consume memory and CPU than what’s actually needed.</p><p>There definitely isn’t a huge use-case for CPU rate limits. However, there are a few that do exist and you should be prepared to handle them if the need arises.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1aadae50b0e0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kubernetes Ingress vs Service Mesh]]></title>
            <link>https://michael-levan.medium.com/kubernetes-ingress-vs-service-mesh-a11200c05099?source=rss-8fa8e88f2ad6------2</link>
            <guid isPermaLink="false">https://medium.com/p/a11200c05099</guid>
            <category><![CDATA[engineering]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Michael Levan]]></dc:creator>
            <pubDate>Thu, 20 Oct 2022 19:20:27 GMT</pubDate>
            <atom:updated>2022-10-20T19:20:27.143Z</atom:updated>
            <content:encoded><![CDATA[<p>Networking in Kubernetes is no easy task. Whether you’re on the application side or the operations side, you need to think about networking. Whether it’s connectivity between clusters, control planes, and worker nodes, or connectivity between Kubernetes Services and Pods, it all becomes a task that needs a large amount of focus and effort.</p><p>In this blog post, you’ll learn about what a service mesh is, what ingress is, and why you need both.</p><h3>What’s A Service Mesh</h3><p>When you deploy applications inside of Kubernetes, there are two primary ways that the apps are talking to each other:</p><ul><li>Services</li><li>Pod-to-Pod communication</li></ul><p>Pod-to-Pod communication isn’t exactly recommended because Pods are ephemeral, which means they aren’t permanent. They are designed to go down at any time and only if they’re part of a StatefulSet would they keep any type of unique identifier. However, Pods still need to be able to communicate with each other because microservices need to talk. Backends need to talk to frontends, middleware needs to talk to backends and frontends, etc…</p><p>The next primary communication is Services. Services are the preferred method because a Service isn’t ephemeral and only gets deleted if specified by an engineer. Pods are able to connect to Services with Selectors (sometimes called Tags), so if a Pod goes down but the Selector in the Kubernetes Manifest that deployed the Pod doesn’t change, the new Pod will be connected to the Service.</p><p>Here’s the problem; all of this traffic is unencrypted. Pod-to-Pod communication, or as some people like to call it, East-West Traffic, is completely unencrypted. That means if for any reason a Pod is compromised or you have some segregation concerns, there’s nothing out-of-the-box that you can do.</p><p>A Service Mesh handles a lot of that for you. A Service Mesh:</p><ul><li>Encrypts traffic between microservices</li><li>Helps with network latency troubleshooting</li><li>Securely connects Kubernetes Services</li><li>Observability for tracing and alerting</li></ul><h3>What’s Ingress</h3><p>Outside of the need for secure communication between microservices, you need a way to interact with frontend apps. The typical way is with a load balancer that’s connected to a Service. You can also use a NodePort, but in the cloud world, you’ll mostly see load balancers being used.</p><p>Here’s the problem; cloud load balancers are expensive literally and figuratively. You have to pay money for each cloud load balancer that you have. Having a few applications may not be a big deal, but what about if you have 50 or 100? Not to mention that you have to manage all of those cloud load balancers. If a Kubernetes Service disconnects from the load balancer for whatever reason, it’s your job to go in and fix it.</p><p>With Kubernetes Ingress Controllers, the management and cost nightmare is abstracted from you. An Ingress Controller allows you to have:</p><ul><li>One load balancer</li><li>Multiple applications (Kubernetes Services) pointing to it</li></ul><p>You can create one load balancer and have every Kubernetes Service point to it. Then, you can access each Kubernetes Service on a different path.</p><p>For example, below is an Ingress Spec that points to a Kubernetes Service called nginxservice and outputs it on the path called /nginxappa</p><pre>apiVersion: networking.k8s.io/v1<br>kind: Ingress<br>metadata:<br>  name: ingress-nginxservice-a<br>spec:<br>  ingressClassName: nginx-servicea<br>  rules:<br>  - host: localhost<br>    http:<br>      paths:<br>      - path: /nginxappa<br>        pathType: Prefix<br>        backend:<br>          service:<br>            name: nginxservice<br>            port:<br>              number: 8080</pre><p>Ingress Controllers are like an Nginx Reverse Proxy.</p><h3>Do You Need Both?</h3><p>There’s a lot of talk between engineers around the idea of not needing both a Service Mesh and an Ingress Controller. My take on it is that you need both. Here’s why.</p><p>They’re both doing two different jobs. I always like to use the hammer analogy. If I need to hammer a nail, I can use the handle to slam the nail in and eventually it’ll work, but why would I do that if I can use the proper end of the hammer?</p><p>An Ingress Controller is used to:</p><ul><li>Make load balancing apps easier</li></ul><p>A Service Mesh is used to:</p><ul><li>Secure communication between apps</li><li>Help out with Kubernetes networking</li></ul><p>Now, here’s the kicker; there are tools that do both. For example, Istio Ingress is an Ingress Controller, but also has the capability of secure gateways using TLS or mTLS. If you’re using one of those tools, great. Just make sure that it handles both communication and security for you in the way that you’re expecting. The recommendation still is to use the proper tool for the job.</p><p>Both Service Mesh and Ingress are incredibly important, especially as your microservice environment grows.</p><h3>Popular Ingress Controllers and Service Mesh Platforms</h3><p>Below is a list of Ingress Controllers and Service Mesh that are popular in today’s cloud-native world.</p><p>For Service Mesh:</p><ul><li><a href="https://www.consul.io/">https://www.consul.io/</a></li><li><a href="https://istio.io/latest/about/service-mesh/">https://istio.io/latest/about/service-mesh/</a></li><li><a href="https://platform9.com/learn/linkerd-as-service-mesh">https://platform9.com/learn/linkerd-as-service-mesh</a></li></ul><p>For Ingress Controllers:</p><ul><li><a href="https://kubernetes.github.io/ingress-nginx/">https://kubernetes.github.io/ingress-nginx/</a></li><li><a href="https://doc.traefik.io/traefik/providers/kubernetes-ingress/">https://doc.traefik.io/traefik/providers/kubernetes-ingress/</a></li><li><a href="https://github.com/Kong/kubernetes-ingress-controller#readme">https://github.com/Kong/kubernetes-ingress-controller#readme</a></li><li><a href="https://istio.io/latest/docs/tasks/traffic-management/ingress/">https://istio.io/latest/docs/tasks/traffic-management/ingress/</a></li></ul><p>If you want to check out how to get started with the Istio, check out my blog post on it <a href="https://dev.to/thenjdevopsguy/kubernetes-service-mesh-istio-edition-2l2p">here</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a11200c05099" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How To Install a Kubernetes Container Storage Interface Driver]]></title>
            <link>https://michael-levan.medium.com/how-to-install-a-kubernetes-container-storage-interface-driver-df3c035c788a?source=rss-8fa8e88f2ad6------2</link>
            <guid isPermaLink="false">https://medium.com/p/df3c035c788a</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[engineering]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Michael Levan]]></dc:creator>
            <pubDate>Thu, 20 Oct 2022 19:19:17 GMT</pubDate>
            <atom:updated>2022-10-20T19:19:17.586Z</atom:updated>
            <content:encoded><![CDATA[<p>When you want to work with storage inside of Kubernetes, as in, utilize Volumes, you need a way to “communicate” with those volumes. That way, you can do things like store Pod data inside of a hard drive or have stateful applications.</p><p>In this blog post, you’ll learn about what CSI is and how you can implement a CSI driver in your Kubernetes cluster.</p><h3>What’s CSI?</h3><p>CSI stands for Container Storage Interface, which is one of the many interfaces that you can use in Kubernetes. An interface is essentially a standard for how you interact with resources outside of Kubernetes. For example, the Container Network Interface (CNI) is a standard for how you get networking components up and running in a Kubernetes cluster. A CSI is a standard for how you get storage components up and running in a Kubernetes cluster.</p><p>With CSI’s, you can interact with storage on a local server or even outside of the cluster itself. For example, there’s a CSI for interacting with Azure Storage and a CSI to interact with AWS S3. The CSI’s allow you to use Azure Storage or AWS S3 to store Pod data. That way, if a Pod goes away (because they’re ephemeral), the data still exists.</p><p>What a CSI really does for vendors, like Azure and AWS, is give them a standard way to expose storage resources to Kubernetes. That way, Kubernetes can utilize the storage.</p><p>There are a lot of CSI’s, not just for the cloud. For example, there’s a CSI for NetApp.</p><p>By definition, a Container Storage Interface is a standard to expose arbitrary block and file storage systems to containerized workloads.</p><h3>Before CSI</h3><p>Even before Container Storage Interfaces existed, you could still interact with volumes and vendors that had storage solutions. The biggest problem was to do this, the Kubernetes release itself, as in, the actual core platform code, needed to include the interface. Without it, the storage interface wouldn’t work. This option was called “in-tree”. For example, if a vendor found a bug in the storage interface, they would have to wait until a new Kubernetes release to get the bug fix out to users. It was a hassle for the vendors, but it was also a hassle for the Kubernetes maintainers because they had to do a ton of extra work to manage the storage interfaces.</p><p>With CSI, all of those problems go away. Now called an “out-tree” approach, vendors can create their own storage interfaces and release them outside of the core Kubernetes platform code, giving vendors the ability to manage and maintain their own release cycles. The only thing they have to ensure is to follow the CSI standards and requirements when building a CSI.</p><h3>Installing A CSI Driver</h3><p>Now that you know why you’d want to use a CSI, let’s learn how to work with them.</p><p>First and foremost, installing the CSI driver in itself is going to be different for almost every vendor.</p><p>For example, you can install a CSI driver in AKS with the following command:</p><pre>az aks update -n myAKSCluster -g myResourceGroup --enable-disk-driver --enable-file-driver --enable-snapshot-controller</pre><p>You can install a CSI driver for EKS with the following command:</p><pre>eksctl create addon --name aws-ebs-csi-driver --cluster my-cluster --service-account-role-arn arn:aws:iam::111122223333:role/AmazonEKS_EBS_CSI_DriverRole --force</pre><p>You can install a CSI driver for GKE with the following command:</p><pre>gcloud container clusters create CLUSTER-NAME \\<br>    --addons=GcePersistentDiskCsiDriver \\<br>    --cluster-version=VERSION</pre><p>As you can see, it’s going to vary based on the vendor. The best thing that you can do is literally Google “how to install CSI for xvendor”</p><p>Here’s a list of drivers that are currently supported: <a href="https://kubernetes-csi.github.io/docs/drivers.html">https://kubernetes-csi.github.io/docs/drivers.html</a></p><p>The good news is, there’s one thing that’s mostly always going to be the same — how you use the CSI. The way to use a CSI is to create a StorageClass inside of your Kubernetes cluster pointing to the CSI.</p><p>For example, here’s what it would look like for Azure.</p><pre>kind: StorageClass<br>apiVersion: storage.k8s.io/v1<br>metadata:<br>  name: custom-managed-premium<br>provisioner: disk.csi.azure.com<br>reclaimPolicy: Delete<br>parameters:<br>  storageAccountType: Premium_LRS</pre><p>And here’s what it would look like for AWS</p><pre>kind: StorageClass<br>apiVersion: storage.k8s.io/v1<br>metadata:<br>  name: gp2<br>  annotations:<br>    storageclass.kubernetes.io/is-default-class: &quot;true&quot;<br>provisioner: kubernetes.io/aws-ebs<br>parameters:<br>  type: gp2<br>  fsType: ext4</pre><p>As you can see, it’s utilizing the same kind along with API spec and similar maps.</p><p>When you’re using a CSI, there are two things to remember:</p><ul><li>Ensure to research how you get a CSI driver installed for a specific vendor.</li><li>Ensure that you initialize the CSI driver with a StorageClass</li></ul><p>That’s how you can get started with CSI! Thanks for reading.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=df3c035c788a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What Problem Is Kubernetes Actually Trying To Solve?]]></title>
            <link>https://michael-levan.medium.com/what-problem-is-kubernetes-actually-trying-to-solve-833eece75a9b?source=rss-8fa8e88f2ad6------2</link>
            <guid isPermaLink="false">https://medium.com/p/833eece75a9b</guid>
            <category><![CDATA[engineering]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Michael Levan]]></dc:creator>
            <pubDate>Thu, 20 Oct 2022 19:18:20 GMT</pubDate>
            <atom:updated>2022-10-20T19:18:20.540Z</atom:updated>
            <content:encoded><![CDATA[<p>When you comb through the marketing around Kubernetes regarding “this product or that product” or the countless amount of buzz-words that come with it, you’re left with a platform. A platform that as you dive deeper and deeper, isn’t as easy and straightforward as abstraction-based services like Kubernetes running in the cloud make it out to be.</p><p>Kubernetes and all of its internals, along with everything under the hood of services like AKS, EKS, GKE, etc. are extremely complex.</p><p>With all of this complexity, what is Kubernetes actually trying to solve?</p><p>The purpose of this blog post is to explain exactly why Kubernetes exists in the first place.</p><h3>Technology Over Platform</h3><p>Kubernetes itself is a platform.</p><p>It’s an extremely large platform. It’s been called things like:</p><ul><li>The datacenter of the cloud</li><li>The operating system of the cloud</li></ul><p>and for good reason — because there’s a lot that happens underneath the hood.</p><p>With that being said, let’s take a step back and think about what Kubernetes actually does for us.</p><p>Kubernetes scales and manages Pods, which consist of containers, and containers run applications. The applications could be frontend apps, backend apps, scripts, or pretty much anything you’d like.</p><p>Kubernetes orchestrates Pods for us by:</p><ul><li>Scheduling the Pods to run on certain nodes.</li><li>Auto-scaling up or down, horizontally or vertically, the Pods for us.</li><li>Allowing the same application to run in multiple Pods with replicas.</li><li>Exposing Pods in the form of services so users and other Pods running inside of the Kubernetes cluster can access them.</li></ul><p>Kubernetes in essence is an orchestrator and manager of applications.</p><p>The core problem that Kubernetes is solving is the ability to manage containerized apps at scale.</p><p>However, Kubernetes isn’t the only platform doing this. Remember, you must keep in mind that “technology over platform” is extremely important. Kubernetes isn’t doing anything that no other platform has done before. It’s just doing it in, arguably, a much better way.</p><p>The most important thing to keep in mind when thinking about what Kubernetes is trying to solve isn’t using Kubernetes, but what Kubernetes actually is, which is an orchestrator and manager of applications.</p><h3>Complexities Of A Datacenter</h3><p>When you have virtual machines running in a datacenter from a bare-metal perspective, you’ll have something that looks like this:</p><ul><li>A datacenter itself</li><li>Racks of servers</li><li>Network equipment (firewalls, routers, switches, etc.)</li><li>Lots of cables</li></ul><p>If you’re running Kubernetes on-prem, you’re still going to have the datacenter, racks of servers, network equipment, and cables. What you won’t have are applications running inside of virtual hardware, like ESXi scaled across countless servers in and out of the datacenter.</p><p>Instead, you’ll have Kubernetes clusters scaled in and out of the datacenter.</p><p>Although this may sound just as complex in the beginning, it’s not. Kubernetes allows you to have one system to manage, with a fraction of time spent vs managing virtualized hardware. You’ll also have one API to interact with vs many UI’s, automation methodologies, and configurations across countless servers. The other reality is, chances are you’re probably not going to run Kubernetes on a bare metal server. You’ll most likely use something like OpenStack or a hybrid cloud model, which will add another layer of abstraction that removes the direct bare-metal management.</p><p>In short, you’ll manage your environment with an API instead of clicking buttons and having tons of different scripts automating for you.</p><h3>Complexities Of The Cloud</h3><p>When you move workloads to the cloud, the core difference between a data center and the cloud is you’re no longer managing the hardware. Everything else is still on you to figure out, including:</p><ul><li>Scaling applications</li><li>Replicas for the applications</li><li>Auto-healing the applications</li><li>How you manage and interact with the applications</li><li>How you automate everything on this list</li></ul><p>The cloud definitely made life easier, but the actual complexities other than managing hardware are the same in the cloud.</p><p>With Kubernetes, it’s meant to make this a bit easier. Let’s take the cloud as an example. If you’re running Kubernetes in the cloud and you need to scale a node out, you can do so automatically. A worker node automatically gets created and is automatically connected to your Kubernetes cluster. Then, if that worker node is needed for Pods to run it, that automatically happens as well based on the Kubernetes Scheduler.</p><p>Another huge, and arguably the most important, aspect of running Kubernetes in the cloud is that the control plane is abstracted for you. This is another layer of abstraction and overall another complexity that you don’t have to worry about.</p><p>Without Kubernetes, you can still have auto-scaling groups for cloud virtual machines, but you still need to create the automation and repeatable processes to get your workloads over to the virtual machines.</p><p>In short, running Kubernetes in the cloud and outside of the cloud helps to resolve the scalability issues of infrastructure.</p><h3>Complexities Of Scaling Servers</h3><p>The ability to scale and have high availability throughout virtualized or bare-metal servers isn’t the easiest task in the world.</p><p>Two ways of scaling servers are:</p><ul><li>Hot</li><li>Cold</li></ul><p>“Hot” means that servers are up and running, but on standby. They’re ready to take the workload as it comes.</p><p>“Cold” means the servers possibly have a golden image on them of the application, binaries, and dependencies that are needed, but they’re turned off.</p><p>In either of the scenarios, you have to worry about updates to those servers, managing them, paying for them, and automating the upkeep of them.</p><p>With Kubernetes, you pretty much just have to add a new worker node or a new control plane. Depending on where you’re deploying the worker node or control plane, you’ll still have to do the upkeep (updates, management, maintenance, etc.) on the server, but it’s far less than having to manage a server that’s running an application.</p><p>With scaling in general, it’s no lite task. There are a lot of repeatable processes that need to occur to get an application, binaries, and dependencies running on a server that makes it look and feel the same way as the other servers. With Kubernetes, when you’re installing a new control plane or worker node with, say, Kubeadm, it’s a one-line command that needs to be run on the server. After a few minutes, you’ve officially scaled out.</p><h3>Complexities Of Scaling Apps</h3><p>Kubernetes helps in creating a container-centric environment.</p><p>In other environments, you may have several other components like:</p><ul><li>Bare metal</li><li>Virtual machines</li><li>A little bit of both</li></ul><p>However, with Kubernetes, you have one type of environment — containerized.</p><p>With a containerized environment comes, of course, complexity on it’s own, but one great thing is you only have to create and plan for one type of environment. This goes for complexities like scaling of applications as well.</p><p>In a traditional environment, there’s a lot of planning that goes into scaling an application. There’s entire workflows that are comprised of how an application will work properly if the load becomes too intense on a server. The workflow (at a high level) looks something like:</p><ul><li>A new server gets created in an auto-scaling group</li><li>The server gets configure</li><li>The server installs all application dependencies</li><li>The application gets deployed</li><li>Tests are run to ensure that the application works properly</li></ul><p>Even at a high level, those steps appear to be complex. The amount of automation code alone that goes into configuring the server is a beast.</p><p>With Kubernetes, it’s pretty much done for you. If a Pod cannot obtain resources on one worker node, it just goes to the next worker node. If more replicas are needed for a Pod, you simply set a mincount and maxcount for how many replicas can be deployed. If you realize that the maxcount is too low, you simply update the maxcount and re-deploy the Kubernetes Manifest.</p><p>Kubernetes is certainly complex, but how easy it makes scaling applications is amazing.</p><h3>Upgrading Applications</h3><p>Thinking about the previous section, there is also the need to upgrade applications. The workflow (at a high level) typically looks something like this on a server:</p><ul><li>SSH into the server</li><li>Copy the new binary to the server</li><li>Shut down the service</li><li>Add in the new binary</li><li>Start the service</li><li>Hope that it all goes well</li></ul><p>With Kubernetes, you don’t have to do that. Most of that work is done for you with Rolling Updates.</p><p>Rolling Updates are like Canary Deployments. Let’s say you have three replicas for the Deployment. A Rolling Update will roll out the new version to one Pod, confirm it works, then move on to the next. This approach is much more straightforward than on a general virtual machine or bare-metal server.</p><h3>Manual Complexities</h3><p>Manual efforts mean:</p><ul><li>Slower build and times</li><li>Human intervention</li><li>No repeatable process</li></ul><p>Let’s break down each of these.</p><p>If you’re attempting to build and deploy an application on a server, it’s going to look something like this — create a server, deploy an operating system, run updates, install app dependencies, deploy the app, and troubleshoot the app. This is not only hours but perhaps days of effort. Instead, it makes sense to containerize an app, package up the dependencies, and deploy it out to be managed by an orchestrator. The feedback loop of what’s wrong with the app and what needs to be fixed is vastly faster than the original, manual effort of deploying a server.</p><p>Humans make mistakes, and that won’t ever stop. Human intervention when it comes to application deployments is awful, and truthfully, there’s no reason for it anymore. 20 years ago engineers needed to be on a keyboard deploying an app. Now, there are so many automation platforms and orchestrators that it doesn’t make sense to spend time doing things manually. Instead, train systems to do the work for you. Kubernetes is one of those systems.</p><p>If you’re manually doing work, that means no repeatable processes are being created. Not only is it not great for you because you’re putting out fires instead of creating value-driven work, but it’s not great for your team. If you’re on vacation or if you quit, the rest of your team has no idea what makes the system you were working on “tick”. Orchestration platforms and creating repeatable processes help mitigate this risk. It’s a huge lift off of any engineer’s shoulders.</p><p>In short, Kubernetes is configuration management for your apps, and one of the reasons it exists is to remove the manual complexity of manually configuring apps.</p><h3>Desired State</h3><p>In any environment, there are two types of state:</p><ul><li>Current state</li><li>Desired state</li></ul><p>Sometimes, the current state of your platforms and application are the desired state. Most times, they aren’t. Whether it’s because what was deployed didn’t work as expected or someone SSH’d into a server manually and changed something. Then, if the environment is re-deployed, the state wouldn’t match the manual change.</p><p>Kubernetes aims to solve this with Controllers, which look at the current state and confirm that it’s the desired state. For example, take the Deployment Controller. The Deployment Controller will, for example, look at an app that’s supposed to have three replicas. If, for whatever reason, only two exist, the Deployment Controller will take action and ensure that the application obtains a third replica (a third Pod).</p><p>There are other Kubernetes-centric systems that do this as well. For example, GitOps. GitOps takes the idea of what a Kubernetes Controller is and puts the same method into having a source control repository become the desired state. GitOps methodologies “check in” with the source control repo to confirm what’s deployed to Kubernetes matches what’s in source control. If it doesn’t, the GitOps Controller (like Flux) kicks into gear and deploys what’s needed for the Kubernetes environment to match the source control repo.</p><h3>The Push For Microservices</h3><p>Monolithic applications are apps that are tightly coupled, which means if any update needs to occur on one part of the app, it’s going to impact the rest of the application.</p><p>Microservices are carved out pieces of an app. For example, let’s say you have a frontend, a middleware, and a backend. In a monolithic environment, all three of those pieces would be together in the same codebase. In a microservice, they’re split into three different parts. That means if you want to update one part, the rest of the app isn’t impacted.</p><p>Because of the way that Pods work, it essentially doesn’t make sense to not split up your application into microservices, or at least split up the code base in separate repos and have a container image for each code base.</p><p>Because of how straightforward Kubernetes makes it to create a Pod, it’s helped a ton for organizations to get the push they’ve need to enable microservices.</p><h3>Closing Thoughts</h3><p>When implementing Kubernetes, the primary thing to remember is it’s complex in the beginning, but as you learn about it, you realize that it’s only complex because it’s a new way of managing applications. Once you truly go in-depth with Kubernetes, you understand that it’s actually removing a lot of the complexities that standard environments face and in turn, allows you to focus on more value-driven work instead of putting out fires or utilizing a ton of your time in manual intervention.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=833eece75a9b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Future Of Kubernetes]]></title>
            <link>https://michael-levan.medium.com/the-future-of-kubernetes-33bacfe909cf?source=rss-8fa8e88f2ad6------2</link>
            <guid isPermaLink="false">https://medium.com/p/33bacfe909cf</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[engineering]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Michael Levan]]></dc:creator>
            <pubDate>Thu, 20 Oct 2022 19:17:06 GMT</pubDate>
            <atom:updated>2022-10-20T19:17:06.163Z</atom:updated>
            <content:encoded><![CDATA[<p>Understanding what will happen to a technology once you learn it, live it, and use it daily pops up a lot in a persons mind. The constant questions of “will this technology stick around?” or “when should I start learning the new thing?” comec into our mind and throw us for a loop.</p><p>With Kubernetes, the question of “is this technology going to stick around” doesn’t come up all that much, and for good reason.</p><p>In this blog post, you’ll learn about what I believe the future of Kubernetes is and how you should think about it for the next several years.</p><h3>Kubernetes Will Be Irrelevant</h3><p>When thinking about the direction that a platform can go, I like to think about it with the end in mind. As we all know in technology, there’s a beginning, a middle, and an end.</p><p>As for the beginning of Kubernetes, I would say that we’re still in it. Organizations are still trying to adopt Kubernetes, and once adopted, engineers are still trying to make it work in the way that they’re hoping for in terms of how they need to run workloads.</p><p>The middle of Kubernetes will be like all technology middle grounds. It’ll be adopted, wildly used in production, and just as much of a day-to-day operation as using the UI of a public cloud. Most of the kinks will be worked out and adoption rates will be steady, along with success rates.</p><p>At the end, Kubernetes won’t be the end of orchestration. It’ll just be another platform that was replaced by something else. Think about it like this — Managed Kubernetes Services like Azure Kubernetes Service and AWS EKS already have the tools to put any label on Kubernetes that they want. With Azure Container Apps and EKS Fargate, they could replace “Kubernetes” and call it something else. How? Because at the end of the day, an orchestration platform is an orchestration platform. It doesn’t matter what name it has. The beginning of the end already exists. All that’s needed is the name of the platform and a scalable approach.</p><p>I do believe that “the end of Kubernetes” will take a little bit longer than typical technology. The reason why is because just as much as Kubernetes is popular, it’s not the easiest platform to manage and deploy. In the past year or so is when products/platforms around making Kubernetes easier started coming out, and Kubernetes hit the shelves for users in 2014, so as you can see there’s a bit of a gap there. It almost feels like Kubernetes just came out, but at the same time, it’s been out for a while.</p><p>I decided to write this section first because thinking with the end in mind is crucial to understand how the rest of the future of Kubernetes, before the end, will play out. It’s also crucial to understand what you’ll need before the end of the Kubernetes.</p><p>The platform may become irrelevant, but the doors that the platform opened up for us won’t be. The key takeaway for Kubernetes in the future will be that Custom Resource Definitions (CRD) and extending the API will be “the future” of orchestration platforms.</p><h3>Environment Instead Of Platform</h3><p>As of right now, Kubernetes is thought of this mystery box. It’s a platform that engineers know, based on their experience level, sort of what’s happening inside of it, but not really. Many engineers take to the cloud and deploy Kubernetes workloads without understanding what’s happening inside of, for example, the control plane because they don’t have to manage it.</p><p>They don’t know about the underlying components, which can in-turn turn into a negative experience later on, which we’ll talk about in the upcoming sections.</p><p>Because of that negative experience, Kubernetes will be thought about less like a platform and more like a datacenter. It’ll be thought of as “the datacenter of the cloud”.</p><p>With all of it’s moving parts, security needs, networking needs, and deployment needs, environments will be built out around the successful deployment of Kubernetes and it’s workloads.</p><h3>What’s Happening Underneath The Hood</h3><p>When it comes to Kubernetes, there’s a lot of abstraction. If you think about Managed Kubernetes Services like AKS and GKE, you don’t have to worry about the control plane. That means you don’t have to think about:</p><ul><li>Scaling the control plane</li><li>The API server</li><li>Scheduler</li><li>Controller Manager</li><li>Etcd</li></ul><p>Which are arguably the most important parts of Kubernetes.</p><p>With Managed Kubernetes Services, you don’t even have the ability to properly know how all of the components work.</p><p>The question that I constantly ask myself is “how does someone troubleshoot if something is going wrong?”. For example, let’s say you’re having a problem with your Kubernetes cluster and it ends up being that the Container Network Interface (CNI) is having issues on the control plane, and therefore, Kubernetes Worker Nodes aren’t able to connect.</p><p>If you don’t know how the CNI works and that worker nodes and other control planes won’t connect to the primary control plane without CNI working, how can you troubleshoot the problem? How could you know if it’s something you did or something that the cloud provider did?</p><p>Understanding what’s going on underneath the hood is a crucial need for any engineer working with Kubernetes. Otherwise, you’re literally flying blind.</p><h3>Hybrid Solutions</h3><p>At the time of writing this, there are four primarily hybrid solutions:</p><ul><li>Azure Stack HCI</li><li>Google Anthos</li><li>AWS Outposts</li><li>EKS Anywhere</li></ul><p>They’re all more or less doing the same thing at a high level — giving engineers the ability to run on-prem workloads like they are on the cloud.</p><p>You may be thinking to yourself “why wouldn’t I just run the workloads in the cloud?” and the answer to that question can be one in a million. Some common scenarios are organizations have legacy apps that they aren’t ready to move to the cloud yet, latency concerns, security concerns, the lift and shift would be too great for the reward, or they simply want a combination of both on-prem and in the cloud (the hybrid model).</p><p>The next question becomes “why would they use these services then?” and I feel that the answer to the question is the same as why people want to use OpenStack. It’s like having a cloud, but on-prem.</p><p>For example, let’s say you already have containerized applications that are running in the cloud. Perhaps you already know EKS or AKS, and you want to containerize legacy apps and orchestrate them, but you don’t want to run them in the cloud. You could run them on-prem, with Kubernetes, the same way you run containerized apps in the cloud. It’ll look and feel the same. The only difference is that it’ll be on-prem instead of in the cloud. Engineering teams would be interacting with the apps and the interface the same exact way.</p><p>There will organizations, at least for a very long time, that’ll always have on-prem workloads. But just because they’re on-prem doesn’t mean they can’t get the benefits of cloud-native.</p><h3>Serverless Kubernetes</h3><p>Before Managed Kubernetes Services like AKS, EKS, and GKE, engineers had to create, manage, and update control planes and worker nodes. With AKS, EKS, and GKE, control planes are now managed from a day-to-day perspective via the cloud provider. With those services, an engineer still has to worry about the worker nodes.</p><p>The next iteration of this would be that worker nodes aren’t managed from a day-to-day perspective by engineers either. That’s where Serverless Kubernetes comes into play.</p><p>There are a few services out there that do exactly this:</p><ul><li>GKE AutoPilot</li><li>AWS Fargate</li><li>Azure Container Apps</li></ul><p>“Serverless Kubernetes”, as in, removing the need to run day-to-day operations for the control planes AND worker nodes is as close as we are to not even needing the Kubernetes platform anymore. At this point, the only thing that engineers have to think about are are Controllers, Customer Resource Definitions, and the Kubernetes API itself.</p><p>With Serverless Kubernetes, engineers are truly only worrying about managing their infrastructure with an API.</p><p>However, there is a catch here. Even though you’re not performing the day-to-day operations of the underlying infrastructure, things can still go wrong and you should still know about it. If the Kubernetes API, scheduler, or other components in the Kubernetes cluster are degrading for whatever reason, you need to know for troubleshooting purposes.</p><p>If you aren’t managing it, it’s not like there’s anything you can do about it, but you can at least know so you aren’t going down a rabbit hole that’s out of your control and so you can let management know what’s happening.</p><p>It’ll still be a waiting game for it to be fixed, but at least you’ll have a root cause analysis for the organization.</p><p>As of right now, I believe Serverless Kubernetes is still too new and not used in production all that much, which makes it an overall future prediction as to how it’ll be used. I believe the primary concern for organizations will be scale and latency. If the environment is completely out of your hands, you’re relying on underlying automation of these services to scale for you. A few folks at Microsoft for example have said that Azure Container App is great for testing right now, but not for production as things like Namespace segregation doesn’t exist. If issues like that get fixed, Serverless Kubernetes might be a solid contender.</p><h3>Creating Kubernetes Resources</h3><p>When Infrastructure-as-Code and Configuration Management started to become more of a reality for engineering teams, there was never an idea around developers working with it unless they really had to. The idea was to keep it more for Sysadmins and infrastructure engineers that needed to create, update, manage, and delete environments.</p><p>Then, this idea started to change a bit. We began to saw tools like Pulumi and the AWS CDK emerge, which allow you to perform Infrastructure-as-Code, but with a general-purpose programming language like Go or Python.</p><p>Now, engineers can choose whatever language they want to create, update, manage, and delete environments.</p><p>I believe the same will occur for creating and managing Kubernetes resources.</p><p>As of right now, the primary method is with YAML. However, that’s far from the only option available.</p><p>When you’re interacting with Kubernetes, all you’re doing is “talking to” an API. Because of that, any language that can “talk to” any API can create resources in Kubernetes.</p><p>For example, <a href="https://github.com/AdminTurnedDevOps/kubernetes-examples/blob/main/pulumi/nginxmanifest/main.go">here’s a link</a> to some Go code that I wrote to create a Kubernetes Deployment and Service for an Nginx app.</p><p>The know-how to do this is already there, but not a lot of engineers are doing it. I believe this will become more relevant as people want to move away from YAML.</p><h3>Dive Deeper</h3><p>I’ve touched on this a bit throughout the blog post so far, but I’d like to call it out more directly.</p><p>There’s far too much abstraction in today’s world, and there was a different purpose for abstraction when it was originally designed for technlogy.</p><p>Automation and repeatability was supposed to be implemented after the knowledge of manual efforts and as a solution for those manual efforts.</p><p>Now, platforms, systems, and overall environments are automated and abstracted away without the engineer even truly understanding what’s happening inside of the environment.</p><p>Without the underlying knowledge, there’s no possible way for an engineer to properly troubleshoot, re-architect, or scale out environments.</p><p>I’ve heard time and time again “why do I need to know this if it’s abstracted away” or “why do I need to know the underlying components of Kubernetes if I’m not working with them”.</p><p>The reason why is because scaling out large environments, troubleshooting, and figuring out solutions to problems isn’t always going to be as easy as clicking a few buttons or running some Terraform code.</p><p>There’s an ongoing joke in today’s engineering world that if folks can’t find the answer on StackOverflow in 10 minutes, they say it cannot be done.</p><p>Engineers cannot continue to work in this fashion or it will destroy environments.</p><p>Kubernetes is no different. Engineers must dive as deep as they can into what’s quickly becoming the datacenter of the cloud, just like Sysadmins had to dive deep into system components.</p><p>Without truly going deep into a topic, you can never really know that topic at all.</p><p>If you’re curious about the prerequisites to Kubernetes and how to dive deep into them, check out <a href="https://dev.to/thenjdevopsguy/prerequisites-to-learning-and-understanding-kubernetes-583p">this blog post</a>.</p><h3>Kubernetes Adoption and Incubators</h3><p>For the past year or so, it seems like new Kubernetes projects, products, and incubators are popping up to solve certain problems in the Kubernetes landscape.</p><p>Some are focused on RBAC, others are focused on logging, some are focused on making Kubernetes easier, and everything in-between.</p><p>The CNCF has been working with a lot of these startups and it’s clear that the CNCF is putting a lot of time into Kubernetes in general, but also into the direction that Kubernetes is going in via these products.</p><p>At some point, once a lot more of these startups pop up, there will be a “tool belt” or some kind of “best practices for your environment” with the products that ended up making it out of the “start up” phase and into the enterprise phase. Perhaps this will be all under the umbrella of CNCF or of a “parent company” of sorts for all of the products.</p><h3>Virtual Machines</h3><p>This prediction is probably the most “hmm, I’m not sure about this” prediction. However, I do believe it makes sense because of legacy applications.</p><p>Think about HashiCorp Nomad — it’s an orchestrator and does the same thing as Kubernetes, except Kubernetes only supports containerized apps. Nomad supports any type of app.</p><p>For Kubernetes to stay competitive, this will have to be thought about and ultimately figured out.</p><p>The good news is, there are solutions that help with this.</p><p><a href="https://kubevirt.io/">Kube-virt</a> provides a platform that engineers can build apps for both containers and virtual machines in the same environment. Essentially, it allows you to run virtual machines on Kubernetes.</p><p>This goes back to the hybrid approach from an earlier section. Some organizations aren’t going to want to move everything to the cloud, and to take it a step further, some organizations aren’t going to want to containerize certain legacy applications. However, what they will want is the “power” behind Kubernetes to manage legacy applications.</p><p>This is where tools like Kube-virt comes into play, and I’m certain that others will enter the market as well once Kubernetes becomes used more in production. If Kubernetes is truly a way to manage infrastructure with an API, that means we should be able to manage all infrastructure.</p><h3>Cluster Management</h3><p>Last but certainly not least, there’s cluster management. With everything from:</p><ul><li>Hybrid-cloud</li><li>Multi-cloud</li><li>Clusters on-prem</li><li>Clusters in OpenStack</li><li>All of the different products and solutions</li></ul><p>there’s a lot of infrastructure flying around all over the place.</p><p>Cluster management tools like Rancher and Azure Arc already exist, but they will become more relevant when organizations begin to incorporate multiple Kubernetes clusters. As of right now, the number is around less than 10% of organizations are running over 50 Kubernetes clusters. Because the number is so low, you have to imagine that a lot of organizations are running a few Kubernetes clusters at most, and that doesn’t mean they’re all in production.</p><p>Once Kubernetes becomes the “environment” instead of the “platform”, more and more clusters will be deployed, and cluster management tools will not be an option, they’ll be needed. Also, those cluster management tools will give you the ability to easily manage the internals of your clusters. For example, overall management of RBAC across all of your clusters instead of managing RBAC per cluster.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=33bacfe909cf" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Breaking Down Kubeconfigs (Quick Start)]]></title>
            <link>https://michael-levan.medium.com/breaking-down-kubeconfigs-quick-start-13fb5cddceb4?source=rss-8fa8e88f2ad6------2</link>
            <guid isPermaLink="false">https://medium.com/p/13fb5cddceb4</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[engineering]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Michael Levan]]></dc:creator>
            <pubDate>Thu, 20 Oct 2022 19:15:26 GMT</pubDate>
            <atom:updated>2022-10-20T19:15:26.894Z</atom:updated>
            <content:encoded><![CDATA[<p>In this quickstart guide, you’re going to learn about what consists of a Kubeconfig and what each piece of the YAML means.</p><p>First, capture one of your Kubeconfigs on your local computer. It’s typically under `~/.kube`.</p><p>![Image description](<a href="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/e2rp4el7wq6jc6g6wa2x.png">https://dev-to-uploads.s3.amazonaws.com/uploads/articles/e2rp4el7wq6jc6g6wa2x.png</a>)</p><p>Next, open up the `config` in an editor of your choosing or by using the `cat` command to view it.</p><p>Below is a sample `config`. Let’s break it down by splitting it piece by piece.</p><p>```jsx<br>clusters:<br>- cluster:<br> certificate-authority-data: some_string_here<br> server: <a href="https://kubernetes.docker.internal:6443/">https://kubernetes.docker.internal:6443</a><br> name: docker-desktop<br>- cluster:<br> certificate-authority: /Users/michael/.minikube/ca.crt<br> extensions:<br> — extension:<br> last-update: Thu, 08 Sep 2022 12:50:41 EDT<br> provider: minikube.sigs.k8s.io<br> version: v1.25.1<br> name: cluster_info<br> server: <a href="https://127.0.0.1:64599/">https://127.0.0.1:64599</a><br> name: minikube<br>contexts:<br>- context:<br> cluster: docker-desktop<br> user: docker-desktop<br> name: docker-desktop<br>- context:<br> cluster: minikube<br> extensions:<br> — extension:<br> last-update: Thu, 08 Sep 2022 12:50:41 EDT<br> provider: minikube.sigs.k8s.io<br> version: v1.25.1<br> name: context_info<br> namespace: default<br> user: minikube<br> name: minikube<br>current-context: minikube<br>kind: Config<br>preferences: {}<br>users:<br>- name: docker-desktop<br> user:<br> client-certificate-data: some_string_here<br> client-key-data: some_string_here<br>- name: minikube<br> user:<br> client-certificate: /Users/michael/.minikube/profiles/minikube/client.crt<br> client-key: /Users/michael/.minikube/profiles/minikube/client.key<br>```</p><p>First things first — a `kubeconfig` is how you, the user, interacts with Kubernetes. To interact with a Kubernetes cluster, the `kubeconfig` needs information about your cluster. The first section of the config is the server name, certificate authority, server URL, Kubernetes API version, and metadata information about when the cluster has been updated. This is the actual connection piece to a cluster. You can have more than one cluster listed in a Kubeconfig depending on how many clusters you connect to.</p><p>```jsx<br>clusters:<br>- cluster:<br> certificate-authority-data: some_string_here<br> server: <a href="https://kubernetes.docker.internal:6443/">https://kubernetes.docker.internal:6443</a><br> name: docker-desktop<br>- cluster:<br> certificate-authority: /Users/michael/.minikube/ca.crt<br> extensions:<br> — extension:<br> last-update: Thu, 08 Sep 2022 12:50:41 EDT<br> provider: minikube.sigs.k8s.io<br> version: v1.25.1<br> name: cluster_info<br> server: <a href="https://127.0.0.1:64599/">https://127.0.0.1:64599</a><br> name: minikube<br>```</p><p>Next, there’s the context. The context is the configuration itself to the cluster. Like the cluster section of a config, you can have multiple contexts. The context consist of the username of the user using the Kubernetes cluster, the users default namespace, and the cluster information that the user is connecting to.</p><p>```jsx<br>contexts:<br>- context:<br> cluster: docker-desktop<br> user: docker-desktop<br> name: docker-desktop<br>- context:<br> cluster: minikube<br> extensions:<br> — extension:<br> last-update: Thu, 08 Sep 2022 12:50:41 EDT<br> provider: minikube.sigs.k8s.io<br> version: v1.25.1<br> name: context_info<br> namespace: default<br> user: minikube<br> name: minikube<br>```</p><p>The current context states what Kubernetes cluster inside of the config you have on your computer you’re currently connected to. You can change this to point to another cluster as long as you have appropriate access to said cluster.</p><p>```jsx<br>current-context: minikube<br>```</p><p>The last piece is the config, which is a Kubernetes resource/object itself. It specifies the user that’ll be used to connect to the current Kubernetes cluster, the name of the current Kubernetes cluster in the context, and the authorization (client cert and key) used so the current user that’s associated with the config can connect to the cluster.</p><p>```jsx<br>kind: Config<br>preferences: {}<br>users:<br>- name: docker-desktop<br> user:<br> client-certificate-data: some_string_here<br> client-key-data: some_string_here<br>- name: minikube<br> user:<br> client-certificate: /Users/michael/.minikube/profiles/minikube/client.crt<br> client-key: /Users/michael/.minikube/profiles/minikube/client.key<br>```</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=13fb5cddceb4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kyverno For Kubernetes]]></title>
            <link>https://michael-levan.medium.com/kyverno-for-kubernetes-290c2b7f142?source=rss-8fa8e88f2ad6------2</link>
            <guid isPermaLink="false">https://medium.com/p/290c2b7f142</guid>
            <category><![CDATA[engineering]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Michael Levan]]></dc:creator>
            <pubDate>Thu, 20 Oct 2022 19:14:22 GMT</pubDate>
            <atom:updated>2022-10-20T19:14:22.236Z</atom:updated>
            <content:encoded><![CDATA[<p>Security is arguably one of the largest issues in Kubernetes environments. Other than the fact that almost all engineers are still trying to figure out Kubernetes, once it’s figured out, the next concern is security.</p><p>Red Hat’s most recent report states that 70% of security issues are Kubernetes misconfigurations. Typically, misconfigurations are due to having poor policies or no policies in place.</p><p>That’s where a platform like Kyverno comes into play.</p><p>In this blog post, you’re going to learn about Kyverno, why it’s important, how it’s compared to Open Policy Agent (OPA), and how you can get started with it in production today.</p><p>## What’s Kyverno?</p><p>When you deploy anything to Kubernetes, whether it’s a Pod, ConfigMap, or an entire containerized microservices, you’ll typically want some type of rules around them. For example, one of the best practices that every engineer should implement is to never use the `latest` tag of a container image inside of a Kubernetes Manifest in production. Why? Because the `latest` tag is typically a dev build that’s still being worked out. Instead, engineers should use a specific version of the containerized app.</p><p>Here’s the thing - how do you block engineers from using the `latest` tag?</p><p>That’s what a policy manager like Kyverno does.</p><p>Kyverno manages policies, whether they’re for security or general best practices, in your Kubernetes environment. You can set up policies for something like the `latest` tag example above, or something more security-centric like verifying container images for a software supply chain security. Kyverno works outside of standard Kubernetes Manifests as well. You can use overlays like Kustomize or Helm and implement Kyverno policies.</p><p>### What About OPA?</p><p>Open Policy Agent (OPA) is gaining increased popularity in the policy management community, primarily because it’s vendor agnostic. Meaning, you can create policies for any environment, not just Kubernetes.</p><p>With Kyverno, it’s only for Kubernetes.</p><p>If you need policy management outside of Kubernetes, Kyverno won’t work and you may be better off going the OPA route.</p><p>What should be said for Kyverno, which is a major upside over OPA, is that Kyverno policies are created via Kubernetes Manifests. OPA policies are created with Rego, a configuration language that sort of looks like Go (golang). With that being said, if you don’t already know Rego, implementing OPA may be a burden. Where-as with Kyverno, you’re already used to Kubernetes Manifests, so it’s less of a lift.</p><p>## How Do Policies Work?</p><p>A policy manager like Kyverno or OPA are Kubernetes Controllers that run inside of a Kubernetes cluster. Because of the underlying features of a Kubernetes Controller, like managing the state of deployed resources, it makes sense for a policy manager to be a Controller. The whole idea is to manage policies, and you can’t manage policies without proper state in place.</p><p>With policy managers, it’s all about webooks. There are policies in place inside of the Kubernetes cluster and whenever a request is made, like creating a Kubernetes Deployment resource, the webhook is received and makes a callback to the Kubernetes API server. It then checks to confirm that all of the policies in place are passed and if they are, the Kubernetes resource will be created. If they aren’t, the resource will be blocked.</p><p>## Deploying Kyverno</p><p>When you deploy Kyverno in a Kubernetes cluster, a few Kubernetes resources are created:</p><p>- A Namespace where Kyverno will live<br>- The Kyverno CRD’s<br>- The Kyverno container image<br>- The Kyverno services<br>- Configmaps<br>- A service account with cluster roles and rolebindings. Please note that the service account does contain `create` and `delete` roles.</p><p>You can find the entire Kubernetes configuration for more detail here: [<a href="https://raw.githubusercontent.com/kyverno/kyverno/main/config/install.yaml](https://raw.githubusercontent.com/kyverno/kyverno/main/config/install.yaml)">https://raw.githubusercontent.com/kyverno/kyverno/main/config/install.yaml](https://raw.githubusercontent.com/kyverno/kyverno/main/config/install.yaml)</a></p><p>Now that you know what the Kubernetes configuration consists of, let’s look at how to install and configure Kyverno.</p><p>First, run the Kubernetes Manifest installation.</p><p>```jsx<br>kubectl create -f <a href="https://raw.githubusercontent.com/kyverno/kyverno/main/config/install.yaml">https://raw.githubusercontent.com/kyverno/kyverno/main/config/install.yaml</a><br>```</p><p>If you’d prefer to use a Helm Chart, use the code below.</p><p>```jsx<br>helm repo add kyverno <a href="https://kyverno.github.io/kyverno/">https://kyverno.github.io/kyverno/</a></p><p>helm repo update</p><p>helm install kyverno kyverno/kyverno -n kyverno --create-namespace<br>```</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=290c2b7f142" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>