<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Arash on Medium]]></title>
        <description><![CDATA[Stories by Arash on Medium]]></description>
        <link>https://medium.com/@kubernauts?source=rss-d6c3d29f67f8------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*hNEgekjWuPjCoOwBCKyGxw.png</url>
            <title>Stories by Arash on Medium</title>
            <link>https://medium.com/@kubernauts?source=rss-d6c3d29f67f8------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 10:52:15 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@kubernauts/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Aia at the Gate: AI Proposes, Humans Hold the Key]]></title>
            <link>https://blog.kubernauts.io/aia-at-the-gate-ai-proposes-humans-hold-the-key-e7466b2f516b?source=rss-d6c3d29f67f8------2</link>
            <guid isPermaLink="false">https://medium.com/p/e7466b2f516b</guid>
            <category><![CDATA[souvernity]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[platform-engineering]]></category>
            <category><![CDATA[cloud-native]]></category>
            <category><![CDATA[artificail-intelligence]]></category>
            <dc:creator><![CDATA[Arash]]></dc:creator>
            <pubDate>Tue, 18 Aug 2026 10:01:42 GMT</pubDate>
            <atom:updated>2026-08-19T14:40:02.545Z</atom:updated>
            <content:encoded><![CDATA[<p><em>How OpenKubes is designing safe agentic cluster lifecycle management</em></p><p><em>By Arash Kaffamanesh</em></p><blockquote><strong>TL;DR:</strong> OpenKubes is proposing a human-approved architecture in which AI may diagnose and draft cluster lifecycle changes but cannot authorize, accept, execute, or reconcile them. The ADR is public and remains Proposed; implementation and acceptance evidence still have to follow.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XcfK8BgPN9sAEMgwkWjPfw.png" /></figure><p>Aia arrives with a purple backpack that is far too full.</p><p>Inside are plans, observations, possible improvements, and probably several ideas nobody asked for yet. She reads quickly, compares clues, notices when a path no longer matches the Book of Promises, and drafts a better one.</p><p>Then she reaches the gate.</p><p>Behind it, a proposal can become a real change. A cluster can be created, upgraded, or deleted. Infrastructure can move. Machines can disappear. The consequences are no longer part of a conversation; they become part of the world.</p><p>Aia looks for the key. The human builder is holding it.</p><p>This is not because the builder distrusts Aia. It is because trust works best when every participant has a clear responsibility. Aia helps with thinking. Sensors check facts. Contracts define the promises. Humans carry responsibility for consequential decisions.</p><p>That scene from the <a href="https://oki.openkubes.org/#start">Oki Universe</a> has become the most useful explanation of how we want AI to participate in the OpenKubes control plane.</p><p>It is also the story behind our new architecture decision, <a href="https://github.com/openkubes/openkubes/blob/main/architecture/decisions/ADR-Platform-035-hybrid-intent-and-control-plane-execution.md">ADR-Platform-035</a>.</p><h3>The Question That Sent Us to the Gate</h3><p>Our starting question sounded simple:</p><blockquote>Could an AI agent replace the custom runner that helps provision and manage OpenKubes workload clusters?</blockquote><p>The runner contains deterministic Go and Python logic. It validates declarations, submits lifecycle changes, observes Cluster API and related controllers, and records evidence. An agent can reason across more context, explain failures, and translate natural language into configuration. Why maintain both?</p><p>Because this is not really a choice between two components.</p><p>The runner itself is replaceable. It is one implementation of a deeper architectural role: the <strong>Contract Executor</strong>. The safety property we need to preserve is not a specific container, language, or API. It is the boundary between reasoning about a change and authoritatively accepting and executing that change.</p><p>Once we framed the problem that way, the answer became clear:</p><blockquote>The agent may help create intent. It must not become the authority, executor, or reconciler merely because it can call tools.</blockquote><h3>Aia Is an Author, Not an Authority</h3><p>In the Oki Universe, the metaphors already line up with the architecture.</p><p>Aia and her backpack represent the untrusted authoring layer and its candidate proposals. The Book of Promises represents versioned OpenKubes Contracts. The sensors represent read-only diagnostics and evidence. Reviewing the plan together represents deterministic canonicalization and a human-readable semantic review. The builder holding the key is the authenticated Human Approver. The gate represents the Policy, Authority, and Contract Executor boundary. Beyond the gate, controllers reconcile the accepted state across the islands.</p><p>The resulting principle is simple enough to remember:</p><blockquote><strong>Aia helps us think. Humans approve. Policies authorize. Executors submit. Controllers reconcile.</strong></blockquote><p>The implementation behind that sentence is deliberately less magical.</p><h3>From a Candidate to a Real Cluster Change</h3><p>An agent-originated lifecycle proposal follows the same control-plane model as any other request. It does not receive a privileged shortcut.</p><pre>User and Aia<br>    -&gt; untrusted candidate input<br>    -&gt; server-attested proposal provenance<br>    -&gt; deterministic canonicalization and transition planning<br>    -&gt; authenticated human review of the exact semantic change<br>    -&gt; policy authorization bound to that exact transition<br>    -&gt; Executor verification and single-use authorization claim<br>    -&gt; atomic compare-and-swap acceptance by the Authority Profile<br>    -&gt; asynchronous reconciliation by GitOps and lifecycle controllers<br>    -&gt; generation-correlated Conditions and durable evidence</pre><p>Each arrow matters.</p><p>The Candidate Proposal service records who produced the proposal using server-attested workload identity. An agent cannot simply label its own work as human-authored.</p><p>The Canonicalizer converts flexible authoring input into one deterministic representation. It rejects duplicate keys, ambiguous fields, unsupported schema versions, and parser-dependent interpretations. Semantically identical input must produce identical canonical bytes and the same digest.</p><p>The human reviews the effective change, including defaults, rather than approving an opaque hash. The human-visible review artifact is derived from the exact canonical predecessor and requested revision. Its digest, renderer version, canonicalization profile, and request digest remain bound together.</p><p>The Policy Authority verifies that protected approval and issues a single-use decision for one exact transition. The Contract Executor independently verifies and durably claims that authorization. Only then may the selected Authority Profile atomically accept the new desired revision.</p><p>Acceptance changes authority. It does not prove convergence.</p><p>Cluster API, infrastructure providers, Cluster Enablement, GitOps, and other responsible controllers still reconcile asynchronously. Their revision- and generation-correlated Conditions determine whether the requested state was realized. An LLM response and a successful runner exit code do not.</p><h3>Human Approval Must Mean More Than Clicking “Approve”</h3><p>Digest binding solves only part of the problem. A person cannot meaningfully approve a digest they do not understand.</p><p>The security property we need is often called <strong>WYSIWYS: What You See Is What You Sign</strong>. The semantic change shown to the reviewer, the canonical representation used to compute the request digest, and the revision later presented for acceptance must remain inseparably correlated.</p><p>If the proposal changes after review, the approval becomes invalid. If the canonicalizer profile changes, the proposal must be reviewed again. If the human-visible artifact changes, its protected binding changes too. A later deployment of the renderer does not rewrite history; the immutable versioned review artifact remains the evidence of what the person actually approved.</p><p>This may sound strict for creating a Kubernetes cluster. It is exactly as strict as we want it to be for deleting one.</p><h3>Why the Agent Never Sees the Full Executor API</h3><p>One tempting design is to expose the entire mutating API through MCP and rely on RBAC to deny dangerous calls.</p><p>We rejected that design.</p><p>An authorization defect, confused-deputy path, or accidentally broadened role would turn an advertised tool into a real mutation capability. Instead, the agent-facing MCP surface is constructed from two intentionally narrow contracts:</p><ul><li>the existing Read-Only Platform Diagnostics Contract; and</li><li>a separate, non-authoritative Candidate Proposal Contract.</li></ul><p>It is not produced by taking a powerful Executor API and subtracting methods until it looks safe. Mutating Executor and Authority operations live behind separate identity, authorization, and network boundaries. They do not appear in the agent-facing tool surface at all.</p><p>Prompts and tool descriptions remain helpful interaction material. They are never validation logic, policy, provenance, authorization, or readiness evidence.</p><h3>Prompt Injection Does Not Disappear at the Gate</h3><p>Read-only is not the same as harmless.</p><p>Poisoned documentation, logs, diagnostic output, or retrieval context can influence the model’s next proposal. An agent can create a syntactically valid but undesirable candidate. It can consume review capacity. It may expose information available to its read identity. And a human reviewer can still be deceived into approving a manipulated proposal.</p><p>Human approval bounds this risk; it does not eliminate it.</p><p>That is why the design also requires least-privilege read identities, server-attested provenance, quotas, rate limits, audit records, deterministic canonicalization, explicit semantic review, and a policy decision bound to the exact transition. The gate is a system of independent controls, not a ceremonial button.</p><h3>What Exists Today — and What Is Still a Target</h3><p>We want to be precise about current reality.</p><p>OpenKubes has already validated the read-only diagnostic foundation in the proof of concept: agents access platform state through scoped contracts and dedicated workload identities, while writes and Secrets are denied. This boundary is documented in ADRs. OpenClaw remains an optional broad assistant, while kagent is the first diagnostics implementation behind the OpenKubes-owned contract. The candidate-to-acceptance lifecycle path defined by ADRs remains target architecture and still requires its acceptance evidence.</p><p>ADR-Platform-035 is the next architectural step. It has been merged into the public repository with <strong>Status: Proposed</strong>. Merging the document means the design is available for review and implementation. It does not mean the end-to-end Candidate Proposal, approval, Authority, and Executor path already exists or has passed conformance.</p><p>Before the ADR can move to Accepted, OpenKubes requires reviewed evidence for fourteen criteria. They cover provenance and identity isolation, GitOps reconciliation isolation, human-approval enforcement, digest and transition locking, stale-write rejection, replay and crash recovery, MCP attack-surface verification, Condition correctness, WYSIWYS review, canonical-form correlation, parser-differential resistance, and renderer binding.</p><p>This distinction matters. We are publishing an architecture and an evidence bar, not announcing an autonomous production feature.</p><h3>Why We Are Publishing Before We Build</h3><p>Agentic infrastructure management is moving quickly. The easy demonstrations are impressive: give a model cluster-admin, ask it to fix something, and watch the terminal.</p><p>The harder questions begin after the demo:</p><ul><li>Who owns the desired state?</li><li>What exactly did the human approve?</li><li>Can the model alter the evidence shown to the reviewer?</li><li>What happens after a stale plan or a lost response?</li><li>Can a candidate branch ever be reconciled accidentally?</li><li>How do we prove that an MCP surface contains no mutation path?</li><li>Which state remains authoritative after the Executor crashes?</li></ul><p>We would rather expose those questions now than hide them behind an apparently seamless chat interface.</p><p>The implementation work is tracked under OpenKubes Epics. We are deliberately starting with this article and community discussion. Feedback that changes a contract will go back through the ADR process. Public feedback is design input; it does not silently become platform authority either.</p><h3>We Would Like Your Feedback</h3><p>If you operate Kubernetes platforms, build agent frameworks, design authorization systems, or work on GitOps control planes, we would especially value your perspective:</p><ol><li>Where do you place the human approval boundary for agent-originated infrastructure changes?</li><li>Have you implemented WYSIWYS review for declarative configuration or GitOps changes?</li><li>How do you perform atomic, transition-correlated acceptance with Git as authority?</li><li>Do your agent-facing APIs expose a safe contract by construction, or a filtered view of a more powerful API?</li><li>Which lifecycle changes, if any, could eventually qualify for scoped standing approval?</li><li>What evidence would you require before trusting this design in production?</li></ol><p>We have intentionally excluded autonomous and standing approval from this ADR. If we ever introduce it, it will require a separate decision, a bounded operation scope, an explicit threat model, and its own acceptance evidence.</p><h3>The Key Stays Visible</h3><p>Aia’s backpack will keep getting fuller. That is a feature.</p><p>We want agents that can read more evidence, connect more clues, explain failures, and prepare better proposals. We want the interaction to feel natural and the underlying platform to remain boringly explicit about authority.</p><p>The goal is not to keep AI away from infrastructure engineering. The goal is to give AI a valuable role without allowing convenience to erase responsibility.</p><p>At the OpenKubes gate, Aia brings the plan. The sensors bring the evidence. The Contracts define the promises. A human turns the key. Deterministic systems carry the accepted change into the world.</p><p>That is how we intend to make agentic cluster lifecycle management useful, reviewable, and worthy of trust.</p><h3>References</h3><ul><li><a href="https://github.com/openkubes/openkubes/blob/main/architecture/decisions/ADR-Platform-035-hybrid-intent-and-control-plane-execution.md">ADR-Platform-035: Hybrid Intent and Control-Plane Execution Architecture</a></li><li><a href="https://github.com/openkubes/openkubes/pull/258">ADR-Platform-035 pull request and design discussion</a></li><li><a href="https://github.com/openkubes/openkubes/blob/main/architecture/decisions/ADR-Platform-015-agentic-ai.md">ADR-Platform-015: Agentic AI</a></li><li><a href="https://github.com/openkubes/openkubes/blob/main/architecture/decisions/ADR-Platform-021-read-only-platform-diagnostics-contract.md">ADR-Platform-021: Read-Only Platform Diagnostics Contract</a></li><li><a href="https://oki.openkubes.org/#start">Oki Universe — Oki und das Geheimnis der Inseln</a></li><li><a href="https://github.com/openkubes/openkubes">OpenKubes on GitHub</a></li></ul><p><em>Author’s note: This article was developed from ADR-Platform-035 through a human-led review involving Codex, Claude, and Gemini. The architectural decision, editorial framing, and final publication authority remain with the author.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e7466b2f516b" width="1" height="1" alt=""><hr><p><a href="https://blog.kubernauts.io/aia-at-the-gate-ai-proposes-humans-hold-the-key-e7466b2f516b">Aia at the Gate: AI Proposes, Humans Hold the Key</a> was originally published in <a href="https://blog.kubernauts.io/">Kubernauts</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Immortal Mind]]></title>
            <link>https://blog.kubernauts.io/the-immortal-mind-d960cc83c065?source=rss-d6c3d29f67f8------2</link>
            <guid isPermaLink="false">https://medium.com/p/d960cc83c065</guid>
            <category><![CDATA[knowledge-management]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[platform-engineering]]></category>
            <dc:creator><![CDATA[Arash]]></dc:creator>
            <pubDate>Sat, 04 Jul 2026 21:47:26 GMT</pubDate>
            <atom:updated>2026-07-04T23:12:41.422Z</atom:updated>
            <content:encoded><![CDATA[<h3>A Manifesto for Organizational Intelligence That Survives</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*H1k2umBE273hYmwWQbpgNA.png" /></figure><h3>A Different Question</h3><p>For centuries, humanity has asked a single question:</p><p><strong>How do we preserve knowledge?</strong></p><p>We built libraries. We built archives. We built databases. We built clouds.</p><p>And yet knowledge continues to disappear.</p><p>Engineers retire. Teams reorganize. Projects are abandoned. Companies are acquired. Entire decades of hard-won experience vanish with the people who created it.</p><p>The problem is not storage.</p><p><strong>The problem is continuity.</strong></p><h3>Infrastructure Can Already Heal Itself</h3><p>We have been building self-healing digital infrastructure with OpenKubes.</p><p>Git stores the desired state. Kubernetes reconciles reality back to that state. Servers fail. Clusters disappear. Entire environments get rebuilt from scratch — automatically, without human intervention.</p><p>Infrastructure survives because it remembers what it should be.</p><p>We call this the Immortal Platform.</p><pre>Git is the contract.<br>Kubernetes is the enforcer.</pre><p>Target recovery time: under ten minutes. No runbook. No 3am call. No tribal knowledge required.</p><p>When we built this, we thought we were solving an infrastructure problem.</p><p>We were actually solving the beginning of a much larger problem.</p><h3>What About Intelligence?</h3><p>Organizations face a different kind of failure — one that no monitoring system detects, no alert fires for, and no on-call engineer gets paged about.</p><p><strong>The slow disappearance of organizational intelligence.</strong></p><p>What happens when the engineer who designed the system retires?</p><p>Who remembers why that architectural decision was made in 2019?</p><p>Who remembers the three failed approaches before the solution that worked?</p><p>Who remembers the lessons from the production incident that took down the factory floor for six hours?</p><p>Most organizations have no answer.</p><p>Their infrastructure is documented. Their intelligence is not.</p><p>We have spent ten years building and operating Kubernetes platforms across automotive plants, financial institutions, industrial facilities, and government agencies. We have seen the same pattern repeat itself dozens of times: a new team inherits a platform built by people who are no longer there. They spend months — sometimes years — reverse-engineering decisions that took the original team weeks to make. They repeat mistakes that were already made and documented somewhere no one can find. They abandon patterns that worked because nobody explained why they were there.</p><p>The cost is not just time. It is confidence. Every inherited system that lacks its original context becomes a system nobody fully trusts, nobody fully understands, and nobody wants to touch.</p><p>This is not a technology problem. This is a memory problem.</p><h3>The Immortal Mind</h3><p>OpenKubes AI begins with a simple idea:</p><p><strong>Knowledge should be as durable as infrastructure.</strong></p><p>The founding principle of OpenKubes is that the platform owns contracts, not components. Infrastructure components come and go — the contracts they fulfill persist. The Immortal Mind extends that same principle to knowledge: decisions, context, and lessons are contracts too. Individual people, teams, and documents come and go — the organizational intelligence they carry must persist.</p><p>If we can build infrastructure that heals itself — that reads a desired state from Git, reconciles toward it, and recovers from failure without human intervention — then we can build intelligence systems that do the same.</p><p>Not storing documents in a folder that nobody reads.</p><p>Not writing runbooks that become outdated before the ink dries.</p><p>But genuinely <strong>living knowledge</strong> — continuously updated, continuously reconciled, continuously connected to the systems and decisions it describes.</p><p>Just as Kubernetes reconciles infrastructure to its desired state, future AI systems can continuously reconcile organizational knowledge to its current reality.</p><pre>Git is the contract.<br>Kubernetes is the enforcer.<br>AI is the memory.</pre><p>To be precise about what we mean: the memory itself does not live inside a model. It lives in Git, in architecture decision records, in the knowledge graph, and in the context that evolves with the platform. AI is what connects, understands, and makes that memory accessible. Models will come and go. The organizational memory must endure.</p><h3>The Architecture of Organizational Immortality</h3><p>OpenKubes AI envisions four foundational layers — not as a product announcement, but as an architectural direction:</p><h3>Layer 1: Knowledge Graph</h3><p>A structured, living representation of organizational knowledge.</p><p>People. Systems. Projects. Decisions. Failures. Lessons. Relationships. Dependencies. Evolution.</p><p>Not a static diagram. A continuously updated graph that reflects the current state of the organization and its history — connected to the actual infrastructure it describes.</p><p>When a cluster is deployed, the knowledge graph knows why. When an architectural decision is made, it is captured — not in a document folder nobody will find, but in a structured, queryable, AI-accessible form.</p><figure><img alt="Not a mockup: the OpenKubes knowledge graph, extracted from Git by a 200-line script — every decision, component, and commit, connected." src="https://cdn-images-1.medium.com/max/1024/1*10E937a-KvX1Lv375AMZSw.png" /><figcaption>Not a mockup: the OpenKubes knowledge graph, extracted from Git by a 200-line script — every decision, component, and commit, connected.</figcaption></figure><p>Explore the interactive version: <a href="https://kubernauts.de/en/openkubes/openkubes_knowledge_graph_force_layout.html">https://kubernauts.de/en/openkubes/openkubes_knowledge_graph_force_layout.html</a></p><h3>Layer 2: Context Store</h3><p>GitOps for knowledge.</p><p>Architecture decision records. Runbooks. Postmortem analyses. Design rationale. Lessons learned. Every significant decision — versioned, auditable, and connected to the code and infrastructure it influenced.</p><p>Not documentation as an afterthought. Documentation as infrastructure — with the same discipline, the same tooling, the same lifecycle.</p><p>When you git blame a Kubernetes manifest, you can trace it back to the incident that caused it. When you ask why a system is designed the way it is, the answer is a git log away.</p><p>This is not theory. It is how OpenKubes is already built: every architectural decision lives as a versioned ADR in the platform repository, and every deviation from upstream in our deployment guides is recorded with its reason and its operational impact. The Context Store simply makes that discipline queryable — and permanent.</p><h3>Layer 3: Model Runtime</h3><p>Open AI runtimes deployed anywhere — on the same infrastructure that runs your workloads.</p><p>Cloud. Edge. Air-gapped factory floors. Sovereign government infrastructure.</p><p>The intelligence follows the workload. Not locked in a vendor’s cloud. Not dependent on an external API that may change, disappear, or become unavailable in an air-gapped facility.</p><p>The same platform engineering principles that make OpenKubes infrastructure sovereign make OpenKubes AI intelligence sovereign.</p><h3>Layer 4: Immortal Platform Integration</h3><p>The platform heals itself. The intelligence remembers itself. The system continuously rebuilds both.</p><p>When a cluster fails and is reprovisioned on fresh bare metal, the AI layer knows the history of that cluster — every deployment, every incident, every change. The infrastructure is new. The memory is intact.</p><h3>Beyond Automation</h3><p>It is important to say clearly what this is not.</p><p>This is not a vision of autonomous machines replacing human engineers.</p><p>This is not digital immortality for individuals.</p><p>This is not artificial general intelligence.</p><p><strong>This is preservation of organizational intelligence.</strong></p><p>A future where critical knowledge no longer disappears when individuals leave.</p><p>A future where the organization remembers — not just what it built, but why it built it.</p><p>The engineer retires. The knowledge stays.</p><p>The team disbands. The context remains.</p><p>The company is acquired. The intelligence survives.</p><h3>Why This Matters for Industrial Systems</h3><p>In a factory, a hospital, a power grid, or a government agency — the stakes of lost knowledge are not measured in developer productivity.</p><p>They are measured in production downtime, patient safety, grid stability, and national security.</p><p>We have seen what happens when a factory floor loses the engineer who understood the control system. We have seen what happens when a hospital’s IT team inherits infrastructure nobody documented. We have seen what happens when a critical system needs to be rebuilt and nobody remembers the original architecture rationale.</p><p><strong>The infrastructure survived. The intelligence did not. The consequences were real.</strong></p><p>This is why OpenKubes AI is not a feature.</p><p>It is a responsibility.</p><h3>The Complete Vision</h3><p>When we look at where OpenKubes is going, we see a platform designed not merely for uptime — but for <strong>continuity</strong>:</p><pre>OpenKubes IMP        → Infrastructure survives<br>OpenKubes AI         → Knowledge survives<br>OpenKubes Robotics   → Actions survive</pre><p>The Robotics layer is not hypothetical. Open-RMF — the open robotics middleware framework for fleet management, traffic coordination, task dispatching, and simulation — runs today as a reference robotics workload on the OpenKubes platform, deployed through the same GitOps-based operational model as everything else, consistently across local, edge, bare-metal, and public cloud environments.</p><p>Together these layers form something that has never existed before:</p><p><strong>A platform where systems, knowledge, and actions persist — regardless of hardware failures, software updates, team changes, or the passage of time.</strong></p><p>Infrastructure that heals itself. Intelligence that remembers itself. Systems that evolve themselves.</p><h3>An Invitation</h3><p>This manifesto is not a product announcement.</p><p>It is a direction.</p><p>OpenKubes AI does not exist yet as a shipping product. But the architectural foundation does — in the Git repositories, the Crossplane compositions, the Cluster API providers, the running robotics reference workload, and the knowledge accumulated across ten years of building and operating critical Kubernetes infrastructure.</p><p>The Immortal Mind is where that foundation leads.</p><p>If you are building systems that cannot afford to forget — factory automation platforms, critical infrastructure, sovereign AI systems, industrial knowledge management — we want to build this with you.</p><p>Not for you. <strong>With you.</strong></p><p>Because the most important knowledge to preserve is the knowledge we build together.</p><blockquote>Git is the contract.<em> </em>Kubernetes is the enforcer.<em> </em>AI is the memory.</blockquote><blockquote><strong><em>Together, they create systems that do not merely survive failure.</em></strong><em> </em><strong><em>They learn from it.</em></strong></blockquote><p>🔗 <a href="https://github.com/openkubes/openkubes">github.com/openkubes/openkubes</a> 📊 <a href="https://kubernauts.de/en/openkubes/OpenKubes-Presentation.html">OpenKubes Platform Presentation</a> 📖 <a href="https://blog.kubernauts.io/">blog.kubernauts.io</a> 🎯 <a href="https://kubernauts.atlassian.net/browse/OK-30">OpenKubes Roadmap: OK-30 Immortal Platform</a></p><p><em>Arash Kaffamanesh is the founder of Clouds Sky GmbH &amp; Kubernauts GmbH and has been building and operating Kubernetes platforms for over ten years across automotive, industrial, financial, and healthcare environments. He is the creator of OpenKubes — the open platform for self-healing sovereign Kubernetes infrastructure.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d960cc83c065" width="1" height="1" alt=""><hr><p><a href="https://blog.kubernauts.io/the-immortal-mind-d960cc83c065">The Immortal Mind</a> was originally published in <a href="https://blog.kubernauts.io/">Kubernauts</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kubernetes Anywhere. Make it. — How We Built OpenKubes with Crossplane and Cluster API]]></title>
            <link>https://blog.kubernauts.io/kubernetes-anywhere-make-it-how-we-built-openkubes-with-crossplane-and-cluster-api-b35062c54c5f?source=rss-d6c3d29f67f8------2</link>
            <guid isPermaLink="false">https://medium.com/p/b35062c54c5f</guid>
            <category><![CDATA[intralogistik]]></category>
            <category><![CDATA[robotics]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Arash]]></dc:creator>
            <pubDate>Tue, 09 Jun 2026 17:54:48 GMT</pubDate>
            <atom:updated>2026-06-11T20:19:45.307Z</atom:updated>
            <content:encoded><![CDATA[<p><em>By Arash Kaffamanesh · Kubernauts GmbH</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dqDPhrQpk5jfcO75HxGwZg.png" /></figure><p>Most Kubernetes stories start the same way.</p><p>A team deploys their first cluster. Everything works. Then comes the second cluster. And the third. Then a request from the industrial IoT team. Then the edge deployments. Then someone asks about AWS. Suddenly, you are managing eight different tools, twelve different workflows, and nobody really knows what runs where.</p><p>We have been living this story for ten years across automotive, industrial, financial, and healthcare environments. And a few months ago, we decided to do something about it.</p><p>We built <strong>OpenKubes</strong>.</p><h3>The Problem We Were Solving</h3><p>The question we kept hearing was not “can we run Kubernetes?” — everyone can run Kubernetes today.</p><p>The real question was: <strong>“Can we run it the same way everywhere?”</strong></p><p>Same process for on-premises bare metal and AWS. Same day-two operations for an edge factory site and a cloud-native microservices platform. Same team, same tooling, same mental model — regardless of where the workload runs.</p><p>That is a much harder problem.</p><h3>What We Built</h3><p>OpenKubes is not a new Kubernetes distribution. It is a <strong>platform engineering toolkit</strong> — a thin, opinionated layer on top of proven open-source projects that your team already knows.</p><p>At its core, OpenKubes combines three ideas:</p><p><strong>One:</strong> Everything is a Kubernetes resource. Clusters, virtual machines, ingress controllers, TLS certificates — all declared as YAML, all reconciled by Crossplane. No manual steps, no imperative scripts that only the senior engineer understands.</p><p><strong>Two:</strong> Everything is operated via make. Not because Make is glamorous, but because it is universal. Any engineer on any machine can run make deploy cluster=ok1 without reading a 40-page runbook first.</p><p><strong>Three:</strong> The platform abstracts the provider, not the operator. You still own your infrastructure. OpenKubes just makes sure you can describe it once and apply it anywhere.</p><h3>The Technology Behind It</h3><p>We chose two CNCF projects as the foundation:</p><p><strong>Crossplane</strong> acts as the platform API layer. It extends Kubernetes with custom resource definitions that represent your infrastructure as first-class objects. Want to create a virtual machine? Apply a OpenKubesVMClaim. Want a complete Kubernetes cluster with CNI, load balancer, and ingress? Apply a KubeVirtClusterClaim. Crossplane handles the rest.</p><p><strong>Cluster API (CAPI)</strong> handles the Kubernetes cluster lifecycle. It models clusters, control planes, and worker nodes as Kubernetes objects — which means GitOps, drift detection, and reconciliation work out of the box.</p><p>Together they give you something remarkable: <strong>infrastructure that heals itself</strong>. If a node disappears, Cluster API recreates it. If a Helm release drifts from its desired state, Crossplane reconciles it. The platform is always moving toward the state you declared, not the state it happens to be in.</p><h3>What It Looks Like in Practice</h3><p>Here is what deploying a production-ready Kubernetes cluster looks like today with OpenKubes:</p><pre>make deploy         cluster=ok1   # ~2 minutes → Kubernetes cluster<br>make kubeconfig     cluster=ok1   # ~5 seconds  → kubeconfig saved<br>make manager-deploy cluster=ok1   # ~30 seconds → Headlamp UI<br>make ingress-setup  cluster=ok1   # ~30 seconds → Traefik ingress<br>make cert-setup     cluster=ok1   # ~90 seconds → Let&#39;s Encrypt TLS</pre><p>Four minutes later, https:<em>//headlamp.openkubes.ai</em> returns HTTP/2 200 with a valid TLS certificate.</p><p>No manual steps. No YAML written by hand. No undocumented tribal knowledge.</p><p>This is what platform engineering looks like when it is working.</p><h3>The Lessons We Learned</h3><p><strong>Abstractions only work when they are consistent.</strong> The moment you have two ways to do the same thing, engineers will find both of them and combine them in ways you did not anticipate. Every operation in OpenKubes goes through make. That constraint is a feature.</p><p><strong>Crossplane compositions are powerful but unforgiving.</strong> Writing a Go-template composition that creates a KubeVirt DataVolume, Service, and VirtualMachine atomically across two clusters took longer than expected. But once it worked, it never needed to be touched again. The investment pays off over time.</p><p><strong>Nested virtualization has limits.</strong> Running KubeVirt inside KubeVirt VMs (our lab setup) means MetalLB Layer 2 advertisement does not propagate to the physical network. We solved this by routing through the INFRA cluster’s existing MetalLB LoadBalancer — no MetalLB on workload clusters at all. Sometimes the pragmatic solution is better than the theoretically correct one.</p><p><strong>Platform engineering is product engineering.</strong> The Makefile is not just a convenience. It is the user interface of the platform. Every target is a feature. Every error message is UX copy. Every ✅ at the end of a command is a tiny moment of delight. Treat it that way.</p><h3>Where We Are Going</h3><p>OpenKubes v1.0.4 is available today on GitHub — Community Preview, Apache 2.0.</p><p>The roadmap has three clear directions:</p><p><strong>OpenKubes Anywhere</strong> — the same make deploy interface for EKS, AKS, and GKE. One platform API, multiple cloud providers. The user should never need to know which Cluster API provider is running underneath.</p><p><strong>OpenKubesMetal</strong> — bare metal lifecycle management via Metal3. For the factory floors, industrial edge sites, and sovereign cloud environments where virtualization is not an option.</p><p><strong>OpenKubes Robotics</strong> — Kubernetes for autonomous systems. Open-RMF, ROS2, fleet orchestration, and industrial AI — all managed through the same platform model.</p><p>The vision is simple:</p><blockquote><strong><em>Deploy a complete robot factory site from Git.</em></strong></blockquote><p>We are not there yet. But we know exactly where we are going.</p><h3>Try It</h3><p>OpenKubes is open source and available today:</p><p>🔗 <a href="https://github.com/openkubes/openkubes">github.com/openkubes/openkubes</a></p><p>📊 <a href="https://kubernauts.de/en/openkubes/OpenKubes-Presentation.html">OpenKubes Platform Presentation</a></p><p>If you are running Kubernetes in production — especially across mixed environments — we would love to hear from you.</p><p><em>Platform Engineering. Sovereign Infrastructure. Kubernetes Anywhere.</em></p><p><em>Arash Kaffamanesh is the founder of Clouds Sky GmbH &amp; Kubernauts GmbH and has been building and operating Kubernetes platforms for over ten years across automotive, industrial, financial, and healthcare environments.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b35062c54c5f" width="1" height="1" alt=""><hr><p><a href="https://blog.kubernauts.io/kubernetes-anywhere-make-it-how-we-built-openkubes-with-crossplane-and-cluster-api-b35062c54c5f">Kubernetes Anywhere. Make it. — How We Built OpenKubes with Crossplane and Cluster API</a> was originally published in <a href="https://blog.kubernauts.io/">Kubernauts</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kubernautic Data Platform for DataOps & MLOps]]></title>
            <link>https://blog.kubernauts.io/kubernautic-data-platform-for-dataops-mlops-bc719cde38a4?source=rss-d6c3d29f67f8------2</link>
            <guid isPermaLink="false">https://medium.com/p/bc719cde38a4</guid>
            <category><![CDATA[dataops]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[iot]]></category>
            <dc:creator><![CDATA[Arash]]></dc:creator>
            <pubDate>Wed, 20 Oct 2021 21:45:46 GMT</pubDate>
            <atom:updated>2021-10-20T22:30:14.035Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/512/1*4xxxS-KnyYYGaQZ-fN3B_A.png" /><figcaption>Kubernautic Data Platform base components</figcaption></figure><h4>Introduction</h4><p>Every digital transformation needs a data platform to transform data and master the challenges of <strong>Data Operations</strong> referred as <strong>DataOps</strong> to bring Data, Apps and Processes made by humans and machines together.</p><p>To make data accessible in real time to the right data scientists by decoupling business decisions from the underlying infrastructure, organizations need to remove bottlenecks from their data projects by implementing a Data Platform based on the best DataOps practices.</p><h4>DataOps Defined</h4><p>According to Wikipedia <a href="https://en.wikipedia.org/wiki/DataOps">DataOps</a> was first introduced by Lenny Liebmann, Contributing Editor, InformationWeek, in a blog post on the IBM Big Data &amp; Analytics Hub titled “3 reasons why DataOps is essential for big data success” on June 19, 2014.</p><p>DataOps is an automated, process-oriented methodology, used by analytic and data teams, to improve the quality and reduce the cycle time of data analytics. While DataOps began as a set of best practices, it has now matured to become a new and independent approach to data analytics.</p><h3>DataOps Re-Defined</h3><p>Eckerson Group re-defines DataOps as follow, which we like so much:</p><p>“DataOps is an engineering methodology and set of practices designed for rapid, reliable, and repeatable delivery of production-ready data and operations-ready analytics and data science models. DataOps enhances an advanced governance through engineering disciplines that support versioning of data, data transformations, data lineage, and analytic models. DataOps supports business operational agility with the ability to meet new and changing data and analysis needs quickly. It also supports portability and technical operations agility with the ability to rapidly redeploy data pipelines and analytic models across multiple platforms in on-premises, cloud, multi-cloud, and hybrid ecosystems.” [1]</p><p><strong>Take it easy: DataOps combines Agile development, DevOps and statistical process controls and applies them to data analytics, but needs a well designed Data Platform!</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1008/0*SXI30EENa3sgqCyB" /></figure><h4>What is a Data Platform?</h4><p>We define a Data Platform as the infrastructure with a minimal set, easy to extend base components to successfully deliver data-driven business outcomes with increased productivity and improved collaboration between data engineers, data and security operators through a governed Self-service operation with the highest possible <strong>automation</strong>.</p><p>That said, we believe <strong>a governed Self-service operation and automation</strong> is key to the success of most DataOps initiatives and needs a Platform to build Platforms. With Kubernautic Platform we made it happen to build a Data Platform based on Kubernetes, which is a Platform to build Platforms.</p><p>Kubernautic Data Platform: DataOps Orchestration on Kubernetes</p><p>Kubernautic Data Platform provides a cloud-native infrastructure with the base components as the foundation to run DataOps initiatives on Kubernetes through orchestration in a Self-service manner <strong>without the need to operate the platform</strong> on top of our <a href="https://kubernauts.de/en/kubernauts-kubernetes-services/kubernautic-the-cloud-agnostic-managed-kubernetes-as-a-service-platform.html">Kubernautic</a> offering.</p><p>Kubernautic Data Platform is designed to enable data scientists to deploy, implement and run their data analytic pipelines built on top of Apache Kafka, Cassandra, MQTT implementations, Spark and Flink with Jupyter notebooks to achieve the following goals:</p><ul><li>Gather and manage data in one secure place</li><li>Build and share interactive dashboards</li><li>Reduce time spent on errors and operational tasks</li><li>Accelerate productivity through team collaboration</li><li>Orchestrate your development and production pipelines for re-usability and security</li><li>Unlock the full potential of DataOps without the need to operate the platform</li><li>Create innovative data analytics and deliver intelligent business value</li></ul><h4>DataOps Needs a Culture made by People to build the Data Factory</h4><p>Like DevOps, DataOps is mainly a principle to create and develop a strategy in your organisation to overcome the cultural obstacles to achieve higher agility by development and delivery of enterprise grade data pipelines to turn data into value through automation and the right processes and product design made by people to build the Data Factory!</p><h4>Kubernautic Data Platform: The DataOps Data Factory</h4><p>These days machines are assembled mostly by machines designed by people and complex processes in a data-driven world to deliver products in factories.</p><p>The following quote from Elon Musk describes the true problem and the solution to thinking of the Factory like a product.</p><blockquote>“We realised that the true problem, the true difficulty, and where the greatest potential is — is building the machine that makes the machine. In other words, it’s building the factory. I’m really thinking of the factory like a product.” Elon Musk</blockquote><p>With Kubernautic Data Platform we deliver a highly automated system to build a Data Factory for DataOps to build enterprise grade data-driven products and services on top of our <a href="https://kubernauts.de/en/kubernauts-kubernetes-services/kubernautic-the-cloud-agnostic-managed-kubernetes-as-a-service-platform.html">Kubernautic Platform</a>.</p><p>Some base components of Kubernautic Data Platform — which mainly provides the persistent layer of the platform — are presented through the following figure:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*E4a46YbpCUBZ3lp2" /></figure><h4>Automation and Self-service at the heart of Kubernautic Data Platform</h4><p>Like our <a href="https://kubernautic.com/">Kubernautic public offering</a>, where developers can get instant access for free to Spot Namespaces on Kubernetes or B2B customers within minutes to a dedicated Kubernetes cluster managed by Rancher Cluster Manager, Kuberbautic Data Platform provides the same agile experience to access the Data Factory for data pipeline and model orchestration, test and deployment automation for data pipelines and analytic models to make decisions faster with high quality data governed across a range of users, use cases, architectures and deployment options.</p><p>According to Gartner’s Survey Analysis in March 2020 titled as “Data Management Struggles to Balance Innovation and Control” [2], only 22% of a data team’s time is spent on new initiatives and innovation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/732/0*5qEBPyLziecu2urh" /></figure><p>The Self-service capability of our Data Platform allows DevOps teams to provision Kubernetes clusters with DataOps base components within a few hours, not days or weeks waiting for IT operation teams to set-up the environment to explore, blend, enrich and visualize data. And exactly this Self-service capability of Kubernautic Data Platform boosts innovation by your DataOps projects.</p><h4>DataOps and MLOps are almost the same, but …</h4><p>MLOps stands for Machine Learning Operations and is almost the sibling of DataOps.</p><p>We love the great blog post by Fernando Velez — Chief Data Technologist by Persistent — in Data-Driven Business and Intelligence titled as:</p><p>“DataOps and MLOps: Almost, But Not Quite The Same” [3]</p><p>and invite you to read through Fernando’s view on this topic, which might help to keep your Data and Machine Learning initiatives accurate over time with NoOps!</p><h3>Related resources</h3><p>[1] DataOps: More Than DevOps for Data Pipelines (by Eckerson Group)<br><a href="https://www.eckerson.com/articles/dataops-more-than-devops-for-data-pipelines">https://www.eckerson.com/articles/dataops-more-than-devops-for-data-pipelines</a></p><p>[2] Survey Analysis: Data Management Struggles to Balance Innovation and Control<br><a href="https://www.gartner.com/en/documents/3982237/survey-analysis-data-management-struggles-to-balance-inn">https://www.gartner.com/en/documents/3982237/survey-analysis-data-management-struggles-to-balance-inn</a></p><p>[3] DataOps and MLOps: Almost, But Not Quite The Same <a href="https://www.persistent.com/blogs/dataops-and-mlops-almost-but-not-quite-the-same/">https://www.persistent.com/blogs/dataops-and-mlops-almost-but-not-quite-the-same/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bc719cde38a4" width="1" height="1" alt=""><hr><p><a href="https://blog.kubernauts.io/kubernautic-data-platform-for-dataops-mlops-bc719cde38a4">Kubernautic Data Platform for DataOps &amp; MLOps</a> was originally published in <a href="https://blog.kubernauts.io/">Kubernauts</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Hey Docker run EKS, with Auto Fleet Spotting Support, please!]]></title>
            <link>https://blog.kubernauts.io/hey-docker-run-eks-with-auto-fleet-spotting-support-please-f91fcc80c60e?source=rss-d6c3d29f67f8------2</link>
            <guid isPermaLink="false">https://medium.com/p/f91fcc80c60e</guid>
            <category><![CDATA[ek]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[kubernauts]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[docker]]></category>
            <dc:creator><![CDATA[Arash]]></dc:creator>
            <pubDate>Sun, 16 Aug 2020 13:51:10 GMT</pubDate>
            <atom:updated>2020-08-27T16:55:53.091Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*dz8UGcmK3IMjlvpM" /><figcaption>Photo by <a href="https://unsplash.com/@el_ham?utm_source=medium&amp;utm_medium=referral">Hamish Weir</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p><strong>Updates:<br>2020/08/27: Git repo updated with only private subnets support option</strong>.</p><h3>Introduction</h3><p>We define Auto Fleet Spotting as a way to provide support for Auto Scaling of a Fleet of Spot Instances on AWS EKS.</p><p>This implementation is based on the official upstream <a href="https://github.com/terraform-aws-modules/terraform-aws-eks">Terraform AWS EKS</a> implementation and was extended to provide an easy way for the deployment of EKS clusters with Kubernetes 1.17.9 in any region with Auto Fleet Spotting support.</p><p>With this simple AWS EKS deployment automation solution with Docker you’ll not only be able to save time by EKS deployments and enforce some security best practices, but also you can reduce your Kubernetes costs by up to 90%.</p><p>I’m not joking, we’re using this implementation for developing and running our own <a href="https://kubernauts.sh/">Kubernautic Platform</a>, which is the basis of our Rancher Shared as a Service offering.</p><p>We’re using the same implementation for other dedicated Kubernetes clusters running on AWS EKS which are running with a ratio of 98% for our Spot to On-Demand Instances and scale roughly from 6 to 40 Spot Instances during the day. Here is the Grafana view on the node count of a real world project over the last 7 days.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oN6nvAX0bha3vR-qm4fb2w.png" /></figure><h3>TL;DR</h3><p>If you’d like to try this implementation, all what you need is to clone the extended <a href="https://github.com/arashkaffamanesh/terraform-aws-eks">AWS Terraform EKS</a> repo and run the docker/eks container to deploy the latest EKS Kubernetes 1.17.9 version (at this time of writing at 20/08/16). If you’d like to build your own image, please refer to <a href="https://github.com/arashkaffamanesh/terraform-aws-eks#build-your-own-image-from-dockerfile-recommended">this section</a> of the README file in the repo.</p><pre># Do this on your own risk, if you might want to trust me :-)</pre><pre>$ git clone <a href="https://github.com/arashkaffamanesh/terraform-aws-eks.git">https://github.com/arashkaffamanesh/terraform-aws-eks.git</a></pre><pre>&amp; cd terraform-aws-eks</pre><pre>$ docker run -it --rm -v &quot;$HOME/.aws/:/root/.aws&quot; -v &quot;$PWD:/tmp&quot; kubernautslabs/eks -c &quot;cd /tmp; ./2-deploy-eks-cluster.sh&quot;</pre><h4>Run EKS without the Docker Image</h4><p>If these <a href="https://github.com/arashkaffamanesh/terraform-aws-eks/tree/master/base#prerequisites">Prerequisites</a> are met on your machine, you can run the deployment script directly without running the docker container:</p><pre>$ ./2-deploy-eks-cluster.sh</pre><h3>Would you like to have a cup of coffee, tee or a cold German beer?</h3><p>The installation should take about 15 minutes, time enough to enjoy a delicious cup of coffee, tee or a cold German beer ;-)</p><p>Please let me describe what the deployment scripts mainly do and how you can extend the cluster with a new worker node group for On-Demand or additional Spot Instances and deploy additional add-ons like Prometheus, HashiCorp Vault, Tekton for CI/CD and GitOps, HelmOps, etc.</p><p>The first script `<em>./1-clone-base-module.sh</em>` copies the base folder from the root of the repo to a new folder which is the cluster name which you provide after running the script and provides a README file in the cluster folder with the needed steps to run the EKS deployment step by step.</p><p>The second script `<a href="https://github.com/volkswagen-onehub-services/vwg.services.infrastructure.eks.terraform/blob/master/2-deploy-eks-cluster.sh"><em>2-deploy-eks-cluster.sh</em></a>` makes the life easier and automates all the steps which is needed to deploy an EKS cluster. The main work which the deployment script does is to ask you about the cluster name and region and apply some few sed operations to substitute some values in the configuration files like setting the S3 bucket name, the cluster name and region and create a key pair, the S3 bucket and then run `<em>make apply</em>` to deploy the cluster with terraform and deploy the cluster autoscaler by setting the autoscaling group name in the <a href="https://github.com/arashkaffamanesh/terraform-aws-eks/blob/master/base/cluster-autoscaler-asg-name.yml">cluster-autoscaler-asg-name.yml</a>.</p><h4>Why using S3?</h4><p>The S3 bucket is very useful, if you want to work with your team to extend the cluster, it mainly saves the cluster state file `terraform.tfstate` in the S3 bucket, the S3 bucket should have a unique name and will be defined as the cluster name in the backend.tf file:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cc96ff9a3a086aaf41b1262a673d7070/href">https://medium.com/media/cc96ff9a3a086aaf41b1262a673d7070/href</a></iframe><p>One of the essential parts to get autoscaling working with spot instances is to define the right IAM Policy Document and attach it to the worker autoscaling group of the worker node group IAM role:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4a26caa805dd79cdb52ed85ad56fa92a/href">https://medium.com/media/4a26caa805dd79cdb52ed85ad56fa92a/href</a></iframe><h3>How does the whole thing work?</h3><p>Each cluster may have one or more Spot Instances or On-Demand Worker groups, sometimes referred as node pools as well. The initial installation will deploy only one worker group named spot-1, which we can easily extend after the initial deployment and run make apply again.</p><p>If you’d like to add a new On-Demand worker group, you’ve to extend the `main.tf` file in your cluster module (in terraform a folder is named module) as shown below and run make plan followed with make apply:</p><pre>$ make plan<br>$ make apply</pre><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6a117e59f75001d9794bd3edfed50df8/href">https://medium.com/media/6a117e59f75001d9794bd3edfed50df8/href</a></iframe><h4>Using Private Subnets</h4><p>I extended the VPC module definition to make use of private subnets with a larger /20 network address range, which allows us to use up-to 4094 IPs for our pods.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/98f725c41df79cb9caeace191ca538eb/href">https://medium.com/media/98f725c41df79cb9caeace191ca538eb/href</a></iframe><h3>Add-Ons</h3><p>Some additional components which we refer as add-ons are provided in the addons folder in the repo. Each addon should have a README file about how to use it.</p><h3>Testing</h3><p>To test how the auto-scaling of spot instances works, you can deploy the provided nginx-to-scaleout.yaml from the addons folder and scale it to 10 replicas and see how the spot instances are scaled-out and decrease the number of replicas to 1 and see how the spot instances are scaled-in back (the scaled-in process may take up to 5 minutes or longer, or even don&#39;t happen, if something was not configured properly!).</p><pre>k create -f ../addons/nginx-to-scaleout.yaml<br>k scale deployment nginx-to-scaleout --replicas=10<br>k get pods -w<br>k get nodes</pre><h3>Conclusion</h3><p>Running an EKS cluster with autoscaling support for spot instances or on-demand instances is a matter of running a single docker run command, even if you don’t have terraform, the aws-iam-authenticator or the kubectl-terraform plugin installed. All needed configurations are provided as code, we didn’t even had to login to the AWS web console to setup the EKS cluster and configure the permissions for the cluster auto-scaler for for our worker groups.</p><p>This is the true Infrastructure as Code made with few lines of code and minor adaptions to the upstream Terraform EKS implementation to run EKS with support of Autoscaling of Spot and On-Demand instances.</p><p>We are using this implementation with Kubernetes 1.15 today, the in-place migration from 1.15 directly to 1.17 can’t work directly, we’re going to use blue green deployments with traffic routing to migrate our workloads and our users who are using our <a href="https://kubernauts.sh/">Kubernautic Platform</a> to a new environment, users would be able to use a full-fledged, fully isolated virtual Kubernetes cluster running in a Stateful Namespace, stay tuned!</p><h3>Questions?</h3><p>Join us on <a href="https://kubernauts-slack-join.herokuapp.com/">Kubernauts Slack channel</a> or add a comment here.</p><h3>We’re hiring!</h3><p>We are looking for engineers who love to work in Open Source communities like Kubernetes, Rancher, Docker, etc.</p><p>If you wish to work on such projects and get involved to build the NextGen Kubernautic Platform, please do visit our <a href="https://kubernauts.de/en/careers/">job offerings page</a> or <a href="https://twitter.com/kubernauts">ping us on twitter</a>.</p><h3>Related resources</h3><p><a href="https://github.com/terraform-aws-modules/terraform-aws-eks">Terraform AWS EKS</a><br><a href="https://github.com/terraform-providers/terraform-provider-aws/tree/master/examples/eks-getting-started">Terraform Provider EKS</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f91fcc80c60e" width="1" height="1" alt=""><hr><p><a href="https://blog.kubernauts.io/hey-docker-run-eks-with-auto-fleet-spotting-support-please-f91fcc80c60e">Hey Docker run EKS, with Auto Fleet Spotting Support, please!</a> was originally published in <a href="https://blog.kubernauts.io/">Kubernauts</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Hello HelmOps with Tekton and Chartmuseum on Kubernetes]]></title>
            <link>https://blog.kubernauts.io/hello-helmops-with-tekton-and-chartmuseum-on-kubernetes-4ffd2038425d?source=rss-d6c3d29f67f8------2</link>
            <guid isPermaLink="false">https://medium.com/p/4ffd2038425d</guid>
            <category><![CDATA[gitops]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[containers]]></category>
            <category><![CDATA[helmops]]></category>
            <category><![CDATA[k3]]></category>
            <dc:creator><![CDATA[Arash]]></dc:creator>
            <pubDate>Mon, 03 Aug 2020 18:48:31 GMT</pubDate>
            <atom:updated>2020-09-27T16:07:17.572Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*o4tUE1aEK-zIQRW_" /><figcaption>Photo by <a href="https://unsplash.com/@jbsinger1970?utm_source=medium&amp;utm_medium=referral">Jonathan Singer</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>This tutorial should help you to get started with Helm Operations, referred to as HelmOps and CI/CD with Tekton on <a href="https://k3s.io/">K3s</a> running on your Laptop deployed on multipass Ubuntu VMs with MetalLB support. We’ll use <a href="https://chartmuseum.com/">Chartmuseum</a> to store a simple Helm Chart in our own Helm Repository in Chartmuseum and deploy our apps from there through Tekton Pipelines in the next section of this tutorial.</p><p>This guide should work with any K8s implementation running locally or on any cloud with some minor adaptations to the scripts provided in the <a href="https://github.com/kubernauts/bonsai">Github Repo of Bonsai</a>.</p><p>To learn more about <a href="https://k3s.io/">K3s</a> and our somehow opinionated Bonsai implementation on Multipass VMs with MetalLB support, please refer to this article about <a href="https://blog.kubernauts.io/k3s-with-metallb-on-multipass-vms-ac2b37298589">K3s with MetalLB on Multipass VMs</a>.</p><h3>Get started</h3><p>With the following commands you should get a K3s cluster running on your machine in about 3 minutes on real Ubuntu VMs, which we refer sometimes as a near-to-production environment.</p><pre>git clone <a href="https://github.com/kubernauts/bonsai.git">https://github.com/kubernauts/bonsai.git<br></a>cd bonsai<br>./deploy-bonsai.sh<br># please accept all default values (1 master and 2 workers)</pre><h3>Deploy Chartmuseum</h3><h4>Prerequisites</h4><ul><li>A running k8s cluster</li><li>LB service (it will be installed by default with Bonsai)</li><li>helm3 version</li></ul><p>We deploy chartmuseum with custom values and create an ingress with the chartmuseum.local domain name.</p><p>Please set the following host entries in your /etc/hosts file:</p><pre># please adapt the IP below with the external IP of the traefik LB (run `kubectl get svc -A` and find the external IP)</pre><pre>192.168.64.26 chart-example.local</pre><pre>192.168.64.26 chartmuseum.local</pre><pre>192.168.64.26 tekton-dashboard.local</pre><pre>192.168.64.26 registry-ui.local</pre><pre>192.168.64.26 getting-started-triggers.local</pre><p>And deploy chartmuseum:</p><pre>cd addons/chartmuseum<br>helm repo add stable <a href="https://kubernetes-charts.storage.googleapis.com/">https://kubernetes-charts.storage.googleapis.com</a><br>kubectl create ns chartmuseum<br>kubectl config set-context --current --namespace chartmuseum<br>helm install chartmuseum stable/chartmuseum --values custom-values.yaml<br>kubectl apply -f ingress.yaml</pre><p>Now you should be able to access chartmuseum through:</p><p><a href="http://chartmuseum.local/">http://chartmuseum.local</a></p><p>If you see the “Welcome to ChartMuseum!”, then you’re fine.</p><p>Now we are ready to add the chartmuseum repo and install the helm-push plugin and package the sample hello-helm chart and push it to our chartmuseum repo running on K3s:</p><pre># add the repo<br>helm repo add chartmuseum <a href="http://chartmuseum.local/">http://chartmuseum.local</a><br># install helm push plugin:<br>helm plugin install <a href="https://github.com/chartmuseum/helm-push.git">https://github.com/chartmuseum/helm-push.git</a><br># build the package:<br>cd hello-helm/chart<br>helm package hello/<br># the helm package name will be hello-1.tgz<br>ls -l hello-1.tgz<br># push the package to chartmuseum<br>helm push hello-1.tgz chartmuseum<br># You should get:<br>Pushing hello-1.tgz to chartmuseum…<br>Done.</pre><h3>Install the Chart</h3><p>To install a chart, this is the basic command used:</p><pre>helm install &lt;chartmuseum-repo-name&gt;/&lt;chart-name&gt; --name &lt;release-name&gt; (helm2)</pre><pre>helm install &lt;release-name&gt; &lt;chartmuseum-repo-name&gt;/&lt;chart-name&gt; (helm3)</pre><p>We need to update our helm repos first and install the chart:</p><pre>helm repo update</pre><pre>helm install hello chartmuseum/hello</pre><p>We should get a similar output like this:</p><pre>NAME: hello</pre><pre>LAST DEPLOYED: Sat Jul 25 17:56:23 2020</pre><pre>NAMESPACE: chartmuseum</pre><pre>STATUS: deployed</pre><pre>REVISION: 1</pre><pre>TEST SUITE: None</pre><h3>Harvest your work</h3><pre>curl <a href="http://chart-example.local/">http://chart-example.local/</a><br><em>Welcome to Hello Chartmuseum for Private Helm Repos on K3s Bonsai!</em></pre><p><strong>What happened here?</strong></p><p>We deployed the chart with helm through the command line, the hello service was defined in the values.yaml with the type LoadBalancer and received an external IP from MetalLB.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/737/0*a5xENzC1CMOsB72t" /></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/262c126dd063c5258cd7e13a936c6f93/href">https://medium.com/media/262c126dd063c5258cd7e13a936c6f93/href</a></iframe><p>In the above values.yaml we define to have an ingress and the hostname which is chart-example.local as the domain name which is mapped in our /etc/hosts file to the IP address of the traefik load balancer, alternatively we could map the domain to the external IP address of the hello-hello service (in my case 192.168.64.27) as well.</p><h3>Delete the chart and the helm hello release</h3><p>Since we want to automate the deployment process in the next section with Tekton Pipelines and do the CI part as well, we’ll clean up the deployment and delete our chart from chartmuseum.</p><pre>helm delete hello<br>curl -i -X DELETE <a href="http://chartmuseum.local/api/charts/hello/1">http://chartmuseum.local/api/charts/hello/1</a></pre><h3><strong>CI/CD and HelmOps on Kubernetes</strong></h3><p>In this section, we’re going to introduce Tekton and Tekton Pipelines which is the technological successor to <a href="https://github.com/knative/build">Knative Build</a> and provides a Kube-Native style for declaring CI/CD pipelines on Kubernetes. Tekton supports many advanced CI/CD patterns, including rolling, blue/green, and canary deployment. To learn more about Tekton, please refer to the <a href="https://tekton.dev/docs/">official documentation page of Tekton</a> and also visit the new neutral home for the next generation of continuous delivery collaboration by <a href="https://cd.foundation/">Continuous Delivery Foundation (CDF</a>).</p><h4>For the impatient</h4><p>Katakoda provides a very nice <a href="https://www.katacoda.com/javajon/courses/kubernetes-pipelines/tekton">Scenario for Tekton</a> which demonstrates building, deploying, and running a Node.js application with Tekton on Kubernetes using a private docker registry.</p><p>By going through the Katakoda scenario in about 20 minutes, you should be able to learn about Tekton concepts and how to define various Tekton resources like Tasks, Resources and Pipelines to kick off a process through PipelineRuns to deploy our apps in a GitOps style manner on Kubernetes.</p><p>In this section, we’re going to extend that scenario with Chartmuseum from the first section and provide some insights about Tekton Triggers which rounds up this tutorial and hopefully helps you to use it for your daily projects.</p><h4>About GitOps and xOps</h4><p>GitOps is buzzing these days after the dust around DevOps, DevSecOps, NoOps, or xOps has settled down. We asked the community about the future of xOps and what’s coming next, if you’d like to provide your feedback, we’d love to have it:</p><h3></h3><p>DevOpsDevSecOpsGitOpsGitSecOpsSimpleOpsNoOpsxOpsWhat&#39;s coming next?#KubeOps#KubeSecOps#KeyOps?😊#keeverneetees #k11s #Kubernetes #k8s #k3s pic.twitter.com/V6HHWVzUsp</p><p>The reality is we did CVS- and SVN- Ops about 15 years ago with <a href="http://cruisecontrol.sourceforge.net/index.html">CruiseControl</a> for both CI and CD. Later Hudson from the awesome folks at Sun Micorsystems was born and renamed to Jenkins, which is still one of the most widely used CI/CD tools today.</p><p>Times have changed and with the rise of Cloud Computing, Kubernetes and the still buzzing term Cloud-Native, which is <a href="https://blog.kubernauts.io/cloud-native-a-new-way-of-thinking-8699133309ad">a new way of thinking</a>, the whole community is bringing new innovations at the speed of light.</p><p><a href="https://blog.kubernauts.io/cloud-native-a-new-way-of-thinking-8699133309ad">Cloud Native: A New Way Of Thinking!</a></p><p>New terms like Kube-Native-Ops, AppsOps, InfraOps and now HelmOps are not so common or buzzing yet at this time of writing. We believe that Tekton or Jenkins-X which is based on Tekton along with other nice solutions like ArgoCD, FluxCD, Spinnaker or Screwdriver are going to change the way we’re going to automate the delivery and roll-out of our apps and services the Kube-Native way on Kubernetes.</p><p>With that said, let’s start with xOps and deploy our private docker registry and Tekton on K3s and run our simple node.js app from the previous section through helm triggered by Tekton Pipelines from Tekton Dashboard.</p><p>Make sure you have helm3 version in place, add the stable and incubator Kubernetes Charts Helm Repo and install the docker registry proxy, the registry UI and the ingress for the registry UI:</p><pre>helm version --short</pre><pre>helm repo add stable <a href="https://kubernetes-charts.storage.googleapis.com/">https://kubernetes-charts.storage.googleapis.com</a></pre><pre>helm repo add incubator <a href="http://storage.googleapis.com/kubernetes-charts-incubator">http://storage.googleapis.com/kubernetes-charts-incubator</a></pre><pre>helm upgrade --install private-docker-registry stable/docker-registry --namespace kube-system</pre><pre>helm upgrade --install registry-proxy incubator/kube-registry-proxy \<br>--set registry.host=private-docker-registry.kube-system \</pre><pre>--set registry.port=5000 \</pre><pre>--set hostPort=5000 \</pre><pre>--namespace kube-system</pre><pre>kubectl apply -f docker-registry-ui.yaml</pre><pre>kubectl apply -f ingress-docker-registry-ui.yaml</pre><h4>Access the Docker Registry</h4><p><a href="http://registry-ui.local/">http://registry-ui.local/</a></p><h4>Deploy Tekton Resources, Triggers and Dashboard</h4><pre>kubectl apply -f pipeline-release.yaml</pre><pre>kubectl apply -f triggers-release.yaml</pre><pre>kubectl config set-context --current --namespace tekton-pipelines</pre><pre>kubectl apply -f tekton-dashboard-release.yaml</pre><pre>kubectl apply -f ingress-tekton-dashboard.yaml</pre><h4>Access the Tekton Dashboard</h4><p>The Tekton dashboard for this installation on K3s can be reached via<br><a href="http://tekton-dashboard.local/">http://tekton-dashboard.local/</a></p><p>The dashboard is great to see how tasks and PipelineRuns are running or if something goes wrong to get the logs for troubleshooting or sell it as your xOps Dashboard ;-)</p><p>For now we don’t have any pipelines running. We’re going to run some tasks first, build a pipeline and initiate it in the next steps along with pipeline runs. But before we fire the next commands, I’d like to explain what we’re going to do.</p><p>We have 2 apps, the first app is named app, which builds an image from the source on github with kaniko executor and pushes it to our private docker registry and deploy it from there into the hello namespace, where we deploy our hello app from our private chartmuseum repo for the HelmOps showcase as well.</p><p>N.B.: we cleaned-up the same hello app at the end of the first section of this tutorial and are going to deploy it through HelmOps.</p><p>Take it easy or not, we’re going to create a task, build a pipeline containing our task, use the TaskRun resource to instantiate and execute a Task outside of the Pipeline and use the PipelineRun to instantiate and run the Pipeline containing our Tasks, oh my God? ;-)</p><h4>Take it easy, let’s start with HelmOps or xOps</h4><p>You need the <a href="https://github.com/tektoncd/cli/releases">tkn cli</a> to follow the next steps, I’m using the latest version at this time of writing.</p><pre>➜ pipeline git:(master) ✗ tkn version</pre><pre>Client version: 0.11.0<br>Pipeline version: v0.15.0<br>Triggers version: v0.7.0</pre><p>We create a namespace named hello and switch to the namespace:</p><pre>kubectl create ns hello<br>kubectl config set-context --current --namespace hello</pre><p>Now we need to define the git-resource, which is the source in our git repo defined through a custom resource definition PipelineResource:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d3cfe785de60fce2658e90bfeb8bf36e/href">https://medium.com/media/d3cfe785de60fce2658e90bfeb8bf36e/href</a></iframe><p>And apply the git-resource:</p><pre>kubectl apply -f node-js-tekton/pipeline/git-resource.yaml</pre><p>With that we are defining where our source is coming from in git, in our case from Github.</p><p>We can now list our first resource with the tkn cli tool as follow:</p><pre>tkn resources list</pre><p>In the next step, we’re going to define a task named “build-image-from-source”. In the spec part of the Task object (CRD) we define the git-source with the type git and some parameters as pathToContext, patToDockerfile, imageUrl and imageTag as inputs and the steps needed to run the task, the list-src (which is optional) and the <strong>build-and-push</strong> step which uses the kanico-project executor image with the command /kaniko/executor/ and some args to build and push the image.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d77e198facd1fbc89e8882dd49015ab9/href">https://medium.com/media/d77e198facd1fbc89e8882dd49015ab9/href</a></iframe><p>Now we need to deploy our task <strong>build and push</strong> Task resource to our cluster with:</p><pre>kubectl apply -f node-js-tekton/pipeline/task-build-src.yaml</pre><p>And deploy another Task which is defined in the task-deploy.yaml</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c2fcf43d3fd076b7c5df2391cc2c5140/href">https://medium.com/media/c2fcf43d3fd076b7c5df2391cc2c5140/href</a></iframe><p>This Task has 4 steps:</p><ul><li>update-yaml # set the image tag</li><li>deploy-app # deploy the first app</li><li>push-to-chartmuseum # push the hello helm chart to chartmuseum</li><li>helm-install-hello # install the helm chart to the cluster</li></ul><p>The last 3 tasks are using a slightly extended helm-kubectl image which has the helm push plugin installed.</p><p>N.B. the Dockerfile for helm-kubectl is provided under addons/helm-kubectl.</p><p>We can now run the task-deploy.yaml and list our tasks with:</p><pre>kubectl apply -f node-js-tekton/pipeline/task-deploy.yaml<br>tkn tasks list</pre><p>And finally we’ll declare the pipeline and run it with the pipeline-run entity.</p><p>Nice to know: In PipelineRun we define a collection of resources that define the elements of our Pipeline which include tasks.</p><pre>kubectl apply -f node-js-tekton/pipeline/pipeline.yaml</pre><pre>tkn pipelines list</pre><pre>kubectl apply -f node-js-tekton/pipeline/service-account.yaml</pre><pre>kubectl get ServiceAccounts</pre><pre>kubectl apply -f node-js-tekton/pipeline/pipeline-run.yaml</pre><pre>tkn pipelineruns list</pre><pre>tkn pipelineruns describe application-pipeline-run</pre><pre>kubectl get all -n hello</pre><pre>curl chart-example.local</pre><pre>Welcome to Hello Chartmuseum for Private Helm Repos on K3s Bonsai!</pre><h4>Harvest your Helm-X-Ops work</h4><p>If all steps worked, in the <a href="http://tekton-dashboard.local/#/namespaces/hello/pipelineruns">Tekton Dashboard</a> you should get what you did in the hello namespace, enjoy :-)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wmGu_-qHTr0QI6enqcHISA.png" /></figure><h3>What’s coming next: Automation</h3><p>In the third section of this tutorial, we’ll see how to use Tekton Triggers to automate our HelmOps process, stay tuned.</p><h3>Related resources</h3><h4>Sample nodes-js app for tekton</h4><p><a href="https://github.com/javajon/node-js-tekton">https://github.com/javajon/node-js-tekton</a></p><h4>Tekton Pipelines on Github</h4><p><a href="https://github.com/tektoncd/pipeline">https://github.com/tektoncd/pipeline</a></p><h4>Katacoda Scenarions</h4><p><a href="https://www.katacoda.com/javajon/courses/kubernetes-pipelines/tekton">https://www.katacoda.com/javajon/courses/kubernetes-pipelines/tekton</a><br><a href="https://katacoda.com/ncskier/scenarios/tekton-dashboard">https://katacoda.com/ncskier/scenarios/tekton-dashboard</a></p><h4>Tekton Pipelines Tutorial</h4><p><a href="https://github.com/tektoncd/pipeline/blob/master/docs/tutorial.md">https://github.com/tektoncd/pipeline/blob/master/docs/tutorial.md</a></p><h4>Open-Toolchain Tekton Catalog</h4><p><a href="https://github.com/open-toolchain/tekton-catalog">open-toolchain/tekton-catalog</a></p><h4>GitOps</h4><p><a href="https://www.gitops.tech/">https://www.gitops.tech/</a></p><h4>Building a ChatOps Bot With Tekton</h4><p><a href="https://medium.com/@dlorenc/building-a-chatops-bot-with-tekton-6d51c4f958f9">Building a ChatOps Bot With Tekton</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4ffd2038425d" width="1" height="1" alt=""><hr><p><a href="https://blog.kubernauts.io/hello-helmops-with-tekton-and-chartmuseum-on-kubernetes-4ffd2038425d">Hello HelmOps with Tekton and Chartmuseum on Kubernetes</a> was originally published in <a href="https://blog.kubernauts.io/">Kubernauts</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploy Ghost in a Spot Namespace on RSaaS from your browser]]></title>
            <link>https://blog.kubernauts.io/deploy-ghost-in-a-spot-namespace-on-rsaas-from-your-browser-d9376db342e0?source=rss-d6c3d29f67f8------2</link>
            <guid isPermaLink="false">https://medium.com/p/d9376db342e0</guid>
            <category><![CDATA[saas]]></category>
            <category><![CDATA[rancher]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[kubernauts]]></category>
            <category><![CDATA[free-hosting]]></category>
            <dc:creator><![CDATA[Arash]]></dc:creator>
            <pubDate>Sun, 28 Jun 2020 18:42:01 GMT</pubDate>
            <atom:updated>2021-03-07T21:20:27.786Z</atom:updated>
            <content:encoded><![CDATA[<h3>Deploy Ghost in a Spot Namespace on Kubernautic from your browser</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8AG65Xax4P25-fnmrxQVSg.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@karim_manjra?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Karim MANJRA</a> on <a href="https://medium.com/s/photos/freedom?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>In <a href="https://blog.kubernauts.io/free-kubernetes-namespaces-for-everyone-80d721ad1882">my previous post</a> Spot Namespaces was introduced.</p><p>In this post I’m going to show you how you can run <a href="https://ghost.org/">ghost</a>, a headless CMS or your own apps in a namespace by defining the proper requests and limits in the deployment manifest file of ghost or your own app on <a href="https://kubernautic.com/">Kubernautic</a>.</p><p>After you’re logged into Rancher you’ll have access to your project on your assigned Kubernetes cluster in form of &lt;user-id&gt;-project-spot-namespace:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/582/0*fKAztIVfplH8Pb_Y" /></figure><p>Click on the project and in the navigation menu bar click on Namespaces, now you can see a the namespace which is assigned to your project:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IjOIfP5FFnVIE7T88p8J9Q.png" /></figure><p>Now that we know the id of our namespace, we’re ready to deploy our Ghost app.</p><p>First you need to select the cluster and click on the “Launch kubectl” button:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*XALtSVe-qo02YeNL" /></figure><p>With that we’ll have Shell access to the cluster in our browser:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/684/1*gLIafJozO10AlDtGfU3c6Q.png" /></figure><p>With the following kubectl command, you can switch to your namespace:</p><pre>kubectl config set-context --current --namespace &lt;your namespace&gt;</pre><p>And deploy Ghost like this:</p><pre>kubectl create -f <a href="https://raw.githubusercontent.com/kubernauts/practical-kubernetes-problems/master/3-ghost-deployment.yaml">https://raw.githubusercontent.com/kubernauts/practical-kubernetes-problems/master/3-ghost-deployment.yaml</a></pre><p>Please note in the spec part of the manifest we’re setting the limits for CPU and Memory in the resources part of our ghost image, unless you’ll not be able to deploy the app.</p><pre>spec:<br>      containers:<br>      - image: ghost:latest<br>        imagePullPolicy: IfNotPresent<br>        name: ghost<br>        resources:<br>          limits:<br>            memory: 512Mi<br>            cpu: 500m<br>          requests:<br>            memory: 256Mi</pre><p>To see if our deployment was successful, we can run:</p><pre>kubectl get deployment</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/905/1*_dMLIryVAYxqRcWkInFOEA.png" /></figure><p>Now we can create a L7 Ingress to ghost easily in Rancher by selecting Resources → Workloads → LoadBalancing → Add ingress.</p><p>In this example we’re going to tell Rancher to generate a .xip.io hostname for us, we select the target which is ghost and provide the <strong>port 2368</strong> in Port filed:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bUQbYu0USEKywk82qy3e3w.png" /></figure><p>After couple of seconds the L7 ghost ingress is ready to go, click on the ghost-ingress-xyz-xip.io link and enjoy:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XyXH2cK2DYA92fJlasiRZQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*K1M8MPPY-Ja2Y4UlC_EHIw.png" /></figure><h3>Questions?</h3><p>If you have any questions, we’d love to welcome you to our <a href="https://kubernauts-slack-join.herokuapp.com/">Kubernauts Slack</a> channel and be of help.</p><h3>We’re hiring!</h3><p>We love people who love building things, New Things!<br>If you wish to work on New Things, please do visit our <a href="https://kubernauts.de/en/careers/">job offerings page</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d9376db342e0" width="1" height="1" alt=""><hr><p><a href="https://blog.kubernauts.io/deploy-ghost-in-a-spot-namespace-on-rsaas-from-your-browser-d9376db342e0">Deploy Ghost in a Spot Namespace on RSaaS from your browser</a> was originally published in <a href="https://blog.kubernauts.io/">Kubernauts</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Free Kubernetes Namespaces for Everyone]]></title>
            <link>https://blog.kubernauts.io/free-kubernetes-namespaces-for-everyone-80d721ad1882?source=rss-d6c3d29f67f8------2</link>
            <guid isPermaLink="false">https://medium.com/p/80d721ad1882</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[rancher]]></category>
            <category><![CDATA[free]]></category>
            <category><![CDATA[containers]]></category>
            <dc:creator><![CDATA[Arash]]></dc:creator>
            <pubDate>Sat, 27 Jun 2020 15:19:40 GMT</pubDate>
            <atom:updated>2020-06-29T09:01:30.125Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Introducing Spot and Stateful Namespaces on RSaaS — Rancher Shared as a Service — on Auto-Pilot</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3lE6rlRkJ1Abh4o5mui8WQ.jpeg" /><figcaption>Photo by Luca Upper on Unsplash</figcaption></figure><p>We launched our Rancher Shared and Dedicated as a Service offering some weeks ago to help individuals and small companies to get started with Rancher Management Server to build their own Kubernetes Platform without having to deal with the Rancher installation, maintenance or upgrade, and the best thing is, it is for free for everyone, forever!</p><p>With RSaaS — <a href="https://kubernauts.de/en/kubernauts-kubernetes-services/rancher-shared-as-a-service.html">Rancher Shared as a Service</a> — you can get instant access to a <strong>shared carefree</strong> Rancher Server with a <strong>free namespace</strong> to deploy your apps or to create new or import existing Kubernetes clusters within minutes.</p><p>With RSaaS we hope to boost up the developer productivity through a zero touch experience and engage users to build their own Kubernetes platform with Rancher everywhere.</p><p>With RDaaS — <a href="https://kubernauts.de/en/kubernauts-kubernetes-services/rancher-dedicated-as-a-service.html">Rancher Dedicated as a Service</a> — small to midsize companies can get access to a full-managed AWS account and an EKS cluster through a dedicated, high available Rancher Management Server running on RKE with <strong>Auto Scaling Support for Spot Instances</strong>, which helps to reduce their cloud costs by up to 90% and save our climate and planet, since wasting your money means hurting our climate!</p><p>If you’re going to use RSaaS for development, you can import your local minikube, k3d or k3sup made k3s, kind or whatever cluster like AKS, EKS or GKE into Rancher or create a Rancher Kubernetes Engine (RKE) cluster on your local machine or on other existing infrastructure through Rancher.</p><p>But with RSaaS we are introducing Spot and Stateful Namespaces to help developers and small businesses to get instant access to one or more namespaces in a project in Rancher. With that we are introducing the <strong>Cloudless</strong> paradigm, where you can use the cloud without having direct access to the cloud and the need to provision the infrastructure and deal with security, hardening, patching, etc.</p><h3>Introducing Spot Namespaces</h3><p>Inspired from <a href="https://aws.amazon.com/de/ec2/spot/">AWS Spot Instances</a> which we are using heavily on most of our managed Kubernetes clusters, we are introducing Spot Namespaces. A Spot Namespace is an ephemeral namespace which RSaaS users can use for a certain time, depending on the capacity and load of the Kubernetes clusters like EKS and soon GKE which are integrated into RSaaS with <strong>dynamic quotas</strong>. Spot Namespaces could last for a day, week or forever, depending on the utilisation of the namespace. Idle namespaces will get purged to save resources and provide the freed-up resources to other namespaces which have a higher utilisation need.</p><p>Since RSaaS with Spot Namespaces are a no-cost and free offering, we had to see how we could finance the whole service without having to get bankrupted. The idea is to provide low-cost namespaces which are Stateful and do persist as long you want.</p><h3>Stateful Namespaces coming soon</h3><p>A Stateful namespace is a low-cost namespace which won’t get purged and will get charged based on the desired quota which the user can define in our backend environment in the near future. With Stateful Namespaces users can apply to get their own On-Demand or Reserved Instances.</p><p>Now that you have an idea about RSaaS with Spot and Stateful Namespaces, we hope you’re ready to get started with RSaaS.</p><p>Get Instant Access to RSaaS</p><p>Getting <a href="https://kubernauts.sh/#section-rsaas-form">instant access to Rancher</a> is a matter of providing your name and email and clicking the Sign Up button. You’ll receive an email with an initial password to get started immediately.</p><p>Here is a short video recording about how the registration process and the instant access to RSaaS might work for you.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F5JitQlS6gmM%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D5JitQlS6gmM&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F5JitQlS6gmM%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/900e0ace7df184f8f07addf78fba9a77/href">https://medium.com/media/900e0ace7df184f8f07addf78fba9a77/href</a></iframe><h3>FAQs</h3><p><strong>How do I get access to a Spot Namespace?</strong></p><p>After the initial registration, you’ll get access to a project in the form of &lt;user-id&gt;-project-spot-namespace and you can create a namespace in this project.</p><p><strong>Do I get informed before the Spot Namespace is purged?</strong></p><p>Yes, the system informs you before the resources in your namespace is purged.</p><p><strong>How many Namespaces can I use?</strong></p><p>The number of namespaces within a project is unlimited.</p><p><strong>How much quota can I use with a Spot Namespace?</strong></p><p>Each namespace has by default a single KU (kube unit) of 0.5 vCPU and 512 MB Memory assigned.</p><p><strong>How much does a Stateful Namespace cost?</strong></p><p>A Stateful Namespace with a single KU (kube unit) of 0.5 vCPU and 512 MB Memory and 1 GB storage will cost probably about €10–€20.</p><p><strong>Can I change the type of my Spot Namespace to a Stateful Namespace?</strong></p><p>Yes, you need to apply for the change. In the future, users will be able to change the type of their namespace in the backend interface.</p><p><strong>How are Stateful Namespaces charged and what does it cost?</strong></p><p>Stateful Namespaces are charged depending on the requested quota and reserved resources monthly.</p><p><strong>How secure and isolated are the namespaces?</strong></p><p>We are implementing our own hard multi-tenancy implementation for namespaces, but don’t guarantee any security breaches. Using a shared environment is always at your own risk. Although we have done our best to provide the best isolation possible.</p><p><strong>Can I provision a managed hosted cluster like AKS, EKS or GCP?</strong></p><p>Yes, but we don’t recommend doing it in a shared environment.</p><p><strong>Can I change from RSaaS to RDaaS?</strong></p><p>Sure, we’ll create a new account with the same user to your RDaaS environment, where you’ll have full-admin privileges.</p><p><strong>Can I use On-Demand Instances with RSaaS?</strong></p><p>Yes, you can get your own On-Demand instance, if you’re using Stateful Namespaces.</p><p><strong>Can I use Reserved Instances?</strong></p><p>Yes, but you should extend your contract for at least 1 year.</p><p><strong>Can I use Auto Scaling with On-Demand Instances?</strong></p><p>Not yet, but in the future you can configure the type of the instance in the backend interface and the system will provision the instance for you.</p><p><strong>Can I choose the Cloud Provider where my workloads shall run?</strong></p><p>Not yet, but we are planning to support GCP and Azure in addition to AWS.</p><p><strong>How does the support process work?</strong></p><p>Paid accounts which are using Stateful Namespaces have access to our <a href="https://kubernauts.atlassian.net/">service desk</a>. Spot Namespace users can get support through our public <a href="https://kubernauts-slack-join.herokuapp.com/">Kubernauts Slack</a> in the <a href="https://kubernauts.slack.com/archives/C015CFYQM6W">#rsaas-support</a> channel.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=80d721ad1882" width="1" height="1" alt=""><hr><p><a href="https://blog.kubernauts.io/free-kubernetes-namespaces-for-everyone-80d721ad1882">Free Kubernetes Namespaces for Everyone</a> was originally published in <a href="https://blog.kubernauts.io/">Kubernauts</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[TK8 Cattle EKS Provisioner with Terraform Rancher Provider]]></title>
            <link>https://blog.kubernauts.io/tk8-cattle-eks-provisioner-with-terraform-rancher-provider-d5f5c4ccf43e?source=rss-d6c3d29f67f8------2</link>
            <guid isPermaLink="false">https://medium.com/p/d5f5c4ccf43e</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[rancher]]></category>
            <category><![CDATA[tk8]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[ek]]></category>
            <dc:creator><![CDATA[Arash]]></dc:creator>
            <pubDate>Tue, 31 Dec 2019 20:34:41 GMT</pubDate>
            <atom:updated>2020-04-29T21:19:23.639Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/734/1*Ln3PAZEWda4_moMJLxdGJQ.png" /></figure><p>In a <a href="https://blog.kubernauts.io/tk8-cattle-aws-provisioner-with-terraform-rancher-provider-db4e3d2edd7e">previous post</a> we introduced how to use a Rancher Server with Terraform Rancher Provider to deploy Rancher’s Kubernetes Engine (RKE) with the TK8 Cattle AWS provisioner on auto-provisioned EC2 machines.</p><p>In this post I’ll introduce the TK8 Cattle EKS provisioner by the awesome <a href="https://twitter.com/ishantanu_">Shantanu Deshpande</a> to deploy an EKS Cluster with the tk8ctl tool talking to a Rancher Server using a valid SSL certificate running on our local machine.</p><h3>Rancher launched EKS vs. Rancher launched RKE Cluster</h3><p>With Rancher Server you can launch or import any Kubernetes cluster on any cloud provider or existing bare-metal servers or virtual machines.</p><p>In the case of AWS, we can either choose to use RKE with new nodes on Amazon EC2 or the managed Amazon EKS offering.</p><p>With EKS one doesn’t need to worry about managing the control plane or even the worker nodes, AWS manages everything for us at the price of a lower Kubernetes version, which is Kubernetes v1.14.8 at this time of writing.</p><p>With RKE, we can use the latest Kubernetes 1.16.x or soon 1.17.x versions, but we need to manage the control plane and worker nodes on our own, which requires skilled Kubernetes and Rancher professionals.</p><p>Harshal Shah shares his experience nicely in this blog post about <a href="https://engineering.hellofresh.com/lessons-learned-from-running-eks-in-production-a6dddb8368b8">Lessons Learned from running EKS in Production</a>, which I highly recommend to read, if you’d like to free-up your time to be able to deal with other challenges.</p><p>In <a href="https://blog.kubernauts.io/thanks-god-its-rancher-1edfa8b2fc82">a previous post</a> I wrote about a dilemma by deciding on how to run and manage multiple Kubernetes clusters using OpenShift, RKE, EKS or Kubeadm on AWS.</p><h3>Let’s get started</h3><p><strong>Prerequisites</strong></p><p>Most probably you have already these tools installed listed below, except mkcert and tk8ctl:</p><p><a href="https://docs.aws.amazon.com/cli/latest/userguide/installing.html">AWS CLI</a><br><a href="https://www.terraform.io/downloads.html">Terraform 0.12</a><br><a href="https://www.docker.com/products/docker-desktop">Docker for Desktop</a><br><a href="https://git-scm.com/book/en/v2/Getting-Started-The-Command-Line">git cli</a><br>mkcert<br>tk8ctl</p><h3>Get the source</h3><pre>git clone <a href="https://github.com/kubernauts/tk8-provisioner-cattle-eks.git">https://github.com/kubernauts/tk8-provisioner-cattle-eks.git<br></a>cd tk8-provisioner-cattle-eks</pre><h3>Install Rancher with Docker and mkcert</h3><p>As mentioned at the beginning we are going to use Rancher Server and Rancher’s API via code to deploy and manage the life cycle of our EKS clusters with tk8ctl and the Cattle EKS provisioner.</p><p>To keep things simple, we’ll install Rancher on our local machine with docker and <a href="https://github.com/FiloSottile/mkcert">mkcert</a> to get a valid SSL certificate in our browser, which we need to talk to with the following simple commands on MacOS (on Linux you need to follow these <a href="https://github.com/FiloSottile/mkcert#linux">mkcert instructions</a> and copy the rootCA.pem from the right directory on linux to your working directory):</p><pre><br>$ brew install mkcert</pre><pre>$ mkcert — install</pre><pre>$ mkcert &#39;*.rancher.svc&#39;</pre><pre># on MacOS</pre><pre># cp $HOME/Library/Application\ Support/mkcert/rootCA.pem cacerts.pem</pre><pre># on Ubuntu Linux</pre><pre># cp /home/ubuntu/.local/share/mkcert/rootCA.pem cacerts.pem</pre><pre># cp _wildcard.rancher.svc.pem cert.pem</pre><pre># cp _wildcard.rancher.svc-key.pem key.pem</pre><pre>$ sudo echo &quot;127.0.0.1 gui.rancher.svc&quot; &gt;&gt; /etc/hosts</pre><pre>docker run -d -p 80:80 -p 443:443 -v $PWD/cacerts.pem:/etc/rancher/ssl/cacerts.pem -v $PWD/key.pem:/etc/rancher/ssl/key.pem -v $PWD/cert.pem:/etc/rancher/ssl/cert.pem rancher/rancher:stable</pre><pre>$ open <a href="https://gui.rancher.svc/">https://gui.rancher.svc</a></pre><p>With that you should be able to access Rancher on <a href="https://gui.rancher.svc/">https://gui.rancher.svc</a> without TLS warnings!</p><h4>Get the tk8ctl CLI</h4><p>Download the <a href="https://github.com/kubernauts/tk8/releases">latest tk8ctl</a> release and place it in your path:</p><pre># On MacOS<br>$ wget <a href="https://github.com/kubernauts/tk8/releases/download/v0.7.7/tk8ctl-darwin-amd64">https://github.com/kubernauts/tk8/releases/download/v0.7.7/tk8ctl-darwin-amd64<br></a>chmod +x tk8ctl<a href="https://github.com/kubernauts/tk8/releases/download/v0.7.7/tk8ctl-darwin-amd64">-</a>darwin-amd64<br>mv tk8ctl<a href="https://github.com/kubernauts/tk8/releases/download/v0.7.7/tk8ctl-darwin-amd64">-</a>darwin-amd64 /usr/local/bin/tk8ctl</pre><pre>$ tk8ctl version</pre><pre># ignore any warnings for now, you’ll get a config.yaml file which we’ll overwrite shortly</pre><pre># On Linux</pre><pre>$ wget <a href="https://github.com/kubernauts/tk8/releases/download/v0.7.7/tk8ctl-linux-amd64">https://github.com/kubernauts/tk8/releases/download/v0.7.7/tk8ctl-linux-amd64<br></a>chmod +x tk8ctl-linux-amd64</pre><pre>$ sudo mv tk8ctl-linux-amd64 /usr/local/bin/tk8ctl</pre><pre>$ tk8ctl version</pre><pre># provide any value for aws access and secret key, you’ll get a config.yaml file which we’ll overwrite</pre><h4>Set AWS and Terraform Rancher Provider variables</h4><p>Get the bearer token from Rancher UI in the menu via API &amp; Keys and provide your AWS access and secret keys in a file called e.g. cattle_eks_env_vars.template:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7e1e05d06680754e8a20465782e29e06/href">https://medium.com/media/7e1e05d06680754e8a20465782e29e06/href</a></iframe><p>and source the file:</p><pre>$ source cattle_eks_env_vars.template</pre><h4>Deploy EKS with tk8ctl</h4><p>Now you’re ready to deploy EKS via Rancher API:</p><pre><br>$ cp example/config-eks-gui.rancher.svc.yaml config.yaml<br>$ tk8ctl cluster install cattle-eks<br></pre><p>After some seconds you should see in the Rancher Server GUI an EKS cluster in the provisioning state, take a cup of coffee or a delicious red wine, your EKS cluster needs about 15 min. to get ready.</p><h4>Access your EKS cluster</h4><p>To access your EKS Cluster you can either get the kubeconfig from Rancher UI and save it as kubeconfig.yaml and run:</p><pre>KUBECONFIG=kubeconfig.yaml kubectl get nodes</pre><p>or you can run the following aws eks command to update your default kubeconfig file with the new context:</p><pre>aws eks update-kubeconfig --name tk8-tpr2-eks</pre><h3>Clean-Up</h3><pre>tk8ctl cluster destroy cattle-eks</pre><h3>We’re hiring!</h3><p>We are looking for engineers who love to work in Open Source communities like Kubernetes, Rancher, Docker, etc.</p><p>If you wish to work on such projects please do visit our <a href="https://kubernauts.de/en/careers/">job offerings page</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d5f5c4ccf43e" width="1" height="1" alt=""><hr><p><a href="https://blog.kubernauts.io/tk8-cattle-eks-provisioner-with-terraform-rancher-provider-d5f5c4ccf43e">TK8 Cattle EKS Provisioner with Terraform Rancher Provider</a> was originally published in <a href="https://blog.kubernauts.io/">Kubernauts</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploy K8s using K8s with Cluster API and CAPA on AWS]]></title>
            <link>https://blog.kubernauts.io/deploy-k8s-using-k8s-with-cluster-api-and-capa-on-aws-107669808367?source=rss-d6c3d29f67f8------2</link>
            <guid isPermaLink="false">https://medium.com/p/107669808367</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[rancher]]></category>
            <category><![CDATA[cluster-api]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[open-source]]></category>
            <dc:creator><![CDATA[Arash]]></dc:creator>
            <pubDate>Thu, 26 Dec 2019 20:07:39 GMT</pubDate>
            <atom:updated>2019-12-28T13:55:22.150Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/653/0*AGoFPzvz-B1PkK3-.png" /></figure><p>This post provides some insights about Cluster API (CAPI), <a href="https://github.com/kubernetes-sigs/cluster-api">a Kubernetes Lifecycle SIG project</a> and how to use it together with <strong>Cluster API Provider for AWS</strong> (CAPA) on any Kubernetes Cluster to create a single or a fleet of Vanilla Kubernetes cluster(s) on AWS or other <a href="https://cluster-api.sigs.k8s.io/reference/providers.html">supported providers</a>.</p><p>Cluster API is very new and was inspired by Kubicorn and Kops projects and I guess some concepts was looked up by Rancher’s way of bootstrapping clusters.</p><p>At this time of writing the Cluster API project has reached the <strong>v1alpha3</strong> phase and I think with the right skills and backup and recovery strategy one can trust the project and use it in production as well.</p><p>In general on AWS I’d recommend to go with EKS with managed node, spot instances and auto-scaling support or in some cases even with the <a href="https://www.learnaws.org/2019/12/16/running-eks-on-aws-fargate/">Serverless Kubernetes on Fargate</a> solution, if you can live with the current Kubernetes 1.14.x supported version.</p><p>If you need to use the most recent K8s versions 1.16.x or soon the 1.17.x version for production on AWS, you might want to go with <a href="https://github.com/rancher/rancher">Rancher</a> and <a href="https://github.com/rancher/rkehttps://github.com/rancher/rke">Rancher’s Kubernetes Engine</a>.</p><p>In my next post I’ll introduce our own <a href="https://github.com/kubernauts/tk8-provisioner-cattle-aws">Cattle AWS</a> and <a href="https://github.com/kubernauts/tk8-provisioner-cattle-eks">Cattle EKS</a> implementations with the <a href="https://github.com/kubernauts/tk8s">tk8ctl</a> tool and show you how we define and deploy production ready clusters in a declarative way with a single YAML file.</p><p>Bootstrapping Kubernetes from within Kubernetes is not something new, Rancher is doing that for a long time, since Rancher is running on an embedded K3s cluster and is using CRDs to bootstrap any Kubernetes Cluster on any Provider.</p><p>Well, let’s stay by CAPI+CAPA and don’t dive into any religious discussions ;-) and see how innovation works and what is going to come soon in the CAPI space.</p><h3>The Cluster API Book and Related Resources</h3><p><a href="https://cluster-api.sigs.k8s.io/introduction.html">The Cluster API Book</a> and the related resources which are linked through this article provide mostly everything which one needs to know about Cluster API and how to get started.</p><p>The main <a href="https://github.com/kubernetes-sigs/cluster-api/blob/master/docs/scope-and-objectives.md">scope and objectives</a> of Cluster API is to provide a set of Kubernetes cluster management APIs and Kubernetes Operators to enable common cluster lifecycle operations like create, scale, upgrade, and destroy of Kubernetes Clusters across infrastructure providers including on-prem environments with bare-metal support.</p><p>In short, Cluster API (a.k.a CAPI) extends the base Kubernetes API to understand concepts such as Clusters, Machines, and Machine Deployments, as <a href="https://twitter.com/crmilsted">Chris Milsted</a> describes very nicely in his blog post with the title <a href="https://blogs.vmware.com/cloudnative/2019/12/12/how-cluster-api-promotes-self-service-infrastructure/">Taking Kubernetes to the People</a>.</p><p>With Cluster API, Kubernetes Clusters, Machines and Machine Deployments become the first class citizens on top of core Kubernetes API to build, deploy and manage the lifecycle of other Kubernetes clusters from within a Kubernetes cluster like minikube or a <a href="https://www.learnaws.org/2019/12/16/running-eks-on-aws-fargate/">serverless EKS cluster on fargate</a> or any other Kubernetes cluster.</p><p>With that said, let’s learn about CAPI’s API and Framework and how to get started on AWS and later on Bare-Metal or other providers.</p><h3>CAPI’s API and Operator Framework</h3><p>Cluster API provides a framework through Kubernetes Operators and a set of APIs defined through <a href="https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/">Custom Resources Definitions</a> (CRDs) to declaratively specify a <strong>Cluster</strong> spec including a <strong>Machine</strong> controlled by a <strong>MachineSet</strong> and a <strong>MachineDeployment</strong> through an infrastructure specific <strong>MachineTemplate</strong>, e.g. the AWSMachineTemplate.</p><h3>The Cluster CRD</h3><p>The CAPI defines a Cluster through a simple Cluster controller definition defined by a Cluster CRD and in our case with an infrastructure specific <strong>AWSCluster</strong> CRD, which will take over the control of our clusters from within an external K8s cluster to build and manage the lifecycle of a target K8s cluster.</p><p>The Cluster and AWSCluster CRDs are defined in the cluster.yaml, here we define the name of our cluster, which is aws-cluster-1, the cluster Pod Network cidrBlocks, the region where we want to deploy our cluster and finally the SSH Key in AWS. We will see later after applying the cluster.yaml a small micro bastion host (Ubuntu 18.04) will be provisioned a new VPC on AWS with a public IP which provides access to the master and worker nodes.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/298e4bfca780543c4ff2c62c53d30f1b/href">https://medium.com/media/298e4bfca780543c4ff2c62c53d30f1b/href</a></iframe><p>The graphical presentation of the Cluster CRD objects can be represented like this (we’ll see later how to create it by using <a href="https://github.com/vmware-tanzu/octant">octant</a>):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/651/0*iOKa5ye2HsdbgMOT" /></figure><h3>The Control Plane</h3><p>The Control Plane consists of a Machine CRD resource controlled by a MachineSet (think of it like a ReplicaSet) which defines a set or group of machines, these are the Control Plane (Master + ETCD) nodes. Through the controlplane.yaml we define one Control Plane, the K8s version (1.16.2), the bootstrapper (KubeadmConfig) and our infrastructure specific <strong>AWSMachine</strong> CRD which is presented like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a465888e0d7b14d443b7055133898f1e/href">https://medium.com/media/a465888e0d7b14d443b7055133898f1e/href</a></iframe><p>The graphical presentation of the ControlPlane CRD objects can be represented like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/455/0*61rn3VS-SkuYF-5C" /></figure><h3>The MachineDeployment CRD</h3><p>The MachineDeployment represents the deployment of a set of worker nodes defined through the spec.replicas filed within a nodepool-0 here. Think about a MachineDeployment like a standard Deployment object in Kubernetes which takes care about the scheduled pods, that said, the MachineDeployment takes care about the scheduled Machines (our worker nodes).</p><p>The YAML presentation of the MachineDeployment CRD has a reference to our infrastructure specific AWSMachineTemplate and the KubeadmConfigTemplate:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e5128ed3277562ff6bd882c355801c1c/href">https://medium.com/media/e5128ed3277562ff6bd882c355801c1c/href</a></iframe><h3>Deploy K8s using K8s with Cluster API</h3><p>As mentioned earlier we need first a Kubernetes Cluster running, either on our local machine or somewhere else. This first Kubernetes cluster will be our bootstrap / management cluster, where we are going to deploy the Cluster API on top and bootstrap a target worker cluster named aws-cluster-1 on AWS.</p><p>For the initial bootstrap cluster on our machine, which is going to become our management cluster, we can use <a href="https://github.com/kubernetes-sigs/kind">Kind</a>, <a href="https://github.com/rancher/k3d">k3d</a>, my own <a href="https://github.com/arashkaffamanesh/kubeadm-multipass">kubeadm-multipass</a> implementation or <a href="https://itnext.io/run-kubernetes-on-your-machine-7ee463af21a2">other solutions</a> over there, you have the freedom of choice.</p><p>By the way I tested the whole thing with this nice <a href="https://gist.github.com/mhausenblas/a1d195745267811b68055320f9844fe1">EKS on ARM</a> implementation by <a href="https://twitter.com/mhausenblas">Michale Hausenblas</a>, it worked like a charm.</p><p>And if you want to try <a href="https://medium.com/@dhimanhalder/how-i-ran-an-inexpensive-sandbox-multi-node-kubernetes-cluster-on-aws-using-ec2-spot-instances-238c74f163bf">this nice implementation</a> by <a href="https://medium.com/@dhimanhalder?source=post_page-----238c74f163bf----------------------">Dhiman Halder</a> with Kubeadm with Spot Instances support for 6$ / month to run your management cluster, please let me know if it worked.</p><h3>Let’s get started</h3><h3>Prerequisites</h3><p>A Kubernetes cluster (&gt; 1.15 due to cert-manager support?)<br>Linux or MacOS (Windows is not supported yet)<br>git cli<br>make (optional and nice to have)<br>Homebrew for Linux of MacOS<br><a href="https://docs.aws.amazon.com/cli/latest/userguide/installing.html">AWS CLI</a><br><a href="https://stedolan.github.io/jq/download/">jq</a><br><a href="https://github.com/kubernetes-sigs/kustomize">kustomize</a><br><a href="https://github.com/kubernetes-sigs/cluster-api-provider-aws/releases">clusterawsadm</a></p><p>As per <a href="https://github.com/kubernetes-sigs/cluster-api-provider-aws/blob/master/docs/prerequisites.md">Cluster API Provider AWS requirements</a> in order to use clusterawsadm you must have an administrative user in an AWS account. Once you have that administrator user you need to set your environment variables:</p><ul><li>AWS_REGION</li><li>AWS_ACCESS_KEY_ID</li><li>AWS_SECRET_ACCESS_KEY</li><li>AWS_SESSION_TOKEN (if you are using Multi-factor authentication)</li></ul><h4>CAPA: Cluster API Provider for AWS</h4><p>The <a href="https://github.com/kubernetes-sigs/cluster-api">Core Cluster API</a> (CAPI) project provides kubeadm as the default bootstrap provider named Cluster API Bootstrap Provider Kubeadm (CABPK) and there are infrastructure specific providers like <a href="https://github.com/kubernetes-sigs/cluster-api-provider-aws">Cluster API Provider for AWS</a> (CAPA) which is developed out-of-tree of the core project and extends the CAPI for their own environment, which we’re going to use now.</p><p>In the following steps we’re going to clone the source of the Cluster API Provider for AWS from Github, install kustomize (if not present on your machine), build the clusterawsadm binary with make, do some minor changes to the generate.sh script to create and apply a couple of manifests and turn our local bootstrap cluster to a management cluster which deploys and manages our first cluster aws-cluster-1.</p><h4>Get the source</h4><pre>git clone <a href="https://github.com/kubernetes-sigs/cluster-api-provider-aws.git">https://github.com/kubernetes-sigs/cluster-api-provider-aws.git</a></pre><h4>Install kustomize</h4><pre>brew install kustomize # if not already installed</pre><h4>Build the binaries with make</h4><pre>cd cluster-api-provider-aws/</pre><pre>make binaries</pre><pre># this will build the clusterawsadm and manager binaries and place it under the bin directory, alternatively you can download clusterawsadm form the release page and place it in the bin directory, since the generate.sh script expects to have the clusterawsadm in the bin directory</pre><h4>Create the CAPA CloudFormation Stack</h4><pre># import your public key-pair</pre><pre>aws ec2 import-key-pair --key-name default --public-key-material &quot;$(cat ~/.ssh/id_rsa.pub)&quot;</pre><pre># set the region</pre><pre>export AWS_REGION=eu-central-1</pre><pre>./bin/clusterawsadm alpha bootstrap create-stack</pre><p>The clusterawsadm command above will create a CloudFormation Stack named cluster-api-provider-aws-sigs-k8s-io which you shall see in the AWS Console with the status “Create Complete” before proceeding with the next steps:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/946/0*41GVSCCaE_MbLnfO" /></figure><h4>Adapt the generate.sh script</h4><p>You’ll find the generate.sh script under the examples folder of the repo, I’ve changed the cluster name and Kubernetes version to 1.16.2 like this:</p><pre># Cluster</pre><pre>export CLUSTER_NAME=”${CLUSTER_NAME:-<strong>aws-cluster-1</strong>}”</pre><pre>export KUBERNETES_VERSION=”${KUBERNETES_VERSION:-<strong>v1.16.2</strong>}”</pre><p>With all that in place we can now generate the needed manifests which will be placed under the examples/_out directory.</p><p>We can now run the following commands to turn our bootstrap cluster into the management cluster and deploy an HA’ed worker cluster with 3 Control Plane and 2 worker nodes:</p><h4>Cost notice</h4><p>You’l make Jeff Bezos happy with approx. $1 for one hour playing with this tutorial. If you’d like to have only 1 master and 1 worker node, you can replace the generated files with those files embedded above from my gist.</p><pre>./examples/generate.sh</pre><pre>kubectl create -f examples/_out/cert-manager.yaml</pre><pre>kubectl get all -n cert-manager</pre><pre># wait for cert-manager to get ready before creating the provider-components</pre><pre>kubectl create -f examples/_out/provider-components.yaml</pre><pre># create the Cluster CRD, this will create the bastion host in a new VPC</pre><pre>kubectl create -f examples/_out/cluster.yaml</pre><pre># create the Control Plane (3 stacked master + etcd nodes)</pre><pre>kubectl create -f examples/_out/controlplane.yaml</pre><pre># Create 2 worker nodes</pre><pre>kubectl create -f examples/_out/machinedeployment.yaml</pre><pre># wait until the machines are ready and create the kubeconfig file</pre><pre>kubectl --namespace=default get secret aws-cluster-1-kubeconfig -o json | jq -r .data.value | base64 --decode &gt; ./aws-cluster-1-kubeconfig</pre><pre># create the CNI (Calico)</pre><pre>kubectl --kubeconfig=./aws-cluster-1-kubeconfig apply -f <a href="https://docs.projectcalico.org/v3.8/manifests/calico.yaml">https://docs.projectcalico.org/v3.8/manifests/calico.yaml</a></pre><pre># or create CNI for Calico with the addon</pre><pre>kubectl --kubeconfig=./aws-cluster-1-kubeconfig create -f examples/addons.yaml</pre><pre># enjoy your cluster and machines</pre><pre>kubectl get clusters</pre><pre>kubectl get machines</pre><pre>kubectl get awsmachines</pre><pre>kubectl get machinedeployment</pre><pre>kubectl get node --kubeconfig=./aws-cluster-1-kubeconfig</pre><p>With that we are running an HA’ed Kubernetes worker cluster on AWS. Our management cluster is running locally, in the near future we’ll be able to move our management cluster to a target cluster by using pivoting with clusterctl.</p><h4>Clean-up and keep our planet happy</h4><p>To clean-up your test environment and keep our planet happy, please run:</p><pre>kubectl delete -f examples/_out/machinedeployment.yaml<br>kubectl delete -f examples/_out/controlplane.yaml<br>kubectl delete -f examples/_out/cluster.yaml</pre><h4>What’s coming next: Day one experience, Day 2 operations</h4><p>If you’ve been familiar with earlier 2 versions of Cluster API v1alpha1 or v1alpha2 or had a look through the CAPI Book, you might have read something about the <a href="https://cluster-api.sigs.k8s.io/tooling/clusterctl.html"><strong>clusterctl</strong></a> and asked yourself why I didn’t mentioned it till now. Well, <a href="https://github.com/kubernetes-sigs/cluster-api/blob/master/docs/proposals/20191016-clusterctl-redesign.md">clusterctl is undertaking a redesign</a> and is currently not available for v1alpha3.</p><p>But the good news is that clusterctl will continue to ease the init process and will be optimized for the “day one” experience of using Cluster API, and it will be possible to create a first management cluster with only two commands:</p><pre>$ clusterctl init --infrastructure aws<br>$ clusterctl config cluster my-first-cluster | kubectl apply -f -</pre><h3>We’re hiring!</h3><p>We are looking for engineers who love to work in Open Source communities like Kubernetes, Rancher, Docker, etc.</p><p>If you wish to work on such projects please do visit our <a href="https://kubernauts.de/en/careers/">job offerings page</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=107669808367" width="1" height="1" alt=""><hr><p><a href="https://blog.kubernauts.io/deploy-k8s-using-k8s-with-cluster-api-and-capa-on-aws-107669808367">Deploy K8s using K8s with Cluster API and CAPA on AWS</a> was originally published in <a href="https://blog.kubernauts.io/">Kubernauts</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>