<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Jay Sadhu on Medium]]></title>
        <description><![CDATA[Stories by Jay Sadhu on Medium]]></description>
        <link>https://medium.com/@jysadhu?source=rss-32ee9105f329------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*OI3_HuDsQpUIvYflG1W5gQ.jpeg</url>
            <title>Stories by Jay Sadhu on Medium</title>
            <link>https://medium.com/@jysadhu?source=rss-32ee9105f329------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 20:26:03 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@jysadhu/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[10 Kubernetes Errors I See Most Often (and How to Fix Them)]]></title>
            <link>https://medium.com/devops-ai-decoded/10-kubernetes-errors-i-see-most-often-and-how-to-fix-them-0589e8ccc8e2?source=rss-32ee9105f329------2</link>
            <guid isPermaLink="false">https://medium.com/p/0589e8ccc8e2</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[docker]]></category>
            <dc:creator><![CDATA[Jay Sadhu]]></dc:creator>
            <pubDate>Tue, 25 Aug 2026 05:41:58 GMT</pubDate>
            <atom:updated>2026-08-25T05:41:58.808Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uOplzauxOlwc_0XuDbHujw.png" /></figure><p>If you’ve spent more than a week running workloads on Kubernetes, you’ve almost certainly stared at a pod stuck in CrashLoopBackOff wondering what you did wrong. The truth is, most Kubernetes failures fall into a small, predictable set of buckets. Once you know the patterns, debugging stops feeling like magic and starts feeling like a checklist.</p><p>Here are the 10 errors I run into most often in production clusters — what causes them, how to diagnose them fast, and how to fix them for good.</p><h3>1. CrashLoopBackOff</h3><p><strong>What it means:</strong> Your container starts, crashes, and Kubernetes keeps restarting it with an exponential backoff delay.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/741/1*-74ECd4SDuqfEciBxJo0VQ.png" /></figure><p><strong>Common causes:</strong></p><ul><li>The application throws an unhandled exception on startup</li><li>A missing environment variable or config file</li><li>The container’s main process exits immediately (e.g., a misconfigured entrypoint)</li><li>A failing liveness probe restarting the pod repeatedly</li></ul><p><strong>How to diagnose:</strong></p><pre>kubectl describe pod &lt;pod-name&gt;<br>kubectl logs &lt;pod-name&gt; --previous</pre><p>The --previous flag is key — it shows logs from the last crashed container, not the current (empty) one.</p><p><strong>Fix:</strong> Nine times out of ten, the logs point straight to the issue — a bad config, a missing secret, or a dependency (like a database) that isn’t reachable yet. If it’s a startup timing issue, add an initContainer or increase initialDelaySeconds on your liveness probe.</p><h3>2. ImagePullBackOff / ErrImagePull</h3><p><strong>What it means:</strong> Kubernetes can’t pull the container image you specified.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*7l_eNzzcopAOkvWKA5Su4g.png" /></figure><p><strong>Common causes:</strong></p><ul><li>Typo in the image name or tag</li><li>Private registry without proper imagePullSecrets</li><li>Rate limiting from Docker Hub</li><li>Image simply doesn’t exist for the target architecture (common with ARM nodes)</li></ul><p><strong>How to diagnose:</strong></p><pre>kubectl describe pod &lt;pod-name&gt;</pre><p>Look at the Events section — it usually tells you exactly why the pull failed.</p><p><strong>Fix:</strong> Double-check the image tag, verify registry credentials are correctly referenced in the pod spec or service account, and confirm the image exists in the target registry for the correct platform.</p><h3>3. OOMKilled</h3><p><strong>What it means:</strong> Your container exceeded its memory limit and the kernel killed it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/898/1*pQnwE91COP018ydw59KWkg.png" /></figure><p><strong>Common causes:</strong></p><ul><li>Memory limits set too low for the actual workload</li><li>A memory leak in the application</li><li>No limits set at all, letting a pod consume node memory until it’s evicted</li></ul><p><strong>How to diagnose:</strong></p><pre>kubectl describe pod &lt;pod-name&gt;</pre><p>Check for Reason: OOMKilled under the last state. Also check kubectl top pod if metrics-server is installed.</p><p><strong>Fix:</strong> Right-size your resources.requests and resources.limits based on real usage data (tools like Vertical Pod Autoscaler or Goldilocks help here). If it&#39;s a genuine leak, fix the app — don&#39;t just throw more memory at it.</p><h3>4. Pending Pods</h3><p><strong>What it means:</strong> The pod is created but never gets scheduled onto a node.</p><p><strong>Common causes:</strong></p><ul><li>Insufficient CPU/memory across all nodes</li><li>Node selectors, taints, or affinity rules that no node satisfies</li><li>PersistentVolumeClaim can’t be bound</li></ul><p><strong>How to diagnose:</strong></p><pre>kubectl describe pod &lt;pod-name&gt;<br>kubectl get events --sort-by=&#39;.lastTimestamp&#39;</pre><p><strong>Fix:</strong> If it’s a resource issue, scale your node pool or adjust requests. If it’s affinity/taints, verify your nodeSelector or tolerations actually match your nodes. For PVC issues, check that your StorageClass and provisioner are working.</p><h3>5. Readiness/Liveness Probe Failures</h3><p><strong>What it means:</strong> Kubernetes thinks your app isn’t healthy, even though it might actually be running fine.</p><p><strong>Common causes:</strong></p><ul><li>Probe endpoint doesn’t exist or returns a non-200 status</li><li>App takes longer to start than initialDelaySeconds allows</li><li>Probe timeout too aggressive for a slow-starting service</li></ul><p><strong>How to diagnose:</strong></p><pre>kubectl describe pod &lt;pod-name&gt;</pre><p>Look for repeated “Readiness probe failed” or “Liveness probe failed” events.</p><p><strong>Fix:</strong> Tune initialDelaySeconds, periodSeconds, and timeoutSeconds to match your app&#39;s real startup behavior. Never copy-paste probe configs between services with very different startup times.</p><h3>6. DNS Resolution Failures Inside the Cluster</h3><p><strong>What it means:</strong> Pods can’t resolve service names like my-service.my-namespace.svc.cluster.local.</p><p><strong>Common causes:</strong></p><ul><li>CoreDNS pods are down or overloaded</li><li>Network policies blocking DNS traffic</li><li>Incorrect dnsPolicy on the pod spec</li></ul><p><strong>How to diagnose:</strong></p><pre>kubectl get pods -n kube-system -l k8s-app=kube-dns<br>kubectl exec -it &lt;pod-name&gt; -- nslookup kubernetes.default</pre><p><strong>Fix:</strong> Restart or scale up CoreDNS if it’s under memory pressure. Check NetworkPolicies aren’t accidentally blocking port 53. This is one of the most common “randomly flaky” issues in busy clusters.</p><h3>7. Service Not Routing Traffic to Pods</h3><p><strong>What it means:</strong> Your Service exists, but requests never reach the backend pods.</p><p><strong>Common causes:</strong></p><ul><li>Label selector on the Service doesn’t match pod labels</li><li>Pods aren’t passing readiness checks (so they’re excluded from Endpoints)</li><li>Wrong targetPort vs container port</li></ul><p><strong>How to diagnose:</strong></p><pre>kubectl get endpoints &lt;service-name&gt;</pre><p>If the endpoint list is empty, that’s your answer.</p><p><strong>Fix:</strong> Compare spec.selector on the Service against metadata.labels on the pods — a single mismatched label breaks everything. Also confirm the container is actually listening on the port you&#39;re forwarding to.</p><h3>8. PVC Stuck in Pending</h3><p><strong>What it means:</strong> Your PersistentVolumeClaim never gets bound to a PersistentVolume.</p><p><strong>Common causes:</strong></p><ul><li>No StorageClass defined or the default one is misconfigured</li><li>Requested storage size exceeds what’s available</li><li>Access mode mismatch (e.g., asking for ReadWriteMany on a provisioner that only supports ReadWriteOnce)</li></ul><p><strong>How to diagnose:</strong></p><pre>kubectl describe pvc &lt;pvc-name&gt;<br>kubectl get storageclass</pre><p><strong>Fix:</strong> Verify the StorageClass exists and its provisioner is healthy. Match your access mode to what your cloud provider or storage backend actually supports.</p><h3>9. Node NotReady</h3><p><strong>What it means:</strong> A node has stopped reporting healthy status to the control plane.</p><p><strong>Common causes:</strong></p><ul><li>Kubelet crashed or lost connectivity</li><li>Disk pressure or memory pressure on the node</li><li>Node-level networking issues (CNI plugin failure)</li></ul><p><strong>How to diagnose:</strong></p><pre>kubectl describe node &lt;node-name&gt;<br>kubectl get events --field-selector involvedObject.kind=Node</pre><p><strong>Fix:</strong> SSH into the node and check systemctl status kubelet and disk space (df -h). Often it&#39;s disk pressure from unpruned images/logs — docker system prune or equivalent buys you time while you fix root cause (usually log rotation or ephemeral storage limits).</p><h3>10. “Forbidden” RBAC Errors</h3><p><strong>What it means:</strong> A user, service account, or controller is denied access to a resource.</p><p><strong>Common causes:</strong></p><ul><li>Missing Role/ClusterRole bindings</li><li>Overly narrow permissions on a ServiceAccount</li><li>Namespace-scoped Role used where a ClusterRole was needed</li></ul><p><strong>How to diagnose:</strong></p><pre>kubectl auth can-i &lt;verb&gt; &lt;resource&gt; --as=&lt;user-or-serviceaccount&gt;</pre><p><strong>Fix:</strong> Use kubectl auth can-i to pinpoint exactly what&#39;s missing, then grant the least privilege necessary — resist the urge to just slap cluster-admin on everything to make the error go away.</p><h3>Closing Thoughts</h3><p>Most Kubernetes errors aren’t actually mysterious once you know where to look. kubectl describe and kubectl logs --previous will solve 80% of your problems on their own. The other 20% usually comes down to three things: resource limits, label/selector mismatches, or RBAC.</p><p>Bookmark this list — chances are you’ll be back the next time a pod refuses to start at 2 AM.</p><p><em>If you found this useful, follow for more hands-on DevOps and Kubernetes content — including deep dives on GitOps, cost optimization, and cluster security.</em></p><p><em>Connect with me on LinkedIn: </em><a href="https://www.linkedin.com/in/jysadhu"><em>linkedin.com/in/jysadhu</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0589e8ccc8e2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/devops-ai-decoded/10-kubernetes-errors-i-see-most-often-and-how-to-fix-them-0589e8ccc8e2">10 Kubernetes Errors I See Most Often (and How to Fix Them)</a> was originally published in <a href="https://medium.com/devops-ai-decoded">Devops &amp; AI Hub</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[10 Docker Best Practices That Separate Hobby Projects From Production Systems]]></title>
            <link>https://medium.com/devops-ai-decoded/10-docker-best-practices-that-separate-hobby-projects-from-production-systems-f52ccfd920ab?source=rss-32ee9105f329------2</link>
            <guid isPermaLink="false">https://medium.com/p/f52ccfd920ab</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Jay Sadhu]]></dc:creator>
            <pubDate>Mon, 24 Aug 2026 10:53:54 GMT</pubDate>
            <atom:updated>2026-08-24T10:53:54.947Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bu_X9rFdPbdRL-jSipDgMw.png" /></figure><h4>A field guide for DevOps engineers who are done getting paged at 2 AM over container issues</h4><p>Anyone can run docker build and docker run and get an app working on their laptop. Getting that same container to survive real production traffic — with security scanners, on-call rotations, and SLAs watching over your shoulder — is a completely different skill.</p><p>Over the years, a fairly consistent set of practices has emerged among teams running Docker at scale. None of them are exotic. All of them are the difference between a container platform that quietly does its job and one that becomes a source of incidents. Here are the ten that matter most.</p><h3>TL;DR — The Checklist Version</h3><pre>[ ] Minimal base image (Alpine / Slim / Distroless)<br>[ ] Non-root USER set in Dockerfile<br>[ ] Pinned versions, not :latest<br>[ ] Multi-stage build to drop build-time bloat<br>[ ] .dockerignore in place (no .git, node_modules, .env)<br>[ ] No secrets baked into image layers<br>[ ] HEALTHCHECK + K8s liveness/readiness/startup probes<br>[ ] CPU + memory limits set and monitored<br>[ ] Log rotation + centralized logging<br>[ ] Vulnerability scan gating CI/CD (Trivy/Scout/Grype)<br>[ ] Immutable, versioned tags — never :latest in prod<br>[ ] Multi-arch builds if your fleet isn&#39;t uniform x86</pre><p>If you’re only skimming, that’s the whole article. If you want the reasoning behind each line, keep reading.</p><h3>1. Start From Minimal Base Images</h3><p>Every package you add to a base image is something you now have to patch, scan, and defend. Alpine, Debian Slim, and distroless images strip out the extras and leave you with a smaller attack surface, a faster pull time, and less noise in your vulnerability reports. If your application doesn’t need a shell or a package manager at runtime, distroless is worth the extra setup effort.</p><h3>2. Stop Running as Root</h3><p>By default, most containers run as root — which means a compromised application has root access inside that container. Creating a dedicated, unprivileged user and switching to it before your app starts is a two-line fix with an outsized security payoff:</p><pre>USER appuser</pre><p>It won’t stop every attack, but it removes an entire category of privilege-escalation paths.</p><h3>3. Pin Your Image Versions</h3><p>FROM node:latest is a promise that your build will behave differently at some unpredictable point in the future. Pin to a specific version instead:</p><pre>FROM node:22-alpine</pre><p>In environments where reproducibility really matters, go a step further and pin by digest so the exact same bytes get pulled every time, regardless of tag mutations upstream.</p><h3>4. Build in Stages</h3><p>Multi-stage builds let you use a full toolchain to compile or bundle your app, then discard all of it in the final image:</p><pre>FROM node:22-alpine AS builder<br>WORKDIR /app<br>COPY package*.json ./<br>RUN npm ci<br>COPY . .<br>RUN npm run build<br><br>FROM node:22-alpine<br>WORKDIR /app<br>COPY --from=builder /app/dist ./dist<br>COPY --from=builder /app/node_modules ./node_modules<br>USER node<br>CMD [&quot;node&quot;, &quot;dist/server.js&quot;]</pre><p>The result: smaller images, fewer build-time dependencies leaking into production, and a smaller surface for vulnerability scanners to flag.</p><blockquote><strong><em>Quick win: use a </em></strong><strong><em>.dockerignore file.</em></strong><em> It&#39;s easy to overlook, and it&#39;s a classic novice trap. Without one, </em><em>COPY . . happily drags </em><em>.git, </em><em>node_modules, local </em><em>.env files, and build artifacts straight into your build context — inflating image size and, worse, sometimes leaking secrets or internal history into a layer. A few lines closes that gap in seconds:</em></blockquote><pre>.git<br>node_modules<br>.env<br>*.md</pre><h3>5. Keep Secrets Out of the Image, Period</h3><p>Hardcoding credentials into a Dockerfile is one of the most common — and most avoidable — security failures:</p><pre>ENV DB_PASSWORD=MySecret123</pre><p>Once that’s baked into a layer, it’s in your image history forever, regardless of whether you delete it in a later step. Use a real secrets manager — Kubernetes Secrets, AWS Secrets Manager, GCP Secret Manager, or HashiCorp Vault — and make sure .env files never make it into version control in the first place.</p><h3>6. Give Your Containers Health Checks</h3><p>A container that’s running isn’t the same as a container that’s <em>working</em>. A basic Docker-level check catches the obvious cases:</p><pre>HEALTHCHECK --interval=30s --timeout=5s --retries=3 \<br>  CMD curl -f http://localhost:3000/health || exit 1</pre><p>In Kubernetes, this concept splits into three distinct probes, each answering a different question:</p><ul><li><strong>Liveness probe</strong> — is this container broken and in need of a restart?</li><li><strong>Readiness probe</strong> — is this container ready to receive traffic right now?</li><li><strong>Startup probe</strong> — is this a slow-starting app that needs extra grace before the other probes kick in?</li></ul><p>Getting these right is often the difference between a rolling deploy that’s invisible to users and one that causes a spike in 502s.</p><h3>7. Put a Ceiling on Resource Usage</h3><p>Without limits, a single misbehaving container can starve everything else on the host:</p><pre>docker run \<br>  --memory=1g \<br>  --cpus=1.5 \<br>  --restart=unless-stopped \<br>  myapp:1.0.0</pre><p>Setting limits is only half the job — you also need visibility into CPU, memory, disk, network, and restart counts over time, so you can tell the difference between “this limit is too tight” and “this app has a memory leak.”</p><h3>8. Treat Logs as a First-Class Concern</h3><p>Unbounded logs will eventually fill a disk and take down a node that has nothing to do with the app that caused it. Rotate logs at the Docker level as a baseline:</p><pre>{<br>  &quot;log-driver&quot;: &quot;json-file&quot;,<br>  &quot;log-opts&quot;: {<br>    &quot;max-size&quot;: &quot;100m&quot;,<br>    &quot;max-file&quot;: &quot;5&quot;<br>  }<br>}</pre><p>But for anything beyond a single-node setup, ship logs to a centralized system — Grafana Loki, Elasticsearch, OpenSearch, or CloudWatch — so you can actually search across containers, nodes, and deployments when something goes wrong.</p><h3>9. Scan Every Image Before It Ships</h3><p>An unscanned image is a guess. Tools like <strong>Trivy</strong>, <strong>Docker Scout</strong>, and <strong>Grype</strong> turn that guess into a fact:</p><pre>trivy image myapp:1.0.0</pre><p>The real value shows up when scanning is wired into CI/CD as a gate — vulnerable images get blocked automatically, rather than relying on someone remembering to run a scan manually before a release.</p><h3>10. Make Every Deployment Immutable and Traceable</h3><p>myapp:latest tells you nothing about what&#39;s actually running. A versioned, immutable tag tells you everything:</p><pre>myapp:2026.08.24-abc123</pre><p>This one habit pays off repeatedly: reproducible builds, fast and confident rollbacks, a clean audit trail, and no ambiguity about what’s live at any given moment.</p><p>One more thing worth planning for as you version and tag images: <strong>architecture</strong>. If your team develops on Apple Silicon but deploys to x86 EC2 nodes — or you’re targeting ARM-based instances for cost savings — a single-architecture image will quietly fail or get silently emulated at a performance cost. docker buildx lets you build and push multi-arch manifests under one tag, so the right binary gets pulled automatically regardless of the host&#39;s CPU architecture. It&#39;s a small addition to your build step that avoids a very confusing class of &quot;works on my machine&quot; bugs.</p><h3>What a Production Pipeline Actually Looks Like</h3><p>Put these practices together and they group naturally into four phases:</p><p><strong>🏗️ Build Phase</strong> Git → CI Pipeline (e.g. Jenkins) → Multi-Stage Docker Build</p><p><strong>🔒 Security Phase</strong> Trivy Scan → Private Registry (e.g. ECR) → Immutable, Versioned Tag</p><p><strong>🚀 Runtime Phase</strong> Kubernetes / EKS → Liveness/Readiness/Startup Probes → CPU + Memory Limits</p><p><strong>📊 Observability Phase</strong> Prometheus + Grafana (metrics) → Loki (centralized logs) → Alerting</p><p>Each phase catches a different class of failure — the build phase keeps bad code out, the security phase keeps known vulnerabilities out, the runtime phase keeps one bad container from taking down its neighbors, and the observability phase makes sure that when something <em>does</em> go wrong, you find out from a dashboard instead of a customer.</p><h3>The Bottom Line</h3><p>Docker’s job in a mature pipeline isn’t to run production traffic directly — that’s what Kubernetes or a managed equivalent like EKS is for. Docker’s job is to package your application into something small, secure, and reproducible enough that the orchestration layer can do its job well.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DlCxJCK-O2iHNudLnuqAKA.png" /></figure><p>A pipeline built around <strong>Git → CI → Docker Build → Vulnerability Scan → Registry → Kubernetes → Monitoring → Alerting</strong> isn’t complicated for complexity’s sake — every stage exists because it closes a specific failure mode. Skip enough of them, and you’re not running a production system; you’re running a demo that hasn’t failed yet.</p><p><strong>Which of these ten has saved your team from a 2 AM page — or which one did you learn the hard way?</strong> Let me know in the comments.</p><p><em>Connect with me on LinkedIn: </em><a href="https://www.linkedin.com/in/jysadhu"><em>linkedin.com/in/jysadhu</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f52ccfd920ab" width="1" height="1" alt=""><hr><p><a href="https://medium.com/devops-ai-decoded/10-docker-best-practices-that-separate-hobby-projects-from-production-systems-f52ccfd920ab">10 Docker Best Practices That Separate Hobby Projects From Production Systems</a> was originally published in <a href="https://medium.com/devops-ai-decoded">Devops &amp; AI Hub</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I Stopped Debugging Cron Jobs at Midnight — Here’s the Workflow Tool That Replaced Them]]></title>
            <link>https://medium.com/devops-ai-decoded/i-stopped-debugging-cron-jobs-at-midnight-heres-the-workflow-tool-that-replaced-them-c70df581a63d?source=rss-32ee9105f329------2</link>
            <guid isPermaLink="false">https://medium.com/p/c70df581a63d</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[kestra]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[scheduling]]></category>
            <dc:creator><![CDATA[Jay Sadhu]]></dc:creator>
            <pubDate>Wed, 19 Aug 2026 11:31:01 GMT</pubDate>
            <atom:updated>2026-08-19T11:31:01.374Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XUsVCRS7nsEQ5YkNHehwIQ.png" /></figure><p>The pipeline failed at 11:47 PM.</p><p>Not loudly — nothing crashed with a helpful stack trace pointing at the problem. It just… stopped. Somewhere in a chain of three Python scripts glued together by a cron job, one step silently failed, and the next two scripts happily ran anyway, on missing data, producing garbage output that nobody noticed until the morning standup.</p><p>I spent the next hour doing what every engineer who’s built automation out of scripts and cron has done at some point: SSH-ing into a server, tailing log files that only existed on that one box, trying to reconstruct what happened by staring at timestamps. There was no visual of the pipeline. No “resume from here” button. Just me, a terminal, and the slow process of piecing together a failure after the fact.</p><p>That night is why <strong>Kestra</strong> earned a place in my toolkit. It’s an open-source, event-driven workflow orchestration platform built on a simple premise: automation shouldn’t be a pile of scripts held together by cron and hope. It should be declarative, visible, and recoverable — something you can actually <em>see</em> running, and actually <em>fix</em> without starting over from scratch.</p><p>This post walks through what Kestra actually is, and how to build a real pipeline with it — one that pulls stories from Hacker News, summarizes them with an LLM, and posts the results to Slack, entirely defined in readable YAML.</p><h3>The Problem With Scripts and Cron Jobs</h3><p>Scripts-and-cron isn’t a bad idea because it doesn’t work — it’s a bad idea because of what happens when it <em>doesn’t</em>.</p><p>A typical setup looks something like: a handful of Python scripts, each doing one part of a job, wired together by a cron schedule and a prayer that they run in the right order, on time, every time. It’s simple to set up. It’s miserable to maintain.</p><p>The core problems compound quietly:</p><ul><li><strong>No visibility.</strong> There’s no dashboard showing you what ran, what failed, or why. You find out something broke because someone downstream noticed bad data — not because the system told you.</li><li><strong>No structured logging or error handling.</strong> Debugging usually means SSH-ing into whatever box the cron job happened to run on and grepping through log files that may or may not still exist.</li><li><strong>No replay.</strong> If step three of five fails, your only real option is often rerunning the entire pipeline from step one — even if steps one and two worked fine and just wasted compute and time doing it again.</li></ul><p>None of this is a knock on the people who build these pipelines. It’s what happens when the tooling never grew past “a script and a schedule.” Kestra exists to fix exactly this gap.</p><h3>What Kestra Actually Is</h3><p>Kestra is a workflow orchestration platform where the entire pipeline — every step, every dependency, every integration — is defined declaratively in YAML instead of imperative code. You’re not writing a script that runs other scripts. You’re describing <em>what</em> should happen, in what order, and Kestra handles the <em>how</em>: execution, retries, logging, and recovery.</p><p>It’s event-driven at its core, meaning workflows aren’t limited to “run on a schedule” — they can be triggered by webhooks, file arrivals, messages on a queue, or the completion of other workflows. Cron jobs can only ask “has enough time passed?” Kestra can ask “did something actually happen?” — which is a fundamentally more useful question for most real automation.</p><h3>Building It for Real: A Hacker News AI Digest</h3><p>Theory is easy to nod along to. Let’s build something real: a pipeline that fetches the top stories from Hacker News, summarizes them with an LLM, and posts a clean digest straight to Slack — the kind of small, genuinely useful automation most teams have on their “someday” list and never actually build.</p><p><strong>The workflow logic</strong> breaks into three clear stages:</p><ol><li><strong>Fetch</strong> — pull the current top stories from the Hacker News API.</li><li><strong>Summarize</strong> — pass each story through an LLM (Kestra supports providers like Anthropic’s Claude or a self-hosted option like Ollama) to generate a short, readable summary.</li><li><strong>Notify</strong> — post the finished digest to a Slack channel via a webhook.</li></ol><p><strong>Environment setup</strong> is refreshingly boring, which is exactly the point. A local .env file holds your secrets and API keys, and a Docker Compose file spins up everything you need to run Kestra locally — the Kestra UI and a Postgres database backing it — with a single command. No manual server provisioning, no separately-managed database setup.</p><p><strong>The pipeline itself is just YAML.</strong> Instead of three separate scripts calling three separate APIs and hoping they hand off cleanly, the entire flow — fetching from Hacker News, calling the LLM, formatting the digest, posting to Slack — lives in one readable, version-controllable file. You can look at the YAML and understand the whole pipeline in the time it takes to read it, which is not something you can say about three scripts spread across a cron table.</p><h3>Debugging: Actually Seeing What Happened</h3><p>Here’s where Kestra earns the “never again” feeling for anyone who’s lived through a midnight cron failure.</p><p><strong>A live topology graph</strong> shows your pipeline visually as it executes — which steps succeeded, which failed, and exactly where in the chain things went wrong. No more reconstructing the failure from scattered log files after the fact. You open the execution, and the failure point is right there, visually, immediately.</p><p><strong>Recovery doesn’t mean starting over.</strong> Say a task fails because of something small and dumb — an invalid header, an expired secret key. In the scripts-and-cron world, your options are usually “rerun everything” or “manually patch state and hope.” Kestra lets you fix the underlying configuration and <strong>restart from the exact point of failure</strong>, picking the workflow back up instead of re-running every step that already succeeded. For a pipeline with expensive or rate-limited steps — like, say, LLM API calls — that difference alone can save real time and real money.</p><h3>From Local Dev to Production</h3><p>The open-source version of Kestra is genuinely excellent for local development and smaller-scale automation — everything in this post runs on it. But production environments usually need more than a good local setup: proper single sign-on, audit logging for compliance, and role-based access control so not everyone on the team has the same level of access to every workflow.</p><p>That’s where <strong>Kestra Enterprise Edition</strong> comes in — the same declarative, event-driven core, with the governance layer production teams actually need once a workflow stops being a personal automation and starts being something the whole organization depends on.</p><h3>The Pipeline That Tells You What Happened</h3><p>Looking back at that 11:47 PM failure, the actual bug wasn’t even the hard part to fix. The hard part was <em>finding</em> it — no visibility, no clean recovery path, just a terminal and a slow reconstruction of what went wrong.</p><p>That’s the real value of a tool like Kestra. It’s not that YAML is inherently better than Python — it’s that a declarative, visible, recoverable pipeline turns “something broke, good luck” into “here’s exactly where it broke, and here’s how to pick back up.” The Hacker News digest in this post is intentionally small — but the same pattern scales to the pipelines that actually matter: data syncs, ML pipelines, notification systems, anything currently held together by a cron job and a script nobody wants to touch.</p><p>If you’re still debugging automation at midnight with nothing but log files and a terminal, it might be time to give your pipelines a place to actually show you what happened.</p><h3>Resources &amp; Links</h3><ul><li><a href="https://fandf.co/4uGuh97">GitHub Repository — All Code &amp; YAML Files</a></li><li><a href="https://fandf.co/3QLAoKN">Kestra Blueprint: Hacker News AI Digest</a></li><li><a href="https://fandf.co/4vphxmS">Kestra Enterprise Edition</a></li><li><a href="https://api.slack.com/apps">Create a Slack App &amp; Webhook</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c70df581a63d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/devops-ai-decoded/i-stopped-debugging-cron-jobs-at-midnight-heres-the-workflow-tool-that-replaced-them-c70df581a63d">I Stopped Debugging Cron Jobs at Midnight — Here’s the Workflow Tool That Replaced Them</a> was originally published in <a href="https://medium.com/devops-ai-decoded">Devops &amp; AI Hub</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[CI/CD vs GitOps vs MLOps: A Complete DevOps Comparison Guide (2026)]]></title>
            <link>https://medium.com/devops-ai-decoded/ci-cd-vs-gitops-vs-mlops-a-complete-devops-comparison-guide-2026-8f256e246403?source=rss-32ee9105f329------2</link>
            <guid isPermaLink="false">https://medium.com/p/8f256e246403</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[mlops]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[Jay Sadhu]]></dc:creator>
            <pubDate>Tue, 18 Aug 2026 05:47:24 GMT</pubDate>
            <atom:updated>2026-08-18T05:47:24.775Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yJjb9TXakJ2XRhVlIZWvvA.png" /></figure><h3>Introduction: Why DevOps Engineers Need to Understand All Three</h3><p>If you work in DevOps, Platform Engineering, or MLOps, you’ve likely heard <strong>CI/CD</strong>, <strong>GitOps</strong>, and <strong>MLOps</strong> used almost interchangeably — but they are not the same thing. Each solves a different automation problem: shipping code, reconciling infrastructure, or deploying machine learning models.</p><p>This guide breaks down each pipeline stage by stage, compares the tools that power them (ArgoCD, Flux, OPA, Kyverno, vLLM, TensorRT), and closes with a proven DevOps best practice you can apply today: <strong>shifting policy validation left</strong>.</p><p>Whether you’re preparing for a DevOps interview, designing a platform engineering roadmap, or just trying to decide which pipeline fits your team, this comparison will save you hours of research.</p><h3>What Is CI/CD? (Continuous Integration / Continuous Deployment)</h3><p><strong>CI/CD</strong> is the foundational DevOps pipeline for software delivery. It answers one question: <em>is this commit safe to deploy?</em></p><h3>CI/CD Pipeline Stages</h3><ol><li><strong>Write Code → Commit &amp; Push</strong> — the starting point of every CI/CD workflow</li><li><strong>Run Tests</strong> — unit tests and integration tests execute in parallel for fast feedback</li><li><strong>Static Analysis</strong> — SAST/lint and dependency scanning catch security vulnerabilities early</li><li><strong>Build Artifact</strong> — produces a reproducible, versioned build</li><li><strong>Push to Registry</strong> — artifacts are tagged, versioned, and cryptographically signed</li><li><strong>Deploy to Staging</strong> — followed by smoke tests, E2E tests, and performance tests</li><li><strong>Approval Gate</strong> — a manual or policy-driven checkpoint before production</li><li><strong>Deploy to Production</strong> — the final release step</li></ol><h3>Why CI/CD Matters</h3><p>CI/CD pipelines are <strong>sequential by design</strong>. Cheap, fast checks (unit tests, lint) run first; expensive, slow checks (E2E tests, human approval) run last. This “fail fast, fail cheap” ordering is the single biggest reason CI/CD reduces the cost of bugs.</p><h3>What Is GitOps? (Git-Driven Infrastructure Automation)</h3><p><strong>GitOps</strong> extends CI/CD principles to infrastructure management. The core idea: <strong>Git is the single source of truth</strong>, and a controller continuously reconciles your live environment to match it.</p><h3>GitOps Pipeline Stages</h3><ol><li><strong>Git as Source of Truth</strong> — all infrastructure state lives in version control</li><li><strong>Declarative Manifests</strong> — written using Helm Charts or Kustomize</li><li><strong>Open Pull Request</strong> — infrastructure changes follow the same review process as application code</li><li><strong>Policy Validation</strong> — enforced automatically via OPA Gatekeeper or Kyverno</li><li><strong>Merge to Main</strong> — changes land in the GitHub/GitLab repository</li><li><strong>GitOps Operator Watches</strong> — ArgoCD or Flux monitors the repo for changes</li><li><strong>Sync to Cluster</strong> — the operator applies changes to the Kubernetes cluster</li><li><strong>Drift Detection</strong> — alerts on configuration drift, with auto-remediation</li><li><strong>Secrets Management</strong> — handled securely via Sealed Secrets or Vault ESO</li><li><strong>Continuous Reconcile</strong> — the loop repeats indefinitely</li></ol><h3>GitOps vs CI/CD: The Key Difference</h3><p>CI/CD <strong>pushes</strong> changes into an environment. GitOps <strong>pulls</strong> changes via an in-cluster operator that constantly reconciles state. This makes GitOps environments <strong>self-healing</strong> — manual changes made outside Git get automatically reverted, which is why ArgoCD and Flux have become the de facto standard for Kubernetes deployment automation.</p><h3>What Is MLOps? (Machine Learning Operations)</h3><p><strong>MLOps</strong> applies DevOps principles to machine learning, but with a critical difference: models degrade silently over time, and correctness is statistical, not binary.</p><h3>MLOps Pipeline Stages</h3><ol><li><strong>Data Ingestion → Feature Engineering</strong> — runs alongside data validation and feature store updates</li><li><strong>Train Model</strong> — with HPO (hyperparameter optimization) tuning and experiment tracking</li><li><strong>Evaluate Model</strong> — measured on accuracy/F1 score <strong>and</strong> fairness checks</li><li><strong>Register Model</strong> — versioned in a model registry, just like a software artifact</li><li><strong>Package &amp; Serve</strong> — using vLLM/TensorRT for optimized inference, exposed via REST/gRPC</li><li><strong>A/B Testing</strong> — validated against live production traffic</li><li><strong>Monitor Inference</strong> — tracking data drift and latency/SLA in real time</li><li><strong>Trigger Retraining</strong> — loops back to model training when drift is detected</li></ol><h3>Why MLOps Pipelines Are Cyclical, Not Linear</h3><p>Unlike CI/CD and GitOps, which terminate once a deployment is stable, <strong>MLOps pipelines never fully stop</strong>. A model that performs well at launch can become inaccurate months later as real-world data drifts away from training data. Continuous monitoring and automated retraining triggers are what separate mature MLOps practices from one-off model deployments.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KD0JeotvnwrPgHTmCEP8pw.png" /></figure><h3>What CI/CD, GitOps, and MLOps Have in Common</h3><p>Despite shipping different artifacts, all three pipelines share the same underlying DevOps philosophy:</p><ul><li><strong>Version everything</strong> — code, manifests, and models</li><li><strong>Automate validation gates</strong> before anything reaches production</li><li><strong>Use a registry</strong> as the handoff between “built” and “deployable”</li><li><strong>Monitor continuously</strong> after release, not just before it</li><li><strong>Define a rollback or remediation path</strong> for when things break</li></ul><p>If you’re designing internal platform tooling, this shared spine is exactly where to standardize — logging, secrets management, and policy enforcement don’t need three separate implementations.</p><h3>DevOps Best Practice: Shift Policy Validation Left</h3><p>The single highest-leverage practice across CI/CD, GitOps, and MLOps is the one GitOps makes most explicit: <strong>enforce policy validation automatically, before merge — not after deployment.</strong></p><p>In CI/CD, this looks like SAST and dependency scanning. In GitOps, it’s OPA Gatekeeper or Kyverno blocking non-compliant Kubernetes manifests at the pull request stage. In MLOps, it’s automated fairness and accuracy thresholds gating model registration.</p><h3>How to Implement Policy-as-Code in Your Pipeline</h3><ol><li><strong>Define policies as code</strong> — use Rego (OPA) or CEL (Kyverno) and version them alongside your manifests</li><li><strong>Run policy checks pre-merge</strong> — never let non-compliant changes reach the reconciliation loop</li><li><strong>Fail closed by default</strong> — if a policy check errors, block the deploy rather than letting it pass silently</li><li><strong>Apply the same rigor to MLOps</strong> — treat a fairness or accuracy regression like a failed unit test, not an optional metric to review later</li></ol><p>A human reviewer approving a pull request is fallible and slow. A policy engine that automatically rejects a privileged container spec — or a model that fails a fairness threshold — is fast, consistent, and never gets tired on a Friday afternoon deploy.</p><h3>Conclusion</h3><p>CI/CD, GitOps, and MLOps solve different problems — shipping code, reconciling infrastructure, and deploying models — but they’re built on the same trust-machine philosophy: <strong>prove automatically that what’s about to go live is safe, as early in the pipeline as possible.</strong></p><p>Whether you’re a DevOps engineer standardizing deployment pipelines, a platform engineer rolling out GitOps with ArgoCD, or an ML engineer building retraining loops, mastering all three pipelines — and the shared best practice of shifting validation left — is what separates a good DevOps engineer from a great one.</p><p><em>Found this comparison useful? Follow for more deep dives into DevOps, GitOps, and MLOps pipeline design.</em></p><p><em>Connect with me on LinkedIn: </em><a href="https://www.linkedin.com/in/jysadhu"><em>linkedin.com/in/jysadhu</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8f256e246403" width="1" height="1" alt=""><hr><p><a href="https://medium.com/devops-ai-decoded/ci-cd-vs-gitops-vs-mlops-a-complete-devops-comparison-guide-2026-8f256e246403">CI/CD vs GitOps vs MLOps: A Complete DevOps Comparison Guide (2026)</a> was originally published in <a href="https://medium.com/devops-ai-decoded">Devops &amp; AI Hub</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Docker for AI/ML: The Crash Course That Gets Your Model Out of the Notebook and Into Production]]></title>
            <link>https://medium.com/devops-ai-decoded/docker-for-ai-ml-the-crash-course-that-gets-your-model-out-of-the-notebook-and-into-production-33d5f7e35b0d?source=rss-32ee9105f329------2</link>
            <guid isPermaLink="false">https://medium.com/p/33d5f7e35b0d</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Jay Sadhu]]></dc:creator>
            <pubDate>Mon, 17 Aug 2026 05:43:37 GMT</pubDate>
            <atom:updated>2026-08-17T05:53:35.951Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ETqTLsUaUf3dXoD2ytoZ6g.png" /></figure><p>A data scientist on my team once handed me a model with a shrug and a sentence that should terrify every ML engineer:</p><p><em>“It ran fine in my notebook.”</em></p><p>I believed her. That was the problem. It probably <em>had</em> run fine — on her laptop, with her exact CUDA version, her exact PyTorch build, her exact pip install history going back eighteen months, and a folder of model weights sitting nowhere except her Downloads folder.</p><p>None of that traveled with the model. So when we tried to serve it, it didn’t just run slow — it didn’t run at all. Missing library. Wrong CUDA version. A weights file the deployment script assumed would just… be there.</p><p>That’s “it works on my machine” — except the ML version is worse than the normal software version. A web app’s dependencies are a few libraries. A model’s dependencies are a deep learning framework, a specific GPU driver stack, gigabytes of weights, and a training environment that was never written down anywhere except one person’s memory.</p><p>Docker is the answer to this — not as a nice-to-have, but as close to mandatory as anything gets in modern ML infrastructure. It’s the difference between “it ran in my notebook” and “it runs, identically, everywhere, forever.” This is the crash course built specifically for that problem: Docker, through the lens of actually shipping models.</p><h3>Why Containers Matter More for ML Than for Almost Anything Else</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*cYsMcqbzf6Fk8P-I" /></figure><p>Every engineering team eventually discovers containers. ML teams discover <em>why they can’t live without them</em> — because the size and fragility of ML dependencies make the old alternative, virtual machines, almost unworkable.</p><p>A VM carries a full guest operating system on top of the host. For a normal web app, that’s wasteful but tolerable. For an ML workload — where you’re already hauling around multi-gigabyte CUDA libraries and model weights — stacking a second full OS on top is a tax you can’t afford, especially when you’re paying for GPU compute by the hour.</p><p>Containers skip that entirely. They share the host’s kernel and package only what the model actually needs to run. The result: containers boot in seconds instead of minutes, and you can pack far more inference workloads onto the same GPU-backed hardware than an equivalent VM setup would ever allow. When your compute bill is dominated by GPU time, that efficiency isn’t a nice bonus — it’s real money.</p><h3>Docker’s Architecture: What’s Actually Running Your Model</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/492/0*0Ysm8vmq2xAL7axl" /></figure><p>Docker works as a client-server system, and for ML workflows this split matters more than people expect.</p><p>The <strong>Docker Client</strong> — the CLI you type docker build and docker run into — doesn&#39;t do any of the actual work. It sends instructions to the <strong>Docker Daemon</strong>, the background engine that manages your images, containers, networks, and volumes.</p><p>Why does this matter for ML specifically? Because training and serving often don’t happen on your laptop at all — they happen on a remote GPU box, or a cluster node. The client-server split is exactly what lets you point your local docker CLI at a daemon running on a completely different, GPU-equipped machine, and build or run there without ever leaving your terminal.</p><h3>Docker Desktop: Native Docker on a Non-Linux Data Science Laptop</h3><p>Containers are fundamentally a Linux technology. Most ML engineers, meanwhile, are on a Mac. <strong>Docker Desktop</strong> bridges that gap by quietly running a lightweight Linux VM in the background to host the daemon, while giving you a native-feeling CLI and interface on top.</p><p>Practically, this means you can prototype your model’s Dockerfile locally on your Mac, using the exact same commands you’ll later run on a Linux GPU server in production — no surprises when you move from laptop to cluster.</p><h3>The ML Dockerfile: Not Your Average Image</h3><p>A typical web app Dockerfile is small and boring. An ML Dockerfile is a different beast entirely — because you’re usually pulling in a deep learning framework, GPU driver libraries, and model weights that can run from hundreds of megabytes to tens of gigabytes.</p><p>A <strong>Dockerfile</strong> is the text file with the build instructions; the <strong>Image</strong> it produces is the read-only template you’ll actually run. For ML specifically, how you write that Dockerfile has an outsized impact on your sanity:</p><ul><li><strong>Pin everything.</strong> Framework version, CUDA version, driver version. “Latest” is exactly how a model that worked last week silently breaks this week, with no code change to blame.</li><li><strong>Order your layers around what actually changes.</strong> Docker caches each layer and skips rebuilding ones that haven’t changed. Install your framework and dependencies early — they rarely change. Copy in your model weights and inference code last — they change constantly. Get this order right, and an update to your serving script rebuilds in seconds instead of re-downloading gigabytes of CUDA libraries from scratch.</li><li><strong>Use multi-stage builds.</strong> Compile or install heavy build tooling in one stage, then copy only the finished artifacts into a slim runtime image. Your training environment doesn’t need to ship to production — just its output.</li></ul><p>This alone is usually the difference between a five-minute rebuild and a forty-five-minute one, multiplied by every engineer on your team, every single day.</p><p>Here’s what that actually looks like in practice — a multi-stage Dockerfile for serving a PyTorch model, with dependencies pinned and layered by how often they change:</p><pre># ---- Stage 1: Build ----<br>FROM nvidia/cuda:12.1.0-runtime-ubuntu22.04 AS builder</pre><pre>WORKDIR /app</pre><pre># Rarely changes — cached almost every rebuild<br>RUN apt-get update &amp;&amp; apt-get install -y python3.11 python3-pip</pre><pre># Pin exact versions — no &quot;latest&quot;, ever<br>COPY requirements.txt .<br>RUN pip install --no-cache-dir -r requirements.txt</pre><pre># ---- Stage 2: Runtime ----<br>FROM nvidia/cuda:12.1.0-runtime-ubuntu22.04</pre><pre>WORKDIR /app</pre><pre># Copy only what&#39;s needed to run — not the build tools<br>COPY --from=builder /usr/lib/python3.11 /usr/lib/python3.11<br>COPY --from=builder /usr/local/lib/python3.11 /usr/local/lib/python3.11</pre><pre># Changes constantly — kept last so it doesn&#39;t bust the cache above<br>COPY ./model_weights /app/model_weights<br>COPY ./src /app/src</pre><pre>EXPOSE 8000<br>CMD [&quot;python3&quot;, &quot;src/serve.py&quot;]</pre><p>Notice the order: base image and system packages first (barely ever change), pinned dependencies next (change occasionally), model weights and serving code last (change constantly). Rebuild after tweaking serve.py, and Docker skips straight past the CUDA and dependency layers — reusing what&#39;s cached and rebuilding only the last two lines. That&#39;s the whole trick.</p><h3>Containers: Where Training and Inference Actually Happen</h3><p>A container is a running instance of an image — the same relationship an object has to a class. For ML workloads, containers typically show up in two very different shapes:</p><p><strong>Training containers</strong> are usually short-lived and resource-hungry: spin up, consume as much GPU as you’ll give them, run for hours or days, then exit. They move through the standard lifecycle — Created, Running, Paused, Stopped, Removed — but the “Running” phase is where all the cost lives, which is exactly why crashes here are so painful. A training container that dies six hours in without checkpointing doesn’t just fail; it burns real GPU-hours for nothing.</p><p><strong>Inference containers</strong> are the opposite: long-running, expected to stay healthy continuously, serving requests indefinitely rather than finishing a job and exiting. The operational bar is different — uptime and latency matter more than raw throughput per run.</p><p>Knowing which kind of container you’re building changes almost every decision downstream, from how much you invest in checkpointing to how aggressively you autoscale.</p><h3>GPU Access: The Step That’s Easy to Silently Get Wrong</h3><p>Here’s the trap that catches almost everyone the first time: <strong>a container cannot see your GPU by default.</strong> Docker isolates containers from host hardware unless you explicitly grant access — typically via the NVIDIA Container Toolkit and a --gpus all flag at runtime.</p><p>Miss this step, and nothing errors out. Your model just quietly falls back to CPU — technically working, running five to ten times slower, and giving you no obvious signal that anything’s wrong until someone asks why inference suddenly takes eight seconds instead of eight hundred milliseconds. This is one of the most common — and most quietly expensive — ML deployment mistakes, precisely because it fails silently instead of loudly.</p><h3>Storage: Model Weights, Datasets, and the Ephemeral Container Problem</h3><p>Containers are ephemeral — stop and remove one, and anything it wrote internally is gone. For ML, that’s a real risk: training checkpoints, logged experiment artifacts, and downloaded datasets can’t afford to vanish when a container restarts.</p><p>Docker gives you two tools for this:</p><ul><li><strong>Volumes</strong> — storage Docker manages independently of any single container’s lifecycle. This is where production model weights and training checkpoints belong: durable, decoupled from the container, safe if the container crashes or gets replaced.</li><li><strong>Bind Mounts</strong> — a direct link between a host directory and a container directory. Ideal for active development: point a bind mount at your local dataset or model code, and changes reflect instantly inside the container without a rebuild.</li></ul><p>The rule of thumb: bind mounts while you’re iterating on training code locally, volumes once you’re persisting real checkpoints, weights, or artifacts that need to survive and move with the workload.</p><h3>Networking: Getting GPUs to Talk to Each Other</h3><p>Single-GPU, single-container inference doesn’t need much networking sophistication. Distributed training does — badly.</p><p>Docker’s default <strong>Bridge</strong> network handles container-to-container communication on a single host, which covers plenty of inference setups. But real distributed training frequently spans <em>multiple physical machines</em>, each with their own GPUs, that need to coordinate as if they were on one network. That’s what <strong>Overlay</strong> networking is built for — and it’s the same underlying capability that lets orchestration platforms coordinate distributed training jobs across a whole cluster instead of a single box.</p><p>If you’ve ever wondered how a training job splits gradient updates across GPUs sitting on entirely different servers, overlay-style networking is the plumbing underneath that magic.</p><h3>Serving the Model: From Weights to an Actual Endpoint</h3><p>A model sitting inside a container isn’t useful by itself — something has to wrap it and expose it to the outside world. Typically that’s a lightweight framework like FastAPI or Flask, serving a /predict endpoint.</p><p>The mistake that quietly kills latency: loading the model <em>inside</em> the request handler instead of once at container startup. Do that, and every single request pays the full cost of loading multi-gigabyte weights from disk — turning a should-be-fast endpoint into one that times out under any real load. Load once, serve many times. It sounds obvious written down; it’s one of the most common real-world performance bugs in model-serving containers.</p><p><strong>Versioning matters here more than almost anywhere else.</strong> Tag every model image with the actual model version — not latest — so that when a model starts behaving strangely in production, you can answer &quot;which exact model and code combination is running right now&quot; without guessing or digging through Slack history.</p><h3>Docker Compose: Standing Up a Full ML Stack in One Command</h3><p>A real ML system is almost never one container. A typical setup might include your model-serving container, a vector database for retrieval, a Postgres instance for metadata, a Redis cache, and a monitoring sidecar — and running each one by hand with individual docker run commands gets unmanageable fast.</p><p><strong>Docker Compose</strong> lets you define the entire stack — every service, network, and volume — in a single YAML file, then bring the whole thing up with one command. Instead of remembering the right order to start five different containers so they can actually find each other, you describe how they connect once, and Compose handles the rest every time.</p><p>This is usually the point where Docker stops feeling like “a way to package one thing” and starts feeling like what it actually is: the foundation your entire ML system runs on.</p><h3>The Notebook Was Never the Problem</h3><p>Looking back, the frustrating part of that handoff was never the model itself. The model worked. The <em>environment</em> was the bug — invisible, undocumented, and living entirely inside one person’s laptop.</p><p>That’s what Docker actually fixes for ML teams. A well-built Dockerfile isn’t just a build script; it’s a precise, version-controlled statement of exactly what a model needs to run — pinned framework, pinned CUDA, weights, code, all of it — traveling with the model instead of living in someone’s memory. Once that’s true, “it worked in my notebook” stops being a warning sign and becomes irrelevant, because the notebook and production are finally running the exact same thing.</p><p>If you’re heading toward Kubernetes next to actually scale this — GPU scheduling, autoscaling inference under real traffic, distributed training across nodes — every bit of it is built directly on top of what’s in this post. Skip this part, and Kubernetes will always feel like magic you’re clicking through. Get this part right, and it starts feeling like something you actually built, one layer at a time.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=33d5f7e35b0d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/devops-ai-decoded/docker-for-ai-ml-the-crash-course-that-gets-your-model-out-of-the-notebook-and-into-production-33d5f7e35b0d">Docker for AI/ML: The Crash Course That Gets Your Model Out of the Notebook and Into Production</a> was originally published in <a href="https://medium.com/devops-ai-decoded">Devops &amp; AI Hub</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Only Kubernetes Architecture Guide You Need — Part 2: Running AI Workloads at Scale]]></title>
            <link>https://medium.com/devops-ai-decoded/the-only-kubernetes-architecture-guide-you-need-part-2-running-ai-workloads-at-scale-cbdbbceaf3de?source=rss-32ee9105f329------2</link>
            <guid isPermaLink="false">https://medium.com/p/cbdbbceaf3de</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[mlops]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Jay Sadhu]]></dc:creator>
            <pubDate>Sat, 15 Aug 2026 14:44:59 GMT</pubDate>
            <atom:updated>2026-08-15T14:46:00.267Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Z_YEJ6gTNfltmVJ-Xm0-2g.png" /></figure><p>Six months after that 2 AM outage, I got a different kind of page.</p><p>Not “the site is down.” This one read: <em>“The training job died overnight. We lost 14 hours of GPU time.”</em></p><p>I remember staring at that message feeling almost nostalgic — because at least this time, nothing was on fire. Customers weren’t affected. Revenue wasn’t bleeding. But somewhere in a data center, a rack of A100s had been idling all night because a pod got evicted and nobody set up the guardrails to stop it.</p><p>That’s when I learned the uncomfortable truth about Kubernetes and AI: <strong>the cluster that flawlessly runs your web servers has no idea what a GPU, a training job, or a model server even is</strong> — unless you teach it.</p><p>In Part 1, we built the skeleton: a working control plane, a joined worker node, pods talking across the network. That skeleton runs stateless web apps beautifully. But machine learning workloads are a different animal — they’re long-running, resource-hungry, stateful, and unforgiving of interruption. Kubernetes wasn’t originally built with any of that in mind.</p><p>This is the guide I wish I’d had before that overnight training job died — how to take a standard cluster and turn it into a real AI/ML platform: one that understands GPUs, survives long training runs, serves models at scale, and tells you exactly what’s happening when something goes wrong.</p><p>Let’s extend the cluster.</p><h3>1. Extending Kubernetes: Teaching It New Tricks</h3><p>Out of the box, Kubernetes understands Pods, Deployments, and Services. It has no concept of “a training job” or “a model.” If you want the cluster to manage things that don’t fit the built-in vocabulary, you have to teach it — and that’s exactly what <strong>Custom Resource Definitions (CRDs)</strong> are for.</p><p>A CRD lets you define a brand-new resource type — say, TrainingJob — that behaves like a first-class Kubernetes object. You can kubectl get trainingjobs the same way you&#39;d kubectl get pods.</p><p>But a CRD alone is just a data shape. It needs a brain. That’s where <strong>Operators</strong> come in — controllers that watch for these custom resources and take real action when they appear, change, or disappear. An Operator is essentially a piece of automated operational knowledge, encoded in software: it knows how to install something, heal it when it breaks, and scale it when needed.</p><p>This pattern — CRD defines <em>what</em>, Operator defines <em>how</em> — is the foundation on which everything else in this guide is built. Every tool below (Kubeflow, KServe, KEDA) is, under the hood, a set of CRDs and Operators that extend Kubernetes to support a new kind of workload.</p><h3>2. GPU Scheduling: Making the Cluster GPU-Aware</h3><p>By default, Kubernetes has no idea a GPU exists. It sees CPU and memory — nothing else. If you drop a workload requesting a GPU onto a vanilla cluster, it just… won’t schedule.</p><p>The fix is the <strong>NVIDIA GPU Operator</strong>, which installs everything the cluster needs to detect, expose, and manage GPUs as schedulable resources: drivers, the container toolkit, and a device plugin that reports GPU availability to the Kubernetes scheduler.</p><p>Once that’s running, your nodes with GPUs attached start advertising nvidia.com/gpu as a resource type — and you can request it in a pod spec exactly like you&#39;d request CPU or memory:</p><pre>resources:<br>  limits:<br>    nvidia.com/gpu: 1</pre><p>But availability isn’t the whole problem. GPU nodes are expensive, and you don’t want your ordinary web app pods accidentally landing on them — nor do you want a GPU-heavy job accidentally scheduled onto a plain CPU node where it’ll simply fail.</p><p>This is where <strong>taints and tolerations</strong> come in. You taint the GPU nodes so that nothing schedules there by default:</p><pre>kubectl taint nodes gpu-node-1 nvidia.com/gpu=true:NoSchedule</pre><p>Then only workloads that explicitly “tolerate” that taint — your training and inference pods — are allowed to land there. It’s a simple mechanism, but it’s the difference between a cluster that wastes your most expensive hardware and one that protects it.</p><h3>3. Training Workloads: Teaching the Cluster to Learn</h3><p>A single-GPU training job is easy. The real complexity starts when training needs to be distributed across multiple GPUs or multiple nodes — which is where most home-grown setups quietly fall apart.</p><p><strong>PyTorch Operators</strong> solve the orchestration problem: they understand the concept of a distributed training job made up of a master and multiple workers, and they handle the coordination, networking, and restart logic between them. Instead of manually wiring up distributed training scripts and hoping nothing crashes halfway through, you describe the job declaratively and let the Operator manage the lifecycle.</p><p>Two other pieces round out a real training stack:</p><ul><li><strong>MLflow</strong> for experiment tracking — logging metrics, parameters, and model artifacts so that six months from now, you can actually answer “which run produced this model, and with what hyperparameters?”</li><li><strong>Kubeflow</strong> for end-to-end workflow orchestration — chaining together data preprocessing, training, evaluation, and deployment into a single reproducible pipeline instead of a folder of scripts run in a fragile, manual order.</li></ul><p>For teams working with heavy distributed Python compute — not just training, but data processing and hyperparameter search — <strong>Ray</strong> has become the layer of choice. It abstracts away a lot of the low-level distributed-systems pain and lets you scale Python workloads across a cluster with far less boilerplate than wiring it by hand.</p><p>The lesson from my dead 14-hour training job: none of this matters if you don’t also handle <em>failure</em>. A distributed job that dies without checkpointing, restart logic, or alerting isn’t a training pipeline — it’s an expensive way to heat a data center.</p><h3>4. Inference Serving: Keeping Models Alive in Production</h3><p>Training gets the attention, but inference is where the money actually gets made — and it’s a completely different operational problem. A training job runs once and finishes. An inference service has to run <em>continuously</em>, handle unpredictable traffic, and never go down.</p><p>Modern LLM inference typically runs on a serving engine like <strong>vLLM</strong>, which is purpose-built for high-throughput, low-latency model serving — handling things like efficient memory management for large models and request batching that a naive serving setup would choke on.</p><p>But an engine alone isn’t a platform. That’s the role of <strong>KServe</strong>, which sits on top and handles the operational concerns: model loading and unloading, versioning multiple models simultaneously, and — critically — <strong>canary rollouts</strong>, so you can shift a small percentage of traffic to a new model version and watch it behave before fully committing. If you’ve ever pushed a new model straight to 100% of production traffic and immediately regretted it, this is the guardrail that prevents that.</p><h3>5. Autoscaling: Beyond CPU-Based Guesswork</h3><p>Kubernetes ships with the <strong>Horizontal Pod Autoscaler (HPA)</strong>, which scales pods up and down based on CPU or memory usage. That’s a reasonable default for a typical web app. It’s almost useless for AI workloads.</p><p>Here’s why: an inference server can be sitting at 20% CPU utilization while its request queue is 500 deep and every user is waiting on a slow response. CPU was never the bottleneck — the GPU and the queue were. HPA, watching only CPU, would see a healthy, underutilized service and do nothing.</p><p><strong>KEDA (Kubernetes Event-driven Autoscaling)</strong> fixes this by letting you scale on virtually any custom metric — inference queue length, requests per second, GPU utilization, messages in a queue, whatever actually reflects load for your workload. Instead of asking “is the CPU busy?”, you ask the question that actually matters: “is work piling up?”</p><p>For AI infrastructure specifically, this is often the single highest-leverage change you can make — the gap between HPA and KEDA is frequently the gap between a service that falls over under load and one that scales ahead of it.</p><h3>6. Observability: Knowing What’s Actually Happening</h3><p>None of the above matters if you’re flying blind. When a training job crashes at 3 AM or an inference service starts timing out, you need to already have the data — not go digging for it after the fact.</p><p>The standard, battle-tested stack is <strong>Prometheus</strong> for metrics collection and <strong>Grafana</strong> for visualization. That combination is well understood for typical infrastructure — but AI workloads need you to go further than the defaults.</p><p>Standard cluster dashboards will happily tell you every pod is “healthy” while a GPU sits at 95% memory and 12% actual compute utilization — quietly wasting the most expensive resource in your entire stack. A production-grade observability setup for AI infrastructure has to specifically track:</p><ul><li><strong>GPU utilization and memory</strong>, not just whether the pod is running</li><li><strong>Custom engine metrics</strong> from your serving layer — queue depth, tokens per second, time-to-first-token, batch sizes — the numbers that actually explain user-facing latency</li></ul><p>Get this right, and the difference shows up exactly where it matters: when something breaks, you already know why, instead of finding out from an angry Slack message.</p><h3>The Cluster That Actually Understands What It’s Running</h3><p>Look back at everything this piece covered, and a pattern emerges: none of it is really about Kubernetes learning to run GPUs, or training jobs, or model servers as isolated tricks. It’s about the same idea from Part 1, just at a bigger scale — <strong>infrastructure that’s designed to survive failure, not just perform when everything goes right.</strong></p><p>CRDs and Operators teach the cluster new vocabulary. Taints protect expensive hardware. MLflow and Kubeflow keep experiments reproducible instead of lost to memory. KServe protects you from your own bad rollouts. KEDA scales on what’s actually true instead of what’s convenient to measure. And observability makes sure that the next time something goes wrong at an ungodly hour, it’s a five-minute fix instead of a 14-hour mystery.</p><p>That dead training job taught me the AI-infrastructure version of the same lesson production taught me at 2 AM: the goal was never a cluster that works when you’re watching it. It keeps working when you’re not.</p><p>If this helped clarify the AI/ML side of Kubernetes for you, Part 3 will get hands-on — walking through the actual manifests to stand up GPU scheduling, KEDA, and a full observability stack from scratch. Let me know in the comments what you’d want covered first.</p><p><em>Connect with me on LinkedIn: </em><a href="https://www.linkedin.com/in/jysadhu"><em>linkedin.com/in/jysadhu</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cbdbbceaf3de" width="1" height="1" alt=""><hr><p><a href="https://medium.com/devops-ai-decoded/the-only-kubernetes-architecture-guide-you-need-part-2-running-ai-workloads-at-scale-cbdbbceaf3de">The Only Kubernetes Architecture Guide You Need — Part 2: Running AI Workloads at Scale</a> was originally published in <a href="https://medium.com/devops-ai-decoded">Devops &amp; AI Hub</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I Broke Production at 2 AM — Here’s the Kubernetes Setup That Made Sure It Never Happened Again]]></title>
            <link>https://medium.com/devops-ai-decoded/i-broke-production-at-2-am-heres-the-kubernetes-setup-that-made-sure-it-never-happened-again-cff97493935f?source=rss-32ee9105f329------2</link>
            <guid isPermaLink="false">https://medium.com/p/cff97493935f</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Jay Sadhu]]></dc:creator>
            <pubDate>Fri, 14 Aug 2026 09:31:01 GMT</pubDate>
            <atom:updated>2026-08-14T14:38:05.156Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Q32-yHiC1GBlhKYRmgcQfA.png" /></figure><p>It was 2:14 AM when my phone buzzed.</p><p><em>“Site is down. Customers can’t check out. Fix it NOW”</em></p><p>I sat up, laptop already open, hands shaking slightly from adrenaline and too much cold coffee. Our app was running on a single EC2 instance — no redundancy, no failover, no orchestration. One process crashed, and the entire business stopped.</p><p>That night cost the company real money. It cost me a full night’s sleep and a very uncomfortable stand-up the next morning. But it also taught me the most valuable lesson of my DevOps career:</p><blockquote><strong><em>If your infrastructure depends on one server behaving perfectly forever, it’s not infrastructure. It’s a countdown timer.</em></strong></blockquote><p>That’s the night I stopped treating Kubernetes as “something the big companies use” and started treating it as the safety net every serious application needs.</p><p>This is the exact setup I now deploy for every production workload — Kubernetes running on EC2, built by hand with kubeadm, so you actually understand what&#39;s happening under the hood instead of just clicking &quot;Create Cluster&quot; and hoping for the best.</p><p>By the end of this guide, you won’t just have a working cluster. You’ll understand <em>why</em> every single piece exists — because one day, at 2 AM, that understanding is the only thing standing between you and a very bad night.</p><p>Let’s build it.</p><h3>Why Kubernetes, and Why Build It Yourself?</h3><p>Managed services like EKS are great — once you know what they’re doing for you. Most engineers skip straight to managed Kubernetes and never learn what’s actually happening underneath: how nodes join a cluster, how pods find each other, how the control plane makes decisions.</p><p>Building your own cluster with kubeadm on plain EC2 instances strips away the magic. You&#39;ll set up networking by hand. You&#39;ll watch nodes join in real time. You&#39;ll break things — and fix them — in a sandbox instead of production.</p><p>This is the same foundational skill that separates engineers who <em>use</em> Kubernetes from engineers who <em>understand</em> it.</p><h3>What You’ll Need</h3><ul><li>Two EC2 instances — one master, one worker</li><li>Ubuntu 22.04 (or similar)</li><li>Basic comfort with the Linux command line and AWS EC2</li></ul><p>That’s it. No prior Kubernetes experience required — just patience and a willingness to type a lot of commands.</p><h3>Step 0: Open the Right Doors (Security Groups)</h3><p>Before any of this works, your two servers need to actually be able to talk to each other. Miss this step, and you’ll spend an hour debugging a cluster that was never broken — just blocked.</p><p><strong>Master Node Security Group</strong></p><ul><li><strong>6443</strong> (TCP) — Kubernetes API server</li><li><strong>2379–2380</strong> (TCP) — etcd client API</li><li><strong>10250</strong> (TCP) — Kubelet API</li><li><strong>10251</strong> (TCP) — kube-scheduler</li><li><strong>10252</strong> (TCP) — kube-controller-manager</li><li><strong>22</strong> (TCP) — SSH access</li></ul><p><strong>Worker Node Security Group</strong></p><ul><li><strong>10250</strong> (TCP) — Kubelet API</li><li><strong>30000–32767</strong> (TCP) — NodePort services</li><li><strong>22</strong> (TCP) — SSH access</li></ul><p>Open these both ways between the two nodes. This is the unglamorous plumbing work — but skip it, and nothing downstream will save you.</p><h3>Step 1: Give Your Nodes an Identity</h3><p>Right now, your two EC2 instances are anonymous. Let’s name them.</p><p>On the master node:</p><pre>sudo hostnamectl set-hostname &quot;master&quot;</pre><p>On the worker node:</p><pre>sudo hostnamectl set-hostname &quot;worker&quot;</pre><p>Then, on <strong>both</strong> nodes, update /etc/hosts so they can find each other by name instead of memorizing IPs:</p><pre>&lt;Master Node IP&gt;  master<br>&lt;Worker Node IP&gt;  worker</pre><p>Small step. But every cluster starts with two machines learning each other’s names.</p><h3>Step 2: Turn Off Swap (on all nodes)</h3><p>Kubernetes and swap memory do not get along — the scheduler makes promises about resources that swap can quietly break. Disable it everywhere:</p><pre>sudo apt update<br>sudo swapoff -a<br>sudo sed -i &#39;/ swap / s/^\(.*\)$/#\1/g&#39; /etc/fstab</pre><h3>Step 3: Prep the Kernel (on all nodes)</h3><p>Kubernetes needs specific kernel modules and network settings to route traffic between pods correctly:</p><pre># Load the required kernel modules<br>sudo tee /etc/modules-load.d/containerd.conf &lt;&lt;EOF<br>overlay<br>br_netfilter<br>EOF</pre><pre>sudo modprobe overlay<br>sudo modprobe br_netfilter</pre><pre># Set sysctl parameters<br>sudo tee /etc/sysctl.d/kubernetes.conf &lt;&lt;EOF<br>net.bridge.bridge-nf-call-ip6tables = 1<br>net.bridge.bridge-nf-call-iptables = 1<br>net.ipv4.ip_forward = 1<br>EOF</pre><pre>sudo sysctl --system</pre><h3>Step 4: Install the Container Runtime (on all nodes)</h3><p>Kubernetes doesn’t run containers itself — it delegates that job to a runtime. We’ll use containerd, the industry standard.</p><pre>sudo apt install -y curl gnupg2 software-properties-common apt-transport-https ca-certificates</pre><pre># Add Docker&#39;s repository (containerd is distributed here)<br>sudo curl -fsSL <a href="https://download.docker.com/linux/ubuntu/gpg">https://download.docker.com/linux/ubuntu/gpg</a> | sudo gpg --dearmour -o /etc/apt/trusted.gpg.d/docker.gpg<br>sudo add-apt-repository &quot;deb [arch=amd64] <a href="https://download.docker.com/linux/ubuntu">https://download.docker.com/linux/ubuntu</a> $(lsb_release -cs) stable&quot;</pre><pre>sudo apt update<br>sudo apt install -y containerd.io</pre><p>Now configure it to use systemd as the cgroup driver — this one setting trips up more first-time cluster builds than anything else in this guide:</p><pre>containerd config default | sudo tee /etc/containerd/config.toml &gt;/dev/null 2&gt;&amp;1<br>sudo sed -i &#39;s/SystemdCgroup = false/SystemdCgroup = true/g&#39; /etc/containerd/config.toml</pre><pre>sudo systemctl restart containerd<br>sudo systemctl enable containerd</pre><h3>Step 5: Add the Kubernetes Repository (on all nodes)</h3><pre>echo &quot;deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.30/deb/ /&quot; | sudo tee /etc/apt/sources.list.d/kubernetes.list<br>curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.30/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg</pre><h3>Step 6: Install the Kubernetes Tools (on all nodes)</h3><pre>sudo apt update<br>sudo apt install -y kubelet kubeadm kubectl<br>sudo apt-mark hold kubelet kubeadm kubectl</pre><p>apt-mark hold matters here — it stops these packages from silently auto-updating and breaking your cluster version compatibility later.</p><h3>Step 7: Bring the Cluster to Life (on the master node)</h3><p>This is the moment it all comes together.</p><pre>sudo kubeadm init --control-plane-endpoint=master</pre><p>Watch the output closely — it prints your kubeadm join command, which you&#39;ll need in the next step. Copy it somewhere safe.</p><p>Now let kubectl talk to your new cluster:</p><pre>mkdir -p $HOME/.kube<br>sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config<br>sudo chown $(id -u):$(id -g) $HOME/.kube/config</pre><p>Check that your master node is alive:</p><pre>kubectl get nodes</pre><p>If you see a node listed — even if it says NotReady for now — you just built a Kubernetes control plane from scratch. Take a second. That&#39;s a real milestone.</p><h3>Step 8: Bring the Worker Into the Family</h3><p>On the worker node, paste the kubeadm join command you copied earlier:</p><pre>kubeadm join &lt;Master Node IP&gt;:6443 --token &lt;token&gt; \<br>       --discovery-token-ca-cert-hash sha256:&lt;hash&gt;</pre><p>This single command is doing a lot of quiet, important work — authenticating the worker, registering it with the control plane, and preparing it to receive workloads.</p><h3>Step 9: Give Your Pods a Way to Talk (on the master node)</h3><p>Right now, your nodes are connected, but your pods have no network to communicate across nodes. Install Calico to fix that:</p><pre>kubectl apply -f https://raw.githubusercontent.com/projectcalico/calico/v3.25.0/manifests/calico.yaml</pre><p>This is the moment your cluster stops being two isolated machines and becomes one coordinated system.</p><h3>Step 10: Confirm Everything Is Alive</h3><pre>kubectl get pods -n kube-system<br>kubectl get nodes</pre><p>Both nodes should now show Ready. If they don&#39;t yet, give it a minute — Calico takes a moment to finish wiring the network.</p><h3>Step 11: Prove It Works</h3><p>Theory is worthless until something actually runs. Deploy a real application:</p><pre># Create a namespace<br>kubectl create ns demo-app</pre><pre># Deploy nginx<br>kubectl create deployment nginx-app --image nginx --replicas 1 --namespace demo-app</pre><pre># Check status<br>kubectl get deployment -n demo-app<br>kubectl get pods -n demo-app</pre><pre># Expose it<br>kubectl expose deployment nginx-app -n demo-app --type NodePort --port 80</pre><pre># Confirm the service<br>kubectl get svc -n demo-app</pre><p>Grab the NodePort and hit &lt;Worker Node IP&gt;:&lt;NodePort&gt; in your browser. When that nginx welcome page loads, that&#39;s not just a test page — that&#39;s proof your cluster can take real traffic.</p><h3>Full Circle</h3><p>I think back to that 2 AM page a lot. Not because it was traumatic — plenty of engineers have a story like it — but because of what it taught me: resilience isn’t something you bolt on after things break. It’s something you design in from the start.</p><p>The cluster you just built is small — two nodes, one test app. But the architecture is the same one running behind Spotify, Airbnb, and half the internet’s checkout pages. You now understand the machinery that most engineers only click a button to deploy.</p><p>Next time production has a bad night, you won’t be guessing. You’ll know exactly where to look — because you built it with your own hands, one sudo command at a time.</p><p>If this helped you, save it for the next time you’re staring down a cluster at 2 AM. And if you’ve got your own “the-night-everything-broke” story, I’d genuinely love to hear it.</p><p><em>Connect with me on LinkedIn: </em><a href="https://www.linkedin.com/in/jysadhu"><em>linkedin.com/in/jysadhu</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cff97493935f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/devops-ai-decoded/i-broke-production-at-2-am-heres-the-kubernetes-setup-that-made-sure-it-never-happened-again-cff97493935f">I Broke Production at 2 AM — Here’s the Kubernetes Setup That Made Sure It Never Happened Again</a> was originally published in <a href="https://medium.com/devops-ai-decoded">Devops &amp; AI Hub</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Only Kubernetes Architecture Guide You Need — Part 1: Architecture & Core Objects]]></title>
            <link>https://jysadhu.medium.com/the-only-kubernetes-architecture-guide-you-need-part-1-architecture-core-objects-164490d0eca6?source=rss-32ee9105f329------2</link>
            <guid isPermaLink="false">https://medium.com/p/164490d0eca6</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[mlops]]></category>
            <category><![CDATA[architecture]]></category>
            <dc:creator><![CDATA[Jay Sadhu]]></dc:creator>
            <pubDate>Thu, 13 Aug 2026 17:28:41 GMT</pubDate>
            <atom:updated>2026-08-13T17:28:42.891Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9pu2AVNAer6ZO2PCVlcsQQ.png" /></figure><p>Transitioning from Docker to Kubernetes often feels like jumping off a quiet footbridge straight into a hurricane of YAML files, custom resources, and abstract networking terminology. When running a local container with docker run or docker-compose up, the scope of operational concerns is tight. You start a process, map a port, mount a folder, and move on.</p><p>Kubernetes changes the paradigm entirely. Instead of managing individual containers, you are managing desired states across an entire distributed system. The sheer volume of objects — Pods, Deployments, Services, Ingresses, StatefulSets, ConfigMaps, Taints — can obscure the underlying simplicity of the architecture. Without a clear mental model, cluster management degenerates into pasting YAML snippets from Stack Overflow until something stops crash-looping.</p><p>This guide provides that mental model. It’s <strong>Part 1</strong> of a technical reference series designed to strip away the jargon and break Kubernetes down into its structural architecture and six fundamental object buckets. <strong>Part 2</strong> will build on this foundation to cover AI/ML workloads, custom operators, GPU scheduling, and distributed training setups.</p><h3>Why Kubernetes Exists: The Evolutionary Path</h3><p>To understand what Kubernetes does, it helps to trace the operational pressures that forced its creation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0J6bA9dSuYwbYn-cNoEtFg.png" /></figure><ul><li><strong>Monoliths</strong>: Applications started as single, massive codebases running directly on bare-metal servers or monolithic virtual machines. Scaling required scaling the entire application, wasting compute resources and creating single points of failure.</li><li><strong>Microservices</strong>: Teams decomposed monoliths into smaller, loosely coupled services. This improved developer velocity but introduced massive operational complexity — hundreds of distinct processes needing deployment, configuration, and monitoring.</li><li><strong>Containers (Docker)</strong>: Packaging applications with their dependencies solved the “works on my machine” problem. But Docker operates primarily at the single-host level. If a host node crashed, the containers running on it died with it unless manually restarted elsewhere.</li><li><strong>Orchestration (Kubernetes)</strong>: As container volume grew into the hundreds or thousands across heterogeneous cloud environments, manual intervention became impossible. The industry needed an automated control system capable of bin-packing containers onto hosts, monitoring health, repairing failed instances, balancing traffic, and reconciling declared state automatically.</li></ul><p>Kubernetes was built to solve exactly these orchestration challenges at scale.</p><h3>Cluster Architecture: Control Plane vs. Data Plane</h3><p>A Kubernetes cluster splits cleanly into two functional areas: the <strong>Control Plane</strong> (which makes global decisions about the cluster) and the <strong>Data Plane</strong> (where your workloads actually run).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HIohpQG8vKVk8_ZqW1JJ9Q.png" /></figure><h3>The Control Plane (The Brain)</h3><p>The Control Plane maintains the declarative state of the cluster, processes incoming commands, and continuously works to align actual state with desired state.</p><ul><li><strong>API Server (</strong><strong>kube-apiserver)</strong>: The central communication hub. It exposes the HTTP/JSON REST API for the cluster. Every component — from the kubectl CLI to internal controllers — talks exclusively through the API Server.</li><li><strong>etcd</strong>: A strongly consistent, distributed key-value store. It acts as the cluster’s single source of truth, persisting all cluster state, configurations, and secrets.</li><li><strong>Kube-Scheduler</strong>: The placement engine. It watches for newly created Pods that have no assigned node and selects the optimal worker node based on resource requests, constraints, and affinity rules.</li><li><strong>Kube-Controller-Manager</strong>: The continuous reconciliation engine. It runs background controller loops (Node Controller, Deployment Controller, EndpointSlice Controller, etc.) that detect state drift and trigger corrections.</li></ul><h3>The Data Plane (The Muscle)</h3><p>The Data Plane consists of worker nodes responsible for executing application workloads.</p><ul><li><strong>kubelet</strong>: The primary agent running on every worker node. It receives Pod specifications from the API Server, talks to the local container runtime to start/stop containers, and reports node health back to the Control Plane.</li><li><strong>Container Runtime</strong>: The software engine that actually runs containers. Modern Kubernetes uses runtimes implementing the Container Runtime Interface (CRI), such as <strong>containerd</strong> or <strong>CRI-O</strong>.</li><li><strong>CNI Plugins (Container Network Interface)</strong>: Software modules (Calico, Cilium, Flannel) that assign IP addresses to Pods and establish overlay or routable network paths across nodes.</li></ul><h3>The Six Core Object Buckets</h3><p>Rather than memorizing dozens of disconnected Kubernetes primitives, group them into six functional categories:</p><p>1. Workloads 2. Scheduling 3. Networking 4. Storage Pod Requests/Limits Service PV Deployment Node Affinity Ingress PVC StatefulSet Taints/Tolerations Gateway API StorageClass DaemonSet Topology Spread CSI Driver Job / CronJob 5. Config &amp; Isolation 6. Security &amp; Access ConfigMap / Secret RBAC (Role, RoleBinding, ServiceAccount) Namespace / ResourceQuota Network Policy LimitRange Pod Security Standards (PSS/PSA)</p><h3>1. Workloads</h3><p><strong>Primary function:</strong> Manages process lifecycles, scalability, and execution rules for containers.</p><ul><li><strong>Pod</strong>: The smallest deployable unit in Kubernetes. A wrapper around one or more tightly coupled containers sharing network and storage namespaces.</li><li><strong>Deployment</strong>: Manages stateless applications. Handles declarative updates, scaling, automated rollouts, and rollbacks.</li><li><strong>StatefulSet</strong>: Manages stateful applications requiring unique network identities, stable persistent storage bindings, and ordered deployment/scaling (e.g., databases).</li><li><strong>DaemonSet</strong>: Ensures a copy of a specific Pod runs on all (or selected) worker nodes. Used for infrastructure agents like log collectors and monitoring daemons.</li><li><strong>Job &amp; CronJob</strong>: Manages batch execution. A Job runs tasks to completion and terminates; a CronJob schedules Jobs periodically using cron syntax.</li></ul><p><strong>Why it matters in production:</strong> Production applications are rarely deployed as bare Pods. Always encapsulate Pod definitions inside higher-level abstractions like Deployments or StatefulSets to enable self-healing and automated lifecycle management.</p><h3>2. Scheduling</h3><p><strong>Primary function:</strong> Determines placement of workloads onto specific nodes based on capacity, topology, and isolation requirements.</p><ul><li><strong>Resource Requests &amp; Limits</strong>: Requests define the guaranteed compute resources (CPU/memory) allocated for scheduling; Limits enforce the maximum a container can consume before being throttled (CPU) or OOM-killed (memory).</li><li><strong>Node Affinity &amp; Anti-Affinity</strong>: Hard or soft rules that instruct the scheduler to place Pods on nodes matching specific key-value labels.</li><li><strong>Taints &amp; Tolerations</strong>: Taints let a node repel a set of Pods; Tolerations applied to Pods allow (but don’t force) them to schedule onto tainted nodes (e.g., reserving GPU nodes for specialized jobs).</li><li><strong>Topology Spread Constraints</strong>: Rules that distribute Pods evenly across failure domains (availability zones, regions, host racks) to prevent a single-zone outage from taking down a service.</li></ul><p><strong>Why it matters in production:</strong> Improper resource definitions destabilize clusters. Omitting requests leads to overcommitted nodes; unrealistic limits trigger unexpected OOM kills.</p><h3>3. Networking</h3><p><strong>Primary function:</strong> Enables service discovery, intra-cluster communication, and ingress/egress traffic management.</p><ul><li><strong>Service</strong>: An abstract boundary defining a logical set of Pods and a policy to access them via a stable virtual IP.</li><li><strong>ClusterIP</strong> (default): Exposes the service internally only.</li><li><strong>NodePort</strong>: Exposes the service on each node’s IP at a static port.</li><li><strong>LoadBalancer</strong>: Provisions an external cloud load balancer (AWS ALB, GCP NLB) pointing to cluster nodes.</li><li><strong>Ingress</strong>: An API object managing external HTTP/HTTPS access to services — layer 7 routing, SSL termination, host-based virtual hosting.</li><li><strong>Gateway API</strong>: The modern replacement for Ingress. An extensible, role-oriented routing interface capable of advanced L4/L7 traffic management (header matching, weighted splits, cross-namespace routing).</li></ul><p><strong>Why it matters in production:</strong> Traditional Ingress controllers struggle with multi-tenant edge routing and advanced traffic management. The industry is rapidly adopting Gateway API for enterprise infrastructure because it separates operational roles across Infrastructure Provider, Cluster Admin, and Application Developer.</p><h3>4. Storage</h3><p><strong>Primary function:</strong> Decouples persistent data management from the transient lifecycle of containers.</p><ul><li><strong>PersistentVolume (PV)</strong>: A piece of cluster storage provisioned statically by an admin or dynamically via a StorageClass. Exists independently of any Pod lifecycle.</li><li><strong>PersistentVolumeClaim (PVC)</strong>: A request for storage by a workload — specifies size, access mode (e.g., ReadWriteOnce, ReadWriteMany), and storage class.</li><li><strong>StorageClass</strong>: Defines the “class” of storage (fast SSD, cheap object storage) and dynamically provisions PVs when a PVC requests them.</li><li><strong>CSI Drivers (Container Storage Interface)</strong>: Standardized interfaces letting cloud providers and storage vendors attach block/file storage directly to Kubernetes clusters.</li></ul><p><strong>Why it matters in production:</strong> Make sure your storage drivers support access modes suited to your architecture. Attaching a single-attach disk (ReadWriteOnce) to a multi-node deployment leads to stuck Pod attachments during node failovers.</p><h3>5. Config &amp; Isolation</h3><p><strong>Primary function:</strong> Separates application code from configuration, enforces multi-tenancy boundaries, and restricts resource consumption.</p><ul><li><strong>ConfigMap</strong>: Stores non-sensitive configuration data as key-value pairs, injectable as environment variables, CLI args, or mounted files.</li><li><strong>Secret</strong>: Stores sensitive data (passwords, tokens, keys) base64-encoded, injectable similarly to ConfigMaps.</li><li><strong>Namespace</strong>: Virtual clusters within a single physical cluster, providing scoping boundaries and logical isolation.</li><li><strong>ResourceQuota &amp; LimitRange</strong>: ResourceQuota restricts total aggregate resource consumption (CPU, memory, Pod count) within a Namespace; LimitRange enforces min/max resource allocation per Pod within that Namespace.</li></ul><p><strong>Why it matters in production:</strong> Base64 encoding in Secrets is not encryption. Configure KMS envelope encryption for etcd or integrate external secrets operators (HashiCorp Vault, AWS Secrets Manager) to secure credentials properly.</p><h3>6. Security &amp; Access</h3><p><strong>Primary function:</strong> Establishes authentication, authorization, container runtime boundaries, and network segmentation.</p><ul><li><strong>RBAC (Role-Based Access Control)</strong>: Regulates API access via Roles/ClusterRoles (permissions) bound to ServiceAccounts, Users, or Groups through RoleBindings/ClusterRoleBindings.</li><li><strong>Network Policies</strong>: Declarative firewalls at layer 3/4 dictating how groups of Pods communicate with each other and external endpoints.</li><li><strong>Pod Security Standards (PSS) &amp; Pod Security Admission (PSA)</strong>: Built-in controls defining security profiles (Privileged, Baseline, Restricted) that enforce container security settings, such as preventing root execution or dropping capabilities.</li></ul><p><strong>Why it matters in production:</strong> Default Kubernetes networking allows all Pods to talk to all other Pods across namespaces. Implement default-deny Network Policies early to enforce zero-trust network boundaries.</p><h3>Deployment Strategies for Zero-Downtime Releases</h3><p>Updating applications without dropping active connections requires choosing the right strategy for your reliability requirements.</p><p>Strategy Mechanism Pros Cons Ideal Use Case <strong>Rolling Update</strong> (default) Replaces old Pods with new ones incrementally No downtime, no extra infrastructure cost Old and new versions temporarily coexist (schema risk) Standard stateless microservices <strong>Blue-Green</strong> Provisions a full duplicate environment (Green), tests it, then switches traffic via a Service update Instant rollback, isolated pre-release testing Requires 2x compute during updates Monoliths, rigid APIs, stateful updates <strong>Canary</strong> Routes a small fraction of real traffic (e.g., 5%) to the new version before full rollout Minimal blast radius for production bugs Requires advanced networking (Service Mesh or Gateway API) Mission-critical apps, high-volume APIs</p><h3>Key Takeaways: The Mental Checklist</h3><p>Use this checklist when building or reviewing cluster manifests:</p><ul><li><strong>Control Plane isolation</strong> — keep the API server and etcd decoupled from data workloads and spread across distinct failure zones.</li><li><strong>Decouple configuration</strong> — store environment parameters in ConfigMaps and sensitive credentials in external secret stores. Never bake configs into container images.</li><li><strong>Enforce resource guardrails</strong> — specify explicit CPU/memory requests and limits on every container, paired with Namespace-level ResourceQuotas.</li><li><strong>Zero-trust network defaults</strong> — restrict cluster networking with Network Policies, blocking cross-namespace traffic by default.</li><li><strong>Select the right workload type</strong> — Deployments for stateless tasks, StatefulSets for persistent data, DaemonSets for infrastructure agents.</li></ul><h3>Looking Ahead: Part 2</h3><p>Mastering Kubernetes architecture and object primitives is the prerequisite for managing modern distributed computing environments. But standard deployments run into unique challenges when handling AI and machine learning infrastructure.</p><p>In <strong>Part 2</strong>, we’ll shift focus to running AI workloads on Kubernetes, covering:</p><ul><li>GPU resource allocation, slicing, and device plugins</li><li>Custom Resource Definitions (CRDs) and the Operator pattern</li><li>Distributed training architectures (Ray, Kubeflow, PyTorch Operators)</li><li>Custom schedulers built for high-performance batch computing</li></ul><p>Follow along to catch Part 2 as soon as it drops!</p><p><em>Connect with me on LinkedIn: </em><a href="https://www.linkedin.com/in/jysadhu"><em>linkedin.com/in/jysadhu</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=164490d0eca6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I Switched My VS Code Copilot to Kimi K3 for a Week — Here’s What 4x Cheaper AI Actually Gets You]]></title>
            <link>https://jysadhu.medium.com/i-switched-my-vs-code-copilot-to-kimi-k3-for-a-week-heres-what-4x-cheaper-ai-actually-gets-you-4cdfc73af2e8?source=rss-32ee9105f329------2</link>
            <guid isPermaLink="false">https://medium.com/p/4cdfc73af2e8</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[openai]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Jay Sadhu]]></dc:creator>
            <pubDate>Fri, 07 Aug 2026 07:06:47 GMT</pubDate>
            <atom:updated>2026-08-07T07:06:47.166Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PxJJEvlBqkXSj1lPehwd4g.png" /></figure><p>My OpenRouter bill last month made me wince. Not because I was doing anything exotic — just the usual daily grind of coding, debugging, and asking my AI assistant to explain why my Docker build kept failing at 2 AM. Frontier model API costs add up fast when you’re running them constantly inside your editor.</p><p>So when I kept seeing “Kimi K3” mentioned in dev forums, I figured it was worth an actual test instead of another bookmark I’d never revisit. Moonshot AI’s new open-weight model has been making the rounds, and the pitch is simple: performance close to the big closed models, at a fraction of the cost. I wanted to know if that was real or just hype.</p><p>Here’s what I found after actually wiring it into my workflow.</p><h3>What Is Kimi K3, Actually?</h3><p>Kimi K3 is a 2.8 trillion parameter open-weight model from Moonshot AI. That number alone is worth pausing on — it’s enormous, even by current frontier standards.</p><p>But the 2.8T figure is a bit misleading on its own. Kimi K3 is a Mixture-of-Experts (MoE) model, meaning it doesn’t activate all 2.8 trillion parameters for every token. It routes each token through 16 of 896 available experts, which works out to roughly 104B active parameters at inference time. That’s the actual compute you’re paying for per request — the full 2.8T is more like the model’s total “knowledge capacity” sitting in reserve. It’s also released with a 1M token context window and ships in MXFP4 weight quantization (MXFP8 for activations), which is part of why it’s cheap to serve at scale.</p><p>Let me be upfront about something: Kimi K3 doesn’t strictly beat the top closed models on every benchmark. What it does is stay competitive across the board. In day-to-day coding and reasoning tasks, it holds its own well enough that the gap, if there is one, isn’t something you’ll notice in normal use.</p><p>That’s not a knock on it. “Competitive with frontier models” from an open-weight release is genuinely impressive, and it’s the right way to frame this rather than pretending it’s some undisputed champion.</p><h3>Why “Open-Weight” Actually Matters Here</h3><p>The open-weight label gets thrown around a lot, so here’s the practical version of why it matters.</p><p><strong>Transparency.</strong> With a closed model, you’re trusting a black box. With open weights, there’s more visibility into how the model was built and trained, which matters if you care about understanding what you’re actually building on top of.</p><p><strong>Fine-tuning flexibility.</strong> This is the bigger deal for most developers. Closed models generally don’t let you meaningfully fine-tune them for your specific domain or use case. With Kimi K3’s weights available, that door is open — you can shape the model around your actual problem instead of prompting your way around its limitations.</p><p>Whether you’ll personally use that fine-tuning capability is another question. But having the option is a real structural advantage over closed alternatives.</p><h3>The Hardware Reality Check (Don’t Get Your Hopes Up)</h3><p>Here’s where I need to manage expectations, because “open-weight” tricked me into thinking I could run this locally. I couldn’t. Neither can you, probably.</p><p>A 2.8 trillion parameter model needs serious virtual memory to run — we’re talking up to 2TB. That’s not a beefy gaming rig or even a maxed-out workstation. That’s data-center territory.</p><p>So despite being “open,” Kimi K3 isn’t self-hostable for the vast majority of developers. The practical path is accessing it through a cloud provider, the same way you’d access any closed model. Open-weight here means open access to the model’s internals and fine-tuning potential — not open access to running it on your laptop.</p><h3>The Real Reason Everyone’s Talking About It: Cost</h3><p>This is the part that actually matters for day-to-day use.</p><p>Through providers like OpenRouter, Kimi K3 runs roughly <strong>4x cheaper</strong> than comparable frontier models. Not a small discount — a genuinely different cost bracket.</p><p>For solo developers, indie hackers, and startups watching every dollar of API spend, that price-to-performance ratio is the actual headline here. You’re not sacrificing much capability, but you’re cutting your inference costs dramatically. If you’re running an AI assistant constantly inside your editor all day, every day, that difference compounds fast.</p><p>This — more than the parameter count, more than the benchmarks — is why developers are actually switching.</p><h3>How to Set It Up: Kimi K3 in VS Code</h3><p>Here’s the exact walkthrough I used to get Kimi K3 running inside my editor. One honest caveat first: stock GitHub Copilot doesn’t natively let you point it at an arbitrary third-party base URL — that kind of custom-provider flexibility lives in open-source Copilot alternatives like <strong>Continue.dev</strong>, <strong>Cline</strong>, or <strong>Aider</strong>. If you’re set on staying inside the Copilot UI specifically, check its latest docs for BYOK (bring-your-own-key) support, since this changes often. For everyone else, any of the extensions below get you there in a few minutes.</p><ol><li><strong>Create an OpenRouter account.</strong> Head to OpenRouter’s site and sign up — this is the API provider that gives you access to Kimi K3 without needing your own GPU cluster.</li><li><strong>Add credits and generate an API key.</strong> Load a small amount of credit onto your account (given the pricing, a little goes a long way), then generate an API key from your dashboard. Keep this key somewhere safe.</li><li><strong>Install a VS Code extension that supports custom OpenAI-compatible providers.</strong> Continue.dev and Cline are the two most popular free options for this.</li><li><strong>Add a custom model config</strong> pointing to OpenRouter’s endpoint, paste in your API key, and set the model string to Kimi K3’s OpenRouter identifier.</li><li><strong>Save and switch your active model</strong> to Kimi K3 in the extension’s model picker.</li><li><strong>Start coding.</strong> Use it for inline completions, chat-based debugging, refactoring, long-context log analysis across multiple files, and talking through deployment steps.</li></ol><p>That’s it. No infrastructure to manage, no local weights to download. You’re just routing your editor’s AI assistant through a cheaper backend.</p><h3>Key Takeaways: Should You Actually Switch?</h3><p>Consider making the switch to Kimi K3 if:</p><ul><li>You’re burning through API credits fast because you use AI assistance constantly throughout the workday</li><li>You’re a solo developer, indie hacker, or early-stage startup where every dollar of tooling cost matters</li><li>You regularly work with large codebases or log files and would benefit from the 1M token context window</li><li>You want the <em>option</em> to fine-tune a model for a specific domain down the line, even if you’re not doing it yet</li><li>You care about transparency into how your AI tooling was built, not just its output</li><li>Your use case doesn’t require squeezing out the absolute last percentage point of benchmark performance</li></ul><p>Maybe stick with a closed frontier model if:</p><ul><li>You need the model to win, not just compete, on the specific benchmarks that matter to your use case</li><li>Your usage volume is low enough that a 4x price difference doesn’t move the needle</li><li>Your workflow depends on proprietary features tied to a specific vendor’s ecosystem</li><li>You have zero interest in self-hosting, fine-tuning, or installing a third-party editor extension</li></ul><h3>Final Thoughts</h3><p>Kimi K3 isn’t going to dethrone the frontier models on raw capability, and Moonshot AI doesn’t need it to. What it offers is a genuinely different value proposition: competitive performance at a quarter of the cost, with the structural benefits of open weights baked in.</p><p>For a lot of developers watching their API bills climb, that trade-off is going to be worth it.</p><p>Have you tried Kimi K3 yet, or switched your Copilot setup to a custom provider? I’d love to hear how it’s holding up for your workflow — drop a comment with what you’re building and whether the cost savings held up in practice.</p><p>📌 Let’s connect on LinkedIn too — <a href="https://www.linkedin.com/in/jysadhu">linkedin.com/in/jysadhu</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4cdfc73af2e8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How I’d Design AWS Systems: A Field Engineer’s Playbook]]></title>
            <link>https://jysadhu.medium.com/how-id-design-aws-systems-a-field-engineer-s-playbook-42f7976f28d7?source=rss-32ee9105f329------2</link>
            <guid isPermaLink="false">https://medium.com/p/42f7976f28d7</guid>
            <category><![CDATA[interview]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[amazon-web-services]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <dc:creator><![CDATA[Jay Sadhu]]></dc:creator>
            <pubDate>Fri, 07 Aug 2026 06:28:56 GMT</pubDate>
            <atom:updated>2026-08-07T06:28:56.074Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Z1n-N_EzCmATRn-az3XOwA.png" /></figure><h3>Three real-world scenarios, one mental model, and zero Click-Ops. This is the AWS design framework I wish I had on day one — built from scars, not theory.</h3><h3>You’ve Been There. The Page Goes Off at 2 AM.</h3><p>A traffic spike your architecture wasn’t built for. A Black Friday crash costing $50K an hour. A manual process burning through budget that AI could handle in seconds. These aren’t hypotheticals — they’re Tuesday for most DevOps engineers.</p><p>The problem isn’t that AWS lacks tools. It’s that most of us design systems reactively — picking services based on what we know, not what the problem demands. The result: fragile architectures that hold until they don’t.</p><p>This post introduces the <strong>Six-Layer Decision Stack</strong> — a mental model for making every AWS design choice with intention. Then we’ll apply it to three real-world scenarios: a viral startup, an e-commerce retailer, and an AI-powered enterprise. No fluff, no vendor marketing — just field-tested patterns.</p><h3>The Six-Layer Decision Stack</h3><p>Every AWS architecture decision maps to one of six layers. Work through them in order — each layer informs the next. Bookmark this.</p><ol><li><strong>Traffic</strong> — Managing user access and data flow. DNS, CDN, load balancing, WAF — how requests enter your system.</li><li><strong>Compute</strong> — Where processing happens. EC2, Lambda, Fargate, ECS — match the workload, not your comfort zone.</li><li><strong>Data</strong> — Storage and movement patterns. S3, RDS, DynamoDB, SQS — right tool, right access pattern.</li><li><strong>Security</strong> — Access control at every layer. IAM, VPC, WAF, encryption — security is a property of the design, not a bolt-on.</li><li><strong>Cost &amp; Budget</strong> — Performance vs. financial efficiency. On-Demand vs. Reserved, serverless vs. provisioned — every choice has a price tag.</li><li><strong>Observability</strong> — Proactive system health tracking. CloudWatch, X-Ray, logs — if you can’t see it, you can’t fix it.</li></ol><blockquote><strong><em>Pro tip:</em></strong><em> Don’t skip layers. I’ve seen teams pick a database before thinking about traffic patterns. It always ends in a migration.</em></blockquote><h3>Scenario 1: The Viral Startup Problem</h3><p><strong>The problem:</strong> A single EC2 instance can’t handle a 100,000-user traffic spike. Budget is effectively zero. You need to scale without knowing if this traffic is permanent or a flash in the pan.</p><p><strong>The stack:</strong></p><ul><li><strong>Route 53</strong> — DNS routing with latency-based policies. First touchpoint for all traffic.</li><li><strong>CloudFront + S3</strong> — Global CDN caching static assets at the edge. S3 as origin — cheap, durable, infinitely scalable.</li><li><strong>WAF</strong> — Block bad actors before they hit your compute. Critical when you’re suddenly visible.</li><li><strong>API Gateway + Lambda</strong> — Serverless compute — zero cost at idle, scales to millions of requests. No servers to manage.</li><li><strong>DynamoDB On-Demand</strong> — Variable database traffic without capacity planning. Pay per request — perfect for unpredictable loads.</li></ul><p><strong>Architecture flow:</strong> Route 53 → CloudFront → WAF → API Gateway → Lambda → DynamoDB On-Demand <em>(S3 serves static assets directly via CloudFront)</em></p><p>This stack costs almost nothing at low traffic and scales automatically. The trade-off: cold starts on Lambda and eventual consistency on DynamoDB — acceptable for a startup validating product-market fit.</p><h3>Why These Choices Work</h3><ul><li><strong>Traffic:</strong> Route 53 handles DNS failover. CloudFront absorbs the spike at the edge — your origin never sees the full blast. WAF stops the scrapers and bots that always follow viral traffic.</li><li><strong>Compute:</strong> Lambda is the only sane choice here. You’re not paying for idle capacity, and it scales in milliseconds. API Gateway gives you a clean REST interface without managing servers.</li><li><strong>Data:</strong> DynamoDB On-Demand means no capacity planning. When traffic is unpredictable, provisioning throughput is a guess — and guesses cost money.</li><li><strong>Cost:</strong> This entire stack is pay-per-use. At 100K requests, you’re looking at pennies. The moment traffic drops, your bill drops with it.</li></ul><h3>The Trade-Offs</h3><ul><li>⚡ <strong>Cold Starts</strong> — Lambda functions can take 100–500ms to initialize. Mitigate with provisioned concurrency for critical paths.</li><li>📦 <strong>Vendor Lock-in</strong> — API Gateway + Lambda + DynamoDB is a very AWS-native stack. Migration later is non-trivial.</li><li>🔍 <strong>Debugging</strong> — Distributed serverless is harder to trace. Enable X-Ray from day one, not after things break.</li></ul><h3>Scenario 2: E-Commerce Resilience</h3><p><strong>The problem:</strong> A mid-size retailer loses $50,000/hour during a Black Friday crash. The root cause: a single point of failure and no auto-scaling capacity. This is the scenario where “works on my laptop” becomes a six-figure incident report.</p><p><strong>The stack:</strong></p><ul><li>🌐 <strong>Application Load Balancer</strong> — Distributes traffic across healthy targets. Health checks remove failed instances automatically — no manual intervention during a crisis.</li><li>🏗️ <strong>Multi-AZ VPC</strong> — Spread EC2 instances across at least two Availability Zones. One AZ goes down, traffic routes to the other. This is non-negotiable for production.</li><li>📈 <strong>Auto Scaling Groups</strong> — EC2 instances scale based on CPU, memory, or custom CloudWatch metrics. Define min/max — let AWS handle the middle.</li><li>🗄️ <strong>RDS Multi-AZ</strong> — Managed relational database with automatic failover. Your EC2 fleet can scale, but if the database is a single instance, you still have a single point of failure.</li></ul><p><strong>Architecture flow:</strong> Route 53 → CloudFront → Application Load Balancer → Auto Scaling Group (EC2 across 2+ AZs) → RDS Multi-AZ</p><h3>Why Resilience Costs Less Than Downtime</h3><p>At $50K/hour, a 4-hour Black Friday outage costs $200,000. The Multi-AZ + ALB + ASG stack adds maybe $3–5K/month in infrastructure costs. The ROI is obvious — this isn’t an optimization, it’s insurance.</p><blockquote><strong><em>The key insight:</em></strong><em> Auto Scaling Groups need warm-up time. If your scale-up policy takes 5 minutes and your traffic spike happens in 30 seconds, you’re already behind. Use predictive scaling or scheduled scaling for known events like Black Friday.</em></blockquote><h3>Layer Mapping</h3><ul><li><strong>Traffic:</strong> ALB + Multi-AZ ensures no single point of failure at the entry layer.</li><li><strong>Compute:</strong> EC2 via ASG — predictable workloads benefit from provisioned compute over serverless.</li><li><strong>Data:</strong> RDS Multi-AZ — relational data with ACID compliance and automatic failover.</li><li><strong>Observability:</strong> CloudWatch alarms on ASG metrics trigger scaling before humans notice the spike.</li></ul><h3>Scenario 3: The AI-Powered Enterprise</h3><p><strong>The problem:</strong> A finance company spends heavily on manual document processing — invoices, contracts, compliance reports. The work is repetitive, error-prone, and expensive. Generative AI can handle this, but the architecture needs to be robust, auditable, and company-specific.</p><p><strong>The stack:</strong></p><ul><li><strong>S3 Intake</strong> — Documents land in S3 — durable, versioned, and event-driven. S3 Event Notifications trigger the processing pipeline automatically.</li><li><strong>SQS Decoupling</strong> — A queue between S3 and processors prevents backpressure. If the AI layer slows down, documents wait safely — nothing drops.</li><li><strong>Fargate Containers</strong> — Sustained, predictable workloads run on Fargate — no server management, but unlike Lambda, no cold starts or execution time limits.</li><li><strong>Bedrock + RAG</strong> — Amazon Bedrock provides the LLM. RAG (Retrieval-Augmented Generation) injects company-specific context — so the AI answers from your documents, not generic training data.</li></ul><p><strong>Architecture flow:</strong> S3 (document upload) → SQS (queue) → Fargate (document processing + embedding) → Bedrock (LLM inference) + RAG vector store → structured output</p><h3>Stop Clicking. Start Coding: Infrastructure as Code</h3><p>If you’re building production infrastructure through the AWS Console, you’re building on sand. Click-Ops works for exploration — it fails in production because it’s not repeatable, not version-controlled, and not reviewable.</p><h3>Why Click-Ops Fails</h3><ul><li><strong>No audit trail</strong> — Who changed the security group? When? Why? The Console doesn’t tell you — CloudTrail logs are a forensic nightmare.</li><li><strong>Environment drift</strong> — Dev, Staging, and Production slowly diverge. A bug that doesn’t reproduce in Staging? Classic Click-Ops symptom.</li><li><strong>No rollback</strong> — Misconfigured a route table at 2 AM? Good luck remembering what you changed. With IaC, you revert a commit.</li></ul><h3>Terraform vs. CloudFormation</h3><p><strong>Terraform</strong> <strong>CloudFormation</strong> Scope Multi-cloud AWS-native Syntax HCL YAML/JSON Ecosystem Massive provider ecosystem Deep AWS service integration State Requires management (S3 + DynamoDB locking) No external state management Multi-account Workspaces/modules StackSets</p><blockquote><em>Start with </em><strong><em>Terraform</em></strong><em> if you might ever use another cloud. Start with </em><strong><em>CloudFormation</em></strong><em> if you’re all-in AWS and want native integration.</em></blockquote><h3>Key Takeaways &amp; Your Next Move</h3><ol><li><strong>Design with the Six-Layer Stack — in order.</strong> Traffic → Compute → Data → Security → Cost → Observability. Each layer constrains the next. Skipping layers creates technical debt you’ll pay for at 2 AM.</li><li><strong>Match the service to the workload, not your resume.</strong> Lambda for spiky, unpredictable traffic. EC2 + ASG for steady, predictable loads. Fargate for sustained containerized work. The right tool isn’t always the trendy one.</li><li><strong>Multi-AZ is not optional for production.</strong> If your architecture fits in one Availability Zone, it’s a demo, not a system. Multi-AZ is the baseline for anything with real users or real money.</li><li><strong>IaC from day one — even for small projects.</strong> The cost of migrating from Click-Ops to Terraform grows with every manual change. Write the HCL. Commit it. Review it. Your future self will thank you.</li><li><strong>Observability is a design requirement, not an afterthought.</strong> Enable CloudWatch, X-Ray, and structured logging before you deploy. You can’t debug what you can’t see — and by the time you realize you need logs, the evidence is gone.</li></ol><p>Which layer do you most often skip when designing systems? Drop a comment — I read every one. And if this was useful, follow for more field notes on AWS architecture, DevOps patterns, and the scars that come with building systems that actually run in production.</p><p>📌 Let’s connect on LinkedIn too — <a href="https://www.linkedin.com/in/jysadhu">linkedin.com/in/jysadhu</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=42f7976f28d7" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>