<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Sagar Pawadi on Medium]]></title>
        <description><![CDATA[Stories by Sagar Pawadi on Medium]]></description>
        <link>https://medium.com/@sagarpawadi?source=rss-67ef7bb588e6------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*BEyoIX7cx3qGwtEA</url>
            <title>Stories by Sagar Pawadi on Medium</title>
            <link>https://medium.com/@sagarpawadi?source=rss-67ef7bb588e6------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 06:08:01 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@sagarpawadi/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[ Deep dive Kubernetes Scheduling: Filtering/Scoring/Preemption/Affinity/PDB]]></title>
            <link>https://medium.com/@sagarpawadi/deep-dive-kubernetes-scheduling-filtering-scoring-preemption-affinity-pdb-42f2806daf00?source=rss-67ef7bb588e6------2</link>
            <guid isPermaLink="false">https://medium.com/p/42f2806daf00</guid>
            <category><![CDATA[kubernetes-scheduler]]></category>
            <category><![CDATA[scheduling]]></category>
            <category><![CDATA[pdb]]></category>
            <category><![CDATA[affinity]]></category>
            <dc:creator><![CDATA[Sagar Pawadi]]></dc:creator>
            <pubDate>Mon, 17 Nov 2025 06:59:43 GMT</pubDate>
            <atom:updated>2025-11-17T06:59:43.421Z</atom:updated>
            <content:encoded><![CDATA[<h3>🔍 Introduction: The Invisible Orchestrator</h3><p><strong>Picture this:</strong> You deploy a pod to your Kubernetes cluster. Within milliseconds, it lands on the perfect node — one with enough resources, the right labels and no conflicting workloads. But have you ever wondered how kube-scheduler makes this magic happen?​​</p><p>Behind the scenes, the Kubernetes scheduler performs an elegant two-step dance: filtering unsuitable nodes and scoring the survivors to find the optimal placement. When resources run tight, preemption kicks in to evict lower-priority pods for critical workloads.​</p><p>In this blog, we’ll demystify the scheduler’s algorithms, explore where anti-affinity, PDB, and taints/tolerations fit into the workflow, and dissect a real production incident where conflicting policies created a deployment deadlock that stumped engineers for hours.</p><h3>⚙️ Part 1: The Kubernetes Scheduler Explained</h3><h3>What is kube-scheduler?</h3><p>The kube-scheduler is the default scheduler running as part of the Kubernetes control plane. Its mission: watch for newly created pods with no assigned node and find the best home for them.​</p><p>When a pod is created, the scheduler evaluates every available node based on:</p><ul><li>Resource availability (CPU, memory, storage)​</li><li>Constraints (node selectors, affinity/anti-affinity)​</li><li>Taints and tolerations​</li><li>Policy restrictions (PodDisruptionBudgets, resource quotas)​</li></ul><h3>The Two-Step Scheduling Process</h3><p>The scheduler uses a two-phase approach: Filtering and Scoring.​</p><h3>Phase 1: Filtering (Finding Feasible Nodes)</h3><p>In the filtering phase, the scheduler eliminates nodes that don’t meet the pod’s requirements. Think of this as a bouncer at an exclusive club — only nodes that pass all entry criteria make it through.​</p><p>Common Filters:</p><ul><li>PodToleratesNodeTaints: Does the pod tolerate node taints?​</li><li>PodFitsResources: Does the node have enough CPU and memory?​</li><li>NodeSelector &amp; NodeAffinity: Does the node match label requirements?​</li><li>PodAntiAffinity (Required): Are conflicting pods already on the node?​</li><li>VolumeZone: Can the node satisfy storage requirements?</li></ul><p>After filtering, you’re left with a list of feasible nodes. If this list is empty, the pod remains unscheduled (stuck in Pending state) until resources become available.​</p><h3>Phase 2: Scoring (Ranking the Best Node)</h3><p>Once feasible nodes are identified, the scheduler ranks them using scoring functions (also called priorities). Each node receives a score between 0 and 100, and the node with the highest score wins.​</p><p><strong>Key Scoring Criteria:</strong></p><ul><li><strong>LeastAllocated</strong>: Favors nodes with more free resources​</li><li><strong>BalancedResourceAllocation</strong>: Prefers balanced CPU and memory usage​</li><li><strong>NodeAffinityPriority</strong>: Rewards nodes matching affinity preferences​</li><li><strong>ImageLocality</strong>: Prefers nodes that already have the container image cached (faster startup!)​</li><li><strong>PodAntiAffinity </strong>(Preferred): Bonus points for nodes without conflicting pods​</li></ul><pre>apiVersion: kubescheduler.config.k8s.io/v1beta3<br>kind: KubeSchedulerConfiguration<br>profiles:<br>- schedulerName: default-scheduler<br>  pluginConfig:<br>  - name: NodeResourcesFit<br>    args:<br>      scoringStrategy:<br>        type: LeastAllocated<br>        resources:<br>        - name: cpu<br>          weight: 2  # CPU gets 2x importance vs memory<br>        - name: memory<br>          weight: 1</pre><p>After scoring, the scheduler binds the pod to the highest-ranked node.​</p><h3>🚨 Part 2: Preemption Logic</h3><h3>When High-Priority Pods Need a Seat</h3><p>What happens when a high-priority pod arrives but all nodes are full? Enter preemption logic.​</p><p><strong>Preemption allows the scheduler to evict lower-priority pods to make room for higher-priority ones. Here’s the process:​​</strong></p><ol><li><strong>Priority Queue: </strong>Pods are arranged in the scheduling queue by priority. High-priority pods jump to the front.​</li><li><strong>Resource Check: </strong>If no node has resources for a high-priority pod, preemption kicks in.​</li><li><strong>Victim Selection:</strong> The scheduler identifies lower-priority pods to evict.​</li><li><strong>Graceful Termination: </strong>Evicted pods get a 30-second grace period (or custom terminationGracePeriodSeconds).​</li><li><strong>Scheduling:</strong> The high-priority pod is placed on the freed-up node.​</li></ol><h3>Creating Priority Classes</h3><pre>apiVersion: scheduling.k8s.io/v1<br>kind: PriorityClass<br>metadata:<br>  name: critical-priority<br>value: 1000000<br>globalDefault: false<br>description: &quot;Reserved for business-critical workloads&quot;<br>---<br>apiVersion: scheduling.k8s.io/v1<br>kind: PriorityClass<br>metadata:<br>  name: standard-priority<br>value: 1000<br>globalDefault: true<br>description: &quot;Default priority for regular workloads&quot;</pre><h3>Using PriorityClass in a Pod</h3><pre>apiVersion: v1<br>kind: Pod<br>metadata:<br>  name: payment-processor<br>spec:<br>  priorityClassName: critical-priority<br>  containers:<br>  - name: app<br>    image: payment-api:v2</pre><h3>🧩 Part 3: Where Anti-Affinity, PDB, and Taints/Tolerations Fit</h3><p>Now that we understand the scheduling phases, let’s map out exactly where each mechanism comes into play.</p><h3>1. Taints/Tolerations → Filtering Phase</h3><p>Taints and tolerations are evaluated during the filtering phase as one of the first checks to eliminate unsuitable nodes.​</p><p><strong>How It Works:</strong></p><ul><li>Taints are applied to nodes to mark them as “off-limits” for pods without matching tolerations.​</li><li>Tolerations are specified in pod specs to allow scheduling on tainted nodes.​</li><li>The scheduler uses the Pod Tolerates Node Taints filter to check compatibility.​</li></ul><pre># Node with taint (e.g., dedicated GPU node)<br>apiVersion: v1<br>kind: Node<br>metadata:<br>  name: gpu-node-1<br>spec:<br>  taints:<br>  - key: &quot;nvidia.com/gpu&quot;<br>    value: &quot;true&quot;<br>    effect: &quot;NoSchedule&quot;  # Blocks pods without toleration</pre><pre># Pod with toleration<br>apiVersion: v1<br>kind: Pod<br>metadata:<br>  name: ml-training-job<br>spec:<br>  tolerations:<br>  - key: &quot;nvidia.com/gpu&quot;<br>    operator: &quot;Equal&quot;<br>    value: &quot;true&quot;<br>    effect: &quot;NoSchedule&quot;  # Allows scheduling on gpu-node-1<br>  containers:<br>  - name: tensorflow<br>    image: tensorflow/tensorflow:latest-gpu</pre><p>Result: Only pods with matching tolerations pass the filtering phase for tainted nodes.​</p><h3>2. Anti-Affinity → Filtering &amp; Scoring Phases</h3><blockquote>Anti-affinity rules are evaluated differently depending on whether they’re “required” (hard) or “preferred” (soft).​</blockquote><h3>Hard Anti-Affinity (Required) → Filtering Phase</h3><pre>affinity:<br>  podAntiAffinity:<br>    requiredDuringSchedulingIgnoredDuringExecution:<br>    - labelSelector:<br>        matchExpressions:<br>        - key: app<br>          operator: In<br>          values:<br>          - database<br>      topologyKey: kubernetes.io/hostname</pre><ul><li>When: During filtering (node elimination)​</li><li>Effect: Nodes with matching pods are immediately disqualified​</li><li>Use Case: Enforce strict separation (e.g., never co-locate database replicas)​</li></ul><h3>Soft Anti-Affinity (Preferred) → Scoring Phase</h3><pre>affinity:<br>  podAntiAffinity:<br>    preferredDuringSchedulingIgnoredDuringExecution:<br>    - weight: 100<br>      podAffinityTerm:<br>        labelSelector:<br>          matchExpressions:<br>          - key: app<br>            operator: In<br>            values:<br>            - web-frontend<br>        topologyKey: kubernetes.io/hostname</pre><ul><li>When: During scoring (node ranking)​</li><li>Effect: Nodes without matching pods get higher scores​</li><li>Use Case: Prefer spreading pods but allow co-location if necessary​</li></ul><h3>3. PDB (Pod Disruption Budget) → Post-Scheduling Protection</h3><p>PDB does NOT directly affect the initial scheduling decision. Instead, it protects running pods from voluntary disruptions like node drains, cluster upgrades, or evictions.​</p><pre>apiVersion: policy/v1<br>kind: PodDisruptionBudget<br>metadata:<br>  name: web-frontend-pdb<br>spec:<br>  minAvailable: 2  # Must keep at least 2 pods running<br>  selector:<br>    matchLabels:<br>      app: web-frontend</pre><p><strong>When PDB Matters:</strong></p><ul><li>During rolling updates​</li><li>During node drains (maintenance)​</li><li>When the eviction API is called​</li><li>When preemption might evict lower-priority pods​</li></ul><h3>Complete Scheduling Workflow</h3><pre>┌───────────────────────────────────────────────────────┐<br>│  1. Pod Created (Unscheduled)                         │<br>└──────────────────────┬────────────────────────────────┘<br>                       │<br>                       ▼<br>┌───────────────────────────────────────────────────────┐<br>│  2. FILTERING PHASE (Eliminate Unsuitable Nodes)      │<br>│                                                       │<br>│  ✓ PodToleratesNodeTaints                             │<br>│    └─ Check: Does pod tolerate node taints?           │<br>│                                                       │<br>│  ✓ PodFitsResources                                   │<br>│    └─ Check: Enough CPU/memory?                       │<br>│                                                       │<br>│  ✓ NodeAffinity/NodeSelector                          │<br>│    └─ Check: Does node match label requirements?      │<br>│                                                       │<br>│  ✓ PodAntiAffinity (Required)                         │<br>│    └─ Check: Are conflicting pods already on node?    │<br>│                                                       │<br>│  ✓ VolumeZone, PodTopologySpread, etc.                │<br>│                                                       │<br>└──────────────────────┬────────────────────────────────┘<br>                       │<br>                       ▼<br>┌───────────────────────────────────────────────────────┐<br>│  3. Feasible Nodes List                               │<br>│     (If empty → Pod stays Pending)                    │<br>└──────────────────────┬────────────────────────────────┘<br>                       │<br>                       ▼<br>┌───────────────────────────────────────────────────────┐<br>│  4. SCORING PHASE (Rank Nodes 0-100)                  │<br>│                                                       │<br>│  ✓ LeastAllocated                                     │<br>│    └─ Score: Favor nodes with more free resources     │<br>│                                                       │<br>│  ✓ BalancedResourceAllocation                         │<br>│    └─ Score: Prefer balanced CPU/memory usage         │<br>│                                                       │<br>│  ✓ PodAntiAffinity (Preferred)                        │<br>│    └─ Score: Bonus points for nodes without conflicts │<br>│                                                       │<br>│  ✓ ImageLocality, NodeAffinityPriority, etc.          │<br>│                                                       │<br>└──────────────────────┬────────────────────────────────┘<br>                       │<br>                       ▼<br>┌───────────────────────────────────────────────────────┐<br>│  5. Highest Scoring Node Selected                     │<br>└──────────────────────┬────────────────────────────────┘<br>                       │<br>                       ▼<br>┌───────────────────────────────────────────────────────┐<br>│  6. BINDING (Pod assigned to Node)                    │<br>└──────────────────────┬────────────────────────────────┘<br>                       │<br>                       ▼<br>┌───────────────────────────────────────────────────────┐<br>│  7. Pod Running                                       │<br>└──────────────────────┬────────────────────────────────┘<br>                       │<br>                       ▼<br>┌───────────────────────────────────────────────────────┐<br>│  8. LATER: PDB Protects Pod During:                   │<br>│     • Rolling updates                                 │<br>│     • Node drains                                     │<br>│     • Cluster upgrades                                │<br>│     • Preemption/eviction requests                    │<br>└───────────────────────────────────────────────────────┘</pre><h3>📚 References</h3><ul><li><a href="https://kubernetes.io/docs/concepts/scheduling-eviction/">Kubernetes Scheduling, Preemption and Eviction</a></li><li><a href="https://kubernetes.io/docs/concepts/scheduling-eviction/kube-scheduler/">Kubernetes Scheduler Documentation</a></li><li><a href="https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/">Taints and Tolerations</a></li><li><a href="https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/">Assigning Pods to Nodes</a></li><li><a href="https://kubernetes.io/docs/concepts/workloads/pods/disruptions/">Pod Disruption Budgets</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=42f2806daf00" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mastering Admission Webhooks in Kubernetes: Mutating vs Validating️]]></title>
            <link>https://medium.com/@sagarpawadi/mastering-admission-webhooks-in-kubernetes-mutating-vs-validating-%EF%B8%8F-0d352b6ff89f?source=rss-67ef7bb588e6------2</link>
            <guid isPermaLink="false">https://medium.com/p/0d352b6ff89f</guid>
            <category><![CDATA[admission-webhook]]></category>
            <category><![CDATA[webhooks]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[mutating]]></category>
            <dc:creator><![CDATA[Sagar Pawadi]]></dc:creator>
            <pubDate>Sat, 15 Nov 2025 08:44:39 GMT</pubDate>
            <atom:updated>2025-11-15T13:28:52.917Z</atom:updated>
            <content:encoded><![CDATA[<p>Want to enforce custom policies, inject sidecars, or validate resource configurations before they hit your Kubernetes cluster? Admission webhooks are your secret weapon. They act as gatekeepers, intercepting API requests before they’re persisted to etcd, giving you full control over what gets into your cluster.​</p><p>In this guide, we’ll demystify mutating and validating admission webhooks, explore real-world use cases, and walk through practical examples with commands and configurations.</p><h3>🔐 What Are Admission Controllers?</h3><p>Admission controllers are plugins in the Kubernetes API server that intercept requests after authentication and authorization but before persistence. They can:​</p><ul><li>✅ Validate requests (reject invalid configurations)</li><li>✏️ Mutate requests (modify objects automatically)</li><li>🔄 Execute custom logic via HTTP webhooks</li></ul><p>Flow Diagram:</p><pre>User → kubectl → API Server → Authentication → Authorization <br>→ Mutating Admission → Validating Admission → etcd (Persist)</pre><p>Check enabled admission controllers:</p><pre>kube-apiserver -h | grep enable-admission-plugins</pre><p>Default controllers in Kubernetes 1.34:</p><pre>CertificateApproval, DefaultIngressClass, LimitRanger, <br>MutatingAdmissionWebhook, NamespaceLifecycle, PodSecurity, <br>ResourceQuota, ValidatingAdmissionWebhook</pre><h3>🔀 Two Types of Admission Webhooks</h3><h3>1️⃣ Mutating Admission Webhooks 🛠️</h3><p>What they do: Modify (mutate) objects before they’re stored.​</p><p>Common Use Cases:</p><ul><li>Inject sidecar containers (e.g., Istio proxy, logging agents)</li><li>Set default resource limits</li><li>Add labels, annotations, or environment variables</li><li>Apply security policies automatically</li></ul><p>Example: Injecting a Sidecar</p><p>Imagine you want to automatically inject a logging sidecar into every Pod in the production namespace.</p><p>Webhook configuration:</p><pre>apiVersion: admissionregistration.k8s.io/v1<br>kind: MutatingWebhookConfiguration<br>metadata:<br>  name: sidecar-injector<br>webhooks:<br>  - name: sidecar.example.com<br>    clientConfig:<br>      service:<br>        name: sidecar-injector<br>        namespace: webhook-system<br>        path: &quot;/mutate&quot;<br>      caBundle: &lt;BASE64_CA_BUNDLE&gt;<br>    rules:<br>      - operations: [&quot;CREATE&quot;]<br>        apiGroups: [&quot;&quot;]<br>        apiVersions: [&quot;v1&quot;]<br>        resources: [&quot;pods&quot;]<br>    admissionReviewVersions: [&quot;v1&quot;]<br>    sideEffects: None<br>    namespaceSelector:<br>      matchLabels:<br>        inject-sidecar: &quot;true&quot;</pre><p>Webhook response (JSON Patch):</p><pre>{<br>  &quot;apiVersion&quot;: &quot;admission.k8s.io/v1&quot;,<br>  &quot;kind&quot;: &quot;AdmissionReview&quot;,<br>  &quot;response&quot;: {<br>    &quot;uid&quot;: &quot;&lt;request-uid&gt;&quot;,<br>    &quot;allowed&quot;: true,<br>    &quot;patchType&quot;: &quot;JSONPatch&quot;,<br>    &quot;patch&quot;: &quot;W3sib3AiOiAiYWRkIiwgInBhdGgiOiAiL3NwZWMvY29udGFpbmVycyIsICJ2YWx1ZSI6IFt7Im5hbWUiOiAibG9nLWFnZW50IiwgImltYWdlIjogImZsdWVudGQ6bGF0ZXN0In1dfV0=&quot;<br>  }<br>}</pre><p>This adds a Fluentd logging container to every Pod.​</p><h3>2️⃣ Validating Admission Webhooks ✅</h3><p>What they do: Validate requests and accept or reject them based on custom logic.​</p><p>Common Use Cases:</p><ul><li>Enforce naming conventions</li><li>Block privileged containers</li><li>Ensure resource quotas are set</li><li>Validate custom security policies</li><li>Check compliance requirements</li></ul><p>Example: Blocking Privileged Pods</p><p>Let’s create a webhook that rejects any Pod trying to run as privileged.</p><p>Webhook configuration:</p><pre>apiVersion: admissionregistration.k8s.io/v1<br>kind: ValidatingWebhookConfiguration<br>metadata:<br>  name: block-privileged-pods<br>webhooks:<br>  - name: validate-privileged.example.com<br>    clientConfig:<br>      service:<br>        name: pod-validator<br>        namespace: webhook-system<br>        path: &quot;/validate&quot;<br>      caBundle: &lt;BASE64_CA_BUNDLE&gt;<br>    rules:<br>      - operations: [&quot;CREATE&quot;, &quot;UPDATE&quot;]<br>        apiGroups: [&quot;&quot;]<br>        apiVersions: [&quot;v1&quot;]<br>        resources: [&quot;pods&quot;]<br>    admissionReviewVersions: [&quot;v1&quot;]<br>    sideEffects: None<br>    failurePolicy: Fail</pre><p>Webhook response (Rejecting request)</p><pre>{<br>  &quot;apiVersion&quot;: &quot;admission.k8s.io/v1&quot;,<br>  &quot;kind&quot;: &quot;AdmissionReview&quot;,<br>  &quot;response&quot;: {<br>    &quot;uid&quot;: &quot;&lt;request-uid&gt;&quot;,<br>    &quot;allowed&quot;: false,<br>    &quot;status&quot;: {<br>      &quot;code&quot;: 403,<br>      &quot;message&quot;: &quot;Privileged containers are not allowed in this cluster&quot;<br>    }<br>  }<br>}</pre><p>The Pod creation is rejected.​</p><h3>🔄 Admission Control Flow</h3><p>Here’s how requests flow through admission controllers:</p><pre>┌─────────────────────────────────────────────────────────┐<br>│  API Request (kubectl create pod ...)                   │<br>└────────────────┬────────────────────────────────────────┘<br>                 │<br>                 ▼<br>┌─────────────────────────────────────────────────────────┐<br>│  Authentication + Authorization                         │<br>└────────────────┬────────────────────────────────────────┘<br>                 │<br>                 ▼<br>┌─────────────────────────────────────────────────────────┐<br>│  MUTATING Admission Phase                               │<br>│  ├── Built-in mutating controllers                      │<br>│  └── MutatingAdmissionWebhook (HTTP callout)            │<br>│      → Modify object (add labels, inject sidecars)      │<br>└────────────────┬────────────────────────────────────────┘<br>                 │<br>                 ▼<br>┌─────────────────────────────────────────────────────────┐<br>│  Schema Validation                                      │<br>└────────────────┬────────────────────────────────────────┘<br>                 │<br>                 ▼<br>┌─────────────────────────────────────────────────────────┐<br>│  VALIDATING Admission Phase                             │<br>│  ├── Built-in validating controllers                    │<br>│  └── ValidatingAdmissionWebhook (HTTP callout)          │<br>│      → Accept or Reject                                 │<br>└────────────────┬────────────────────────────────────────┘<br>                 │<br>                 ▼<br>┌─────────────────────────────────────────────────────────┐<br>│  Persist to etcd ✅                                     │<br>└─────────────────────────────────────────────────────────┘</pre><p>Key points:</p><ul><li>Mutating webhooks run first​</li><li>Validating webhooks run last and see the final object​</li><li>If any webhook rejects, the entire request fails immediately​</li></ul><h3>📊 Mutating vs Validating: Quick Comparison</h3><p>Feature Mutating Webhook Validating Webhook Purpose Modify objects Accept or reject objects Execution Order First (before validation)Last (after mutations)Can Change Object✅ Yes❌ No Use Case Examples Inject sidecars, set defaults Enforce policies, compliance Response Type JSON Patch or allow Allow or deny with message Side Effects Possible Should be read-only</p><h3>🛠️ Common Webhook Patterns</h3><h3>Pattern 1: Default Value Injection 🔧</h3><p>Use case: Automatically set resource limits if not specified.</p><pre># Mutating webhook adds default limits<br>{<br>  &quot;op&quot;: &quot;add&quot;,<br>  &quot;path&quot;: &quot;/spec/containers/0/resources/limits&quot;,<br>  &quot;value&quot;: {<br>    &quot;cpu&quot;: &quot;500m&quot;,<br>    &quot;memory&quot;: &quot;512Mi&quot;<br>  }<br>}</pre><h3>Pattern 2: Label-Based Policy Enforcement 🏷️</h3><p>Use case: Validate that Pods have required labels.</p><p>Webhook logic:</p><pre>def validate_pod(pod):<br>    required_labels = [&quot;team&quot;, &quot;environment&quot;, &quot;app&quot;]<br>    pod_labels = pod.get(&quot;metadata&quot;, {}).get(&quot;labels&quot;, {})<br>    <br>    for label in required_labels:<br>        if label not in pod_labels:<br>            return reject(f&quot;Missing required label: {label}&quot;)<br>    <br>    return allow()</pre><h3>Pattern 3: Namespace-Specific Mutations 🌐</h3><p>Use case: Inject monitoring sidecars only in production namespaces.</p><pre>namespaceSelector:<br>  matchLabels:<br>    environment: production</pre><h3>Pattern 4: Conditional Validation with matchConditions 🎯</h3><p>Use case: Skip validation for admin users.</p><pre>matchConditions:<br>  - name: &#39;exclude-admins&#39;<br>    expression: &#39;!(&quot;system:masters&quot; in request.userInfo.groups)&#39;</pre><h3>📝 Building Your Own Admission Webhook</h3><h3>Step 1: Create Webhook Server</h3><p>Simple Go example</p><pre>func handleAdmission(w http.ResponseWriter, r *http.Request) {<br>    var admissionReview v1.AdmissionReview<br>    json.NewDecoder(r.Body).Decode(&amp;admissionReview)<br>    <br>    // Your validation/mutation logic here<br>    response := v1.AdmissionResponse{<br>        UID:     admissionReview.Request.UID,<br>        Allowed: true,<br>    }<br>    <br>    admissionReview.Response = &amp;response<br>    json.NewEncoder(w).Encode(admissionReview)<br>}</pre><h3>Step 2: Deploy as Service</h3><pre>apiVersion: v1<br>kind: Service<br>metadata:<br>  name: my-webhook<br>  namespace: webhook-system<br>spec:<br>  selector:<br>    app: webhook<br>  ports:<br>    - port: 443<br>      targetPort: 8443</pre><h3>Step 3: Register Webhook Configuration</h3><pre>kubectl apply -f mutating-webhook-config.yaml<br>kubectl apply -f validating-webhook-config.yaml</pre><h3>Step 4: Test Your Webhook</h3><pre># Check webhook configuration<br>kubectl get mutatingwebhookconfigurations<br>kubectl get validatingwebhookconfigurations</pre><pre># Test by creating a resource<br>kubectl create deployment test-app --image=nginx</pre><pre># View webhook logs<br>kubectl logs -n webhook-system -l app=webhook</pre><h3>🔍 Debugging Admission Webhooks</h3><p>Check webhook endpoints:</p><pre>kubectl describe validatingwebhookconfiguration &lt;name&gt;</pre><p>View admission events:</p><pre>kubectl get events --all-namespaces | grep Admission</pre><p>Test webhook connectivity:</p><pre># From inside cluster<br>kubectl run test-pod --image=curlimages/curl -it --rm -- \<br>  curl -k https://my-webhook.webhook-system.svc:443/validate</pre><h3>⚙️ Webhook Configuration Best Practices</h3><h3>Failure Policy 🚨</h3><pre>failurePolicy: Fail  # Reject if webhook fails (secure)<br>failurePolicy: Ignore  # Allow if webhook fails (flexible)</pre><p>Recommendation: Use Fail for critical validations, Ignore for optional enhancements.​</p><h3>Timeout Configuration ⏱️</h3><pre>timeoutSeconds: 5  # Must be between 1-30 seconds</pre><p>Recommendation: Keep webhooks fast (&lt;5s) to avoid blocking API requests.​</p><h3>Scope Filtering 🎯</h3><pre>rules:<br>  - operations: [&quot;CREATE&quot;]<br>    apiGroups: [&quot;apps&quot;]<br>    apiVersions: [&quot;v1&quot;]<br>    resources: [&quot;deployments&quot;]<br>    scope: &quot;Namespaced&quot;  # Only namespaced resources</pre><h3>Dry Run Support 🧪</h3><pre>sideEffects: None  # No side effects<br>sideEffects: NoneOnDryRun  # Skip side effects on dry-run</pre><p>Always handle dryRun: true requests properly.​</p><h3>🎯 Real-World Use Cases</h3><p>🔹 Istio Service Mesh: Mutating webhook injects Envoy proxy sidecars<br>🔹 OPA Gatekeeper: Validating webhook enforces policy-as-code<br>🔹 Vault Agent Injector: Mutating webhook injects secrets agent<br>🔹 Pod Security Admission: Validating webhook enforces security standards<br>🔹 cert-manager: Mutating webhook injects TLS certificates</p><h3>Top tools that actually helps with this :)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ov9my-Bvo7hsHsd7qDsLfg.png" /></figure><p>References:</p><ul><li><a href="https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/">Kubernetes Admission Controllers Documentation</a></li><li><a href="https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/">Dynamic Admission Control</a></li><li>GitHub: <a href="https://github.com/CrowdStrike/falcon-operator">CrowdStrike/falcon-operator</a>​</li><li>Red Hat Catalog: <a href="https://catalog.redhat.com/software/container-stacks/detail/crowdstrike-falcon-operator">falcon-operator</a>​</li><li>Helm Chart: <a href="https://artifacthub.io/packages/helm/crowdstrike/falcon-kac">falcon-kac</a>​</li></ul><p>Have you built custom admission webhooks? Share your use cases in the comments! 💬</p><p><strong>#Kubernetes #OpenShift #AdmissionWebhooks #DevOps #SRE #CloudNative #PlatformEngineering #SecurityAutomation</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0d352b6ff89f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Decoding Kube-Proxy: iptables vs IPVS Mode Explained ]]></title>
            <link>https://medium.com/@sagarpawadi/decoding-kube-proxy-iptables-vs-ipvs-mode-explained-4050f6af288b?source=rss-67ef7bb588e6------2</link>
            <guid isPermaLink="false">https://medium.com/p/4050f6af288b</guid>
            <category><![CDATA[iptables]]></category>
            <category><![CDATA[ipv]]></category>
            <category><![CDATA[large-clusters]]></category>
            <category><![CDATA[kubernetes-networking]]></category>
            <category><![CDATA[kube-proxy]]></category>
            <dc:creator><![CDATA[Sagar Pawadi]]></dc:creator>
            <pubDate>Sat, 15 Nov 2025 07:50:16 GMT</pubDate>
            <atom:updated>2025-11-15T08:00:36.797Z</atom:updated>
            <content:encoded><![CDATA[<h3>Decoding Kube-Proxy: iptables vs IPVS vs nftables Mode Explained 🚀</h3><p>Ever wondered how Kubernetes routes traffic to your Services? The magic happens through <strong>kube-proxy</strong> — a critical component running on every node that implements virtual IP (VIP) mechanisms for Services. But here’s the kicker: kube-proxy can operate in different modes and choosing the right one can significantly impact your cluster’s performance and scalability.​</p><p>In this article, we’ll break down the two most popular modes — iptables and IPVS — with practical examples, commands, and clear explanations to help you master Kubernetes networking.</p><h3>🔍 What is Kube-Proxy?</h3><p>Kube-proxy is the node-level component responsible for traffic forwarding from Service ClusterIPs to backend Pods. It watches the Kubernetes control plane for Service and EndpointSlice changes, then configures the node’s networking stack to route traffic accordingly.​</p><p>Key Responsibilities:</p><ul><li>Implement virtual IPs for Services</li><li>Load balance traffic across healthy endpoints</li><li>Handle traffic policies and session affinity</li><li>Update networking rules dynamically as Pods come and go</li></ul><p>Check kube-proxy mode</p><pre>kubectl get configmap kube-proxy -n kube-system -o yaml | grep mode</pre><h3>⚙️ iptables Proxy Mode</h3><p>The iptables mode is the default and most widely used mode in Kubernetes. It uses Linux iptables rules to redirect traffic from Service VIPs to backend Pods.​</p><h3>How It Works 🔧</h3><p>When you create a Service with ClusterIP 10.0.0.1:1234, kube-proxy installs a series of iptables rules:</p><ol><li>Service rules redirect traffic from the VIP to per-Service chains</li><li>Endpoint rules randomly select a backend Pod (or use session affinity)</li><li>Destination NAT (DNAT) rewrites packets to route them to the selected Pod without changing the client IP​</li></ol><h3>Example: Image Processing Service</h3><p>Let’s say you have a Service with 3 backend Pods:</p><pre>kubectl create deployment image-processor --image=nginx:latest --replicas=3<br>kubectl expose deployment image-processor --port=80 --target-port=80 --type=ClusterIP</pre><p>Check the Service:</p><pre>kubectl get svc image-processor</pre><p>Output:</p><pre>NAME               TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)   AGE<br>image-processor    ClusterIP   10.0.0.1      &lt;none&gt;        80/TCP    10s</pre><p>When kube-proxy sees this Service, it creates iptables rules. You can inspect them:</p><pre>sudo iptables -t nat -L -n | grep 10.0.0.1</pre><h3>Performance Considerations ⚡</h3><p>In large clusters with tens of thousands of Pods and Services, iptables mode can become slow because:</p><ul><li>It creates one rule per endpoint (thousands of rules)</li><li>Linear rule processing (O(n) complexity)</li><li>Slow rule updates during Service changes​</li></ul><p>Optimization Tips:</p><pre># kube-proxy config<br>iptables:<br>  minSyncPeriod: 1s   # Aggregate rapid changes<br>  syncPeriod: 30s     # Full sync interval</pre><h3>🚀 IPVS Proxy Mode</h3><p>IPVS (IP Virtual Server) mode was introduced to address iptables’ performance limitations. It uses the Linux kernel’s IPVS module with a hash table for faster lookups and better throughput.​</p><h3>How It Works 🔧</h3><p>IPVS operates in kernel space and provides:</p><ul><li>Hash table-based lookups (O(1) complexity)</li><li>Better performance at scale</li><li>Advanced load balancing algorithms</li></ul><h3>Load Balancing Algorithms 🎯</h3><p>IPVS offers 10+ scheduling algorithms:​</p><p>AlgorithmIconDescriptionrr (Round Robin)🔄Equal distribution across backendslc (Least Connection)📊Route to servers with fewer connectionssh (Source Hashing)🔐Consistent routing based on source IPdh (Destination Hashing)🎯Route based on destination IPsed (Shortest Expected Delay)⏱️Route to server with shortest delay</p><p>Configure IPVS mode:</p><pre>apiVersion: v1<br>kind: ConfigMap<br>metadata:<br>  name: kube-proxy<br>  namespace: kube-system<br>data:<br>  config.conf: |<br>    mode: ipvs<br>    ipvs:<br>      scheduler: rr  # Round Robin</pre><h3>Enabling IPVS Mode</h3><p>Step 1: Verify IPVS kernel modules are loaded</p><pre># Load IPVS modules<br>sudo modprobe ip_vs<br>sudo modprobe ip_vs_rr<br>sudo modprobe ip_vs_wrr<br>sudo modprobe ip_vs_sh</pre><pre># Verify modules<br>lsmod | grep ip_vs</pre><p>Step 2: Update kube-proxy ConfigMap</p><pre>kubectl edit configmap kube-proxy -n kube-system</pre><p>Change mode: &quot;iptables&quot; to mode: &quot;ipvs&quot;</p><p>Step 3: Restart kube-proxy</p><pre>kubectl rollout restart daemonset kube-proxy -n kube-system</pre><p>Step 4: Verify IPVS is active</p><pre>sudo ipvsadm -Ln</pre><p>Output:</p><pre>IP Virtual Server version 1.2.1 (size=4096)<br>Prot LocalAddress:Port Scheduler Flags<br>  -&gt; RemoteAddress:Port           Forward Weight ActiveConn InActConn<br>TCP  10.0.0.1:80 rr<br>  -&gt; 10.244.1.5:80                Masq    1      0          0<br>  -&gt; 10.244.2.6:80                Masq    1      0          0<br>  -&gt; 10.244.3.7:80                Masq    1      0          0</pre><h3>📊 iptables vs IPVS: The Showdown</h3><p>Feature iptables Mode IPVS ModePerformance Good for small clusters Excellent at scaleRule ComplexityO(n) — LinearO(1) — Hash table Load BalancingRandom selection only10+ algorithms Update SpeedSlow with many endpointsFast, incremental Memory Usage Higher with many rules LowerMaturity Stable, widely used Less tested, deprecated ⚠️</p><h3>⚠️ Important Note on IPVS</h3><p>While IPVS offers better performance, Kubernetes documentation notes that it’s expected to be deprecated in the future because it doesn’t fully support all edge cases of the Kubernetes Services API. The newer nftables mode (Kubernetes 1.33+) is the recommended replacement.​</p><h3>🆕 nftables Mode: The Future</h3><p>nftables is the successor to both iptables and IPVS, offering:​</p><ul><li>✅ Better performance than iptables</li><li>✅ Fully compliant with Kubernetes Services API (unlike IPVS)</li><li>✅ Efficient packet processing</li><li>⚠️ Requires Linux kernel 5.13+</li></ul><p>Enable nftables mode:</p><pre>mode: nftables</pre><h3>🛠️ Troubleshooting Commands</h3><p>Check current kube-proxy mode:</p><pre>kubectl logs -n kube-system -l k8s-app=kube-proxy | grep &quot;Using&quot;</pre><p>View iptables rules for a Service:</p><pre>sudo iptables-save | grep &lt;service-name&gt;</pre><p>View IPVS connections:</p><pre>sudo ipvsadm -Ln --stats</pre><p>Check Service endpoints:</p><pre>kubectl get endpoints &lt;service-name&gt;</pre><p>Describe Service for detailed info:</p><pre>kubectl describe svc &lt;service-name&gt;</pre><h3>🎯 Key Takeaways</h3><ul><li>Kube-proxy implements virtual IPs and load balancing for Kubernetes Services.​</li><li>iptables mode is the default, reliable, and works well for most clusters.​</li><li>IPVS mode offers better performance at scale but has limitations and is being deprecated.​</li><li>nftables mode is the future — faster, more efficient, and fully compliant.​</li><li>Choose based on your cluster size, kernel version, and performance requirements.</li></ul><p>For most production workloads, stick with iptables unless you’re running a massive cluster or can leverage nftables on modern kernels. Understanding these modes helps you optimize network performance and troubleshoot Service connectivity issues like a pro! 🚀</p><p>References: <a href="https://kubernetes.io/docs/reference/networking/virtual-ips/">Kubernetes Documentation: Virtual IPs and Service Proxies</a></p><p>What’s your experience with kube-proxy modes? Have you migrated to IPVS or nftables? Share your insights.</p><p><strong>#Kubernetes #OpenShift #KubeProxy #Networking #DevOps #SRE #CloudNative #ContainerOrchestration</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4050f6af288b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Pod Lifecycle in Kubernetes: From Birth to Termination ]]></title>
            <link>https://medium.com/@sagarpawadi/pod-lifecycle-in-kubernetes-from-birth-to-termination-82e1802347ca?source=rss-67ef7bb588e6------2</link>
            <guid isPermaLink="false">https://medium.com/p/82e1802347ca</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[pods]]></category>
            <category><![CDATA[pod-lifecycle]]></category>
            <category><![CDATA[kubernetes-probes]]></category>
            <dc:creator><![CDATA[Sagar Pawadi]]></dc:creator>
            <pubDate>Sat, 15 Nov 2025 07:10:12 GMT</pubDate>
            <atom:updated>2025-11-15T07:10:12.146Z</atom:updated>
            <content:encoded><![CDATA[<p>Running containers in Kubernetes is easy — but understanding what happens <strong>behind the scenes</strong> from the moment a Pod is created until it’s gracefully terminated? That’s where the magic happens. Let’s dive into the <strong>Pod Lifecycle</strong> and uncover the phases, states, and probes that keep your applications running smoothly.</p><h3>🌱 The Journey Begins: Pod Phases</h3><p>Every Pod goes through a well-defined lifecycle with distinct phases. Think of it like the life stages of an application — each with its own purpose and behavior.​</p><h3>Pod Phases Explained:</h3><p><strong>Pending</strong>⏳ Pod accepted by the cluster but waiting for scheduling, image pulls, or volume mounts​.<br><strong>Running</strong>✅ Pod bound to a node, at least one container is running or restarting.<br><strong>Succeeded</strong>🎉All containers terminated successfully; no restarts needed. <strong>Failed</strong>❌All containers terminated, with at least one failing (non-zero exit) <strong>Unknown</strong>❓Pod state cannot be determined — usually due to node communication issues.</p><p><strong>Check Pod Phase</strong></p><pre>kubectl get pods<br>kubectl get pod &lt;pod-name&gt; -o jsonpath=&#39;{.status.phase}&#39;</pre><p><strong>💡 Pro Tip:</strong> When you see CrashLoopBackOff in your kubectl get pods output, it&#39;s not a phase—it&#39;s a <strong>status indicator</strong> showing the Pod is stuck in a failure-restart loop with exponential backoff delays.​</p><h3>🔍 Container States: What’s Happening Inside?</h3><p>While Pod phases give the big picture, <strong>container states</strong> show what’s happening at the micro level. Kubernetes tracks three container states:bash​</p><ul><li><strong>⏸️ Waiting:</strong> Container isn’t running yet — still pulling images or applying configs.</li><li><strong>🟢 Running:</strong> Container executing normally; postStart hook (if any) has completed.</li><li><strong>🛑 Terminated:</strong> Container finished execution or failed; you’ll see exit codes and timestamps here.</li></ul><p><strong>Inspect Container State</strong></p><pre>kubectl describe pod &lt;pod-name&gt;<br>kubectl get pod &lt;pod-name&gt; -o jsonpath=&#39;{.status.containerStatuses[*].state}&#39;</pre><h3>🔄 Restart Policies: Keeping Pods Alive</h3><p>Kubernetes doesn’t give up easily. When containers fail, the <strong>restart policy</strong> determines what happens next:​</p><ul><li><strong>Always:</strong> Restart the container no matter what (default for most workloads).</li><li><strong>OnFailure:</strong> Restart only if the container exits with an error.</li><li><strong>Never:</strong> Let it fail and don’t restart.</li></ul><p><strong>Check Restart Policy:</strong></p><pre>kubectl get pod &lt;pod-name&gt; -o jsonpath=&#39;{.spec.restartPolicy}&#39;</pre><p><strong>Exponential Backoff:</strong> After crashes, Kubernetes waits progressively longer (10s → 20s → 40s… up to 5 minutes) before retrying, preventing system overload.​</p><h3>🩺 Health Checks: Probes That Keep Things Running</h3><p>Kubernetes uses <strong>probes</strong> to monitor container health and decide when to restart, route traffic, or wait for startup.​</p><h3>Three Types of Probes:</h3><p>Probe Type Icon Purpose <strong>Liveness Probe</strong>💓Checks if the container is alive; restarts it if it fails. <strong>Readiness Probe</strong>🚦Determines if the container is ready to receive traffic. <strong>Startup Probe</strong>🚀Gives slow-starting apps extra time before liveness/readiness checks begin.</p><p><strong>Check Mechanisms:</strong></p><ul><li>exec – Run a command inside the container</li><li>httpGet – HTTP request to a specified endpoint</li><li>tcpSocket – Check if a port is open</li><li>grpc – gRPC health check​</li></ul><p><strong>View Probe Configuration</strong></p><pre>kubectl get pod &lt;pod-name&gt; -o yaml | grep -A 10 livenessProbe<br>kubectl get pod &lt;pod-name&gt; -o yaml | grep -A 10 readinessProbe</pre><p><strong>When to use what?</strong></p><ul><li>Use <strong>liveness</strong> probes to restart unhealthy containers — If the process in your container is able to crash on its own whenever it encounters an issue or becomes unhealthy, you do not necessarily need a liveness probe; the kubelet will automatically perform the correct action in accordance with the Pod’s restartPolicy.If you’d like your container to be killed and restarted if a probe fails, then specify a liveness probe, and specify a restartPolicy of Always or OnFailure</li><li>Use <strong>readiness</strong> probes to control traffic flow — pods won’t get requests until they pass.​</li><li>Use <strong>startup</strong> probes for apps with long initialization times. ​</li></ul><h3>🛑 Graceful Termination: Saying Goodbye the Right Way</h3><p>When you delete a Pod, Kubernetes doesn’t just kill it instantly — it follows a <strong>graceful shutdown flow:</strong>​</p><ol><li><strong>TERM Signal Sent:</strong> Kubelet sends SIGTERM to containers, giving them time to clean up (default: 30 seconds​</li><li><strong>PreStop Hook Runs:</strong> If defined, this hook executes before termination.​</li><li><strong>Grace Period Expires:</strong> If containers are still running, Kubernetes sends SIGKILL for forceful shutdown.​</li><li><strong>Endpoints Removed:</strong> The Pod is removed from Service endpoints so no new traffic is routed.​</li></ol><p><strong>Delete Pod with Custom Grace Period:</strong></p><pre>kubectl delete pod &lt;pod-name&gt; --grace-period=60<br>kubectl delete pod &lt;pod-name&gt; --grace-period=0 --force  # Force delete</pre><p><strong>💡 Sidecar Containers:</strong> These are terminated <strong>last</strong>, ensuring they support main containers until the very end.​</p><h3>⚠️ Troubleshooting CrashLoopBackOff</h3><p>Seeing CrashLoopBackOff? Here&#39;s your debugging checklist.​</p><p><strong>Essential Commands:</strong></p><pre># Check logs<br>kubectl logs &lt;pod-name&gt;<br>kubectl logs &lt;pod-name&gt; --previous  # Logs from crashed container</pre><pre># Inspect events and details<br>kubectl describe pod &lt;pod-name&gt;</pre><pre># Check resource usage<br>kubectl top pod &lt;pod-name&gt;</pre><pre># Get Pod status and conditions<br>kubectl get pod &lt;pod-name&gt; -o jsonpath=&#39;{.status.conditions[*]}&#39;</pre><p><strong>Common Causes:</strong></p><ul><li><strong>Application errors</strong> causing container exits</li><li><strong>Configuration issues</strong> (wrong env vars, missing configs)</li><li><strong>Resource constraints</strong> (insufficient CPU/memory)</li><li><strong>Probe failures</strong> (liveness/startup probes timing out)​</li></ul><h3>🧹 Pod Garbage Collection</h3><p>Kubernetes doesn’t let terminated Pods pile up forever. The <strong>Pod Garbage Collector</strong> cleans up Pods in Succeeded or Failed phases once a threshold is reached, preventing resource leaks.​</p><p><strong>View Terminated Pods:</strong></p><pre>kubectl get pods --field-selector=status.phase=Failed<br>kubectl get pods --field-selector=status.phase=Succeeded</pre><h3>🎯 Key Takeaways</h3><ul><li>Pods go through <strong>five phases</strong>: Pending, Running, Succeeded, Failed, Unknown.​</li><li><strong>Container states</strong> (Waiting, Running, Terminated) provide granular insight.​</li><li><strong>Restart policies</strong> and exponential backoff prevent runaway failures.​</li><li><strong>Probes</strong> (liveness, readiness, startup) are critical for health monitoring and traffic management.​</li><li>Graceful termination ensures clean shutdowns with hooks and signal handling.​</li></ul><p>Mastering the Pod lifecycle means fewer outages, faster debugging, and more reliable Kubernetes deployments. 🚀</p><p><strong>What’s your biggest challenge with Pod management?</strong></p><p>#Kubernetes #DevOps #CloudNative #PodLifecycle #SRE #ContainerOrchestration #OpenShift</p><p>Ref: <a href="https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/">https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=82e1802347ca" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kubelet Internals — The Unsung Hero of Kubernetes]]></title>
            <link>https://medium.com/@sagarpawadi/kubelet-internals-the-unsung-hero-of-kubernetes-d9b728252b0d?source=rss-67ef7bb588e6------2</link>
            <guid isPermaLink="false">https://medium.com/p/d9b728252b0d</guid>
            <category><![CDATA[kubelet]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Sagar Pawadi]]></dc:creator>
            <pubDate>Thu, 13 Nov 2025 13:07:43 GMT</pubDate>
            <atom:updated>2025-11-13T13:07:43.913Z</atom:updated>
            <content:encoded><![CDATA[<h3>Kubelet Internals — The Unsung Hero of Kubernetes</h3><h3>Understanding Kubelet ⚙️</h3><p>Kubernetes orchestrates containers at scale, but the real work happens at the node level — where Kubelet lives. Think of Kubelet as the loyal worker on every node, making sure your containers run exactly as you want them to. Whether you’re managing a handful of pods or thousands, understanding Kubelet is essential.</p><p>In this article, we’ll answer 5 critical questions about Kubelet to help you master node-level operations in Kubernetes.</p><h3>🏗️ Architecture Overview</h3><p>Kubelet sits on every worker node and acts as the bridge between the Kubernetes control plane and the container runtime. It takes orders from the API server and ensures containers start, stay healthy, and restart when needed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/971/0*dwJva4T7zWL7WONG.jpeg" /></figure><h3>1️⃣ What Are Kubelet’s Core Responsibilities?</h3><p>Kubelet is the node-level agent responsible for keeping your pods alive and healthy. Here’s what it does:</p><ul><li>📝 Node Registration: Registers the node with the cluster so pods can be scheduled on it.</li><li>👀 Pod Monitoring: Watches the API server for pod assignments and ensures they’re running.</li><li>🐳 Container Management: Talks to the container runtime (via CRI) to pull images, start, stop, and restart containers.</li><li>💚 Health Checks: Continuously checks pod health and restarts failed containers to maintain uptime.</li></ul><p>💡 Example:<br>You deploy an Nginx web server pod on Node 1. Kubelet pulls the Nginx image, starts the container, and monitors it. If Nginx crashes, Kubelet automatically restarts it — no human intervention needed.</p><h3>2️⃣ How Does Kubelet Know Which Pods to Run?</h3><p>Kubelet doesn’t decide on its own — it follows orders from the control plane. Here’s the flow:</p><ol><li>The kube-scheduler assigns a pod to a specific node.</li><li>Kubelet watches the API server for pod assignments.</li><li>Once assigned, Kubelet pulls the container image, allocates resources (CPU, memory), and starts the pod.</li></ol><p>💡 Example:<br>A PostgreSQL database pod is scheduled on Node 2. Kubelet on Node 2 fetches the assignment from the API server, pulls the PostgreSQL image, and launches the database container.</p><h3>3️⃣ What Happens If Kubelet Goes Down? ⚠️</h3><p>If Kubelet crashes or stops, the node enters a degraded state:</p><ul><li>❌ No new pods can be scheduled on the node.</li><li>❌ Health checks stop — crashed pods won’t auto-restart.</li><li>❌ The API server marks the node as NotReady after a timeout.</li><li>🔄 The scheduler may move critical pods to other healthy nodes.</li></ul><p>💡 Example:<br>Kubelet on Node 3 crashes while running a critical app. The app continues running temporarily, but if it crashes, it won’t restart. Eventually, Kubernetes reschedules the app to another node to ensure availability.</p><h3>4️⃣ How Does Kubelet Interact with CRI and CSI? 🔗</h3><p>Kubelet uses two key interfaces to manage containers and storage:</p><h3>CRI (Container Runtime Interface) 🐋</h3><ul><li>Manages the container lifecycle: pulling images, starting/stopping containers, and enforcing resource limits.</li><li>Works with runtimes like containerd, CRI-O, and Docker.</li></ul><h3>CSI (Container Storage Interface) 💾</h3><ul><li>Manages persistent storage: provisioning, mounting, and unmounting volumes.</li><li>Enables pods to use external storage like AWS EBS, Azure Disk, or NFS.</li></ul><p>💡 Example:<br>A stateful MySQL pod needs persistent storage. Kubelet uses CRI to start the MySQL container and CSI to mount an external volume where the database stores its data.</p><h3>5️⃣ Does Kubelet Run as a DaemonSet? 🤔</h3><p>No! While Kubelet operates on every node (like a DaemonSet), it’s not managed by Kubernetes itself. Instead, Kubelet runs as a system-level service (like systemd on Linux).</p><ul><li>Kubelet: A system service installed directly on the node, independent of Kubernetes’ pod-scheduling system.</li></ul><p>💡 Example:<br>Think of Kubelet like an operating system daemon — it’s installed and managed outside Kubernetes, unlike a DaemonSet which Kubernetes deploys and manages internally.</p><h3>Few Important Kubelet Commands:</h3><h4>Service Management:</h4><ul><li>systemctl status kubelet - Check status</li><li>sudo systemctl start kubelet - Start service</li><li>sudo systemctl stop kubelet - Stop service</li><li>sudo systemctl restart kubelet - Restart service</li><li>sudo systemctl enable kubelet - Enable auto-start</li><li>sudo systemctl disable kubelet - Disable auto-start</li></ul><p><strong>Logs &amp; Troubleshooting:</strong></p><ul><li>journalctl -u kubelet -f - View real-time logs</li><li>journalctl -u kubelet -n 100 - Last 100 log lines</li><li>journalctl -u kubelet -b - Logs since boot</li><li>journalctl -u kubelet --since &quot;1 hour ago&quot; - Logs from last hour</li></ul><p><strong>Configuration &amp; Info:</strong></p><ul><li>kubelet --version - Check version</li><li>cat /var/lib/kubelet/config.yaml - View config</li><li>ps aux |- ps aux | grep kubelet` - Check process</li><li>cat /etc/systemd/system/kubelet.service - View service file</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d9b728252b0d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deployment Strategies in Kubernetes]]></title>
            <link>https://medium.com/@sagarpawadi/deployment-strategies-in-kubernetes-71d576e52b7a?source=rss-67ef7bb588e6------2</link>
            <guid isPermaLink="false">https://medium.com/p/71d576e52b7a</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[interview-questions]]></category>
            <category><![CDATA[implementation]]></category>
            <category><![CDATA[strategy-deployment]]></category>
            <category><![CDATA[realtime]]></category>
            <dc:creator><![CDATA[Sagar Pawadi]]></dc:creator>
            <pubDate>Sat, 04 May 2024 06:53:00 GMT</pubDate>
            <atom:updated>2024-05-04T06:53:00.925Z</atom:updated>
            <content:encoded><![CDATA[<h3>What is Deployment ?</h3><p>Deployment generally refers to the process of releasing a software application or system update from a development environment to a production environment where it can be used by end-users. It involves several interrelated activities that ensure a smooth transition and minimize disruptions.</p><h3>Types Of Deployments:</h3><p>1. Rolling 2. Recreate 3. Blue Green 4. Canary</p><h3>Rolling Deployment Strategy</h3><p>The Deployment primitive employs rolling deployment as the default deployment strategy, also referred to as ramped deployment. It’s called “ramped” because the Deployment gradually transitions replicas from the old version to a new version in batches. The Deployment will automatically create a new ReplicaSet for the desired change after the user updates the Pod template.</p><p>Below diagram shows a snapshot in time during the rollout process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Y81xUwgzqKOP4noW.png" /><figcaption>The rolling deployment strategy</figcaption></figure><p>In this scenario, the user initiated an update of the application version from 1.0.0 to 2.0.0. As a result, the Deployment creates a new ReplicaSet and starts up Pods running the new application version while at the same time scaling down the old version. The Service routes network traffic to either the old or new version of the application.</p><h4>Implementation</h4><p>A Deployment uses the rolling deployment strategy by default. The runtime value for the attribute spec.strategy.type is RollingUpdate. Users can fine-tune this strategy. You can use the attributes spec.strategy.rollingUpdate.maxUnavailable and spec.strategy.rollingUpdate.maxSurge to change the rollout rate. Both attributes can use a fixed integer (e.g. 3), or assign a percentage of the total required number of Pods (e.g. 33%). The default value for maxUnavailable and maxSurge is set to 25%.</p><p>The attribute maxUnavailable specifies the maximum number of Pods that can be unavailable during the update process. For example, if you set the value to 40%, then the old ReplicaSet can scaled down to 60% immediately when the rolling update starts.</p><p>The attribute maxSurge specifies the maximum number of Pods that can be created over the desired number of Pods. For example, if you set the value to 10%, the total number of new and old Pods cannot exceed a total of 110% after the new ReplicaSet has been created.</p><p>Independent of the values assigned to the attributes maxUnavailable and maxSurge, all replicas controlled by the old ReplicaSet will be ramped down to 0 over time until all replicas controlled by the new ReplicaSet equals the value of spec.replicas.</p><p>It’s recommended to define a readiness probe for the Pod template to ensure that a replica is ready to handle incoming requests. The attribute spec.minReadySeconds specifies the number of seconds a replica needs to be available for before it is made available to incoming requests.</p><p>Ex shows the usage of those attributes in the context of a full Deployment YAML manifest stored in the file <em>deployment-rolling-update.yaml</em>.</p><p>A Deployment configured with a rolling update strategy</p><pre>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: web-server<br>spec:<br>  replicas: 4<br>  strategy:<br>    type: RollingUpdate<br>    rollingUpdate:<br>      maxUnavailable: 40%     1<br>      maxSurge: 10%           2<br>  minReadySeconds: 60         3<br>  selector:<br>    matchLabels:<br>      app: httpd<br>  template:<br>    metadata:<br>      labels:<br>        app: httpd<br>    spec:<br>      containers:<br>      - name: httpd<br>        image: httpd:2.4.23-alpine<br>        ports:<br>        - containerPort: 80<br>          protocol: TCP<br>        readinessProbe:     4<br>          httpGet:<br>            path: /<br>            port: 80</pre><ol><li>The percentage of Pods that can be unavailable during the update.</li></ol><p>2. The percentage of Pods that can temporarily exceed the total number of replicas.</p><p>3. The number of seconds for which the readiness probe in a Pod needs to be healthy until the rollout process can continue.</p><p>4. The readiness probe for all replicas referred to by spec.minReadySeconds.</p><p>The combination of assigned values to maxUnavailable and maxSurge will determine the runtime behavior and speed of a rollout. You will want to adjust those parameters to find the most-suitable combination for your application.</p><h4>Use Cases and Tradeoffs</h4><p>The rolling deployment is a fitting deployment strategy for rolling out a new application version with zero downtime. Depending on the number of replicas managed by the Deployment, this process can be relatively slow, as old versions of the applications are ramped down and new versions of the application are ramped up in batches.</p><p>It’s important to mention that this deployment strategy comes with a potential risk. Old and new versions of the application run in parallel. Breaking changes introduced with the new version can lead to unexpected and hard-to-debug errors for consumers if they haven’t adapted their client software to the latest changes yet. It’s a good idea to roll out a new application version in a backward-compatible fashion, for example by using a versioned API, to avoid running into this situation.</p><h3>Recreate Deployment Strategy</h3><p>The recreate deployment strategy will terminate replicas with the old application version at once before creating another ReplicaSet that controls replicas running the new application version.</p><p>Below Fig. illustrates the rollout process while updating the Pod template from application version 1.0.0 to 2.0.0. All replicas of the old ReplicaSet are shut down simultaneously. Afterwards, the replicas controlled by the new ReplicaSet are started. During this process, the Service may not be able to reach any of the replicas which can lead to unnecessary downtime for consumers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*rnewO6h4mygKsVn1.png" /><figcaption>Recreate deployment strategy</figcaption></figure><h4>Implementation</h4><p>To configure the recreate deployment strategy for a Deployment, set the attribute spec.strategy.type to Recreate. Internally, this strategy type will automatically assign the total number of replicas to the attribute maxUnavailable. No other configuration options need to be provided.</p><p>Below manifest shows the Recreate strategy type in the context of a full Deployment YAML manifest stored in the file <em>deployment-recreate.yaml</em>.</p><p>Example 11–2. A Deployment configured with a fixed deployment strategy</p><pre>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: web-server<br>spec:<br>  replicas: 4<br>  strategy:<br>    type: Recreate       1<br>  selector:<br>    matchLabels:<br>      app: httpd<br>  template:<br>    metadata:<br>      labels:<br>        app: httpd<br>    spec:<br>      containers:<br>      - name: httpd<br>        image: httpd:2.4.23-alpine<br>        ports:<br>        - containerPort: 80<br>          protocol: TCP</pre><ol><li>The strategy type for configuring fixed deployment.</li></ol><p>Assigning a readiness probe to containers defined by the Pod template isn’t strictly necessary because all replicas with the old application version will be shut down at once. Nevertheless, it still makes sense to verify that the application is up and running before incoming traffic can reach the container by defining a readiness probe.</p><h4>Use Cases and Tradeoffs</h4><p>The fixed deployment strategy is suitable for situations where application downtime is acceptable. For example, it’s great if you want to roll out a new application to a developer environment for testing purposes. For production environments, this deployment strategy may work if you announce an outage time window to customers.</p><h3>Blue-Green Deployment Strategy</h3><p>The blue-green deployment strategy (sometimes referred to as red-black deployment strategy) figuratively uses blue as a representation of the old application version, and green as a representation of the new application version. Both application versions will be operated at the same time with an equal number of replicas.</p><p>Kubernetes routes traffic to the blue deployment, while the green deployment is rolled out and tested by the development or test team. Traffic will be switched over to the green deployment as soon as it is considered production-ready. At that point, the team managing the application can decommission the blue deployment.</p><p>Below fig. shows two Deployments managing replicas with different application versions. The Services can switch network traffic from the old application version to the new application version by changing the label selection.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*GSk47O2JJYydF32Z.png" /><figcaption>The blue-green deployment strategy</figcaption></figure><h4>Implementation</h4><p>Blue-green deployment is not a built-in strategy you can configure within the Deployment resource. You will have to create a Deployment object for both application versions. The Service routes traffic to replicas managed by either the blue or the green Deployment.</p><p>Below shows a blue Deployment YAML manifest stored in the file <em>deployment-blue.yaml</em> specifying the container image httpd:2.4.23-alpine in the Pod template.</p><p><strong>A blue Deployment</strong></p><pre>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: web-server-blue<br>spec:<br>  replicas: 4<br>  selector:<br>    matchLabels:<br>      type: blue<br>  template:<br>    metadata:<br>      labels:<br>        type: blue                   1<br>    spec:<br>      containers:<br>      - name: httpd<br>        image: httpd:2.4.23-alpine   2<br>        ports:<br>        - containerPort: 80<br>          protocol: TCP</pre><ol><li>Uses the label assignment type: blue to any replica managed by the corresponding ReplicaSet.</li></ol><p>2. The old application version 2.4.23-alpine.</p><p>To set up a green deployment that runs the newer container image httpd:2.4.57-alpine, simply create another Deployment object. Note that the label used for the Pod template is different than for the blue deployment. Below shows the green Deployment definition in the file <em>deployment-green.yaml</em>.</p><p><strong>A green Deployment</strong></p><pre>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: web-server-green<br>spec:<br>  replicas: 4<br>  selector:<br>    matchLabels:<br>      type: green<br>  template:<br>    metadata:<br>      labels:<br>        type: green                  1<br>    spec:<br>      containers:<br>      - name: httpd<br>        image: httpd:2.4.57-alpine   2<br>        ports:<br>        - containerPort: 80<br>          protocol: TCP</pre><ol><li>Uses the label assignment type: green to any replica managed by the corresponding ReplicaSet.</li></ol><p>2. The new application version 2.4.57-alpine.</p><p>As mentioned earlier, the Kubernetes object responsible for routing network traffic to the old or new application version is the Service. Ex shows an example for the Service object.</p><p><strong>The Service routing network traffic to a blue deployment</strong></p><pre>apiVersion: v1<br>kind: Service<br>metadata:<br>  name: web-server<br>spec:<br>  selector:<br>    type: blue   1<br>  ports:<br>    - protocol: TCP<br>      port: 80<br>      targetPort: 80</pre><ol><li>The label selector pointing to replicas managed by the blue deployment.</li></ol><p>The resource declaration currently points to the blue deployment, however, to switch to green, simply change the label selection from type: blue to type: green. At that point, you can delete the blue Deployment.</p><h4>Use Cases and Tradeoffs</h4><p>The blue-green deployment strategy is suitable for deployment scenarios where complex upgrades need to be performed without downtime to consumers. This situation may arise if a rollout requires a data migration or if multiple, dependent software components need to be changed at once. Should a rollback to the old application version be required, a simple change of the label selection in the Service will do.</p><p>On the downside, it’s worth mentioning that you will need more hardware resources than for other deployment strategies. Say you need 5 replicas to run the old application version, then you will need the same amount of resources for the new application version assuming the resource requirements won’t differ.</p><h3>Canary Deployment Strategy</h3><p>The canary deployment strategy is similar to the blue-green deployment, however, you’d only make the new application version available to a subset of consumers. With this approach, you can implement A/B testing of new features or if you need to gather metrics about consumer behavior. Based on the defined set of success criteria, traffic to the new application version can be increased gradually. The end goal is to shut down the old application version completely.</p><p>Below fig. shows how the Service sends traffic to both application versions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*vL43Nh_DYeRnKwjO.png" /><figcaption>The canary deployment strategy</figcaption></figure><p>Deployment 1 controls application version v1.0.0. Deployment 2 controls application version v2.0.0. Deployment 1 and 2 use the same label assignment in their Pod template. The Service selects the label key-value pair(s) defined by both Deployments.</p><h4>Implementation</h4><p>In a Kubernetes cluster, you would represent each application version with the help of a Deployment object. You will want to roll out the new application version with less replicas than the current application version by assigning a smaller value to the attribute spec.replicas.</p><p>The following code snippet shows the truncated definition of the Deployment controlling the old application version.</p><pre>kind: Deployment<br>spec:<br>  replicas: 4<br>  selector:<br>    matchLabels:<br>      app: httpd</pre><p>For the new application version, assign a <strong>smaller amount of replicas</strong>, as shown below.</p><pre>kind: Deployment<br>spec:<br>  replicas: 1<br>  selector:<br>    matchLabels:<br>      app: httpd</pre><p>To ensure that replicas for old and new application versions receive requests from consumers, the assigned Pod template label(s) in both Deployment objects need to be the same. Ensure that the Service selects those label(s), as shown in the truncated Service definition below.</p><pre>kind: Service<br>spec:<br>  selector:<br>    app: httpd   1</pre><ol><li>Selects the label assigned to both Deployments.</li></ol><h4>Use Cases and Tradeoffs</h4><p>Organization typically use the canary deployment strategy to roll out experimental features or changes with a potential impact on system performance. You can evaluate your success criteria while making the new feature available to only a subset of consumers. Implementing a canary deployment usually requires less hardware resources than the blue-green deployment as the number of replicas with the new application version is much lower.</p><h3>Summary</h3><p>A <em>deployment</em> is the process of making a software change available to end users or programs. You need to consider two aspects: the procedure of <em>how</em> to deploy a change and the routing of network traffic to the application. Select an appropriate deployment strategy based on use case, application type, and tradeoffs.</p><p>Kubernetes natively supports two deployment strategies with the Deployment primitive, the rolling deployment and the recreate deployment. The rolling deployment, specified by the RollingUpdate strategy, rolls out a change gradually in badges. The fixed deployment, configured by the Recreate strategy, first shuts down the old application version and then brings up the new application version.</p><p>The blue-green and canary deployment strategies can be set up by creating a second Deployment object that manages the new application version in parallel to the old one. The Service would then route network traffic to replicas of both application versions (blue-green) or transition consumers to the new application version over time (canary).</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=71d576e52b7a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Integrate Jenkins with Azure AD]]></title>
            <link>https://medium.com/@sagarpawadi/integrate-jenkins-with-azure-ad-621daa9c4cb4?source=rss-67ef7bb588e6------2</link>
            <guid isPermaLink="false">https://medium.com/p/621daa9c4cb4</guid>
            <category><![CDATA[ldap]]></category>
            <category><![CDATA[azure-ad]]></category>
            <category><![CDATA[jenkins-integration]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[authentication-solution]]></category>
            <dc:creator><![CDATA[Sagar Pawadi]]></dc:creator>
            <pubDate>Fri, 03 May 2024 10:27:27 GMT</pubDate>
            <atom:updated>2024-05-03T10:31:33.782Z</atom:updated>
            <content:encoded><![CDATA[<h3>Configuring user authentication and permissions</h3><p><strong>Scenario: </strong>Suppose if 100 users wants to access you Jenkins for running the pipeline ? How do you integrate Jenkins with Azure AD for that ?</p><p>Jenkins initiates its own user database out of the box to allow you to create users. However, using Azure AD or LDAP-based authentication to grant people access to Jenkins is a popular choice among organizations.</p><p><strong>The following section will walk you through the process of integrating Jenkins with Azure AD to establish a robust and secure authentication and permissions management system:</strong></p><h3><strong>Registering Jenkins on Azure AD</strong></h3><p>This section provides a step-by-step guide to registering your Jenkins application in Azure AD. This involves navigating to the Azure portal, accessing the <strong>App registrations</strong> section, and creating a new registration. You will learn how to set up a reply URL and generate a client secret, which will be utilized for authentication purposes in Jenkins.</p><p><strong>Follow these steps:</strong></p><ol><li>Navigate to the Azure portal (<a href="http://portal.azure.com/">portal.azure.com</a>) and sign in with your Azure account.</li><li>From the dashboard, click on <strong>Azure Active Directory</strong> and then select <strong>App registrations</strong> from the menu.</li><li>Click on the <strong>New registration</strong> button to create a new app registration. This can be seen in the following screenshot:</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*qAqs0pQxcupmf4IGKFb31Q.png" /><figcaption>Initiating a new app for registration</figcaption></figure><p>4. Provide a name for your Jenkins app using the <strong>Name</strong> field and ensure that <strong>Supported account types</strong> is set to the appropriate option based on yourrequirements (for example, <strong>Accounts in this organizational directory only</strong>).</p><p>5. In the <strong>Redirect URI</strong> section, set the app type to <strong>Web</strong> and set the reply to the URL to <a href="https://%3cyour_jenkins_host%3e/securityRealm/finishLogin">https://&lt;your_jenkins_host&gt;/securityRealm/finishLogin</a>. Make sure you replace <strong>&lt;your_jenkins_host&gt;</strong> with the actual host name of your Jenkins instance.</p><p>6. Click on <strong>Register</strong> to create the app registration:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*0n7-35y7oXFF7aUhWlrf0g.png" /><figcaption>Registering Jenkins on Azure AD</figcaption></figure><p>7. Next, on the <strong>App registration</strong> page, click on <strong>Certificates &amp; secrets</strong>. On the resultant page, under the <strong>Client secrets</strong> section, click on <strong>+ New client secret</strong> to generate a new secret key. On the pop-up page, provide a description and set an expiration period for your new secret key. Finally, click on the <strong>Add</strong> button:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*B_vj-Cw09ZSRqSbvfvvFDA.png" /><figcaption>Creating a client secret</figcaption></figure><p>8. Once the client secret has been generated, copy and securely store the value as it will be used in Jenkins to configure the Azure AD plugin.</p><p>9. Next, from the left-hand side menu, click on <strong>Authentication</strong> and, under <strong>Implicit grant and hybrid flows</strong>, enable the <strong>ID tokens</strong> option:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*xSOonzuQpHa9OkBE4Dvlkw.png" /><figcaption>Selecting ID tokens for implicit and hybrid flows</figcaption></figure><p>10. (Optional) If you want to enable Azure AD group support, go to the <strong>Manifest</strong> tab and change the <strong>groupMembershipClaims</strong> value from <strong>null</strong> to <strong>SecurityGroup</strong>. Then, save your changes:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*4uz0pvYTZSho0X27fThqCA.png" /><figcaption>Enabling Azure AD group support</figcaption></figure><p>11. Next, navigate to the <strong>API permissions</strong> section and click on <strong>+ Add a permission</strong> | <strong>Microsoft Graph</strong> | <strong>Application permissions</strong>. Add permissions such as <strong>User.Read.All</strong>, <strong>Group.Read.All</strong>, and <strong>People.Read.All</strong>, as shown here:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*z5pSpQabEPJJMxpKM7zYAQ.png" /><figcaption>Adding API permissions to User, Group, and People</figcaption></figure><p>12. Next, grant admin consent for these permissions, ensuring Jenkins has the required access to retrieve user and group information. This can be seen here:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*QVUUHYzux2GZTUjFh5y-AQ.png" /><figcaption>Granting API permissions and admin consent for Default Directory</figcaption></figure><p>13. With that, your Jenkins app will be registered in Azure AD.</p><p>14. Before we move on, make note of the <strong>Application (client) ID</strong> and <strong>Directory (tenant) ID</strong> values, as you will need them to configure Jenkins. You will find these details on the <strong>Overview</strong> page of your app, as shown here:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*OOHr42hKEYVvDop-q_STxg.png" /><figcaption>The app’s overview page</figcaption></figure><p>15. Additionally, visit the <strong>Users</strong> section and take note of the <strong>Object ID</strong> value of a user from the Azure AD. We will use this to test the Azure AD configuration inside Jenkins to see whether Jenkins can retrieve user information from Azure AD.</p><p><strong>Installing the Jenkins plugin for Azure AD</strong></p><p>To install the Azure AD Jenkins plugin, you have two options: use the Jenkins dashboard or install it via the <strong>values.yaml</strong> file with the Jenkins Helm chart. Since we already know how to install or upgrade a Jenkins plugin using the <strong>values.yaml</strong> file, let’s explore the alternative process. Follow these steps to install the plugin manually:</p><ol><li>Log in to your Jenkins instance as an administrator and, from the Jenkins dashboard, click on <strong>Manage Jenkins</strong> in the left-hand menu.</li><li>From the options that appear, select <strong>Manage Plugins</strong>. On the resultant page, in the <strong>Available</strong> tab, locate the <strong>Azure AD</strong> plugin by either scrolling through the list or using the search bar:</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*wU0U3Jfi8kw7_zWiSEd2qA.png" /><figcaption>Installing the Jenkins plugin for Azure AD</figcaption></figure><p>3. Check the checkbox next to the <strong>Azure AD</strong> plugin to select it for installation and click on the <strong>Install without restart</strong> button to begin the installation process. Wait for Jenkins to download and install the Azure AD plugin. This may take a moment, depending on your internet connection and the size of the plugin. Once the installation is complete, you will see a notification at the top of the page indicating the success of the installation.</p><p>4. You do not necessarily need to restart Jenkins after installing the Azure AD plugin. The plugin should be ready to use immediately after the installation is complete. However, in some cases, Jenkins may recommend or require a restart to ensure proper functioning or to activate certain plugin features.</p><p><strong>Configuring Azure AD settings in Jenkins</strong></p><p>Moving on to the Jenkins configuration, we will explore the security settings. Here, you will learn how to enable Azure AD authentication, input the necessary credentials, and verify the application’s validity within Jenkins. The following steps will guide you through the process of saving the configuration and testing the login functionality with Azure AD:</p><ol><li>Log in to your Jenkins instance as an administrator and, from the Jenkins dashboard, click on <strong>Manage Jenkins</strong> in the left-hand menu. From the options that appear, select <strong>Configure Global Security</strong>.</li><li>On the <strong>Security</strong> page, scroll down to the <strong>Security Realm</strong> section and select the <strong>Azure Active Directory</strong> option from the drop-down menu.</li><li>Fill in the required Azure AD credential information:</li></ol><ul><li><strong>Client ID</strong>: Enter the application (client) ID of your registered Jenkins app in Azure AD.</li><li><strong>Client Secret</strong>: Enter the client secret key generated during the Azure AD app registration.</li><li><strong>Tenant</strong>: Enter the Azure AD tenant ID:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*a6_y3fssCNxnOrCNS7eIjw.png" /><figcaption>Configuring the Azure AD settings in Jenkins</figcaption></figure><p>4. Now, verify your settings and see whether you can also find an AD user. To do that, add the <strong>Object ID</strong> property of an AD user under the <strong>Test user principal name or object id</strong> field, and click on the <strong>Verify Application</strong> button to ensure that the provided configuration is valid. Jenkins will attempt to authenticate with Azure AD using the provided credentials. If the verification is successful, you will see a message indicating that the application was verified successfully:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*9ccDhRQ9OIXl7viA8V3gRA.png" /><figcaption>Verifying the Azure AD configuration in Jenkins</figcaption></figure><p><strong>Configuring user permissions in Jenkins</strong></p><p>Now, let’s explore the authorization aspect, which involves utilizing the Azure AD matrix-based security option. This option enables fine-grained permissions management based on user and group object IDs. This section demonstrates how to configure authorization rules by searching for users or groups, assigning appropriate permissions, and applying the changes:</p><ol><li>Scroll down to the <strong>Authorization</strong> section on the <strong>Security</strong> page. You’ll see that the default setting is <strong>Logged-in users can do anything</strong>; we don’t want that. Replace it with <strong>Azure Active Directory Matrix-based security</strong>.</li><li>First, grant overall administrative privileges to <strong>Authenticated Users</strong> while ensuring <strong>Anonymous</strong> users have no access to the Jenkins dashboard. This ensures that users who are not logged in will see no content on the Jenkins dashboard. Click the <strong>Save</strong> button to apply the changes:</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*1oqycB9hGpSC2MEdNjsESg.png" /><figcaption>Providing admin privileges to Authenticated Users</figcaption></figure><p>3. Now, log out from Jenkins, and log in using an AD user. Return to the <strong>Authorization</strong> section on the <strong>Security</strong> page.</p><p>4. To add a user or group for authorization, in the search field next to <strong>Azure User/group to add</strong>, enter the display name of the user or group you want to add and click on the <strong>Add</strong> button:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*p8krsY_lip3bK4GRtsczSA.png" /><figcaption>Providing admin privileges to an AD user</figcaption></figure><p>5. In the permission column, provide overall administrative privileges to the logged-in AD user, which is the new admin user on Jenkins, and change the <strong>Authenticated Users</strong> permissions to read overall. If you tilt your head and read the columns of the following matrix, you will notice various privileges segregated based on the Jenkins items. This mechanism allows you to control who can do what. In the preceding figure, you can see I have given zero access to <strong>Anonymous</strong> users, read access to <strong>Authenticated Users</strong>, and admin privileges to myself.</p><p>6. Continue adding users or groups as needed, assigning the desired permissions for each. To remove a user or group, click the <strong>X</strong> icon next to their entry in the list.</p><p>7. Once you have configured the authorization rules, click on the <strong>Save</strong> button to apply the changes.</p><p>After completing these steps, AD users can now log in to Jenkins using their Azure AD credentials, and permissions can be managed based on the Azure AD user and group settings.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=621daa9c4cb4" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>