<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by yuresh Gurung on Medium]]></title>
        <description><![CDATA[Stories by yuresh Gurung on Medium]]></description>
        <link>https://medium.com/@ygce108?source=rss-ea26bead1eaa------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*7Ka84p8ZK569vCag</url>
            <title>Stories by yuresh Gurung on Medium</title>
            <link>https://medium.com/@ygce108?source=rss-ea26bead1eaa------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 03:46:20 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ygce108/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Deploying HashiCorp Vault on Kubernetes with Raft Storage, MetalLB, and Helm]]></title>
            <link>https://medium.com/@ygce108/deploying-hashicorp-vault-on-kubernetes-with-raft-storage-metallb-and-helm-bae9ac8b3312?source=rss-ea26bead1eaa------2</link>
            <guid isPermaLink="false">https://medium.com/p/bae9ac8b3312</guid>
            <category><![CDATA[secrets]]></category>
            <category><![CDATA[k8s]]></category>
            <category><![CDATA[vault]]></category>
            <category><![CDATA[hashicorp-vault]]></category>
            <dc:creator><![CDATA[yuresh Gurung]]></dc:creator>
            <pubDate>Thu, 27 Aug 2026 10:52:43 GMT</pubDate>
            <atom:updated>2026-08-27T10:52:43.177Z</atom:updated>
            <content:encoded><![CDATA[<blockquote><strong><em>A practical step-by-step guide for installing HashiCorp Vault 1.20.4 on Kubernetes without TLS and without persistent PVC storage</em></strong></blockquote><p>HashiCorp Vault is a secrets-management platform used to securely store and control access to sensitive information such as database passwords, API keys, application credentials, certificates, and tokens.</p><p>In this guide, we will deploy <strong>HashiCorp Vault 1.20.4 on Kubernetes using Helm</strong>, configure <strong>Raft Integrated Storage</strong>, expose Vault through <strong>MetalLB</strong>, and use <strong>emptyDir</strong> instead of a PVC.</p><p>This setup is intended primarily for <strong>learning, development, and lab environments</strong>.</p><blockquote><strong><em>Important:</em></strong><em> The exact </em><em>values.yaml used in this guide is intentionally kept as provided. No PVC, TLS, or Vault Injector is added.</em></blockquote><h3>1. Architecture</h3><p>The resulting setup looks like this:</p><pre>                        User / Browser<br>                              |<br>                              | HTTP :8200<br>                              |<br>                              v<br>                     +-------------------+<br>                     |      MetalLB      |<br>                     | LoadBalancer IP   |<br>                     +---------+---------+<br>                               |<br>                               v<br>                     +-------------------+<br>                     |   Vault Service   |<br>                     |    ClusterIP /    |<br>                     |    LoadBalancer   |<br>                     +---------+---------+<br>                               |<br>                               v<br>                     +-------------------+<br>                     |     vault-0       |<br>                     |  Vault 1.20.4     |<br>                     |                   |<br>                     |  Raft Storage     |<br>                     |  /vault/data      |<br>                     |                   |<br>                     |    emptyDir       |<br>                     +-------------------+</pre><p>The Kubernetes components are:</p><pre>Kubernetes<br>│<br>├── Namespace: vault<br>│<br>├── StatefulSet<br>│   └── vault-0<br>│<br>├── Service<br>│   └── vault<br>│<br>├── MetalLB<br>│   └── External LoadBalancer IP<br>│<br>└── Vault<br>    ├── HTTP :8200<br>    ├── Raft :8201<br>    └── /vault/data → emptyDir</pre><h3>2. Important Characteristics of This Installation</h3><p>This installation uses the following design:</p><p>ComponentConfigurationVault version1.20.4InstallationHelmHelm charthashicorp/vaultNamespacevaultStorageRaft Integrated StoragePersistent storageDisabledData volumeemptyDirTLSDisabledVault InjectorDisabledServiceLoadBalancerExternal IPProvided by MetalLBReplicas1CPU request250mCPU limit500mMemory request512MiMemory limit1GiKubernetes service registrationEnabled</p><h3>3. Important HA Clarification</h3><p>The Helm configuration contains:</p><pre>ha:<br>  enabled: true</pre><p>but it also contains:</p><pre>replicas: 1</pre><p>Therefore, <strong>this particular installation is not a 3-node highly available Vault cluster</strong>.</p><p>It is:</p><pre>Vault<br>  |<br>  +-- Raft storage<br>  |<br>  +-- 1 Vault node<br>      └── vault-0</pre><p>The ha.enabled: true setting enables the HA/Raft-related configuration in the chart, but <strong>one replica means there is only one Vault server</strong>.</p><p>For a real HA Raft cluster, you would normally run multiple Vault servers, such as three replicas.</p><p>However, because this guide intentionally uses your exact values.yaml, we will keep:</p><pre>replicas: 1</pre><p>throughout this installation.</p><h3>4. Prerequisites</h3><p>Before installing Vault, make sure you have:</p><ul><li>A working Kubernetes/RKE2 cluster</li><li>kubectl</li><li>Helm 3</li><li>Sufficient Kubernetes permissions</li><li>MetalLB installed and configured</li><li>Network connectivity between Kubernetes nodes</li><li>A Kubernetes namespace available for Vault</li></ul><p>Check Kubernetes:</p><pre>kubectl get nodes</pre><p>Example:</p><pre>NAME   STATUS   ROLES<br>m1     Ready    control-plane<br>w1     Ready    &lt;none&gt;<br>w2     Ready    &lt;none&gt;<br>w3     Ready    &lt;none&gt;</pre><p>Check Helm:</p><pre>helm version</pre><p>Check MetalLB:</p><pre>kubectl get pods -n metallb-system</pre><p>You should see MetalLB components running.</p><h3>5. Create the Vault Namespace</h3><p>Create a dedicated namespace:</p><pre>kubectl create namespace vault</pre><p>Verify:</p><pre>kubectl get namespace vault</pre><p>Expected:</p><pre>NAME     STATUS<br>vault    Active</pre><p>If the namespace already exists, Kubernetes will report:</p><pre>Error from server (AlreadyExists)</pre><p>This is harmless. You can continue.</p><h3>6. Add the HashiCorp Helm Repository</h3><p>Add the official HashiCorp Helm repository:</p><pre>helm repo add hashicorp https://helm.releases.hashicorp.com</pre><p>Expected:</p><pre>&quot;hashicorp&quot; has been added to your repositories</pre><p>Update the repository:</p><pre>helm repo update</pre><p>Expected:</p><pre>...Successfully got an update from the &quot;hashicorp&quot; chart repository<br>Update Complete.</pre><p>Verify:</p><pre>helm search repo hashicorp/vault</pre><h3>7. Create the Vault values.yaml</h3><p>Create your configuration file:</p><pre>cd ~/vault<br>nano values.yaml</pre><p>Use the following configuration.</p><h3>values.yaml</h3><pre>injector:<br>  enabled: false</pre><pre>server:<br>  image:<br>    repository: hashicorp/vault<br>    tag: &quot;1.20.4&quot;<br>    pullPolicy: IfNotPresent</pre><pre>  updateStrategyType: &quot;OnDelete&quot;</pre><pre>  resources:<br>    requests:<br>      cpu: 250m<br>      memory: 512Mi<br>    limits:<br>      cpu: 500m<br>      memory: 1Gi</pre><pre>  ha:<br>    enabled: true<br>    replicas: 1</pre><pre>    raft:<br>      enabled: true</pre><pre>      config: |<br>        ui = true</pre><pre>        listener &quot;tcp&quot; {<br>          address         = &quot;[::]:8200&quot;<br>          cluster_address = &quot;[::]:8201&quot;<br>          tls_disable     = 1<br>        }</pre><pre>        storage &quot;raft&quot; {<br>          path = &quot;/vault/data&quot;<br>        }</pre><pre>        service_registration &quot;kubernetes&quot; {}</pre><pre>  dataStorage:<br>    enabled: false</pre><pre>  auditStorage:<br>    enabled: false</pre><pre>  volumes:<br>    - name: vault-data<br>      emptyDir: {}</pre><pre>  volumeMounts:<br>    - name: vault-data<br>      mountPath: /vault/data</pre><pre>  service:<br>    enabled: true<br>    type: LoadBalancer</pre><p>Save:</p><pre>CTRL + O<br>ENTER<br>CTRL + X</pre><h3>8. Understanding the values.yaml</h3><p>Before installing, it is important to understand what the configuration does.</p><h3>8.1 Disable Vault Injector</h3><pre>injector:<br>  enabled: false</pre><p>The Vault Agent Injector automatically injects secrets into Kubernetes pods.</p><p>In this installation, it is disabled.</p><p>Therefore:</p><pre>Application Pod<br>      |<br>      X<br>Vault Agent Injector</pre><p>is not being used.</p><p>Vault can still be accessed using the Vault CLI/API.</p><h3>9. Vault Image</h3><pre>image:<br>  repository: hashicorp/vault<br>  tag: &quot;1.20.4&quot;<br>  pullPolicy: IfNotPresent</pre><p>This deploys:</p><pre>hashicorp/vault:1.20.4</pre><p>Verify after deployment:</p><pre>kubectl get pod -n vault</pre><h3>10. Resource Configuration</h3><p>The Vault server has:</p><pre>requests:<br>  cpu: 250m<br>  memory: 512Mi</pre><pre>limits:<br>  cpu: 500m<br>  memory: 1Gi</pre><p>Meaning:</p><h3>Requested resources</h3><pre>CPU:    250m<br>Memory: 512Mi</pre><h3>Maximum resources</h3><pre>CPU:    500m<br>Memory: 1Gi</pre><p>These values are suitable for a small learning environment.</p><h3>11. Raft Integrated Storage</h3><p>The important part is:</p><pre>ha:<br>  enabled: true<br>  replicas: 1</pre><pre>  raft:<br>    enabled: true</pre><p>Vault uses:</p><pre>Raft Integrated Storage</pre><p>instead of an external database.</p><p>The Raft data directory is:</p><pre>/vault/data</pre><p>configured here:</p><pre>storage &quot;raft&quot; {<br>  path = &quot;/vault/data&quot;<br>}</pre><h3>12. Why emptyDir Is Required in This Setup</h3><p>Your configuration disables the Helm-created persistent storage:</p><pre>dataStorage:<br>  enabled: false</pre><p>Therefore, Kubernetes does not automatically create a Vault data volume.</p><p>We explicitly create an emptyDir:</p><pre>volumes:<br>  - name: vault-data<br>    emptyDir: {}</pre><p>and mount it:</p><pre>volumeMounts:<br>  - name: vault-data<br>    mountPath: /vault/data</pre><p>The resulting structure is:</p><pre>Vault container<br>     |<br>     v<br>/vault/data<br>     |<br>     v<br>emptyDir volume</pre><p>This is what allows Vault Raft to create files such as:</p><pre>/vault/data/node-id</pre><h3>13. Why You Previously Got the node-id Error</h3><p>Your earlier Vault log showed:</p><pre>Error initializing storage of type raft:<br>error parsing config:<br>open /vault/data/node-id: no such file or directory</pre><p>The root cause was that Vault was configured to use:</p><pre>/vault/data</pre><p>for Raft storage, but /vault/data was <strong>not mounted as a writable volume</strong>.</p><p>You had:</p><pre>dataStorage:<br>  enabled: false</pre><p>but initially did not provide another volume for:</p><pre>/vault/data</pre><p>Therefore:</p><pre>Vault<br>  |<br>  └── Raft<br>       |<br>       └── /vault/data<br>              |<br>              └── missing</pre><p>Vault could not initialize its Raft storage.</p><p>The fix was:</p><pre>volumes:<br>  - name: vault-data<br>    emptyDir: {}</pre><pre>volumeMounts:<br>  - name: vault-data<br>    mountPath: /vault/data</pre><p>Now:</p><pre>Vault<br>  |<br>  └── /vault/data<br>         |<br>         └── emptyDir</pre><p>and Vault can create:</p><pre>node-id<br>raft/</pre><p>and other Raft files.</p><h3>14. TLS Is Disabled</h3><p>The listener contains:</p><pre>listener &quot;tcp&quot; {<br>  address         = &quot;[::]:8200&quot;<br>  cluster_address = &quot;[::]:8201&quot;<br>  tls_disable     = 1<br>}</pre><p>Therefore Vault is using HTTP instead of HTTPS.</p><p>The API endpoint is:</p><pre>http://&lt;vault-ip&gt;:8200</pre><p>The UI is:</p><pre>http://&lt;vault-ip&gt;:8200/ui/</pre><p>The browser will show:</p><pre>Not secure</pre><p>because there is no TLS certificate.</p><p>This is expected for this lab configuration.</p><h3>15. Install Vault with Helm</h3><p>Once values.yaml is ready:</p><pre>helm install vault hashicorp/vault \<br>  -n vault \<br>  -f values.yaml</pre><p>Expected:</p><pre>NAME: vault<br>NAMESPACE: vault<br>STATUS: deployed<br>REVISION: 1</pre><p>Check the Helm release:</p><pre>helm list -n vault</pre><p>Expected:</p><pre>NAME    NAMESPACE    STATUS<br>vault   vault        deployed</pre><h3>16. Verify the Vault Resources</h3><p>Check all resources:</p><pre>kubectl get all -n vault</pre><p>Check pods:</p><pre>kubectl get pods -n vault</pre><p>Expected:</p><pre>NAME      READY   STATUS    RESTARTS   AGE<br>vault-0   0/1     Running   0          ...</pre><p>Initially, however, the pod may show:</p><pre>0/1 Running</pre><p>because Vault has not yet been initialized and unsealed.</p><p>That is normal.</p><h3>17. Check the Vault StatefulSet</h3><pre>kubectl get statefulset -n vault</pre><p>Expected:</p><pre>NAME    READY<br>vault   0/1</pre><p>Check details:</p><pre>kubectl describe statefulset vault -n vault</pre><h3>18. Check the Vault Service</h3><p>Run:</p><pre>kubectl get svc -n vault</pre><p>Because your configuration uses:</p><pre>service:<br>  enabled: true<br>  type: LoadBalancer</pre><p>MetalLB should provide an external IP.</p><p>Example:</p><pre>NAME             TYPE           CLUSTER-IP      EXTERNAL-IP<br>vault            LoadBalancer   10.43.x.x       10.10.10.25<br>vault-active     LoadBalancer   10.43.x.x       10.10.10.23<br>vault-standby    LoadBalancer   10.43.x.x       10.10.10.24<br>vault-internal   ClusterIP      None            &lt;none&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/893/1*26y9822tsQWLfanTTXcTjA.png" /></figure><p>Your actual IP may be different.</p><p>Always check:</p><pre>kubectl get svc -n vault</pre><p>rather than assuming a particular MetalLB address.</p><h3>19. Verify the Vault Logs</h3><p>Run:</p><pre>kubectl logs vault-0 -n vault</pre><p>You should no longer see:</p><pre>open /vault/data/node-id: no such file or directory</pre><p>The Vault server should start successfully.</p><h3>20. Check Vault Status</h3><p>Run:</p><pre>kubectl exec -it vault-0 -n vault -- vault status</pre><p>Before initialization, you will typically see something similar to:</p><pre>Initialized    false<br>Sealed         true<br>Storage Type   raft</pre><p>This means Vault is running but has not yet been initialized.</p><h3>21. Initialize Vault</h3><p>Initialization is performed only once for the new Vault cluster.</p><p>Run:</p><pre>kubectl exec -it vault-0 -n vault -- \<br>  vault operator init \<br>  -key-shares=5 \<br>  -key-threshold=3</pre><p>Vault will generate:</p><pre>Unseal/Recovery Key 1<br>Unseal/Recovery Key 2<br>Unseal/Recovery Key 3<br>Unseal/Recovery Key 4<br>Unseal/Recovery Key 5</pre><pre>Initial Root Token</pre><p>output:(please donot share this other — if you work in production )</p><pre>start@m1:~/vault$ kubectl exec -it vault-0 -n vault -- \<br>  vault operator init \<br>  -key-shares=5 \<br>  -key-threshold=3<br>Unseal Key 1: CbDcvhDatRAuY43dJgti6y1Pq038u9OSP4gi7Q+Eg4Uv<br>Unseal Key 2: rWglDgYuc7Bal6tajLGW4fkRgigZmhv+gKhOgG/jgrhZ<br>Unseal Key 3: pxDT6e/1wmiNJmeSsyqL4NoOu7NFUXp45qVqapkSIMIg<br>Unseal Key 4: A/OHiWfPnTgpzr5uPG1gDYx9gYo7TefWmNwUD8SKRK1m<br>Unseal Key 5: pfy29bN5/9xocpmtIYwcwDyrie9VEHYb9O1TloayaB4R<br><br>Initial Root Token: hvs.etLFasRBvCv1jnDwYL97PiRn<br><br>Vault initialized with 5 key shares and a key threshold of 3. Please securely<br>distribute the key shares printed above. When the Vault is re-sealed,<br>restarted, or stopped, you must supply at least 3 of these keys to unseal it<br>before it can start servicing requests.<br><br>Vault does not store the generated root key. Without at least 3 keys to<br>reconstruct the root key, Vault will remain permanently sealed!<br><br>It is possible to generate new unseal keys, provided you have a quorum of<br>existing unseal keys shares. See &quot;vault operator rekey&quot; for more information.</pre><h3>22. VERY IMPORTANT — Protect the Initialization Output</h3><p>The generated keys are extremely sensitive.</p><p>Do <strong>not</strong> put real values in:</p><ul><li>GitHub repositories</li><li>GitLab repositories</li><li>public blogs</li><li>screenshots</li><li>Slack messages</li><li>documentation</li><li>values.yaml</li></ul><p>Never publish actual keys.</p><h3>23. Understanding the 5/3 Configuration</h3><p>You configured:</p><pre>-key-shares=5<br>-key-threshold=3</pre><p>This means:</p><pre>5 total key shares<br>        |<br>        +-- Key 1<br>        +-- Key 2<br>        +-- Key 3<br>        +-- Key 4<br>        +-- Key 5</pre><pre>At least 3 keys are required<br>to unseal Vault.</pre><p>For example:</p><pre>Key 1 + Key 2 + Key 3<br>        ↓<br>      Unseal</pre><p>or:</p><pre>Key 1 + Key 3 + Key 5<br>        ↓<br>      Unseal</pre><p>You do not need all five keys.</p><p>You need the configured threshold of three.</p><h3>24. Unseal Vault</h3><p>Use three different recovery/unseal keys.</p><pre>kubectl exec -it vault-0 -n vault -- \<br>  vault operator unseal</pre><p>Paste:</p><pre>&lt;UNSEAL_KEY_1&gt;<br><br><br><br>start@m1:~/vaultkubectl exec -it vault-0 -n vault -- vault operator unsealal<br>Unseal Key (will be hidden): <br>Key                     Value<br>---                     -----<br>Seal Type               shamir<br>Initialized             true<br>Sealed                  true<br>Total Shares            5<br>Threshold               3<br>Unseal Progress         1/3<br>Unseal Nonce            8fa99676-1781-b3ba-9423-7cbb771285e5<br>Version                 1.20.4<br>Build Date              2025-09-23T13:22:38Z<br>Storage Type            raft<br>Removed From Cluster    false<br>HA Enabled              true</pre><p>Then:</p><pre>kubectl exec -it vault-0 -n vault -- \<br>  vault operator unseal</pre><p>Paste:</p><pre>&lt;UNSEAL_KEY_2&gt;<br><br><br>start@m1:~/vault$ kubectl exec -it vault-0 -n vault -- vault operator unseal<br>Unseal Key (will be hidden): <br>Key                     Value<br>---                     -----<br>Seal Type               shamir<br>Initialized             true<br>Sealed                  true<br>Total Shares            5<br>Threshold               3<br>Unseal Progress         2/3<br>Unseal Nonce            8fa99676-1781-b3ba-9423-7cbb771285e5<br>Version                 1.20.4<br>Build Date              2025-09-23T13:22:38Z<br>Storage Type            raft<br>Removed From Cluster    false<br>HA Enabled              true</pre><p>Then:</p><pre>kubectl exec -it vault-0 -n vault -- \<br>  vault operator unseal</pre><p>Paste:</p><pre>&lt;UNSEAL_KEY_3&gt;</pre><h3>25. Verify Vault Is Unsealed</h3><p>Run:</p><pre>kubectl exec -it vault-0 -n vault -- vault status</pre><p>You should see:</p><pre>Initialized     true<br>Sealed          false<br>Storage Type    raft<br>HA Enabled      true<br>HA Mode         active</pre><p>The important values are:</p><pre>Initialized     true<br>Sealed          false<br>Storage Type    raft</pre><h3>26. Verify Kubernetes Pod Status</h3><p>Run:</p><pre>kubectl get pods -n vault</pre><p>Expected:</p><pre><br>start@m1:~/vault$ kubectl get pods -n vault<br>NAME      READY   STATUS    RESTARTS   AGE<br>vault-0   1/1     Running   0          39m</pre><p>Now Vault is ready.</p><h3>27. Login to Vault from the CLI</h3><p>First obtain your MetalLB IP:</p><pre>kubectl get svc -n vault</pre><p>Suppose the Vault LoadBalancer IP is:</p><pre>10.10.10.25</pre><p>Set:</p><pre>export VAULT_ADDR=http://10.10.10.25:8200</pre><p>Then:</p><pre>vault login</pre><p>When prompted, enter:</p><pre>Initial Root Token</pre><p>For example:</p><pre>hvs.xxxxxxxxxxxxxxxxx</pre><p>Do <strong>not</strong> use a recovery/unseal key as the login token.</p><h3>28. Recovery Keys and Root Token Are Different</h3><p>This is one of the most important concepts.</p><h3>Recovery/unseal key</h3><p>Used for:</p><pre>Vault unseal operation</pre><p>Example:</p><pre>Recovery Key 1<br>Recovery Key 2<br>Recovery Key 3</pre><h3>Root token</h3><p>Used for:</p><pre>Vault authentication</pre><p>Example:</p><pre>Initial Root Token: hvs.xxxxxxxxx</pre><p>Therefore:</p><pre>Recovery Key<br>     |<br>     └── Unseal Vault</pre><pre>Root Token<br>     |<br>     └── Login/authenticate to Vault</pre><p>Do not enter an unseal key into the Vault UI Token field.</p><h3>29. Verify Root Token</h3><p>After login:</p><pre>vault token lookup</pre><p>You should see policies including:</p><pre>policies<br>--------<br>[&quot;root&quot;]</pre><p>You can also test:</p><pre>vault status</pre><h3>30. Access the Vault UI</h3><p>Find the LoadBalancer address:</p><pre>kubectl get svc -n vault</pre><p>For example:</p><pre>vault   LoadBalancer   10.43.x.x   10.10.10.25   8200:30307/TCP</pre><p>Open:</p><pre>http://10.10.10.25:8200/ui/</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3skHVDSWqi-cEbHksMU_3g.png" /></figure><p>The exact IP depends on your MetalLB configuration.</p><h3>31. Login to the Vault UI</h3><p>On the Vault login page:</p><pre>Method: Token</pre><p>Then enter:</p><pre>Initial Root Token</pre><p>For example:</p><pre>hvs.xxxxxxxxxxxxxxxxx</pre><p>Click:</p><pre>Sign In</pre><p>Do <strong>not</strong> enter:</p><pre>Recovery Key 1</pre><p>or:</p><pre>Recovery Key 2</pre><p>in the Token field.</p><h3>32. Why the Browser Says “Not Secure”</h3><p>Because your configuration contains:</p><pre>tls_disable = 1</pre><p>Vault is running with HTTP:</p><pre>http://10.10.10.25:8200</pre><p>instead of:</p><pre>https://10.10.10.25:8200</pre><p>Therefore Chrome/Firefox will display:</p><pre>Not secure</pre><p>This is expected.</p><p>For a production deployment, TLS should be enabled.</p><h3>33. Test the Vault API</h3><p>From m1:</p><pre>curl http://10.10.10.25:8200/v1/sys/health</pre><p>You should receive a JSON response.</p><p>You can also test:</p><pre>curl http://10.10.10.25:8200/v1/sys/seal-status</pre><h3>34. Check Vault’s Raft Storage</h3><p>Run:</p><pre>kubectl exec -it vault-0 -n vault -- \<br>  vault operator raft list-peers</pre><p>Because you are running:</p><pre>replicas: 1</pre><p>you should have only one Raft server.</p><p>Conceptually:</p><pre>Raft Cluster<br>     |<br>     └── vault-0<br>          └── leader</pre><p>There are no vault-1 or vault-2 pods in this configuration.</p><h3>35. Important: Do Not Run raft join Commands</h3><p>With your current configuration:</p><pre>replicas: 1</pre><p>you should <strong>not</strong> run:</p><pre>vault operator raft join ...</pre><p>for vault-1 and vault-2.</p><p>Those pods do not exist.</p><p>Commands such as:</p><pre>kubectl exec -it vault-1 ...</pre><p>will fail because:</p><pre>vault-1</pre><p>does not exist.</p><h3>36. Why Your Previous Installation Failed</h3><p>Your previous installation had:</p><pre>dataStorage:<br>  enabled: false</pre><p>and:</p><pre>storage &quot;raft&quot; {<br>  path = &quot;/vault/data&quot;<br>}</pre><p>but there was no volume mounted at:</p><pre>/vault/data</pre><p>Vault therefore reported:</p><pre>Error initializing storage of type raft:<br>error parsing config:<br>open /vault/data/node-id: no such file or directory</pre><h3>The problem</h3><pre>Raft configured<br>      |<br>      v<br>/vault/data<br>      |<br>      X<br>No writable volume<br>      |<br>      v<br>Vault crashes</pre><h3>The solution</h3><p>Your current configuration adds:</p><pre>volumes:<br>  - name: vault-data<br>    emptyDir: {}</pre><p>and:</p><pre>volumeMounts:<br>  - name: vault-data<br>    mountPath: /vault/data</pre><p>Now:</p><pre>Raft<br> |<br> v<br>/vault/data<br> |<br> v<br>emptyDir<br> |<br> v<br>Vault starts</pre><h3>37. Verify the emptyDir Mount</h3><p>You can confirm the mount:</p><pre>kubectl describe pod vault-0 -n vault</pre><p>Look for:</p><pre>Mounts:<br>  /vault/data from vault-data</pre><p>and:</p><pre>Volumes:<br>  vault-data:<br>    Type: EmptyDir</pre><p>You can also check from inside the container:</p><pre>kubectl exec -it vault-0 -n vault -- ls -la /vault/data</pre><p>After Vault has initialized, you should see Raft-related data.</p><h3>38. Troubleshooting</h3><h3>Problem 1 — CrashLoopBackOff</h3><p>Check:</p><pre>kubectl get pods -n vault</pre><p>Then:</p><pre>kubectl logs vault-0 -n vault --previous</pre><p>Also:</p><pre>kubectl describe pod vault-0 -n vault</pre><p>Look at:</p><pre>State<br>Last State<br>Exit Code<br>Events</pre><h3>39. Problem 2 — /vault/data/node-id: no such file or directory</h3><p>Error:</p><pre>Error initializing storage of type raft:<br>error parsing config:<br>open /vault/data/node-id: no such file or directory</pre><h3>Cause</h3><p>Raft is configured for:</p><pre>/vault/data</pre><p>but no volume is mounted there.</p><h3>Solution</h3><p>Your current configuration must contain:</p><pre>volumes:<br>  - name: vault-data<br>    emptyDir: {}</pre><p>and:</p><pre>volumeMounts:<br>  - name: vault-data<br>    mountPath: /vault/data</pre><p>Then upgrade:</p><pre>helm upgrade vault hashicorp/vault \<br>  -n vault \<br>  -f values.yaml</pre><p>Delete the pod so StatefulSet recreates it:</p><pre>kubectl delete pod vault-0 -n vault</pre><p>Check:</p><pre>kubectl get pods -n vault -w</pre><h3>40. Problem 3 — Pod Is Running but 0/1</h3><p>Example:</p><pre>vault-0   0/1   Running</pre><p>This can happen because Vault is:</p><pre>sealed</pre><p>or:</p><pre>not initialized</pre><p>Check:</p><pre>kubectl exec -it vault-0 -n vault -- vault status</pre><p>If:</p><pre>Initialized    false</pre><p>initialize:</p><pre>kubectl exec -it vault-0 -n vault -- \<br>  vault operator init \<br>  -key-shares=5 \<br>  -key-threshold=3</pre><p>If:</p><pre>Initialized    true<br>Sealed         true</pre><p>unseal it using three keys.</p><h3>41. Problem 4 — permission denied / invalid token</h3><p>If the UI displays:</p><pre>Authentication failed:<br>permission denied<br>invalid token</pre><p>check that you are using the:</p><pre>Initial Root Token</pre><p>and not:</p><pre>Recovery Key</pre><p>Correct:</p><pre>Method: Token<br>Token: &lt;Initial Root Token&gt;</pre><p>Incorrect:</p><pre>Method: Token<br>Token: &lt;Recovery Key&gt;</pre><h3>42. Problem 5 — Vault UI Cannot Be Opened</h3><p>Check:</p><pre>kubectl get svc -n vault</pre><p>Verify that the service has an external IP:</p><pre>EXTERNAL-IP<br>10.10.10.x</pre><p>Check MetalLB:</p><pre>kubectl get pods -n metallb-system</pre><p>Check service details:</p><pre>kubectl describe svc vault -n vault</pre><p>Test connectivity:</p><pre>curl http://&lt;EXTERNAL-IP&gt;:8200/v1/sys/health</pre><p>For example:</p><pre>curl http://10.10.10.25:8200/v1/sys/health</pre><h3>43. Problem 7 — Helm Release Already Exists</h3><p>If you receive:</p><pre>Error: INSTALLATION FAILED:<br>cannot re-use a name that is still in use</pre><p>Check:</p><pre>helm list -n vault</pre><p>If the release already exists, use:</p><pre>helm upgrade vault hashicorp/vault \<br>  -n vault \<br>  -f values.yaml</pre><p>Do not run helm install again for the existing release.</p><h3>45. Problem 8 — Helm Upgrade Says no deployed releases</h3><p>Check:</p><pre>helm list -A</pre><p>Then:</p><pre>helm status vault -n vault</pre><p>If the old release exists in the wrong namespace, remove that old release and install into the intended namespace.</p><p>For example:</p><pre>helm uninstall vault -n default</pre><p>Then:</p><pre>helm install vault hashicorp/vault \<br>  -n vault \<br>  -f values.yaml</pre><p>Always confirm the namespace before deleting a release.</p><h3>46. Problem 9 — PodSecurity Warning</h3><p>You may see a warning similar to:</p><pre>would violate PodSecurity &quot;restricted:latest&quot;</pre><p>For example:</p><pre>unrestricted capabilities<br>seccompProfile</pre><p>This is a Kubernetes Pod Security warning.</p><p>If the pod is successfully created and running, this warning did not prevent deployment.</p><p>Check:</p><pre>kubectl get pods -n vault</pre><p>If:</p><pre>vault-0   1/1   Running</pre><p>then the pod is running successfully.</p><p>For a production deployment, however, security context and Pod Security settings should be reviewed properly.</p><h3>47. Problem 10 — unknown field ... emptyDir.defaultMode</h3><p>If you see:</p><pre>Warning: unknown field &quot;spec.template.spec.volumes[1].emptyDir.defaultMode&quot;</pre><p>then the volume definition is incorrectly structured.</p><p>For emptyDir, use:</p><pre>volumes:<br>  - name: vault-data<br>    emptyDir: {}</pre><p>Do not put:</p><pre>defaultMode</pre><p>under emptyDir.</p><h3>48. Problem 11 — Vault Is Unsealed but Data Disappears</h3><p>This is expected with:</p><pre>emptyDir: {}</pre><p>The data lifecycle is:</p><pre>Pod<br> |<br> +-- emptyDir<br>       |<br>       +-- Vault Raft data</pre><p>When the pod/container restarts, emptyDir normally survives a container restart while the pod remains, but <strong>it is deleted when the pod is removed/recreated</strong>.</p><p>For example:</p><pre>kubectl delete pod vault-0 -n vault</pre><p>can result in the new pod getting a new empty emptyDir.</p><p>That means your Vault Raft data can be lost.</p><h3>49. This Is the Biggest Limitation of my Current Setup</h3><p>Your current configuration is excellent for learning:</p><pre>Helm<br>+<br>Vault<br>+<br>Raft<br>+<br>Kubernetes<br>+<br>MetalLB<br>+<br>Unseal<br>+<br>Vault UI</pre><p>But it is <strong>not suitable for production secrets storage</strong> because:</p><pre>dataStorage:<br>  enabled: false</pre><p>and:</p><pre>emptyDir: {}</pre><p>mean there is no persistent storage for the Vault data.</p><p>If the Vault pod and its emptyDir are permanently removed, the Raft database is lost.</p><h3>50. What Happens If vault-0 Restarts?</h3><p>There are two different scenarios.</p><h3>Container restart inside the same Pod</h3><p>The emptyDir generally remains available to that Pod.</p><h3>Pod deletion/recreation</h3><p>For example:</p><pre>kubectl delete pod vault-0 -n vault</pre><p>The StatefulSet creates a replacement Pod.</p><p>The new Pod receives a new emptyDir.</p><p>Therefore:</p><pre>Old vault-0<br>     |<br>     └── old emptyDir<br>           |<br>           └── Raft data</pre><pre>Pod deleted<br>     |<br>     X<br>     |<br>New vault-0<br>     |<br>     └── new emptyDir<br>           |<br>           └── empty</pre><p>This can cause Vault to appear uninitialized again.</p><h3>51. Verify the Current Configuration</h3><p>Run:</p><pre>helm get values vault -n vault</pre><p>You should see your configuration.</p><p>You can also inspect the rendered StatefulSet:</p><pre>kubectl get statefulset vault -n vault -o yaml</pre><p>Verify:</p><pre>/vault/data</pre><p>is mounted.</p><h3>52. Useful Operational Commands</h3><h3>Check Vault pods</h3><pre>kubectl get pods -n vault -o wide</pre><h3>Check services</h3><pre>kubectl get svc -n vault</pre><h3>Check StatefulSet</h3><pre>kubectl get statefulset -n vault</pre><h3>Check PVCs</h3><p>Because you intentionally disabled persistent storage:</p><pre>kubectl get pvc -n vault</pre><p>Expected:</p><pre>No resources found in vault namespace.</pre><p>That is expected for this configuration.</p><h3>Check logs</h3><pre>kubectl logs vault-0 -n vault</pre><h3>Follow logs</h3><pre>kubectl logs -f vault-0 -n vault</pre><h3>Check Vault status</h3><pre>kubectl exec -it vault-0 -n vault -- vault status</pre><h3>Check Raft peers</h3><pre>kubectl exec -it vault-0 -n vault -- \<br>  vault operator raft list-peers</pre><h3>53. Complete Installation Flow</h3><p>The entire installation can be summarized as:</p><pre>1. Create namespace<br>        |<br>        v<br>2. Add HashiCorp Helm repository<br>        |<br>        v<br>3. Create values.yaml<br>        |<br>        v<br>4. Configure Vault<br>   ├── Vault 1.20.4<br>   ├── Raft<br>   ├── TLS disabled<br>   ├── emptyDir<br>   ├── no PVC<br>   ├── Injector disabled<br>   └── LoadBalancer<br>        |<br>        v<br>5. helm install<br>        |<br>        v<br>6. vault-0 starts<br>        |<br>        v<br>7. Check Vault status<br>        |<br>        v<br>8. Initialize Vault<br>        |<br>        v<br>9. Receive 5 keys + root token<br>        |<br>        v<br>10. Unseal with 3 keys<br>        |<br>        v<br>11. Vault becomes active<br>        |<br>        v<br>12. Get MetalLB IP<br>        |<br>        v<br>13. Login using root token<br>        |<br>        v<br>14. Access Vault UI</pre><h3>54. Complete Command Reference</h3><p>For convenience, here is the complete basic installation sequence.</p><h3>Create namespace</h3><pre>kubectl create namespace vault</pre><h3>Add Helm repository</h3><pre>helm repo add hashicorp https://helm.releases.hashicorp.com<br>helm repo update</pre><h3>Create configuration</h3><pre>mkdir -p ~/vault<br>cd ~/vault<br>nano values.yaml</pre><p>Use:</p><pre>injector:<br>  enabled: false</pre><pre>server:<br>  image:<br>    repository: hashicorp/vault<br>    tag: &quot;1.20.4&quot;<br>    pullPolicy: IfNotPresent</pre><pre>  updateStrategyType: &quot;OnDelete&quot;</pre><pre>  resources:<br>    requests:<br>      cpu: 250m<br>      memory: 512Mi<br>    limits:<br>      cpu: 500m<br>      memory: 1Gi</pre><pre>  ha:<br>    enabled: true<br>    replicas: 1</pre><pre>    raft:<br>      enabled: true</pre><pre>      config: |<br>        ui = true</pre><pre>        listener &quot;tcp&quot; {<br>          address         = &quot;[::]:8200&quot;<br>          cluster_address = &quot;[::]:8201&quot;<br>          tls_disable     = 1<br>        }</pre><pre>        storage &quot;raft&quot; {<br>          path = &quot;/vault/data&quot;<br>        }</pre><pre>        service_registration &quot;kubernetes&quot; {}</pre><pre>  dataStorage:<br>    enabled: false</pre><pre>  auditStorage:<br>    enabled: false</pre><pre>  volumes:<br>    - name: vault-data<br>      emptyDir: {}</pre><pre>  volumeMounts:<br>    - name: vault-data<br>      mountPath: /vault/data</pre><pre>  service:<br>    enabled: true<br>    type: LoadBalancer</pre><h3>Install</h3><pre>helm install vault hashicorp/vault \<br>  -n vault \<br>  -f values.yaml</pre><h3>Check pod</h3><pre>kubectl get pods -n vault</pre><h3>Check service</h3><pre>kubectl get svc -n vault</pre><h3>Check Vault status</h3><pre>kubectl exec -it vault-0 -n vault -- vault status</pre><h3>Initialize</h3><pre>kubectl exec -it vault-0 -n vault -- \<br>  vault operator init \<br>  -key-shares=5 \<br>  -key-threshold=3</pre><h3>Unseal</h3><pre>kubectl exec -it vault-0 -n vault -- vault operator unseal</pre><p>Run it three times with three different keys.</p><h3>Check status</h3><pre>kubectl exec -it vault-0 -n vault -- vault status</pre><h3>Get MetalLB IP</h3><pre>kubectl get svc -n vault</pre><h3>Configure CLI</h3><pre>export VAULT_ADDR=http://&lt;METALLB-IP&gt;:8200</pre><h3>Login</h3><pre>vault login</pre><p>Use:</p><pre>Initial Root Token</pre><h3>Open UI</h3><pre>http://&lt;METALLB-IP&gt;:8200/ui/</pre><h3>55. Production Recommendations</h3><p>The configuration in this article is intentionally designed around your current lab setup. For a production Vault deployment, I recommend changing several things.</p><h3>55.1 Use Persistent Storage</h3><p>Current:</p><pre>dataStorage:<br>  enabled: false</pre><pre>volumes:<br>  - name: vault-data<br>    emptyDir: {}</pre><p>Production should use durable storage.</p><p>For example:</p><pre>Vault<br> |<br> +-- Raft<br>      |<br>      +-- Persistent Volume<br>           |<br>           +-- NFS / CSI / Cloud Disk</pre><p>The important requirement is that Vault’s Raft data survives pod recreation and node failures.</p><h3>56. Use Multiple Vault Nodes for Real HA</h3><p>Current:</p><pre>replicas: 1</pre><p>This is not a highly available cluster.</p><p>A production architecture should generally use multiple Vault servers, commonly three or more:</p><pre>Vault Raft Cluster</pre><pre>              +-------------------+<br>              |     vault-0       |<br>              |      Leader       |<br>              +---------+---------+<br>                        |<br>              +---------+---------+<br>              |                   |<br>              v                   v<br>       +-------------+     +-------------+<br>       |   vault-1   |     |   vault-2   |<br>       |   Standby   |     |   Standby   |<br>       +-------------+     +-------------+</pre><p>This provides Raft quorum and node failover.</p><h3>57. Enable TLS in Production</h3><p>Current configuration:</p><pre>tls_disable = 1</pre><p>This is suitable for a controlled lab but should not be used for production traffic.</p><p>Production should use:</p><pre>HTTPS<br>   |<br>   v<br>Vault</pre><p>with properly managed certificates.</p><h3>58. Do Not Use the Root Token for Applications</h3><p>The initial root token should be treated as an administrative bootstrap credential.</p><p>Do not configure:</p><pre>React application<br>       |<br>       v<br>Root Token<br>       |<br>       v<br>Vault</pre><p>Instead use:</p><pre>Application<br>     |<br>     v<br>Kubernetes authentication<br>     |<br>     v<br>Vault Role<br>     |<br>     v<br>Vault Policy<br>     |<br>     v<br>Required Secret</pre><p>This follows the principle of least privilege.</p><h3>59. Recommended Production Secret Flow for Your ShopStack</h3><p>For a Kubernetes application such as your React + FastAPI + MySQL application, a better architecture is:</p><pre>Kubernetes<br>                       |<br>                       |<br>                FastAPI Backend<br>                       |<br>                       | Kubernetes Auth<br>                       v<br>                +-------------+<br>                |   Vault     |<br>                |             |<br>                | KV Secrets  |<br>                +------+------+<br>                       |<br>                       v<br>              MySQL credentials</pre><p>For example:</p><pre>Vault<br>└── secret/shopstack<br>    ├── DB_HOST<br>    ├── DB_PORT<br>    ├── DB_NAME<br>    ├── DB_USER<br>    └── DB_PASSWORD</pre><p>The backend receives only the secrets it needs.</p><h3>60. Recommended Future Architecture</h3><p>Once you move from learning to production, I recommend evolving the setup toward:</p><pre>Users<br>                           |<br>                           v<br>                       MetalLB<br>                           |<br>                           v<br>                    Vault Gateway<br>                           |<br>                           v<br>                +----------------------+<br>                |      Vault HA        |<br>                |                      |<br>                |   vault-0            |<br>                |   vault-1             |<br>                |   vault-2             |<br>                |                      |<br>                |   Raft Integrated    |<br>                |   Storage             |<br>                +----------+-----------+<br>                           |<br>                           v<br>                    Persistent Storage</pre><p>With:</p><pre>3 Vault replicas<br>+<br>Raft<br>+<br>Persistent storage<br>+<br>TLS<br>+<br>Kubernetes authentication<br>+<br>least-privilege policies<br>+<br>regular backups<br>+<br>audit logging<br>+<br>monitoring</pre><h3>61. Final Troubleshooting Checklist</h3><p>When Vault does not start, check these in order:</p><h3>1. Pod</h3><pre>kubectl get pods -n vault</pre><h3>2. Logs</h3><pre>kubectl logs vault-0 -n vault --previous</pre><h3>3. Describe</h3><pre>kubectl describe pod vault-0 -n vault</pre><h3>4. Vault status</h3><pre>kubectl exec -it vault-0 -n vault -- vault status</pre><h3>5. Check /vault/data</h3><pre>kubectl exec -it vault-0 -n vault -- ls -la /vault/data</pre><h3>6. Check service</h3><pre>kubectl get svc -n vault</pre><h3>7. Check MetalLB</h3><pre>kubectl get pods -n metallb-system</pre><h3>8. Check Helm</h3><pre>helm status vault -n vault</pre><h3>9. Check configuration</h3><pre>helm get values vault -n vault</pre><h3>10. Check Raft</h3><pre>kubectl exec -it vault-0 -n vault -- \<br>  vault operator raft list-peers</pre><h3>62. Final Summary</h3><p>This guide deploys HashiCorp Vault with the exact lab configuration:</p><pre>Vault 1.20.4<br>      |<br>      v<br>Helm<br>      |<br>      v<br>Kubernetes<br>      |<br>      +----------------------+<br>      |                      |<br>      v                      v<br>MetalLB                  vault-0<br>External IP              Vault Server<br>                         |<br>                         v<br>                    Raft Storage<br>                         |<br>                         v<br>                      /vault/data<br>                         |<br>                         v<br>                      emptyDir</pre><p>The most important point from the troubleshooting process was the missing Raft data directory.</p><p>Initially:</p><pre>storage &quot;raft&quot;<br>      |<br>      v<br>/vault/data<br>      |<br>      X<br>No volume mounted<br>      |<br>      v<br>CrashLoopBackOff</pre><p>After adding your emptyDir volume:</p><pre>storage &quot;raft&quot;<br>      |<br>      v<br>/vault/data<br>      |<br>      v<br>emptyDir<br>      |<br>      v<br>Vault starts successfully</pre><p>Your current installation is therefore a <strong>single-node Raft Vault lab deployment exposed through MetalLB without TLS and without PVC storage</strong>.</p><p>For learning Vault concepts, this is a good starting point. For production, the next improvements should be <strong>persistent storage, multiple Vault replicas for real HA, TLS, Kubernetes authentication, least-privilege policies, backups, and audit logging</strong>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bae9ac8b3312" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Argo CD Installation Methods]]></title>
            <link>https://medium.com/@ygce108/argo-cd-installation-methods-76cae9d52944?source=rss-ea26bead1eaa------2</link>
            <guid isPermaLink="false">https://medium.com/p/76cae9d52944</guid>
            <category><![CDATA[gitops-with-argocd]]></category>
            <category><![CDATA[helm-argocd]]></category>
            <category><![CDATA[kubernetes-argocd]]></category>
            <category><![CDATA[argo-cd]]></category>
            <dc:creator><![CDATA[yuresh Gurung]]></dc:creator>
            <pubDate>Mon, 17 Aug 2026 10:07:43 GMT</pubDate>
            <atom:updated>2026-08-17T10:08:39.704Z</atom:updated>
            <content:encoded><![CDATA[<p>Argo CD can be installed in several ways. The right method depends on <strong>simplicity, customization, lifecycle management, and environment size</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/837/1*QnbR9_u11Ciu5jx7Jiurgg.png" /></figure><h3>1. Kubernetes Manifest</h3><pre>kubectl create namespace argocd</pre><pre>kubectl apply -n argocd \<br>  -f <a href="https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml">https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml</a></pre><p><strong>Use when:</strong> learning Argo CD or needing a quick, simple installation.</p><p><strong>Pros:</strong> Simple, no Helm required, easy to understand.<br><strong>Cons:</strong> Customization and lifecycle management can be less convenient.</p><h3>2. Helm ⭐ Recommended</h3><pre>helm repo add argo https://argoproj.github.io/argo-helm<br>helm repo update</pre><pre>kubectl create namespace argocd</pre><pre>helm install argocd argo/argo-cd \<br>  -n argocd</pre><p>Custom configuration:</p><pre>helm install argocd argo/argo-cd \<br>  -n argocd \<br>  -f values.yaml</pre><p>Upgrade:</p><pre>helm upgrade argocd argo/argo-cd \<br>  -n argocd \<br>  -f values.yaml</pre><p>Rollback:</p><pre>helm rollback argocd 1 -n argocd</pre><p><strong>Use when:</strong> you need customization, repeatable deployments, upgrades, or production-style management.</p><p><strong>Pros:</strong> Flexible, versioned, configurable, upgrade/rollback support.<br><strong>Cons:</strong> Requires Helm knowledge.</p><h3>3. Kustomize</h3><p>Kustomize uses a <strong>base + overlays</strong> approach:</p><pre>base/<br>  ↓<br> ┌───────────────┐<br> ↓               ↓<br>dev            production</pre><p>Apply:</p><pre>kubectl apply -k overlays/dev/</pre><p><strong>Use when:</strong> managing different environments through Git and customizing Kubernetes YAML without Helm.</p><p><strong>Pros:</strong> GitOps-friendly, clean environment customization, built into kubectl.<br><strong>Cons:</strong> Complex overlays can become difficult to maintain.</p><h3>4. Argo CD Operator</h3><p>The Operator uses Kubernetes Custom Resources to manage Argo CD:</p><pre>ArgoCD Custom Resource<br>          ↓<br>    Argo CD Operator<br>          ↓<br>      Argo CD</pre><p><strong>Use when:</strong> an organization wants Kubernetes-native lifecycle management and already uses Operators.</p><p><strong>Pros:</strong> Declarative, automated lifecycle management, Kubernetes-native.<br><strong>Cons:</strong> More components and complexity; generally unnecessary for beginners.</p><h3>Which Method Should You Choose?</h3><h3>Learning</h3><p><strong>Manifest → Helm → Kustomize → Operator</strong></p><h3>Production</h3><p><strong>Helm</strong> is a strong general choice when you want configurable and repeatable installations.</p><h3>GitOps-heavy environments</h3><p><strong>Kustomize or Helm</strong> are commonly useful because configuration can be maintained in Git.</p><h3>Simple lab</h3><p><strong>Manifest</strong> is the easiest.</p><h3>Enterprise/operator-based platform</h3><p><strong>Operator</strong> can be appropriate.</p><blockquote><strong><em>Key point:</em></strong><em> These are different </em><strong><em>installation methods</em></strong><em>, not different types of Argo CD. They ultimately install and manage the same Argo CD platform.</em></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=76cae9d52944" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Best Linux Books for Every Stage of Your Career (2026 Edition)]]></title>
            <link>https://medium.com/@ygce108/the-best-linux-books-for-every-stage-of-your-career-2026-edition-643c512cd97c?source=rss-ea26bead1eaa------2</link>
            <guid isPermaLink="false">https://medium.com/p/643c512cd97c</guid>
            <category><![CDATA[linux]]></category>
            <category><![CDATA[linux-book]]></category>
            <category><![CDATA[linux-roadmap]]></category>
            <dc:creator><![CDATA[yuresh Gurung]]></dc:creator>
            <pubDate>Thu, 06 Aug 2026 11:17:39 GMT</pubDate>
            <atom:updated>2026-08-06T11:19:32.127Z</atom:updated>
            <content:encoded><![CDATA[<p>Linux is the foundation of modern cloud computing, DevOps, Kubernetes, cybersecurity, and Site Reliability Engineering (SRE). Whether you’re deploying containers, automating infrastructure, or managing production systems, a strong understanding of Linux is one of the most valuable technical skills you can develop.</p><p>Over the years, I’ve explored many Linux learning resources. While there are countless books available, only a handful consistently stand out for their depth, clarity, and long-term value. This guide organizes the best Linux books by skill level so you can build expertise in a structured way.</p><h3>🟢 Beginner: Build a Strong Foundation</h3><h3>1. How Linux Works (3rd Edition) — Brian Ward</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/484/0*fNeY-yRAi6Tsb-5P" /></figure><p><strong>Best for:</strong> Beginners and aspiring DevOps engineers</p><p>This is one of the best books for understanding <em>why</em> Linux behaves the way it does, rather than simply memorizing commands. It explains core concepts such as the boot process, processes, memory management, filesystems, networking, and systemd in a clear and approachable way.</p><p><strong>You’ll learn:</strong></p><ul><li>Linux architecture</li><li>The boot process</li><li>Processes and memory</li><li>Filesystems</li><li>Networking fundamentals</li><li>Shell basics</li></ul><p><strong>Why read it?</strong><br>If you’re starting your Linux journey, this book provides the conceptual understanding that will make everything else easier.</p><p><strong>Rating:</strong> ⭐⭐⭐⭐⭐ (5/5)</p><h3>2. The Linux Command Line (2nd Edition) — William Shotts</h3><p><strong>Best for:</strong> Learning the command line and Bash</p><p>The command line is where Linux truly shines. This book focuses on practical skills, including navigation, text processing, shell scripting, permissions, and automation.</p><p><strong>You’ll learn:</strong></p><ul><li>Bash commands</li><li>Shell scripting</li><li>File management</li><li>Permissions</li><li>Text processing</li></ul><p><strong>Rating:</strong> ⭐⭐⭐⭐⭐ (5/5)</p><h3>🟡 Intermediate: Become a Linux Administrator</h3><h3>3. Linux Bible (10th Edition) — Christopher Negus</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/718/0*koNhU_JLOI5XjEA5" /></figure><p><strong>Best for:</strong> Intermediate learners</p><p>This comprehensive guide covers Linux administration in depth, including networking, storage, security, virtualization, and server management.</p><p><strong>Highlights:</strong></p><ul><li>System administration</li><li>Networking</li><li>Security</li><li>Storage management</li><li>Containers and virtualization</li></ul><p><strong>Rating:</strong> ⭐⭐⭐⭐⭐ (5/5)</p><h3>4. UNIX and Linux System Administration Handbook (5th Edition)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ob3enJaXPD6H5EcB" /></figure><p><strong>Best for:</strong> System administrators, DevOps engineers, and SREs</p><p>Often referred to as the “System Administrator’s Bible,” this book is widely respected in the industry. It covers real-world production topics, from automation and networking to troubleshooting and security.</p><p><strong>Topics include:</strong></p><ul><li>systemd</li><li>SSH</li><li>DNS</li><li>Storage</li><li>Monitoring</li><li>Security</li><li>Automation</li><li>Production troubleshooting</li></ul><p><strong>Rating:</strong> ⭐⭐⭐⭐⭐ (5/5)</p><h3>🔴 Advanced: Understand Linux Internals</h3><h3>5. The Linux Programming Interface — Michael Kerrisk</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*sLxus5MGirv8vli9" /></figure><p><strong>Best for:</strong> Advanced Linux users</p><p>This book explores Linux at the system-call level, explaining how applications interact with the operating system.</p><p><strong>You’ll learn:</strong></p><ul><li>System calls</li><li>Processes</li><li>Threads</li><li>Signals</li><li>IPC</li><li>Sockets</li><li>Filesystems</li></ul><p><strong>Rating:</strong> ⭐⭐⭐⭐⭐ (5/5)</p><h3>6. Linux Kernel Development — Robert Love</h3><p>This book explains the architecture of the Linux kernel, including scheduling, interrupts, memory management, and device drivers.</p><p>It is an excellent choice for readers who want to understand what happens beneath user space.</p><p><strong>Rating:</strong> ⭐⭐⭐⭐⭐ (5/5)</p><h3>⭐ Expert: Performance and Observability</h3><h3>7. Systems Performance (2nd Edition) — Brendan Gregg</h3><p>Performance optimization is a critical skill for engineers operating production environments.</p><p>This book covers:</p><ul><li>CPU analysis</li><li>Memory optimization</li><li>Disk performance</li><li>Networking</li><li>Performance tuning</li><li>Production troubleshooting</li></ul><p>It is considered one of the definitive references on Linux performance engineering.</p><p><strong>Rating:</strong> ⭐⭐⭐⭐⭐ (5/5)</p><h3>8. BPF Performance Tools — Brendan Gregg</h3><p>Modern Linux observability increasingly relies on eBPF. This book demonstrates how to use BPF tools to diagnose performance issues in real systems without modifying applications.</p><p>If you’re interested in SRE, platform engineering, or production Kubernetes clusters, it’s an excellent next step.</p><p><strong>Rating:</strong> ⭐⭐⭐⭐⭐ (5/5)</p><h3>Recommended Learning Path</h3><ol><li>How Linux Works</li><li>The Linux Command Line</li><li>Linux Bible</li><li>UNIX and Linux System Administration Handbook</li><li>Systems Performance</li><li>The Linux Programming Interface</li><li>BPF Performance Tools</li><li>Linux Kernel Development</li></ol><p>Following this sequence will take you from Linux fundamentals to advanced administration, performance analysis, and kernel-level understanding.</p><h3>Final Thoughts</h3><p>Linux is more than an operating system — it’s the backbone of modern infrastructure. Investing time in the right books can dramatically improve your understanding of cloud platforms, containers, Kubernetes, DevOps workflows, and production operations.</p><p>If I had to recommend just four books for a long-term professional library, they would be:</p><ul><li><strong>How Linux Works</strong> — Best for fundamentals</li><li><strong>UNIX and Linux System Administration Handbook</strong> — Best for administration</li><li><strong>The Linux Programming Interface</strong> — Best for Linux internals</li><li><strong>Systems Performance</strong> — Best for production troubleshooting</li></ul><p>Master these books, combine them with hands-on practice in a Linux lab or Kubernetes cluster, and you’ll develop a strong foundation for a successful career in DevOps, SRE, or platform engineering.</p><p>Happy learning!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=643c512cd97c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Structure Large Ansible Projects Like a DevOps Engineer]]></title>
            <link>https://medium.com/@ygce108/how-to-structure-large-ansible-projects-like-a-devops-engineer-e1d7d247ad88?source=rss-ea26bead1eaa------2</link>
            <guid isPermaLink="false">https://medium.com/p/e1d7d247ad88</guid>
            <category><![CDATA[ansiblestructure]]></category>
            <category><![CDATA[ansible-playbook]]></category>
            <category><![CDATA[ansible]]></category>
            <dc:creator><![CDATA[yuresh Gurung]]></dc:creator>
            <pubDate>Thu, 30 Jul 2026 04:20:55 GMT</pubDate>
            <atom:updated>2026-07-30T04:20:55.016Z</atom:updated>
            <content:encoded><![CDATA[<p>When you first start learning Ansible, most tutorials demonstrate a simple project structure with a single role. While this approach is excellent for understanding the fundamentals, it quickly becomes difficult to manage as your infrastructure grows.</p><p>In production environments, infrastructure often consists of dozens of services such as Docker, Kubernetes, NGINX, Prometheus, Grafana, Istio, Harbor, databases, and custom applications. Organising all of these in a single playbook or role leads to duplication, poor maintainability, and increased complexity.</p><p>This article explains how to structure Ansible projects professionally so they remain modular, reusable, and easy to maintain.</p><h3>Why Project Structure Matters</h3><p>As your infrastructure expands, you may encounter challenges such as:</p><ul><li>Hundreds of Ansible tasks in one playbook</li><li>Repeated variables across projects</li><li>Difficulty reusing automation</li><li>Multiple engineers modifying the same files</li><li>Challenging debugging and maintenance</li></ul><p>A well-designed project structure solves these issues by separating responsibilities.</p><h3>A Basic Ansible Project</h3><p>Many beginners start with a structure similar to this:</p><pre>my-ansible-project/<br>├── ansible.cfg<br>├── inventory/<br>│   └── hosts<br>├── playbooks/<br>│   └── site.yml<br>├── roles/<br>│   └── nginx/<br>│       ├── tasks/<br>│       ├── handlers/<br>│       ├── templates/<br>│       ├── files/<br>│       ├── vars/<br>│       ├── defaults/<br>│       └── meta/</pre><p>This is perfect for learning, but once your environment includes multiple services, a single role is no longer sufficient.</p><h3>Think in Components, Not Servers</h3><p>A common mistake is creating one massive role that installs everything.</p><p>Instead, treat each infrastructure component as its own reusable building block.</p><p>Examples include:</p><ul><li>Docker</li><li>Kubernetes (RKE2)</li><li>Helm</li><li>MetalLB</li><li>Cert-Manager</li><li>Istio</li><li>Prometheus</li><li>Grafana</li><li>Loki</li><li>Tempo</li><li>Alloy</li><li>NGINX</li><li>Harbour</li><li>GitLab</li><li>Redis</li><li>PostgreSQL</li><li>Custom Applications</li></ul><p>Each component should have its own role.</p><h3>Professional Project Structure</h3><p>A scalable project may look like this:</p><pre>my-devops-project/<br>│<br>├── ansible.cfg<br>│<br>├── inventory/<br>│   ├── production/<br>│   ├── staging/<br>│   └── development/<br>│<br>├── playbooks/<br>│   ├── 01_prepare_servers.yml<br>│   ├── 02_install_docker.yml<br>│   ├── 03_install_rke2.yml<br>│   ├── 04_install_metallb.yml<br>│   ├── 05_install_cert_manager.yml<br>│   ├── 06_install_istio.yml<br>│   ├── 07_install_monitoring.yml<br>│   ├── 08_install_logging.yml<br>│   ├── 09_deploy_applications.yml<br>│   └── site.yml<br>│<br>├── roles/<br>│   ├── common/<br>│   ├── docker/<br>│   ├── helm/<br>│   ├── rke2/<br>│   ├── metallb/<br>│   ├── cert_manager/<br>│   ├── istio/<br>│   ├── prometheus/<br>│   ├── grafana/<br>│   ├── loki/<br>│   ├── tempo/<br>│   ├── alloy/<br>│   ├── ingress_nginx/<br>│   ├── harbor/<br>│   ├── gitlab/<br>│   ├── nfs_server/<br>│   ├── nfs_csi/<br>│   ├── app1/<br>│   ├── app2/<br>│   └── app3/<br>│<br>├── templates/<br>├── files/<br>└── vars/</pre><p>This structure scales well from small home labs to enterprise environments.</p><h3>Each Role Has One Responsibility</h3><p>Every role should focus on a single task.</p><p>Example:</p><pre>roles/<br>└── grafana/<br>    ├── defaults/<br>    ├── files/<br>    ├── handlers/<br>    ├── meta/<br>    ├── tasks/<br>    ├── templates/<br>    └── vars/</pre><p>This follows the Single Responsibility Principle.</p><p>Benefits include:</p><ul><li>Easier testing</li><li>Better reusability</li><li>Cleaner code reviews</li><li>Independent versioning</li><li>Faster troubleshooting</li></ul><h3>Use Playbooks as Orchestrators</h3><p>Instead of placing every task in one playbook, organise playbooks by deployment stage.</p><p>Example:</p><pre>playbooks/<br>├── 01_prepare_servers.yml<br>├── 02_install_rke2.yml<br>├── 03_install_monitoring.yml<br>├── 04_install_logging.yml<br>└── 05_deploy_apps.yml</pre><p>Each playbook simply calls the required roles.</p><p>Example:</p><pre>---<br>- name: Install Monitoring Stack<br>  hosts: masters<br>  become: true</pre><pre>  roles:<br>    - prometheus<br>    - grafana<br>    - loki<br>    - tempo<br>    - alloy</pre><p>The playbook becomes easy to read because the implementation details remain inside the roles.</p><h3>Use site.yml as the Entry Point</h3><p>The site.yml file acts as the main orchestrator.</p><pre>---<br>- import_playbook: 01_prepare_servers.yml<br>- import_playbook: 02_install_rke2.yml<br>- import_playbook: 03_install_monitoring.yml<br>- import_playbook: 04_install_logging.yml<br>- import_playbook: 05_deploy_apps.yml</pre><p>Now the entire infrastructure can be deployed with a single command:</p><pre>ansible-playbook playbooks/site.yml</pre><h3>Separate Environments</h3><p>Production, staging, and development environments should have separate inventories.</p><p>Example:</p><pre>inventory/<br>├── production/<br>├── staging/<br>└── development/</pre><p>This enables the same automation to be reused across environments by changing only the inventory.</p><h3>Store Variables Properly</h3><p>Avoid hardcoding configuration inside tasks.</p><p>A recommended layout is:</p><pre>inventory/<br>├── group_vars/<br>│   ├── all.yml<br>│   ├── masters.yml<br>│   └── workers.yml<br>│<br>└── host_vars/<br>    ├── master01.yml<br>    └── worker01.yml</pre><p>This keeps infrastructure configuration separate from automation logic.</p><h3>Benefits of a Modular Structure</h3><p>A modular Ansible project provides several advantages:</p><ul><li>Reusable roles</li><li>Smaller playbooks</li><li>Easier debugging</li><li>Simpler maintenance</li><li>Better collaboration</li><li>Consistent deployments</li><li>Faster onboarding for new engineers</li><li>Clear separation of responsibilities</li><li>Easier CI/CD integration</li><li>Improved scalability</li></ul><h3>Common Mistakes</h3><p>Avoid these common pitfalls:</p><p>❌ One role that installs everything</p><p>❌ Thousands of lines in a single playbook</p><p>❌ Hardcoded IP addresses</p><p>❌ Duplicate variables</p><p>❌ Copy-and-paste automation</p><p>❌ No separation between environments</p><h3>Best Practices</h3><ul><li>Design one role per component.</li><li>Keep playbooks focused on orchestration.</li><li>Store variables outside task files.</li><li>Use templates for configuration.</li><li>Keep roles independent and reusable.</li><li>Separate inventories by environment.</li><li>Make deployments idempotent.</li><li>Test changes in development before production.</li><li>Use version control for all automation.</li><li>Document each role with a README.</li></ul><h3>Final Thoughts</h3><p>Professional Ansible projects are built around modularity. Instead of creating one large playbook that performs every task, divide your infrastructure into reusable roles and orchestrate them with dedicated playbooks.</p><p>This approach scales from personal labs to enterprise platforms, making automation easier to maintain, test, and extend as your infrastructure grows.</p><p>Whether you’re deploying Kubernetes clusters, monitoring stacks, service meshes, storage systems, or business applications, a well-structured Ansible project will save time and reduce operational complexity.</p><p>Investing in a clean project structure early pays dividends as your automation evolves.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e1d7d247ad88" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why Your /etc/hosts Changes Disappear After Reboot in Linux VMs (Proxmox VM)]]></title>
            <link>https://medium.com/@ygce108/why-your-etc-hosts-changes-disappear-after-reboot-in-linux-cloud-vms-proxmox-vm-f8f13c9ba51f?source=rss-ea26bead1eaa------2</link>
            <guid isPermaLink="false">https://medium.com/p/f8f13c9ba51f</guid>
            <dc:creator><![CDATA[yuresh Gurung]]></dc:creator>
            <pubDate>Wed, 08 Jul 2026 12:17:52 GMT</pubDate>
            <atom:updated>2026-07-08T12:18:38.924Z</atom:updated>
            <content:encoded><![CDATA[<h3>1. The Original Problem</h3><p>You manually added Kubernetes host entries:</p><pre>10.10.10.27 master-hardway.kubernetes.local master-hardway<br>10.10.10.28 worker1-hardway.kubernetes.local worker1-hardway<br>10.10.10.29 worker2-hardway.kubernetes.local worker2-hardway</pre><p>into:</p><pre>/etc/hosts</pre><p>Before reboot:</p><pre>cat /etc/hosts</pre><p>showed the Kubernetes entries.</p><p>After reboot:</p><pre>cat /etc/hosts</pre><p>the entries were gone.</p><h3>2. Why Did This Happen?</h3><p>The reason is <strong>cloud-init</strong>.</p><p>Your VM is created using a <strong>Proxmox Cloud-Init image</strong>.</p><p>Cloud-init automatically configures the VM during boot.</p><p>It manages files like:</p><ul><li>/etc/hostname</li><li>/etc/hosts</li><li>network configuration</li><li>users</li><li>SSH keys</li></ul><p>Your /etc/hosts file contained this message:</p><pre># Your system has configured &#39;manage_etc_hosts&#39; as True.</pre><p>This is the important part.</p><p>It means:</p><blockquote><em>Cloud-init is responsible for generating </em><em>/etc/hosts.</em></blockquote><p>So the boot process was:</p><pre>VM starts<br>     |<br>     |<br>     v<br>Cloud-init starts<br>     |<br>     |<br>     v<br>Reads its template/configuration<br>     |<br>     |<br>     v<br>Generates /etc/hosts<br>     |<br>     |<br>     v<br>Your manual Kubernetes entries are removed</pre><h3>3. What is Cloud-init?</h3><h3>Simple definition</h3><p><strong>Cloud-init is an automatic VM initialization system.</strong></p><p>It prepares a new virtual machine automatically.</p><p>Example:</p><p>Without cloud-init:</p><pre>Create VM<br>   |<br>   |<br>Login manually<br>   |<br>   |<br>Set hostname<br>Configure network<br>Create users<br>Add SSH keys<br>Edit files</pre><p>With cloud-init:</p><pre>Create VM<br>   |<br>   |<br>Cloud-init runs automatically<br>   |<br>   |<br>Set hostname<br>Configure network<br>Create users<br>Add SSH keys<br>Modify /etc/hosts</pre><h3>4. How Cloud-init Works in Proxmox</h3><p>Your Proxmox flow is usually:</p><pre>Proxmox<br>   |<br>   |<br>   +---- Ubuntu Cloud Image<br>              |<br>              |<br>              v<br>          VM boots<br>              |<br>              |<br>              v<br>        Cloud-init starts<br>              |<br>              |<br>              +---- Metadata<br>              |<br>              +---- User-data<br>              |<br>              +---- Network config<br>              |<br>              |<br>              v<br>        Configure Linux VM</pre><p>Proxmox provides information to cloud-init:</p><p>Example:</p><pre>Hostname:<br>master-hardway</pre><pre>IP:<br>10.10.10.27</pre><pre>User:<br>master</pre><p>Cloud-init applies these settings.</p><h3>5. Why /etc/cloud/templates/hosts.debian.tmpl Exists</h3><p>Cloud-init does not directly store the final /etc/hosts.</p><p>It uses a template:</p><pre>/etc/cloud/templates/hosts.debian.tmpl</pre><p>Template:</p><pre>|<br>        |<br> vCloud-init processes template<br>        |<br>        |<br>        v<br>Creates:<br>/etc/hosts<br></pre><p>Your original template:</p><pre>127.0.1.1 {{fqdn}} {{hostname}}<br>127.0.0.1 localhost</pre><p>only created:</p><pre>127.0.1.1 master-hardway master-hardway<br>127.0.0.1 localhost</pre><p>It did not know about Kubernetes machines.</p><h3>6. The Solution You Applied</h3><p>You edited:</p><pre>sudo nano /etc/cloud/templates/hosts.debian.tmpl</pre><p>and added:</p><pre>10.10.10.27 master-hardway.kubernetes.local master-hardway<br>10.10.10.28 worker1-hardway.kubernetes.local worker1-hardway<br>10.10.10.29 worker2-hardway.kubernetes.local worker2-hardway</pre><p>Now the template contains Kubernetes hostname mappings.</p><h3>7. Applying the Template</h3><p>You ran:</p><pre>sudo cloud-init single --name update_etc_hosts</pre><p>This forced cloud-init to regenerate /etc/hosts.</p><p>Before:</p><pre>/etc/cloud/templates/hosts.debian.tmpl<br>|<br>        |<br>        v<br>/etc/hosts</pre><p>After:</p><pre>Template<br>   |<br>   |<br>   v<br>/etc/hosts</pre><pre>contains Kubernetes entries</pre><h3>8. Verification</h3><p>Your current /etc/hosts:</p><pre>127.0.1.1 master-hardway master-hardway<br>127.0.0.1 localhost<br></pre><pre>10.10.10.27 master-hardway.kubernetes.local master-hardway<br>10.10.10.28 worker1-hardway.kubernetes.local worker1-hardway<br>10.10.10.29 worker2-hardway.kubernetes.local worker2-hardway</pre><p>This means:</p><p>HostnameIPmaster-hardway10.10.10.27worker1-hardway10.10.10.28worker2-hardway10.10.10.29</p><p>Now Linux can resolve:</p><pre>ping worker1-hardway</pre><p>into:</p><pre>10.10.10.28</pre><h3>9. Why Kubernetes Needs This</h3><p>Kubernetes components communicate using names.</p><p>Example:</p><p>kubectl uses:</p><pre>master-hardway.kubernetes.local</pre><p>instead of:</p><pre>10.10.10.27</pre><p>Certificates also contain names:</p><p>Example:</p><pre>CN = kubernetes</pre><pre>SAN:<br>DNS:master-hardway.kubernetes.local</pre><p>If the name cannot resolve:</p><pre>kubectl<br>   |<br>   |<br>   v<br>master-hardway.kubernetes.local<br>   |<br>   |<br>   X<br>No IP found</pre><p>Connection fails.</p><h3>10. Alternative Solution</h3><p>Instead of editing the template, you could disable cloud-init host management.</p><p>Edit:</p><pre>sudo nano /etc/cloud/cloud.cfg</pre><p>Change:</p><pre>manage_etc_hosts: true</pre><p>to:</p><pre>manage_etc_hosts: false</pre><p>Then you manage /etc/hosts manually.</p><p>However, in a Proxmox cloud environment, using the template method is often better.</p><h3>11. Final Architecture</h3><p>Your current setup:</p><pre>                Proxmox<br>                   |<br>                   |<br>             Cloud-init<br>                   |<br>                   |<br>      /etc/cloud/templates/hosts.debian.tmpl<br>                   |<br>                   |<br>                   v<br>               /etc/hosts<br>                   |<br>                   |<br>        ------------------------<br>        |          |           |<br>        v          v           v<br>  master-hardway worker1    worker2</pre><pre> 10.10.10.27  10.10.10.28 10.10.10.29</pre><h3>Final Summary</h3><p>ItemExplanationProblem/etc/hosts entries disappeared after rebootCauseCloud-init regenerated /etc/hostsEnvironmentProxmox VM using Cloud-initEvidencemanage_etc_hosts: True messageFixAdd entries to hosts.debian.tmplApplycloud-init single --name update_etc_hostsResult/etc/hosts survives rebootKubernetes benefitHostname resolution works correctly</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f8f13c9ba51f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Ansible:]]></title>
            <link>https://medium.com/@ygce108/ansible-3121dc2959c0?source=rss-ea26bead1eaa------2</link>
            <guid isPermaLink="false">https://medium.com/p/3121dc2959c0</guid>
            <category><![CDATA[ansible]]></category>
            <dc:creator><![CDATA[yuresh Gurung]]></dc:creator>
            <pubDate>Mon, 29 Jun 2026 12:10:03 GMT</pubDate>
            <atom:updated>2026-06-29T12:10:03.089Z</atom:updated>
            <content:encoded><![CDATA[<h3>1. Definition of Ansible</h3><p><strong>Ansible</strong> is an open-source IT automation tool used to automate system administration, software deployment, configuration management, application provisioning, and orchestration. It enables administrators to manage multiple remote systems from a single control node using simple, human-readable YAML playbooks. Ansible follows an <strong>agentless architecture</strong>, communicating with managed nodes through <strong>SSH</strong> (Linux/Unix) or <strong>WinRM</strong> (Windows), eliminating the need to install additional software on target systems.</p><p>Ansible is designed around the principles of <strong>simplicity, agentless automation, scalability, idempotence, and low maintenance overhead</strong>, making it suitable for automating physical, virtual, cloud, and containerized infrastructures.</p><p><em>source-linke: 1. </em><a href="https://docs.ansible.com/projects/ansible/latest/getting_started/introduction.html"><em>https://docs.ansible.com/projects/ansible/latest/getting_started/introduction.html</em></a></p><p><a href="https://spacelift.io/blog/ansible-architecture#key-points">2. https://spacelift.io/blog/ansible-architecture#key-points</a></p><h3>2. Ansible Architecture</h3><p>The core Ansible architecture consists of three primary components (Control Node, Inventory, and Managed Nodes). Additional components such as Modules, Playbooks, Plugins, and Roles support the automation workflow.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/537/1*sA3rek0JuO_n03wh_TphXg.png" /></figure><h3>Components of Ansible Architecture</h3><h3>2.1. Control Node</h3><p>The <strong>Control Node</strong> is the system where Ansible is installed. It is responsible for executing Ansible commands, storing playbooks and inventory files, and managing communication with remote systems. The control node sends automation tasks to managed nodes and receives execution results.</p><p><strong>Responsibilities:</strong></p><ul><li>Executes playbooks and ad-hoc commands</li><li>Stores inventory and configuration files</li><li>Connects to managed nodes via SSH or WinRM</li><li>Transfers and executes Ansible modules</li></ul><h3>2.2. Inventory</h3><p>The <strong>Inventory</strong> is a file or dynamic source that contains information about all managed nodes. It organizes hosts into logical groups, allowing administrators to apply automation tasks to specific servers or collections of servers.</p><p><strong>Responsibilities:</strong></p><ul><li>Lists managed hosts</li><li>Organizes hosts into groups</li><li>Stores connection details such as IP addresses and usernames</li></ul><h3>2.3. Managed Nodes</h3><p><strong>Managed Nodes</strong> are the remote systems controlled by Ansible. These systems do not require any Ansible software or agent installation. The control node connects securely using SSH (Linux/Unix) or WinRM (Windows), transfers the required module, executes it, and removes it after completion.</p><p><strong>Responsibilities:</strong></p><ul><li>Receive automation tasks</li><li>Execute temporary Ansible modules</li><li>Return execution results to the control node</li></ul><h3>2.4. Modules</h3><p><strong>Modules</strong> are small programs that perform specific automation tasks such as installing packages, creating users, copying files, or managing services. The control node temporarily copies modules to managed nodes, executes them, and then removes them.</p><h3>2.5. Playbooks</h3><p>A <strong>Playbook</strong> is a YAML file that defines automation tasks in a human-readable format. Playbooks describe the desired state of systems and enable repeatable, consistent automation across multiple managed nodes.</p><h3>Workflow of Ansible Architecture</h3><ol><li>The administrator executes an Ansible command or playbook from the <strong>Control Node</strong>.</li><li>The <strong>Inventory</strong> identifies the target managed nodes.</li><li>The control node establishes a secure SSH or WinRM connection.</li><li>Required <strong>Ansible modules</strong> are copied temporarily to the managed nodes.</li><li>The managed nodes execute the modules.</li><li>Execution results are returned to the control node.</li><li>Temporary modules are removed from the managed nodes.</li></ol><h3>Key Characteristics of Ansible :</h3><ul><li><strong>Agentless:</strong> No software agent is required on managed nodes.</li><li><strong>SSH/WinRM Based:</strong> Uses secure native communication protocols.</li><li><strong>Push-Based:</strong> The control node pushes tasks directly to managed nodes.</li><li><strong>Idempotent:</strong> Running the same playbook multiple times produces the same desired state without unnecessary changes.</li><li><strong>Scalable:</strong> A single control node can manage hundreds or thousands of remote systems.</li><li><strong>Human-Readable:</strong> Automation is written using simple YAML syntax, improving readability and maintainability.</li></ul><h3>3. Ansible Installation Methods</h3><p>Ansible can be installed using several methods depending on the operating system, environment, and administrative requirements. The following are the most common installation methods.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2ScHAf7Y7SZUXcVvrPgRxQ.png" /></figure><blockquote>source link: <a href="https://docs.ansible.com/projects/ansible/latest/installation_guide/intro_installation.html">https://docs.ansible.com/projects/ansible/latest/installation_guide/intro_installation.html</a></blockquote><h3>3.2 Installation Method</h3><p>The recommended installation method depends on the user’s requirements:</p><ul><li><strong>Package Manager (APT/DNF/YUM):</strong> Suitable for stable production environments and beginners.</li><li><strong>pipx:</strong> Recommended by the Ansible community for most users because it installs Ansible in an isolated environment without affecting the system Python.</li><li><strong>pip:</strong> Appropriate when integrating Ansible into an existing Python environment or when specific Python package management is required.</li><li><strong>Container:</strong> Recommended for DevOps workflows, CI/CD pipelines, and execution environments.</li><li><strong>Source Code:</strong> Intended for Ansible developers and contributors who need access to development versions.</li></ul><h3>4. Select the Ansible Package</h3><p>Before installing Ansible, choose one of the two official Python packages.</p><h3>4.1. ansible (Recommended)</h3><ul><li>Complete Ansible distribution.</li><li>Includes <strong>ansible-core</strong> and a curated set of Ansible Collections.</li><li>Provides commonly used command-line tools: ansible ansible-playbook, ansible-galaxy, ansible-vault,.......</li></ul><p>Suitable for:</p><ul><li>Beginners</li><li>Learning Ansible</li><li>Small and medium-sized environments</li><li>Most production deployments</li><li>Installation example:</li></ul><pre>python3 -m pip install --user ansible</pre><h3>4.2. ansible-core</h3><ul><li>Minimal Ansible distribution.</li><li>Includes only the Ansible automation engine, built-in modules, and plugins.</li><li>Does <strong>not</strong> include community Ansible Collections.</li><li>Additional collections must be installed manually using:</li></ul><pre>ansible-galaxy collection install &lt;collection_name&gt;</pre><p>Suitable for:</p><ul><li>Advanced users</li><li>Developers</li><li>Custom or lightweight deployments</li><li>Installation example:</li></ul><pre>python3 -m pip install --user ansible-core</pre><h3><strong>5. Install Ansible on your Local Machine </strong>:</h3><ul><li><strong>Control Node:</strong> Your <strong>local machine</strong></li><li><strong>Managed Node 1:</strong> Proxmox VM1</li><li><strong>Managed Node 2:</strong> Proxmox VM2</li></ul><p>the architecture looks like this:</p><pre>                +----------------------+<br>                |  Local Machine       |<br>                |  Ansible Control     |<br>                |  Ubuntu/Linux        |<br>                +----------+-----------+<br>                           |<br>                    SSH (Port 22)<br>                -----------+-----------<br>               |                       |<br>               |                       |<br>      +--------+--------+     +--------+--------+<br>      |  Proxmox VM1    |     |  Proxmox VM2    |<br>      | Managed Node 1  |     | Managed Node 2  |<br>      | SSH Server      |     | SSH Server      |<br>      +-----------------+     +-----------------+</pre><h3>Step 1: Install Ansible on your Local Machine</h3><p>If your local machine is <strong>Ubuntu/Debian</strong>:</p><pre>sudo apt update<br>sudo apt install ansible -y</pre><p>Verify installation:</p><pre>ansible --version</pre><p>Example output:</p><pre>ansible [core 2.18.x]<br>python version = 3.x.x</pre><h3>Step 2: Check SSH on Both VMs</h3><p>On <strong>VM1</strong></p><pre>sudo apt update<br>sudo apt install openssh-server -y</pre><p><strong>openssh-server</strong> → The package that provides the SSH server (sshd), allowing other computers to connect to this machine remotely using SSH.</p><p>Start SSH</p><pre>sudo systemctl enable ssh<br>sudo systemctl start ssh</pre><p>Check</p><pre>sudo systemctl status ssh</pre><p>Repeat the same on <strong>VM2</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/942/1*yoUrWS0CoILaZgxMDnuikg.png" /></figure><h3>Step 3: Find VM IP Addresses</h3><p>On each VM:</p><pre>ip a</pre><p>Example</p><p>VM1</p><pre>ansible-m-y= 10.10.10.75</pre><p>VM2</p><pre>ansible-w-y= 10.10.10.76</pre><h3>Step 4: Test SSH from Local Machine</h3><p>From your local machine:</p><pre>ssh user@ip-adress<br>ssh ansible-my@10.10.10.75</pre><p>Example:</p><pre><br>ssh ansible-wy@10.10.10.76</pre><p>Then test:</p><pre>ssh user@192.168.1.102</pre><p>If you can log in, SSH is working.</p><h3>Step 5: Generate an SSH Key</h3><p>On your local machine:</p><pre>ssh-keygen<br><br>Enter file in which to save the key:<br>/home/yuresh/.ssh/ansible_key</pre><p>Press <strong>Enter</strong> for all prompts.</p><p>This creates:</p><pre>~/.ssh/ansible-key<br>~/.ssh/ansible-key.pub</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/917/1*mquismaUl3iDM9Lzt2Dntw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/623/1*KpZEIkT0p0TFR4CVOPcuWQ.png" /></figure><h3>Step 6: Copy the SSH Key to Both VMs</h3><p>For VM1:</p><pre>ssh-copy-id -i ~/.ssh/ansible-key.pub ansible-my@10.10.10.75</pre><pre>ssh-copy-id -i ~/.ssh/ansible-key.pub ansible-wy@10.10.10.76</pre><p><strong>Why use </strong><strong>IdentityFile in </strong><strong>~/.ssh/config?</strong></p><ul><li>Avoids typing -i ~/.ssh/ansible-key every time.</li><li>Automatically uses the correct private key.</li><li>Simplifies SSH and Ansible connections.</li><li>Makes managing multiple servers easier.</li></ul><p><strong>Example:</strong></p><pre>Host node1<br>    HostName 192.168.1.101<br>    User yuresh<br>    IdentityFile ~/.ssh/ansible-key</pre><p>Now, instead of:</p><pre>ssh -i ~/.ssh/ansible-key yuresh@192.168.1.101</pre><p>you can simply run:</p><pre>ssh node1</pre><p>IdentityFile in ~/.ssh/config is the configuration-file equivalent of the -i option.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/391/1*LHoOzrUnn5pdoPbduy8JvQ.png" /></figure><h3>Step 7: Create an Ansible Inventory</h3><p>Create a project directory:</p><pre>mkdir ansible-project<br>cd ansible-project</pre><p>Create an inventory file:</p><pre>nano inventory.ini</pre><p>Example:</p><pre>[webservers]<br>vm1 ansible_host=192.168.1... ansible_user=user...<br>vm2 ansible_host=192.168.1.... ansible_user=user...</pre><p>Replace user with the username on your VMs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/560/1*MoMdjne75ZyeR9lVY9OKMw.png" /></figure><h3>Step 8: Test the Connection</h3><p>Run:</p><pre>ansible all -i inventory.ini -m ping</pre><p>Expected output:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/732/1*gYkkGGjkviDWJlNifqZ11Q.png" /></figure><h3>Step 9: Run an Ad-hoc Command</h3><p>Check uptime:</p><pre>ansible all -i inventory.ini -a &quot;uptime&quot;</pre><p>Check hostname:</p><pre>ansible all -i inventory.ini -a &quot;hostname&quot;</pre><p>Check disk usage:</p><pre>ansible all -i inventory.ini -a &quot;df -h&quot;</pre><h3>Step 10: Create Your First Playbook</h3><p>Create a file:</p><pre>nano ping.yml</pre><p>Add:</p><pre>---<br>- name: Test connection<br>  hosts: all<br>  become: yes<br>tasks:<br>    - name: Ping managed nodes<br>      ansible.builtin.ping:</pre><p>Run it:</p><pre>ansible-playbook -i inventory.ini ping.yml</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/941/1*rxNVpBotra2BLNbU_u28xw.png" /></figure><h3>Directory Structure</h3><pre>ansible-project/<br>│<br>├── inventory.ini<br>├── ping.yml<br>└── ansible.cfg    (optional)</pre><p>Ansible provides a simple, agentless, and scalable approach to infrastructure automation. By using SSH-based communication, YAML playbooks, and reusable modules, administrators can efficiently manage multiple systems while maintaining consistency and reducing manual effort.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3121dc2959c0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kubernetes Node Maintenance: Cordon, Drain, and Uncordon]]></title>
            <link>https://medium.com/@ygce108/kubernetes-node-maintenance-cordon-drain-and-uncordon-306a9d9d20f5?source=rss-ea26bead1eaa------2</link>
            <guid isPermaLink="false">https://medium.com/p/306a9d9d20f5</guid>
            <category><![CDATA[cordon]]></category>
            <category><![CDATA[k8s]]></category>
            <category><![CDATA[drain]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[yuresh Gurung]]></dc:creator>
            <pubDate>Thu, 25 Jun 2026 08:48:50 GMT</pubDate>
            <atom:updated>2026-06-25T08:48:50.565Z</atom:updated>
            <content:encoded><![CDATA[<h3>1. Why do we need Cordon and Drain?</h3><p>In Kubernetes production clusters, worker nodes sometimes require maintenance:</p><p>Examples:</p><ul><li>Memory leak</li><li>CPU issues</li><li>Disk problems</li><li>OS patching</li><li>Kernel upgrade</li><li>Hardware replacement</li><li>Kubernetes node upgrade</li></ul><p>A production node may be running important applications:</p><pre>Kubernetes Cluster</pre><pre>             Control Plane<br>                  |<br>     --------------------------------<br>     |              |               |<br>   node01         node02          node03</pre><pre>   Pods:<br>   - frontend<br>   - backend<br>   - database</pre><p>You cannot simply stop node01.</p><p>If you shut down immediately:</p><pre>node01 OFF</pre><pre>       |<br>       |<br>Pods unavailable<br>       |<br>       |<br>Application downtime</pre><p>Kubernetes provides a safe maintenance process:</p><pre>1. Cordon<br>      |<br>      v<br>2. Drain<br>      |<br>      v<br>3. Maintenance<br>      |<br>      v<br>4. Uncordon</pre><h3>2. Kubernetes Scheduler Role</h3><p>Before understanding cordon, understand the scheduler.</p><p>The scheduler decides:</p><blockquote><em>Which node should run a new Pod?</em></blockquote><p>Example:</p><p>A new Pod is created:</p><pre>Deployment<br>     |<br>     |<br>New Pod</pre><p>Scheduler checks:</p><pre>node01  ?<br>node02  ?<br>node03  ?</pre><p>Then selects the best node.</p><p>Example:</p><pre>Pod<br> |<br> |<br>Scheduler<br> |<br> |<br>node02</pre><h3>3. Cordon: Mark Node Unschedulable</h3><h3>Definition</h3><p><strong>Cordon marks a node as unschedulable.</strong></p><p>Meaning:</p><blockquote><em>The scheduler will not place any new Pods on this node.</em></blockquote><p>Command:</p><pre>kubectl cordon node01</pre><h3>Before Cordon</h3><p>Node:</p><pre>node01</pre><pre>Status:<br>Ready</pre><pre>Scheduling:<br>Enabled<br></pre><pre>Running Pods:</pre><pre>frontend<br>backend<br>api</pre><p>New Pod scheduling:</p><pre>Scheduler<br>     |<br>     |<br> node01  ✅</pre><h3>After Cordon</h3><pre>node01</pre><pre>Status:<br>Ready,SchedulingDisabled</pre><pre>Scheduling:<br>Disabled</pre><p>New Pod scheduling:</p><pre>Scheduler<br>     |<br>     |<br> node01 ❌</pre><pre> node02 ✅<br> node03 ✅</pre><h3>Important: Cordon Does NOT Remove Pods</h3><p>Many beginners misunderstand this.</p><p>Cordon only affects <strong>future scheduling</strong>.</p><p>It does:</p><pre>YES:<br>- Block new Pods</pre><pre>NO:<br>- Delete existing Pods<br>- Move existing Pods<br>- Stop applications</pre><p>Example:</p><p>Before:</p><pre>node01</pre><pre>Pod A<br>Pod B<br>Pod C</pre><p>After cordon:</p><pre>node01</pre><pre>Pod A<br>Pod B<br>Pod C</pre><p>Everything continues running.</p><h3>4. Verify Cordon Status</h3><p>Command:</p><pre>kubectl get nodes</pre><p>Example output:</p><pre>NAME       STATUS<br>node01     Ready,SchedulingDisabled<br>node02     Ready<br>node03     Ready</pre><p>Meaning:</p><pre>node01<br> |<br> +-- Existing workloads: Running<br> |<br> +-- New workloads: Blocked</pre><h3>5. Drain: Evict Existing Pods</h3><h3>Definition</h3><p><strong>Drain safely removes workloads from a node.</strong></p><p>Command:</p><pre>kubectl drain node01 --ignore-daemonsets</pre><h3>What happens during Drain?</h3><p>Before:</p><pre>node01</pre><pre>frontend Pod<br>backend Pod<br>api Pod</pre><p>Drain starts:</p><pre>kubectl drain</pre><pre>       |<br>       |<br> Kubernetes API</pre><pre>       |<br>       |<br> Pod eviction</pre><p>After:</p><pre>node01</pre><pre>(empty)<br></pre><pre>node02/node03</pre><pre>frontend<br>backend<br>api</pre><h3>6. Why Drain Instead of Delete Pods?</h3><p>Wrong way:</p><pre>kubectl delete pod nginx</pre><p>Problem:</p><pre>Pod deleted immediately</pre><pre>Application interruption possible</pre><p>Drain is safer:</p><pre>Drain<br> |<br> |<br>Checks Pod controllers<br> |<br> |<br>Evicts Pods gracefully<br> |<br> |<br>Controllers create replacements</pre><h3>7. Deployment Pod Example During Drain</h3><p>Before:</p><p>Deployment:</p><pre>replicas: 3</pre><p>Pods:</p><pre>node01:<br> nginx-pod-1</pre><pre>node02:<br> nginx-pod-2</pre><pre>node03:<br> nginx-pod-3</pre><p>Drain node01:</p><pre>nginx-pod-1 removed</pre><p>Deployment controller notices:</p><pre>Desired:<br>3 Pods</pre><pre>Current:<br>2 Pods</pre><p>Creates replacement:</p><pre>node02:<br> nginx-new-pod</pre><p>Final:</p><pre>node02:<br> nginx-pod-2<br> nginx-new-pod<br></pre><pre>node03:<br> nginx-pod-3</pre><p>Application remains available.</p><h3>8. Why Use — ignore-daemonsets?</h3><p>Command:</p><pre>kubectl drain node01 --ignore-daemonsets</pre><p>DaemonSet example:</p><pre>DaemonSet</pre><pre>   |<br>   |<br>-----------------<br>|       |       |<br>node01 node02 node03<br></pre><pre>Pods:<br>- fluent-bit<br>- node-exporter<br>- calico</pre><p>DaemonSets are designed:</p><blockquote><em>One Pod on every node.</em></blockquote><p>They are not normally removed during drain.</p><p>So we tell Kubernetes:</p><pre>Ignore DaemonSet Pods</pre><p>using:</p><pre>--ignore-daemonsets</pre><h3>9. Verify Drain Completed</h3><p>Command:</p><pre>kubectl get pods -o wide | grep node01</pre><p>Explanation:</p><pre>kubectl get pods</pre><p>Shows Pods.</p><pre>-o wide</pre><p>Shows extra details:</p><p>Example:</p><pre>NAME          NODE</pre><pre>nginx-abc     node01</pre><p>After drain:</p><pre>(no output)</pre><p>Meaning:</p><pre>No workload Pods running on node01</pre><h3>10. Maintenance Phase</h3><p>Now node01 is safe.</p><p>You can perform:</p><pre>- Fix memory leak<br>- Restart machine<br>- Upgrade packages<br>- Replace hardware<br>- Upgrade Kubernetes version</pre><p>Example:</p><pre>node01</pre><pre>Maintenance Mode</pre><h3>11. Uncordon: Return Node to Service</h3><p>After maintenance:</p><p>Command:</p><pre>kubectl uncordon node01</pre><p>Meaning:</p><blockquote><em>Allow scheduling on this node again.</em></blockquote><p>Before:</p><pre>node01</pre><pre>Scheduling:<br>Disabled</pre><p>After:</p><pre>node01</pre><pre>Scheduling:<br>Enabled</pre><h3>12. Complete Production Workflow</h3><h3>Step 1: Check Nodes</h3><pre>kubectl get nodes</pre><p>Output:</p><pre>node01 Ready<br>node02 Ready<br>node03 Ready</pre><h3>Step 2: Stop New Scheduling</h3><pre>kubectl cordon node01</pre><p>Verify:</p><pre>kubectl get nodes</pre><p>Output:</p><pre>node01 Ready,SchedulingDisabled</pre><h3>Step 3: Move Workloads Away</h3><pre>kubectl drain node01 --ignore-daemonsets</pre><h3>Step 4: Confirm Node Empty</h3><pre>kubectl get pods -o wide | grep node01</pre><p>No result = successful drain.</p><h3>Step 5: Perform Maintenance</h3><p>Example:</p><pre>Upgrade OS<br>Fix memory<br>Restart server</pre><h3>Step 6: Return Node</h3><pre>kubectl uncordon node01</pre><h3>13. Cordon vs Drain vs Uncordon</h3><p>CommandPurposeExisting PodsNew PodscordonDisable schedulingContinue runningBlockeddrainEvict workloadsRemovedAlready blockeduncordonEnable schedulingContinueAllowed</p><h3>14. Real Production Scenario</h3><p>Company Kubernetes cluster:</p><pre>Production Cluster</pre><pre>worker-1<br>worker-2<br>worker-3</pre><p>Alert:</p><pre>worker-1 memory leak detected</pre><p>Engineer performs:</p><h3>1. Protect node</h3><pre>kubectl cordon worker-1</pre><p>Result:</p><pre>No new applications go to worker-1</pre><h3>2. Move applications</h3><pre>kubectl drain worker-1 --ignore-daemonsets</pre><p>Result:</p><pre>Applications move to worker-2/worker-3</pre><h3>3. Repair node</h3><pre>Fix memory problem</pre><h3>4. Return node</h3><pre>kubectl uncordon worker-1</pre><p>Result:</p><pre>worker-1 available again</pre><h3>Professional Summary</h3><pre>CORDON<br>=<br>Prevent new Pods from scheduling<br></pre><pre>DRAIN<br>=<br>Safely evacuate existing Pods<br></pre><pre>MAINTENANCE<br>=<br>Repair the node<br></pre><pre>UNCORDON<br>=<br>Allow scheduling again</pre><p>This process is a standard Kubernetes administrator/SRE procedure for safe node maintenance with minimal application impact.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=306a9d9d20f5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Assign Pods to Nodes Using Node Affinity in Kubernetes]]></title>
            <link>https://medium.com/@ygce108/assign-pods-to-nodes-using-node-affinity-in-kubernetes-991e49d70070?source=rss-ea26bead1eaa------2</link>
            <guid isPermaLink="false">https://medium.com/p/991e49d70070</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[k8s]]></category>
            <category><![CDATA[pods]]></category>
            <category><![CDATA[kubernetes-node-affinity]]></category>
            <dc:creator><![CDATA[yuresh Gurung]]></dc:creator>
            <pubDate>Wed, 24 Jun 2026 11:38:34 GMT</pubDate>
            <atom:updated>2026-06-24T11:38:34.689Z</atom:updated>
            <content:encoded><![CDATA[<h3>What is Node Affinity?</h3><p><strong>Node Affinity</strong> is a Kubernetes scheduling feature that controls <strong>which node a Pod can run on</strong> based on <strong>node labels</strong>.</p><p>In simple words:</p><blockquote><em>Node Affinity allows you to tell the Kubernetes Scheduler:</em></blockquote><blockquote><strong><em>“Run this Pod only on nodes that match specific labels.”</em></strong></blockquote><p>or</p><blockquote><strong><em>“Prefer these nodes, but if unavailable, run somewhere else.”</em></strong></blockquote><p>Node Affinity is the advanced version of <strong>nodeSelector</strong> and provides more flexible scheduling rules.</p><h3>Why Do We Need Node Affinity?</h3><p>In real production environments, not all nodes are identical.</p><p>Example cluster:</p><pre>+----------------+<br>| worker-1       |<br>| SSD Disk       |<br>| High CPU       |<br>+----------------+<br><br>+----------------+<br>| worker-2       |<br>| HDD Disk       |<br>| Standard CPU   |<br>+----------------+<br><br>+----------------+<br>| worker-3       |<br>| GPU Enabled    |<br>+----------------+</pre><p>Different workloads require different hardware.</p><p>ApplicationRequirementDatabaseSSDAI/MLGPULoggingHDDHigh Traffic AppHigh CPU</p><p>Without Node Affinity:</p><pre>Scheduler<br>    ↓<br>Random Node</pre><p>With Node Affinity:</p><pre>Scheduler<br>    ↓<br>Check Labels<br>    ↓<br>Find Matching Node<br>    ↓<br>Schedule Pod</pre><h3>Real Production Example</h3><p>Imagine:</p><pre>E-commerce Application</pre><p>Components:</p><pre>Frontend<br>Backend<br>Database<br>Redis</pre><p>Database requires:</p><pre>Fast SSD Storage</pre><p>Nodes:</p><pre>worker-1 -&gt; SSD<br>worker-2 -&gt; HDD<br>worker-3 -&gt; HDD</pre><p>You don’t want MySQL running on HDD nodes.</p><p>Node Affinity ensures:</p><pre>MySQL Pod<br>    ↓<br>Only SSD Node</pre><h3>How Node Affinity Works Internally</h3><h3>Step 1: Node Labels Exist</h3><p>Node:</p><pre>worker-1</pre><p>Label:</p><pre>disktype=ssd</pre><p>Kubernetes stores this inside:</p><pre>etcd</pre><h3>Step 2: Pod Requests Node Affinity</h3><p>Pod says:</p><pre>I need:<br>disktype=ssd</pre><h3>Step 3: Scheduler Checks Nodes</h3><p>Scheduler scans all nodes.</p><pre>worker-1 -&gt; disktype=ssd<br>worker-2 -&gt; disktype=hdd<br>worker-3 -&gt; disktype=hdd</pre><h3>Step 4: Match Found</h3><pre>worker-1</pre><p>matches.</p><h3>Step 5: Pod Scheduled</h3><pre>Pod<br> ↓<br>worker-1</pre><h3>Node Affinity Architecture</h3><pre>                  Kubernetes Scheduler <br>                        │<br>                        │<br>                        ▼<br>              Read Pod Affinity Rules<br>                        │<br>                        ▼<br>              Check Node Labels<br>                        │<br>        ┌───────────────┼───────────────┐<br>        │               │               │<br>        ▼               ▼               ▼<br>    worker-1       worker-2       worker-3<br>  disktype=ssd   disktype=hdd   disktype=hdd<br>        │<br>        │ Match<br>        ▼<br>      Pod<br></pre><h3>Step 1: View Existing Nodes</h3><pre>kubectl get nodes</pre><p>Example:</p><pre>NAME       STATUS   ROLES<br>worker-1   Ready<br>worker-2   Ready<br>worker-3   Ready</pre><h3>Step 2: View Node Labels</h3><pre>kubectl get nodes --show-labels</pre><p>Example:</p><pre>NAME       LABELS<br>worker-1   disktype=ssd<br>worker-2   disktype=hdd<br>worker-3   disktype=hdd</pre><h3>Step 3: Add Label to Node</h3><h3>Command</h3><pre>kubectl label node worker-1 disktype=ssd</pre><h3>Breakdown</h3><pre>kubectl</pre><p>Kubernetes CLI.</p><pre>label</pre><p>Add or modify labels.</p><pre>node</pre><p>Target resource.</p><pre>worker-1</pre><p>Node name.</p><pre>disktype=ssd</pre><p>Key-value label.</p><p>Verify:</p><pre>kubectl get nodes --show-labels</pre><p>Output:</p><pre>worker-1   disktype=ssd</pre><h3>Required Node Affinity</h3><h3>Definition</h3><p><strong>requiredDuringSchedulingIgnoredDuringExecution</strong></p><p>Means:</p><pre>Must match</pre><p>If no node matches:</p><pre>Pod remains Pending</pre><h3>YAML Manifest</h3><pre>apiVersion: v1<br>kind: Pod</pre><pre>metadata:<br>  name: nginx-required</pre><pre>spec:</pre><pre>  affinity:<br>    nodeAffinity:</pre><pre>      requiredDuringSchedulingIgnoredDuringExecution:</pre><pre>        nodeSelectorTerms:</pre><pre>        - matchExpressions:</pre><pre>          - key: disktype<br>            operator: In<br>            values:<br>            - ssd</pre><pre>  containers:</pre><pre>  - name: nginx<br>    image: nginx</pre><h3>Complete Code Explanation</h3><pre>affinity:</pre><p>Enable affinity rules.</p><pre>nodeAffinity:</pre><p>Apply rules on nodes.</p><pre>requiredDuringSchedulingIgnoredDuringExecution:</pre><p>Mandatory scheduling condition.</p><pre>nodeSelectorTerms:</pre><p>List of conditions.</p><pre>matchExpressions:</pre><p>Label matching expressions.</p><pre>key: disktype</pre><p>Node label key.</p><pre>operator: In</pre><p>Check if value exists.</p><pre>values:<br>- ssd</pre><p>Required label value.</p><p>Meaning:</p><pre>Schedule Pod ONLY if:</pre><pre>disktype=ssd</pre><h3>Deploy Pod</h3><pre>kubectl apply -f nginx-required.yaml</pre><p>Check Pod:</p><pre>kubectl get pods -o wide</pre><p>Example:</p><pre>NAME             READY   STATUS<br>nginx-required   1/1     Running</pre><p>Check Node:</p><pre>kubectl get pod nginx-required -o wide</pre><p>Output:</p><pre>NAME             NODE<br>nginx-required   worker-1</pre><h3>What If No Matching Node Exists?</h3><p>Suppose:</p><pre>worker-1 -&gt; hdd<br>worker-2 -&gt; hdd<br>worker-3 -&gt; hdd</pre><p>Pod requests:</p><pre>disktype=ssd</pre><p>Scheduler:</p><pre>No Match Found</pre><p>Result:</p><pre>STATUS = Pending</pre><p>Check:</p><pre>kubectl describe pod nginx-required</pre><p>Output:</p><pre>0/3 nodes available:<br>node(s) didn&#39;t match node affinity.</pre><h3>Preferred Node Affinity</h3><h3>Definition</h3><p><strong>preferredDuringSchedulingIgnoredDuringExecution</strong></p><p>Means:</p><pre>Try to match</pre><p>but</p><pre>Not mandatory</pre><p>If no matching node exists:</p><pre>Pod still runs elsewhere</pre><h3>YAML Example</h3><pre>apiVersion: v1<br>kind: Pod</pre><pre>metadata:<br>  name: nginx-preferred</pre><pre>spec:</pre><pre>  affinity:<br>    nodeAffinity:</pre><pre>      preferredDuringSchedulingIgnoredDuringExecution:</pre><pre>      - weight: 1</pre><pre>        preference:</pre><pre>          matchExpressions:</pre><pre>          - key: disktype<br>            operator: In<br>            values:<br>            - ssd</pre><pre>  containers:</pre><pre>  - name: nginx<br>    image: nginx</pre><h3>Code Explanation</h3><pre>preferredDuringSchedulingIgnoredDuringExecution</pre><p>Soft rule.</p><pre>weight: 1</pre><p>Preference score.</p><p>Range:</p><pre>1 - 100</pre><p>Higher weight:</p><pre>Higher scheduling priority</pre><p>Example:</p><pre>weight: 100</pre><p>strong preference.</p><pre>preference:</pre><p>Desired node condition.</p><p>Meaning:</p><pre>Prefer SSD node.</pre><pre>If unavailable,<br>use another node.</pre><h3>Required vs Preferred</h3><p>FeatureRequiredPreferredMust MatchYesNoScheduler EnforcesStrictlySoftlyNo Matching NodePendingScheduled ElsewhereProduction Critical AppsYesNoFlexible SchedulingNoYes</p><p>Example:</p><p>Required:</p><pre>Database<br> ↓<br>SSD Only</pre><p>Preferred:</p><pre>Web Server<br> ↓<br>Prefer SSD<br> ↓<br>Otherwise Any Node</pre><h3>Supported Operators</h3><h3>In</h3><pre>operator: In</pre><p>Example:</p><pre>values:<br>- ssd</pre><p>Means:</p><pre>disktype=ssd</pre><h3>NotIn</h3><pre>operator: NotIn</pre><p>Example:</p><pre>values:<br>- hdd</pre><p>Means:</p><pre>Anything except HDD</pre><h3>Exists</h3><pre>operator: Exists</pre><p>Means:</p><pre>Label key exists</pre><p>Example:</p><pre>key: gpu<br>operator: Exists</pre><h3>DoesNotExist</h3><pre>operator: DoesNotExist</pre><p>Means:</p><pre>Label must not exist</pre><h3>Gt</h3><pre>operator: Gt</pre><p>Example:</p><pre>key: cpu<br>operator: Gt<br>values:<br>- &quot;8&quot;</pre><p>Means:</p><pre>cpu &gt; 8</pre><h3>Lt</h3><pre>operator: Lt</pre><p>Example:</p><pre>key: memory<br>operator: Lt<br>values:<br>- &quot;64&quot;</pre><p>Means:</p><pre>memory &lt; 64</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/691/1*IlvzTZSeVIFzqFs6kyZiRw.png" /></figure><h3>IgnoredDuringExecution Meaning</h3><p>This phrase appears in:</p><pre>requiredDuringSchedulingIgnoredDuringExecution</pre><p>and</p><pre>preferredDuringSchedulingIgnoredDuringExecution</pre><p>Meaning:</p><pre>Check affinity ONLY during scheduling.</pre><p>After Pod starts:</p><pre>Node label changes</pre><p>Kubernetes does NOT evict Pod.</p><p>Example:</p><pre>Pod running on:</pre><pre>worker-1<br>disktype=ssd</pre><p>Later:</p><pre>kubectl label node worker-1 disktype=hdd --overwrite</pre><p>Pod:</p><pre>Still Running</pre><p>because:</p><pre>Ignored During Execution</pre><h3>Production Use Cases</h3><h3>Database Workloads</h3><pre>MySQL<br>PostgreSQL<br>MongoDB</pre><p>Run only on SSD nodes.</p><pre>disktype=ssd</pre><h3>GPU Workloads</h3><pre>TensorFlow<br>PyTorch<br>ML Training</pre><p>Run only on GPU nodes.</p><pre>gpu=true</pre><h3>High Memory Applications</h3><pre>Elasticsearch<br>Redis<br>Kafka</pre><p>Run only on memory-optimized nodes.</p><pre>memory=high</pre><h3>Dedicated Team Nodes</h3><pre>Dev Team<br>QA Team<br>Production Team</pre><p>Labels:</p><pre>environment=prod<br>environment=dev</pre><p>Pods run only on designated nodes.</p><h3>Useful Commands</h3><p>View nodes:</p><pre>kubectl get nodes</pre><p>View labels:</p><pre>kubectl get nodes --show-labels</pre><p>Add label:</p><pre>kubectl label node worker-1 disktype=ssd</pre><p>Remove label:</p><pre>kubectl label node worker-1 disktype-</pre><p>Describe node:</p><pre>kubectl describe node worker-1</pre><p>View pod node:</p><pre>kubectl get pod nginx -o wide</pre><p>Describe pod:</p><pre>kubectl describe pod nginx</pre><h3>Node Selector vs Node Affinity</h3><p>FeaturenodeSelectorNode AffinitySimple MatchingYesYesComplex RulesNoYesOperatorsNoYesPreferred RulesNoYesRequired RulesNoYesProduction UsageLimitedRecommended</p><p>Example nodeSelector:</p><pre>nodeSelector:<br>  disktype: ssd</pre><p>Example Node Affinity:</p><pre>nodeAffinity:<br>  requiredDuringSchedulingIgnoredDuringExecution:</pre><p>Node Affinity is more powerful and is the preferred approach in modern Kubernetes clusters.</p><h3>Summary</h3><p>Node Affinity is a Kubernetes scheduling mechanism that uses <strong>node labels</strong> to control where Pods are placed. The Kubernetes Scheduler evaluates node labels and Pod affinity rules before scheduling.</p><p>Two main modes exist:</p><ol><li><strong>requiredDuringSchedulingIgnoredDuringExecution</strong></li></ol><ul><li>Mandatory rule.</li><li>Pod runs only on matching nodes.</li><li>Otherwise remains Pending.</li></ul><ol><li><strong>preferredDuringSchedulingIgnoredDuringExecution</strong></li></ol><ul><li>Soft rule.</li><li>Scheduler tries to honor it.</li><li>Pod can run on other nodes if necessary.</li></ul><p>Node Affinity is commonly used in production for SSD-based databases, GPU workloads, high-memory applications, environment isolation, and workload placement optimization. It provides significantly more flexibility than nodeSelector through operators such as In, NotIn, Exists, Gt, and Lt.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=991e49d70070" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Create New User in Kubernetes]]></title>
            <link>https://medium.com/@ygce108/create-new-user-in-kubernetes-0137ce556a5f?source=rss-ea26bead1eaa------2</link>
            <guid isPermaLink="false">https://medium.com/p/0137ce556a5f</guid>
            <category><![CDATA[k8s-newuser]]></category>
            <category><![CDATA[k8s]]></category>
            <category><![CDATA[new-user]]></category>
            <category><![CDATA[rabc]]></category>
            <dc:creator><![CDATA[yuresh Gurung]]></dc:creator>
            <pubDate>Wed, 24 Jun 2026 08:08:01 GMT</pubDate>
            <atom:updated>2026-06-24T08:08:01.975Z</atom:updated>
            <content:encoded><![CDATA[<p>This lab teaches <strong>how to create a human user in Kubernetes and give limited permissions using RBAC</strong>.</p><h3>1. What is RBAC?</h3><p><strong>RBAC = Role-Based Access Control</strong></p><p>RBAC controls <strong>who can do what in Kubernetes</strong>.</p><p>Think of it like a company:</p><p>PersonPermissionEmployeeView documentsManagerEdit documentsAdminFull access</p><p>Kubernetes works the same way.</p><p>Example:</p><ul><li>Dev team → Can view pods</li><li>QA team → Can view logs</li><li>Admin team → Full cluster access</li></ul><h3>Why RBAC is Needed?</h3><p>Without RBAC:</p><p>Everyone could:</p><pre>kubectl delete pods --all</pre><p>or</p><pre>kubectl delete namespace production</pre><p>This would be dangerous.</p><p>RBAC allows:</p><p>✅ Least Privilege Principle</p><p>Users get only the permissions they need.</p><h3>RBAC Components</h3><p>There are 4 main components:</p><pre>User<br>  ↓<br>Role<br>  ↓<br>RoleBinding<br>  ↓<br>Resource</pre><p>Example:</p><pre>carlton<br>   ↓<br>pod-reader<br>   ↓<br>RoleBinding<br>   ↓<br>Pods</pre><p>Meaning:</p><pre>Carlton can view pods</pre><h3>Step 1: Create Namespace</h3><pre>kubectl create ns web</pre><p>Namespace created:</p><pre>web</pre><p>Think of namespace as a separate room.</p><pre>Cluster<br> ├── default<br> ├── kube-system<br> └── web</pre><p>We want Carlton to work only inside the <strong>web</strong> room.</p><h3>Step 2: Create Role</h3><p>Command:</p><pre>kubectl -n web create role pod-reader \<br>--verb=get,list \<br>--resource=pods</pre><h3>What is a Role?</h3><p>A Role defines:</p><pre>What actions?<br>On what resources?<br>In which namespace?</pre><h3>This role means:</h3><p>Resource:</p><pre>pods</pre><p>Actions:</p><pre>get<br>list</pre><p>Visual:</p><pre>Role: pod-reader</pre><pre>Can:<br>✓ get pods<br>✓ list pods</pre><pre>Cannot:<br>✗ delete pods<br>✗ create pods<br>✗ edit pods</pre><h3>get vs list</h3><h3>get</h3><p>View one object</p><pre>kubectl get pod nginx</pre><h3>list</h3><p>View all objects</p><pre>kubectl get pods</pre><h3>Step 3: Create RoleBinding</h3><p>Command:</p><pre>kubectl -n web create rolebinding pod-reader-binding \<br>--role=pod-reader \<br>--user=carlton</pre><h3>What is RoleBinding?</h3><p>Role exists.</p><p>But nobody uses it yet.</p><p>RoleBinding connects:</p><pre>User<br>  ↓<br>Role</pre><p>Visual:</p><pre>Carlton<br>    ↓<br>RoleBinding<br>    ↓<br>pod-reader Role</pre><p>Now Carlton receives permissions from pod-reader role.</p><h3>Verify</h3><pre>kubectl -n web get role,rolebinding</pre><p>Output:</p><pre>ROLE<br>pod-reader</pre><pre>ROLEBINDING<br>pod-reader-binding</pre><h3>Step 4: Create a Pod</h3><p>Create something for Carlton to view.</p><pre>kubectl -n web run pod1 --image=nginx</pre><p>Verify:</p><pre>kubectl -n web get pods</pre><p>Output:</p><pre>NAME   READY   STATUS<br>pod1   1/1     Running</pre><h3>Step 5: Why Create Certificates?</h3><p>Important:</p><p>Kubernetes has:</p><pre>No User Object</pre><p>Unlike pods or deployments.</p><p>You cannot do:</p><pre>kubectl create user carlton</pre><p>because User resource doesn’t exist.</p><h3>How Kubernetes Identifies Users</h3><p>Using certificates.</p><p>Inside the certificate:</p><pre>CN = Common Name</pre><p>Example:</p><pre>CN=carlton</pre><p>Kubernetes reads:</p><pre>CN=carlton</pre><p>and treats the request as user:</p><pre>carlton</pre><h3>Step 6: Create Private Key</h3><p>Command:</p><pre>openssl genrsa -out carlton.key 2048</pre><p>Creates:</p><pre>carlton.key</pre><p>Think:</p><pre>Password</pre><p>for Carlton.</p><p>Keep it private.</p><h3>Step 7: Create CSR</h3><p>CSR = Certificate Signing Request</p><p>Command:</p><pre>openssl req -new \<br>-key carlton.key \<br>-subj &quot;/CN=carlton&quot; \<br>-out carlton.csr</pre><p>Creates:</p><pre>carlton.csr</pre><h3>Why?</h3><p>Carlton is asking Kubernetes:</p><pre>Please issue me a certificate.<br>My username is carlton.</pre><p>Visual:</p><pre>Carlton<br>   ↓<br>CSR<br>   ↓<br>Kubernetes CA</pre><h3>Kubernetes CA</h3><p>CA = Certificate Authority</p><p>Kubernetes control plane acts as CA.</p><p>It signs certificates.</p><p>Visual:</p><pre>Kubernetes CA<br>    ↓<br>Signs<br>    ↓<br>Carlton Certificate</pre><h3>Step 8: Convert CSR to Base64</h3><pre>export REQUEST=$(cat carlton.csr | base64 -w 0)</pre><p>Why?</p><p>YAML cannot directly store binary data.</p><p>We encode it.</p><h3>Step 9: Submit CSR to Kubernetes</h3><pre>apiVersion: certificates.k8s.io/v1<br>kind: CertificateSigningRequest<br>metadata:<br>  name: carlton</pre><pre>spec:<br>  groups:<br>  - system:authenticated</pre><pre>  request: &lt;base64 csr&gt;</pre><pre>  signerName: kubernetes.io/kube-apiserver-client</pre><pre>  usages:<br>  - client auth</pre><p>Apply:</p><pre>kubectl apply -f csr.yaml</pre><h3>What Happens?</h3><pre>Carlton CSR<br>      ↓<br>Kubernetes API<br>      ↓<br>Pending Approval</pre><p>Check:</p><pre>kubectl get csr</pre><p>Output:</p><pre>carlton Pending</pre><h3>Step 10: Approve CSR</h3><p>Admin approves:</p><pre>kubectl certificate approve carlton</pre><p>Now:</p><pre>Pending<br>   ↓<br>Approved<br>   ↓<br>Issued</pre><p>Check:</p><pre>kubectl get csr</pre><p>Output:</p><pre>Approved,Issued</pre><h3>Step 11: Get Certificate</h3><p>Extract certificate:</p><pre>kubectl get csr carlton \<br>-o jsonpath=&#39;{.status.certificate}&#39; \<br>| base64 -d &gt; carlton.crt</pre><p>Creates:</p><pre>carlton.crt</pre><p>Now we have:</p><pre>carlton.key<br>carlton.crt</pre><h3>Step 12: Add User to kubeconfig</h3><pre>kubectl config set-credentials carlton \<br>--client-key=carlton.key \<br>--client-certificate=carlton.crt \<br>--embed-certs</pre><p>What happens?</p><p>Adds user:</p><pre>carlton</pre><p>to kubeconfig.</p><h3>Kubeconfig Structure</h3><pre>Cluster<br>Users<br>Contexts</pre><p>Example:</p><pre>Cluster:<br>  kubernetes</pre><pre>Users:<br>  kubernetes-admin<br>  carlton</pre><pre>Contexts:<br>  admin<br>  carlton</pre><h3>Step 13: Create Context</h3><pre>kubectl config set-context carlton \<br>--user=carlton \<br>--cluster=kubernetes</pre><h3>What is Context?</h3><p>Context =</p><pre>Cluster + User + Namespace</pre><p>Think:</p><pre>Login profile</pre><h3>Step 14: Switch User</h3><pre>kubectl config use-context carlton</pre><p>Now kubectl acts as:</p><pre>Carlton</pre><p>instead of:</p><pre>Admin</pre><h3>Step 15: Test Access</h3><p>Run:</p><pre>kubectl -n web get pods</pre><p>Output:</p><pre>NAME<br>pod1</pre><p>Success.</p><p>Because:</p><pre>Carlton<br>   ↓<br>RoleBinding<br>   ↓<br>pod-reader<br>   ↓<br>get/list pods</pre><h3>What Carlton Can Do</h3><p>✅ Allowed</p><pre>kubectl -n web get pods</pre><pre>kubectl -n web get pod pod1</pre><pre>kubectl -n web get pods -o wide</pre><h3>What Carlton Cannot Do</h3><p>❌ Create pod</p><pre>kubectl -n web run nginx --image=nginx</pre><p>Error:</p><pre>Forbidden</pre><p>❌ Delete pod</p><pre>kubectl -n web delete pod pod1</pre><p>Error:</p><pre>Forbidden</pre><p>❌ Access other namespace</p><pre>kubectl -n default get pods</pre><p>Error:</p><pre>Forbidden</pre><p>Because role exists only in:</p><pre>web namespace</pre><h3>Real Production Example</h3><p>Imagine a company:</p><h3>Developers</h3><pre>View Pods<br>View Logs<br>Restart Deployments</pre><h3>QA Team</h3><pre>View Pods<br>View Services</pre><h3>DevOps Team</h3><pre>Full Namespace Access</pre><h3>Platform Admin</h3><pre>Cluster Admin</pre><p>RBAC is used every day in production to ensure teams only have the permissions they need.</p><h3>Complete Flow Diagram</h3><pre>Create Namespace<br>        ↓<br>Create Role<br>        ↓<br>Create RoleBinding<br>        ↓<br>Create Key<br>        ↓<br>Create CSR<br>        ↓<br>Submit CSR<br>        ↓<br>Approve CSR<br>        ↓<br>Get Certificate<br>        ↓<br>Add User to kubeconfig<br>        ↓<br>Create Context<br>        ↓<br>Switch Context<br>        ↓<br>Test Access</pre><p><strong>Key takeaway:</strong> Kubernetes does not have a built-in User resource. Human users are typically authenticated using certificates (or external identity providers), and RBAC (Roles + RoleBindings) determines exactly what they are allowed to do.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0137ce556a5f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kubernetes PriorityClass and Preemption]]></title>
            <link>https://medium.com/@ygce108/kubernetes-priorityclass-and-preemption-professional-explanation-e698b86bd38d?source=rss-ea26bead1eaa------2</link>
            <guid isPermaLink="false">https://medium.com/p/e698b86bd38d</guid>
            <category><![CDATA[priorityclass]]></category>
            <category><![CDATA[k8s]]></category>
            <category><![CDATA[kubernetes-cluster]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[yuresh Gurung]]></dc:creator>
            <pubDate>Tue, 23 Jun 2026 06:40:07 GMT</pubDate>
            <atom:updated>2026-06-23T06:40:59.447Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>PriorityClass</strong> is a Kubernetes resource that defines the relative importance of Pods within a cluster. It allows administrators to ensure that critical workloads receive scheduling preference over less important workloads when cluster resources become constrained.</p><p>When a Pod is assigned a PriorityClass, Kubernetes assigns a numerical priority value to that Pod. Pods with higher priority values are scheduled before Pods with lower priority values. If sufficient resources are unavailable, the Kubernetes scheduler can invoke <strong>preemption</strong>, which terminates lower-priority Pods to free resources for higher-priority Pods.</p><h3>Built-in Priority Classes</h3><p>Every Kubernetes cluster includes two reserved high-priority classes intended for critical cluster components:</p><ul><li>system-cluster-critical</li><li>system-node-critica</li></ul><p>These are used by essential services such as DNS, networking, and other control-plane-related components to ensure cluster stability.</p><p>View existing PriorityClasses:</p><pre>kubectl get priorityclass</pre><p>Example:</p><pre>NAME                      VALUE<br>system-node-critical      2000001000<br>system-cluster-critical   2000000000</pre><h3>Creating a Custom PriorityClass</h3><p>To protect important application workloads, a custom PriorityClass can be created:</p><pre>kubectl create priorityclass high-priority --value=1000000</pre><p>This creates a PriorityClass named high-priority with a priority value of 1000000.</p><p>Verify:</p><pre>kubectl get priorityclass</pre><h3>Low-Priority Workloads</h3><p>Pods that do not specify a PriorityClass automatically receive a priority value of:</p><pre>0</pre><p>These Pods are considered lower priority and are candidates for preemption when resources become scarce.</p><p>Example deployment:</p><pre>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: low-prio<br>spec:<br>  replicas: 3<br>  selector:<br>    matchLabels:<br>      app: low-prio<br>  template:<br>    metadata:<br>      labels:<br>        app: low-prio<br>    spec:<br>      containers:<br>      - name: stress<br>        image: polinux/stress<br>        command: [&quot;stress&quot;]<br>        args: [&quot;--vm&quot;, &quot;1&quot;, &quot;--vm-bytes&quot;, &quot;400M&quot;, &quot;--timeout&quot;, &quot;600s&quot;]<br>        resources:<br>          requests:<br>            memory: &quot;500Mi&quot;<br>            cpu: &quot;100m&quot;</pre><p>Each replica requests:</p><ul><li>500Mi memory</li><li>100m CPU</li></ul><p>With three replicas, the deployment consumes a significant portion of node resources.</p><h3>High-Priority Workloads</h3><p>Critical workloads can be associated with the custom PriorityClass:</p><pre>apiVersion: v1<br>kind: Pod<br>metadata:<br>  name: high-prio<br>spec:<br>  priorityClassName: high-priority<br>  containers:<br>  - name: stress<br>    image: polinux/stress<br>    command: [&quot;stress&quot;]<br>    args: [&quot;--cpu&quot;, &quot;1&quot;, &quot;--vm&quot;, &quot;1&quot;, &quot;--vm-bytes&quot;, &quot;512M&quot;, &quot;--timeout&quot;, &quot;300s&quot;]<br>    resources:<br>      requests:<br>        memory: &quot;200Mi&quot;<br>        cpu: &quot;200m&quot;</pre><p>This Pod receives a priority value of 1000000, making it more important than the low-priority Pods.</p><h3>Understanding Preemption</h3><p>Initially, all Pods can run successfully because sufficient resources are available:</p><pre>Node<br>├── low-prio Pod 1<br>├── low-prio Pod 2<br>├── low-prio Pod 3<br>└── high-prio Pod</pre><p>Next, the high-priority Pod’s memory request is increased:</p><pre>resources:<br>  requests:<br>    memory: &quot;600Mi&quot;</pre><p>After recreating the Pod:</p><pre>kubectl replace -f high-prio.yaml --force</pre><p>the scheduler attempts to place the Pod on a node.</p><p>However, the node no longer has enough available memory.</p><p>At this point, Kubernetes evaluates Pod priorities and determines that the high-priority Pod is more important than the low-priority Pods.</p><p>The scheduler then:</p><ol><li>Selects one or more lower-priority Pods.</li><li>Terminates those Pods.</li><li>Frees the required resources.</li><li>Schedules the high-priority Pod.</li></ol><p>This process is called <strong>preemption</strong>.</p><h3>Observing Preemption</h3><p>During the exercise, output similar to the following appears:</p><pre>low-prio-55c4ff8b4f-g55jh   Terminating</pre><p>This indicates Kubernetes is evicting a lower-priority Pod.</p><p>Shortly afterward:</p><pre>high-prio   Running</pre><p>The high-priority Pod is successfully scheduled using the resources that were reclaimed.</p><h3>Business Use Cases</h3><p>PriorityClass is commonly used for:</p><ul><li>Monitoring platforms (Prometheus)</li><li>Logging systems (Fluentd, Logstash)</li><li>Message brokers (Kafka, RabbitMQ)</li><li>Databases</li><li>Payment processing services</li><li>Security agents</li><li>Critical production applications</li></ul><p>By assigning higher priorities to these workloads, organizations ensure that essential services remain available even when the cluster experiences resource pressure.</p><h3>Key Exam Concept</h3><blockquote><strong><em>PriorityClass defines the importance of Pods within a Kubernetes cluster. During resource shortages, the scheduler prioritizes higher-priority Pods and may preempt lower-priority Pods to ensure critical workloads can be scheduled and remain operational.</em></strong></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e698b86bd38d" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>