<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Manohar Shetty on Medium]]></title>
        <description><![CDATA[Stories by Manohar Shetty on Medium]]></description>
        <link>https://medium.com/@tradingcontentdrive?source=rss-1304b20de7e5------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*bzBE3EFQgkUqFriV1SJN6Q.png</url>
            <title>Stories by Manohar Shetty on Medium</title>
            <link>https://medium.com/@tradingcontentdrive?source=rss-1304b20de7e5------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 00:19:42 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@tradingcontentdrive/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Kubernetes Node Failure: How the Control Plane Detects and Diagnoses Node Issues]]></title>
            <link>https://medium.com/@tradingcontentdrive/kubernetes-node-failure-how-the-control-plane-detects-and-diagnoses-node-issues-cdbd7e6877be?source=rss-1304b20de7e5------2</link>
            <guid isPermaLink="false">https://medium.com/p/cdbd7e6877be</guid>
            <dc:creator><![CDATA[Manohar Shetty]]></dc:creator>
            <pubDate>Mon, 09 Nov 2026 13:51:36 GMT</pubDate>
            <atom:updated>2026-11-09T13:51:36.611Z</atom:updated>
            <content:encoded><![CDATA[<p>What happens when a Kubernetes worker node goes down?</p><p>n Kubernetes, the control plane does not instantly know that a worker node has failed. It detects the failure through missed heartbeats, updates the node’s condition to NotReady or Unknown, and may eventually reschedule affected workloads onto healthy nodes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1JxEcFpJFv9jkRlHUvPPvQ.png" /></figure><p>The important distinction is that detecting a node failure, marking it unhealthy, and replacing its workloads are three separate stages.</p><p>Let’s walk through the complete sequence, from the moment a node fails until Kubernetes recovers the application.</p><h3>1. The components involved</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/705/1*5T3SKO4DkIcvN7vHB6NuZQ.png" /></figure><p>The central component for detecting a failed node is the Node lifecycle controller, which runs inside kube-controller-manager.</p><p>The scheduler does not continuously ping workers. The kubelet does not normally ask the scheduler whether its node is alive. These components have different responsibilities.</p><h3>2. Before the failure: how Kubernetes tracks node health</h3><p>A worker node normally reports its health using two mechanisms.</p><h4>A. Node status updates</h4><p>The kubelet periodically communicates information about the node to the API server. This includes conditions such as:</p><ul><li>Ready</li><li>MemoryPressure</li><li>DiskPressure</li><li>PIDPressure</li><li>NetworkUnavailable</li></ul><p>For example:</p><pre>kubectl get nodes</pre><pre>NAME       STATUS   ROLES    AGE   VERSION<br>master-1   Ready    control-plane   30d   v1.34.8<br>worker-1   Ready    &lt;none&gt;          30d   v1.34.8<br>worker-2   Ready    &lt;none&gt;          30d   v1.34.8</pre><h4>B. Node Lease heartbeats</h4><p>In modern Kubernetes, the kubelet renews a Lease object in the kube-node-lease namespace. Each node normally has a Lease with the same name as the node.</p><p>Check it using:</p><pre>kubectl get leases -n kube-node-lease</pre><p>Inspect a particular worker:</p><pre>kubectl get lease worker-1 -n kube-node-lease -o yaml</pre><p>Look at:</p><pre>spec:<br>  holderIdentity: worker-1<br>  renewTime: &quot;2026-10-25T16:40:00Z&quot;<br>  leaseDurationSeconds: 40</pre><p>The renewTime indicates the last successful renewal recorded in the API. The Lease is a lightweight heartbeat mechanism, so the kubelet does not need to send the entire Node status on every heartbeat.</p><p>A Lease renewal is not proof that every workload on the node is healthy. It primarily helps the control plane determine whether the kubelet is still communicating.</p><h3>3. The exact sequence when a worker node goes down</h3><p>Assume you have three nodes:</p><ul><li>master-1: control plane</li><li>worker-1: running your application Pods</li><li>worker-2: healthy and available for replacement workloads</li></ul><p>At time T=0, worker-1 suddenly loses power.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/627/1*kSVvSJHPC5Ng8f8IQm40gg.png" /></figure><p>The Node status and taint behavior is documented in the official</p><p><a href="https://kubernetes.io/docs/reference/node/node-status">Node Status</a></p><p><a href="https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/?utm_source=chatgpt.com">Taints and Tolerations documentation</a>.</p><p><a href="https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration">Taints and Tolerations</a></p><h3>4. How does the master know the node is down?</h3><p>This is the most important part.</p><p>The control plane does not generally detect a failed worker by repeatedly SSHing into it, pinging its IP address, or checking whether its containers respond.</p><p>Instead, Kubernetes relies primarily on heartbeats recorded through the API server.</p><p>The process is:</p><ol><li>The kubelet on worker-1 periodically renews its Lease.</li><li>The API server persists the updated Lease in etcd.</li><li>The Node lifecycle controller checks node health.</li><li>The controller sees that the Lease has become stale beyond the configured monitoring grace period.</li><li>The controller updates the Node object’s status to indicate that the node is unreachable.</li></ol><p>The key point: the control plane detects the absence of expected communication, not necessarily the physical cause of the failure.</p><p>The same symptom can result from a powered-off VM, a crashed kubelet, a network partition, a firewall problem, or an unavailable API endpoint.</p><p>That is why Kubernetes may report Unknown rather than immediately claiming that the machine is dead.</p><h3>How to verify the heartbeat</h3><pre>kubectl get leases -n kube-node-lease</pre><pre>kubectl get lease worker-1 -n kube-node-lease -o yaml</pre><pre>kubectl get node worker-1 -o yaml</pre><p>Look at the Node condition:</p><pre>status:<br>  conditions:<br>    - type: Ready<br>      status: &quot;Unknown&quot;<br>      reason: NodeStatusUnknown<br>      message: Kubelet stopped posting node status.</pre><p>The precise reason and message can vary by Kubernetes version and failure scenario.</p><p>Also inspect the timestamps:</p><pre>kubectl describe node worker-1</pre><p>Pay attention to Ready, LastHeartbeatTime, LastTransitionTime, and the node events.</p><h3>5. NotReady versus Unknown: what is the difference?</h3><p>These are not interchangeable states.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/621/1*kSgSAHzxkhUTyr3QfLiaaw.png" /></figure><p>The control plane has not received sufficient recent information to determine whether the node is healthy.</p><p>For example, if the kubelet is running but detects a serious local problem, it may report Ready=False.</p><p>If the machine loses power and cannot report anything, the control plane generally transitions the condition to Unknown after the monitoring grace period.</p><p>kubectl get nodes may display both False and Unknown as NotReady. You need kubectl describe node or the Node YAML to distinguish the underlying condition.</p><p>One additional distinction: a manually cordoned node displays SchedulingDisabled in kubectl get nodes. That is not a Node condition and does not mean the machine has failed.</p><h3>6. What happens to the Pods on the failed node?</h3><p>Suppose worker-1 is running these Pods:</p><pre>worker-1<br>├── frontend-7f6d<br>├── frontend-8a2c<br>├── payment-api-5b7c<br>└── nginx-6d4f</pre><p>When the node becomes unreachable, Kubernetes applies a taint that affects existing Pods as well as new scheduling decisions.</p><p>For an unreachable node, the typical taint is:</p><pre>node.kubernetes.io/unreachable:NoExecute</pre><p>NoExecute has two effects:</p><ul><li>It prevents new Pods from being placed on the node unless they tolerate the taint.</li><li>It causes existing Pods without a suitable toleration to become eligible for eviction.</li></ul><p>By default, ordinary Pods receive automatic tolerations for not-ready and unreachable with tolerationSeconds: 300. Therefore, a failed node does not necessarily cause immediate Pod eviction. The default toleration is five minutes, and the detection timeout happens before that period begins.</p><p>DaemonSet Pods normally tolerate these node taints indefinitely, so they are handled differently. See the <a href="https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/?utm_source=chatgpt.com">official</a></p><p><a href="https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/">Taints and Tolerations</a></p><p>Example timeline</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/659/1*FcKzhdPRIXGBAl0YS6CQ-w.png" /></figure><p>These times are illustrative, not guaranteed deadlines. Controller configuration, taint propagation, eviction rate limits, Pod-specific tolerations and cluster-wide failures can change the timeline.<br>7. How are replacement Pods created?</p><p>This is where Kubernetes’ declarative architecture becomes important.</p><p>The scheduler does not independently decide to recreate every Pod that was on the failed node. Instead, workload controllers reconcile the desired state.</p><p>Consider a Deployment:</p><pre>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: frontend<br>spec:<br>  replicas: 3<br>  selector:<br>    matchLabels:<br>      app: frontend<br>  template:<br>    metadata:<br>      labels:<br>        app: frontend<br>    spec:<br>      containers:<br>        - name: nginx<br>          image: nginx:1.27<br>          ports:<br>            - containerPort: 80</pre><p>The Deployment controller ensures that three replicas are desired. The ReplicaSet controller manages the corresponding Pod objects.</p><p>If a Pod is deleted or otherwise ceases to count toward the desired replica count, the controller can create a replacement Pod. The scheduler then assigns an eligible, unscheduled Pod to a healthy node with sufficient resources.</p><p>For example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/636/1*3Zme_f7rMJjpdNkAGCiktw.png" /></figure><p>Replacement placement depends on available CPU and memory, scheduling constraints, affinity rules, taints, topology constraints and other policies. If there is insufficient capacity, the replacement Pods can remain Pending.</p><p>Also, Kubernetes does not automatically guarantee that an application remains available throughout this process. That depends on replica count, distribution across nodes, readiness probes, Service endpoints, storage and application design.</p><h3>8. What if the failed node comes back?</h3><p>Suppose worker-1 was down for two minutes and then reconnects before its Pods are evicted.</p><p>The kubelet can resume renewing its Lease and reporting Node status. Once the controller observes the node as healthy, its Ready condition can return to True, and relevant automatically managed taints can be removed.</p><p>Existing Pods may continue running if they were never terminated and their containers survived. Kubernetes does not necessarily recreate all Pods merely because the node briefly became NotReady.</p><p>If Pods have already been evicted and replacement Pods created elsewhere, recovery is more complicated. The original node may return while replacement workloads are running on other nodes. Stateful workloads, in particular, can require additional coordination around storage attachment and application-level recovery.</p><p>A network partition is a special risk: the original node may still be running application processes even though the control plane cannot communicate with it. Kubernetes cannot safely assume that a disconnected machine has physically stopped.</p><h3>9. Commands to observe the complete process in your cluster</h3><p>Run these from a machine with working cluster access.</p><p>Watch Node status:</p><pre>kubectl get nodes -w</pre><p>Watch Pods across all namespaces:</p><pre>kubectl get pods -A -o wide -w</pre><p>Watch Node and Pod events:</p><pre>kubectl get events -A --sort-by=.metadata.creationTimestamp</pre><p>Inspect a specific node’s conditions and taints:</p><pre>kubectl describe node worker-1</pre><p>Inspect its Lease:</p><pre>kubectl get lease worker-1 -n kube-node-lease -o yaml</pre><p>Inspect the controller-manager configuration and logs (for kubeadm clusters, where the control-plane component runs as a static Pod):</p><pre>kubectl -n kube-system get pods -o wide | grep kube-controller-manager</pre><pre>kubectl -n kube-system logs &lt;controller-manager-pod-name&gt;</pre><p>Controller-manager logs depend on the logging configuration and verbosity; not every health check produces a visible log entry.</p><h3>10. Important differences in real environments</h3><ul><li>Deployment: The ReplicaSet controller can create replacement Pods to maintain the desired replica count.</li><li>StatefulSet: Replacement Pods retain their stable identity, and persistent storage may delay recovery.</li><li>Standalone Pod: There is no Deployment or ReplicaSet to maintain a desired replica count. Do not assume a replacement will be created.</li><li>DaemonSet: Its Pods are managed per eligible node and have special tolerations for node failures.</li><li>Single-node cluster: If the only node fails, there is no healthy worker on which to schedule replacements. The control plane must also remain operational for recovery to proceed.</li><li>Multiple-node or zone-wide failure: Eviction rate limiting and safeguards against widespread connectivity failures can slow or stop mass eviction.</li></ul><p>The behavior is described in the official <a href="https://kubernetes.io/docs/concepts/architecture/nodes/?utm_source=chatgpt.com">Kubernetes Nodes documentation</a> and <a href="https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/?utm_source=chatgpt.com">Taints and Tolerations documentation</a>.</p><p>The full chain to remember is:</p><p>Worker failure → heartbeat stops → Node lifecycle controller detects the missing heartbeat → Node condition becomes Unknown or False → taints are applied → eligible Pods are evicted → workload controllers reconcile desired replicas → scheduler places replacements on eligible healthy nodes.</p><p>The key distinction is that the kubelet reports node health, the Node lifecycle controller detects missed heartbeats, the taint/eviction mechanism handles unhealthy-node workloads, and workload controllers plus the scheduler provide recovery.</p><h3>Conclusion</h3><p>Understanding how Kubernetes detects and diagnoses node failures is essential for maintaining cluster stability and application availability. By monitoring heartbeats, evaluating node conditions, and applying taints, Kubernetes identifies nodes that are no longer healthy or reachable and prevents workloads from being scheduled onto them.</p><p>Knowing how the Node lifecycle controller, kubelet, API server, and scheduler interact makes it easier to troubleshoot node-related issues and understand why Pods become unavailable or get rescheduled. This knowledge helps DevOps engineers identify potential failure points, investigate cluster events, and build more resilient Kubernetes environments.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cdbd7e6877be" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Troubleshooting OpenShift Authentication Errors: Fixing Kubeconfig Context and Restoring Cluster…]]></title>
            <link>https://medium.com/@tradingcontentdrive/troubleshooting-openshift-authentication-errors-fixing-kubeconfig-context-and-restoring-cluster-6363dfa52d27?source=rss-1304b20de7e5------2</link>
            <guid isPermaLink="false">https://medium.com/p/6363dfa52d27</guid>
            <category><![CDATA[redhat-linux]]></category>
            <category><![CDATA[red-hat-openshift]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[openshift]]></category>
            <category><![CDATA[kubernetes-cluster]]></category>
            <dc:creator><![CDATA[Manohar Shetty]]></dc:creator>
            <pubDate>Sat, 31 Oct 2026 06:01:21 GMT</pubDate>
            <atom:updated>2026-10-31T06:01:21.223Z</atom:updated>
            <content:encoded><![CDATA[<h3>Troubleshooting OpenShift Authentication Errors: Fixing Kubeconfig Context and Restoring Cluster Access</h3><h3>Introduction</h3><p>I recently encountered an authentication issue while accessing my OpenShift Single Node OpenShift (SNO) cluster from a bastion server.</p><p>The cluster was already running, and I was using the kubeconfig generated during OpenShift installation. However, when I tried executing basic oc commands such as oc get nodes and oc get co, the commands either hung or returned authentication errors.</p><p>Initially, I suspected that there might be an issue with the API server or the cluster itself. However, after troubleshooting the kubeconfig, I discovered that the problem was related to the authentication user configured in the active context.</p><p>In this blog, I’ll walk through the issue, the troubleshooting steps, the root cause, and how I resolved it without reinstalling or modifying the OpenShift cluster.</p><p>This blog might be useful if you are troubleshooting authentication issues in OpenShift, especially errors such as “the server has asked for the client to provide credentials.” It covers how to inspect kubeconfig users, validate client certificates, extract certificate and private key data, create a new authentication context, and restore administrative access without reinstalling the cluster.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tJldCjcK_NQq_46YgpZHPA.png" /></figure><h3>1. Environment Details</h3><p>The environment used for this troubleshooting:</p><p>ComponentConfigurationPlatformRed Hat OpenShiftInstallation typeSingle Node OpenShift (SNO)OpenShift version4.21.19Kubernetes versionv1.34.8Node hostnamesno.manohar.inAPI endpointapi.sno.manohar.in:6443Access methodBastion serverKubeconfig location/root/sno/auth/kubeconfig</p><p>The cluster was operational, and I wanted to restore administrative access from the bastion server.</p><h3>2. Initial Problem</h3><p>I started by exporting the kubeconfig path.</p><pre>export KUBECONFIG=/root/sno/auth/kubeconfig</pre><p>Then I tried checking the cluster nodes.</p><pre>oc get nodes</pre><p>However, the command did not return any output. I waited for some time and eventually interrupted it using Ctrl+C.</p><p>I also tried checking the cluster operators.</p><pre>oc get co</pre><p>This command also appeared to hang.</p><p>After trying again, I received the following error:</p><pre>E1031 09:48:19.101307    2867 memcache.go:265]<br>&quot;Unhandled Error&quot; err=&quot;couldn&#39;t get current server API group list:<br>the server has asked for the client to provide credentials&quot;</pre><p>This error indicated that the API server was requesting authentication credentials that were not being accepted.</p><p>At this point, I needed to determine whether the problem was with the cluster, the API server, or the kubeconfig.</p><h3>3. Inspecting the Kubeconfig</h3><p>Instead of making changes to the cluster, I decided to inspect the existing kubeconfig.</p><p>I first checked the available users:</p><pre>grep -A10 &#39;users:&#39; /root/sno/auth/kubeconfig</pre><p>The output showed three authentication entries:</p><ul><li>admin</li><li>alex</li><li>david</li></ul><p>The admin user contained two important fields:</p><pre>client-certificate-data: &lt;BASE64_ENCODED_CERTIFICATE&gt;<br>client-key-data: &lt;BASE64_ENCODED_PRIVATE_KEY&gt;</pre><p>The other two users contained token-based authentication:</p><pre>- name: alex/api-sno-manohar-in:6443<br>  user:<br>    token: &lt;TOKEN&gt;<br>- name: david/api-sno-manohar-in:6443<br>  user:<br>    token: &lt;TOKEN&gt;</pre><p>This was an important discovery.</p><p>The kubeconfig contained certificate-based authentication for the admin user, but it also contained separate token-based authentication entries.</p><p>I needed to identify which authentication method the active context was actually using.</p><h3>4. Checking the Current Context</h3><p>I executed the following commands:</p><pre>export KUBECONFIG=/root/sno/auth/kubeconfig<br>oc config current-context<br>oc config view --minify</pre><p>The output was:</p><pre>[root@bastion auth]# export KUBECONFIG=/root/sno/auth/kubeconfig<br>oc config current-context<br>oc config view --minify<br>vault/api-sno-manohar-in:6443/kube:admin<br>apiVersion: v1<br>clusters:<br>- cluster:<br>    insecure-skip-tls-verify: true<br>    server: https://api.sno.manohar.in:6443<br>  name: api-sno-manohar-in:6443<br>contexts:<br>- context:<br>    cluster: api-sno-manohar-in:6443<br>    namespace: vault<br>    user: kube:admin/api-sno-manohar-in:6443<br>  name: vault/api-sno-manohar-in:6443/kube:admin<br>current-context: vault/api-sno-manohar-in:6443/kube:admin<br>kind: Config<br>users:<br>- name: kube:admin/api-sno-manohar-in:6443<br>  user:<br>    token: REDACTED<br>[root@bastion auth]#</pre><p>This output revealed something important.</p><p>My active context was:</p><pre>vault/api-sno-manohar-in:6443/kube:admin</pre><p>The context was configured with:</p><pre>namespace: vault<br>user: kube:admin/api-sno-manohar-in:6443</pre><p>The active context was therefore selecting the token-based kube:admin user, not the certificate-based admin user.</p><p>This explained why inspecting the admin certificate alone would not automatically change the authentication method used by my regular oc commands.</p><h3>5. Testing the Admin Certificate Authentication</h3><p>Before making any changes, I wanted to verify whether the existing admin certificate was still valid.</p><p>I executed:</p><pre>oc --user=admin whoami</pre><p>The output was:</p><pre>system:admin</pre><p>This confirmed that the admin certificate credentials could successfully authenticate against the OpenShift API server.</p><p>Next, I checked the certificate validity dates.</p><pre>openssl x509 -in &lt;(base64 -d &lt;&lt;&lt; &quot;$(oc config view --raw -o jsonpath=&#39;{.users[0].user.client-certificate-data}&#39;)&quot;) -noout -dates</pre><p>Output:</p><pre>notBefore=Jun 23 15:00:20 2026 GMT<br>notAfter=Jun 20 15:00:21 2036 GMT</pre><p>The certificate was valid until June 20, 2036.</p><p>Therefore, the admin certificate was not expired.</p><p>At this point, I had established two important facts:</p><ol><li>The existing admin certificate could authenticate successfully.</li><li>The active context was using a different token-based user.</li></ol><p>This narrowed the issue down to kubeconfig context selection and authentication.</p><h3>6. First Attempt: Creating a New Credential Entry</h3><p>My first attempt was to create a new credential entry named sno-admin.</p><p>I executed:</p><pre>oc config set-credentials sno-admin \<br>  --client-certificate=/root/sno/auth/admin.crt \<br>  --client-key=/root/sno/auth/admin.key</pre><p>The command returned:</p><pre>User &quot;sno-admin&quot; set.</pre><p>However, when I executed:</p><pre>oc get nodes</pre><p>I received the same authentication error:</p><pre>E1031 09:50:46.538988    3785 memcache.go:265]<br>&quot;Unhandled Error&quot; err=&quot;couldn&#39;t get current server API group list:<br>the server has asked for the client to provide credentials&quot;</pre><p>Why did this happen?</p><p>Because creating a new credential entry does not automatically change the active context.</p><p>My existing context was still pointing to:</p><pre>kube:admin/api-sno-manohar-in:6443</pre><p>Additionally, the certificate and key files had not yet been extracted from the original kubeconfig when I first attempted this configuration.</p><p>I needed to extract the actual certificate and private key before configuring the new credentials.</p><h3>7. Extracting the Admin Certificate and Private Key</h3><p>The original kubeconfig stored the certificate and private key in Base64-encoded format.</p><p>Instead of generating new credentials, I extracted the existing credentials.</p><p>First, I extracted the client certificate:</p><pre>oc config view --raw -o jsonpath=&#39;{.users[0].user.client-certificate-data}&#39; | base64 -d &gt; /root/sno/auth/admin.crt</pre><p>Next, I extracted the private key:</p><pre>oc config view --raw -o jsonpath=&#39;{.users[0].user.client-key-data}&#39; | base64 -d &gt; /root/sno/auth/admin.key</pre><p>I then restricted access to the private key:</p><pre>chmod 600 /root/sno/auth/admin.key</pre><p>Now I had two files:</p><pre>/root/sno/auth/admin.crt<br>/root/sno/auth/admin.key</pre><p>These files contained the existing admin client certificate and its corresponding private key.</p><h3>8. Creating a New Admin Credential</h3><p>Now I configured the new user using the extracted certificate and private key.</p><pre>oc config set-credentials sno-admin \<br>  --client-certificate=/root/sno/auth/admin.crt \<br>  --client-key=/root/sno/auth/admin.key</pre><p>Output:</p><pre>User &quot;sno-admin&quot; set.</pre><p>This created a new authentication entry named sno-admin.</p><p>However, creating credentials alone was not enough. I also needed a context that connected the new user to the existing OpenShift cluster.</p><h3>9. Creating a New Context</h3><p>I created a new context using the existing cluster configuration.</p><pre>oc config set-context sno-admin \<br>  --cluster=api-sno-manohar-in:6443 \<br>  --user=sno-admin \<br>  --namespace=default</pre><p>Output:</p><pre>Context &quot;sno-admin&quot; created.</pre><p>This context contained three important configurations:</p><ul><li>Cluster: api-sno-manohar-in:6443</li><li>User: sno-admin</li><li>Namespace: default</li></ul><p>The new context would use certificate-based authentication instead of the previously selected token-based authentication.</p><h3>10. Switching to the New Context</h3><p>Next, I switched the active context.</p><pre>oc config use-context sno-admin</pre><p>Output:</p><pre>Switched to context &quot;sno-admin&quot;.</pre><p>This was the key step.</p><p>Until now, the existing context had continued using the token-based user. By switching to the newly created context, I instructed oc to use the admin certificate credentials.</p><h3>11. Verifying OpenShift Authentication</h3><p>Now it was time to test the configuration.</p><p>First, I checked the authenticated user:</p><pre>oc whoami</pre><p>Output:</p><pre>system:admin</pre><p>Authentication was successful.</p><p>Next, I checked the cluster nodes:</p><pre>oc get nodes</pre><p>Output:</p><pre>NAME             STATUS   ROLES                         AGE    VERSION<br>sno.manohar.in   Ready    control-plane,master,worker   129d   v1.34.8</pre><p>The SNO node was reporting Ready.</p><p>Finally, I checked all cluster operators:</p><pre>oc get co</pre><p>Output:</p><pre>NAME                                       VERSION   AVAILABLE   PROGRESSING   DEGRADED   SINCE   MESSAGE<br>authentication                             4.21.19   True        False         False      13d<br>baremetal                                  4.21.19   True        False         False      129d<br>cloud-controller-manager                   4.21.19   True        False         False      129d<br>cloud-credential                           4.21.19   True        False         False      129d<br>cluster-autoscaler                         4.21.19   True        False         False      129d<br>config-operator                            4.21.19   True        False         False      129d<br>console                                    4.21.19   True        False         False      30d<br>control-plane-machine-set                  4.21.19   True        False         False      129d<br>csi-snapshot-controller                    4.21.19   True        False         False      67d<br>dns                                        4.21.19   True        False         False      13d<br>etcd                                       4.21.19   True        False         False      129d<br>image-registry                             4.21.19   True        False         False      67d<br>ingress                                    4.21.19   True        False         False      129d<br>insights                                   4.21.19   True        False         False      129d<br>kube-apiserver                             4.21.19   True        False         False      129d<br>kube-controller-manager                    4.21.19   True        False         False      129d<br>kube-scheduler                             4.21.19   True        False         False      129d<br>kube-storage-version-migrator              4.21.19   True        False         False      67d<br>machine-api                                4.21.19   True        False         False      129d<br>machine-approver                           4.21.19   True        False         False      129d<br>machine-config                             4.21.19   True        False         False      129d<br>marketplace                                4.21.19   True        False         False      129d<br>monitoring                                 4.21.19   True        False         False      23d<br>network                                    4.21.19   True        False         False      129d<br>node-tuning                                4.21.19   True        False         False      67d<br>olm                                        4.21.19   True        False         False      129d<br>openshift-apiserver                        4.21.19   True        False         False      14d<br>openshift-controller-manager               4.21.19   True        False         False      14d<br>openshift-samples                          4.21.19   True        False         False      129d<br>operator-lifecycle-manager                 4.21.19   True        False         False      129d<br>operator-lifecycle-manager-catalog         4.21.19   True        False         False      129d<br>operator-lifecycle-manager-packageserver   4.21.19   True        False         False      13d<br>service-ca                                 4.21.19   True        False         False      129d<br>storage                                    4.21.19   True        False         False      129d</pre><p>All listed cluster operators were reporting:</p><pre>AVAILABLE   True<br>PROGRESSING False<br>DEGRADED    False</pre><p>I ran the node command once more to confirm:</p><pre>oc get nodes</pre><p>Output:</p><pre>NAME             STATUS   ROLES                         AGE    VERSION<br>sno.manohar.in   Ready    control-plane,master,worker   129d   v1.34.8</pre><p>The issue was resolved.</p><h3>12. What Was the Actual Root Cause?</h3><p>The problem was not caused by an expired admin certificate or an unhealthy OpenShift cluster.</p><p>The original kubeconfig contained multiple authentication entries. However, the active context was configured to use:</p><pre>user: kube:admin/api-sno-manohar-in:6443</pre><p>That user used token-based authentication.</p><p>The same kubeconfig also contained a certificate-based admin user. We verified that this user successfully authenticated as:</p><pre>system:admin</pre><p>The solution was to extract the existing certificate and private key, create a separate credential entry, create a context that referenced those credentials, and switch to that context.</p><p>The important distinction is:</p><p><strong>Creating credentials and selecting credentials are two different operations in Kubernetes.</strong></p><p>oc config set-credentials creates or updates a user entry.</p><p>oc config set-context associates a user with a cluster.</p><p>oc config use-context selects which configuration oc actually uses.</p><p>Missing the final context selection can leave you using the original authentication method.</p><h3>13. Useful Commands for Future Troubleshooting</h3><p>These commands are useful when troubleshooting similar kubeconfig authentication problems.</p><p>PurposeCommandCheck kubeconfig pathecho $KUBECONFIGCheck active contextoc config current-contextView active contextoc config view --minifyList contextsoc config get-contextsList usersoc config get-usersCheck authenticated useroc whoamiTest a specific useroc --user=admin whoamiCheck certificate validityopenssl x509 -in admin.crt -noout -datesSwitch contextoc config use-context sno-adminCheck nodesoc get nodesCheck cluster operatorsoc get co</p><h3>14. Important Security Considerations</h3><p>The kubeconfig contains sensitive authentication material, including client certificates, private keys, and access tokens.</p><p>A few precautions are important:</p><ol><li>Never share the complete kubeconfig in public troubleshooting forums.</li><li>Do not publish Base64-encoded certificate or private-key data. Base64 is encoding, not encryption.</li><li>Restrict private-key permissions using chmod 600.</li><li>Avoid exposing authentication tokens in screenshots or terminal logs.</li><li>Since authentication material was exposed during troubleshooting, treat the exposed credentials as compromised and arrange appropriate credential rotation.</li></ol><p>For production environments, use dedicated service accounts and appropriately scoped permissions for automation instead of distributing cluster-admin credentials.</p><h3>Conclusion</h3><p>This troubleshooting exercise reinforced an important lesson about Kubernetes and OpenShift authentication.</p><p>A valid certificate does not automatically mean that the active kubeconfig context is using it.</p><p>When oc get nodes fails with an authentication error, do not immediately assume that the cluster is unhealthy or that the API server is down.</p><p>Start by checking the current context, identifying the user associated with it, and verifying the available authentication credentials.</p><p>In my case, creating a separate admin credential and switching to a new context resolved the issue without reinstalling OpenShift or making changes to the running SNO cluster.</p><p>The final verification confirmed that the node was Ready and all listed cluster operators were available and not degraded.</p><pre>[root@bastion auth]# oc get nodes<br>NAME             STATUS   ROLES                         AGE    VERSION<br>sno.manohar.in   Ready    control-plane,master,worker   129d   v1.34.8</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6363dfa52d27" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Quick VM Creation on Ubuntu Using Multipass CLI]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/devops-cloud-engineering-hub/quick-vm-creation-on-ubuntu-using-multipass-cli-d81d4132d920?source=rss-1304b20de7e5------2"><img src="https://cdn-images-1.medium.com/max/1672/1*WgnDiBJyQxpvJesMBoRJrw.png" width="1672"></a></p><p class="medium-feed-snippet">Introduction</p><p class="medium-feed-link"><a href="https://medium.com/devops-cloud-engineering-hub/quick-vm-creation-on-ubuntu-using-multipass-cli-d81d4132d920?source=rss-1304b20de7e5------2">Continue reading on devops-cloud-engineering-hub »</a></p></div>]]></description>
            <link>https://medium.com/devops-cloud-engineering-hub/quick-vm-creation-on-ubuntu-using-multipass-cli-d81d4132d920?source=rss-1304b20de7e5------2</link>
            <guid isPermaLink="false">https://medium.com/p/d81d4132d920</guid>
            <category><![CDATA[ansible]]></category>
            <category><![CDATA[ubuntu]]></category>
            <category><![CDATA[ubuntu-server]]></category>
            <category><![CDATA[linux]]></category>
            <category><![CDATA[multipass]]></category>
            <dc:creator><![CDATA[Manohar Shetty]]></dc:creator>
            <pubDate>Tue, 27 Oct 2026 07:45:22 GMT</pubDate>
            <atom:updated>2026-10-27T07:45:40.187Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Onboarding a New User in OpenShift: HTPasswd Authentication, RBAC, and CLI Access]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/@tradingcontentdrive/onboarding-a-new-user-in-openshift-htpasswd-authentication-rbac-and-cli-access-6e19361f9f2c?source=rss-1304b20de7e5------2"><img src="https://cdn-images-1.medium.com/max/1672/1*vaJKAmURwXQBtOycAIi_Sg.png" width="1672"></a></p><p class="medium-feed-snippet">Introduction</p><p class="medium-feed-link"><a href="https://medium.com/@tradingcontentdrive/onboarding-a-new-user-in-openshift-htpasswd-authentication-rbac-and-cli-access-6e19361f9f2c?source=rss-1304b20de7e5------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://medium.com/@tradingcontentdrive/onboarding-a-new-user-in-openshift-htpasswd-authentication-rbac-and-cli-access-6e19361f9f2c?source=rss-1304b20de7e5------2</link>
            <guid isPermaLink="false">https://medium.com/p/6e19361f9f2c</guid>
            <category><![CDATA[openshift]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[kubernetes-cluster]]></category>
            <category><![CDATA[k8s]]></category>
            <category><![CDATA[red-hat-openshift]]></category>
            <dc:creator><![CDATA[Manohar Shetty]]></dc:creator>
            <pubDate>Sun, 18 Oct 2026 10:10:21 GMT</pubDate>
            <atom:updated>2026-10-18T10:10:21.191Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Ansible Vault Password Validation Before Running a Playbook]]></title>
            <link>https://medium.com/@tradingcontentdrive/ansible-vault-password-validation-before-running-a-playbook-5599e7238119?source=rss-1304b20de7e5------2</link>
            <guid isPermaLink="false">https://medium.com/p/5599e7238119</guid>
            <category><![CDATA[ansible-roles]]></category>
            <category><![CDATA[ansible-tutorial]]></category>
            <category><![CDATA[ansible-playbook]]></category>
            <category><![CDATA[ansible-tower]]></category>
            <category><![CDATA[ansible]]></category>
            <dc:creator><![CDATA[Manohar Shetty]]></dc:creator>
            <pubDate>Wed, 14 Oct 2026 16:34:26 GMT</pubDate>
            <atom:updated>2026-10-14T16:34:26.407Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction</h3><p>When working with Ansible, we often need to store sensitive information such as passwords, usernames, and other credentials. Keeping these values in plain text files is risky because anyone who can access those files can read the information.</p><p>Ansible Vault helps solve this problem by encrypting sensitive files with a password. Instead of storing readable content, the file contains encrypted data that can only be viewed or edited after entering the correct Vault password.</p><p>Ansible Vault encrypts sensitive data such as passwords and secret variables. However, running ansible-playbook with --ask-vault-pass does not guarantee that the password will be validated immediately. Ansible validates it when it needs to decrypt Vault-encrypted content.</p><p>In this blog, we will create an encrypted Vault file, verify the password, and configure a custom error message to display when an incorrect password is entered. We will also see how to validate the password before running an Ansible playbook.</p><p>Press enter or click to view image in full size</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/635/0*VD7RUtsagFhvyVvz.png" /></figure><h3>Terraform Ansible Setup</h3><p><a href="https://medium.com/@tradingcontentdrive/ansible-automation-on-aws-complete-setup-nginx-and-qemu-guest-agent-b3b927f4057b">Terraform &amp; Ansible Automation on AWS: Complete Setup, Nginx and QEMU Guest Agent</a></p><h3>Installation of Ansible Tower on Kind Cluster</h3><p><a href="https://medium.com/devops-cloud-engineering-hub/executing-ansible-jobs-using-aap-on-openshift-dbcd9994eaa3">Executing Ansible Jobs using AAP on Openshift</a></p><h3>Installation of Ansible AAP Platform on Openshift</h3><p><a href="https://medium.com/devops-cloud-engineering-hub/deploying-ansible-automation-platform-on-openshift-19bcbb31a9d0">Deploying Ansible Automation Platform on OpenShift</a></p><h3>Environment</h3><pre>┌────────────────────┬───────────────┬────────────────────────┐<br>│ Hostname           │ IP Address    │ Vault Password         │<br>├────────────────────┼───────────────┼────────────────────────┤<br>│ ansible_controller │ 10.20.4.23    │ captain_jack_sparrow   │<br>├────────────────────┼───────────────┼────────────────────────┤<br>│ ansible_client     │ 10.20.4.115   │ captain_jack_sparrow   │<br>└────────────────────┴───────────────┴────────────────────────┘</pre><h3>1. Create an Ansible Vault file</h3><p>Run this command on the Ansible controller:</p><pre>ansible-vault create ansible/vault.yml</pre><p>Enter the Vault password:</p><pre>captain_jack_sparrow</pre><p>Add the following content in ansible/vault.yaml:</p><pre>vault_check: true</pre><p>Save and exit the editor.</p><h3>List the directory</h3><pre>root@ip-10-20-4-23:/home/ansible-user/terraform-ansible-setup# ls<br> README.md                                   ansible       change-root-password-qemu.yaml   nginx_install.yaml   policy.json   testing-dev-1.pem       user-script-controller.sh<br>&#39;Terraform and Ansible AWS Automation.png&#39;   ansible.cfg   main.tf                          ping.yaml            steps.txt     user-script-client.sh   validate-vault.sh<br>root@ip-10-20-4-23:/home/ansible-user/terraform-ansible-setup# ls ansible<br>inventories  vault.yml</pre><p>The file will contain encrypted data rather than readable YAML when viewed with cat.</p><h3>2. Validate the Vault password</h3><p>Run:</p><pre>ansible-vault view ansible/vault.yml</pre><p>Enter the Vault password when prompted.</p><pre>root@ip-10-20-4-23:/home/ansible-user/terraform-ansible-setup# ansible-vault view ansible/vault.yml<br>Vault password:<br>vault_check: true</pre><p>If the password is correct, Ansible decrypts and displays the file contents.</p><p>If the password is incorrect, decryption fails.</p><pre>cat vault.yaml</pre><p>Output:</p><pre>root@ip-10-20-4-23:/home/ansible-user/terraform-ansible-setup# cat ansible/vault.yml<br>$ANSIBLE_VAULT;1.1;AES256<br>61383362396635663965383534643663636632626161643733393635626139636539663037316164<br>6362656138316431343938643237666564396163383439610a643234356638363632653732386464<br>62336365376332343464666463383465653733376266336230633838356461656162313466396538<br>3131343332626530360a376631383635313735623566303232386463636665383065376133393662<br>33393736653833356361353934323538613138343964663431613164373635643336<br>root@ip-10-20-4-23:/home/ansible-user/terraform-ansible-setup#</pre><p>The file ansible/vault.yml is encrypted using Ansible Vault. The $ANSIBLE_VAULT;1.1;AES256 header identifies it as an encrypted Vault file. cat displays the encrypted content, not the original password or YAML values.</p><h3>3. Load the Vault file into the playbook</h3><p>Update ping.yaml to include the encrypted file:</p><pre>---<br>- name: Ping all hosts<br>  hosts: all<br>  gather_facts: false</pre><pre>vars_files:<br>    - ansible/vault.yml<br>  tasks:<br>    - name: Check connectivity<br>      ansible.builtin.ping:</pre><h3>4. Execute the playbook</h3><p>Run:</p><pre>ansible-playbook -i ansible/inventories/inventory.ini ping.yaml --ask-vault-pass</pre><p>Ansible prompts:</p><pre>Vault password:</pre><p>Enter captain_jack_sparrow.</p><p><strong>Expected behavior:</strong></p><ul><li>Correct password: Ansible decrypts the Vault file and executes the playbook.</li><li>Incorrect password: Ansible reports a decryption failure and stops before executing the playbook tasks.</li></ul><p>If you enter the wrong Vault password, Ansible displays a decryption error and stops execution before running the playbook tasks.</p><pre>Vault password:<br>ERROR! Decryption failed (no vault secrets were found that could decrypt) on /home/ansible-user/terraform-ansible-setup/ansible/vault.yml</pre><h3>5. Important limitation</h3><p>The default Ansible Vault error message is:</p><pre>ERROR! Decryption failed (no vault secrets were found that could decrypt)</pre><p>Ansible does not provide a built-in command-line option to replace this message with a custom message such as:</p><pre>Incorrect password, please check it.</pre><p>Custom error handling requires an additional wrapper or custom execution logic.</p><h3>6. Customised Error — Ansible Vault Password Validation Script</h3><p>This script validates the Ansible Vault password and displays a custom error message if decryption fails. It does not execute any playbook.</p><p>Script creation:</p><pre>cat &gt;&gt; ~/.bashrc &lt;&lt;&#39;EOF&#39;<br>ansible-playbook() {<br>    local password_file status arg<br>    local -a args=()<br>    password_file=$(mktemp) || return 1<br>    chmod 600 &quot;$password_file&quot;<br>    trap &#39;rm -f &quot;$password_file&quot;; unset VAULT_PASSWORD&#39; RETURN<br>    read -rsp &quot;Vault password: &quot; VAULT_PASSWORD<br>    echo<br>    printf &#39;%s&#39; &quot;$VAULT_PASSWORD&quot; &gt; &quot;$password_file&quot;<br>    unset VAULT_PASSWORD<br>    if ! ansible-vault view ansible/vault.yml --vault-password-file &quot;$password_file&quot; &gt;/dev/null 2&gt;&amp;1; then<br>        echo &quot;Incorrect password, please check it.&quot;<br>        return 1<br>    fi<br>    for arg in &quot;$@&quot;; do<br>        [[ &quot;$arg&quot; != &quot;--ask-vault-pass&quot; ]] &amp;&amp; args+=(&quot;$arg&quot;)<br>    done<br>    command ansible-playbook &quot;${args[@]}&quot; --vault-password-file &quot;$password_file&quot;<br>    status=$?<br>    return &quot;$status&quot;<br>}<br>EOF<br>source ~/.bashrc</pre><h3>Explanation of each part:</h3><ul><li>ansible-playbook(): Defines a shell function that intercepts your command.</li><li>mktemp: Creates a temporary file to hold the password.</li><li>chmod 600: Restricts file access to the current user.</li><li>read -rsp: Prompts for the Vault password without displaying the typed characters.</li><li>printf: Writes the entered password into the temporary file.</li><li>unset VAULT_PASSWORD: Removes the password variable from the shell.</li><li>ansible-vault view: Tests the password by attempting to decrypt ansible/vault.yml.</li><li>&gt;/dev/null 2&gt;&amp;1: Hides Ansible&#39;s default output and error messages.</li><li>if ! ...: Detects a failed password validation.</li><li>echo &quot;Incorrect password, please check it.&quot;: Displays your custom error.</li><li>rm -f: Deletes the temporary password file when validation fails.</li><li>command ansible-playbook &quot;$@&quot;: Executes the actual Ansible command with the original arguments. command bypasses the shell function to prevent recursion.</li><li>--vault-password-file: Passes the validated password file to Ansible.</li><li>local status=$?: Captures the playbook&#39;s exit status.</li><li>return &quot;$status&quot;: Returns the actual playbook result.</li></ul><p>Why ~/.bashrc matters: It loads the function into your interactive Bash shell. Running source ~/.bashrc activates it immediately without requiring a new terminal.</p><p>Important limitation: This custom validation checks only ansible/vault.yml. Keep that file protected, and avoid running the playbook with --ask-vault-pass, because the function already handles the Vault password prompt.</p><h3>Customized Output:</h3><pre>root@ip-10-20-4-23:/home/ansible-user/terraform-ansible-setup# ansible-playbook -i ansible/inventories/inventory.ini ping.yaml --ask-vault-pass<br>Vault password:<br>Incorrect password, please check it.</pre><p>Correct password: Vault password is correct.</p><p>Incorrect password: Incorrect password, please check it.</p><h3>Conclusion</h3><p>Ansible Vault provides a simple way to protect sensitive information by encrypting files with a password. In this setup, we created a Vault file, verified the password, and added custom validation to display a clear error message when an incorrect password is entered. This also helps prevent the playbook from running when password validation fails.</p><p>For production environments, keep the Vault password secure, restrict access to encrypted files, and avoid storing passwords directly in scripts or configuration files. These precautions help protect credentials while running Ansible automation.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5599e7238119" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Ansible Vault Password Validation Before Running a Playbook]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/devops-cloud-engineering-hub/ansible-vault-password-validation-before-running-a-playbook-eaefbc2dd9b3?source=rss-1304b20de7e5------2"><img src="https://cdn-images-1.medium.com/max/1672/1*wRzawP3XHUIwhLAMPMXtig.png" width="1672"></a></p><p class="medium-feed-snippet">Introduction</p><p class="medium-feed-link"><a href="https://medium.com/devops-cloud-engineering-hub/ansible-vault-password-validation-before-running-a-playbook-eaefbc2dd9b3?source=rss-1304b20de7e5------2">Continue reading on devops-cloud-engineering-hub »</a></p></div>]]></description>
            <link>https://medium.com/devops-cloud-engineering-hub/ansible-vault-password-validation-before-running-a-playbook-eaefbc2dd9b3?source=rss-1304b20de7e5------2</link>
            <guid isPermaLink="false">https://medium.com/p/eaefbc2dd9b3</guid>
            <category><![CDATA[ansible]]></category>
            <category><![CDATA[ansible-roles]]></category>
            <category><![CDATA[ansible-playbook]]></category>
            <category><![CDATA[ansible-tutorial]]></category>
            <category><![CDATA[ansible-tower]]></category>
            <dc:creator><![CDATA[Manohar Shetty]]></dc:creator>
            <pubDate>Wed, 14 Oct 2026 16:24:27 GMT</pubDate>
            <atom:updated>2026-10-14T16:35:24.048Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Installing Virtulization Operator on Openshift 4:21]]></title>
            <link>https://medium.com/@tradingcontentdrive/installing-virtulization-operator-on-openshift-4-21-12887bf240c1?source=rss-1304b20de7e5------2</link>
            <guid isPermaLink="false">https://medium.com/p/12887bf240c1</guid>
            <category><![CDATA[kubernetes-cluster]]></category>
            <category><![CDATA[red-hat-openshift]]></category>
            <category><![CDATA[k8s]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[openshift]]></category>
            <dc:creator><![CDATA[Manohar Shetty]]></dc:creator>
            <pubDate>Tue, 13 Oct 2026 12:06:01 GMT</pubDate>
            <atom:updated>2026-10-13T12:06:01.522Z</atom:updated>
            <content:encoded><![CDATA[<p>OpenShift Virtualization enables you to run and manage virtual machines (VMs) alongside containerized applications on an OpenShift cluster. Built on KubeVirt, it integrates virtualization workloads into the Kubernetes platform, allowing administrators to manage VMs using the OpenShift web console, Kubernetes APIs, and command-line tools.</p><p>In this guide, we will install the OpenShift Virtualization Operator on an OpenShift Single Node OpenShift (SNO) cluster, configure the HyperConverged resource, verify the virtualization components, configure storage for VM disks, and create a Fedora virtual machine.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lzQ0D1vQv-bo3x_hJsC43g.png" /></figure><h3>Prerequisites</h3><p>Before starting, ensure that you have:</p><ul><li>A running OpenShift 4.21 cluster.</li><li>Cluster administrator access to the OpenShift web console or oc CLI.</li><li>Hardware virtualization support enabled and exposed to the OpenShift node.</li><li>Sufficient CPU, memory, and disk capacity for virtualization workloads.</li><li>A storage backend suitable for VM disks</li><li>Network connectivity to pull the required container images.</li></ul><h3>Step 1: Verify the OpenShift Cluster</h3><p>Log in to the OpenShift cluster and check the node status.</p><pre>oc login https://api.&lt;cluster-domain&gt;:6443<br>oc get nodes<br>oc get clusterversion</pre><p>Confirm that the node is Ready and the cluster version is appropriate for your installation.</p><h3>Step 2: Install the OpenShift Virtualization Operator</h3><ol><li>Log in to the OpenShift web console as a cluster administrator.</li><li>Navigate to <strong>Operators → OperatorHub</strong></li><li>Search for <strong>OpenShift Virtualization</strong>.</li><li>Select the operator.</li><li>Click <strong>Install</strong>.</li><li>Select the recommended update channel compatible with your OpenShift release.</li><li>Select the operator installation namespace specified by the installation workflow, typically openshift-cnv.</li><li>Complete the installation.</li></ol><p>Wait for the operator to appear under <strong>Operators → Installed Operators</strong>.</p><h3>Step 3: Create the HyperConverged Resource</h3><p>The HyperConverged resource configures and manages the OpenShift Virtualization components.</p><ol><li>Navigate to <strong>Operators → Installed Operators</strong>.</li><li>Open <strong>OpenShift Virtualization</strong>.</li><li>Select the <strong>HyperConverged</strong> tab.</li><li>Click <strong>Create HyperConverged</strong>.</li><li>Keep the default configuration unless your environment requires specific settings.</li><li>Click <strong>Create</strong>.</li></ol><p>OpenShift Virtualization enables you to run and manage virtual machines (VMs) alongside containerized applications on an OpenShift cluster. Built on KubeVirt, it integrates virtualization workloads into the Kubernetes platform, allowing administrators to manage VMs using the OpenShift web console, Kubernetes APIs, and command-line tools.</p><p>In this guide, we will install the OpenShift Virtualization Operator on an OpenShift Single Node OpenShift (SNO) cluster, configure the HyperConverged resource, verify the virtualization components, configure storage for VM disks, and create a Fedora virtual machine.</p><h3>Prerequisites</h3><p>Before starting, ensure that you have:</p><ul><li>A running OpenShift 4.21 cluster.</li><li>Cluster administrator access to the OpenShift web console or oc CLI.</li><li>Hardware virtualization support enabled and exposed to the OpenShift node.</li><li>Sufficient CPU, memory, and disk capacity for virtualization workloads.</li><li>A storage backend suitable for VM disks.</li><li>Network connectivity to pull the required container images.</li></ul><h3>Step 4: Verify the OpenShift Cluster</h3><p>Log in to the OpenShift cluster and check the node status.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yryJUDagewfjMRlrTpX1OA.png" /></figure><pre>oc login console-openshift-console.apps.sno.manohar.in:6443<br>oc get nodes<br>oc get clusterversion<br><br>NAME             STATUS   ROLES                         AGE    VERSION<br>sno.manohar.in   Ready    control-plane,master,worker   111d   v1.34.8<br><br><br>NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS<br>version   4.21.19   True        False         111d    Cluster version is 4.21.19</pre><p>Confirm that the node is Ready and the cluster version is appropriate for your installation.</p><h3>Step 6: Install the OpenShift Virtualization Operator</h3><ol><li>Log in to the OpenShift web console as a cluster administrator.</li><li>Navigate to <strong>Operators → OperatorHub</strong>.</li><li>Search for <strong>OpenShift Virtualization</strong>.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_b9ZGWBILJac059fjPS6ng.png" /></figure><ol><li>Select the operator.</li><li>Click <strong>Install</strong>.</li><li>Select the recommended update channel compatible with your OpenShift release.</li><li>Select the operator installation namespace specified by the installation workflow, typically openshift-cnv.</li><li>Complete the installation.</li></ol><p>Wait for the operator to appear under <strong>Operators → Installed Operators</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9IzMe-VjtkqPHjZBfA-vtA.png" /></figure><p>Ouput of created pods</p><pre>[root@bastion ~]# oc get pods -n openshift-cnv -w<br>NAME                                                   READY   STATUS    RESTARTS   AGE<br>aaq-operator-79c97dbdb-n4ghs                           1/1     Running   0          6m<br>cdi-operator-6cb4557654-h64cz                          1/1     Running   0          6m1s<br>cluster-network-addons-operator-5769f94bb8-l4w7k       2/2     Running   0          6m1s<br>hco-operator-55989ff5fb-47gjn                          1/1     Running   0          6m2s<br>hco-webhook-674cd58994-fjqd6                           1/1     Running   0          6m2s<br>hostpath-provisioner-operator-5884f9784b-5rvxn         1/1     Running   0          6m<br>hyperconverged-cluster-cli-download-566f558c56-m7bc9   1/1     Running   0          6m2s<br>kubevirt-migration-operator-654787db8c-q9c5s           1/1     Running   0          6m<br>ssp-operator-54c778fff9-trqb5                          1/1     Running   0          6m1s<br>virt-operator-68fc9d5956-dggvw                         1/1     Running   0          6m1s<br>virt-operator-68fc9d5956-fvnhq                         1/1     Running   0          6m1s</pre><h3>Step 7: Create the HyperConverged Resource</h3><p>The HyperConverged resource configures and manages the OpenShift Virtualization components.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YRlRxtkaIceA5FmIUW_Vsg.png" /></figure><ol><li>Navigate to <strong>Operators → Installed Operators</strong>.</li><li>Open <strong>OpenShift Virtualization</strong>.</li><li>Select the <strong>HyperConverged</strong> tab.</li><li>Click <strong>Create HyperConverged</strong>.</li><li>Keep the default configuration unless your environment requires specific settings.</li><li>Click <strong>Create</strong>.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aFmpPj64aVJRx4yypJP2hw.png" /></figure><p>The operator will begin deploying components such as KubeVirt, Containerized Data Importer (CDI), and virtualization networking components.</p><pre>[root@bastion ~]# oc get pods -n openshift-cnv<br>NAME                                                   READY   STATUS    RESTARTS      AGE<br>aaq-operator-79c97dbdb-n4ghs                           1/1     Running   0             77m<br>bridge-marker-pnsh9                                    1/1     Running   0             70m<br>cdi-apiserver-5679d7b9-qd4nd                           1/1     Running   0             70m<br>cdi-deployment-6c67878657-7w5lp                        1/1     Running   0             70m<br>cdi-operator-6cb4557654-h64cz                          1/1     Running   0             77m<br>cdi-uploadproxy-74f6bb6c9c-cz5m2                       1/1     Running   0             70m<br>cluster-network-addons-operator-5769f94bb8-l4w7k       2/2     Running   0             77m<br>hco-operator-55989ff5fb-47gjn                          1/1     Running   0             77m<br>hco-webhook-674cd58994-fjqd6                           1/1     Running   0             77m<br>hostpath-provisioner-csi-szjx9                         4/4     Running   0             49m<br>hostpath-provisioner-operator-5884f9784b-5rvxn         1/1     Running   0             77m<br>hyperconverged-cluster-cli-download-566f558c56-m7bc9   1/1     Running   0             77m<br>kube-cni-linux-bridge-plugin-q96pr                     1/1     Running   0             70m<br>kubemacpool-cert-manager-748bdc4994-9jht8              1/1     Running   0             70m<br>kubemacpool-mac-controller-manager-8bb5c49df-74tm4     2/2     Running   0             70m<br>kubevirt-apiserver-proxy-58b86958db-gbz9g              1/1     Running   0             67m<br>kubevirt-console-plugin-56f784c8c6-kp96w               1/1     Running   0             67m<br>kubevirt-ipam-controller-manager-8584589448-nvdrk      1/1     Running   0             70m<br>kubevirt-ipam-controller-manager-8584589448-pwzn2      1/1     Running   0             70m<br>kubevirt-migration-controller-bb8d9bf8c-vz7gt          1/1     Running   0             70m<br>kubevirt-migration-operator-654787db8c-q9c5s           1/1     Running   0             77m<br>ssp-operator-54c778fff9-trqb5                          1/1     Running   1 (68m ago)   77m<br>virt-api-5cb944f94d-2pswb                              1/1     Running   0             68m<br>virt-controller-5b44c66d9f-kvcwg                       1/1     Running   0             67m<br>virt-exportproxy-7698b9bb89-gxj7f                      1/1     Running   0             67m<br>virt-handler-n5f6t                                     1/1     Running   0             67m<br>virt-launcher-fedora-vm-m4f2b                          3/3     Running   0             36m<br>virt-operator-68fc9d5956-dggvw                         1/1     Running   0             77m<br>virt-operator-68fc9d5956-fvnhq                         1/1     Running   0             77m<br>virt-template-validator-ddc9bb9d7-t2c45                1/1     Running   0             67m</pre><h3>Step 8: Verify the Installation</h3><p>Check the HyperConverged resource:</p><pre>oc get hyperconverged -A<br><br>NAMESPACE       NAME                      AGE<br>openshift-cnv   kubevirt-hyperconverged   71m</pre><p>Check the virtualization pods:</p><pre>oc get pods -n openshift-cnv<br><br>NAME                                                   READY   STATUS    RESTARTS      AGE<br>aaq-operator-79c97dbdb-n4ghs                           1/1     Running   0             77m<br>bridge-marker-pnsh9                                    1/1     Running   0             70m<br>cdi-apiserver-5679d7b9-qd4nd                           1/1     Running   0             70m<br>cdi-deployment-6c67878657-7w5lp                        1/1     Running   0             70m<br>cdi-operator-6cb4557654-h64cz                          1/1     Running   0             77m<br>cdi-uploadproxy-74f6bb6c9c-cz5m2                       1/1     Running   0             70m<br>cluster-network-addons-operator-5769f94bb8-l4w7k       2/2     Running   0             77m<br>hco-operator-55989ff5fb-47gjn                          1/1     Running   0             77m<br>hco-webhook-674cd58994-fjqd6                           1/1     Running   0             77m<br>hostpath-provisioner-csi-szjx9                         4/4     Running   0             49m<br>hostpath-provisioner-operator-5884f9784b-5rvxn         1/1     Running   0             77m<br>hyperconverged-cluster-cli-download-566f558c56-m7bc9   1/1     Running   0             77m<br>kube-cni-linux-bridge-plugin-q96pr                     1/1     Running   0             70m<br>kubemacpool-cert-manager-748bdc4994-9jht8              1/1     Running   0             70m<br>kubemacpool-mac-controller-manager-8bb5c49df-74tm4     2/2     Running   0             70m<br>kubevirt-apiserver-proxy-58b86958db-gbz9g              1/1     Running   0             67m<br>kubevirt-console-plugin-56f784c8c6-kp96w               1/1     Running   0             67m<br>kubevirt-ipam-controller-manager-8584589448-nvdrk      1/1     Running   0             70m<br>kubevirt-ipam-controller-manager-8584589448-pwzn2      1/1     Running   0             70m<br>kubevirt-migration-controller-bb8d9bf8c-vz7gt          1/1     Running   0             70m<br>kubevirt-migration-operator-654787db8c-q9c5s           1/1     Running   0             77m<br>ssp-operator-54c778fff9-trqb5                          1/1     Running   1 (68m ago)   77m<br>virt-api-5cb944f94d-2pswb                              1/1     Running   0             68m<br>virt-controller-5b44c66d9f-kvcwg                       1/1     Running   0             67m<br>virt-exportproxy-7698b9bb89-gxj7f                      1/1     Running   0             67m<br>virt-handler-n5f6t                                     1/1     Running   0             67m<br>virt-launcher-fedora-vm-m4f2b                          3/3     Running   0             36m<br>virt-operator-68fc9d5956-dggvw                         1/1     Running   0             77m<br>virt-operator-68fc9d5956-fvnhq                         1/1     Running   0             77m<br>virt-template-validator-ddc9bb9d7-t2c45                1/1     Running   0             67m</pre><p>Check the KubeVirt resource:</p><pre>oc get kubevirt -n openshift-cnv<br><br>NAME                               AGE   PHASE<br>kubevirt-kubevirt-hyperconverged   71m   Deployed</pre><p>Wait until the required components are running and the HyperConverged resource reports that the installation is available.</p><p>The operator will begin deploying components such as KubeVirt, Containerized Data Importer (CDI), and virtualization networking components.</p><h3>Step 9: Creating a Fedora VM on OpenShift Virtualization Using HostPath Storage</h3><p>OpenShift Virtualization allows you to create and manage virtual machines alongside containerized applications. In this setup, we configure a HostPathProvisioner, create a StorageClass, and deploy a Fedora virtual machine using a Kubernetes YAML manifest.</p><p>The existing cluster has a local-storage StorageClass, but it uses kubernetes.io/no-provisioner, which does not dynamically provision volumes. The configuration below adds a separate hostpath-csi StorageClass for node-local storage and creates a Fedora VM in the openshift-cnv namespace.</p><h3>10. Create the HostPathProvisioner</h3><p>File: hostpath-provisioner.yaml</p><pre>apiVersion: hostpathprovisioner.kubevirt.io/v1beta1<br>kind: HostPathProvisioner<br>metadata:<br>  name: hostpath-provisioner<br>spec:<br>  storagePools:<br>    - name: local<br>      path: /var/hpvolumes<br>  workload:<br>    nodeSelector:<br>      kubernetes.io/hostname: sno.manohar.in</pre><p>Apply the configuration:</p><pre>oc apply -f hostpath-provisioner.yaml</pre><p>The configuration defines a storage pool named local, using /var/hpvolumes on the node sno.manohar.in.</p><p>The HostPathProvisioner manages node-local storage for workloads. It does not create a VM by itself.</p><h3>11. Create the StorageClass</h3><p>File: hostpath-csi.yaml</p><pre>apiVersion: storage.k8s.io/v1<br>kind: StorageClass<br>metadata:<br>  name: hostpath-csi<br>provisioner: kubevirt.io.hostpath-provisioner<br>volumeBindingMode: WaitForFirstConsumer<br>reclaimPolicy: Delete<br>allowVolumeExpansion: false<br>parameters:<br>  storagePool: local</pre><p>Apply the configuration:</p><pre>oc apply -f hostpath-csi.yaml</pre><p>The StorageClass identifies the HostPath provisioner and selects the local storage pool.</p><ul><li>provisioner: Identifies the storage provisioner.</li><li>volumeBindingMode: WaitForFirstConsumer: Delays volume binding until a workload consumes the volume.</li><li>reclaimPolicy: Delete: Specifies the reclaim policy for dynamically provisioned volumes.</li><li>allowVolumeExpansion: false: Disables volume expansion.</li><li>storagePool: local: Selects the storage pool configured in the HostPathProvisioner.</li></ul><p>The existing local-storage StorageClass remains separate. Creating hostpath-csi does not replace it.</p><h3>12. Create the Fedora Virtual Machine</h3><p>File: fedora-vm.yaml</p><pre>apiVersion: kubevirt.io/v1<br>kind: VirtualMachine<br>metadata:<br>  name: fedora-vm<br>  namespace: openshift-cnv<br>spec:<br>  runStrategy: Always<br>  template:<br>    metadata:<br>      labels:<br>        kubevirt.io/domain: fedora-vm<br>    spec:<br>      domain:<br>        cpu:<br>          cores: 2<br>        resources:<br>          requests:<br>            memory: 2Gi<br>        devices:<br>          disks:<br>            - name: rootdisk<br>              disk:<br>                bus: virtio<br>            - name: cloudinitdisk<br>              disk:<br>                bus: virtio<br>          interfaces:<br>            - name: default<br>              masquerade: {}<br>              model: virtio<br>      networks:<br>        - name: default<br>          pod: {}<br>      volumes:<br>        - name: rootdisk<br>          containerDisk:<br>            image: quay.io/containerdisks/fedora:latest<br>        - name: cloudinitdisk<br>          cloudInitNoCloud:<br>            userData: |<br>              #cloud-config<br>              user: fedora<br>              password: FedoraVM-ChangeMe123!<br>              chpasswd:<br>                expire: false</pre><p>Apply the configuration:</p><pre>oc apply -f fedora-vm.yaml</pre><p><strong>Note:</strong> The Fedora VM configuration uses containerDisk, so it does not require a StorageClass or PVC. This approach is suitable for testing but provides ephemeral storage. To retain VM disk data across VM recreation, use a PVC or DataVolume backed by a StorageClass such as hostpath-csi.</p><h3>VM configuration</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/685/1*-6xVwU5jTC39OzaRwmro3A.png" /></figure><p>The VM uses a Fedora container disk image as its root disk. The cloudinitdisk provides initial guest configuration, including the username and password.</p><p><strong>Important:</strong> This VM manifest does not reference hostpath-csi or create a PVC. The root disk is an ephemeral container disk, so changes to the guest disk are not guaranteed to persist across VM recreation.</p><h3>13. Verify the Configuration</h3><p>Check the HostPathProvisioner:</p><pre>oc get hostpathprovisioner<br><br>NAME                   AGE<br>hostpath-provisioner   52m</pre><p>Check the StorageClasses:</p><pre>oc get sc<br><br>NAME            PROVISIONER                        RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE<br>hostpath-csi    kubevirt.io.hostpath-provisioner   Delete          WaitForFirstConsumer   false                  49m</pre><p>Check the VM and its instance:</p><pre>oc get vm,vmi -n openshift-cnv<br><br>NAME                                   AGE   STATUS    READY<br>virtualmachine.kubevirt.io/fedora-vm   40m   Running   True<br><br>NAME                                           AGE   PHASE     IP             NODENAME         READY<br>virtualmachineinstance.kubevirt.io/fedora-vm   40m   Running   10.128.0.145   sno.manohar.in   True</pre><p>Check the VM-related pods:</p><pre>oc get pods -n openshift-cnv | grep fedora<br>virt-launcher-fedora-vm-m4f2b                          3/3     Running   0             40m</pre><h3>Conclusion</h3><p>In this guide, we covered the complete process of installing the OpenShift Virtualization Operator on an OpenShift 4.21 cluster, creating the HyperConverged resource, and verifying the deployment of virtualization components. We then configured HostPathProvisioner and a dedicated hostpath-csi StorageClass for node-local storage, created a Fedora virtual machine using a YAML manifest, and configured its CPU, memory, networking, and cloud-init settings.</p><p>This walkthrough demonstrates how OpenShift Virtualization integrates virtual machines into the Kubernetes platform and how virtualization resources can be managed through the OpenShift web console and oc CLI. For VMs that require persistent disk data, ensure the VM uses a PVC or DataVolume backed by the configured storage provisioner rather than an ephemeral container disk.</p><p>These steps provide a foundation for building and experimenting with virtual machine workloads in an OpenShift SNO lab.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=12887bf240c1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Add a Windows Server 2022 Worker Node to a Kubernetes Cluster (Ubuntu + Windows)]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/devops-cloud-engineering-hub/how-to-add-a-windows-server-2022-worker-node-to-a-kubernetes-cluster-ubuntu-windows-5ae90eab8e3c?source=rss-1304b20de7e5------2"><img src="https://cdn-images-1.medium.com/max/635/0*uJZr_2qymLfDd1NN.png" width="635"></a></p><p class="medium-feed-snippet">Introduction</p><p class="medium-feed-link"><a href="https://medium.com/devops-cloud-engineering-hub/how-to-add-a-windows-server-2022-worker-node-to-a-kubernetes-cluster-ubuntu-windows-5ae90eab8e3c?source=rss-1304b20de7e5------2">Continue reading on devops-cloud-engineering-hub »</a></p></div>]]></description>
            <link>https://medium.com/devops-cloud-engineering-hub/how-to-add-a-windows-server-2022-worker-node-to-a-kubernetes-cluster-ubuntu-windows-5ae90eab8e3c?source=rss-1304b20de7e5------2</link>
            <guid isPermaLink="false">https://medium.com/p/5ae90eab8e3c</guid>
            <category><![CDATA[windows]]></category>
            <category><![CDATA[ubuntu]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[k8s]]></category>
            <category><![CDATA[openshift]]></category>
            <dc:creator><![CDATA[Manohar Shetty]]></dc:creator>
            <pubDate>Mon, 12 Oct 2026 15:42:30 GMT</pubDate>
            <atom:updated>2026-10-12T18:07:21.881Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[How to Add a Windows Server 2022 Worker Node to a Kubernetes Cluster (Ubuntu + Windows)]]></title>
            <link>https://medium.com/devops-cloud-engineering-hub/how-to-add-a-windows-server-2022-worker-node-to-a-kubernetes-cluster-ubuntu-windows-dc4f56e24a2b?source=rss-1304b20de7e5------2</link>
            <guid isPermaLink="false">https://medium.com/p/dc4f56e24a2b</guid>
            <category><![CDATA[openshift]]></category>
            <category><![CDATA[windows]]></category>
            <category><![CDATA[ubuntu]]></category>
            <category><![CDATA[k8s]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Manohar Shetty]]></dc:creator>
            <pubDate>Mon, 12 Oct 2026 15:39:49 GMT</pubDate>
            <atom:updated>2026-10-12T18:07:03.975Z</atom:updated>
            <content:encoded><![CDATA[<h4>Adding hybrid worker nodes for kubernetes Cluster</h4><h3>Introduction</h3><p>Let’s imagine two developers, David and Alex, working in the same company. David is working on an application which runs on Linux containers, while Alex is developing an application using .NET Framework and some Windows-specific components. Alex’s application needs Windows containers to run properly.</p><p>Now the problem is, David can run his application on Linux worker nodes, but Alex cannot run his Windows containers directly on those Linux nodes. So, how can we run both applications in the same Kubernetes cluster without maintaining two separate clusters?</p><p>This is where Windows worker nodes comes into the picture.</p><p>Kubernetes allows us to run both Linux and Windows workloads in the same cluster. We can have an Ubuntu machine as our control-plane node and use a Windows Server 2022 machine as a Windows worker node. Linux containers will run on Linux nodes and Windows containers will run on Windows nodes, while Kubernetes manages everything from one place.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IM2QycS2sHPZZRrPwG5vFA.png" /></figure><p>In this blog, we will see how to configure a Windows Server 2022 machine and join it to an existing Kubernetes cluster running on Ubuntu. We will also look at the container runtime, node configuration, networking, and how to verify that our Windows worker node is successfully joined to the cluster.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/660/1*cPmHuAGuKmwnHs04L2mRwQ.png" /></figure><h4>Ubuntu Control Plane</h4><p>The Ubuntu VM acts as the Kubernetes control-plane node, hosting the API server and other control-plane components. It manages the cluster and coordinates workloads running on the Windows Server 2022 worker node. Calico provides cluster networking, with the pod network configured as 10.244.0.0/16.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/680/1*YngthSgabPhYeqGfIVh7MA.png" /></figure><h4>Windows Worker Node</h4><p>The Windows Server 2022 VM runs as a worker node in the Kubernetes cluster hosted on Proxmox. It uses containerd as its container runtime and Calico with the Windows HNS dataplane for networking. The worker connects to the Ubuntu control-plane through the Kubernetes API server at 192.168.0.151:6443.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/603/1*3PFoj0CkhfHgQlQBZRJYJw.png" /></figure><p>Let’s begin by downloading the Windows Server 2022 ISO from the official Microsoft website. We will use this ISO to create our Windows worker node in Proxmox.</p><p>Download: <a href="https://www.microsoft.com/en-us/evalcenter/download-windows-server-2022">Windows Server 2022 Evaluation ISO — Microsoft Evaluation Center</a></p><h3>1. Download the Windows Server 2022 ISO</h3><p>Download the Windows Server 2022 Evaluation ISO from Microsoft’s official website:</p><p><a href="https://www.microsoft.com/en-us/evalcenter/download-windows-server-2022">Windows Server 2022 | Microsoft Evaluation Center</a></p><h4>You can refer the blog:</h4><p><a href="https://medium.com/@tradingcontentdrive/how-to-install-windows-11-on-proxmox-tpm-2-0-secure-boot-virtio-setup-guide-abfc2f7df3d3">How to Install Windows 11 on Proxmox (TPM 2.0, Secure Boot &amp; VirtIO Setup Guide)</a></p><p>Upload it after downloading it, The Windows Server 2022 ISO is already available. No download is required. Next, upload it to Proxmox under Node lab → local → ISO Images if it is not already there.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/710/1*hInoqeqEaJmVbmpddokIZw.png" /></figure><h3>2. Download the VirtIO Driver ISO</h3><p>Open the official Fedora VirtIO Windows driver download page:</p><p><a href="https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/">Making sure you&#39;re not a bot!</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/1*YM3fRU1tJFXQRblbbAfo1Q.png" /></figure><p>You may need to download the VirtIO Windows driver ISO. In this lab, the downloaded file is named:</p><p>virtio-win-new.iso</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oHKBv8TEGO3UbnhncQdXag.png" /></figure><p>Run on the Proxmox host to download the VirtIO ISO:</p><pre>cd /var/lib/vz/template/iso &amp;&amp; wget -O virtio-win-new.iso https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/stable-virtio/virtio-win.iso</pre><h3>3. Upload the Windows ISO to Proxmox</h3><ol><li>Log in to the Proxmox web console at <a href="https://192.168.0.150:8006.">https://192.168.0.150:8006.</a></li><li>Select <strong>Datacenter → lab → local (lab) → ISO Images</strong>.</li><li>Click <strong>Upload</strong>.</li><li>Select the Windows Server 2022 ISO and upload it.</li></ol><p>The VirtIO ISO is already available on the Proxmox host at:</p><p>/var/lib/vz/template/iso/virtio-win-new.iso</p><h3>4. Create the Windows Virtual Machine</h3><p>In the Proxmox console, click <strong>Create VM</strong> and configure the following settings.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/610/1*TESBgt_xLEYQdny_SFIvGw.png" /></figure><h3>5. Mount the VirtIO ISO</h3><p>The VirtIO ISO was mounted as a CD-ROM on the existing Windows VM using the following Proxmox configuration:</p><pre>qm set 105 --ide0 local:iso/virtio-win-new.iso,media=cdrom</pre><p>This attaches the driver ISO to VM 105 through the ide0 CD-ROM device.</p><p>The ISO provides drivers that Windows Setup may need if the virtual disk or another VirtIO device is not detected automatically.</p><p>For Our VM 105, the configuration uses a VirtIO SCSI single controller. The steps are:</p><ol><li>Attach virtio-win-new.iso to the VM as a CD-ROM — already done.</li><li>Boot from the Windows Server 2022 ISO.</li><li>At “Where do you want to install Windows?”, if no disk appears, click Load driver → Browse.</li><li>Open the VirtIO CD-ROM and navigate to:</li></ol><p>vioscsi\2k22\amd64</p><ol><li>Select the compatible driver and click Next.</li><li>Once the virtual disk appears, select it and continue Windows installation.</li></ol><p>The exact driver directory can vary with the VirtIO ISO version. If the disk is already visible, you do not need to load a storage driver manually.</p><p>No additional Windows configuration changes are needed beforehand to make the disk visible when the correct driver is loaded during installation.</p><h3>6. Configure the Network Adapter</h3><p>The VM’s network adapter was configured as follows:</p><pre>qm set 105 --net0 e1000,bridge=vmbr0,firewall=1</pre><p>This sets the Intel E1000 network adapter, connects it to vmbr0, and enables the Proxmox firewall for that interface.</p><h3>7. Install Windows Server 2022</h3><ol><li>Attach the Windows Server 2022 ISO as a CD-ROM.</li><li>Start the VM and open its Proxmox console.</li><li>Boot into Windows Setup.</li><li>Select the Windows Server 2022 edition.</li><li>Choose <strong>Custom installation</strong>.</li><li>Select the virtual disk.</li></ol><p>If the disk is not visible, select <strong>Load driver</strong> and browse the VirtIO CD-ROM. For a VirtIO SCSI disk, the driver is typically located in a directory similar to vioscsi\2k22\amd64, depending on the ISO version.</p><p>Continue the Windows installation.</p><h3>8. Enable RDP through Server Manager</h3><p>After installing Windows Server 2022 on your Proxmox VM:</p><ol><li>Search &amp; Open Server Manager.</li><li>Select Local Server from the left panel.</li><li>Locate Remote Desktop; its current status may show Disabled.</li><li>Click Disabled.</li><li>In the System Properties window, select Allow remote connections to this computer.</li><li>If prompted, confirm the firewall exception.</li><li>Click Apply, then OK.</li></ol><h3>Verify Windows Firewall</h3><p>In Server Manager, confirm that Remote Desktop is enabled. Windows normally configures the required firewall rules when you enable RDP through System Properties.</p><h3>Connect from another computer</h3><p>Use the Windows VM’s IP address, 192.168.0.225.</p><p>From Windows, open Remote Desktop Connection (mstsc.exe), enter the IP address, and connect using the Windows account credentials.</p><p>Note: Ensure TCP port 3389 is allowed through any applicable network firewall.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LrCXz1jF3LujzxOCfU_LBA.png" /></figure><h3>After enabling, connnect it locally</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/395/1*K1WvY819KoCQRkmxKrd-VQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*V0j4kO-p1j6A4QPtw2D4Eg.png" /></figure><h3>Install Prerequisites on Windows Before Joining to the Cluster</h3><p>Before joining the Windows Server 2022 worker to the existing Kubernetes cluster, install the container runtime and prepare the Kubernetes components.</p><h3>9. Install containerd on Windows Server 2022</h3><p>Open PowerShell as Administrator on the Windows worker VM.</p><p>Download the containerd installation script:</p><pre>Invoke-WebRequest https://raw.githubusercontent.com/kubernetes-sigs/sig-windows-tools/master/hostprocess/Install-Containerd.ps1 -OutFile C:\Install-Containerd.ps1</pre><p>Install containerd version 1.7.22 with the hypervisor support check skipped:</p><pre>.\Install-Containerd.ps1 -ContainerDVersion 1.7.22 -SkipHypervisorSupportCheck</pre><p>Verify that the containerd service is running:</p><pre>Get-Service containerd</pre><h3>10. Download Kubernetes Windows Components</h3><p>Download the Windows PrepareNode.ps1 preparation script:</p><pre>curl.exe -LO https://raw.githubusercontent.com/kubernetes-sigs/sig-windows-tools/master/hostprocess/PrepareNode.ps1</pre><p>Download kubeadm.exe version v1.34.12:</p><pre>curl.exe -fL https://dl.k8s.io/v1.34.12/bin/windows/amd64/kubeadm.exe -o C:\k\kubeadm.exe</pre><p>Verify that the Kubernetes executables exist:</p><pre>Get-Item C:\k\kubelet.exe,C:\k\kubeadm.exe</pre><h3>11. Prepare the Windows Kubernetes Worker</h3><p>Run the Windows node preparation script for Kubernetes v1.34.12:</p><pre>.\PrepareNode.ps1 -KubernetesVersion v1.34.12</pre><p>Verify the kubelet service:</p><pre>Get-Service kubelet</pre><p>This prepares the Windows machine for joining the existing Kubernetes cluster.</p><h3>12. Verify Connectivity to the Control Plane</h3><p>The Ubuntu control-plane IP address is 192.168.0.151, and the Kubernetes API server listens on port 6443.</p><p>From Windows PowerShell, verify connectivity:</p><pre>Test-NetConnection 192.168.0.151 -Port 6443</pre><p>The connection should succeed before joining the worker to the cluster.</p><h3>13. Install and Configure Calico for Windows</h3><p>The cluster uses Calico v3.33.0 for networking. The Windows worker requires Windows-compatible Calico components and the Host Networking Service (HNS) dataplane configuration.</p><p>The containerd installation script can also be downloaded explicitly to C:\Install-Containerd.ps1 using:</p><pre>Invoke-WebRequest https://raw.githubusercontent.com/kubernetes-sigs/sig-windows-tools/master/hostprocess/Install-Containerd.ps1 -OutFile C:\Install-Containerd.ps1</pre><p>The cluster’s Calico configuration was applied on Ubuntu, including the Windows HNS dataplane, pod CIDR 10.244.0.0/16, service CIDR 10.96.0.0/12, and strict IPAM affinity.</p><p>After configuration, the Windows Calico pod reached 2/2 Running, and the Windows node reached Ready.</p><h3>Create a Kubernetes Cluster on Ubuntu Using kubeadm and Calico</h3><h3>1. Introduction</h3><p>In this section, we will create a Kubernetes cluster on Ubuntu using kubeadm and configure Calico as the Container Network Interface (CNI).</p><p>The Ubuntu machine acts as the Kubernetes control-plane node. Once the cluster is initialized, Calico provides pod networking, allowing Kubernetes workloads to communicate across the cluster.</p><p><strong>Lab configuration</strong></p><p>ComponentConfigurationOperating systemUbuntuKubernetes versionv1.34.xContainer runtimecontainerdCluster initialization toolkubeadmPod CIDR10.244.0.0/16CNICalico v3.33.0</p><h3>2. Update Ubuntu and Install Prerequisites</h3><p>Update the APT package index and install the packages required to configure the Kubernetes repository and container runtime.</p><pre>sudo apt-get update<br>sudo apt-get install -y apt-transport-https ca-certificates curl gpg containerd</pre><p><strong>Command explanation</strong></p><ul><li>apt-get update refreshes the local package index.</li><li>apt-transport-https supports HTTPS package repository access.</li><li>ca-certificates provides trusted certificate authorities for TLS connections.</li><li>curl downloads files from remote URLs.</li><li>gpg provides GNU Privacy Guard tools used in repository key management.</li><li>containerd installs the container runtime used to run Kubernetes containers.</li><li>-y automatically confirms package installation prompts.</li></ul><h3>3. Configure the Kubernetes APT Repository</h3><p>Create the directory for repository signing keys:</p><pre>sudo install -m 0755 -d /etc/apt/keyrings</pre><p>This creates /etc/apt/keyrings if it does not exist. The -m 0755 option sets directory permissions.</p><p>Download the Kubernetes repository signing key:</p><pre>sudo curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.34/deb/Release.key -o /etc/apt/keyrings/kubernetes-apt-keyring.asc</pre><p><strong>Command explanation</strong></p><ul><li>-f makes curl fail on HTTP errors.</li><li>-sS suppresses normal progress output while displaying errors.</li><li>-L follows HTTP redirects.</li><li>-o specifies the destination file.</li></ul><p>The signing key allows APT to verify packages from the Kubernetes repository.</p><p>Add the Kubernetes v1.34 repository:</p><pre>echo &#39;deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.asc] https://pkgs.k8s.io/core:/stable:/v1.34/deb/ /&#39; | sudo tee /etc/apt/sources.list.d/kubernetes.list</pre><p>This command writes the repository configuration to /etc/apt/sources.list.d/kubernetes.list. The signed-by option tells APT which key to use when verifying packages from this repository.</p><p>Refresh the package index:</p><pre>sudo apt-get update</pre><p>APT can now retrieve package information from the Kubernetes v1.34 repository.</p><h3>4. Install Kubernetes Components</h3><p>Install kubelet, kubeadm, and kubectl:</p><pre>sudo apt-get install -y kubelet kubeadm kubectl</pre><p>These components have different responsibilities:</p><ul><li><strong>kubelet:</strong> Runs on each node and manages its pods and containers.</li><li><strong>kubeadm:</strong> Initializes the control-plane and joins additional nodes to the cluster.</li><li><strong>kubectl:</strong> Provides the command-line interface for communicating with the Kubernetes API server.</li></ul><p>Prevent automatic package upgrades from changing these components:</p><pre>sudo apt-mark hold kubelet kubeadm kubectl</pre><p>The hold setting keeps these packages at their installed versions until the hold is removed.</p><p>Enable containerd and start it immediately:</p><pre>sudo systemctl enable --now containerd</pre><ul><li>enable configures containerd to start at boot.</li><li>--now starts the service immediately</li></ul><p>Enable kubelet at boot:</p><pre>sudo systemctl enable kubelet</pre><p>Kubelet may restart or report errors before the cluster is initialized. This is expected during initial setup because it does not yet have a complete cluster configuration.</p><h3>5. Disable Swap</h3><p>Kubernetes kubelet generally requires swap to be disabled unless the cluster is explicitly configured to support swap.</p><p>Disable swap immediately:</p><pre>swapoff -a</pre><p>This disables active swap devices and files until the next boot.</p><p>To prevent swap entries from being activated again at boot, comment out matching entries in /etc/fstab:</p><pre>sed -i &#39;/\sswap\s/s/^/#/&#39; /etc/fstab</pre><p>The sed expression finds lines containing swap as a separate whitespace-delimited field and adds # at the beginning of those lines.</p><h3>6. Load Required Kernel Modules</h3><p>Kubernetes networking and container operations require specific Linux kernel modules.</p><p>Load the modules immediately:</p><pre>modprobe overlay<br>modprobe br_netfilter</pre><ul><li>overlay provides the overlay filesystem functionality used by container storage.</li><li>br_netfilter enables bridged network traffic to be processed by netfilter</li></ul><p>Configure the modules to load automatically at boot:</p><pre>cat &lt;&lt;EOF &gt; /etc/modules-load.d/k8s.conf<br>overlay<br>br_netfilter<br>EOF</pre><p>This creates /etc/modules-load.d/k8s.conf with the two module names. The system loads them during subsequent boots.</p><h3>7. Configure Linux Networking Parameters</h3><p>Create the Kubernetes sysctl configuration file:</p><pre>cat &lt;&lt;EOF &gt; /etc/sysctl.d/k8s.conf<br>net.bridge.bridge-nf-call-iptables = 1<br>net.bridge.bridge-nf-call-ip6tables = 1<br>net.ipv4.ip_forward = 1<br>EOF</pre><p><strong>Explanation</strong></p><ul><li>net.bridge.bridge-nf-call-iptables = 1 allows bridged IPv4 traffic to be processed by iptables.</li><li>net.bridge.bridge-nf-call-ip6tables = 1 enables corresponding processing for bridged IPv6 traffic.</li><li>net.ipv4.ip_forward = 1 enables IPv4 packet forwarding, which is needed for routing between network interfaces.</li></ul><p>Apply the configuration:</p><pre>sysctl --system</pre><p>This reloads sysctl settings from the system configuration files, including /etc/sysctl.d/k8s.conf.</p><h3>8. Configure containerd</h3><p>Create the containerd configuration directory:</p><pre>mkdir -p /etc/containerd</pre><p>Generate the default containerd configuration:</p><pre>containerd config default &gt; /etc/containerd/config.toml</pre><p>This writes the default containerd configuration to /etc/containerd/config.toml.</p><p>Configure containerd to use systemd cgroups:</p><pre>sed -i &#39;s/SystemdCgroup = false/SystemdCgroup = true/&#39; /etc/containerd/config.toml</pre><p>The sed command replaces SystemdCgroup = false with SystemdCgroup = true in the configuration file.</p><p>Using systemd for cgroup management aligns containerd with kubelet’s systemd cgroup driver configuration.</p><p>Restart containerd to apply the configuration:</p><pre>systemctl restart containerd<br>systemctl enable containerd</pre><p>The first command restarts containerd. The second ensures that the service starts automatically at boot.</p><h3>9. Initialize the Kubernetes Control Plane</h3><p>Initialize the Kubernetes cluster:</p><pre>kubeadm init --pod-network-cidr=10.244.0.0/16</pre><p><strong>Command explanation</strong></p><ul><li>kubeadm init initializes the Kubernetes control-plane.</li><li>--pod-network-cidr=10.244.0.0/16 specifies the IP address range reserved for pod networking.</li></ul><p>The selected pod CIDR must be consistent with the networking configuration used by the CNI plugin. In this lab, Calico uses 10.244.0.0/16.</p><p>After successful initialization, kubeadm configures the control-plane components and generates the cluster’s administrative kubeconfig.</p><h3>10. Configure kubectl</h3><p>Create the kubeconfig directory for the current user:</p><pre>mkdir -p $HOME/.kube</pre><p>Copy the administrative kubeconfig:</p><pre>cp -i /etc/kubernetes/admin.conf $HOME/.kube/config</pre><p>This provides kubectl with the API server address and authentication information needed to manage the cluster.</p><p>Set ownership so the current user can access the file:</p><pre>chown $(id -u):$(id -g) $HOME/.kube/config</pre><p>Here:</p><ul><li>id -u returns the current user&#39;s numeric UID.</li><li>id -g returns the current user&#39;s primary group ID.</li><li>chown assigns ownership of the kubeconfig file to that user and group.</li></ul><p>These commands assume they are run as root or in an appropriately privileged shell. When running them as an ordinary user, use sudo where necessary.</p><h3>11. Download the Calico Operator</h3><p>Calico provides pod networking for the Kubernetes cluster. Download the Calico v3.33.0 operator manifest:</p><pre>curl -fL https://raw.githubusercontent.com/projectcalico/calico/v3.33.0/manifests/tigera-operator.yaml -o tigera-operator.yaml</pre><p><strong>Command explanation</strong></p><ul><li>curl retrieves the manifest from the Calico GitHub repository.</li><li>-f fails on HTTP errors.</li><li>-L follows redirects.</li><li>-o tigera-operator.yaml saves the downloaded manifest to a local file.</li></ul><p>The manifest defines the resources needed to deploy the Tigera operator, which manages the Calico installation.</p><h3>12. Create the Calico Configuration</h3><p>Create custom-resources.yaml:</p><pre>cat &lt;&lt;&#39;EOF&#39; &gt; custom-resources.yaml<br>apiVersion: operator.tigera.io/v1<br>kind: Installation<br>metadata:<br>  name: default<br>spec:<br>  calicoNetwork:<br>    ipPools:<br>    - blockSize: 26<br>      cidr: 10.244.0.0/16<br>      encapsulation: VXLAN<br>      natOutgoing: Enabled<br>      nodeSelector: all()<br>---<br>apiVersion: operator.tigera.io/v1<br>kind: APIServer<br>metadata:<br>  name: default<br>spec: {}<br>EOF</pre><p>This file defines two Calico resources.</p><p><strong>Installation resource</strong></p><ul><li>blockSize: 26 divides the configured IP pool into /26 allocation blocks.</li><li>cidr: 10.244.0.0/16 defines the pod IP address pool.</li><li>encapsulation: VXLAN enables VXLAN encapsulation for pod traffic.</li><li>natOutgoing: Enabled enables source NAT for outgoing traffic from the Calico IP pool.</li><li>nodeSelector: all() selects all applicable nodes for this IP pool.</li></ul><p><strong>APIServer resource</strong></p><p>The APIServer resource named default requests deployment of the Calico API server using the operator&#39;s default settings.</p><h3>13. Apply the Calico Configuration</h3><p>Apply the custom resources:</p><pre>kubectl apply -f custom-resources.yaml</pre><p>This submits the Installation and APIServer resources to the Kubernetes API server. The Tigera operator reconciles them and deploys the corresponding Calico components.</p><p><strong>Important:</strong> The supplied commands create the custom resources but do not apply tigera-operator.yaml. The operator manifest must also be applied for the operator to manage the installation.</p><h3>14. Verify Calico Status</h3><p>Check the Calico installation status:</p><pre>kubectl get tigerastatus</pre><p>This displays the health and availability of Calico components managed by the operator.</p><p>Check all cluster pods and their node placement:</p><pre>kubectl get pods -A -o wide</pre><p>This command lists pods across all namespaces and displays their status, IP addresses, and assigned nodes. Use it to check whether the networking components have started successfully.</p><p>The Ubuntu control-plane is initialized using kubeadm, containerd is configured to use systemd cgroups, and Calico is configured with the 10.244.0.0/16 pod network and VXLAN encapsulation.</p><p>This establishes the Linux control-plane and networking foundation for the lab. The Windows Server 2022 worker requires additional Windows-specific preparation and Calico HNS configuration before Windows workloads can use cluster networking.</p><p>Cluster Creation Script</p><pre>sudo apt-get install -y kubelet kubeadm kubectl<br>sudo apt-mark hold kubelet kubeadm kubectl<br>sudo systemctl enable --now containerd<br>sudo systemctl enable kubelet<br><br>=============<br>swapoff -a<br>sed -i &#39;/\sswap\s/s/^/#/&#39; /etc/fstab<br><br>modprobe overlay<br>modprobe br_netfilter<br><br>cat &lt;&lt;EOF &gt; /etc/modules-load.d/k8s.conf<br>overlay<br>br_netfilter<br>EOF<br><br>cat &lt;&lt;EOF &gt; /etc/sysctl.d/k8s.conf<br>net.bridge.bridge-nf-call-iptables = 1<br>net.bridge.bridge-nf-call-ip6tables = 1<br>net.ipv4.ip_forward = 1<br>EOF<br><br>sysctl --system<br><br>mkdir -p /etc/containerd<br>containerd config default &gt; /etc/containerd/config.toml<br>sed -i &#39;s/SystemdCgroup = false/SystemdCgroup = true/&#39; /etc/containerd/config.toml<br><br>systemctl restart containerd<br>systemctl enable containerd<br><br>kubeadm init --pod-network-cidr=10.244.0.0/16<br><br>mkdir -p $HOME/.kube<br>cp -i /etc/kubernetes/admin.conf $HOME/.kube/config<br>chown $(id -u):$(id -g) $HOME/.kube/config</pre><p>Cluster status</p><pre>NAME     STATUS   ROLES           AGE   VERSION    INTERNAL-IP     EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION      CONTAINER-RUNTIME<br>ubuntu   NotReady    control-plane   70s   v1.34.12   192.168.0.151   &lt;none&gt;        Ubuntu 24.04.5 LTS   6.8.0-146-generic   containerd://2.2.1</pre><p><strong>Explanation:</strong> After initializing the Kubernetes control-plane, the node may remain in the NotReady state until a Container Network Interface (CNI) plugin is installed and configured. Kubernetes requires a CNI plugin to provide pod networking. Once Calico is installed and its networking components become ready, the node can transition to Ready, provided the other node health checks also pass.</p><h3>Install Calico</h3><p>Download the Calico Tigera Operator manifest:</p><pre>curl -fL https://raw.githubusercontent.com/projectcalico/calico/v3.33.0/manifests/tigera-operator.yaml -o tigera-operator.yaml</pre><p>Create the Calico custom resources configuration:</p><pre>cat &lt;&lt;&#39;EOF&#39; &gt; custom-resources.yaml<br>apiVersion: operator.tigera.io/v1<br>kind: Installation<br>metadata:<br>  name: default<br>spec:<br>  calicoNetwork:<br>    ipPools:<br>    - blockSize: 26<br>      cidr: 10.244.0.0/16<br>      encapsulation: VXLAN<br>      natOutgoing: Enabled<br>      nodeSelector: all()<br>---<br>apiVersion: operator.tigera.io/v1<br>kind: APIServer<br>metadata:<br>  name: default<br>spec: {}<br>EOF</pre><p>Apply the Calico configuration:</p><pre>kubectl apply -f custom-resources.yaml</pre><p>Check the Calico installation status:</p><pre>kubectl get tigerastatus<br><br>NAME             AVAILABLE   PROGRESSING   DEGRADED   SINCE   MESSAGE<br>apiserver        True        False         False      72m     All objects available<br>calico           True        False         False      66m     All objects available<br>ippools          True        False         False      73m     All objects available<br>tiers            True        False         False      72m     All objects availableVerify the pods across all namespaces:</pre><pre>kubectl get pods -A -o wide<br><br>NAMESPACE         NAME                                       READY   STATUS    RESTARTS      AGE    IP              NODE              NOMINATED NODE   READINESS GATES<br>calico-system     calico-apiserver-7d67c8ff5-vpzrm           1/1     Running   0             73m    10.244.0.8      ubuntu            &lt;none&gt;           &lt;none&gt;<br>calico-system     calico-apiserver-7d67c8ff5-x227w           1/1     Running   0             73m    10.244.0.6      ubuntu            &lt;none&gt;           &lt;none&gt;<br>calico-system     calico-kube-controllers-575f59bd64-nhjxj   1/1     Running   0             73m    10.244.0.9      ubuntu            &lt;none&gt;           &lt;none&gt;<br>calico-system     calico-node-5zm2k                          1/1     Running   0             67m    192.168.0.151   ubuntu            &lt;none&gt;           &lt;none&gt;<br>calico-system     calico-typha-854b89694f-kshb6              1/1     Running   0             67m    192.168.0.151   ubuntu            &lt;none&gt;           &lt;none&gt;<br>calico-system     csi-node-driver-j76pq                      2/2     Running   0             73m    10.244.0.7      ubuntu            &lt;none&gt;           &lt;none&gt;<br>kube-system       coredns-66bc5c9577-28mv7                   1/1     Running   1 (75m ago)   176m   10.244.0.5      ubuntu            &lt;none&gt;           &lt;none&gt;<br>kube-system       coredns-66bc5c9577-wdnv2                   1/1     Running   1 (75m ago)   176m   10.244.0.4      ubuntu            &lt;none&gt;           &lt;none&gt;<br>kube-system       etcd-ubuntu                                1/1     Running   1 (75m ago)   176m   192.168.0.151   ubuntu            &lt;none&gt;           &lt;none&gt;<br>kube-system       kube-apiserver-ubuntu                      1/1     Running   1 (75m ago)   176m   192.168.0.151   ubuntu            &lt;none&gt;           &lt;none&gt;<br>kube-system       kube-controller-manager-ubuntu             1/1     Running   1 (75m ago)   176m   192.168.0.151   ubuntu            &lt;none&gt;           &lt;none&gt;<br>kube-system       kube-proxy-krtnw                           1/1     Running   1 (75m ago)   176m   192.168.0.151   ubuntu            &lt;none&gt;           &lt;none&gt;<br>kube-system       kube-scheduler-ubuntu                      1/1     Running   1 (75m ago)   176m   192.168.0.151   ubuntu            &lt;none&gt;           &lt;none&gt;<br>tigera-operator   tigera-operator-7d54855d7f-qdxgs           1/1     Running   5 (79m ago)   81m    192.168.0.151   ubuntu            &lt;none&gt;           &lt;none&gt;</pre><h3>Add the Windows Worker Node to the Kubernetes Control Plane</h3><p>After installing containerd and preparing the Windows Server 2022 worker, join it to the existing Kubernetes cluster.</p><p>On the Ubuntu control-plane (192.168.0.151), generate the join command:</p><pre>sudo kubeadm token create --print-join-command<br><br><br>kubeadm join 192.168.0.151:6443 --token ct4l6q.6rbwklsk3cdyrwom --discovery-token-ca-cert-hash sha256:3359054afd0d2432ca13fb5d09a97e0cdecfcf5dd15af69c3046c7b29ea78868</pre><p>Copy the generated command and execute it in an elevated PowerShell session on the Windows worker. This registers the Windows node with the existing Kubernetes control-plane.</p><p>Verify the node status on Ubuntu:</p><pre>kubectl get nodes -o wide<br><br>root@ubuntu:/home/ubuntu# kubectl get nodes<br>NAME              STATUS     ROLES           AGE    VERSION<br>ubuntu            Ready      control-plane   106m   v1.34.12<br>win-dhjep0ugae7   NotReady   &lt;none&gt;          18m    v1.34.12</pre><p>The Windows worker should appear alongside the Ubuntu control-plane. Initially, it may remain NotReady until the Windows-compatible Calico networking components are initialized.</p><h3>Configure Calico Networking for the Windows Worker</h3><h3>* Download the Containerd Installation Script on Windows</h3><pre>Invoke-WebRequest https://raw.githubusercontent.com/kubernetes-sigs/sig-windows-tools/master/hostprocess/Install-Containerd.ps1 -OutFile C:\Install-Containerd.ps1</pre><p>This downloads the containerd installation script and saves it to C:\Install-Containerd.ps1.</p><p>Configmap is missing</p><pre>Events:<br>  Type    Reason     Age    From               Message<br>  ----    ------     ----   ----               -------<br>  Normal  Scheduled  2m21s  default-scheduler  Successfully assigned calico-system/calico-node-windows-9r5fw to win-dhjep0ugae7<br>root@ubuntu:/home/ubuntu# ^C<br>root@ubuntu:/home/ubuntu# kubectl get pod calico-node-windows-9r5fw -n calico-system -o wide<br>NAME                        READY   STATUS     RESTARTS   AGE     IP              NODE              NOMINATED NODE   READINESS GATES<br>calico-node-windows-9r5fw   0/2     Init:0/2   0          2m44s   192.168.0.225   win-dhjep0ugae7   &lt;none&gt;           &lt;none&gt;</pre><p><strong>Check initialization container logs:</strong></p><pre>kubectl logs calico-node-windows-9r5fw -n calico-system --all-containers=true</pre><p>During troubleshooting, the Windows Calico setup reported that the kubernetes-services-endpoint ConfigMap was missing. This ConfigMap supplies the Kubernetes API server address and port to the Windows networking configuration.</p><h3>* Create the Kubernetes Services Endpoint ConfigMap</h3><p>Run the following command on the Ubuntu control-plane:</p><pre>kubectl create configmap kubernetes-services-endpoint -n kube-system --from-literal=KUBERNETES_SERVICE_HOST=192.168.0.151 --from-literal=KUBERNETES_SERVICE_PORT=6443</pre><p>This creates a ConfigMap named kubernetes-services-endpoint in the kube-system namespace. It specifies the Kubernetes API server address (192.168.0.151) and port (6443).</p><p>This defines the API server endpoint as 192.168.0.151:6443.</p><p><strong>Note:</strong> The pod status above is the recorded output. The exact error message from the logs was not included in the available output, so the missing ConfigMap is described based on the troubleshooting result rather than an invented log excerpt.</p><h3>* Configure Calico for Windows</h3><pre>kubectl patch installation default --type=merge -p &#39;{&quot;spec&quot;:{&quot;serviceCIDRs&quot;:[&quot;10.96.0.0/12&quot;],&quot;calicoNetwork&quot;:{&quot;windowsDataplane&quot;:&quot;HNS&quot;,&quot;bgp&quot;:&quot;Disabled&quot;}}}&#39;</pre><p>This updates the Calico Installation resource with the following settings:</p><ul><li>serviceCIDRs: Sets the Kubernetes Service IP range to 10.96.0.0/12.</li><li>windowsDataplane: HNS: Configures the Windows Host Networking Service dataplane.</li><li>bgp: Disabled: Disables BGP for this Calico installation.</li></ul><p><strong>Verify the Calico status:</strong></p><pre>kubectl get tigerastatus<br><br>NAME             AVAILABLE   PROGRESSING   DEGRADED   SINCE   MESSAGE<br>apiserver        True        False         False      86m     All objects available<br>calico           True        False         False      80m     All objects available<br>calico-windows   True        False         False      79m     All objects available<br>ippools          True        False         False      86m     All objects available<br>tiers            True        False         False      86m     All objects available</pre><p><strong>Check the Windows Calico pod:</strong></p><pre>kubectl get pods -n calico-system -o wide<br><br>NAME                                       READY   STATUS    RESTARTS   AGE   IP              NODE              NOMINATED NODE   READINESS GATES<br>calico-apiserver-7d67c8ff5-vpzrm           1/1     Running   0          87m   10.244.0.8      ubuntu            &lt;none&gt;           &lt;none&gt;<br>calico-apiserver-7d67c8ff5-x227w           1/1     Running   0          87m   10.244.0.6      ubuntu            &lt;none&gt;           &lt;none&gt;<br>calico-kube-controllers-575f59bd64-nhjxj   1/1     Running   0          87m   10.244.0.9      ubuntu            &lt;none&gt;           &lt;none&gt;<br>calico-node-5zm2k                          1/1     Running   0          80m   192.168.0.151   ubuntu            &lt;none&gt;           &lt;none&gt;<br>calico-node-windows-9r5fw                  2/2     Running   0          80m   192.168.0.225   win-dhjep0ugae7   &lt;none&gt;           &lt;none&gt;<br>calico-typha-854b89694f-kshb6              1/1     Running   0          80m   192.168.0.151   ubuntu            &lt;none&gt;           &lt;none&gt;<br>csi-node-driver-j76pq                      2/2     Running   0          87m   10.244.0.7      ubuntu            &lt;none&gt;           &lt;none&gt;</pre><p><strong>Check the Windows Calico pod logs:</strong></p><pre>kubectl logs calico-node-windows-9r5fw -n calico-system --all-containers=true<br><br>2026-10-12 06:13:13.800 [INFO][7260] felix/config_params.go 865: Parsed value for HealthPort: 9099 (from datastore (global))<br>2026-10-12 06:13:13.800 [INFO][7260] felix/config_params.go 829: Parsing value for BPFEnabled: false (from datastore (global))<br>2026-10-12 06:13:13.800 [INFO][7260] felix/config_params.go 865: Parsed value for BPFEnabled: false (from datastore (global))<br>2026-10-12 06:13:13.800 [INFO][7260] felix/config_params.go 829: Parsing value for BPFHostNetworkedNATWithoutCTLB: Enabled (from datastore (global))<br>2026-10-12 06:13:13.800 [INFO][7260] felix/config_params.go 865: Parsed value for BPFHostNetworkedNATWithoutCTLB: Enabled (from datastore (global))<br>2026-10-12 06:13:13.800 [INFO][7260] felix/config_params.go 829: Parsing value for NFTablesMode: Disabled (from datastore (global))<br>2026-10-12 06:13:13.800 [INFO][7260] felix/config_params.go 865: Parsed value for NFTablesMode: Disabled (from datastore (global))<br>2026-10-12 06:13:13.800 [INFO][7260] felix/config_params.go 829: Parsing value for ReportingIntervalSecs: 0 (from datastore (global))<br>2026-10-12 06:13:13.800 [INFO][7260] felix/config_params.go 865: Parsed value for ReportingIntervalSecs: 0s (from datastore (global))<br>2026-10-12 06:13:13.800 [INFO][7260] felix/config_params.go 829: Parsing value for VXLANVNI: 4096 (from datastore (global))<br>2026-10-12 06:13:13.801 [INFO][7260] felix/config_params.go 865: Parsed value for VXLANVNI: 4096 (from datastore (global))<br>2026-10-12 06:13:13.801 [INFO][7260] felix/config_params.go 868: Skipping config value for VXLANVNI from datastore (global); already have a value from environment variable<br>2026-10-12 06:13:13.801 [INFO][7260] felix/config_params.go 829: Parsing value for CalicoVersion: v3.33.0 (from datastore (global))<br>2026-10-12 06:13:13.801 [INFO][7260] felix/config_params.go 865: Parsed value for CalicoVersion: v3.33.0 (from datastore (global))<br>2026-10-12 06:13:13.801 [INFO][7260] felix/config_params.go 829: Parsing value for BPFConnectTimeLoadBalancing: TCP (from datastore (global))<br>2026-10-12 06:13:13.801 [INFO][7260] felix/config_params.go 865: Parsed value for BPFConnectTimeLoadBalancing: TCP (from datastore (global))<br>2026-10-12 06:13:13.801 [INFO][7260] felix/config_params.go 829: Parsing value for ProgramClusterRoutes: EnabledIPIPOnly (from datastore (global))<br>2026-10-12 06:13:13.801 [INFO][7260] felix/config_params.go 865: Parsed value for ProgramClusterRoutes: EnabledIPIPOnly (from datastore (global))<br>2026-10-12 06:13:13.801 [INFO][7260] felix/async_calc_graph.go 221: First flush after becoming in sync, sending InSync message.<br>2026-10-12 06:13:13.801 [INFO][7260] felix/daemon.go 1452: Datastore now in sync.<br>2026-10-12 06:13:13.801 [INFO][7260] felix/win_dataplane.go 252: Received *proto.Encapsulation update from calculation graph msg=vxlan_enabled:true<br>2026-10-12 06:13:13.802 [INFO][7260] felix/win_dataplane.go 252: Received *proto.InSync update from calculation graph msg=<br>2026-10-12 06:13:13.802 [INFO][7260] felix/win_dataplane.go 259: Datastore in sync, flushing the dataplane for the first time... timeSinceStart=465.1693ms<br>2026-10-12 06:13:13.802 [INFO][7260] felix/win_dataplane.go 303: Applying dataplane updates<br>2026-10-12 06:13:13.809 [INFO][7260] felix/vxlan_mgr.go 238: All VXLAN route updates succeeded.<br>2026-10-12 06:13:13.810 [INFO][7260] felix/win_dataplane.go 310: Finished applying updates to dataplane. msecToApply=7.8352<br>2026-10-12 06:13:13.810 [INFO][7260] felix/win_dataplane.go 316: Completed first update to dataplane. secsSinceStart=0.4730045<br>2026-10-12 06:13:13.810 [INFO][7260] felix/health.go 203: Health of component changed name=&quot;WindowsDataplaneMainLoop&quot; newReport=&quot;live,ready&quot; oldReport=&quot;live,non-ready&quot;<br>2026-10-12 06:13:15.117 [INFO][7260] felix/health.go 334: Overall health status changed: live=true ready=true<br>+--------------------------+---------+----------------+-----------------+--------+<br>| COMPONENT                | TIMEOUT | LIVENESS       | READINESS       | DETAIL |<br>+--------------------------+---------+----------------+-----------------+--------+<br>| CalculationGraph         | 30s     | reporting live | reporting ready |        |<br>| FelixStartup             | -       | reporting live | reporting ready |        |<br>| WindowsDataplaneMainLoop | 1m30s   | reporting live | reporting ready |        |<br>+--------------------------+---------+----------------+-----------------+--------+<br>2026-10-12 06:18:15.018 [INFO][7260] felix/usagerep.go 116: Initial delay complete, doing first report<br>2026-10-12 06:18:15.019 [INFO][7260] felix/usagerep.go 209: Reporting cluster usage/checking for deprecation warnings. alpEnabled=false calicoVersion=&quot;v3.33.0&quot; clusterGUID=&quot;d4dd016f4d8143479a9ac7222ad13e8b&quot; clusterType=&quot;typha,kdd,k8s,operator,bgp,kubeadm,windows,win&quot; gitRevision=&quot;d2060a8417a60c37946901f1beb951011ea24557&quot; kubernetesVersion=&quot;v1.34.12&quot; stats=calc.StatsUpdate{NumHosts:2, NumWorkloadEndpoints:6, NumHostEndpoints:0, NumTiers:4, NumPolicies:4, NumProfiles:58, NumALPPolicies:0} version=&quot;v3.33.0&quot;<br>2026-10-12 06:18:15.767 [INFO][7260] felix/usagerep.go 118: First report done, starting ticker<br>Setting environment variables if not set...<br>Environment variable KUBE_NETWORK is already set: Calico.*<br>Environment variable CALICO_NETWORKING_BACKEND is already set: vxlan<br>Environment variable DNS_SEARCH is not set. Setting it to the default value: svc.cluster.local<br>Environment variable VXLAN_VNI is already set: 4096<br>Environment variable VXLAN_MAC_PREFIX is not set. Setting it to the default value: 0E-2A<br>Environment variable VXLAN_ADAPTER is not set. Setting it to the default value:<br>Environment variable NODENAME is already set: win-dhjep0ugae7<br>Environment variable CALICO_K8S_NODE_REF is not set. Setting it to the default value: win-dhjep0ugae7<br>Environment variable STARTUP_VALID_IP_TIMEOUT is not set. Setting it to the default value: 90</pre><h3>Enable Strict IPAM Affinity</h3><p>Run the following command on the Ubuntu control-plane:</p><pre>kubectl patch ipamconfiguration default --type=merge -p &#39;{&quot;spec&quot;:{&quot;strictAffinity&quot;:true}}&#39;</pre><p>This enables strict IP address management (IPAM) affinity in Calico.</p><p><strong>Why is it required?</strong></p><p>Calico IPAM allocates pod IP addresses from IP pools divided into smaller blocks. Strict affinity enforces that IP address blocks are allocated to specific nodes and prevents addresses from being allocated from blocks that are not strictly associated with the allocating node.</p><p>In this lab, the cluster contains both Ubuntu and Windows nodes. Enabling strict affinity is part of the Windows Calico IPAM configuration, helping Calico manage pod IP allocation according to node-specific block ownership.</p><p><strong>Verify the configuration:</strong></p><pre>kubectl get ipamconfiguration default -o yaml</pre><p>Check that the output contains:</p><pre>spec:<br>  strictAffinity: true</pre><p>This confirms that strict IPAM affinity is enabled.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/674/1*WYwZPNYjjRt8oUq0lX5dsw.png" /></figure><h3>Troubleshoot Calico Windows Pod Initialization</h3><p>The Windows Calico pod was stuck in the initialization phase:</p><pre>kubectl get pod calico-node-windows-9r5fw -n calico-system -o wide</pre><p>Output:</p><pre>NAME                        READY   STATUS     RESTARTS   AGE     IP              NODE<br>calico-node-windows-9r5fw   0/2     Init:0/2   0          2m44s   192.168.0.225   win-dhjep0ugae7</pre><p>The pod was assigned to the Windows worker at 192.168.0.225, but neither of its two containers was ready because initialization had not completed.</p><p>During troubleshooting, the issue was traced to the missing kubernetes-services-endpoint ConfigMap required by the Windows Calico setup. The ConfigMap provides the Kubernetes API server address and port used by the networking components.</p><h3>Troubleshoot Windows CNI Initialization</h3><p>After joining the Windows worker, check its Kubernetes node conditions from the Ubuntu control-plane:</p><pre>kubectl get node win-dhjep0ugae7 -o jsonpath=&#39;{range .status.conditions[*]}{.type}={.status}{&quot;  &quot;}{.reason}{&quot;  &quot;}{.message}{&quot;\n&quot;}{end}&#39;</pre><p>The output showed:</p><pre>MemoryPressure=False  KubeletHasSufficientMemory  kubelet has sufficient memory available<br>DiskPressure=False  KubeletHasNoDiskPressure  kubelet has no disk pressure<br>PIDPressure=False  KubeletHasSufficientPID  kubelet has sufficient PID available<br>Ready=False  KubeletNotReady  container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin not initialized</pre><p><strong>Explanation:</strong> The Windows node had sufficient memory, disk space, and available process IDs. However, Ready=False indicated that kubelet could not mark the node ready because the container network interface (CNI) had not initialized.</p><p>If the CNI plugin is not initialized on the Windows node, the expected error would look like the output you captured earlier:</p><pre>Ready=False  KubeletNotReady  container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin not initialized</pre><p>This indicates that kubelet cannot mark the Windows node as ready because the container runtime’s network plugin has not initialized.</p><p>Your latest output shows the issue is resolved:</p><pre>NetworkUnavailable=False  CalicoIsUp  Calico is running on this node<br>Ready=True  KubeletReady  kubelet is posting ready status</pre><h3>Inspect the Windows CNI Configuration</h3><p>Run the following command in Windows PowerShell:</p><pre>Get-ChildItem C:\etc\cni\net.d -Force -ErrorAction SilentlyContinue</pre><p>Specifically, it only lists the contents of C:\etc\cni\net.d.</p><ul><li>Get-ChildItem — lists files and directories.</li><li>-Force — includes hidden items.</li><li>-ErrorAction SilentlyContinue — suppresses errors, such as when the directory does not exist.</li></ul><p>Therefore, this command alone cannot explain why the CNI issue was resolved. If running it coincided with the fix, another action or process may have created the required configuration, or the node may have recovered independently. The command does not create the directory, write CNI configuration files, or restart networking services.</p><p>Important: In our lab, the Windows Calico pod was initially stuck at Init:0/2, and the missing kubernetes-services-endpoint ConfigMap was identified during troubleshooting. If the CNI directory is missing, investigate Calico initialization and its configuration first rather than manually creating CNI files.</p><p>Nodes Status</p><pre>root@ubuntu:/home/ubuntu# kubectl get nodes -o wide<br>NAME              STATUS   ROLES           AGE     VERSION    INTERNAL-IP     EXTERNAL-IP   OS-IMAGE                                  KERNEL-VERSION      CONTAINER-RUNTIME<br>ubuntu            Ready    control-plane   3h28m   v1.34.12   192.168.0.151   &lt;none&gt;        Ubuntu 24.04.5 LTS                        6.8.0-146-generic   containerd://2.2.1<br>win-dhjep0ugae7   Ready    &lt;none&gt;          121m    v1.34.12   192.168.0.225   &lt;none&gt;        Windows Server 2022 Standard Evaluation   10.0.20348.587      containerd://1.7.22</pre><p>Pod Status</p><pre>root@ubuntu:/home/ubuntu# kubectl get pods -A<br>NAMESPACE         NAME                                       READY   STATUS    RESTARTS       AGE<br>calico-system     calico-apiserver-7d67c8ff5-vpzrm           1/1     Running   0              105m<br>calico-system     calico-apiserver-7d67c8ff5-x227w           1/1     Running   0              105m<br>calico-system     calico-kube-controllers-575f59bd64-nhjxj   1/1     Running   0              105m<br>calico-system     calico-node-5zm2k                          1/1     Running   0              99m<br>calico-system     calico-node-windows-9r5fw                  2/2     Running   0              98m<br>calico-system     calico-typha-854b89694f-kshb6              1/1     Running   0              99m<br>calico-system     csi-node-driver-j76pq                      2/2     Running   0              105m<br>kube-system       coredns-66bc5c9577-28mv7                   1/1     Running   1 (107m ago)   3h28m<br>kube-system       coredns-66bc5c9577-wdnv2                   1/1     Running   1 (107m ago)   3h28m<br>kube-system       etcd-ubuntu                                1/1     Running   1 (107m ago)   3h28m<br>kube-system       kube-apiserver-ubuntu                      1/1     Running   1 (107m ago)   3h28m<br>kube-system       kube-controller-manager-ubuntu             1/1     Running   1 (107m ago)   3h28m<br>kube-system       kube-proxy-krtnw                           1/1     Running   1 (107m ago)   3h28m<br>kube-system       kube-scheduler-ubuntu                      1/1     Running   1 (107m ago)   3h28m<br>tigera-operator   tigera-operator-7d54855d7f-qdxgs           1/1     Running   5 (111m ago)   113m</pre><h3>Labeling of windows node</h3><p>You only need to add this to your Deployment YAML to schedule the application exclusively on Windows nodes:</p><pre>spec:<br>  template:<br>    spec:<br>      nodeSelector:<br>        kubernetes.io/os: windows</pre><p>This tells Kubernetes to schedule the pod only on nodes labeled kubernetes.io/os=windows.</p><p>Create a Windows test Deployment using a Windows Server 2022 container image.</p><p>1. Create windows-app.yaml</p><pre>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: windows-app<br>spec:<br>  replicas: 1<br>  selector:<br>    matchLabels:<br>      app: windows-app<br>  template:<br>    metadata:<br>      labels:<br>        app: windows-app<br>    spec:<br>      nodeSelector:<br>        kubernetes.io/os: windows<br>      containers:<br>        - name: windows-app<br>          image: mcr.microsoft.com/windows/servercore:ltsc2022<br>          command:<br>            - powershell.exe<br>            - -Command<br>            - &quot;while ($true) { Write-Output &#39;Windows application is running&#39;; Start-Sleep -Seconds 30 }&quot;</pre><p>2. Apply the manifest</p><pre>kubectl apply -f windows-app.yaml</pre><p>3. Verify the pod is running on the Windows node</p><pre>kubectl get pods -o wide</pre><p>The pod should be scheduled on win-dhjep0ugae7. The Windows Server Core image must be compatible with the Windows Server version on your worker node.</p><p>The pod is scheduled</p><pre>kubectl get pods -o wide<br><br>NAME                           READY   STATUS              RESTARTS   AGE     IP       NODE              NOMINATED NODE   READINESS GATES<br>windows-app-845d7f7b66-wrh92   0/1     ContainerCreating   0          6m35s   &lt;none&gt;   win-dhjep0ugae7   &lt;none&gt;           &lt;none&gt;</pre><p>It took more than 30 mints for it to be in running state</p><pre>^Croot@ubuntu:/home/ubuntu# kubectl get pods -w<br>NAME                           READY   STATUS              RESTARTS   AGE<br>windows-app-845d7f7b66-wrh92   1/1     Running             0          30m</pre><p>Logs of the pod</p><pre>root@ubuntu:/home/ubuntu# kubectl logs windows-app-845d7f7b66-wrh92<br>Windows application is running<br>Windows application is running<br>Windows application is running</pre><h3>Browser based app</h3><pre>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: windows-iis<br>spec:<br>  replicas: 1<br>  selector:<br>    matchLabels:<br>      app: windows-iis<br>  template:<br>    metadata:<br>      labels:<br>        app: windows-iis<br>    spec:<br>      nodeSelector:<br>        kubernetes.io/os: windows<br>      containers:<br>        - name: iis<br>          image: mcr.microsoft.com/windows/servercore/iis:windowsservercore-ltsc2022<br>          ports:<br>            - containerPort: 80<br>---<br>apiVersion: v1<br>kind: Service<br>metadata:<br>  name: windows-iis<br>spec:<br>  type: NodePort<br>  selector:<br>    app: windows-iis<br>  ports:<br>    - port: 80<br>      targetPort: 80<br>      nodePort: 30080</pre><p>his image serves the default IIS page. To display the exact text “Hello from Windows”, the IIS page must also be customized.</p><p>Apply the manifest:</p><pre>kubectl apply -f windows-iis.yaml</pre><p>Check the pod placement:</p><pre>kubectl get pods -o wide<br><br>NAME                           READY   STATUS              RESTARTS   AGE     IP       NODE              NOMINATED NODE   READINESS GATES<br>windows-app-845d7f7b66-wrh92   0/1     ContainerCreating   0          6m35s   &lt;none&gt;   win-dhjep0ugae7   &lt;none&gt;           &lt;none&gt;<br>windows-iis-58c55d8784-f6zhq   0/1     ContainerCreating   0          3m50s   &lt;none&gt;   win-dhjep0ugae7   &lt;none&gt;           &lt;none&gt;</pre><p>Get Service</p><pre>NAME          TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE<br>windows-iis   NodePort    10.106.72.159   &lt;none&gt;        80:30080/TCP   32m</pre><p>They are scheduled on windows node.</p><p>Then access it in your browser:</p><p><a href="http://192.168.0.225:30080/">http://192.168.0.225:30080</a></p><pre>root@ubuntu:/home/ubuntu# kubectl exec windows-iis-58c55d8784-f6zhq -- powershell.exe -Command &quot;Invoke-WebRequest http://localhost -UseBasicParsing&quot;<br><br><br>StatusCode        : 200<br>StatusDescription : OK<br>Content           : &lt;!DOCTYPE html PUBLIC &quot;-//W3C//DTD XHTML 1.0 Strict//EN&quot;<br>                    &quot;http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd&quot;&gt;<br>                    &lt;html xmlns=&quot;http://www.w3.org/1999/xhtml&quot;&gt;<br>                    &lt;head&gt;<br>                    &lt;meta http-equiv=&quot;Content-Type&quot; cont...<br>RawContent        : HTTP/1.1 200 OK<br>                    Accept-Ranges: bytes<br>                    Content-Length: 703<br>                    Content-Type: text/html<br>                    Date: Mon, 12 Oct 2026 15:25:07 GMT<br>                    ETag: &quot;68b74864d53fdd1:0&quot;<br>                    Last-Modified: Tue, 08 Sep 2026 21:02:43 GMT<br>                    Serve...<br>Forms             :<br>Headers           : {[Accept-Ranges, bytes], [Content-Length, 703],<br>                    [Content-Type, text/html], [Date, Mon, 12 Oct 2026<br>                    15:25:07 GMT]...}<br>Images            : {}<br>InputFields       : {}<br>Links             : {@{outerHTML=&lt;a href=&quot;http://go.microsoft.com/fwlink/?linki<br>                    d=66138&amp;amp;clcid=0x409&quot;&gt;&lt;img src=&quot;iisstart.png&quot; alt=&quot;IIS&quot;<br>                    width=&quot;960&quot; height=&quot;600&quot; /&gt;&lt;/a&gt;; tagName=A; href=http://go.<br>                    microsoft.com/fwlink/?linkid=66138&amp;amp;clcid=0x409}}<br>ParsedHtml        :<br>RawContentLength  : 703<br></pre><p>The command kubectl port-forward svc/windows-iis 8080:80 --address 0.0.0.0 forwards traffic from Ubuntu to your IIS application running inside the Windows container.</p><ul><li>kubectl port-forward — forwards network traffic to a Kubernetes resource.</li><li>svc/windows-iis — specifies the Kubernetes Service named windows-iis, which routes traffic to your IIS pod.</li><li>8080:80 — maps port 8080 on Ubuntu to port 80 on the IIS Service.</li><li>--address 0.0.0.0 — makes the forwarded port listen on all network interfaces of Ubuntu, rather than only localhost.</li></ul><p>The output:</p><pre>Forwarding from 0.0.0.0:8080 -&gt; 80</pre><p>means the port-forward is listening on port 8080.</p><p>To access the IIS page, open **http://192.168.0.151:8080** in your browser. Keep the command running while accessing it.</p><p>Output</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*O724BAT2iMpVfpDAE1v64A.png" /></figure><h3>Conclusion</h3><p>In this blog, we have seen how to add a Windows Server 2022 machine as a worker node to an existing Kubernetes cluster running on Ubuntu. We configured the Windows container runtime, prepared the Windows node, configured networking, and successfully joined it to our Kubernetes cluster.</p><p>We also deployed Windows containers on the Windows worker node and verified that both applications were running successfully. To access the IIS application in our browser, we used kubectl port-forward to forward traffic from the Ubuntu control-plane node to the Windows IIS service. Finally, we accessed the IIS page through our browser and confirmed that the application was working.</p><p>This setup helps us understand how Kubernetes manages workloads running on different operating systems. For example, applications built using .NET Framework or other Windows-specific components can run on Windows worker nodes, while Linux-based applications continue to run on Linux nodes. Both can be managed through a single Kubernetes control plane.</p><p>With this hands-on lab, we have successfully demonstrated a hybrid Kubernetes cluster with an Ubuntu control-plane node and a Windows Server 2022 worker node. This provides a good starting point for exploring more advanced Windows workloads, application deployments, and networking in Kubernetes.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dc4f56e24a2b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/devops-cloud-engineering-hub/how-to-add-a-windows-server-2022-worker-node-to-a-kubernetes-cluster-ubuntu-windows-dc4f56e24a2b">How to Add a Windows Server 2022 Worker Node to a Kubernetes Cluster (Ubuntu + Windows)</a> was originally published in <a href="https://medium.com/devops-cloud-engineering-hub">devops-cloud-engineering-hub</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[OpenShift HTPasswd Authentication: User Creation, OAuth Integration & RBAC Access Control]]></title>
            <link>https://medium.com/devops-cloud-engineering-hub/openshift-htpasswd-authentication-user-creation-oauth-integration-rbac-access-control-6813d6f3c8ef?source=rss-1304b20de7e5------2</link>
            <guid isPermaLink="false">https://medium.com/p/6813d6f3c8ef</guid>
            <category><![CDATA[redhat-openshift]]></category>
            <category><![CDATA[red-hat-openshift]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[ldap]]></category>
            <category><![CDATA[openshift]]></category>
            <dc:creator><![CDATA[Manohar Shetty]]></dc:creator>
            <pubDate>Sat, 10 Oct 2026 08:41:58 GMT</pubDate>
            <atom:updated>2026-10-12T08:53:08.811Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction</h3><p>In a real-world OpenShift environment, administrators need a secure and controlled way to onboard users and grant access to cluster resources. In this hands-on implementation, we configure HTPasswd authentication from scratch, create a local user named alex, integrate the HTPasswd file with OpenShift OAuth using a Secret, and verify authentication through the web console.</p><p>We then implement Role-Based Access Control (RBAC) by creating a custom ClusterRole that grants Alex read-only access to nodes and pods without granting access to unrelated resources such as Deployments and Secrets. Finally, we validate the permissions using the OpenShift CLI and test actual access using Alex’s login credentials.</p><p>This practical exercise demonstrates how authentication and authorization work together in OpenShift and how the principle of least privilege can be applied when onboarding users.</p><h4>OpenShift user access: real-world onboarding and authentication methods</h4><p>When a new employee joins an OpenShift environment, there are two separate things to configure:</p><ol><li>Authentication — Who is the user? For example, FreeIPA/LDAP credentials, HTPasswd credentials, or an identity provider using OpenID Connect.</li><li>Authorization — What can the user do? For example, view pods and nodes, access a specific project, deploy applications, or administer the cluster.</li></ol><p>Your current environment uses OpenShift SNO, FreeIPA at ipa.manohar.in, and an HTPasswd provider named local-users. You have created the HTPasswd user alex and assigned a custom ClusterRole for read-only access to pods and nodes. Those are separate from the FreeIPA authentication flow.</p><h3>* Main ways users can access OpenShift</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/980/1*ejM8zgV1NeAU4a1nqcYMQA.png" /></figure><h3>Redhat Documentation:</h3><h3>Understanding identity provider configuration</h3><p><a href="https://docs.redhat.com/en/documentation/openshift_container_platform/4.21/html/authentication_and_authorization/understanding-identity-provider">Chapter 6. Understanding identity provider configuration</a></p><h3>Authentication and authorization</h3><p><a href="https://docs.redhat.com/en/documentation/openshift_container_platform/4.21/observability/authentication_and_authorization/understanding-and-creating-service-accounts">Chapter 11. Understanding and creating service accounts</a></p><h3>* How FreeIPA / LDAP authentication works</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/778/1*ApFvNFM0USWb2KxY5T4-jA.png" /></figure><p>For example, your FreeIPA domain is manohar.in, and the LDAP server is ipa.manohar.in.</p><p>The employee already has an account in FreeIPA. When they log in to OpenShift using the FreeIPA provider, OpenShift validates their credentials against LDAP and establishes an identity mapping. The employee does not need a separate HTPasswd password.</p><p>Important: Your current OAuth configuration, as established in our recent steps, contains only local-users (HTPasswd). The freeipa provider was removed when we cleared the OAuth identity providers. FreeIPA authentication will not work through OpenShift until that LDAP provider is restored.</p><h3>* How HTPasswd authentication works in your current setup</h3><p>You created Alex using this sequence:</p><ol><li>Create users.htpasswd containing Alex’s bcrypt password hash.</li><li>Store the file in the htpass-secret Secret in openshift-config.</li><li>Configure the local-users HTPasswd identity provider.</li><li>Create the OpenShift User object.</li><li>Alex logs in through the local-users option on the console.</li><li>OpenShift establishes the local-users:alex identity mapping.</li></ol><p>HTPasswd is suitable for your lab. For a larger organization, centrally managed LDAP/FreeIPA or OIDC SSO is usually easier to maintain.</p><h3>Configuring identity providers</h3><p><a href="https://docs.redhat.com/en/documentation/openshift_container_platform/4.21/html/authentication_and_authorization/configuring-identity-providers">Chapter 7. Configuring identity providers</a></p><h3>* Real-world employee onboarding workflow</h3><p>When a new employee joins, the administrator typically follows this workflow:</p><ol><li>Create the employee identity. Create an account in FreeIPA/LDAP or the organization’s SSO provider.</li><li>Configure OpenShift authentication. Ensure the required identity provider is configured and healthy.</li><li>Determine the required access. For example, read-only pod and node access, developer access to one project, or cluster administration.</li><li>Create or map the OpenShift identity. With mappingMethod: claim, OpenShift can provision and map the user on successful authentication. Manual provisioning is needed for workflows using lookup.</li><li>Assign permissions. Use custom Roles or ClusterRoles with RoleBindings or ClusterRoleBindings, following least privilege.</li><li>Verify effective access. Use oc auth can-i and test with the employee&#39;s own login or token.</li><li>Provide access instructions. Share the console URL, login method, project names, and CLI login procedure through your organization’s approved channel.</li><li>Handle offboarding. Disable the employee’s directory account or SSO access and remove their OpenShift role bindings as appropriate.</li></ol><h3>* What users use after authentication</h3><p>A user can interact with OpenShift in several ways:</p><ul><li>Web console: Browser-based access to projects, workloads, pods, nodes, and other resources permitted by RBAC.</li><li>OpenShift CLI (oc): Commands such as oc get pods, oc get nodes, and oc logs.</li><li>Kubernetes API: Direct API requests authenticated with an access token or supported client credentials.</li><li>Automation: Pipelines and tools such as Ansible/AWX use dedicated service accounts or other supported credentials, with narrowly scoped permissions.</li></ul><p>The same authorization policies apply regardless of whether a user accesses the cluster through the console, CLI, or API.</p><p>For your lab, the next logical exercise is to restore FreeIPA as an additional login option alongside HTPasswd, then compare how a FreeIPA employee and an HTPasswd employee receive their identities and RBAC permissions. See the official</p><p>OpenShift 4.21 authentication documentation</p><p><a href="https://docs.redhat.com/en/documentation/openshift_container_platform/4.21/html/authentication_and_authorization/understanding-identity-provider">Chapter 6. Understanding identity provider configuration</a></p><p>This blog focuses exclusively on configuring HTPasswd authentication in OpenShift, from creating a local user and configuring the HTPasswd identity provider to logging in through the web console and assigning custom, read-only permissions to view nodes and pods. LDAP and FreeIPA authentication are covered separately in another blog.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vaJKAmURwXQBtOycAIi_Sg.png" /></figure><h3>Read the blog on IPA/LDAP Server Integration with Openshift</h3><p>How I Integrated FreeIPA with OpenShift for Centralized Authentication</p><p><a href="https://medium.com/devops-cloud-engineering-hub/how-i-integrated-freeipa-with-openshift-for-centralized-authentication-97679442ade1">How I Integrated FreeIPA with OpenShift for Centralized Authentication</a></p><h3>Steps Covered in the blog</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/745/1*YTc_65GYAztUn6pFMikTEw.png" /></figure><h3>Prerequisite 1: Check and Create the HTPasswd Secret</h3><h3>Step 1: Create the OpenShift User</h3><p>Run this command on the bastion:</p><pre>oc create user alex</pre><p>This creates the OpenShift User object named alex. It does not set Alex&#39;s password; the password is configured through the HTPasswd file.</p><p>Expected output:</p><pre>user.user.openshift.io/alex created</pre><h3>Step 2: Create the HTPasswd File and Add Alex’s Credentials</h3><p>If you are starting from scratch and users.htpasswd does not exist, run:</p><pre>htpasswd -c -B -b users.htpasswd alex &#39;alex123&#39;</pre><p>This creates the HTPasswd file and adds Alex with password alex123.</p><ul><li>-c creates a new file. Use it only when starting fresh, because it overwrites an existing file.</li><li>-B uses bcrypt to hash the password.</li><li>-b accepts the password directly in the command.</li></ul><p>If the file already exists and contains other users you want to preserve, omit -c:</p><pre>htpasswd -B -b users.htpasswd alex &#39;alex123&#39;</pre><p>Before configuring the local-users HTPasswd identity provider, check whether the htpass-secret Secret exists in the openshift-config namespace.</p><p>Check the Secret:</p><pre>oc get secret htpass-secret -n openshift-config</pre><p>If the Secret does not exist, create it using the users.htpasswd file:</p><p><strong>Explanation:</strong></p><ul><li>oc create secret generic htpass-secret creates a Secret named htpass-secret.</li><li>--from-file=htpasswd=users.htpasswd stores the contents of the local HTPasswd file under the key htpasswd.</li><li>-n openshift-config creates the Secret in the namespace used for OpenShift OAuth configuration.</li></ul><p>The Secret stores the username and password hashes that OpenShift uses to authenticate users.</p><p><strong>Important:</strong> If the Secret already exists, skip the creation command. If you are starting from scratch and the file does not exist, create the HTPasswd file in Step 2 before creating the Secret.</p><h3>Step 3: Update the HTPasswd Secret</h3><p>After adding Alex to the file, update the existing Secret:</p><pre>oc create secret generic htpass-secret \<br>  --from-file=htpasswd=users.htpasswd \<br>  -n openshift-config \<br>  --dry-run=client -o yaml | oc apply -f -</pre><p>This updates htpass-secret with the current contents of users.htpasswd. Existing entries in the Secret are replaced by the contents of the file, so ensure the file contains every user you intend to retain.</p><h3>Prerequisite 2: Verify the Secret contains Alex’s credentials:</h3><pre>oc get secret htpass-secret -n openshift-config -o jsonpath=&#39;{.data.htpasswd}&#39; | base64 -d</pre><p>The output confirms that the Secret contains Alex’s HTPasswd entry.</p><ul><li>oc get secret htpass-secret -n openshift-config retrieves the Secret from the openshift-config namespace.</li><li>-o jsonpath=&#39;{.data.htpasswd}&#39; extracts the htpasswd data field. Kubernetes stores Secret data in Base64 encoding.</li><li>| base64 -d decodes that data to display the file contents.</li></ul><p>Output:</p><pre>alex:$2y$05$exmVWJaP8l8PsLegHCf0PeUU52tCP8hEq2gqYWv/VLWHcKwPgrmiK</pre><p>alex is the username. The string beginning with $2y$ is the bcrypt password hash, not the plaintext password. OpenShift uses this hash to verify the password during authentication.</p><p>Current status: The Secret was created successfully and contains Alex’s hashed credentials. The next step is to configure the OpenShift OAuth HTPasswd identity provider to reference htpass-secret, if it is not already configured.</p><h3>Prerequisite 3: Configure the local-users HTPasswd Identity Provider</h3><p>First, check the configured identity providers:</p><pre>oc get oauth cluster -o jsonpath=&#39;{range .spec.identityProviders[*]}{.name}{&quot;\t&quot;}{.type}{&quot;\n&quot;}{end}&#39;</pre><p>If local-users is absent, add it:</p><pre>oc patch oauth cluster --type=json -p=&#39;[{&quot;op&quot;:&quot;add&quot;,&quot;path&quot;:&quot;/spec/identityProviders/-&quot;,&quot;value&quot;:{&quot;name&quot;:&quot;local-users&quot;,&quot;mappingMethod&quot;:&quot;claim&quot;,&quot;type&quot;:&quot;HTPasswd&quot;,&quot;htpasswd&quot;:{&quot;fileData&quot;:{&quot;name&quot;:&quot;htpass-secret&quot;}}}}]&#39;</pre><p>This configures OpenShift OAuth to use the htpass-secret Secret when authenticating users through the local-users provider. The patch preserves existing identity providers.</p><p>Verify that local-users appears in the output of the first command.</p><p><strong>Important:</strong> Configure this provider only once. Do not run the patch if local-users already exists.</p><h3>Step 4: Verify the HTPasswd Identity Provider</h3><p>Run:</p><pre>oc get oauth cluster -o jsonpath=&#39;{range .spec.identityProviders[*]}{.name}{&quot;\t&quot;}{.type}{&quot;\n&quot;}{end}&#39;</pre><p>Expected output includes:</p><pre>local-users     HTPasswd        htpass-secret</pre><p>This confirms that the local-users identity provider is configured with the HTPasswd authentication type.</p><h3>Step 5: Verify the HTPasswd Secret Reference</h3><p>Run:</p><pre>oc get oauth cluster -o jsonpath=&#39;{.spec.identityProviders[?(@.name==&quot;local-users&quot;)].htpasswd.fileData.name}{&quot;\n&quot;}&#39;</pre><p>Expected output:</p><pre>htpass-secret</pre><p>This confirms that the local-users provider references the htpass-secret Secret.</p><h3>Step 6: Verify Alex’s HTPasswd Entry</h3><p>Run:</p><pre>oc get secret htpass-secret -n openshift-config -o jsonpath=&#39;{.data.htpasswd}&#39; | base64 -d</pre><p>The output should contain Alex’s username and password hash, for example:</p><pre>alex:$2y$...</pre><p>The hash will differ depending on the generated password hash.</p><p>At this point, the HTPasswd credentials and identity provider are configured. Alex can attempt to authenticate through the local-users provider using username alex and password alex123. This procedure does not create or assign any ClusterRole or permissions.</p><h3>Step 7 — Verify the user and identity mapping:</h3><pre>oc get user alex</pre><p>This checks whether OpenShift has created Alex’s User object and whether an HTPasswd identity mapping exists. Run it and share the output.</p><p>The OpenShift User object alex exists. The empty IDENTITIES column is expected because the identity mapping has not yet been established through authentication.</p><pre>[root@bastion ~]# oc get users<br>NAME      UID                                    FULL NAME        IDENTITIES<br>alex      85e7cb74-ee3d-4ba2-814a-bd0153451761</pre><p>Get the OpenShift dashboard URL</p><pre>oc get console cluster -o jsonpath=&#39;{.status.consoleURL}{&quot;\n&quot;}&#39;</pre><p>Open the OpenShift dashboard in any browser:</p><pre>https://console-openshift-console.apps.sno.manohar.in</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UntrRFGy_SydXuoWtiibHg.png" /></figure><h4>Click on local-users</h4><p>Next: Click local-users on the login page and sign in with:</p><ul><li>Username: alex</li><li>Password: alex123</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pXP0FLFb5ssibPyOXwmUxA.png" /></figure><p>Logged in as alex</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dZx3NRw4RZ0AzP29Jdv8hw.png" /></figure><p>The IDENTITIES column should show local-users:alex once the identity mapping is established.</p><pre>[root@bastion ~]# oc get user alex<br>NAME   UID                                    FULL NAME   IDENTITIES<br>alex   85e7cb74-ee3d-4ba2-814a-bd0153451761               local-users:alex<br>[root@bastion ~]#</pre><p>If Alex needs access to cluster resources, assign the appropriate role. Creating a user does not automatically grant permissions.</p><h3>Step 8 — Understanding OpenShift RoleBindings Automatically Created for Users</h3><p>When a user is created in OpenShift and logs in to the web console, certain resources may be created automatically to support console functionality. During the configuration of an HTPasswd user named alex, a RoleBinding appeared in the openshift-console-user-settings namespace even though no permissions had been explicitly assigned to Alex.</p><p>This article explains how to identify this RoleBinding, understand its purpose, and investigate the permissions it grants.</p><h3>Step8.1 — Identify Alex’s RoleBindings</h3><p>To identify RoleBindings and ClusterRoleBindings that reference Alex, run:</p><pre>oc get rolebindings -A -o wide | grep alex<br>oc get clusterrolebindings -o wide | grep alex</pre><p>The output showed:</p><pre>openshift-console-user-settings \<br>user-settings-85e7cb74-ee3d-4ba2-814a-bd0153451761-rolebinding \<br>Role/user-settings-85e7cb74-ee3d-4ba2-814a-bd0153451761-role \<br>24m alex</pre><p>This identifies a RoleBinding in the openshift-console-user-settings namespace.</p><h3>Step8.2 - Understand the RoleBinding</h3><p>The output contains four important details:</p><ul><li><strong>Namespace:</strong> openshift-console-user-settings — the namespace where the RoleBinding exists.</li><li><strong>RoleBinding name:</strong> user-settings-85e7cb74-ee3d-4ba2-814a-bd0153451761-rolebinding — the object that connects a subject to a role.</li><li><strong>Role:</strong> user-settings-85e7cb74-ee3d-4ba2-814a-bd0153451761-role — the Role whose permissions are referenced.</li><li><strong>Subject:</strong> alex — the user to whom those permissions apply.</li></ul><p>A RoleBinding grants the permissions defined by its referenced Role, within the RoleBinding’s namespace. It does not grant those permissions cluster-wide.</p><h3>Step8.3 — Why Does the RoleBinding Reappear?</h3><p>Deleting the RoleBinding did not permanently remove it. It reappeared after Alex accessed the console.</p><p>This behavior suggests that an OpenShift console component or another controller is recreating the resource to maintain console user-settings functionality. The exact component responsible should be confirmed before attempting to change the behavior.</p><p>Repeatedly deleting the RoleBinding is not a reliable solution because the resource may be recreated.</p><h3>Step8.4- Inspect the Role’s Permissions</h3><p>Before modifying or deleting the RoleBinding, inspect the Role itself:</p><pre>oc get rolebindings -A -o wide | grep alex; oc get clusterrolebindings -o wide | grep alex<br>openshift-console-user-settings                    user-settings-85e7cb74-ee3d-4ba2-814a-bd0153451761-rolebinding       Role/user-settings-85e7cb74-ee3d-4ba2-814a-bd0153451761-role              24m     alex<br><br># describe<br>oc describe role \<br>  user-settings-85e7cb74-ee3d-4ba2-814a-bd0153451761-role \<br>  -n openshift-console-user-settings</pre><p>The output lists the API groups, resources, and verbs permitted by the Role.</p><p>For example, verbs such as get, list, and watch represent read operations, while create, update, patch, and delete permit various forms of modification.</p><p>The RoleBinding’s name alone does not establish what Alex can do. Its actual permissions must be determined from the referenced Role.</p><p>The RoleBinding in openshift-console-user-settings will be recreated automatically even after it is deleted.</p><pre>[root@bastion ~]# oc get rolebindings -A -o wide | grep alex<br>openshift-console-user-settings                    user-settings-85e7cb74-ee3d-4ba2-814a-bd0153451761-rolebinding       Role/user-settings-85e7cb74-ee3d-4ba2-814a-bd0153451761-role              6m37s   alex</pre><h3>Step8.5- Verify Effective Permissions Separately</h3><p>To check whether Alex can list pods across namespaces, list nodes, or access Secrets in a particular namespace, run:</p><pre>oc auth can-i list pods --all-namespaces --as=alex<br>oc auth can-i list nodes --as=alex<br>oc auth can-i list secrets -n kubevirt-os-images --as=alex</pre><p>Alex currently has no permission to list pods across namespaces, list nodes, or list Secrets in kubevirt-os-images. All three checks returned no.</p><p>Output:</p><pre>[root@bastion ~]# oc auth can-i list pods --all-namespaces --as=alex<br>no<br>[root@bastion ~]# oc auth can-i list nodes --as=alex<br>Warning: resource &#39;nodes&#39; is not namespace scoped<br><br>no<br>[root@bastion ~]# oc auth can-i list secrets -n kubevirt-os-images --as=alex<br>no<br>[root@bastion ~]#</pre><p>The console user-settings RoleBinding does not grant these permissions. We can now create your custom Role and bind it to Alex for only the node and pod access you want.</p><p>A no result means the requested operation is not authorized under the permissions evaluated by OpenShift.</p><p>To inspect Alex’s effective permissions more broadly:</p><pre>oc auth can-i --list --as=alex</pre><p>This lists Alex’s effective permissions — This are default permission</p><pre>Resources                                                            Non-Resource URLs                           Resource Names    Verbs<br>buildconfigs/webhooks                                                []                                          []                [create get]<br>buildconfigs.build.openshift.io/webhooks                             []                                          []                [create get]<br>builds/docker                                                        []                                          []                [create]<br>builds/jenkinspipeline                                               []                                          []                [create]<br>builds/optimizeddocker                                               []                                          []                [create]<br>builds/source                                                        []                                          []                [create]<br>selfsubjectrulesreviews                                              []                                          []                [create]<br>selfsubjectreviews.authentication.k8s.io                             []                                          []                [create]<br>selfsubjectaccessreviews.authorization.k8s.io                        []                                          []                [create]<br>selfsubjectrulesreviews.authorization.k8s.io                         []                                          []                [create]<br>selfsubjectrulesreviews.authorization.openshift.io                   []                                          []                [create]<br>builds.build.openshift.io/docker                                     []                                          []                [create]<br>builds.build.openshift.io/jenkinspipeline                            []                                          []                [create]<br>builds.build.openshift.io/optimizeddocker                            []                                          []                [create]<br>builds.build.openshift.io/source                                     []                                          []                [create]<br>oauthaccesstokens                                                    []                                          []                [delete]<br>oauthauthorizetokens                                                 []                                          []                [delete]<br>oauthaccesstokens.oauth.openshift.io                                 []                                          []                [delete]<br>oauthauthorizetokens.oauth.openshift.io                              []                                          []                [delete]<br>cdiconfigs.cdi.kubevirt.io                                           []                                          []                [get list watch]<br>storageprofiles.cdi.kubevirt.io                                      []                                          []                [get list watch]<br>consoleclidownloads.console.openshift.io                             []                                          []                [get list watch]<br>consoleexternalloglinks.console.openshift.io                         []                                          []                [get list watch]<br>consolelinks.console.openshift.io                                    []                                          []                [get list watch]<br>consolenotifications.console.openshift.io                            []                                          []                [get list watch]<br>consoleplugins.console.openshift.io                                  []                                          []                [get list watch]<br>consolequickstarts.console.openshift.io                              []                                          []                [get list watch]<br>consolesamples.console.openshift.io                                  []                                          []                [get list watch]<br>consoleyamlsamples.console.openshift.io                              []                                          []                [get list watch]<br>helmchartrepositories.helm.openshift.io                              []                                          []                [get list watch]<br>virtualmachineclusterinstancetypes.instancetype.kubevirt.io          []                                          []                [get list watch]<br>virtualmachineclusterpreferences.instancetype.kubevirt.io            []                                          []                [get list watch]<br>clusterroles.rbac.authorization.k8s.io                               []                                          []                [get list watch]<br>volumesnapshotclasses.snapshot.storage.k8s.io                        []                                          []                [get list watch]<br>clusterroles                                                         []                                          []                [get list]<br>clusterroles.authorization.openshift.io                              []                                          []                [get list]<br>kubevirts.kubevirt.io                                                []                                          []                [get list]<br>storageclasses.storage.k8s.io                                        []                                          []                [get list]<br>guestfs.subresources.kubevirt.io                                     []                                          []                [get list]<br>version.subresources.kubevirt.io                                     []                                          []                [get list]<br>                                                                     [/.well-known/*]                            []                [get]<br>                                                                     [/.well-known/oauth-authorization-server]   []                [get]<br>                                                                     [/.well-known/oauth-authorization-server]   []                [get]<br>                                                                     [/.well-known]                              []                [get]<br>                                                                     [/.well-known]                              []                [get]<br>                                                                     [/.well-known]                              []                [get]<br>                                                                     [/]                                         []                [get]<br>                                                                     [/]                                         []                [get]<br>                                                                     [/api/*]                                    []                [get]<br>                                                                     [/api/*]                                    []                [get]<br>                                                                     [/api/*]                                    []                [get]<br>                                                                     [/api]                                      []                [get]<br>                                                                     [/api]                                      []                [get]<br>                                                                     [/api]                                      []                [get]<br>                                                                     [/apis/*]                                   []                [get]<br>                                                                     [/apis/*]                                   []                [get]<br>                                                                     [/apis/*]                                   []                [get]<br>                                                                     [/apis]                                     []                [get]<br>                                                                     [/apis]                                     []                [get]<br>                                                                     [/apis]                                     []                [get]<br>                                                                     [/healthz/]                                 []                [get]<br>                                                                     [/healthz]                                  []                [get]<br>                                                                     [/healthz]                                  []                [get]<br>                                                                     [/healthz]                                  []                [get]<br>                                                                     [/livez]                                    []                [get]<br>                                                                     [/livez]                                    []                [get]<br>                                                                     [/oapi/*]                                   []                [get]<br>                                                                     [/oapi/*]                                   []                [get]<br>                                                                     [/oapi]                                     []                [get]<br>                                                                     [/oapi]                                     []                [get]<br>                                                                     [/openapi/*]                                []                [get]<br>                                                                     [/openapi/v2]                               []                [get]<br>                                                                     [/openapi/v2]                               []                [get]<br>                                                                     [/openapi]                                  []                [get]<br>                                                                     [/osapi/]                                   []                [get]<br>                                                                     [/osapi/]                                   []                [get]<br>                                                                     [/osapi]                                    []                [get]<br>                                                                     [/osapi]                                    []                [get]<br>                                                                     [/readyz]                                   []                [get]<br>                                                                     [/readyz]                                   []                [get]<br>                                                                     [/swagger-2.0.0.pb-v1]                      []                [get]<br>                                                                     [/swagger-2.0.0.pb-v1]                      []                [get]<br>                                                                     [/swagger.json]                             []                [get]<br>                                                                     [/swagger.json]                             []                [get]<br>                                                                     [/swaggerapi/*]                             []                [get]<br>                                                                     [/swaggerapi/*]                             []                [get]<br>                                                                     [/swaggerapi]                               []                [get]<br>                                                                     [/swaggerapi]                               []                [get]<br>                                                                     [/version/*]                                []                [get]<br>                                                                     [/version/*]                                []                [get]<br>                                                                     [/version/]                                 []                [get]<br>                                                                     [/version/]                                 []                [get]<br>                                                                     [/version]                                  []                [get]<br>                                                                     [/version]                                  []                [get]<br>                                                                     [/version]                                  []                [get]<br>                                                                     [/version]                                  []                [get]<br>users                                                                []                                          [~]               [get]<br>users.user.openshift.io                                              []                                          [~]               [get]<br>userextras.authentication.k8s.io/scopes.authorization.openshift.io   []                                          []                [impersonate]<br>projects                                                             []                                          []                [list watch]<br>projects.project.openshift.io                                        []                                          []                [list watch]<br>projectrequests                                                      []                                          []                [list]<br>projectrequests.project.openshift.io                                 []                                          []                [list]<br>securitycontextconstraints.security.openshift.io                     []                                          [restricted-v2]   [use]<br>securitycontextconstraints.security.openshift.io                     []                                          [restricted-v3]   [use]<br>[root@bastion ~]# oc auth can-i --list --as=alex<br>Resources                                                            Non-Resource URLs                           Resource Names    Verbs<br>buildconfigs/webhooks                                                []                                          []                [create get]<br>buildconfigs.build.openshift.io/webhooks                             []                                          []                [create get]<br>builds/docker                                                        []                                          []                [create]<br>builds/jenkinspipeline                                               []                                          []                [create]<br>builds/optimizeddocker                                               []                                          []                [create]<br>builds/source                                                        []                                          []                [create]<br>selfsubjectrulesreviews                                              []                                          []                [create]<br>selfsubjectreviews.authentication.k8s.io                             []                                          []                [create]<br>selfsubjectaccessreviews.authorization.k8s.io                        []                                          []                [create]<br>selfsubjectrulesreviews.authorization.k8s.io                         []                                          []                [create]<br>selfsubjectrulesreviews.authorization.openshift.io                   []                                          []                [create]<br>builds.build.openshift.io/docker                                     []                                          []                [create]<br>builds.build.openshift.io/jenkinspipeline                            []                                          []                [create]<br>builds.build.openshift.io/optimizeddocker                            []                                          []                [create]<br>builds.build.openshift.io/source                                     []                                          []                [create]<br>oauthaccesstokens                                                    []                                          []                [delete]<br>oauthauthorizetokens                                                 []                                          []                [delete]<br>oauthaccesstokens.oauth.openshift.io                                 []                                          []                [delete]<br>oauthauthorizetokens.oauth.openshift.io                              []                                          []                [delete]<br>cdiconfigs.cdi.kubevirt.io                                           []                                          []                [get list watch]<br>storageprofiles.cdi.kubevirt.io                                      []                                          []                [get list watch]<br>consoleclidownloads.console.openshift.io                             []                                          []                [get list watch]<br>consoleexternalloglinks.console.openshift.io                         []                                          []                [get list watch]<br>consolelinks.console.openshift.io                                    []                                          []                [get list watch]<br>consolenotifications.console.openshift.io                            []                                          []                [get list watch]<br>consoleplugins.console.openshift.io                                  []                                          []                [get list watch]<br>consolequickstarts.console.openshift.io                              []                                          []                [get list watch]<br>consolesamples.console.openshift.io                                  []                                          []                [get list watch]<br>consoleyamlsamples.console.openshift.io                              []                                          []                [get list watch]<br>helmchartrepositories.helm.openshift.io                              []                                          []                [get list watch]<br>virtualmachineclusterinstancetypes.instancetype.kubevirt.io          []                                          []                [get list watch]<br>virtualmachineclusterpreferences.instancetype.kubevirt.io            []                                          []                [get list watch]<br>clusterroles.rbac.authorization.k8s.io                               []                                          []                [get list watch]<br>volumesnapshotclasses.snapshot.storage.k8s.io                        []                                          []                [get list watch]<br>clusterroles                                                         []                                          []                [get list]<br>clusterroles.authorization.openshift.io                              []                                          []                [get list]<br>kubevirts.kubevirt.io                                                []                                          []                [get list]<br>storageclasses.storage.k8s.io                                        []                                          []                [get list]<br>guestfs.subresources.kubevirt.io                                     []                                          []                [get list]<br>version.subresources.kubevirt.io                                     []                                          []                [get list]<br>                                                                     [/.well-known/*]                            []                [get]<br>                                                                     [/.well-known/oauth-authorization-server]   []                [get]<br>                                                                     [/.well-known/oauth-authorization-server]   []                [get]<br>                                                                     [/.well-known]                              []                [get]<br>                                                                     [/.well-known]                              []                [get]<br>                                                                     [/.well-known]                              []                [get]<br>                                                                     [/]                                         []                [get]<br>                                                                     [/]                                         []                [get]<br>                                                                     [/api/*]                                    []                [get]<br>                                                                     [/api/*]                                    []                [get]<br>                                                                     [/api/*]                                    []                [get]<br>                                                                     [/api]                                      []                [get]<br>                                                                     [/api]                                      []                [get]<br>                                                                     [/api]                                      []                [get]<br>                                                                     [/apis/*]                                   []                [get]<br>                                                                     [/apis/*]                                   []                [get]<br>                                                                     [/apis/*]                                   []                [get]<br>                                                                     [/apis]                                     []                [get]<br>                                                                     [/apis]                                     []                [get]<br>                                                                     [/apis]                                     []                [get]<br>                                                                     [/healthz/]                                 []                [get]<br>                                                                     [/healthz]                                  []                [get]<br>                                                                     [/healthz]                                  []                [get]<br>                                                                     [/healthz]                                  []                [get]<br>                                                                     [/livez]                                    []                [get]<br>                                                                     [/livez]                                    []                [get]<br>                                                                     [/oapi/*]                                   []                [get]<br>                                                                     [/oapi/*]                                   []                [get]<br>                                                                     [/oapi]                                     []                [get]<br>                                                                     [/oapi]                                     []                [get]<br>                                                                     [/openapi/*]                                []                [get]<br>                                                                     [/openapi/v2]                               []                [get]<br>                                                                     [/openapi/v2]                               []                [get]<br>                                                                     [/openapi]                                  []                [get]<br>                                                                     [/osapi/]                                   []                [get]<br>                                                                     [/osapi/]                                   []                [get]<br>                                                                     [/osapi]                                    []                [get]<br>                                                                     [/osapi]                                    []                [get]<br>                                                                     [/readyz]                                   []                [get]<br>                                                                     [/readyz]                                   []                [get]<br>                                                                     [/swagger-2.0.0.pb-v1]                      []                [get]<br>                                                                     [/swagger-2.0.0.pb-v1]                      []                [get]<br>                                                                     [/swagger.json]                             []                [get]<br>                                                                     [/swagger.json]                             []                [get]<br>                                                                     [/swaggerapi/*]                             []                [get]<br>                                                                     [/swaggerapi/*]                             []                [get]<br>                                                                     [/swaggerapi]                               []                [get]<br>                                                                     [/swaggerapi]                               []                [get]<br>                                                                     [/version/*]                                []                [get]<br>                                                                     [/version/*]                                []                [get]<br>                                                                     [/version/]                                 []                [get]<br>                                                                     [/version/]                                 []                [get]<br>                                                                     [/version]                                  []                [get]<br>                                                                     [/version]                                  []                [get]<br>                                                                     [/version]                                  []                [get]<br>                                                                     [/version]                                  []                [get]<br>users                                                                []                                          [~]               [get]<br>users.user.openshift.io                                              []                                          [~]               [get]<br>userextras.authentication.k8s.io/scopes.authorization.openshift.io   []                                          []                [impersonate]<br>projects                                                             []                                          []                [list watch]<br>projects.project.openshift.io                                        []                                          []                [list watch]<br>projectrequests                                                      []                                          []                [list]<br>projectrequests.project.openshift.io                                 []                                          []                [list]<br>securitycontextconstraints.security.openshift.io                     []                                          [restricted-v2]   [use]<br>securitycontextconstraints.security.openshift.io                     []                                          [restricted-v3]   [use]</pre><p>Now by default this user will not have any permission of the resources in Openshift Cluster</p><p>These checks are more reliable than inferring access from the presence of a RoleBinding or the visibility of a project name in the console.</p><p>The RoleBinding found for Alex belongs to the openshift-console-user-settings namespace and references a namespace-scoped Role. Its presence does not, by itself, indicate that Alex has access to pods, nodes, Secrets, or other workload resources.</p><p>The correct troubleshooting process is to inspect the referenced Role, verify effective permissions using oc auth can-i, and identify the component recreating the binding before attempting to change its behavior.</p><h4>Note:</h4><p>OpenShift has default RBAC permissions. You did not attach a role to Alex, but the cluster can still grant certain permissions through default roles and bindings.</p><p>Your result confirms Alex can list projects. It does not mean he has access to the resources inside those projects.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*19JxJ0GLcqfAnrckVt-4lw.png" /></figure><h3>Step 9— Create a custom ClusterRole for nodes and pods only:</h3><pre>oc create clusterrole alex-node-pod-view --verb=get,list,watch --resource=nodes,pods<br><br># Output<br>[root@bastion ~]# oc create clusterrole alex-node-pod-view --verb=get,list,watch --resource=nodes,pods<br>clusterrole.rbac.authorization.k8s.io/alex-node-pod-view created</pre><p>This grants only get, list, and watch permissions on nodes and pods.</p><h3>Step 10- Verify if it is created</h3><pre>[root@bastion ~]# oc get clusterrole alex-node-pod-view<br>NAME                 CREATED AT<br>alex-node-pod-view   2026-10-10T06:31:45Z<br>[root@bastion ~]#</pre><h3>Step 11 — Assign the alex-node-pod-view ClusterRole to Alex:</h3><pre>oc adm policy add-cluster-role-to-user alex-node-pod-view alex<br><br># Get role<br>[root@bastion ~]# oc adm policy add-cluster-role-to-user alex-node-pod-view alex<br>clusterrole.rbac.authorization.k8s.io/alex-node-pod-view added: &quot;alex&quot;</pre><p>This grants Alex the read-only node and pod permissions defined in your custom ClusterRole.</p><h3>Step 12- Verify Alex’s permissions:</h3><pre>oc auth can-i list pods --all-namespaces --as=alex<br>oc auth can-i list nodes --as=alex<br>oc auth can-i list secrets -n kubevirt-os-images --as=alex</pre><h3>13. Understand the Results</h3><p><strong>Pods — </strong><strong>yes:</strong> Alex can list pods across all namespaces. This is because the ClusterRoleBinding grants the list permission on pods cluster-wide.</p><p><strong>Nodes — </strong><strong>yes:</strong> Alex can list cluster nodes. Nodes are cluster-scoped resources, so Kubernetes displays a warning that they are not namespace-scoped. This warning is informational and does not indicate an error.</p><p><strong>Secrets — </strong><strong>no:</strong> Alex cannot list Secrets in the kubevirt-os-images namespace. The custom ClusterRole does not grant permissions on Secrets.</p><p>Alex now has read-only access to nodes and pods across the cluster, without permission to list Secrets in kubevirt-os-images through this custom role. The get, list, and watch verbs allow reading resource details and monitoring changes; they do not allow creating, updating, or deleting nodes or pods.</p><p><strong>Note:</strong> These results reflect Alex’s effective permissions at the time of verification. Any additional RoleBindings or ClusterRoleBindings could grant further permissions independently.</p><h3>View pods in all project</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6cgjY2vNG3FJCZ12QLfIAQ.png" /></figure><h3>Take token from console</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*D9hMEaj0j-vOOCw0VNP9aA.png" /></figure><h3>Click on local-users</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fCqheeWTY8sfPylBIJyAgA.png" /></figure><h3>Click on Display Token</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/649/1*okFmSsUZaZNL-7SyPD83Xw.png" /></figure><h3>Copy the token</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7FYE5t5J6D-sNmcoQQTlpw.png" /></figure><h3>Login to the Cluster</h3><pre>[root@bastion auth]# oc login - token=sha256~M61f6e9Jn9KvWtuvDkLg6hjN8eUqycA0HC6-B_M6BNk - server=https://api.sno.manohar.in:6443<br>Logged into &quot;https://api.sno.manohar.in:6443&quot; as &quot;alex&quot; using the token provided.<br>You have one project on this server: &quot;kubevirt-os-images&quot;<br>Using project &quot;kubevirt-os-images&quot;.</pre><h3>Verify the accesses</h3><pre>[root@bastion auth]# oc  get nodes -o wide<br>NAME             STATUS   ROLES                         AGE    VERSION   INTERNAL-IP     EXTERNAL-IP   OS-IMAGE                                                KERNEL-VERSION                  CONTAINER-RUNTIME<br>sno.manohar.in   Ready    control-plane,master,worker   108d   v1.34.8   192.168.0.250   &lt;none&gt;        Red Hat Enterprise Linux CoreOS 9.6.20260602-0 (Plow)   5.14.0-570.118.1.el9_6.x86_64   cri-o://1.34.8-3.rhaos4.21.git91b4b0b.el9</pre><p>Node level access is available.</p><p>Pod Access is available</p><pre>[root@bastion auth]# oc get pods -A<br>NAMESPACE                                          NAME                                                              READY   STATUS      RESTARTS       AGE<br>default                                            guestbook-ui-84774bdc6f-5kdvq                                     1/1     Running     10             88d<br>default                                            snomanoharin-debug-lvmnp                                          0/1     Completed   0              46d<br>default                                            snomanoharin-debug-pgr97                                          0/1     Completed   0              46d<br>kubevirt-hyperconverged                            aaq-operator-84495dc84c-dnd7k                                     1/1     Running     2              2d16h<br>kubevirt-hyperconverged                            bridge-marker-h2bxr                                               1/1     Running     2              2d16h<br>kubevirt-hyperconverged                            cdi-apiserver-766767d995-g429t                                    1/1     Running     3              2d16h<br>kubevirt-hyperconverged                            cdi-deployment-c89c55cb-trwxh                                     1/1     Running     6 (16h ago)    2d16h<br>kubevirt-hyperconverged                            cdi-operator-78864cbf64-zknth                                     1/1     Running     9 (16h ago)    2d16h<br>kubevirt-hyperconverged                            cdi-uploadproxy-5d85f54c8c-sccrq                                  1/1     Running     2              2d16h<br>kubevirt-hyperconverged                            cluster-network-addons-operator-75f884cfdf-kzzqc                  2/2     Running     4              2d16h<br>kubevirt-hyperconverged                            hco-operator-bfcf4cd85-667rl                                      1/1     Running     9 (16h ago)    2d16h<br>kubevirt-hyperconverged                            hco-webhook-55b8ff7f78-9rrdp                                      1/1     Running     2              2d16h<br>kubevirt-hyperconverged                            hostpath-provisioner-operator-54dd865576-9279t                    1/1     Running     2              2d16h<br>kubevirt-hyperconverged                            hyperconverged-cluster-cli-download-5dfd4c5899-pgwsv              1/1     Running     2              2d16h<br>kubevirt-hyperconverged                            kube-cni-linux-bridge-plugin-grhh9                                1/1     Running     2              2d16h<br>kubevirt-hyperconverged                            kubemacpool-cert-manager-7d4b96867f-vs9f6                         1/1     Running     2              2d16h<br>kubevirt-hyperconverged                            kubemacpool-mac-controller-manager-9d874bd4f-n29vv                2/2     Running     4              2d16h<br>kubevirt-hyperconverged                            kubevirt-apiserver-proxy-7b95bb657d-hcnxm                         1/1     Running     2              2d16h<br>kubevirt-hyperconverged                            kubevirt-console-plugin-5fc4687f6d-zq5fk                          1/1     Running     2              2d16h<br>kubevirt-hyperconverged                            kubevirt-ipam-controller-manager-c77d55866-wh6zm                  1/1     Running     2              2d16h<br>kubevirt-hyperconverged                            ssp-operator-f994f6c7d-s9b7v                                      1/1     Running     3              2d16h<br>kubevirt-hyperconverged                            virt-api-59447797c4-c5vv9                                         1/1     Running     2              2d15h<br>kubevirt-hyperconverged                            virt-controller-6d8f6b6dc7-smlsq                                  1/1     Running     2              2d15h<br>kubevirt-hyperconverged                            virt-exportproxy-67c465df58-qvnd2                                 1/1     Running     2              2d15h<br>kubevirt-hyperconverged                            virt-handler-q9hrz                                                1/1     Running     2              2d15h<br>kubevirt-hyperconverged                            virt-operator-578c864ddf-4jjpb                                    1/1     Running     2              2d16h<br>kubevirt-hyperconverged                            virt-operator-578c864ddf-grjst                                    1/1     Running     2              2d16h<br>kubevirt-hyperconverged                            virt-template-validator-85459896b9-qrfcb                          1/1     Running     6 (16h ago)    2d15h<br>openshift-apiserver-operator                       openshift-apiserver-operator-665864cf9-8bl4b                      1/1     Running     14             108d<br>openshift-apiserver                                apiserver-7949c9d647-dmgd4                                        2/2     Running     26             108d<br>openshift-authentication-operator                  authentication-operator-5844f99654-vgkgb                          1/1     Running     15             108d<br>openshift-authentication                           oauth-openshift-c9788c546-vvcdh                                   1/1     Running     0              109m<br>openshift-catalogd                                 catalogd-controller-manager-5c994f9475-pks95                      1/1     Running     14             108d<br>openshift-cloud-controller-manager-operator        cluster-cloud-controller-manager-operator-59d865cd6c-54gnn        3/3     Running     39             108d<br>openshift-cloud-credential-operator                cloud-credential-operator-5cf95ff9bf-cdch9                        2/2     Running     27             108d<br>openshift-cluster-machine-approver                 machine-approver-d85c9648d-t5wlw                                  2/2     Running     47 (16h ago)   108d<br>openshift-cluster-node-tuning-operator             cluster-node-tuning-operator-74dbfb886d-jnjgf                     1/1     Running     13             108d<br>openshift-cluster-node-tuning-operator             tuned-tdcw9                                                       1/1     Running     13             108d<br>openshift-cluster-olm-operator                     cluster-olm-operator-d59cfccf8-flf4g                              1/1     Running     13             108d<br>openshift-cluster-samples-operator                 cluster-samples-operator-b4cb7d45f-4xvqd                          2/2     Running     26             108d<br>openshift-cluster-storage-operator                 cluster-storage-operator-54c9f5cbd4-l8k4d                         1/1     Running     14             108d<br>openshift-cluster-storage-operator                 csi-snapshot-controller-65c97c7f9b-45lg2                          1/1     Running     18             108d<br>openshift-cluster-storage-operator                 csi-snapshot-controller-operator-7948fd979-7stpm                  1/1     Running     13             108d<br>openshift-cluster-storage-operator                 volume-data-source-validator-d9db7c44f-qcsjz                      1/1     Running     13             108d<br>openshift-cluster-version                          cluster-version-operator-786fc65f7-zk464                          1/1     Running     13             108d<br>openshift-config-operator                          openshift-config-operator-74bdfbc48d-2ghpn                        1/1     Running     14             108d<br>openshift-console-operator                         console-operator-74cd95dbf6-xhdht                                 1/1     Running     18             108d<br>openshift-console                                  console-79757b884b-5vjcn                                          1/1     Running     2              2d16h<br>openshift-console                                  downloads-59fbd76858-mwws5                                        1/1     Running     13             108d<br>openshift-controller-manager-operator              openshift-controller-manager-operator-66644d79d6-vtxg6            1/1     Running     14             108d<br>openshift-controller-manager                       controller-manager-6fb9bbd974-kkdrm                               1/1     Running     3              9d<br>openshift-dns-operator                             dns-operator-86759fdcbf-bv8kd                                     2/2     Running     26             108d<br>openshift-dns                                      dns-default-wrfvv                                                 2/2     Running     26             108d<br>openshift-dns                                      node-resolver-hm6p5                                               1/1     Running     13             108d<br>openshift-etcd-operator                            etcd-operator-6b44d9f47d-lmtsg                                    1/1     Running     15             108d<br>openshift-etcd                                     etcd-sno.manohar.in                                               5/5     Running     65             108d<br>openshift-etcd                                     installer-1-sno.manohar.in                                        0/1     Completed   0              108d<br>openshift-etcd                                     installer-2-sno.manohar.in                                        0/1     Completed   0              108d<br>openshift-image-registry                           cluster-image-registry-operator-fcf84c859-7mxsw                   1/1     Running     13             108d<br>openshift-image-registry                           image-pruner-29802240-rzvvl                                       0/1     Completed   0              40d<br>openshift-image-registry                           image-pruner-29856960-6vwl2                                       0/1     Completed   0              2d8h<br>openshift-image-registry                           image-pruner-29859840-248cs                                       0/1     Completed   0              8h<br>openshift-image-registry                           node-ca-x7q8q                                                     1/1     Running     13             108d<br>openshift-ingress-canary                           ingress-canary-5b4jm                                              1/1     Running     13             108d<br>openshift-ingress-operator                         ingress-operator-5b65c66d56-rpxkp                                 2/2     Running     35             108d<br>openshift-ingress                                  router-default-7f7c9c5f95-9lwmc                                   1/1     Running     10 (16h ago)   9d<br>openshift-insights                                 insights-operator-94469c6b6-ttglw                                 1/1     Running     14             108d<br>openshift-insights                                 insights-runtime-extractor-tq5sn                                  3/3     Running     40             108d<br>openshift-kube-apiserver-operator                  kube-apiserver-operator-7bf9f54c6c-x6wvj                          1/1     Running     6              46d<br>openshift-kube-apiserver                           installer-10-sno.manohar.in                                       0/1     Completed   0              56d<br>openshift-kube-apiserver                           installer-11-sno.manohar.in                                       0/1     Completed   0              47d<br>openshift-kube-apiserver                           installer-14-sno.manohar.in                                       0/1     Completed   0              46d<br>openshift-kube-apiserver                           installer-15-sno.manohar.in                                       0/1     Completed   0              46d<br>openshift-kube-apiserver                           installer-16-sno.manohar.in                                       0/1     Completed   0              46d<br>openshift-kube-apiserver                           installer-17-sno.manohar.in                                       0/1     Completed   0              46d<br>openshift-kube-apiserver                           installer-18-sno.manohar.in                                       0/1     Completed   0              43d<br>openshift-kube-apiserver                           installer-19-sno.manohar.in                                       0/1     Completed   0              39d<br>openshift-kube-apiserver                           installer-21-sno.manohar.in                                       0/1     Completed   0              9d<br>openshift-kube-apiserver                           installer-3-retry-1-sno.manohar.in                                0/1     Completed   0              108d<br>openshift-kube-apiserver                           installer-4-sno.manohar.in                                        0/1     Completed   0              108d<br>openshift-kube-apiserver                           installer-5-sno.manohar.in                                        0/1     Completed   0              108d<br>openshift-kube-apiserver                           installer-6-sno.manohar.in                                        0/1     Completed   0              107d<br>openshift-kube-apiserver                           installer-7-sno.manohar.in                                        0/1     Completed   0              90d<br>openshift-kube-apiserver                           installer-8-sno.manohar.in                                        0/1     Completed   0              90d<br>openshift-kube-apiserver                           installer-9-sno.manohar.in                                        0/1     Completed   0              69d<br>openshift-kube-apiserver                           kube-apiserver-sno.manohar.in                                     5/5     Running     15             9d<br>openshift-kube-controller-manager-operator         kube-controller-manager-operator-57b5dfd764-zhmh5                 1/1     Running     14             108d<br>openshift-kube-controller-manager                  installer-4-sno.manohar.in                                        0/1     Completed   0              108d<br>openshift-kube-controller-manager                  installer-5-sno.manohar.in                                        0/1     Completed   0              108d<br>openshift-kube-controller-manager                  installer-6-sno.manohar.in                                        0/1     Completed   0              108d<br>openshift-kube-controller-manager                  kube-controller-manager-sno.manohar.in                            4/4     Running     89             46d<br>openshift-kube-scheduler-operator                  openshift-kube-scheduler-operator-785c776cfd-wdnlg                1/1     Running     14             108d<br>openshift-kube-scheduler                           installer-5-sno.manohar.in                                        0/1     Completed   0              108d<br>openshift-kube-scheduler                           openshift-kube-scheduler-sno.manohar.in                           3/3     Running     39             108d<br>openshift-kube-storage-version-migrator-operator   kube-storage-version-migrator-operator-5977c8bf77-7f9t9           1/1     Running     14             108d<br>openshift-kube-storage-version-migrator            migrator-7dd45b7bbc-r58rj                                         2/2     Running     26             108d<br>openshift-machine-api                              cluster-autoscaler-operator-f75bccff9-v7ch4                       2/2     Running     28             108d<br>openshift-machine-api                              cluster-baremetal-operator-598cbfb9-97p72                         2/2     Running     28             108d<br>openshift-machine-api                              control-plane-machine-set-operator-694d6d4559-wkx4r               1/1     Running     13             108d<br>openshift-machine-api                              machine-api-operator-668b876bb6-qktgx                             2/2     Running     27             108d<br>openshift-machine-config-operator                  kube-rbac-proxy-crio-sno.manohar.in                               1/1     Running     296            108d<br>openshift-machine-config-operator                  machine-config-controller-585947c96f-xrp7l                        2/2     Running     26             108d<br>openshift-machine-config-operator                  machine-config-daemon-26nrs                                       2/2     Running     26             108d<br>openshift-machine-config-operator                  machine-config-operator-55d6f586cd-njvf9                          1/1     Running     13             108d<br>openshift-machine-config-operator                  machine-config-server-s4q2q                                       1/1     Running     13             108d<br>openshift-marketplace                              0d834871c1631b0c45cd6b7a8675c9a7b5a6b66c346a5ac0a3ca8b5bc1lclq5   0/1     Completed   0              55d<br>openshift-marketplace                              3774f6eb5037dd71cd75f6c41dae039b9a57d27c4f7d39c0084d4ee966c456v   0/1     Completed   0              45d<br>openshift-marketplace                              5175b81f93af54073c0ef008fcd1c13715ca81506f457b2a2ad4cec667cfjqz   0/1     Completed   0              41d<br>openshift-marketplace                              5ce75dcee0168fb41531bb322398d7807d342d3c498f52562173c4b89952bmq   0/1     Completed   0              9d<br>openshift-marketplace                              61c5fcacf340959c58a72034dfff2b5a2eaf496379168c0111e398692cd4vdz   0/1     Completed   0              9d<br>openshift-marketplace                              a7c1ab7cf2eb8f83756a06be5ae637269d44cd85e453332be9c16f7aafg9l25   0/1     Completed   0              2d16h<br>openshift-marketplace                              certified-operators-7zz4w                                         1/1     Running     0              12h<br>openshift-marketplace                              community-operators-dgg7w                                         1/1     Running     0              42m<br>openshift-marketplace                              ddb2fe37bd005b9c3de918b4b093dd8d23ace2750fd83284d97baaf4139pf59   0/1     Completed   0              2d16h<br>openshift-marketplace                              fc41476e0ee23173d9cd9ca701f3457b38f68b02be3ef059ec1d7e297dr6smv   0/1     Completed   0              56d<br>openshift-marketplace                              ff19cdf0c48a3cfd69a66537cb4a311de99fae7d2ffc85147330cdba84ll89p   0/1     Completed   0              55d<br>openshift-marketplace                              marketplace-operator-84ddc8df69-5dhgf                             1/1     Running     18             108d<br>openshift-marketplace                              redhat-marketplace-s6wwp                                          1/1     Running     0              42m<br>openshift-marketplace                              redhat-operators-jpd5v                                            1/1     Running     0              12h<br>openshift-monitoring                               alertmanager-main-0                                               6/6     Running     24             41d<br>openshift-monitoring                               cluster-monitoring-operator-59bc95f786-nkmr4                      1/1     Running     13             108d<br>openshift-monitoring                               kube-state-metrics-77cf47f584-vcl4q                               3/3     Running     39             108d<br>openshift-monitoring                               metrics-server-c944c8c-7g9jv                                      1/1     Running     20             9d<br>openshift-monitoring                               monitoring-plugin-654d5fdf85-gnprs                                1/1     Running     13             108d<br>openshift-monitoring                               node-exporter-qr799                                               2/2     Running     26             108d<br>openshift-monitoring                               openshift-state-metrics-5c4f88ddcb-mv6zw                          3/3     Running     39             108d<br>openshift-monitoring                               prometheus-k8s-0                                                  6/6     Running     78             108d<br>openshift-monitoring                               prometheus-operator-7679c8bbb6-85sg9                              2/2     Running     26             108d<br>openshift-monitoring                               prometheus-operator-admission-webhook-78b64f6b88-b6j9x            1/1     Running     13             108d<br>openshift-monitoring                               telemeter-client-57777fdbf7-rcjl2                                 3/3     Running     39             108d<br>openshift-monitoring                               thanos-querier-5cb7c774fc-ptsmd                                   6/6     Running     78             108d<br>openshift-mtv                                      forklift-api-5bdfb4d474-4v86v                                     1/1     Running     2              2d16h<br>openshift-mtv                                      forklift-cli-download-74b4458b98-6w7m5                            1/1     Running     2              2d16h<br>openshift-mtv                                      forklift-controller-5c798587d9-ljlpd                              2/2     Running     4              2d16h<br>openshift-mtv                                      forklift-operator-845cc6c5ff-rrvhz                                1/1     Running     3              9d<br>openshift-mtv                                      forklift-ova-proxy-54f9548d8c-rdjxh                               1/1     Running     2              2d16h<br>openshift-mtv                                      forklift-ui-plugin-68dc754ddc-4jrl9                               1/1     Running     2              2d16h<br>openshift-mtv                                      forklift-validation-68585cb87-lrm9r                               1/1     Running     2              2d16h<br>openshift-mtv                                      forklift-volume-populator-controller-7d6699f664-swjwq             1/1     Running     2              2d16h<br>openshift-multus                                   multus-additional-cni-plugins-thc82                               1/1     Running     13             108d<br>openshift-multus                                   multus-admission-controller-6cb58b49cf-hbgb7                      2/2     Running     24             108d<br>openshift-multus                                   multus-h694r                                                      1/1     Running     27 (16h ago)   108d<br>openshift-multus                                   network-metrics-daemon-w94z4                                      2/2     Running     26             108d<br>openshift-network-console                          networking-console-plugin-59578b75ff-vrskr                        1/1     Running     13             108d<br>openshift-network-diagnostics                      network-check-source-85d946846d-b8xqx                             1/1     Running     13             108d<br>openshift-network-diagnostics                      network-check-target-w95f5                                        1/1     Running     13             108d<br>openshift-network-node-identity                    network-node-identity-tqhbw                                       2/2     Running     27             108d<br>openshift-network-operator                         iptables-alerter-wfdm2                                            1/1     Running     13             108d<br>openshift-network-operator                         network-operator-cfd7b86bd-ngnxv                                  1/1     Running     14             108d<br>openshift-oauth-apiserver                          apiserver-7dbc979c4c-q55cj                                        1/1     Running     13             108d<br>openshift-operator-controller                      operator-controller-controller-manager-5f4c47dd99-8g8tz           1/1     Running     14             108d<br>openshift-operator-lifecycle-manager               catalog-operator-7f68c544c9-dp4wx                                 1/1     Running     15             108d<br>openshift-operator-lifecycle-manager               olm-operator-6b97b47b45-wk8gv                                     1/1     Running     15             108d<br>openshift-operator-lifecycle-manager               package-server-manager-658d6847d-2gpg9                            2/2     Running     26             108d<br>openshift-operator-lifecycle-manager               packageserver-5c48dc6b44-t7b2n                                    1/1     Running     13             108d<br>openshift-ovn-kubernetes                           ovnkube-control-plane-646fcb7b49-kn5vs                            2/2     Running     28             108d<br>openshift-ovn-kubernetes                           ovnkube-node-sj2vd                                                8/8     Running     26 (16h ago)   9d<br>openshift-route-controller-manager                 route-controller-manager-7b74899984-qmkns                         1/1     Running     3              9d<br>openshift-service-ca-operator                      service-ca-operator-56b798cf56-s9pg8                              1/1     Running     14             108d<br>openshift-service-ca                               service-ca-5cb69dc9b9-qrgwq                                       1/1     Running     13             108d</pre><p>We don’t have access to any other resources than pod and node</p><pre>[root@bastion auth]# oc get deploy<br>Error from server (Forbidden): deployments.apps is forbidden: User &quot;alex&quot; cannot list resource &quot;deployments&quot; in API group &quot;apps&quot; in the namespace &quot;kubevirt-os-images&quot;<br>[root@bastion auth]# oc get secret<br>Error from server (Forbidden): secrets is forbidden: User &quot;alex&quot; cannot list resource &quot;secrets&quot; in API group &quot;&quot; in the namespace &quot;kubevirt-os-images&quot;</pre><ul><li>oc get deploy → Forbidden because Alex has no permission to list Deployments.</li><li>oc get secret → Forbidden because Alex has no permission to list Secrets.</li><li>oc get nodes and oc get pods -A should work with the custom ClusterRole you assigned.</li></ul><p>Alex has read-only access to nodes and pods only, not Deployments or Secrets.</p><h3>Conclusion</h3><p>In this blog, we explored how to configure HTPasswd authentication in OpenShift, from creating a local user and securely storing password hashes in a Secret to configuring the OAuth identity provider and verifying login through the web console. We also created a custom ClusterRole and assigned read-only access to nodes and pods, demonstrating how to apply the principle of least privilege using OpenShift RBAC.</p><p>This setup provides a straightforward way to manage local users in lab environments and understand the relationship between authentication and authorization. By granting only the permissions users require, administrators can maintain tighter access control and reduce unnecessary privileges. LDAP and FreeIPA integration are covered separately in another blog.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6813d6f3c8ef" width="1" height="1" alt=""><hr><p><a href="https://medium.com/devops-cloud-engineering-hub/openshift-htpasswd-authentication-user-creation-oauth-integration-rbac-access-control-6813d6f3c8ef">OpenShift HTPasswd Authentication: User Creation, OAuth Integration &amp; RBAC Access Control</a> was originally published in <a href="https://medium.com/devops-cloud-engineering-hub">devops-cloud-engineering-hub</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>