<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ahmed Draz on Medium]]></title>
        <description><![CDATA[Stories by Ahmed Draz on Medium]]></description>
        <link>https://medium.com/@ahmeddraz?source=rss-55d320153012------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*XR_PLdw8QwpdxJF9</url>
            <title>Stories by Ahmed Draz on Medium</title>
            <link>https://medium.com/@ahmeddraz?source=rss-55d320153012------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 04:17:04 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ahmeddraz/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Deploy LLM Models on OpenShift]]></title>
            <link>https://medium.com/@ahmeddraz/deploy-llm-models-on-openshift-84ecb014f09a?source=rss-55d320153012------2</link>
            <guid isPermaLink="false">https://medium.com/p/84ecb014f09a</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[openshift]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[mlops]]></category>
            <dc:creator><![CDATA[Ahmed Draz]]></dc:creator>
            <pubDate>Sat, 17 Jan 2026 12:02:52 GMT</pubDate>
            <atom:updated>2026-01-17T12:02:52.901Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CntqZpgtYBSYcqxQm-9aaw.png" /></figure><p>Operators make life easier, but they are not always an option. In this post, I’ll walk through a practical way to deploy large language models on OpenShift <strong>without</strong> relying on the OpenShift AI or NVIDIA operators. The approach uses <strong>llama.cpp</strong> as a lightweight runtime engine and runs a <strong>quantized GGUF model</strong> — <em>granite-4.0-h-350m-Q4_K_M.gguf</em> — to enable efficient inference with minimal dependencies.</p><p>This is not intended to replace production-grade solutions. Platforms like <strong>OpenShift AI</strong>, which leverage <strong>KServe</strong> in the background and provide a rich set of optimized runtimes for different AI frameworks, are better suited for enterprise-scale deployments. Instead, this approach focuses on <strong>development, experimentation, and environments where only native Kubernetes and OpenShift resources are available or desired</strong>.</p><h3>Why This Model?</h3><p>The model choice in this setup is driven primarily by <strong>infrastructure constraints and deployment simplicity</strong>, rather than raw model size or accuracy. The selected model — <em>granite-4.0-h-350m-Q4_K_M.gguf</em> — is a <strong>quantized GGUF model</strong>, making it well-suited for environments that rely solely on <strong>CPU-based inference</strong>.</p><p>Thanks to its compatibility with <strong>llama.cpp</strong> and the <strong>GGML</strong> backend, the model can run efficiently without GPU acceleration while maintaining reasonable performance and memory usage. Quantization significantly reduces the model footprint, allowing it to operate within the limits of a modest OpenShift cluster that does not provide GPU-enabled nodes.</p><p>This makes the model a practical choice for development, testing, and lightweight inference workloads, especially in clusters where adding GPUs or specialized operators is not feasible.</p><h3>Deployment Procedure</h3><p>The deployment is intentionally kept simple, relying only on native OpenShift and Kubernetes resources. The process is divided into three main steps:</p><ol><li><strong>Build the model-serving container image</strong><br>We start by creating a container image that serves the LLM using <strong>llama.cpp</strong>. A <strong>multi-stage Docker build</strong> is used to compile the runtime and package the quantized model efficiently, resulting in a minimal and portable image. This approach helps reduce image size, improve build reproducibility, and avoid unnecessary runtime dependencies. The details of this build process are covered in the next section.</li><li><strong>Deploy the model on OpenShift</strong><br>Once the image is available, we create a standard <strong>Kubernetes Deployment</strong> that runs the model server. The deployment defines resource requirements, container arguments, and basic configuration needed to start the llama.cpp inference service. No custom resources or operators are involved — only a familiar Kubernetes deployment.</li><li><strong>Expose and validate the model endpoint</strong><br>Finally, the deployment is exposed using a Kubernetes <strong>Service</strong> and <strong>ingress </strong>(and optionally an OpenShift <strong>Route</strong>) to make the model API accessible. With the endpoint in place, we can send test requests to verify that the model is running correctly and responding as expected.</li></ol><p>By following these steps, you’ll have a fully functional LLM inference endpoint running on OpenShift, suitable for development, testing, or environments where operator-based solutions are not available.</p><h3>Build the Model-Serving Container Image</h3><p>To keep the final container image <strong>secure, lightweight, and free of unnecessary build-time dependencies</strong>, we use a <strong>multi-stage Docker build</strong>. This approach allows us to separate model acquisition from model serving, ensuring that only what is strictly required at runtime is included in the final image.</p><h4>Stage 1: The Builder</h4><p>The first stage is responsible solely for <strong>downloading and validating the model artifacts.</strong></p><ul><li><strong>Base Image:</strong> registry.access.redhat.com/ubi9/python-311</li><li><strong>Purpose:</strong> Securely fetch the quantized model weights.</li><li><strong>Process:<br>1. </strong>Install the huggingface-hub Python package.<br><strong>2.</strong> Execute a small helper script (download_model.py) to download the required .gguf model file.<br><strong>3.</strong> Verify the integrity of the downloaded file to ensure it has not been corrupted or tampered with.</li></ul><p>This stage includes Python tooling and dependencies, but none of them are carried forward into the final image.</p><h3>Stage 2: The Runtime (Final Image)</h3><p>The second stage produces the <strong>final runnable image</strong> that will be deployed on OpenShift.</p><ul><li><strong>Base Image:</strong> ghcr.io/ggml-org/llama.cpp:server</li><li><strong>Purpose:</strong> Run the LLM inference engine using llama.cpp.</li><li><strong>Process:<br>1. </strong>Copy <strong>only</strong> the required .gguf model file from the builder stage.<br><strong>2.</strong> Discard all build-time dependencies, including Python packages, pip caches, and intermediate files.<br><strong>3.</strong> Set the container entry point to start the llama-server process.</li></ul><p>By isolating the model download in the builder stage and keeping the runtime image minimal, this design improves security, reduces image size, and aligns well with OpenShift best practices for containerized workloads.</p><pre>FROM registry.access.redhat.com/ubi9/python-311:latest as builder<br><br>USER root<br><br># Install the huggingface library<br>RUN pip install huggingface-hub<br><br>WORKDIR /build<br><br># Copy the python script<br>COPY download_model.py .<br><br># Run the download (resulting file will be in /build/granite-4.0-h-350m-Q4_K_M.gguf)<br>RUN python download_model.py<br><br>FROM ghcr.io/ggml-org/llama.cpp:server<br><br># Copy ONLY the model file from the builder stage<br>COPY --from=builder /build/granite-4.0-h-350m-Q4_K_M.gguf /model.gguf<br><br># Expose the API port<br>EXPOSE 8080<br><br># Command to start the server with the copied model<br># --host 0.0.0.0: Listen on all interfaces<br># --port 8080: OpenAI compatible port<br>CMD [&quot;-m&quot;, &quot;/model.gguf&quot;, &quot;--host&quot;, &quot;0.0.0.0&quot;, &quot;--port&quot;, &quot;8080&quot;]</pre><p><strong>download_model.py</strong> script is responsible for securely and efficiently acquiring the specific model file required</p><pre>from huggingface_hub import hf_hub_download  <br>import shutil<br>import os<br><br># Define the model repo and specific file<br>repo_id = &quot;ibm-granite/granite-4.0-h-350m-GGUF&quot;  <br>filename = &quot;granite-4.0-h-350m-Q4_K_M.gguf&quot;<br><br>print(f&quot;Downloading {filename} from {repo_id}...&quot;)<br><br># Download the file to the local cache<br>model_path = hf_hub_download(<br>    repo_id=repo_id,<br>    filename=filename,<br>    local_dir=&quot;.&quot;,        # Download to current directory<br>    local_dir_use_symlinks=False # Ensure it&#39;s a real file, not a symlink<br>)</pre><p>We use the official Hugging Face Python library. This is more robust than using curl or wget because it handles authentication (if needed), redirects, and verification automatically.</p><pre>podman build -t granite-api .</pre><h3>Deploy the Model on OpenShift</h3><p>With the model-serving image built, the next step is to deploy it using a standard <strong>Kubernetes Deployment</strong>. This keeps the setup simple and avoids any dependency on custom resources or operators.</p><h3>Deployment Configuration</h3><p><strong>A. The Deployment</strong></p><ul><li><strong>Replicas:</strong> 1<br>The deployment starts with a single replica, which is sufficient for development and testing. The workload can be scaled horizontally on demand; however, scaling LLM requires some modification on loadbalancing infra maintian consistancy (using the same KV cache) of the same session.</li><li><strong>Resources:</strong><br>Resource requests and limits are defined to ensure predictable scheduling and prevent resource starvation:</li><li><strong>Requests:<br> </strong>CPU: 1 , Memory: 1Gi</li><li><strong>Limits:<br> </strong>CPU: 4 , Memory: 2Gi</li><li>This configuration provides enough headroom for CPU-based inference while keeping the workload bounded within the cluster’s available resources.</li><li><strong>Health Checks:</strong><br>Probes are configured to improve reliability and ensure the service behaves correctly within OpenShift:</li><li><strong>Readiness Probe:</strong><br>Ensures the model is fully loaded and ready to serve requests before the pod is added to the service endpoints.</li><li><strong>Liveness Probe:</strong><br>Monitors the API responsiveness and triggers a container restart if the inference server becomes unresponsive.</li></ul><p>This deployment model aligns well with CPU-only OpenShift clusters and provides a solid baseline for running lightweight LLM inference workloads.</p><pre>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: granite-deployment<br>  labels:<br>    app: granite-api<br>spec:<br>  replicas: 1  # Number of pods to run<br>  selector:<br>    matchLabels:<br>      app: granite-api<br>  template:<br>    metadata:<br>      labels:<br>        app: granite-api<br>    spec:<br>      containers:<br>        - name: granite-container<br>          image: granite-api:latest # Change it to match the name of your image name<br>          imagePullPolicy: IfNotPresent<br>          args: <br>            - &quot;-m&quot;<br>            - &quot;/model.gguf&quot;<br>            - &quot;--host&quot;<br>            - &quot;0.0.0.0&quot;<br>            - &quot;--port&quot;<br>            - &quot;8080&quot;<br>            - &quot;--ctx-size&quot;<br>            - &quot;8192&quot;<br>          ports:<br>            - containerPort: 8080<br>              name: http          <br>          # Resources: Adjusted for the 350M model with 8192 context<br>          resources:<br>            requests:<br>              memory: &quot;4Gi&quot;<br>              cpu: &quot;2&quot;<br>            limits:<br>              memory: &quot;8Gi&quot;<br>              cpu: &quot;4&quot;<br><br>          # Health Checks<br>          readinessProbe:<br>            tcpSocket:<br>              port: 8080<br>            initialDelaySeconds: 5<br>            periodSeconds: 10<br>          livenessProbe:<br>            tcpSocket:<br>              port: 8080<br>            initialDelaySeconds: 15<br>            periodSeconds: 20</pre><pre>oc apply -f granite-deploy.yaml</pre><h3>Expose the Model Endpoint</h3><p>Once the deployment is running, the model API needs to be exposed so it can be accessed and tested. This is done in two layers: an internal <strong>Service</strong> and an external <strong>Route</strong>.</p><h4>B. The Service</h4><p>The Service provides a <strong>stable internal endpoint</strong> for the model server and abstracts away the dynamic nature of pod IP addresses.</p><ul><li><strong>Type:</strong> ClusterIP</li><li><strong>Function:</strong><br>Acts as a consistent internal access point for the deployment, allowing other pods or OpenShift components to communicate with the model API without needing to know individual pod IPs.</li></ul><h4>C. The Route</h4><p>The Route exposes the Service externally through the OpenShift router.</p><ul><li><strong>Function:</strong><br>Makes the model API accessible via a public URL.</li><li><strong>Security:</strong><br>Uses <strong>Edge TLS termination</strong>, ensuring traffic is encrypted between the client and the OpenShift router while keeping the backend service configuration simple.</li></ul><pre>apiVersion: v1<br>kind: Service<br>metadata:<br>  name: granite-service<br>spec:<br>  selector:<br>    app: granite-api<br>  ports:<br>    - name: http<br>      protocol: TCP<br>      port: 80         # The port other pods use to talk to this service<br>      targetPort: 8080 # The actual port on the container<br>  type: ClusterIP<br><br>---<br>apiVersion: route.openshift.io/v1<br>kind: Route<br>metadata:<br>  name: granite-route<br>spec:<br>  to:<br>    kind: Service<br>    name: granite-service<br>    weight: 100  <br>  port:<br>    targetPort: http  <br>  tls:<br>    termination: edge<br>    insecureEdgeTerminationPolicy: Redirect</pre><pre>oc apply -f granite-expose.yaml</pre><h3>Verification Test</h3><p>With the Service and Route in place, the final step is to verify that the model is reachable and responding correctly. The llama.cpp server exposes an <strong>OpenAI-compatible API</strong>, which allows us to test inference using a simple HTTP request.</p><p>Replace &lt;YOUR-ROUTE-URL&gt; with the hostname generated by the OpenShift Route, then run the following command:</p><pre>curl https://&lt;YOUR-ROUTE-URL&gt;/v1/chat/completions \<br> -H &quot;Content-Type: application/json&quot; \<br> -d &#39;{<br>   &quot;model&quot;: &quot;granite-350m&quot;,<br>   &quot;messages&quot;: [ { &quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;How many pyramids are in Egypt?&quot; } ] }&#39;</pre><p>If the deployment is successful, the API will return a JSON response containing the model’s generated answer. This confirms that the model has been loaded correctly, the inference server is running, and the endpoint is accessible through the OpenShift Route.</p><h3>Conclusion</h3><p>This article demonstrated a lightweight and practical approach to deploying a large language model on OpenShift without relying on specialized operators or GPU acceleration. By using a quantized GGUF model with llama.cpp, a multi-stage container build, and standard Kubernetes resources, it’s possible to run LLM inference efficiently even in constrained, CPU-only environments.</p><p>While this setup is well suited for development, experimentation, and operator-restricted clusters, it’s not a replacement for production-grade platforms like OpenShift AI and KServe, which provide advanced model management, scaling, and observability capabilities. Instead, it offers a flexible alternative when simplicity, portability, or minimal dependencies are the primary goals.</p><p>Ultimately, this approach shows that running LLMs on OpenShift doesn’t always require complex tooling — with the right model and runtime choices, core Kubernetes primitives can be enough to get started.</p><h3><strong>Interesting Links</strong></h3><p><a href="https://www.codecademy.com/article/llama-cpp">How to Use llama.cpp to Run LLaMA Models Locally | Codecademy</a><br><a href="https://github.com/ggml-org/llama.cpp">GitHub — ggml-org/llama.cpp: LLM inference in C/C++</a><br><a href="https://developers.redhat.com/articles/2025/09/30/vllm-or-llamacpp-choosing-right-llm-inference-engine-your-use-case#">vLLM or llama.cpp: Choosing the right LLM inference engine for your use case | Red Hat Developer</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=84ecb014f09a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Connect OpenShift to the External Network]]></title>
            <link>https://medium.com/@ahmeddraz/connect-openshift-to-the-external-network-70a0362d5d03?source=rss-55d320153012------2</link>
            <guid isPermaLink="false">https://medium.com/p/70a0362d5d03</guid>
            <category><![CDATA[virtualization]]></category>
            <category><![CDATA[network]]></category>
            <category><![CDATA[openshift]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Ahmed Draz]]></dc:creator>
            <pubDate>Tue, 24 Jun 2025 13:38:24 GMT</pubDate>
            <atom:updated>2025-06-24T13:38:24.137Z</atom:updated>
            <content:encoded><![CDATA[<p>Connecting your OpenShift cluster to external networks is a critical capability for any organization looking to run virtualized workloads that need to communicate beyond the Kubernetes overlay. Whether you’re hosting traditional web services, integrating with on-premises data centers, or simply need virtual machines with direct access to your company LAN, exposing OpenShift Virtualization workloads to an outside network bridges the gap between container-native environments and the rest of your infrastructure. In this article, we’ll explore how to prepare your cluster nodes, create and configure Linux bridges, and leverage Multus NetworkAttachmentDefinitions to attach external interfaces to your VirtualMachineInstances. By the end, you’ll have a repeatable, declarative process for giving your VMs the same network footprint as any physical or virtual server in your data center — without sacrificing the benefits of OpenShift’s platform services.</p><p>Below is a simplified view of how an OpenShift worker node can be wired to both the cluster overlay and an external VLAN. On the right, you see a CoreOS host with two NICs:</p><ul><li><strong>eth0</strong> plugged into the OpenShift/VXLAN overlay via OVS’s external &amp; integration bridge (br-ex &amp; br-int)</li><li><strong>eth1</strong> trunked back to your physical switch carrying VLAN 10 (or any VLAN of your choosing), then enslaved under a Linux bridge (br0) and exposed to pods or KubeVirt VMs via Multus.</li></ul><p>On the left, three VMs on VLAN 10 (VM1–3) connect through that same bridge, demonstrating “outside” traffic flowing in and out of your cluster as if it were a native part of your datacenter network.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-A5i1GaKDwHjfMV9PRyVzA.png" /><figcaption>Network Diagram</figcaption></figure><p>In the next sections, we’ll show how NMState’s NodeNetworkConfigurationPolicy creates and manages that br0 Linux bridge (and enslaves eth1), and how a Multus NetworkAttachmentDefinition lets your pods and VMs attach directly to it—no manual SSH fiddling required.</p><h3>Overview of NMState</h3><p>NMState is the Kubernetes-native engine that declaratively manages host‐level network configurations on your OpenShift nodes, turning traditionally manual, error-prone steps into repeatable CRDs and controllers. At the heart of NMState are two primary components:</p><ul><li><strong>NodeNetworkConfigurationPolicy (NNCP)</strong><br> Defines your desired network state — whether that’s bringing up physical interfaces, creating Linux bridges, VLAN tagging, or bonding. You author an NNCP to specify exactly how each node’s interfaces should look, and the NMState controller continuously converges them to that defined state.</li><li><strong>NodeNetworkState (NNS)</strong><br> Represents the actual, observed network configuration on each node. After your NNCP is applied, NMState writes the resulting configuration into an NNS object, giving you transparent feedback and auditability (“What does the bridge look like right now?”).</li></ul><p>Behind the scenes, the <strong>nmstate-handler</strong> DaemonSet runs on every worker node, listening for changes to NNCPs and translating your high-level YAML into low-level ip, bridge, and ethtool commands. This removes the need to SSH into nodes for manual setup and ensures that reboots or node replacements automatically reapply your network settings.</p><p>With those definitions out of the way, you can move on to configuring network interfaces on your OpenShift nodes.</p><h4>Linux bridge interface node network configuration policy</h4><p>Create a Linux bridge interface on nodes in the cluster by applying a NodeNetworkConfigurationPolicy manifest to the cluster.</p><pre>apiVersion: nmstate.io/v1<br>kind: NodeNetworkConfigurationPolicy<br>metadata:<br>  name: br0-eth1-policy <br>spec:<br>  nodeSelector: <br>    kubernetes.io/hostname: &lt;node01&gt; ## Change the key and value for more selection criteria.<br>  desiredState:<br>    interfaces:<br>      - name: br0 <br>        description: Linux bridge with eth1 as a port <br>        type: linux-bridge <br>        state: up <br>        ipv4:<br>          dhcp:  true<br>          enabled: true <br>        bridge:<br>          options:<br>            stp:<br>              enabled: false <br>          port:<br>            - name: eth1 </pre><pre>$ oc apply -f br0.yaml</pre><p>This will, on each selected node:</p><ul><li>Bring up eth1</li><li>Create a Linux bridge device called br0 with eth1 enslaved.</li></ul><h3>NetworkAttachmentDefinition Configuration</h3><p>To bridge your VMs or pods onto the <strong><em>br0 </em></strong>Linux bridge, you simply declare a Multus NetworkAttachmentDefinition (NAD). This CRD tells Multus to attach a “bridge”-type interface to br0 and use DHCP (or static IPAM) on your external VLAN. Applying this NAD makes it trivial to add an external network interface to any workload — no imperative CNI tweaks needed.</p><pre>apiVersion: k8s.cni.cncf.io/v1<br>kind: NetworkAttachmentDefinition<br>metadata:<br>  name: vlan-10<br>  namespace: vms-dev  # or your VM’s namespace<br>spec:<br>  config: &#39;{<br>    &quot;name&quot;: &quot;vlan-10&quot;<br>    &quot;cniVersion&quot;: &quot;0.3.1&quot;,<br>    &quot;type&quot;: &quot;cnv-bridge&quot;,<br>    &quot;bridge&quot;: &quot;br0&quot;,<br>    &quot;ipam&quot;: {<br>      &quot;type&quot;: &quot;dhcp&quot;<br>    }<br>  }&#39;oc apply -f br0.yaml</pre><pre>$ oc apply -f vlan10.yaml</pre><p>With the NAD in place, you can simply reference it in your VirtualMachine spec, adding an bridge interface and a matching multus network entry. Once applied, your VM will get a secondary NIC on br0, pick up an IP via DHCP (or your chosen IPAM), and be routable on your external VLAN as if it were a physical host.</p><h3>Attach the external network to your VM</h3><p>With your NetworkAttachmentDefinition in place, wiring a VM to the external bridge is as easy as adding two lines to its manifest. Under devices.interfaces, you declare a new interface of type bridge, for example:</p><pre>apiVersion: kubevirt.io/v1<br>kind: VirtualMachine<br>metadata:<br>  name: vm-demo<br>spec:<br>  running: true<br>  template:<br>    spec:<br>      domain:<br>        devices:<br>          interfaces:<br>            # default pod-only interface<br>            - name: default<br>              masquerade: {}<br>            # external interface<br>            - name: nic0<br>              bridge: {}     # use Multus “bridge” type<br>      networks:<br>        - name: default<br>          pod: {}<br>        - name: vlan-10<br>          multus:<br>            networkName: vlan-10</pre><p>When you apply this updated VirtualMachine (or VirtualMachineInstance) YAML, KubeVirt will hot-plug a secondary vNIC on br0. Inside the VM you’ll see something like eth1 come up automatically, request an address via DHCP (or use your chosen static IPAM), and suddenly your VM is peer-to-peer reachable alongside any other host on VLAN 10.</p><p>Exposing your OpenShift Virtualization workloads to an external VLAN is both powerful and surprisingly straightforward once you embrace the declarative CNI model. By leveraging NMState’s <strong>NodeNetworkConfigurationPolicy</strong>, you automate the creation of host-level bridges (like br0) and enslave physical NICs without ever logging into individual nodes. Then, with a simple <strong>Multus NetworkAttachmentDefinition</strong>, you package up that bridge into a reusable network CRD. Finally, a two-line tweak in your VM spec wires your instance directly onto your datacenter LAN—no more ad-hoc scripts or manual ip link gymnastics.</p><h3>References</h3><p><a href="https://docs.redhat.com/en/documentation/openshift_container_platform/4.18/html/networking/kubernetes-nmstate#virt-example-bridge-nncp_k8s-nmstate-updating-node-network-config">Chapter 21. Kubernetes NMState</a></p><p><a href="https://www.redhat.com/en/blog/access-external-networks-with-openshift-virtualization">Red Hat OpenShift Virtualization: Configuring virtual machines to use external networks</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=70a0362d5d03" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Simplifying Multi-Network Connectivity in OpenShift with Multus CNI]]></title>
            <link>https://blog.devops.dev/simplifying-multi-network-connectivity-in-openshift-with-multus-cni-dabe4da209f8?source=rss-55d320153012------2</link>
            <guid isPermaLink="false">https://medium.com/p/dabe4da209f8</guid>
            <category><![CDATA[network]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[openshift]]></category>
            <category><![CDATA[containers]]></category>
            <dc:creator><![CDATA[Ahmed Draz]]></dc:creator>
            <pubDate>Tue, 16 Jul 2024 07:54:00 GMT</pubDate>
            <atom:updated>2024-07-18T17:15:03.355Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Multus </strong>CNI is such a plug-in, referred to as a <strong><em>meta-plug-in</em></strong>: a CNI plug-in that can run other CNI plug-ins. It works like a wrapper that calls other CNI plug-ins for attaching multiple network interfaces to pods in OpenShift (Kubernetes). First, let’s take a quick look at the CNI project.</p><h3><strong>CNI</strong></h3><p>CNI is the container network interface that provides a pluggable application programming interface to configure network interfaces in Linux containers.</p><p>Initially, containers (pods) don’t have a network interface. To create a network interface for a container, the container runtime (<strong><em>CRI) </em></strong>sends an ADD command to the CNI plugin (the runtime can call the plugin with commands like ADD, DEL, and CHECK). Once the new network interface is created, a JSON payload passes the details of what needs to be added to the CNI.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*2OqZ1j9HxakKafAE.png" /><figcaption>how CRI talks to CNI plugin to attach a network interface to the newly created pod.</figcaption></figure><p>After the CNI creates the network interface for a pod, the pod becomes fully operational from a networking perspective, able to communicate with other pods, services, and external endpoints as defined by its configuration and network policies.</p><h3>Multi-Network Connectivity using Multus</h3><p>Red Hat OpenShift Container Platform uses the Multus CNI plug-in to allow chaining of CNI plug-ins — <em>CNI Chaining enables the combination of two CNI plugins so one can provide the base network connectivity and IP address management and the other can add more advanced capabilities </em>— . During cluster installation, you can configure your default pod network. The default network handles all ordinary network traffic for the cluster.</p><p>You can use Multus to define an extra network based on the available CNI plug-ins and attach one or more of these networks to your pods. To do that and attach more network interfaces to any pod, you must create <strong><em>NetworkAttachmentDefinition (CR) </em></strong>that defines how the interfaces are attached to a pod.</p><p><em>NetworkAttachmentDefinition </em>is additional network configurations based on Multus are managed by the <strong><em>Cluster Network Operator (CNO)</em></strong> in OpenShift, it allows attaching additional interfaces to a pod. Reconciliation is facilitated according to <strong><em>.spec.additionalNetworks</em></strong> in the <strong><em>networks.operator.openshift.io</em></strong> CR. The following diagram describes this process in detail.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1007/1*7sFpOUj4GtXbJ1S8pgCe0w.png" /></figure><p>Even though you can also directly create <strong><em>NetworkAttachmentDefinition</em></strong> CR without CNO, it is recommended that you use the CNO for the centralized management of your additional networks.</p><h3>Multus CNI Plug-ins</h3><p>Plugins are modular, interchangeable pieces of software that implement specific networking functions for containers. Multus has three types of plug-ins:</p><h4>Main plug-in:</h4><p>Responsible for inserting a network interface into the container network namespace. Examples of these plug-ins are <strong><em>bridge, host-device, ipvlan, macvlan</em></strong>, and so on.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/592/1*Sa0MeK9kORLs3dxWeYlYbg.png" /></figure><p>Well, we discuss each one separately in the following sections.</p><h4>IPAM plug-in:</h4><p>Responsible for managing IP allocation within the Main plug-in. Dhcp, host-local, static, whereabouts are examples of IPAM plug-ins.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/590/1*F64ORAo_XvEkF2UdXSkmYg.png" /></figure><p><strong>host-local: </strong>IP allocation management within specified ranges on a single node host only. Storage of IP address allocation is specified in flat files local to each host, and hence, host-local.</p><p><strong>DHCP: </strong>IP allocated by an external DHCP server.</p><p><strong>whereabouts: </strong>IP allocation management within specified ranges across an OpenShift cluster. Storage of IP address allocation is specified in Kubernetes Custom Resources to enable the allocation of an IP across any host in a cluster.</p><p><strong>static: </strong>IP allocated statically.</p><p>Basically, IPAM CNI plug-in can be combined with any main plug-in, but that does not mean it works well in any environment. First, you should consider the plug-ins that are appropriate for your network environments before implementing any of them.</p><h4>Other plug-in:</h4><p>Typically does not work on the primary interface directly. The attributes are set on those interfaces. Tuning, route-override are examples.</p><h4><strong>Note:</strong></h4><p>Multiple plug-ins can be configured for a single container using <strong><em>Network Configuration Lists</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/951/1*O1jzqQHrHerYpGioC89dKw.png" /></figure><p>In this article, we will just focus on the Main and IPAM plug-ins.</p><h3>Main Plug-ins &amp; IPAM examples</h3><p>As we have seen that plugins are modular and each plugin implement specific networking functions for containers, therefore, newtork attachment definitions combine those plugins to create functional interface, In the following examples we will discuss the combination of Main and IPAM plugins.</p><h4><strong>Bridge</strong></h4><p>This acts as a network switch between multiple pods on the same node host. The containers receive one end of the <strong><em>veth </em></strong>pair with the other end connected to the bridge. An IP address is only assigned to one end of the <strong><em>veth</em></strong><em> </em>pair — one residing in the container. In its current form, a bridge interface is created that does not link any physical host interface. As a result, connections are not made to any external networks including other pods on the other host nodes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/935/1*PKz4ev-0pXaA0nihV78mPQ.png" /><figcaption>Bridge Diagram</figcaption></figure><p>Bridge’s <em>NetworkAttachmentDefinition </em>Configuration, In the following configuration we see that the main plugin is bridge and it uses host-local as ipam plugin to allow the bridge to distribute IPs from certain range of a single node network level IPs.</p><pre>$ oc edit networks.operator.openshift.io cluster<br>apiVersion: operator.openshift.io/v1<br>kind: Network<br>metadata:<br>  name: cluster<br>spec:<br>  additionalNetworks: <br>  - name: bridge-main<br>    namespace: multus-test<br>    type: Raw<br>    rawCNIConfig: &#39;{<br>      &quot;cniVersion&quot;: &quot;0.3.1&quot;,<br>      &quot;type&quot;: &quot;bridge&quot;,<br>      &quot;bridge&quot;: &quot;mybr0&quot;,<br>      &quot;ipam&quot;: {<br>          &quot;type&quot;: &quot;host-local&quot;,<br>          &quot;subnet&quot;: &quot;192.168.12.0/24&quot;,<br>          &quot;rangeStart&quot;: &quot;192.168.12.10&quot;,<br>          &quot;rangeEnd&quot;: &quot;192.168.12.200&quot;<br>      }<br>    }&#39;</pre><p>To allow a pod to use this network configuration we have to annotate the pod with “<strong><em>k8s.v1.cni.cncf.io/networks”: &lt;additional-network-name &gt;</em></strong></p><pre>$ oc patch deploy/pod-a -n multus-test --patch &#39;{<br>  &quot;spec&quot;: {<br>    &quot;template&quot;: {<br>      &quot;metadata&quot;: {<br>        &quot;annotations&quot;: {<br>           &quot;k8s.v1.cni.cncf.io/networks&quot;: &quot;bridge-main&quot;<br>       }<br>      }<br>    }<br>  }<br>}&#39;</pre><h4><strong>Host-device</strong></h4><p>This plug-in makes a physical host interface move to a pod network namespace. When enabled, the specified host interface disappears in the root network namespace (default host network namespace). This behavior might affect re-creating the pod in place on the same host as the host interface may not be found as it is specified by host-device plug-in configuration.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/935/1*NY5TzghMZ9cHGDS30Q0Jwg.png" /><figcaption>Host-device Diagram</figcaption></figure><p>This time, dhcp IPAM is configured, and it would trigger the creation of the dhcp-daemon daemonset pods in openshift-multus namespace. The pod in daemon mode listens for an address from a DHCP server on OpenShift, whereas the DHCP server itself is not provided. In other words, it requires an existing DHCP server in the same network. This demonstration shows you the MAC address of the parent is kept in the pod network namespace. Additionally, the source IP and MAC address can be identified by using an external web server access test.</p><p><em>NetworkAttachmentDefinition </em>Configurations:</p><pre>$ oc edit networks.operator.openshift.io cluster<br>apiVersion: operator.openshift.io/v1<br>kind: Network<br>metadata:<br>  name: cluster<br>spec:<br>  additionalNetworks: <br>  - name: host-device-main<br>    namespace: multus-test<br>    type: Raw<br>    rawCNIConfig: &#39;{<br>      &quot;cniVersion&quot;: &quot;0.3.1&quot;,<br>      &quot;name&quot;: &quot;host-device-main&quot;,<br>      &quot;type&quot;: &quot;host-device&quot;,<br>      &quot;device&quot;: &quot;ens4&quot;,<br>      &quot;ipam&quot;: {<br>          &quot;type&quot;: &quot;dhcp&quot;<br>      }<br>    }&#39;</pre><h4><strong>Ipvlan</strong></h4><p>The ipvlan plug-in may be used in the event that there are limited MAC addresses that can be used. This issue is common on some switch devices that restrict the maximum number of MAC addresses per physical port due to port security configurations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/810/1*HB4dNIcl06yDQ7mwRUjxtA.png" /><figcaption>Ipvlan Diagram</figcaption></figure><p>The sub-interface of the ipvlan can use distinct IP addresses with the same MAC address of the parent host interface. So, it would not work well with a DHCP server which depends on the MAC addresses. Parent host interface acts as a bridge (switch) with L2 mode, and it acts as a router with L3 mode of the ipvlan plug-in.</p><p><em>NetworkAttachmentDefinition </em>Configurations:</p><pre>$ oc edit networks.operator.openshift.io cluster<br>apiVersion: operator.openshift.io/v1<br>kind: Network<br>metadata:<br>  name: cluster<br>spec:<br>  additionalNetworks:<br>  - name: ipvlan-main<br>namespace: multus-test<br>type: Raw<br>rawCNIConfig: &#39;{<br>   &quot;cniVersion&quot;: &quot;0.3.1&quot;,<br>   &quot;name&quot;: &quot;ipvlan-main&quot;,<br>   &quot;type&quot;: &quot;ipvlan&quot;,<br>   &quot;mode&quot;: &quot;l2&quot;,<br>   &quot;master&quot;: &quot;ens4&quot;,<br>     &quot;ipam&quot;: {<br>           &quot;type&quot;: &quot;whereabouts&quot;,<br>           &quot;range&quot;: &quot;192.168.12.0/24&quot;,<br>           &quot;range_start&quot;: &quot;192.168.12.50&quot;,<br>           &quot;range_end&quot;: &quot;192.168.12.200&quot;<br>     }<br>}&#39;</pre><p>With ipvlan and whereabouts IPAM, it’s easy to test connectivity among pods and an external web server without external system dependencies. The ipvlan sub-interfaces work like the Linux IP alias.</p><h4><strong>Macvlan</strong></h4><p>With macvlan, it’s simple to use as it aligns to traditional network connectivity. Since the connectivity is directly bound with the underlying network using sub-interfaces each having MAC address.</p><p>macvlan functions like a switch that is already connected to the host interface. A host interface gets “enslaved” with the virtual interfaces sharing the physical device but having distinct MAC addresses. Since each macvlan interface has its own MAC address, it makes it easy to use with existing DHCP servers already present on the network.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/823/1*tF5iSylNUKGPEV3Nlfum-w.png" /><figcaption>Macvlan Diagram</figcaption></figure><p><em>NetworkAttachmentDefinition </em>Configurations:</p><pre>$ oc edit networks.operator.openshift.io cluster<br>apiVersion: operator.openshift.io/v1<br>kind: Network<br>metadata:<br>  name: cluster<br>spec:<br>  additionalNetworks:<br>  - name: macvlan-main<br>    namespace: multus-test<br>    type: Raw<br>    rawCNIConfig: &#39;{<br>       &quot;cniVersion&quot;: &quot;0.3.1&quot;,<br>   &quot;name&quot;: &quot;macvlan-main&quot;,<br>   &quot;type&quot;: &quot;macvlan&quot;,<br>   &quot;mode&quot;: &quot;bridge&quot;,<br>   &quot;master&quot;: &quot;ens4&quot;,<br>   &quot;ipam&quot;: {<br>         &quot;type&quot;: &quot;static&quot;<br>   }<br>    }&#39;</pre><p>Up to this point, we demonstrated the main and IPAM plug-ins. In most cases, you may configure additional networks using one of the CNI plug-in combinations that has been previously covered or reviewed. However, you might need to customize the base network configuration depending on your specific use case or environment dependencies. If this is indeed the case, Other plug-in features may be helpful. Other plug-in explained in refrences below.</p><h3>Refrences</h3><p><a href="https://www.cni.dev/">CNI</a></p><p><a href="https://www.redhat.com/en/blog/using-the-multus-cni-in-openshift">Using the Multus CNI in OpenShift (redhat.com)</a></p><p><a href="https://www.redhat.com/en/blog/demystifying-multus">Demystifying Multus (redhat.com)</a></p><p><a href="https://two-oes.medium.com/openshift-4-with-macvlan-and-whereabouts-f6108addb574">OpenShift 4 with MacVLAN and whereabouts | by Oren Oichman | Medium</a></p><p><a href="https://github.com/k8snetworkplumbingwg/multus-cni">k8snetworkplumbingwg/multus-cni: A CNI meta-plugin for multi-homed pods in Kubernetes (github.com</a>)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dabe4da209f8" width="1" height="1" alt=""><hr><p><a href="https://blog.devops.dev/simplifying-multi-network-connectivity-in-openshift-with-multus-cni-dabe4da209f8">Simplifying Multi-Network Connectivity in OpenShift with Multus CNI</a> was originally published in <a href="https://blog.devops.dev/">DevOps.dev</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[OpenShift Sharded Routers]]></title>
            <link>https://medium.com/@ahmeddraz/openshift-sharded-routers-7f223a5b2624?source=rss-55d320153012------2</link>
            <guid isPermaLink="false">https://medium.com/p/7f223a5b2624</guid>
            <category><![CDATA[openshift]]></category>
            <category><![CDATA[sharding]]></category>
            <category><![CDATA[ingresscontroller]]></category>
            <category><![CDATA[networking]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Ahmed Draz]]></dc:creator>
            <pubDate>Tue, 26 Dec 2023 12:20:04 GMT</pubDate>
            <atom:updated>2023-12-26T12:20:04.766Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1001/1*b5j5k_89cfPtxa0bRHtH7g.png" /></figure><p>In OpenShift deploying applications and exposing it to the public are managed by Ingress Operator which is an OpenShift operator that enables<br>external access to applications within the cluster but <strong><em>how does that happen?</em></strong> OpenShift ingress operator configures ingress controller instance which route traffic as specified by route/ingress resources to the named destinations and in our case, they are (pods). In more simple way the ingress controller can be considered as reverse proxy and each time we create route/ingress the URL configured in the route resource is going to be added to the configuration of the ingress controller to redirect the traffic requests related to this URL to its configured destination.</p><p>In OpenShift 4 Cluster there is by default an ingress controller that comes with two replicas of router pods each one placed on different node (anti-affinity) handling the incoming traffic, however, we can increase number of routers for more distribution for the traffic through more node to prevent any bottleneck could happen.</p><ul><li>To see the default OpenShift ingresscontroller.</li></ul><pre>$ oc get ingresscontroller -n openshift-ingress-operator</pre><ul><li>To get number of routers in the default ingresscontroller.</li></ul><pre>$ oc get ingresscontroller default -n openshift-ingress-operator -o jsonpath=&#39;{.spec.replicas}{&quot;\n&quot;}&#39;<br>$ oc get pod -n openshift-ingress</pre><h3>Ingress Controller sharding</h3><p>Also known as router sharding, routers sharding is way to distribute a set of routes across multiple routers by adding labels to routes, namespaces, or both. The Ingress Controller uses a corresponding set of selectors to admit only the routes that have a specified label. Each Ingress shard comprises the routes that are filtered using a given selection expression.</p><p>As the primary mechanism for traffic to enter the cluster, the demands on the Ingress Controller can be significant. As a cluster administrator, you can shard the routes to:</p><ul><li>Balance Ingress Controllers, or routers, with several routes to speed up responses to changes.</li><li>Allocate certain routes to have different reliability guarantees than other routes.</li><li>Allow certain Ingress Controllers to have different policies defined.</li><li>Allow only specific routes to use additional features.</li><li>Expose different routes on different addresses so that internal and external users can see different routes, for example.</li><li>Transfer traffic from one version of an application to another during a blue green deployment.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/618/1*-NtGOgNdZ4b-NLxYXGfipQ.png" /></figure><h3>Sharding Types</h3><p>Sharding can happen according to one of two options or both:<br>1. Namespace Label<br>2. Route Label</p><p>Namespace label, with this option each route is created within any namespace has the label that is spcified in the sharded ingresscontroller config will automattically managed by this ingresscontroller.</p><p>On the other hand, router label means that any route has the label which specified in the ingresscontroller config will be automatically managed by this ingresscontroller.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/887/1*S6otl8rsBhwgalVxyQDj0g.png" /></figure><ul><li><strong>domain</strong>: a wildcard A record *.prod references a load balancer VIP.</li><li><strong>nodePlacement</strong>: NodePlacement describes node scheduling configuration for an ingress controller. In our case have a matchLabel to deploy the ingress controller only into Workers.</li><li><strong>routeSelector</strong>: routeSelector is used to filter the set of Routes serviced by the ingress controller.</li><li><strong>namespaceSelector: </strong>namespaceSelector is used to filter the set of Routes serviced by the ingress controller based on its namespace label.</li></ul><h3>Steps</h3><ol><li>Create the external ingresscontroller:</li></ol><pre>$ cat &gt; ingresscontroller_external.yaml &lt;&lt; EOF<br>  apiVersion: operator.openshift.io/v1<br>  kind: IngressController<br>  metadata:<br>     name: external<br>     namespace: openshift-ingress-operator<br>  spec:<br>    domain: prod.ocp4.example.com<br>    nodePlacement:<br>      nodeSelector:<br>        matchLabels:<br>           node-role.kubernetes.io/infra: &quot;&quot;<br>    routeSelector:<br>      matchLabels:<br>        type: external<br>  EOF<br>$ oc apply -f ingresscontroller_external.yaml<br>$ oc get ingresscontroller -n openshift-ingress-operator s## check if it is created in cluster<br>$ oc get pod -n openshift-ingress   ## list the new routers.</pre><h4>2. Testing the Route Sharding</h4><p>2.1 Create a new project:</p><pre>$ oc new-project production-app</pre><p>2.2 Create an application for test:</p><pre>$ oc new-app django-psql-example<br>$ watch oc get pods ## Watch untill all pods become running<br>$ oc get route ## The default created route</pre><p>2.3 Reconfigure the route to make it managed by the external ingresscontroller. Add the label that matches to the routeSelector defined into our internal ingresscontroller:</p><pre>$ oc edit route django-psql-example</pre><pre>metadata:<br>  labels:<br>    app: django-psql-example<br>    template: django-psql-example<br>    type: external ## the label configured in ingresscontroller config.<br>...<br>...<br>spec:<br>  host: django-psql-example-production-app.prod.ocp4.example.com ## new host referencing the ingresscontroller wildcard.</pre><p>2.4 Try to connect using the new URL:</p><pre>$ curl -I http://django-psql-example-production-app.prod.ocp4.example.com<br>HTTP/1.1 200 OK<br>Server: gunicorn/19.5.0<br>Date: Tue, 26 Dec 2023 11:48:16 GMT<br>Content-Type: text/html; charset=utf-8<br>X-Frame-Options: SAMEORIGIN<br>Content-Length: 18501<br>Cache-control: private</pre><p>Vola! It works, we can now access the application through a sharded routers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*prGkKTQDiIPV9xGUwEkaEg.gif" /></figure><p>In conclusion, Sharding routing could be used for numerous use cases and knowing how to configure it is important for OpenShift administrator. I tried my best to explain how it works and I wish you grasp the knowledge behind sharding.</p><h3>References:</h3><p><a href="https://docs.openshift.com/container-platform/4.12/networking/ingress-sharding.html#nw-ingress-sharding_ingress-sharding">Ingress sharding | Networking | OpenShift Container Platform 4.12</a></p><p><a href="https://rcarrata.com/openshift/ocp4_route_sharding/">Deep dive of Route Sharding in OpenShift 4 — Rcarrata’s Blog</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7f223a5b2624" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Openshift Egress Traffic]]></title>
            <link>https://medium.com/@ahmeddraz/openshift-egress-traffic-e1d97da4b6d1?source=rss-55d320153012------2</link>
            <guid isPermaLink="false">https://medium.com/p/e1d97da4b6d1</guid>
            <category><![CDATA[openshift]]></category>
            <category><![CDATA[administration]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[network]]></category>
            <dc:creator><![CDATA[Ahmed Draz]]></dc:creator>
            <pubDate>Fri, 05 May 2023 09:41:45 GMT</pubDate>
            <atom:updated>2023-05-05T09:41:45.811Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*X6lbDQ2W-cSFf6I_87XZhw.jpeg" /></figure><p>I wrote this brief article to explain how actually the traffic leaves an openshift cluster to reach external endpoints because I have read many articles explaining the ingress traffic within openshift clusters but very few about the opposite case “How the internal openshift components reach the outside services?”. In the following paragraphs, I will try to explain how egress traffic leaves openshift in a simple way as far as I can.</p><h3><strong>Egress Traffic within Openshift</strong></h3><p>So what are the sources of egress traffic in the openshift cluster? Or in other words, what are the openshift components that try to communicate with services outside the cluster network?</p><p>The openshift cluster originates two types of egress traffic:<br><strong>1. <em>Platform Traffic.</em><br>2. <em>Applications Traffic.</em></strong></p><p><strong>Platform traffic</strong> originates from openshift resources and it can be something like DNS resolution, connection to a proxy server, NTP server, external monitoring system, or integration with an active directory. This type of traffic only leaves using the <strong>node’s IP</strong> “the IP of VM itself” on which the resources run on.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/723/1*9aECBn-QN4iVs_pldMeI1w.png" /></figure><p><strong>Application traffic</strong> originates when application pods in openshift want to access external services like an external database or any endpoint over the internet. From now on we will discuss the possible approaches of how can applications within openshift access external services.</p><h3><strong>How The Traffic leaves the cluster?</strong></h3><h3><strong>Node IP</strong></h3><p>The default behavior in openshift is that all traffic either from the platform or from the applications is going to originate from the node’s IP.</p><figure><img alt="asd" src="https://cdn-images-1.medium.com/max/632/1*Y_yAECTgSzIwEAXRT4GtPA.png" /></figure><p>Here we have two projects; project A and Project B. Project B has only one pod running in node 1, and Project A has two pods running in both node 1 and node 2. Also, we have an external service which could be a database service in a different zone or any kind of application outside the openshift cluster network. Besides, there is a firewall between the cluster network and the external service network. In this approach to allow the application in project A to communicate with external service we have to allow traffic comes from the node’s IP on which the application run; in this case node 1 and node 2 because project A has two replicas in both nodes but here firewall doesn&#39;t allow the traffic from node 2, therefore, only the pod in node 1 can reach the external service. Because the firewall allows all traffic that comes from node 1 the application in <strong><em>project B</em></strong> exposes the external service, and we don’t that to happen from a security perspective.</p><p>This approach has many cons compared to pos; the single pos I can mention is “it’s simple for a lab environment with nearly no restrictions to care about”. To overcome the node IP model cons there are two other approaches are used in production environments and we will discuss them in the following sections.</p><h3><strong>Egress Router</strong></h3><p>In the previous approach, we see how it allows all the applications to expose the external service whether it should or shouldn&#39;t connect to the external service; to overcome this issue other egress models are introduced: Egress Router and Egress IP and I am going to start with Egress Router.</p><p>Egress Router is another application deployed in the project that we want its applications to connect to the external service.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F1LxrbwgxAMoa2T5AUJheQ.png" /></figure><p>In the above diagram like the previous scenario, we have a firewall between the cluster network and the external service and we have two projects A and B. The new in this approach is that in project A in which we want the apps the expose the external service we have an egress router that will take the egress traffic from the application and forward it to the external service. But before the pod traffic is sent to the external service through the egress router a must egress service is located between the egress router and the application pod to forward the traffic from the pods to the egress router.</p><p>Firstly, In Project A the replica which tries to connect directly to the external service fails because it’s not allowed through the firewall. The other replica communicates with the egress service that later on forward the traffic to the egress router which has an IP in the firewall whitelist of allowed IPs.</p><p>Secondly, In project B the application tries to reach the egress service but because of network policies, it’s not allowed to communicate with any other resources outside its namespace and of course, if it tries to connect directly to the external service it will fail because of the firewall.</p><p><strong>Note<em>:<br></em></strong><em>It is noteworthy to mention that the default network policy in these examples is denay all the traffic which means that pods can communicate with resources within their namespace but not resources in other namespaces.</em></p><h3>Egress IP</h3><p>The last approach of the three and simpler than the egress router. Egress IP is configured at the namespace level so we can originate the egress traffic of applications from them. Each node on which the application [Pods] runs can be assigned one or more egress IPs which are allocated to the application namespace within the cluster’s node network free IPs, which means the nodes may have multiple IPs; the node IP itself and egress IPs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/862/1*T-fo1FYCjX6nhJV5NcbLbA.png" /></figure><p>In the above scenario application in Project B tries to connect to the external service but as we have mentioned before, the firewall will block the traffic coming from the IPs not in its whitelist. On the contrary, the application of Project A can reach the external service because pods in Project A forward their traffic through egress IPs that are assigned to their namespace, and these IPs are allowed through the firewall.</p><h3><strong>Conclusion</strong></h3><p>In conclusion, after we have discussed the egress approaches in the openshift cluster; we can tell that there is no best approach each one of them has its own suitable environment. For example, node IP is suitable for lab env for its simplicity, and egress IPs for their high availability.</p><p>I hope this article gives you a teeny tiny glimpse of how egress in openshift works.</p><p>Here are worth mentioning references:</p><p><a href="https://docs.openshift.com/container-platform/4.10/networking/openshift_sdn/assigning-egress-ips.html">https://docs.openshift.com/container-platform/4.10/networking/openshift_sdn/assigning-egress-ips.html</a></p><p><a href="https://docs.openshift.com/container-platform/4.10/networking/openshift_sdn/deploying-egress-router-layer3-redirection.html">https://docs.openshift.com/container-platform/4.10/networking/openshift_sdn/deploying-egress-router-layer3-redirection.html</a></p><p><a href="https://docs.openshift.com/container-platform/4.10/networking/openshift_sdn/deploying-egress-router-layer3-redirection.html">https://docs.openshift.com/container-platform/4.10/networking/openshift_sdn/deploying-egress-router-layer3-redirection.html</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e1d97da4b6d1" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>