<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Arnob on Medium]]></title>
        <description><![CDATA[Stories by Arnob on Medium]]></description>
        <link>https://medium.com/@arn-ob?source=rss-c7040ca07397------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*ENkPy4yN-cpKhf5Ne68V6g.jpeg</url>
            <title>Stories by Arnob on Medium</title>
            <link>https://medium.com/@arn-ob?source=rss-c7040ca07397------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 20:58:51 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@arn-ob/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Deploy OpenStack on Ubuntu 24.04 Using Kolla-Ansible]]></title>
            <link>https://arn-ob.medium.com/deploy-openstack-on-ubuntu-24-04-using-kolla-ansible-8add3132e716?source=rss-c7040ca07397------2</link>
            <guid isPermaLink="false">https://medium.com/p/8add3132e716</guid>
            <category><![CDATA[openstack]]></category>
            <category><![CDATA[ubuntu-server]]></category>
            <category><![CDATA[ansible]]></category>
            <category><![CDATA[ubuntu]]></category>
            <category><![CDATA[containers]]></category>
            <dc:creator><![CDATA[Arnob]]></dc:creator>
            <pubDate>Wed, 23 Sep 2026 07:54:23 GMT</pubDate>
            <atom:updated>2026-09-23T07:54:23.627Z</atom:updated>
            <content:encoded><![CDATA[<p>Kolla-Ansible simplifies OpenStack deployment by running services in containers and using Ansible to automate configuration and orchestration</p><h3>Introduction</h3><p>Building a private cloud infrastructure is a valuable way to understand how modern cloud platforms work under the hood. While public cloud providers such as AWS, Azure, and GCP offer managed infrastructure, OpenStack gives you the opportunity to build and operate your own cloud platform.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5rBBTBNdQCTrFUU-7LG00w.png" /></figure><p>However, deploying OpenStack manually can be challenging. It involves multiple services, networking components, databases, message queues, and configuration dependencies.</p><p>This is where <strong>Kolla-Ansible</strong> comes in.</p><p>Kolla-Ansible simplifies OpenStack deployment by using Ansible to automate configuration and run OpenStack services inside containers. This makes it a practical choice for engineers who want to learn OpenStack without manually configuring every service from scratch.</p><p>In this tutorial, we will deploy OpenStack on <strong>Ubuntu 24.04 using Kolla-Ansible</strong>, starting with an all-in-one lab environment.</p><p>By the end of this guide, you will be able to:</p><ul><li>Prepare Ubuntu 24.04 for OpenStack deployment.</li><li>Configure management and external network interfaces.</li><li>Install Kolla-Ansible in a Python virtual environment.</li><li>Configure the OpenStack deployment.</li><li>Run bootstrap, prechecks, and deployment playbooks.</li><li>Generate admin credentials.</li><li>Verify OpenStack using the CLI.</li><li>Access the Horizon web dashboard.</li></ul><p>Let’s get started.</p><h3><strong>Prepare Ubuntu</strong></h3><p>Use a fresh Ubuntu 24.04 VM with at least 2 network interfaces recommended by the deployment documentation. The exact resource requirements depend on the services you enable and your deployment topology.</p><pre><br>sudo apt update<br>sudo apt install -y git python3-dev python3-venv \<br> libffi-dev gcc libssl-dev libdbus-glib-1-dev<br> <br>python -m pip install docker<br><br>/root/kolla-venv/bin/python3.12 -m pip install docker<br> python -m pip install dbus-python<br> <br>sudo apt update<br>sudo apt install -y \<br> python3.12-dev \<br> libdbus-1-dev \<br> libdbus-glib-1-dev \<br> pkg-config \<br> gcc \<br> g++</pre><h3>Networking</h3><p>OpenStack networking is one of the most important parts of building a private cloud. After deploying OpenStack with Kolla-Ansible, you need to configure Neutron so that virtual machines can communicate with each other and, if required, access external networks through floating IPs.</p><p>In this guide, we’ll focus on OpenStack network configuration using Kolla-Ansible, including:</p><ul><li>Management and external network interfaces.</li><li>Neutron provider networks.</li><li>OpenStack external networks and subnets.</li><li>Router configuration.</li><li>Floating IPs for virtual machines.</li><li>Network troubleshooting.</li></ul><p>The examples are designed for a learning lab using Ubuntu 24.04 and Kolla-Ansible. Adjust the IP ranges and interfaces to match your own environment.</p><pre><br>                 External Network<br>                  192.168.1.0/24<br>                         |<br>                 External Interface<br>                         |<br>                  OpenStack Router<br>                         |<br>              ---------------------<br>              |                   |<br>         Tenant Network       Tenant Network<br>          10.10.10.0/24       10.10.20.0/24<br>              |                   |<br>           VM-01                VM-02</pre><h3>Install Kolla-Ansible</h3><p>The following uses the OpenStack 2026.1 branch and a Python virtual environment, following the official quick-start flow.</p><p>kolla-ansible 22.2.1.dev9 documentation</p><pre>python3 -m venv ~/kolla-venv<br>source ~/kolla-venv/bin/activate<br>pip install -U pip<br>pip install git+https://opendev.org/openstack/kolla-ansible@stable/2026.1</pre><p><strong>Install Ansible dependencies:</strong></p><pre>kolla-ansible install-deps</pre><h3>Prepare configuration</h3><pre>sudo mkdir -p /etc/kolla<br>sudo chown $USER:$USER /etc/kolla</pre><p>Copy the example configuration:</p><pre>cp -r ~/kolla-venv/share/kolla-ansible/etc_examples/kolla/* /etc/kolla/</pre><p>Copy the all-in-one inventory:</p><pre>cp ~/kolla-venv/share/kolla-ansible/ansible/inventory/all-in-one .</pre><p>Generate passwords:</p><pre>kolla-genpwd</pre><h3>Configure `globals.yml`</h3><pre>nano /etc/kolla/globals.yml</pre><p>Minimum example for a lab:</p><pre><br>kolla_base_distro: “ubuntu”<br><br>network_interface: &quot;ens18&quot;<br>neutron_external_interface: &quot;ens19&quot;<br><br># Ip need to give that ip not even used<br>kolla_internal_vip_address: &quot;192.168.1.100&quot;</pre><p>Change these values to match your environment. `ens18` and `ens19` are examples, not guaranteed interface names. Use `ip -br addr` to identify the correct interfaces.</p><p>The internal VIP must be an unused IP address on your management network and must be appropriate for your chosen topology.</p><h3>Deploy OpenStack</h3><p>Run the following in order:</p><p>NOTE: If you need to test purposes, use this argument: ` — use-test-images`</p><pre>kolla-ansible bootstrap-servers -i ./all-in-one</pre><p>Checking the deployment</p><pre>kolla-ansible prechecks -i ./all-in-one</pre><p>Deploying Playbooks</p><pre>kolla-ansible deploy -i ./all-in-one <br></pre><p>Installing OpenStack client</p><pre>pip install python-openstackclient</pre><h3>Move all-in-one to the Ansible inventory folder</h3><pre>mkdir -p /etc/kolla/ansible/inventory<br>cp all-in-one /etc/kolla/ansible/inventory</pre><p>These are the core preparation, validation, and deployment stages in the Kolla-Ansible workflow.</p><h3>kolla-ansible 22.2.1.dev9 documentation</h3><p>Generate OpenStack admin credentials</p><pre>kolla-ansible post-deploy</pre><p>Load the generated admin credentials:</p><pre>source /etc/kolla/admin-openrc.sh</pre><p>Test the deployment:</p><pre>openstack token issue<br>openstack service list<br>openstack compute service list</pre><h3>Access Horizon</h3><p>Open the configured internal VIP in your browser:</p><pre>http://192.168.1.100</pre><p>Use the admin password from:</p><pre>grep keystone_admin_password /etc/kolla/passwords.yml</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8add3132e716" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Install Ceph on Ubuntu 22.04 Using Cephadm]]></title>
            <link>https://arn-ob.medium.com/how-to-install-ceph-on-ubuntu-22-04-using-cephadm-c5e23c9b1f21?source=rss-c7040ca07397------2</link>
            <guid isPermaLink="false">https://medium.com/p/c5e23c9b1f21</guid>
            <category><![CDATA[clustering]]></category>
            <category><![CDATA[storage]]></category>
            <category><![CDATA[ceph]]></category>
            <category><![CDATA[distributed-systems]]></category>
            <dc:creator><![CDATA[Arnob]]></dc:creator>
            <pubDate>Sun, 20 Sep 2026 08:55:57 GMT</pubDate>
            <atom:updated>2026-09-20T08:55:57.557Z</atom:updated>
            <content:encoded><![CDATA[<p>Ceph is a distributed storage platform that provides <strong>object, block, and file storage</strong> through a unified storage cluster. It is widely used in private clouds, virtualization platforms, Kubernetes environments, and large-scale infrastructure.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*86-ZL5_MerLuWJs-6jcoxA.png" /></figure><p>In this guide, we will install Ceph on <strong>Ubuntu 22.04 LTS</strong> using <strong>Cephadm</strong>, bootstrap the initial cluster, inspect storage devices, deploy an OSD, access the dashboard, and review how CephFS, Ceph RBD, and Ceph RGW can be used with OpenCloud and OpenStack.</p><blockquote><strong><em>Important:</em></strong><em> This guide is suitable for a learning environment and initial lab setup. A production deployment requires careful planning for high availability, networking, capacity, security, and failure domains.</em></blockquote><h3>1. Understanding Ceph Architecture</h3><p>Before installing Ceph, it is important to understand its main components.</p><pre>| Component | Purpose |<br>|---|---|<br>| **Ceph MON** | Maintains cluster maps, authentication, and quorum |<br>| **Ceph MGR** | Provides management modules, metrics, and dashboards |<br>| **Ceph OSD** | Stores data on disks and handles replication |<br>| **Ceph MDS** | Provides metadata services required by CephFS |<br>| **Ceph RGW** | Provides an S3-compatible object storage interface |<br>| **Ceph RBD** | Provides virtual block devices for VMs and OpenStack |</pre><h3>Ceph Storage Interfaces</h3><p>Ceph supports multiple storage interfaces, each designed for a different workload.</p><h3>CephFS</h3><p>CephFS is a distributed POSIX-compatible filesystem that exposes directories and files.</p><p>It can be useful for:</p><ul><li>Shared file storage</li><li>Applications that require filesystem semantics</li><li>OpenCloud deployments when CephFS is supported and configured correctly</li></ul><h3>Ceph RBD</h3><p>Ceph RBD, or RADOS Block Device, provides virtual block devices.</p><p>It is commonly used for:</p><ul><li>OpenStack Cinder</li><li>QEMU/KVM virtual machines</li><li>Block storage workloads</li><li>Applications that require raw block devices</li></ul><h3>Ceph RGW</h3><p>Ceph RGW, or RADOS Gateway, provides an S3-compatible object storage API.</p><p>It is useful for:</p><ul><li>S3-compatible applications</li><li>Object storage services</li><li>Backup repositories</li><li>Private cloud object storage</li></ul><blockquote><strong><em>Practical recommendation:</em></strong><em> For an OpenCloud and OpenStack lab, first understand CephFS and RBD. Add RGW later if you need S3-compatible object storage.</em></blockquote><h3>2. Choosing the Installation Method</h3><p>There are two common approaches for installing Ceph on Ubuntu 22.04.</p><pre>| Method | Best suited for |<br>|---|---|<br>| **MicroCeph** | Quick single-node labs and simple Ubuntu-focused deployments |<br>| **Cephadm** | Multi-node clusters and deployments requiring more control over services |</pre><p>This guide focuses on <strong>Cephadm</strong>, which is designed to deploy and manage Ceph services using an orchestrator.</p><p>MicroCeph is another option for users who want a simpler Ubuntu-oriented installation, especially for learning or small test environments.</p><blockquote><em>Do not mix the management workflows of MicroCeph and Cephadm in the same cluster.</em></blockquote><h3>3. Lab Architecture</h3><p>A basic Ceph lab may look like this:</p><pre>MikroTik Router<br>                          |<br>                Management Network<br>                          |<br>        +-----------------+-----------------+<br>        |                 |                 |<br>   ceph-node1       ceph-node2       ceph-node3<br>   MON/MGR/OSD      MON/MGR/OSD      MON/MGR/OSD<br>        |                 |                 |<br>        +-----------------+-----------------+<br>                          |<br>                 Ceph Storage Pools<br>              CephFS | RBD | RGW<br>                    /       \<br>               OpenCloud   OpenStack<br>                CephFS       RBD</pre><p>For a production-style replicated cluster, use multiple physical hosts and distribute monitors and OSDs across appropriate failure domains.</p><p>A single-node cluster can be useful for learning, but it does not provide meaningful host-level high availability.</p><h3>4. Prerequisites</h3><p>Before installing Ceph, prepare the Ubuntu 22.04 host.</p><h3>Recommended lab requirements</h3><pre>| Requirement | Recommendation |<br>|---|---|<br>| Operating system | Ubuntu Server 22.04 LTS |<br>| CPU | At least 2–4 vCPUs for a basic lab |<br>| RAM | 8 GB or more for a basic lab |<br>| OS disk | 20–30 GB or more |<br>| OSD disk | Separate unused disk |<br>| Network | Stable IP address and hostname resolution |<br>| Time synchronization | Chrony or another supported time-sync service |<br>| Runtime | Supported container runtime |<br>| Access | Sudo/root access and SSH for multi-node deployments |</pre><p>Cephadm requires a suitable Python environment, systemd, a container runtime, time synchronization, and LVM2.</p><h3>Check the current system</h3><pre>hostnamectl<br>ip -br addr<br>lsblk<br>free -h</pre><blockquote><strong><em>Warning:</em></strong><em> Never use a disk containing important data as a Ceph OSD. OSD provisioning can erase existing partitions, filesystem signatures, and data.</em></blockquote><h3>5. Update Ubuntu and Install Dependencies</h3><p>Run the following commands on the Ceph host:</p><pre>sudo apt update<br>sudo apt upgrade -y</pre><p>Install the basic dependencies:</p><pre>sudo apt install -y \<br>  curl \<br>  wget \<br>  lvm2 \<br>  chrony \<br>  openssh-server \<br>  ca-certificates \<br>  gnupg \<br>  python3</pre><p>Enable time synchronization:</p><pre>sudo systemctl enable --now chrony</pre><p>Check the system time:</p><pre>timedatectl status</pre><p>Check Chrony:</p><pre>chronyc tracking</pre><p>A stable system clock is important for reliable distributed cluster operations.</p><h3>6. Configure the Hostname</h3><p>Each Ceph node should have a unique hostname.</p><p>For the first node:</p><pre>sudo hostnamectl set-hostname ceph-node1</pre><p>Check the hostname:</p><pre>hostnamectl<br>hostname -I</pre><p>For a three-node cluster, you could use:</p><pre>ceph-node1<br>ceph-node2<br>ceph-node3</pre><h3>Configure /etc/hosts</h3><p>Edit the hosts file:</p><pre>sudo nano /etc/hosts</pre><p>Example:</p><pre>192.168.10.11 ceph-node1<br>192.168.10.12 ceph-node2<br>192.168.10.13 ceph-node3</pre><p>Use your actual IP addresses.</p><p>Make sure all nodes can resolve one another:</p><pre>ping -c 3 ceph-node1<br>ping -c 3 ceph-node2<br>ping -c 3 ceph-node3</pre><p>For production environments, use reliable DNS or consistently managed host entries.</p><h3>7. Install Cephadm</h3><p>First, check whether Ubuntu provides the cephadm package:</p><pre>sudo apt update<br>apt-cache policy cephadm</pre><p>Install Cephadm:</p><pre>sudo apt install -y cephadm</pre><p>Verify the installation:</p><pre>which cephadm<br>cephadm version</pre><p>The executable may be located at:</p><pre>/usr/sbin/cephadm</pre><h3>Version consistency</h3><p>If the Ubuntu package is not suitable for your target Ceph release, use the official Ceph repository-based installation method for a supported release.</p><p>Avoid mixing different installation methods or combining repositories and container images from incompatible Ceph releases.</p><h3>8. Bootstrap the Ceph Cluster</h3><p>Bootstrapping creates the initial Ceph cluster on the first node.</p><p>Run:</p><pre>sudo cephadm bootstrap \<br>  --mon-ip 192.168.10.11</pre><p>Replace 192.168.10.11 with the IP address of your first Ceph node.</p><p>During bootstrap, Cephadm creates initial cluster services, including:</p><ul><li>Ceph Monitor</li><li>Ceph Manager</li><li>Initial cluster configuration</li><li>Management credentials</li><li>Dashboard access</li></ul><p>After bootstrap completes, check the cluster status:</p><pre>sudo cephadm shell -- ceph -s</pre><p>You can also enter the Ceph shell:</p><pre>sudo cephadm shell</pre><p>Then run:</p><pre>ceph -s</pre><p>Exit the shell:</p><pre>exit</pre><h3>Initial cluster warnings</h3><p>A newly bootstrapped cluster may report warnings related to:</p><ul><li>Missing OSDs</li><li>Monitor count</li><li>Replication settings</li><li>Single-node deployment</li><li>Network configuration</li></ul><p>A successful bootstrap does not automatically mean that the cluster is ready for production workloads.</p><h3>9. Inspect Available Storage Devices</h3><p>Ceph stores data on OSDs. Ideally, use dedicated disks that do not contain the operating system.</p><p>List storage devices from Ubuntu:</p><pre>lsblk -o NAME,SIZE,TYPE,FSTYPE,MOUNTPOINTS</pre><p>Example:</p><pre>NAME   SIZE TYPE FSTYPE MOUNTPOINTS<br>sda     50G disk        /<br>sdb    100G disk<br>sdc    100G disk</pre><p>In this example, /dev/sdb and /dev/sdc may be potential OSD devices, provided they are unused and suitable for Ceph.</p><p>You can also inspect devices through the Ceph orchestrator:</p><pre>sudo cephadm shell -- ceph orch device ls</pre><p>Before provisioning a disk, verify:</p><ul><li>It is the correct device.</li><li>It is not mounted.</li><li>It does not contain important data.</li><li>It is not already used by another storage system.</li><li>It meets the requirements of the selected Ceph release.</li></ul><h3>10. Deploy an OSD</h3><p>For a dedicated, unused disk, run:</p><pre>sudo cephadm shell -- \<br>  ceph orch daemon add osd ceph-node1:/dev/sdb</pre><p>Replace:</p><ul><li>ceph-node1 with the correct host</li><li>/dev/sdb with the correct unused disk</li></ul><p>Check the OSD status:</p><pre>sudo cephadm shell -- ceph osd status</pre><p>Display the OSD tree:</p><pre>sudo cephadm shell -- ceph osd tree</pre><p>Check overall cluster health:</p><pre>sudo cephadm shell -- ceph -s</pre><blockquote><strong><em>Disk safety warning:</em></strong><em> Adding a disk as an OSD can erase existing data and partition information. Always verify the device path before proceeding.</em></blockquote><h3>11. Understand the TOO_FEW_OSDS Warning</h3><p>You may see a warning similar to:</p><pre>TOO_FEW_OSDS:<br>OSD count 0 &lt; osd_pool_default_size 3</pre><p>This means:</p><ul><li>The cluster currently has zero OSDs.</li><li>The default pool replication setting expects three copies.</li><li>The cluster does not yet have enough storage daemons to satisfy that configuration.</li></ul><p>The warning is expected in a freshly bootstrapped cluster without OSDs.</p><p>After adding OSDs, the warning may change depending on:</p><ul><li>Number of active OSDs</li><li>Pool replication size</li><li>Failure-domain configuration</li><li>Cluster health</li><li>Placement group state</li></ul><p>For a three-replica production design, plan for sufficient OSDs across multiple physical hosts and appropriate failure domains.</p><p>Do not simply change the replica count to 1 to hide the warning in a production environment. Doing so reduces data redundancy.</p><h3>12. Access the Ceph Dashboard</h3><p>Cephadm generally deploys the dashboard during bootstrap.</p><p>Check the manager services:</p><pre>sudo cephadm shell -- ceph mgr services</pre><p>Example output:</p><pre>{<br>  &quot;dashboard&quot;: &quot;https://192.168.10.11:8443/&quot;<br>}</pre><p>Open the displayed URL in your browser:</p><pre>https://192.168.10.11:8443/</pre><p>Use the credentials generated during the bootstrap process.</p><p>If you need to retrieve credentials later, check the bootstrap output or the Cephadm-generated credential file according to the Ceph release you installed.</p><blockquote><em>Avoid exposing the management dashboard directly to the public internet. Use firewall rules, VPN access, or another secure administrative access method.</em></blockquote><h3>13. Add Additional Ceph Nodes</h3><p>After bootstrapping the first node, you can add additional hosts.</p><p>First, retrieve the Cephadm SSH public key:</p><pre>sudo ceph cephadm get-pub-key</pre><p>Add the public key to the remote host’s root SSH authorized_keys file, or use the supported host onboarding process for your Ceph release.</p><p>Then add the host:</p><pre>sudo cephadm shell -- \<br>  ceph orch host add ceph-node2 192.168.10.12</pre><p>Check the registered hosts:</p><pre>sudo cephadm shell -- ceph orch host ls</pre><p>Repeat the process for additional nodes.</p><p>Make sure the remote hosts have:</p><ul><li>Correct hostname resolution</li><li>SSH access</li><li>Time synchronization</li><li>Supported operating system and dependencies</li><li>Proper network connectivity</li></ul><h3>14. Verify the Ceph Installation</h3><p>Use the following commands to verify the cluster.</p><h3>Check cluster health</h3><pre>sudo cephadm shell -- ceph -s</pre><h3>Display detailed health information</h3><pre>sudo cephadm shell -- ceph health detail</pre><h3>Display the OSD tree</h3><pre>sudo cephadm shell -- ceph osd tree</pre><h3>List storage pools</h3><pre>sudo cephadm shell -- ceph osd pool ls</pre><h3>List deployed services</h3><pre>sudo cephadm shell -- ceph orch ps</pre><h3>List registered hosts</h3><pre>sudo cephadm shell -- ceph orch host ls</pre><h3>Useful command summary</h3><p>CommandPurposeceph -sDisplay cluster statusceph health detailShow detailed health warningsceph osd treeDisplay the OSD hierarchyceph osd statusShow OSD statusceph dfShow storage usageceph orch host lsList cluster hostsceph orch psList deployed servicesceph orch device lsList available storage devices</p><h3>15. CephFS, RBD, and RGW: Which One Should You Use?</h3><p>The correct Ceph interface depends on the workload.</p><p>Storage interfaceTypical use<strong>CephFS</strong>Shared filesystem and file-based applications<strong>Ceph RBD</strong>VM disks, OpenStack Cinder, and block storage<strong>Ceph RGW</strong>S3-compatible object storage</p><h3>OpenCloud</h3><p>If your OpenCloud deployment requires a shared filesystem, CephFS may be relevant, provided the OpenCloud version and configuration support it.</p><h3>OpenStack</h3><p>For OpenStack VM storage, Ceph RBD is commonly used with services such as Cinder and compatible compute integrations.</p><h3>S3 service</h3><p>If your goal is to build an S3-compatible storage service, investigate Ceph RGW and its user, bucket, authentication, and gateway configuration.</p><p>These storage interfaces are different and should not be treated as interchangeable.</p><h3>16. Example Lab Design for Three Computers</h3><p>If you have three computers with approximately 12–16 GB of RAM each, you can use them as a Ceph and OpenStack learning environment.</p><p>ComputerRAMPossible roleIntel Core i512 GBCeph / OpenStack labIntel Core i516 GBCeph / OpenStack labAMD Ryzen 516 GBCeph / OpenStack lab</p><p>This is a constrained environment for running Ceph, OpenStack, and other services together.</p><p>Before using it for production workloads, evaluate:</p><ul><li>CPU capacity</li><li>RAM usage</li><li>Disk type and capacity</li><li>Network throughput</li><li>Failure-domain design</li><li>Power and hardware reliability</li><li>Workload requirements</li></ul><p>Use dedicated storage disks where possible, and monitor memory consumption closely.</p><h3>17. Common Mistakes to Avoid</h3><h3>Mistake 1: Using the OS disk as an OSD</h3><p>Always inspect disks before provisioning them.</p><pre>lsblk -o NAME,SIZE,TYPE,FSTYPE,MOUNTPOINTS</pre><h3>Mistake 2: Running a production cluster on one node</h3><p>A single node is useful for learning but does not provide host-level redundancy.</p><h3>Mistake 3: Ignoring time synchronization</h3><p>Distributed storage systems rely on reliable system clocks.</p><pre>timedatectl status<br>chronyc tracking</pre><h3>Mistake 4: Mixing Cephadm and MicroCeph management</h3><p>Choose one deployment method for a cluster and follow its management workflow consistently.</p><h3>Mistake 5: Treating bootstrap as a complete production deployment</h3><p>Bootstrap creates the initial services, but production readiness also requires:</p><ul><li>Multiple monitors</li><li>Sufficient OSD capacity</li><li>Replication planning</li><li>Network design</li><li>Monitoring and alerting</li><li>Backup and recovery procedures</li><li>Security hardening</li></ul><h3>18. Final Checklist</h3><p>Before considering the initial installation complete, verify the following:</p><ul><li>Ubuntu 22.04 is installed and updated</li><li>Hostnames and IP addresses are configured</li><li>All relevant nodes can resolve one another</li><li>Time synchronization is enabled</li><li>Required dependencies are installed</li><li>Cephadm is installed and verified</li><li>The cluster has been bootstrapped</li><li>Cluster health has been checked</li><li>Dedicated OSD disks have been identified</li><li>OSDs have been deployed</li><li>Additional hosts have been added if required</li><li>Dashboard access has been verified</li><li>Storage interfaces have been selected based on workload</li></ul><h3>Conclusion</h3><p>Installing Ceph on Ubuntu 22.04 using Cephadm is a practical way to start learning distributed storage and build a foundation for private cloud infrastructure.</p><p>The initial workflow is:</p><ol><li>Prepare Ubuntu and configure networking.</li><li>Install Cephadm.</li><li>Bootstrap the cluster.</li><li>Inspect and provision dedicated OSD disks.</li><li>Add additional hosts when required.</li><li>Verify cluster health and deployed services.</li><li>Configure CephFS, RBD, or RGW based on the application.</li></ol><p>For an OpenCloud and OpenStack lab, understanding the difference between <strong>CephFS, Ceph RBD, and Ceph RGW</strong> is essential because each interface solves a different storage problem.</p><h3>References</h3><ul><li><a href="https://docs.ceph.com/">Ceph Documentation</a></li><li><a href="https://docs.ceph.com/en/latest/cephadm/install/">Cephadm Installation Guide</a></li><li><a href="https://docs.ceph.com/en/latest/mgr/orchestrator_modules/">Ceph Orchestrator Documentation</a></li><li><a href="https://docs.ceph.com/en/latest/cephfs/">CephFS Documentation</a></li><li><a href="https://docs.ceph.com/en/latest/rbd/">Ceph RBD Documentation</a></li><li><a href="https://ubuntu.com/ceph">Ubuntu Ceph Documentation</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c5e23c9b1f21" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AWS CloudFront Under the Hood: What Really Happens When a User Requests Your Website?]]></title>
            <link>https://arn-ob.medium.com/aws-cloudfront-under-the-hood-what-really-happens-when-a-user-requests-your-website-4d2a995f53e6?source=rss-c7040ca07397------2</link>
            <guid isPermaLink="false">https://medium.com/p/4d2a995f53e6</guid>
            <category><![CDATA[website]]></category>
            <category><![CDATA[dns]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[cloudfront]]></category>
            <category><![CDATA[caching]]></category>
            <dc:creator><![CDATA[Arnob]]></dc:creator>
            <pubDate>Tue, 08 Sep 2026 11:22:38 GMT</pubDate>
            <atom:updated>2026-09-08T11:22:38.321Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>A deep dive into DNS, Anycast, Edge PoPs, caching, WAF, Regional Edge Cache, Origin Shield, and the complete CloudFront request lifecycle</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1012/0*VDYY39bCIPTnRLTx" /><figcaption>From sls.guru/blog</figcaption></figure><p>When we first learn AWS CloudFront, it is tempting to think of it as simply:</p><blockquote><strong><em>“A server that stores my website closer to users.”</em></strong></blockquote><p>That mental model is useful but incomplete.</p><p>CloudFront is much more interesting than that.</p><p>It is a <strong>globally distributed content delivery and edge-computing platform</strong> that sits between your users and your origin. It combines global network routing, edge locations, caching, TLS termination, security controls, request processing, and optimized connectivity to your origin.</p><p>The interesting question is:</p><blockquote><strong><em>What actually happens when a user in Canada requests an image from an application whose origin is running in Dhaka?</em></strong></blockquote><p>Let’s follow that request from beginning to end.</p><h3>1. The Architecture We Are Going to Follow</h3><p>Imagine that your application is running in AWS in Dhaka.</p><p>A simplified architecture might look like this:</p><pre>                         INTERNET<br>                             │<br>                             ▼<br>                      ┌─────────────┐<br>                      │   Route 53  │<br>                      └──────┬──────┘<br>                             │<br>                             ▼<br>                      ┌─────────────┐<br>                      │ CloudFront  │<br>                      │   Network   │<br>                      └──────┬──────┘<br>                             │<br>                       Cache / WAF<br>                             │<br>                             ▼<br>                           ALB<br>                             │<br>                             ▼<br>                            EKS<br>                             │<br>                             ▼<br>                            RDS</pre><p>Your website could be:</p><pre>https://www.example.com</pre><p>And an image might be:</p><pre>https://www.example.com/images/logo.png</pre><p>Now imagine a user in <strong>Toronto, Canada</strong> opens the website.</p><p>The origin, however, is in <strong>Dhaka, Bangladesh</strong>.</p><p>The question is:</p><blockquote><em>Does the Canadian user have to travel all the way to Dhaka every time they request </em><em>logo.png?</em></blockquote><p><strong>No, not if CloudFront already has the object cached.</strong></p><p>That’s the fundamental idea behind a CDN.</p><h3>2. CloudFront Is Not One Server</h3><p>The first thing to understand is that CloudFront is <strong>not a single server sitting somewhere in AWS</strong>.</p><p>Instead, think of it as a globally distributed network:</p><pre>CloudFront<br>                          │<br>          ┌───────────────┼───────────────┐<br>          │               │               │<br>          ▼               ▼               ▼<br>        Canada          Europe           Asia<br>         PoP              PoP              PoP<br>          │               │               │<br>          ▼               ▼               ▼<br>        Cache           Cache            Cache</pre><p>Users around the world enter the CloudFront network through appropriate edge locations.</p><p>CloudFront then determines whether the requested content can be served from its cache or whether it needs to retrieve the content from your origin.</p><p>The source material describes this as CloudFront being a <strong>globally distributed HTTP/HTTPS reverse-proxy and caching system sitting between users and the origin</strong>.</p><h3>3. Step 1 — The User Requests Your Website</h3><p>The Canadian user enters:</p><pre>https://www.example.com</pre><p>The browser first needs to resolve:</p><pre>www.example.com</pre><p>Your DNS configuration points the domain toward CloudFront.</p><p>Conceptually:</p><pre>www.example.com<br>       │<br>       ▼<br>CloudFront distribution<br>       │<br>       ▼<br>CloudFront network</pre><p>The important thing is that you don’t normally tell DNS:</p><pre>Canada → Toronto PoP<br>Japan → Tokyo PoP<br>Bangladesh → Dhaka PoP</pre><p>You configure your domain to use CloudFront, and CloudFront’s global infrastructure handles the traffic entry.</p><h3>4. DNS Does Not Simply Mean “Choose the Nearest Server”</h3><p>This is a common misunderstanding.</p><p>You might imagine that CloudFront calculates:</p><pre>Toronto → Toronto PoP = 20 km<br>Toronto → New York PoP = 500 km<br>Toronto → London PoP = 5,000 km</pre><p>and simply selects Toronto.</p><p>Real Internet routing is more complicated.</p><p>CloudFront’s global network uses routing mechanisms including <strong>Anycast/BGP</strong>, and the actual PoP selection and internal routing logic is managed by AWS.</p><p>Therefore:</p><blockquote><strong><em>“Nearest CloudFront location” should not be interpreted as simply “the geographically closest server.”</em></strong></blockquote><p>Network topology, routing policies, connectivity, capacity, and other factors can influence where traffic enters the network.</p><h3>5. Anycast and BGP: How Does the Request Enter CloudFront?</h3><p>This is where things get interesting from a networking perspective.</p><p>CloudFront exposes a global service rather than requiring you to configure a separate public IP for every edge location.</p><p>Conceptually:</p><pre>CloudFront<br>                 Global Network<br>                       │<br>          ┌────────────┼────────────┐<br>          │            │            │<br>          ▼            ▼            ▼<br>        PoP A        PoP B        PoP C<br>       Canada       Europe        Asia</pre><p>The Internet uses <strong>BGP Border Gateway Protocol</strong> to exchange routing information between networks.</p><p>So the flow is roughly:</p><pre>Canadian User<br>      │<br>      ▼<br>    ISP<br>      │<br>      ▼<br>  Internet/BGP<br>      │<br>      ▼<br>CloudFront Edge</pre><p>The exact internal CloudFront routing architecture is AWS-managed and proprietary.</p><p>Therefore, it is better to think:</p><pre>User<br>  ↓<br>Internet routing<br>  ↓<br>Appropriate CloudFront entry point<br>  ↓<br>CloudFront network</pre><p>rather than:</p><pre>User<br>  ↓<br>Nearest physical server</pre><h3>6. Step 2 — The Request Reaches a CloudFront Edge PoP</h3><p>Suppose the request reaches an appropriate CloudFront edge location.</p><p>Conceptually:</p><pre>🇨🇦 Toronto User<br>       │<br>       │ HTTPS<br>       ▼<br>┌─────────────────────┐<br>│  CloudFront Edge    │<br>│       PoP           │<br>│                     │<br>│ TLS                 │<br>│ WAF                 │<br>│ Cache               │<br>│ Edge processing     │<br>└─────────────────────┘</pre><p>Now CloudFront can process the request.</p><p>Several things may happen at the edge:</p><ol><li>TLS processing</li><li>WAF inspection</li><li>CloudFront Functions / Lambda@Edge processing, if configured</li><li>Cache-key construction</li><li>Cache lookup</li><li>Origin request if required</li></ol><h3>7. Step 3 — TLS/HTTPS Processing</h3><p>The user requested:</p><pre>https://www.example.com</pre><p>Therefore, an HTTPS connection is established with CloudFront.</p><p>Conceptually:</p><pre>🇨🇦 User<br>    │<br>    │ HTTPS<br>    ▼<br>CloudFront<br>    │<br>    │ HTTPS/HTTP<br>    ▼<br>Origin</pre><p>CloudFront can terminate the viewer-side TLS connection using a certificate configured for the distribution.</p><p>So CloudFront becomes an important security boundary between the Internet and your origin.</p><h3>8. Step 4 — AWS WAF Inspects the Request</h3><p>If AWS WAF is associated with CloudFront, the request can be inspected before it reaches your origin.</p><p>For example:</p><pre>Request<br>                    │<br>                    ▼<br>               CloudFront<br>                    │<br>                    ▼<br>                  WAF<br>                /     \<br>               /       \<br>          Malicious   Legitimate<br>             │           │<br>             ▼           ▼<br>           BLOCK       Continue</pre><p>WAF can evaluate information such as:</p><pre>Source IP<br>HTTP headers<br>URI<br>Query parameters<br>HTTP method<br>Cookies<br>Request patterns<br>Rate-based rules<br>Managed rules</pre><p>This means malicious traffic can potentially be rejected at the edge rather than consuming resources in your ALB, EKS cluster, or application.</p><h3>9. Step 5 — CloudFront Builds the Cache Key</h3><p>Now we reach one of the most important parts of CloudFront:</p><h4>The cache.</h4><p>Suppose the user requested:</p><pre>GET /images/logo.png</pre><p>CloudFront needs to determine which cached object corresponds to that request.</p><p>Conceptually:</p><pre>Cache Key =<br>    Host<br>    +<br>    Path<br>    +<br>    Selected query strings<br>    +<br>    Selected headers<br>    +<br>    Selected cookies</pre><p>The exact components depend on your CloudFront cache policy.</p><p>For example:</p><pre>www.example.com<br>/images/logo.png</pre><p>could represent one cache entry.</p><p>But:</p><pre>/images/logo.png?version=2</pre><p>could represent a different cache key if query strings are included in the cache policy.</p><p>This is why <strong>cache policy design is extremely important for CDN performance</strong>.</p><h3>10. Step 6 — Cache HIT</h3><p>CloudFront now asks:</p><blockquote><strong><em>Do I already have this object cached?</em></strong></blockquote><p>Suppose the answer is yes.</p><p>Then:</p><pre>🇨🇦 User<br>    │<br>    ▼<br>CloudFront PoP<br>    │<br>    ▼<br>Cache<br>    │<br>    │ HIT<br>    ▼<br>logo.png<br>    │<br>    ▼<br>🇨🇦 User</pre><p>The origin is not contacted.</p><p>This is the real power of CloudFront.</p><p>Imagine:</p><pre>100,000 requests<br>        │<br>        ▼<br>    CloudFront<br>      /    \<br>     /      \<br>95,000 HIT  5,000 MISS<br>               │<br>               ▼<br>             Origin</pre><p>Instead of your origin handling 100,000 requests, it may only need to handle the requests necessary to populate or refresh the cache.</p><p>Your source material highlights exactly this distinction: on a cache hit, the origin isn’t involved.</p><h3>11. Step 7 — Cache MISS</h3><p>Now imagine that /images/logo.png is not available in the relevant cache.</p><p>CloudFront gets:</p><pre>CACHE MISS</pre><p>The request then has to move deeper into the CloudFront caching hierarchy and, if necessary, toward the origin.</p><p>Conceptually:</p><pre>User<br> │<br> ▼<br>CloudFront Edge PoP<br> │<br> │ MISS<br> ▼<br>Regional Edge Cache<br> │<br> │ MISS<br> ▼<br>Origin</pre><p>CloudFront supports origins such as:</p><pre>S3<br>ALB<br>EC2<br>API Gateway</pre><p>and an ALB can then send the request toward workloads such as EKS.</p><h3>12. Regional Edge Cache</h3><p>The next important component is the <strong>Regional Edge Cache (REC)</strong>.</p><p>Think of it as a larger caching layer within CloudFront’s architecture.</p><p>Conceptually:</p><pre>Origin<br>                     ▲<br>                     │<br>            ┌────────┴────────┐<br>            │ Regional Edge   │<br>            │      Cache      │<br>            └────────┬────────┘<br>                     │<br>            ┌────────┼────────┐<br>            │        │        │<br>            ▼        ▼        ▼<br>          PoP A    PoP B    PoP C</pre><p>So a request doesn’t necessarily have to go directly from an edge PoP to your origin every time.</p><p>The hierarchy can be thought of as:</p><pre>User<br>  ↓<br>Edge PoP<br>  ↓<br>Regional Edge Cache<br>  ↓<br>Origin</pre><p>The purpose is to reduce unnecessary trips back to the origin.</p><h3>13. Origin Shield: Another Layer of Protection for Your Origin</h3><p>For some architectures, CloudFront can use <strong>Origin Shield</strong>.</p><p>Conceptually:</p><pre>            CloudFront PoPs<br>           /       |       \<br>          /        |        \<br>        PoP       PoP       PoP<br>          \        |        /<br>           \       |       /<br>            ▼      ▼      ▼<br>             Origin Shield<br>                   │<br>                   ▼<br>                 Origin</pre><p>Without Origin Shield:</p><pre>PoP A ─────┐<br>PoP B ─────┼──────► Origin<br>PoP C ─────┤<br>PoP D ─────┘</pre><p>With Origin Shield:</p><pre>PoP A ─────┐<br>PoP B ─────┤<br>PoP C ─────┼──► Origin Shield ──► Origin<br>PoP D ─────┘</pre><p>Origin Shield provides another caching/consolidation layer and can reduce the number of requests that ultimately reach your origin.</p><p>This can be particularly useful when your origin is expensive, distant, or receives traffic from many different CloudFront locations.</p><h3>14. Now Let’s Follow the Request to Dhaka</h3><p>Let’s make the architecture concrete.</p><p>The Canadian user requests:</p><pre>https://www.example.com/images/logo.png</pre><p>Your backend is:</p><pre>🇧🇩 Dhaka<br>    │<br>    ▼<br>   ALB<br>    │<br>    ▼<br>   EKS</pre><p>The complete path can conceptually look like:</p><pre>🇨🇦 Canada<br>    │<br>    │ HTTPS<br>    ▼<br>CloudFront Edge PoP<br>    │<br>    ├── TLS<br>    │<br>    ├── WAF<br>    │<br>    └── Cache<br>          │<br>          │ MISS<br>          ▼<br>   Regional Edge Cache<br>          │<br>          │ MISS<br>          ▼<br>        Origin<br>          │<br>          ▼<br>      🇧🇩 Dhaka<br>          │<br>          ▼<br>         ALB<br>          │<br>          ▼<br>         EKS<br>          │<br>          ▼<br>      Application</pre><p>The application returns:</p><pre>HTTP 200 OK</pre><pre>Content-Type: image/png</pre><pre>[logo.png]</pre><h3>15. The Response Travels Back Through CloudFront</h3><p>The response then travels back toward CloudFront.</p><p>Conceptually:</p><pre>Application<br>    │<br>    ▼<br>   EKS<br>    │<br>    ▼<br>   ALB<br>    │<br>    ▼<br>CloudFront<br>    │<br>    ├── Store/cache object<br>    │<br>    ▼<br>🇨🇦 Canadian User</pre><p>CloudFront can store the object according to the distribution’s caching configuration and relevant HTTP cache metadata.</p><p>For example:</p><pre>/images/logo.png</pre><pre>Object:<br>    logo.png</pre><pre>TTL:<br>    86400 seconds</pre><p>The source material describes the cached entry as containing the object, TTL, headers, and metadata.</p><h3>16. The Second Request Is Completely Different</h3><p>This is where CDN architecture becomes powerful.</p><h3>First request</h3><pre>Canada<br>   │<br>   ▼<br>CloudFront PoP<br>   │<br>   ▼<br>Cache MISS<br>   │<br>   ▼<br>Regional Edge Cache<br>   │<br>   ▼<br>Origin in Dhaka<br>   │<br>   ▼<br>CloudFront<br>   │<br>   ▼<br>Canada</pre><h3>Second request</h3><pre>Canada<br>   │<br>   ▼<br>CloudFront PoP<br>   │<br>   ▼<br>CACHE HIT<br>   │<br>   ▼<br>Canada</pre><p>The Dhaka origin doesn’t need to participate in the second request.</p><p>This is the fundamental CDN optimization.</p><h3>17. Does CloudFront Copy My Entire Website Everywhere?</h3><p><strong>No.</strong></p><p>This is another important misconception.</p><p>CloudFront doesn’t work like:</p><pre>Upload website<br>       ↓<br>Copy entire website<br>       ↓<br>Every PoP<br>       ↓<br>     Done</pre><p>Instead, caching is generally <strong>on demand</strong>.</p><p>For example:</p><pre>User requests:<br>/images/logo.png</pre><p>If the object isn’t cached:</p><pre>User<br> ↓<br>CloudFront<br> ↓<br>MISS<br> ↓<br>Origin<br> ↓<br>logo.png<br> ↓<br>CloudFront Cache<br> ↓<br>User</pre><p>The object becomes available in the caching infrastructure according to the applicable caching configuration.</p><p>So you should think of CloudFront as a <strong>distributed caching system</strong>, not as a mechanism that pre-copies your entire application to every country.</p><h3>18. What Happens When Users Are in Different Countries?</h3><p>Suppose you have users in:</p><pre>🇨🇦 Canada<br>🇬🇧 United Kingdom<br>🇯🇵 Japan<br>🇦🇺 Australia<br>🇧🇩 Bangladesh</pre><p>They all request:</p><pre>/images/logo.png</pre><p>CloudFront can cache that object in the appropriate parts of its distributed infrastructure as requests arrive.</p><p>Conceptually:</p><pre>                      🇧🇩 Origin<br>                           │<br>              ┌────────────┴────────────┐<br>              │                         │<br>              ▼                         ▼<br>       Regional Cache             Regional Cache<br>              │                         │<br>       ┌──────┼──────┐          ┌──────┼──────┐<br>       ▼      ▼      ▼          ▼      ▼      ▼<br>      🇨🇦     🇺🇸     🇬🇧         🇯🇵     🇦🇺    ...<br>      PoP     PoP     PoP        PoP     PoP</pre><p>The exact internal hierarchy and placement are managed by AWS; this diagram is a conceptual model, not a literal map of CloudFront’s infrastructure.</p><h3>19. What If the Image Changes?</h3><p>Suppose your origin originally contains:</p><pre>logo.png → Version 1</pre><p>CloudFront caches it.</p><p>Later you update the origin:</p><pre>logo.png → Version 2</pre><p>CloudFront might still have Version 1 cached until its caching rules cause the object to expire or otherwise be refreshed.</p><p>This is why CDN applications commonly use <strong>versioned asset names</strong>:</p><pre>logo-v1.png<br>logo-v2.png</pre><p>or:</p><pre>app.abc123.js<br>app.def456.js</pre><p>You can also use CloudFront invalidations when appropriate.</p><h3>20. CloudFront + EKS</h3><p>For a modern AWS application, a common architecture is:</p><pre>                    INTERNET<br>                        │<br>                        ▼<br>                   Route 53<br>                        │<br>                        ▼<br>                   CloudFront<br>                        │<br>                 ┌──────┴──────┐<br>                 │             │<br>                WAF          Cache<br>                 │             │<br>                 └──────┬──────┘<br>                        │<br>                        ▼<br>                       ALB<br>                        │<br>                 ┌──────┴──────┐<br>                 │             │<br>              EKS Node      EKS Node<br>                 │             │<br>                 ▼             ▼<br>                Pod           Pod<br>                 │<br>                 ▼<br>                RDS</pre><p>For static content:</p><pre>CloudFront<br>    ↓<br>Cache<br>    ↓<br>User</pre><p>For dynamic APIs:</p><pre>CloudFront<br>    ↓<br>ALB<br>    ↓<br>EKS<br>    ↓<br>Application<br>    ↓<br>RDS</pre><p>CloudFront doesn’t replace the ALB.</p><p>The roles are different.</p><h3>21. CloudFront vs ALB</h3><p>This distinction is extremely important.</p><h3>CloudFront</h3><p>Primarily provides:</p><pre>Global distribution<br>Caching<br>Edge processing<br>TLS<br>Security integration<br>Origin acceleration</pre><h3>ALB</h3><p>Primarily provides:</p><pre>Load balancing<br>       ↓<br>EC2 / EKS targets</pre><p>Therefore:</p><pre>                  Internet<br>                      │<br>                      ▼<br>                 CloudFront<br>                  /       \<br>                 /         \<br>          Cache HIT       Cache MISS<br>             │               │<br>             ▼               ▼<br>           User             ALB<br>                              │<br>                              ▼<br>                             EKS</pre><p>CloudFront and ALB are complementary technologies, not interchangeable ones.</p><h3>22. CloudFront Can Also Protect a Private Origin</h3><p>Modern AWS architectures can use <strong>CloudFront VPC Origins</strong> to reach supported private resources.</p><p>Conceptually:</p><pre>Internet<br>    │<br>    ▼<br>CloudFront<br>    │<br>    │ AWS-managed connectivity<br>    ▼<br>Private VPC<br>    │<br>    ▼<br>Internal ALB<br>    │<br>    ▼<br>EKS</pre><p>This can allow an architecture where the ALB does not need to be directly exposed to the public Internet.</p><p>A strong architecture can therefore look like:</p><pre>              INTERNET<br>                  │<br>                  ▼<br>             CloudFront<br>                  │<br>               AWS WAF<br>                  │<br>                  ▼<br>            Private ALB<br>                  │<br>                  ▼<br>                 EKS<br>                  │<br>                  ▼<br>                 RDS</pre><p>The source material specifically discusses VPC Origins as an option for reaching supported private resources without requiring the origin to be publicly accessible.</p><h3>23. What Does “Deploying” Mean When Creating a CloudFront Distribution?</h3><p>Now let’s address another confusing part of CloudFront.</p><p>You create a distribution and see:</p><pre>Last modified: Deploying</pre><p>What is AWS actually doing?</p><p>AWS is <strong>not creating a new EC2 server for your distribution</strong>.</p><p>It is not:</p><pre>Create CloudFront<br>       ↓<br>Create EC2 servers<br>       ↓<br>Copy website to every PoP</pre><p>Instead, your CloudFront distribution contains configuration such as:</p><pre>Domain<br>Origin<br>Cache policies<br>Behaviors<br>TLS certificate<br>WAF association<br>Edge functions<br>Security settings</pre><p>AWS needs to validate and propagate/activate that configuration across the globally distributed CloudFront serving infrastructure.</p><p>Conceptually:</p><pre>             CloudFront Control Plane<br>                       │<br>                       ▼<br>                Distribution Config<br>                       │<br>          ┌────────────┼────────────┐<br>          ▼            ▼            ▼<br>        PoP A        PoP B        PoP C<br>          │            │            │<br>          ▼            ▼            ▼<br>       Config       Config       Config</pre><p>That’s what the <strong>Deploying</strong> state represents at a high level.</p><h3>24. Control Plane vs Data Plane</h3><p>This is perhaps the cleanest mental model for understanding CloudFront.</p><h3>Control Plane</h3><p>This is where you configure CloudFront:</p><pre>CloudFront Console<br>       │<br>       ▼<br>CloudFront Control Plane<br>       │<br>       ├── Distribution<br>       ├── Origins<br>       ├── Behaviors<br>       ├── Cache policies<br>       ├── Certificates<br>       ├── WAF<br>       └── Functions</pre><p>When you see:</p><pre>Deploying</pre><p>you’re primarily looking at a <strong>configuration rollout</strong>.</p><p>The source material explicitly distinguishes this control-plane rollout from the traffic-handling data plane.</p><h3>Data Plane</h3><p>Once deployed, actual user traffic flows through the CloudFront data plane:</p><pre>User<br> │<br> ▼<br>CloudFront Edge<br> │<br> ├── TLS<br> ├── WAF<br> ├── Functions<br> └── Cache lookup<br>          │<br>          ▼<br>       Origin</pre><p>The <strong>data plane handles the actual requests</strong>.</p><h3>25. The Complete CloudFront Request Lifecycle</h3><p>Now let’s combine everything.</p><pre>🇨🇦 USER<br>                            │<br>                            │ HTTPS<br>                            ▼<br>                    ┌───────────────┐<br>                    │ DNS / Route53 │<br>                    └───────┬───────┘<br>                            │<br>                            ▼<br>                    CloudFront Network<br>                            │<br>                       Anycast/BGP<br>                            │<br>                            ▼<br>                    ┌───────────────┐<br>                    │    Edge PoP   │<br>                    │               │<br>                    │ TLS           │<br>                    │ WAF           │<br>                    │ Functions     │<br>                    │ Cache         │<br>                    └───────┬───────┘<br>                            │<br>                       Cache Lookup<br>                         /       \<br>                       HIT       MISS<br>                        │          │<br>                        │          ▼<br>                        │    Regional Edge<br>                        │       Cache<br>                        │          │<br>                        │        MISS<br>                        │          │<br>                        │          ▼<br>                        │    Origin Shield<br>                        │          │<br>                        │          ▼<br>                        │        Origin<br>                        │          │<br>                        │     ┌────┼────┐<br>                        │     │    │    │<br>                        │     ▼    ▼    ▼<br>                        │    S3   ALB  API GW<br>                        │          │<br>                        │          ▼<br>                        │         EKS<br>                        │          │<br>                        │          ▼<br>                        │         RDS<br>                        │<br>                        ▼<br>                     🇨🇦 USER</pre><p>This captures the central request flow described throughout your source material.</p><h3>26. The Most Important Mental Model</h3><p>Don’t think:</p><pre>CloudFront =<br>Copy my entire website to every country</pre><p>Think:</p><pre>CloudFront =<br>    Global Network<br>          +<br>    Edge Locations<br>          +<br>    Distributed Cache<br>          +<br>    Intelligent Routing<br>          +<br>    Edge Processing<br>          +<br>    Security<br>          +<br>    Origin Connectivity</pre><p>Then the request becomes:</p><pre>USER<br>                  │<br>                  ▼<br>                 DNS<br>                  │<br>                  ▼<br>           CloudFront Edge<br>                  │<br>                  ▼<br>            Cache Lookup<br>             /          \<br>           HIT          MISS<br>            │             │<br>            │             ▼<br>            │       Regional Cache<br>            │             │<br>            │             ▼<br>            │       Origin Shield<br>            │             │<br>            │             ▼<br>            │          Origin<br>            │          Dhaka<br>            │             │<br>            └───────┬─────┘<br>                    ▼<br>                   USER</pre><h3>27. The Dhaka → Canada Example in One Picture</h3><p>Let’s reduce the entire explanation to one practical example.</p><h4>First request</h4><p>A Canadian user requests:</p><pre>/images/logo.png</pre><p>The object isn’t cached.</p><pre>🇨🇦 Canada<br>     │<br>     ▼<br>CloudFront Edge<br>     │<br>     ▼<br>Cache MISS<br>     │<br>     ▼<br>Regional Edge Cache<br>     │<br>     ▼<br>Origin Shield<br>     │<br>     ▼<br>🇧🇩 Dhaka Origin<br>     │<br>     ▼<br>ALB<br>     │<br>     ▼<br>EKS<br>     │<br>     ▼<br>Application<br>     │<br>     ▼<br>logo.png<br>     │<br>     ▼<br>CloudFront<br>     │<br>     ▼<br>🇨🇦 Canada</pre><p>CloudFront can cache the response.</p><h4>Second request</h4><p>Another Canadian user requests the same object:</p><pre>🇨🇦 Canada<br>     │<br>     ▼<br>CloudFront Edge<br>     │<br>     ▼<br>CACHE HIT<br>     │<br>     ▼<br>logo.png<br>     │<br>     ▼<br>🇨🇦 Canada</pre><p><strong>Dhaka doesn’t need to be involved.</strong></p><p>That’s the essence of CDN caching.</p><h3>28. What CloudFront Is Really Doing</h3><p>At a high level, CloudFront is performing three major jobs.</p><h4>Bring the request into AWS close to the user</h4><pre>User<br> ↓<br>Internet routing<br> ↓<br>CloudFront Edge</pre><h4>Try to satisfy the request from cache</h4><pre>Edge<br> ↓<br>Cache<br> ↓<br>HIT → User</pre><h4>If necessary, retrieve the content from the origin</h4><pre>Edge<br> ↓<br>Regional Cache<br> ↓<br>Origin Shield<br> ↓<br>Origin<br> ↓<br>CloudFront<br> ↓<br>User</pre><p>That is why CloudFront can improve:</p><ul><li>latency</li><li>origin scalability</li><li>bandwidth utilization</li><li>availability characteristics</li><li>security posture</li><li>global application performance</li></ul><h3>Conclusion</h3><p>CloudFront is much more than a simple cache.</p><p>It is a <strong>globally distributed edge network</strong> sitting between your users and your application.</p><p>When a user in Canada requests content from an application hosted in Dhaka, the request doesn’t necessarily travel to Dhaka every time.</p><p>Instead:</p><pre>                  🇨🇦 USER<br>                     │<br>                     ▼<br>                DNS / Routing<br>                     │<br>                     ▼<br>              CloudFront Edge<br>                     │<br>              ┌──────┴──────┐<br>              │             │<br>             HIT           MISS<br>              │             │<br>              ▼             ▼<br>             User      Regional Cache<br>                            │<br>                            ▼<br>                       Origin Shield<br>                            │<br>                            ▼<br>                       🇧🇩 Origin<br>                            │<br>                            ▼<br>                           ALB<br>                            │<br>                            ▼<br>                           EKS<br>                            │<br>                            ▼<br>                         RDS/API</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4d2a995f53e6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Hardware Virtualization Deep Dive: What Really Happens Inside the CPU, Memory, and IOMMU?]]></title>
            <link>https://arn-ob.medium.com/hardware-virtualization-deep-dive-what-really-happens-inside-the-cpu-memory-and-iommu-8bb510540243?source=rss-c7040ca07397------2</link>
            <guid isPermaLink="false">https://medium.com/p/8bb510540243</guid>
            <category><![CDATA[memory-management]]></category>
            <category><![CDATA[hardware]]></category>
            <category><![CDATA[virtualization]]></category>
            <category><![CDATA[cpu]]></category>
            <dc:creator><![CDATA[Arnob]]></dc:creator>
            <pubDate>Tue, 11 Aug 2026 08:41:01 GMT</pubDate>
            <atom:updated>2026-08-11T08:41:01.104Z</atom:updated>
            <content:encoded><![CDATA[<p>When we create a virtual machine, it looks deceptively simple:</p><pre>Physical Server<br>      │<br>      ▼<br>   Hypervisor<br>      │<br> ┌────┴────┐<br> ▼         ▼<br> VM 1      VM 2<br>Linux     Windows</pre><p>We allocate some CPU, RAM, disk, and network to a VM, install an operating system, and everything appears to behave like a physical computer.</p><p>But there is a much deeper question:</p><blockquote><strong><em>How can multiple operating systems simultaneously believe they own the same physical CPU, memory, and hardware?</em></strong></blockquote><p>A VM has its own kernel. Its kernel thinks it controls CPU registers, physical memory, interrupts, disks, and network devices.</p><p>But it doesn’t.</p><p>Behind the scenes, the CPU and hardware virtualization technologies create a carefully controlled boundary between the <strong>guest operating system</strong>, the <strong>hypervisor</strong>, and the <strong>real hardware</strong>.</p><p>This article goes deep into that architecture.</p><h3>1. What Is Hardware Virtualization?</h3><p>Hardware virtualization is the process of running multiple independent virtual machines on a single physical computer.</p><p>The physical machine provides:</p><ul><li>CPU</li><li>RAM</li><li>Storage</li><li>Network interfaces</li><li>PCIe devices</li></ul><p>The hypervisor divides and manages those resources among multiple VMs.</p><p>A simplified architecture looks like this:</p><pre>┌───────────────────────────────────────────────┐<br>│                 VM 1                          │<br>│        Guest OS + Applications                │<br>├───────────────────────────────────────────────┤<br>│                 VM 2                          │<br>│        Guest OS + Applications                │<br>├───────────────────────────────────────────────┤<br>│                 VM 3                          │<br>│        Guest OS + Applications                │<br>├───────────────────────────────────────────────┤<br>│                  Hypervisor                   │<br>│       CPU / Memory / I/O Management           │<br>├───────────────────────────────────────────────┤<br>│              Physical Hardware               │<br>│ CPU │ RAM │ Storage │ NIC │ PCIe Devices     │<br>└───────────────────────────────────────────────┘</pre><p>The important part is that the guest OS does <strong>not</strong> directly control the physical hardware.</p><p>The hypervisor provides an abstraction layer.</p><h3>2. The Hypervisor Is More Than a Resource Allocator</h3><p>It is common to think of a hypervisor as simply:</p><blockquote><em>“Software that creates VMs.”</em></blockquote><p>That’s incomplete.</p><p>A hypervisor must enforce isolation.</p><p>For example, suppose:</p><pre>VM 1 → 4 GB RAM<br>VM 2 → 8 GB RAM</pre><p>VM 1 must never be able to access VM 2’s memory.</p><p>Likewise:</p><pre>VM 1 → CPU<br>VM 2 → CPU</pre><p>VM 1 must not be able to execute privileged operations that compromise the hypervisor or another VM.</p><p>Therefore, virtualization requires cooperation between:</p><ul><li>CPU hardware</li><li>Memory Management Unit</li><li>Hypervisor</li><li>IOMMU</li><li>PCIe subsystem</li><li>Guest operating system</li></ul><p>Modern CPUs provide hardware virtualization extensions such as:</p><ul><li>Intel VT-x</li><li>AMD-V</li></ul><p>For device virtualization and DMA isolation, we also have:</p><ul><li>Intel VT-d</li><li>AMD-Vi</li></ul><h3>3. Type 1 vs Type 2 Hypervisors</h3><p>There are two major hypervisor architectures.</p><h3>Type 1 — Bare Metal</h3><p>The hypervisor runs directly on physical hardware.</p><pre>Physical Hardware<br>       │<br>       ▼<br>   Hypervisor<br>       │<br> ┌─────┼─────┐<br> ▼     ▼     ▼<br>VM1   VM2   VM3</pre><p>Examples include:</p><ul><li>VMware ESXi</li><li>Microsoft Hyper-V</li><li>KVM-based virtualization</li></ul><p>The hypervisor has direct control over CPU and memory resources.</p><h3>Type 2 — Hosted</h3><p>A traditional operating system exists underneath the virtualization software.</p><pre>Physical Hardware<br>       │<br>       ▼<br>    Host OS<br>       │<br>       ▼<br>  Virtualization<br>    Software<br>       │<br> ┌─────┼─────┐<br> ▼     ▼     ▼<br>VM1   VM2   VM3</pre><p>Examples include:</p><ul><li>VirtualBox</li><li>VMware Workstation</li></ul><p>The extra host OS layer can introduce additional overhead.</p><h3>4. The CPU Privilege Problem</h3><p>To understand hardware virtualization, we need to understand CPU privilege levels.</p><p>Traditional x86 systems have privilege rings.</p><p>The two most important are:</p><pre>Ring 0 → Kernel<br>Ring 3 → User applications</pre><p>Normally:</p><pre>Application<br>    │<br>    ▼<br>Ring 3<br>    │<br>    ▼<br>Operating System Kernel<br>    │<br>    ▼<br>Ring 0<br>    │<br>    ▼<br>Hardware</pre><p>The operating system kernel runs in Ring 0 because it needs privileged access to hardware.</p><p>But now imagine two VMs:</p><pre>VM 1 Kernel → Ring 0<br>VM 2 Kernel → Ring 0<br>Hypervisor  → Ring 0</pre><p>We cannot simply allow all of them to control the physical hardware.</p><p>This is one of the fundamental problems virtualization had to solve.</p><h3>5. Intel VT-x and AMD-V</h3><p>Modern CPUs introduced hardware virtualization modes.</p><p>For Intel VT-x, the CPU provides:</p><pre>VMX Root Operation<br>VMX Non-Root Operation</pre><p>Conceptually:</p><pre>Physical CPU<br>                  │<br>          ┌───────┴────────┐<br>          │                │<br>      VMX Root        VMX Non-Root<br>          │                │<br>     Hypervisor         Guest VM</pre><p>The hypervisor runs in VMX Root mode.</p><p>Guest operating systems run in VMX Non-Root mode.</p><p>A guest OS can still behave as though it has Ring 0 privileges, but the CPU controls what happens when it attempts certain privileged operations.</p><p>This is where <strong>VM Exit</strong> becomes important.</p><h3>6. What Is a VM Exit?</h3><p>Suppose a guest operating system executes an operation that requires hypervisor intervention.</p><p>The CPU detects it.</p><p>Instead of allowing the guest to directly manipulate protected hardware state:</p><pre>Guest OS<br>   │<br>   │ privileged operation<br>   ▼<br>CPU<br>   │<br>   │ VM Exit<br>   ▼<br>Hypervisor</pre><p>The CPU saves the guest execution state and transfers control to the hypervisor.</p><p>The hypervisor handles the event and eventually resumes the guest using a <strong>VM Entry</strong>.</p><p>Conceptually:</p><pre>Guest<br>  │<br>  │ privileged event<br>  ▼<br>VM Exit<br>  │<br>  ▼<br>Hypervisor<br>  │<br>  │ handle event<br>  ▼<br>VM Entry<br>  │<br>  ▼<br>Guest</pre><p>This hardware mechanism is one of the foundations of modern virtualization.</p><h3>7. The Hardest Part: Memory Virtualization</h3><p>CPU virtualization is only one part of the problem.</p><p>Memory is even more interesting.</p><p>In a normal operating system, there are two major address spaces:</p><pre>Virtual Address<br>      │<br>      ▼<br>Physical Address</pre><p>For example:</p><pre>Application<br>    │<br>    │ Virtual Address<br>    ▼<br>Guest/Host OS Page Tables<br>    │<br>    │ Physical Address<br>    ▼<br>RAM</pre><p>Virtualization introduces another layer.</p><p>We now have:</p><pre>Virtual Address<br>      │<br>      ▼<br>Guest Physical Address<br>      │<br>      ▼<br>Host Physical Address</pre><p>These are commonly represented as:</p><pre>VA  = Virtual Address<br>GPA = Guest Physical Address<br>HPA = Host Physical Address</pre><p>Therefore:</p><pre>Application<br>    │<br>    │ VA<br>    ▼<br>Guest Page Tables<br>    │<br>    │ GPA<br>    ▼<br>Hypervisor Translation<br>    │<br>    │ HPA<br>    ▼<br>Physical RAM</pre><p>This is the foundation of memory virtualization.</p><h3>8. Why Do We Need GPA?</h3><p>Consider a VM configured with:</p><pre>8 GB RAM</pre><p>The guest OS believes that it has physical memory:</p><pre>GPA:</pre><pre>0x00000000<br>      ↓<br>0x1FFFFFFF</pre><p>But that address range isn’t necessarily the real RAM location.</p><p>The hypervisor could map it anywhere:</p><pre>Guest:</pre><pre>GPA 0x100000<br>       │<br>       ▼<br>Host:</pre><pre>HPA 0x840000000</pre><p>The guest doesn’t need to know the real physical address.</p><p>That is what allows multiple VMs to safely share physical RAM.</p><h3>9. Shadow Page Tables: The Old Solution</h3><p>Before modern hardware-assisted memory virtualization, hypervisors could use <strong>Shadow Page Tables</strong>.</p><p>The guest maintains:</p><pre>VA → GPA</pre><p>The hypervisor maintains:</p><pre>GPA → HPA</pre><p>The hypervisor combines them into:</p><pre>VA → HPA</pre><p>Conceptually:</p><pre>Guest Page Table</pre><pre>VA ───────────────► GPA<br>                     │<br>                     ▼<br>               Hypervisor<br>                     │<br>                     ▼<br>                    HPA</pre><p>The hypervisor creates a shadow page table that directly represents:</p><pre>VA → HPA</pre><p>This allows the CPU to use a single mapping during execution.</p><p>However, there is a major problem.</p><h3>10. The Problem With Shadow Page Tables</h3><p>Guest operating systems constantly modify their page tables.</p><p>For example:</p><pre>Process starts<br>    ↓<br>Memory allocated<br>    ↓<br>Page table modified<br>    ↓<br>Process exits<br>    ↓<br>Page table modified</pre><p>The hypervisor has to track these changes.</p><p>It may write-protect guest page-table memory.</p><p>When the guest attempts to modify the table:</p><pre>Guest writes page table<br>        │<br>        ▼<br>Page fault<br>        │<br>        ▼<br>VM Exit<br>        │<br>        ▼<br>Hypervisor<br>        │<br>        ▼<br>Update Shadow Page Table<br>        │<br>        ▼<br>VM Entry<br>        │<br>        ▼<br>Guest resumes</pre><p>Doing this repeatedly introduces significant overhead.</p><p>Hardware-assisted virtualization was designed to solve this problem.</p><h3>11. Second Level Address Translation — SLAT</h3><p>Modern CPUs provide hardware support for the second address translation layer.</p><p>Intel calls it:</p><blockquote><strong><em>Extended Page Tables — EPT</em></strong></blockquote><p>AMD calls it:</p><blockquote><strong><em>Nested Page Tables — NPT</em></strong></blockquote><p>Together, these are commonly referred to as:</p><blockquote><strong><em>Second Level Address Translation — SLAT</em></strong></blockquote><p>The architecture becomes:</p><pre>Guest Virtual Address<br>          │<br>          ▼<br>   Guest Page Tables<br>          │<br>          ▼<br>Guest Physical Address<br>          │<br>          ▼<br>       EPT/NPT<br>          │<br>          ▼<br>Host Physical Address<br>          │<br>          ▼<br>       Physical RAM</pre><p>The important change is:</p><blockquote><em>The CPU hardware performs the second translation.</em></blockquote><p>The hypervisor doesn’t need to software-emulate every memory translation.</p><h3>12. EPT and Two-Dimensional Page Translation</h3><p>Let’s look deeper.</p><p>A traditional x86–64 system may use a four-level page table:</p><pre>PML4<br> │<br> ▼<br>PDPT<br> │<br> ▼<br>PD<br> │<br> ▼<br>PT<br> │<br> ▼<br>Physical Page</pre><p>The guest has its own page tables.</p><p>The hypervisor also maintains EPT tables.</p><p>Therefore, hardware effectively performs nested translation:</p><pre>VA<br> │<br> ▼<br>Guest Page Tables<br> │<br> ▼<br>GPA<br> │<br> ▼<br>EPT Page Tables<br> │<br> ▼<br>HPA</pre><p>This is sometimes called <strong>two-dimensional paging</strong>.</p><h3>13. The 24-Memory-Access Problem</h3><p>There is an interesting mathematical consequence.</p><p>With a four-level guest page table and four-level EPT, a worst-case nested page walk can involve many memory accesses.</p><p>A commonly presented calculation is:</p><pre>(4 guest levels × 4 EPT levels) + 4 final EPT lookups<br>= 16 + 4<br>= 20</pre><p>Depending on the exact page-walk model and which structures are cached, discussions may present different counts.</p><p>The important point is not the exact number.</p><p>The important point is:</p><blockquote><strong><em>Nested page walks can be expensive if every translation requires walking multiple levels of page tables.</em></strong></blockquote><p>So CPUs use another critical optimization.</p><h3>14. The TLB Saves the Day</h3><p>The CPU contains a cache called the:</p><blockquote><strong><em>Translation Lookaside Buffer — TLB</em></strong></blockquote><p>Instead of repeatedly walking page tables, the CPU caches recently used address translations.</p><p>Conceptually:</p><pre>Virtual Address<br>      │<br>      ▼<br>     TLB<br>      │<br>      ├── Hit ──► Physical Address<br>      │<br>      └── Miss<br>            │<br>            ▼<br>        Page Walk</pre><p>With virtualization, the TLB becomes even more important.</p><p>Without caching, nested page-table translation would be extremely expensive.</p><h3>15. VPID: Keeping VM Translations Separate</h3><p>There is another problem.</p><p>Suppose VM 1 has:</p><pre>VA 0x1000 → HPA A</pre><p>VM 2 might have:</p><pre>VA 0x1000 → HPA B</pre><p>The same virtual address can mean completely different physical memory depending on the VM.</p><p>If the CPU simply reused the TLB entry, it could become a security problem.</p><p>One solution is to tag translations.</p><p>Intel provides:</p><blockquote><strong><em>VPID — Virtual Processor ID</em></strong></blockquote><p>Conceptually:</p><pre>TLB Entry</pre><pre>VM1 + VA → HPA<br>VM2 + VA → HPA</pre><p>The CPU knows which VM’s translation context is currently active.</p><p>This reduces the need to completely flush the TLB during every VM transition.</p><h3>16. Guest Page Fault vs EPT Violation</h3><p>This distinction is extremely important.</p><h3>Guest Page Fault</h3><p>A guest application accesses memory that the guest OS hasn’t mapped.</p><pre>Application<br>    │<br>    ▼<br>Guest Page Fault<br>    │<br>    ▼<br>Guest Kernel</pre><p>The hypervisor may never need to become involved.</p><h3>EPT Violation</h3><p>The guest accesses a GPA that isn’t currently permitted or mapped by the hypervisor’s EPT configuration.</p><pre>Guest<br>  │<br>  ▼<br>GPA<br>  │<br>  ▼<br>EPT<br>  │<br>  X<br>EPT Violation<br>  │<br>  ▼<br>VM Exit<br>  │<br>  ▼<br>Hypervisor</pre><p>This is a hardware-enforced boundary between guest memory and host memory.</p><h3>17. Memory Overcommit</h3><p>Physical RAM is expensive.</p><p>Suppose a server has:</p><pre>128 GB physical RAM</pre><p>You might provision:</p><pre>VM1 = 64 GB<br>VM2 = 64 GB<br>VM3 = 64 GB</pre><p>That’s:</p><pre>192 GB virtual memory</pre><p>even though the host only has:</p><pre>128 GB physical RAM</pre><p>This is called <strong>memory overcommitment</strong>.</p><p>Hypervisors can use several techniques to manage this.</p><h3>18. Memory Ballooning</h3><p>One important mechanism is:</p><blockquote><strong><em>Memory Ballooning</em></strong></blockquote><p>A balloon driver runs inside the guest OS.</p><p>When the host experiences memory pressure:</p><pre>Host<br> │<br> │ Memory pressure<br> ▼<br>Hypervisor<br> │<br> ▼<br>Balloon Driver<br> │<br> ▼<br>Guest OS</pre><p>The balloon driver requests memory from the guest.</p><p>The guest OS gives up pages that it can reclaim.</p><p>The hypervisor can then reuse the underlying physical RAM for another VM.</p><p>Conceptually:</p><pre>VM1<br>64 GB allocated<br>     │<br>     │ Balloon inflates<br>     ▼<br>VM1 gives back unused pages<br>     │<br>     ▼<br>Hypervisor<br>     │<br>     ▼<br>VM2 receives physical RAM</pre><p>When pressure decreases, the balloon can deflate.</p><h3>19. Transparent Page Sharing</h3><p>Another technique is memory deduplication.</p><p>Suppose ten Linux VMs contain identical memory pages.</p><p>Instead of storing:</p><pre>Page A<br>Page A<br>Page A<br>Page A<br>...</pre><p>the hypervisor can potentially store one physical copy:</p><pre>┌── VM1<br>             │<br>             ├── VM2<br>             │<br>Shared Page ─┼── VM3<br>             │<br>             ├── VM4<br>             │<br>             └── VM5</pre><p>The shared page is typically marked read-only.</p><p>If one VM modifies it:</p><pre>VM1 writes<br>   │<br>   ▼<br>Copy-on-Write<br>   │<br>   ▼<br>New private page</pre><p>Other VMs continue using the original page.</p><h3>20. But What Happens Inside the Actual Hardware?</h3><p>Now we can go one level deeper.</p><p>The CPU’s <strong>Memory Management Unit — MMU</strong> is responsible for translating memory addresses.</p><p>A simplified non-virtualized system looks like:</p><pre>CPU<br> │<br> │ Virtual Address<br> ▼<br>MMU<br> │<br> ▼<br>Page Tables<br> │<br> ▼<br>Physical Address<br> │<br> ▼<br>RAM</pre><p>With virtualization:</p><pre>CPU<br> │<br> │ Guest Virtual Address<br> ▼<br>MMU<br> │<br> ▼<br>Guest Page Tables<br> │<br> │ GPA<br> ▼<br>EPT/NPT<br> │<br> │ HPA<br> ▼<br>RAM</pre><p>This is not merely software.</p><p>The CPU contains hardware specifically designed to perform these operations.</p><h3>21. The Dual-Pointer Concept</h3><p>One way to understand hardware-assisted virtualization is through two page-table roots.</p><p>The guest maintains its own page-table root, conceptually represented by:</p><pre>gCR3</pre><p>The hypervisor controls the EPT root:</p><pre>EPTP</pre><p>So conceptually:</p><pre>CPU MMU<br>                │<br>        ┌───────┴────────┐<br>        │                │<br>      gCR3              EPTP<br>        │                │<br>        ▼                ▼<br> Guest Page Tables    EPT Tables<br>        │                │<br>        ▼                ▼<br>       GPA              HPA</pre><p>The CPU hardware combines these translation mechanisms.</p><h3>22. What Happens During a Memory Access?</h3><p>Imagine an application inside a VM executes:</p><pre>load 0x7f123456</pre><p>The CPU receives a Guest Virtual Address.</p><p>First:</p><pre>GVA<br> │<br> ▼<br>Guest Page Tables<br> │<br> ▼<br>GPA</pre><p>Then the hardware translates:</p><pre>GPA<br> │<br> ▼<br>EPT<br> │<br> ▼<br>HPA</pre><p>Finally:</p><pre>HPA<br> │<br> ▼<br>Physical RAM</pre><p>The complete conceptual pipeline is:</p><pre>Application<br>     │<br>     ▼<br>Guest Virtual Address<br>     │<br>     ▼<br>Guest Page Tables<br>     │<br>     ▼<br>Guest Physical Address<br>     │<br>     ▼<br>EPT/NPT<br>     │<br>     ▼<br>Host Physical Address<br>     │<br>     ▼<br>DRAM</pre><p>And much of this happens without a software function executing for every memory access.</p><h3>23. What About GPUs and Network Cards?</h3><p>CPU memory virtualization isn’t enough.</p><p>Consider a GPU.</p><p>A GPU can perform:</p><blockquote><em>DMA — Direct Memory Access</em></blockquote><p>The device can read or write RAM without asking the CPU to execute every operation.</p><p>That creates a major security problem.</p><p>Suppose a VM controls a physical GPU.</p><p>The guest tells the GPU:</p><pre>Write data to GPA 0x100000</pre><p>But the physical GPU doesn’t naturally understand the VM’s memory isolation model.</p><p>If the device could blindly write to physical RAM, it could potentially overwrite:</p><pre>Host memory<br>VM1 memory<br>VM2 memory<br>Hypervisor memory</pre><p>This is where the <strong>IOMMU</strong> becomes critical.</p><h3>24. What Is an IOMMU?</h3><p>IOMMU stands for:</p><blockquote><strong><em>Input-Output Memory Management Unit</em></strong></blockquote><p>The CPU MMU manages memory accesses generated by CPUs.</p><p>The IOMMU manages memory accesses generated by I/O devices.</p><p>So:</p><pre>CPU<br> │<br> ▼<br>MMU<br> │<br> ▼<br>RAM</pre><p>while:</p><pre>GPU / NIC / NVMe<br>       │<br>       ▼<br>     IOMMU<br>       │<br>       ▼<br>      RAM</pre><p>The IOMMU provides address translation and access control for device DMA.</p><h3>25. IOMMU as a Hardware Firewall</h3><p>Think of the IOMMU as a firewall for DMA.</p><p>Without an IOMMU:</p><pre>GPU<br> │<br> │ DMA<br> ▼<br>Physical RAM</pre><p>The device could potentially access memory it shouldn’t.</p><p>With an IOMMU:</p><pre>GPU<br> │<br> │ DMA Request<br> ▼<br>IOMMU<br> │<br> ├── Allowed ──► RAM<br> │<br> └── Denied ───► Block</pre><p>This makes PCIe passthrough possible while preserving memory isolation.</p><h3>26. Intel VT-d and AMD-Vi</h3><p>Intel’s IOMMU technology is called:</p><blockquote><strong><em>Intel VT-d</em></strong></blockquote><p>AMD’s equivalent is commonly called:</p><blockquote><strong><em>AMD-Vi</em></strong></blockquote><p>They solve essentially the same class of problem:</p><pre>PCIe Device<br>     │<br>     ▼<br>IOMMU<br>     │<br>     ▼<br>I/O Page Tables<br>     │<br>     ▼<br>Physical RAM</pre><p>The hardware tracks which device is allowed to access which memory.</p><h3>27. How Does the IOMMU Identify a Device?</h3><p>PCIe devices have identifiers.</p><p>A common identifier is:</p><pre>BDF<br>Bus<br>Device<br>Function</pre><p>For example:</p><pre>0000:03:00.0</pre><p>The IOMMU uses device identity to determine which translation and permission structures apply to that device.</p><h3>28. Intel VT-d Architecture</h3><p>Intel VT-d uses structures including:</p><pre>Requester ID<br>     │<br>     ▼<br>Root Table<br>     │<br>     ▼<br>Context Entry<br>     │<br>     ▼<br>I/O Page Tables<br>     │<br>     ▼<br>Physical Memory</pre><p>The device’s identity determines which context and page tables are used.</p><h3>29. AMD-Vi Architecture</h3><p>AMD-Vi uses a device table structure.</p><p>Conceptually:</p><pre>Requester ID<br>      │<br>      ▼<br>Device Table<br>      │<br>      ▼<br>I/O Page Tables<br>      │<br>      ▼<br>Physical Memory</pre><p>The implementation details differ from Intel VT-d, but the fundamental goal is similar:</p><blockquote><strong><em>Translate and restrict DMA requests in hardware.</em></strong></blockquote><h3>30. Interrupt Remapping</h3><p>Memory isn’t the only problem.</p><p>Devices also need to interrupt CPUs.</p><p>For example:</p><pre>NIC receives packet<br>        │<br>        ▼<br>NIC generates interrupt<br>        │<br>        ▼<br>CPU</pre><p>With device passthrough, the interrupt must safely reach the correct VM.</p><p>IOMMU-related virtualization hardware supports <strong>interrupt remapping</strong> to control how device interrupts are delivered.</p><p>Modern systems can optimize this further so that interrupts reach a virtual CPU efficiently without unnecessary hypervisor involvement.</p><h3>31. IOTLB</h3><p>The CPU has:</p><pre>TLB</pre><p>The IOMMU has a similar caching concept:</p><pre>IOTLB</pre><p>The IOTLB caches recent I/O address translations.</p><p>Without it, a high-speed NIC repeatedly performing DMA could require repeated page-table walks.</p><p>With IOTLB caching:</p><pre>Device DMA<br>    │<br>    ▼<br>  IOTLB<br>    │<br>    ├── Hit → Fast translation<br>    │<br>    └── Miss<br>          │<br>          ▼<br>     I/O Page Walk</pre><p>This improves DMA performance.</p><h3>32. Is VT-d or AMD-Vi Software?</h3><p>No.</p><p>This distinction is extremely important.</p><p>The software configures the IOMMU.</p><p>But the actual DMA translation and enforcement happen in hardware.</p><p>Think about the responsibilities this way:</p><pre>SOFTWARE<br>                │<br>        Configure mappings<br>                │<br>                ▼<br>        ┌───────────────┐<br>        │     IOMMU     │<br>        │   HARDWARE    │<br>        └───────────────┘<br>                │<br>       DMA translation<br>       Permission checks<br>       Address remapping<br>       Interrupt handling<br>                │<br>                ▼<br>             HARDWARE</pre><p>The hypervisor creates the rules.</p><p>The hardware enforces the rules.</p><h3>33. What Happens When a VM Starts?</h3><p>A simplified VM startup sequence looks like this:</p><pre>VM creation<br>    │<br>    ▼<br>Hypervisor allocates VM resources<br>    │<br>    ▼<br>Guest memory mappings created<br>    │<br>    ▼<br>EPT/NPT configured<br>    │<br>    ▼<br>Virtual CPU state configured<br>    │<br>    ▼<br>Virtual devices configured<br>    │<br>    ▼<br>IOMMU mappings configured<br>    │<br>    ▼<br>VM starts</pre><p>For PCIe passthrough:</p><pre>Physical Device<br>       │<br>       ▼<br>IOMMU Group<br>       │<br>       ▼<br>VFIO / Hypervisor<br>       │<br>       ▼<br>VM</pre><p>The device is isolated from other devices and mapped to the guest.</p><h3>34. Putting Everything Together</h3><p>We can now draw the complete architecture.</p><pre>                       ┌───────────────────────┐<br>                       │     VM 1              │<br>                       │                       │<br>                       │ Applications          │<br>                       │ Guest Kernel          │<br>                       └──────────┬────────────┘<br>                                  │<br>                           Guest Virtual<br>                              Address<br>                                  │<br>                                  ▼<br>                       ┌───────────────────────┐<br>                       │   Guest Page Tables   │<br>                       └──────────┬────────────┘<br>                                  │<br>                                  ▼<br>                              GPA<br>                                  │<br>                                  ▼<br>                       ┌───────────────────────┐<br>                       │      EPT / NPT        │<br>                       │   Hardware MMU        │<br>                       └──────────┬────────────┘<br>                                  │<br>                                  ▼<br>                              HPA<br>                                  │<br>                                  ▼<br>                       ┌───────────────────────┐<br>                       │     Physical RAM      │<br>                       └───────────────────────┘<br>          PCIe DEVICE<br>          ┌───────────┐<br>          │ GPU / NIC │<br>          └─────┬─────┘<br>                │<br>               DMA<br>                │<br>                ▼<br>        ┌─────────────────┐<br>        │      IOMMU      │<br>        │  VT-d / AMD-Vi  │<br>        └────────┬────────┘<br>                 │<br>            I/O Translation<br>                 │<br>                 ▼<br>          Physical Memory<br></pre><p>Now we can see that virtualization isn’t simply:</p><pre>VM = Process</pre><p>It is a coordinated system involving multiple hardware subsystems.</p><h3>35. The Complete Hardware Virtualization Stack</h3><p>At the highest level:</p><pre>┌────────────────────────────────────────────┐<br>│             Applications                   │<br>├────────────────────────────────────────────┤<br>│             Guest OS                       │<br>├────────────────────────────────────────────┤<br>│        Virtual CPU / Devices               │<br>├────────────────────────────────────────────┤<br>│             Hypervisor                     │<br>├────────────────────────────────────────────┤<br>│ VMX / AMD-V │ EPT / NPT │ IOMMU            │<br>├────────────────────────────────────────────┤<br>│       CPU MMU │ TLB │ IOTLB                │<br>├────────────────────────────────────────────┤<br>│        CPU │ RAM │ PCIe │ NIC │ GPU        │<br>└────────────────────────────────────────────┘</pre><p>Each layer has a specific responsibility.</p><h3>CPU Virtualization</h3><p>Provides:</p><pre>VMX Root / Non-Root<br>VM Exit / VM Entry<br>Virtual CPU state</pre><h3>Memory Virtualization</h3><p>Provides:</p><pre>VA → GPA → HPA</pre><p>using:</p><pre>EPT / NPT</pre><h3>Translation Caching</h3><p>Provides:</p><pre>TLB<br>VPID</pre><p>to avoid expensive repeated page walks.</p><h3>I/O Virtualization</h3><p>Provides:</p><pre>IOMMU<br>VT-d<br>AMD-Vi<br>IOTLB<br>Interrupt Remapping</pre><p>to isolate devices and DMA.</p><h3>36. Why This Matters for Proxmox, KVM, and Cloud Infrastructure</h3><p>When you use platforms such as Proxmox or KVM, you are interacting with a much deeper hardware architecture.</p><p>For example:</p><pre>Proxmox<br>   │<br>   ▼<br>Linux Kernel<br>   │<br>   ▼<br>KVM<br>   │<br>   ▼<br>Intel VT-x / AMD-V<br>   │<br>   ├── CPU virtualization<br>   │<br>   ├── EPT / NPT<br>   │<br>   └── VM Exit / Entry</pre><p>For PCIe passthrough:</p><pre>Proxmox<br>   │<br>   ▼<br>VFIO<br>   │<br>   ▼<br>IOMMU<br>   │<br>   ├── Intel VT-d<br>   │<br>   └── AMD-Vi<br>        │<br>        ▼<br>      PCIe<br>        │<br>        ▼<br>       GPU</pre><p>This is why enabling virtualization in BIOS is only the beginning.</p><p>The real virtualization architecture extends from the BIOS and hypervisor all the way down into CPU and chipset hardware.</p><h3>37. The Most Important Mental Model</h3><p>If you remember only one diagram from this article, remember this:</p><pre>VM<br>                  │<br>          Guest Virtual Address<br>                  │<br>                  ▼<br>          Guest Page Tables<br>                  │<br>                  ▼<br>          Guest Physical Address<br>                  │<br>                  ▼<br>             EPT / NPT<br>                  │<br>                  ▼<br>          Host Physical Address<br>                  │<br>                  ▼<br>                RAM</pre><p>And for devices:</p><pre>GPU / NIC<br>                  │<br>                 DMA<br>                  │<br>                  ▼<br>                IOMMU<br>                  │<br>          I/O Page Tables<br>                  │<br>                  ▼<br>             Physical RAM</pre><p>The CPU MMU protects CPU-generated memory accesses.</p><p>The IOMMU protects device-generated DMA accesses.</p><p>Together, they form a critical part of modern VM isolation.</p><h3>38. Final Thoughts</h3><p>Virtual machines look like software objects:</p><pre>VM = configuration + disk + RAM + CPU</pre><p>But underneath that abstraction is a sophisticated hardware system.</p><p>A modern VM depends on:</p><ul><li>CPU privilege separation</li><li>VMX Root / Non-Root execution</li><li>VM Exit / VM Entry</li><li>Guest page tables</li><li>EPT / NPT</li><li>TLB caching</li><li>VPID</li><li>Memory overcommit</li><li>Ballooning</li><li>Copy-on-Write</li><li>IOMMU</li><li>Intel VT-d</li><li>AMD-Vi</li><li>DMA remapping</li><li>Interrupt remapping</li><li>IOTLB</li></ul><p>The key idea is simple:</p><blockquote><strong><em>The hypervisor defines the virtualization rules, but modern CPU and I/O hardware enforce many of those rules at silicon speed.</em></strong></blockquote><p>That is what makes it possible for multiple operating systems to safely share one physical server while still behaving as if each owns a separate machine.</p><p>And once you understand this architecture, technologies such as <strong>KVM, Proxmox, VMware ESXi, PCIe passthrough, GPU virtualization, SR-IOV, and cloud VM infrastructure</strong> become much easier to understand.</p><p>The next interesting layer is even deeper:</p><p><strong>How does a CPU physically perform VM Entry and VM Exit? How does the hardware save and restore VM state? And how do Huge Pages reduce the cost of nested page translation?</strong></p><p>Those are the mechanisms that take us from the virtualization architecture into the actual CPU microarchitecture.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8bb510540243" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why Doesn’t OpenStack Have Its Own Operating System Like Proxmox?]]></title>
            <link>https://arn-ob.medium.com/why-doesnt-openstack-have-its-own-operating-system-like-proxmox-5cd40e3976b5?source=rss-c7040ca07397------2</link>
            <guid isPermaLink="false">https://medium.com/p/5cd40e3976b5</guid>
            <category><![CDATA[openstack]]></category>
            <category><![CDATA[operating-systems]]></category>
            <category><![CDATA[openstack-private-cloud]]></category>
            <category><![CDATA[proxmox]]></category>
            <dc:creator><![CDATA[Arnob]]></dc:creator>
            <pubDate>Mon, 10 Aug 2026 12:28:56 GMT</pubDate>
            <atom:updated>2026-08-10T12:28:56.713Z</atom:updated>
            <content:encoded><![CDATA[<p>When you first start working with OpenStack, one question naturally comes to mind:</p><blockquote><strong><em>Why doesn’t OpenStack provide its own operating system like Proxmox VE?</em></strong></blockquote><p>With Proxmox, you download an ISO, install it directly on a physical server, and you immediately have a complete virtualization platform.</p><p>With OpenStack, things are different.</p><p>You normally start with an operating system such as Ubuntu, Debian, RHEL, or another supported Linux distribution and then deploy OpenStack services on top of it.</p><p>This raises an interesting question:</p><blockquote><strong><em>Wouldn’t an OpenStack-specific OS provide better performance, simpler deployment, and better internal communication?</em></strong></blockquote><p>The answer is more complicated than it first appears.</p><h3>Proxmox and OpenStack Have Different Philosophies</h3><p>The first thing to understand is that <strong>Proxmox and OpenStack were designed for different purposes</strong>.</p><p>Proxmox is a complete virtualization platform.</p><p>Its architecture is roughly:</p><pre>┌──────────────────────────────┐<br>│          Proxmox VE          │<br>│                              │<br>│ Web UI                       │<br>│ API                          │<br>│ VM Management                │<br>│ Cluster Management           │<br>│ Storage Management           │<br>│ Network Management           │<br>├──────────────────────────────┤<br>│          KVM / QEMU          │<br>│            LXC               │<br>├──────────────────────────────┤<br>│        Linux Kernel          │<br>├──────────────────────────────┤<br>│          Hardware            │<br>└──────────────────────────────┘</pre><p>You install Proxmox and get an integrated platform.</p><p>OpenStack takes a different approach:</p><pre>┌──────────────────────────────┐<br>│          OpenStack           │<br>│                              │<br>│ Nova                         │<br>│ Neutron                      │<br>│ Keystone                     │<br>│ Glance                       │<br>│ Cinder                       │<br>│ Placement                    │<br>│ Horizon                      │<br>├──────────────────────────────┤<br>│       Linux Operating OS     │<br>├──────────────────────────────┤<br>│          Hardware            │<br>└──────────────────────────────┘</pre><p>OpenStack is primarily a <strong>cloud infrastructure control plane</strong>, rather than a complete host operating system.</p><h3>Would an OpenStack OS Make Communication Faster?</h3><p>At first glance, it sounds logical.</p><p>If OpenStack controlled the entire operating system, perhaps its components could communicate more efficiently.</p><p>But there is an important distinction between <strong>control-plane communication</strong> and <strong>data-plane communication</strong>.</p><p>For example, when you create a VM:</p><pre>User<br> │<br> ▼<br>Horizon / API<br> │<br> ▼<br>Keystone<br> │<br> ▼<br>Nova<br> │<br> ▼<br>Scheduler<br> │<br> ▼<br>Compute Node<br> │<br> ▼<br>KVM<br> │<br> ▼<br>VM</pre><p>This is control-plane communication.</p><p>But once the VM is running, its network traffic doesn’t normally travel through Nova or Keystone for every packet.</p><p>The data path looks more like:</p><pre>VM<br> │<br> ▼<br>Virtual NIC<br> │<br> ▼<br>OVS / Linux Bridge<br> │<br> ▼<br>Physical NIC<br> │<br> ▼<br>Network</pre><p>Therefore, replacing Ubuntu with an OpenStack-specific OS wouldn’t automatically make VM-to-VM networking faster.</p><h3>Control Plane vs Data Plane</h3><p>This distinction is extremely important.</p><h3>Control Plane</h3><p>The control plane decides what should happen.</p><p>Examples:</p><pre>Create VM<br>Delete VM<br>Attach volume<br>Assign IP<br>Create network<br>Authenticate user<br>Schedule workload</pre><p>Services involved include:</p><pre>Keystone<br>Nova<br>Neutron<br>Cinder<br>Glance<br>Placement</pre><h3>Data Plane</h3><p>The data plane actually carries the workload.</p><p>Examples:</p><pre>VM CPU<br>VM memory<br>VM network packets<br>VM disk I/O<br>GPU workload</pre><p>The data plane might look like:</p><pre>VM<br>              │<br>       ┌──────┴──────┐<br>       │             │<br>     Network        Disk<br>       │             │<br>      OVS           Ceph<br>       │             │<br>     NIC          Storage</pre><p>The data plane is where performance-sensitive technologies such as:</p><ul><li>DPDK</li><li>SR-IOV</li><li>hugepages</li><li>CPU pinning</li><li>NUMA</li><li>PCI passthrough</li></ul><p>become important.</p><h3>Then Why Does Proxmox Feel More Integrated?</h3><p>Because Proxmox controls the entire host environment.</p><p>You install:</p><pre>Proxmox ISO<br>     ↓<br>Physical Server<br>     ↓<br>Proxmox VE<br>     ↓<br>KVM<br>     ↓<br>VMs</pre><p>The operating system and virtualization platform are packaged together.</p><p>OpenStack intentionally doesn’t require this.</p><p>You can have:</p><pre>Ubuntu + OpenStack</pre><p>or:</p><pre>RHEL + OpenStack</pre><p>or other supported host environments.</p><p>OpenStack essentially says:</p><blockquote><em>“I will provide the cloud control plane. You provide the underlying operating system and infrastructure.”</em></blockquote><p>This separation gives OpenStack much more flexibility.</p><h3>Why OpenStack Doesn’t Force Its Own OS</h3><p>One major reason is <strong>vendor neutrality</strong>.</p><p>Imagine OpenStack required a specific operating system.</p><p>Then the architecture would become:</p><pre>OpenStack OS<br>     │<br>OpenStack<br>     │<br>Hardware</pre><p>Every organization would have to adopt the same host OS.</p><p>But OpenStack is used in environments with different requirements.</p><p>For example:</p><pre>Company A<br>Ubuntu + KVM<br>Company B<br>RHEL + KVM<br>Company C<br>SUSE + KVM<br>Company D<br>GPU infrastructure<br>Company E<br>Custom enterprise storage</pre><p>OpenStack’s design allows these environments to participate in an OpenStack cloud.</p><p>That flexibility is valuable in large enterprises.</p><h3>OpenStack Is More Like AWS Infrastructure Software</h3><p>A better mental model is to compare OpenStack with the software behind a cloud provider rather than directly with Proxmox.</p><p>Proxmox:</p><pre>Hardware<br>   ↓<br>Proxmox<br>   ↓<br>VM</pre><p>OpenStack:</p><pre>Hardware<br>   ↓<br>Linux<br>   ↓<br>KVM<br>   ↓<br>OpenStack<br>   ↓<br>Cloud resources<br>   ↓<br>VM</pre><p>OpenStack provides the APIs, scheduling, identity, networking, storage integration, and orchestration needed to build a cloud.</p><p>It doesn’t necessarily need to own the operating system.</p><h3>But Could We Build an OpenStack OS?</h3><p>Absolutely.</p><p>In fact, the idea is technically very interesting.</p><p>You could create:</p><pre>OpenCloud OS<br>       │<br>       ├── Linux Kernel<br>       ├── KVM<br>       ├── Open vSwitch<br>       ├── OpenStack<br>       ├── Ceph Client<br>       ├── Monitoring<br>       └── Management Agent</pre><p>Then distribute it as:</p><pre>opencloud-os.iso</pre><p>A user could install:</p><pre>opencloud-os.iso<br>        ↓<br>Physical Server<br>        ↓<br>Cloud Node<br>        ↓<br>Join Cluster<br>        ↓<br>Ready</pre><p>This would provide a user experience much closer to Proxmox.</p><h3>What Would an OpenStack OS Improve?</h3><p>A specialized OS could provide several advantages.</p><h3>1. Easier installation</h3><p>Instead of:</p><pre>Install Linux<br>       ↓<br>Configure repositories<br>       ↓<br>Install dependencies<br>       ↓<br>Install OpenStack<br>       ↓<br>Configure networking<br>       ↓<br>Configure storage<br>       ↓<br>Configure services</pre><p>you could have:</p><pre>Install ISO<br>    ↓<br>Configure network<br>    ↓<br>Join cluster<br>    ↓<br>Ready</pre><p>This is a major advantage.</p><h3>2. Standardized configuration</h3><p>The OS could ship with optimized defaults for:</p><pre>KVM<br>Open vSwitch<br>Hugepages<br>CPU isolation<br>NUMA<br>Networking<br>Storage</pre><p>Instead of every administrator configuring these manually.</p><h3>3. Better lifecycle management</h3><p>You could manage:</p><pre>OS<br>OpenStack<br>KVM<br>Networking<br>Drivers<br>Security patches</pre><p>from a unified update mechanism.</p><h3>4. Better hardware optimization</h3><p>A cloud-specific OS could detect:</p><pre>CPU<br>NUMA topology<br>NIC<br>GPU<br>NVMe<br>Storage controllers</pre><p>and automatically tune the host.</p><p>For example:</p><pre>GPU detected<br>      ↓<br>Install NVIDIA driver<br>      ↓<br>Configure IOMMU<br>      ↓<br>Configure PCI passthrough<br>      ↓<br>Register GPU resource<br>      ↓<br>OpenStack scheduler</pre><p>This could be extremely useful for GPU clouds.</p><h3>But There Is a Big Problem</h3><p>Creating an OS isn’t free.</p><p>Now the OpenStack project would have to maintain:</p><pre>Linux kernel<br>Bootloader<br>Drivers<br>Networking<br>Security updates<br>Package repositories<br>Hardware compatibility<br>KVM<br>Storage<br>OpenStack<br>Upgrade mechanism</pre><p>That’s a huge maintenance burden.</p><p>Instead, OpenStack can leverage existing Linux distributions.</p><p>For example:</p><pre>Ubuntu<br>   │<br>Linux kernel<br>   │<br>Drivers<br>   │<br>KVM<br>   │<br>OpenStack</pre><p>The Linux distribution already handles a massive amount of operating-system work.</p><p>OpenStack can focus on <strong>cloud infrastructure</strong>.</p><h3>Another Important Reason: Hardware Diversity</h3><p>A cloud platform can run on very different hardware.</p><p>For example:</p><pre>Intel CPU<br>AMD CPU<br>NVIDIA GPU<br>AMD GPU<br>Intel NIC<br>Mellanox NIC<br>Broadcom NIC<br>NVMe<br>SAS<br>Ceph<br>Enterprise SAN</pre><p>A dedicated OS would need to support all of them.</p><p>Linux already provides much of that hardware support.</p><p>So OpenStack can build on top of Linux instead of reinventing it.</p><h3>What About Internal OpenStack Communication?</h3><p>Another misconception is:</p><blockquote><em>“If everything were inside one OS, OpenStack services would communicate faster.”</em></blockquote><p>Not necessarily.</p><p>Consider:</p><pre>Nova<br> │<br> ▼<br>RabbitMQ<br> │<br> ▼<br>Neutron</pre><p>These services communicate through APIs and messaging systems.</p><p>Even if they were running on a custom OpenStack OS, you would still have:</p><pre>Nova<br> ↓<br>RabbitMQ<br> ↓<br>Neutron</pre><p>The architecture itself doesn’t disappear just because the operating system is customized.</p><p>For a distributed cloud:</p><pre>Controller 1<br>      │<br>      │<br>Controller 2<br>      │<br>      │<br>Controller 3<br>      │<br>      │<br>Compute Nodes</pre><p>network latency and distributed-system design remain important.</p><p>The OS is only one layer of the system.</p><h3>Where Performance Actually Matters</h3><p>If your goal is a high-performance cloud, I would focus more on the <strong>data plane</strong>.</p><p>For networking:</p><pre>VM<br> ↓<br>vNIC<br> ↓<br>OVS<br> ↓<br>DPDK / SR-IOV<br> ↓<br>NIC</pre><p>For CPU:</p><pre>VM<br> ↓<br>vCPU<br> ↓<br>CPU pinning<br> ↓<br>NUMA-aware scheduling<br> ↓<br>Physical CPU</pre><p>For storage:</p><pre>VM<br> ↓<br>VirtIO<br> ↓<br>QEMU<br> ↓<br>Ceph RBD<br> ↓<br>NVMe</pre><p>For GPU:</p><pre>VM<br> ↓<br>PCI Passthrough<br> ↓<br>NVIDIA GPU</pre><p>These optimizations can have a much larger impact than simply creating a new operating system.</p><h3>Proxmox vs OpenStack</h3><p>The difference can be summarized like this:</p><pre>| Feature                | Proxmox                         | OpenStack      |<br>| ---------------------- | ------------------------------- | -------------- |<br>| Complete host platform | Yes                             | No             |<br>| Own installation ISO   | Yes                             | No             |<br>| Built around Linux     | Yes                             | Yes            |<br>| KVM                    | Yes                             | Yes            |<br>| Web UI                 | Yes                             | Yes            |<br>| VM management          | Yes                             | Yes            |<br>| Cloud API              | Yes                             | Yes            |<br>| Large-scale cloud      | Limited compared with OpenStack | Excellent      |<br>| Modular services       | Less                            | Highly modular |<br>| Multi-tenant cloud     | Good                            | Excellent      |<br>| Hardware flexibility   | Good                            | Excellent      |<br>| Deployment complexity  | Lower                           | Higher         |</pre><h3>The Interesting Middle Ground</h3><p>There is actually a compelling architecture between Proxmox and traditional OpenStack.</p><p>Imagine:</p><pre>                Cloud OS<br>                    │<br>        ┌───────────┼───────────┐<br>        │           │           │<br>       KVM         OVS         Ceph<br>        │           │           │<br>        └───────────┼───────────┘<br>                    │<br>               OpenStack<br>                    │<br>              Cloud API<br>                    │<br>                Dashboard</pre><p>The user gets:</p><pre>Download ISO<br>      ↓<br>Install<br>      ↓<br>Configure cluster<br>      ↓<br>Create VM</pre><p>just like Proxmox.</p><p>But internally:</p><pre>Cloud OS<br>   ↓<br>OpenStack<br>   ↓<br>KVM<br>   ↓<br>Ceph<br>   ↓<br>OVS</pre><p>This could provide:</p><ul><li>Proxmox-like simplicity</li><li>OpenStack-like scalability</li><li>standardized hardware tuning</li><li>easier GPU support</li><li>easier cluster deployment</li><li>centralized lifecycle management</li></ul><h3>So Why Doesn’t OpenStack Simply Do It?</h3><p>Because OpenStack’s primary goal isn’t:</p><blockquote><strong><em>“Create the easiest virtualization operating system.”</em></strong></blockquote><p>Its goal is closer to:</p><blockquote><strong><em>“Provide an open, scalable cloud infrastructure control plane that can operate across different infrastructure environments.”</em></strong></blockquote><p>Proxmox optimizes for <strong>integration and simplicity</strong>.</p><p>OpenStack optimizes for <strong>scale, modularity, flexibility and cloud infrastructure</strong>.</p><p>Neither approach is universally better.</p><p>They solve different problems.</p><h3>Final Architecture Comparison</h3><h3>Proxmox</h3><pre>┌───────────────────────┐<br>│      Proxmox VE       │<br>│                       │<br>│ API + UI + Cluster    │<br>│ Storage + Network     │<br>│ VM Management         │<br>├───────────────────────┤<br>│       KVM / LXC       │<br>├───────────────────────┤<br>│      Linux Kernel     │<br>├───────────────────────┤<br>│       Hardware        │<br>└───────────────────────┘</pre><h3>OpenStack</h3><pre>┌────────────────────────────┐<br>│         OpenStack          │<br>│                            │<br>│ Nova │ Neutron │ Keystone  │<br>│ Glance │ Cinder │ Horizon  │<br>├────────────────────────────┤<br>│       Linux / KVM          │<br>├────────────────────────────┤<br>│         Hardware           │<br>└────────────────────────────┘</pre><h3>Hypothetical Cloud OS</h3><pre>┌────────────────────────────┐<br>│        Cloud OS             │<br>│                            │<br>│ OpenStack + Management     │<br>│ Monitoring + Automation    │<br>├────────────────────────────┤<br>│ KVM │ OVS │ Ceph │ Drivers │<br>├────────────────────────────┤<br>│       Linux Kernel         │<br>├────────────────────────────┤<br>│         Hardware           │<br>└────────────────────────────┘</pre><p>And this last model is particularly interesting if the goal is to build a <strong>simpler OpenStack-based private cloud or a RunPod-like GPU cloud</strong>.</p><p>The key lesson is:</p><blockquote><strong><em>OpenStack doesn’t lack its own OS because an OS would be useless. It deliberately avoids owning the OS because its architecture prioritizes flexibility and cloud-scale infrastructure. A specialized OpenStack OS could simplify deployment and improve integration, but the biggest performance gains would still come from optimizing the compute, network, storage, and GPU data planes.</em></strong></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5cd40e3976b5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Scaling HAProxy for Billions of Requests: Architecture, Strategies, and Best Practices]]></title>
            <link>https://arn-ob.medium.com/scaling-haproxy-for-billions-of-requests-architecture-strategies-and-best-practices-eee00b4ba786?source=rss-c7040ca07397------2</link>
            <guid isPermaLink="false">https://medium.com/p/eee00b4ba786</guid>
            <category><![CDATA[best-practices]]></category>
            <category><![CDATA[scaling]]></category>
            <category><![CDATA[load-balancer]]></category>
            <category><![CDATA[architecture]]></category>
            <category><![CDATA[haproxy]]></category>
            <dc:creator><![CDATA[Arnob]]></dc:creator>
            <pubDate>Mon, 10 Aug 2026 07:22:56 GMT</pubDate>
            <atom:updated>2026-08-11T06:18:30.247Z</atom:updated>
            <content:encoded><![CDATA[<p>HAProxy is one of the most popular and efficient load balancers and reverse proxies used in modern infrastructure.</p><p>It is extremely fast, lightweight, and capable of handling a very large number of concurrent connections.</p><p>But what happens when your infrastructure grows from thousands of requests to <strong>millions or even billions of requests?</strong></p><p>Can you simply increase maxconn?</p><p>No.</p><p>At large scale, HAProxy itself becomes part of a larger distributed architecture.</p><p>This article explains how to scale HAProxy properly from a single server to a multi-node, multi-region architecture capable of handling extremely large traffic volumes.</p><h3>First, Understand What 10 Billion Requests Means</h3><p>Before designing infrastructure, convert the total request count into Requests Per Second (RPS).</p><p>For example:</p><pre>| Traffic volume |  Average RPS |<br>| -------------- | -----------: |<br>| 10B/month      |   ~3,858 RPS |<br>| 10B/week       |  ~16,534 RPS |<br>| 10B/day        | ~115,741 RPS |<br>| 10B/hour       |   ~2.78M RPS |<br>| 10B/minute     |  ~166.7M RPS |</pre><p>This is extremely important.</p><p>A system processing:</p><p><strong>10 billion requests per month</strong></p><p>is completely different from:</p><p><strong>10 billion requests per day.</strong></p><p>Also, real traffic isn’t evenly distributed.</p><p>You might have:</p><pre>Average:     100K RPS<br>Peak:        500K RPS<br>Traffic spike: 1M RPS</pre><p>Therefore, infrastructure should be designed around <strong>peak traffic</strong>, not just average traffic.</p><h3>The Simplest Approach: Vertical Scaling</h3><p>The first way to scale HAProxy is to make the server bigger.</p><p>Instead of:</p><pre>HAProxy<br>4 CPU<br>4 GB RAM<br>1 Gbps</pre><p>you can move to:</p><pre>HAProxy<br>16–32 CPU<br>32–64 GB RAM<br>25/40/100 Gbps NIC<br>NVMe</pre><p>HAProxy supports multithreading, so you can utilize multiple CPU cores.</p><p>For example:</p><pre>global<br>    nbthread 16<br>    maxconn 500000</pre><p>Linux file descriptor limits must also be configured appropriately.</p><p>For example:</p><pre>ulimit -n</pre><p>For a high-load environment, a low value such as:</p><pre>1024</pre><p>will quickly become a limitation.</p><p>You may need something like:</p><pre>200000<br>500000<br>1000000</pre><p>depending on the architecture.</p><p>However, vertical scaling has a physical limit.</p><p>Eventually:</p><pre>CPU<br> ↓<br>NIC<br> ↓<br>PCIe<br> ↓<br>Memory<br> ↓<br>Kernel networking</pre><p>becomes the bottleneck.</p><p>At that point, we need horizontal scaling.</p><h3>Horizontal Scaling HAProxy</h3><p>Instead of running one large HAProxy server:</p><pre>                 Clients<br>                    |<br>                 HAProxy<br>                    |<br>                 Backend</pre><p>run multiple HAProxy instances:</p><pre>                    Clients<br>                       |<br>                 Load Balancer<br>                       |<br>          ┌────────────┼────────────┐<br>          ↓            ↓            ↓<br>      HAProxy-1    HAProxy-2    HAProxy-3<br>          |            |            |<br>          └────────────┼────────────┘<br>                       |<br>                    Backend</pre><p>Now traffic can be distributed across multiple HAProxy servers.</p><p>For example:</p><pre>300,000 RPS<br>     |<br>     +---- HAProxy-1 → 100K<br>     +---- HAProxy-2 → 100K<br>     +---- HAProxy-3 → 100K</pre><p>When traffic increases:</p><pre>Add HAProxy-4<br>Add HAProxy-5<br>Add HAProxy-6</pre><p>This is the fundamental principle of horizontal scaling.</p><h3>HAProxy Cluster With a Load Balancer</h3><p>A common architecture is:</p><pre>                         Internet<br>                            |<br>                    Global/External LB<br>                            |<br>             ┌──────────────┼──────────────┐<br>             ↓              ↓              ↓<br>        HAProxy-01      HAProxy-02      HAProxy-03<br>             |              |              |<br>             └──────────────┼──────────────┘<br>                            |<br>                       Application</pre><p>The external load balancer distributes connections across the HAProxy nodes.</p><p>Each HAProxy server becomes an independent processing unit.</p><p>This gives us:</p><ul><li>horizontal scalability</li><li>high availability</li><li>fault tolerance</li><li>easier capacity planning</li><li>rolling maintenance</li></ul><p>If one HAProxy server fails, traffic can continue through the others.</p><h3>Keepalived + VRRP (Virtual Router Redundancy Protocol)</h3><p>Another common HA design is Keepalived with VRRP.</p><p>For example:</p><pre>                 Virtual IP<br>                 10.0.0.100<br>                      |<br>             ┌────────┴────────┐<br>             ↓                 ↓<br>        HAProxy-01        HAProxy-02<br>         MASTER             BACKUP<br>             |                 |<br>             └────────┬────────┘<br>                      |<br>                   Backend</pre><p>If HAProxy-01 fails:</p><pre>HAProxy-01<br>    X<br>    |<br>    ↓<br>HAProxy-02<br>    |<br>    ↓<br>Virtual IP</pre><p>HAProxy-02 takes over the Virtual IP.</p><p>This provides excellent <strong>high availability</strong>.</p><p>However, there is an important distinction:</p><p><strong>Keepalived primarily provides failover, not active-active scaling.</strong></p><p>If only one node handles the VIP at a time, you aren’t doubling your throughput.</p><p>For actual capacity scaling, use multiple active HAProxy nodes behind a traffic-distribution layer.</p><h3>Use CDN and Edge Infrastructure</h3><p>If your service is HTTP/HTTPS based, you shouldn’t necessarily send every request directly to HAProxy.</p><p>Instead:</p><pre>                    Users<br>                      |<br>                    CDN<br>                      |<br>                    WAF<br>                      |<br>                 Load Balancer<br>                      |<br>                  HAProxy<br>                      |<br>                 Application</pre><p>A CDN can absorb huge amounts of traffic.</p><p>For example:</p><pre>10 Billion Requests<br>        |<br>       CDN<br>        |<br>        +---- 9B cache hits<br>        |<br>        +---- 1B origin requests<br>                  |<br>               HAProxy</pre><p>The exact numbers will depend on your cache hit ratio, but the principle is extremely powerful.</p><p><strong>Caching is often more effective than simply adding more load balancers.</strong></p><h3>Separate HAProxy by Workload</h3><p>Another scaling strategy is to avoid putting every service behind the same HAProxy cluster.</p><p>For example:</p><pre>                       Traffic<br>                          |<br>            ┌─────────────┼─────────────┐<br>            ↓             ↓             ↓<br>       HTTP HAProxy   MySQL HAProxy   TCP HAProxy<br>            ↓             ↓             ↓<br>          Apps          MySQL         Services</pre><p>This is especially useful in infrastructure environments where you have completely different workloads.</p><p>For example:</p><pre>HTTP<br>→ Kubernetes applications<br>TCP<br>→ MySQL<br>TCP<br>→ SMTP<br>TCP<br>→ Internal services</pre><p>If MySQL suddenly receives a large amount of traffic, it doesn’t consume all resources allocated to your HTTP load balancers.</p><h3>HAProxy and Kubernetes</h3><p>HAProxy can also sit in front of Kubernetes.</p><p>A typical architecture might look like:</p><pre>                       Internet<br>                          |<br>                       CDN/WAF<br>                          |<br>                     Load Balancer<br>                          |<br>                 ┌────────┴────────┐<br>                 ↓                 ↓<br>             HAProxy-1         HAProxy-2<br>                 |                 |<br>                 └────────┬────────┘<br>                          |<br>                     Kubernetes<br>                          |<br>              ┌───────────┼───────────┐<br>              ↓           ↓           ↓<br>             Pod         Pod         Pod</pre><p>Then Kubernetes can independently scale the application:</p><pre>HAProxy<br>   |<br>Kubernetes Service<br>   |<br>HPA<br>   |<br>More Pods</pre><p>This is important because scaling HAProxy doesn’t automatically solve application bottlenecks.</p><h3>Backend Scaling Is Just as Important</h3><p>Imagine:</p><pre>1 million requests/sec<br>        |<br>    HAProxy<br>        |<br>      MySQL</pre><p>If MySQL can only process:</p><pre>50,000 operations/sec</pre><p>adding more HAProxy nodes doesn’t solve the problem.</p><p>You might end up with:</p><pre>HAProxy-01 ─┐<br>HAProxy-02 ─┤<br>HAProxy-03 ─┼──→ MySQL<br>HAProxy-04 ─┤       💥<br>HAProxy-05 ─┘</pre><p>The bottleneck has simply moved to the database.</p><p>Large-scale systems therefore need to scale every layer:</p><pre>Internet<br>   ↓<br>CDN<br>   ↓<br>WAF<br>   ↓<br>Load Balancer<br>   ↓<br>HAProxy<br>   ↓<br>Application<br>   ↓<br>Cache<br>   ↓<br>Database</pre><h3>Connection Reuse</h3><p>At high traffic, connection management becomes extremely important.</p><p>For HTTP workloads, HAProxy can reuse backend connections.</p><p>Example:</p><pre>backend application<br>    balance leastconn<br>    http-reuse safe<br><br>    server app1 10.0.0.11:8080 check<br>    server app2 10.0.0.12:8080 check<br>    server app3 10.0.0.13:8080 check</pre><p>Without efficient connection reuse, the infrastructure may spend significant resources establishing and destroying connections.</p><p>This becomes particularly important when:</p><pre>Clients<br>   ↓<br>HAProxy<br>   ↓<br>Application<br>   ↓<br>Database</pre><p>has millions of client requests but the application/database doesn’t need millions of simultaneous backend connections.</p><h3>Don’t Blindly Increase maxconn</h3><p>A common mistake is:</p><pre>global<br>    maxconn 1000000</pre><p>assuming HAProxy can now handle one million connections safely.</p><p>That’s not how capacity planning works.</p><p>maxconn must be based on:</p><ul><li>available memory</li><li>CPU</li><li>file descriptors</li><li>backend capacity</li><li>network capacity</li><li>connection lifetime</li><li>request rate</li><li>TLS workload</li></ul><p>For example:</p><pre>100,000 connections</pre><p>can be easy in one workload and extremely expensive in another.</p><p>Always load-test your actual configuration.</p><h3>Network Bandwidth Can Become the Bottleneck</h3><p>Suppose your system handles:</p><pre>500,000 requests/sec</pre><p>and each request/response transfers:</p><pre>20 KB</pre><p>Approximate traffic:</p><pre>500,000 × 20 KB<br>= 10 GB/sec<br>≈ 80 Gbps</pre><p>Now your HAProxy server needs substantially more than a 10-Gbps NIC.</p><p>This is why high-scale architecture must consider:</p><pre>RPS<br>+<br>Concurrent Connections<br>+<br>Bandwidth<br>+<br>Packet Rate<br>+<br>CPU</pre><p>not just request count.</p><h3>Kernel and NIC Optimization</h3><p>At high traffic rates, Linux networking becomes important.</p><p>A high-performance HAProxy server may require tuning around:</p><pre>RSS<br>RPS<br>IRQ affinity<br>CPU pinning<br>NIC queues<br>TCP backlog<br>File descriptors<br>Socket buffers</pre><p>For example:</p><pre>net.core.somaxconn = 65535<br>net.ipv4.tcp_max_syn_backlog = 65535<br>net.ipv4.ip_local_port_range = 1024 65535</pre><p>These values should be tuned based on measurements rather than copied blindly.</p><p>At extremely high packet rates, properly distributing NIC interrupts across CPU cores can have a major impact.</p><h3>Monitoring Becomes Critical</h3><p>At large scale, you cannot manage HAProxy based only on CPU usage.</p><p>Monitor:</p><pre>RPS<br>Concurrent connections<br>Connection rate<br>Bandwidth<br>Queue length<br>Backend latency<br>HTTP errors<br>Connection errors<br>Server health<br>CPU<br>Memory<br>File descriptors</pre><p>For example:</p><pre>HAProxy-01<br>CPU:             85%<br>Connections:     420K<br>Connection rate: 75K/sec<br>Network:         32 Gbps<br>Queue:           0<br>5xx:             0.01%</pre><p>This gives you much more information than simply saying:</p><pre>CPU = 85%</pre><h3>Multi-Region HAProxy</h3><p>For global-scale infrastructure, you can distribute HAProxy geographically.</p><pre>                        Users<br>                           |<br>                    Global Routing<br>                           |<br>          ┌────────────────┼────────────────┐<br>          ↓                ↓                ↓<br>      Bangladesh        Singapore         Europe<br>          |                |                |<br>      HAProxy           HAProxy           HAProxy<br>          |                |                |<br>       Backend          Backend          Backend</pre><p>Traffic can be routed to the appropriate region.</p><p>This reduces:</p><ul><li>latency</li><li>cross-region bandwidth</li><li>regional load</li><li>blast radius</li></ul><h3>Anycast Architecture</h3><p>At even larger scale, Anycast can be used.</p><pre>                     Users<br>                       |<br>                  Anycast IP<br>                       |<br>          ┌────────────┼────────────┐<br>          ↓            ↓            ↓<br>       Region A     Region B     Region C<br>       HAProxy      HAProxy      HAProxy</pre><p>Each region advertises the same IP address using BGP.</p><p>The network determines where traffic goes.</p><p>This is much more complex than a normal HAProxy deployment, but it is a powerful architecture for globally distributed systems.</p><h3>A Large-Scale HAProxy Architecture</h3><p>Putting everything together:</p><pre>                           INTERNET<br>                              |<br>                         CDN / WAF<br>                              |<br>                     Global Load Balancer<br>                              |<br>            ┌─────────────────┼─────────────────┐<br>            ↓                 ↓                 ↓<br>        Region A           Region B           Region C<br>            |                 |                 |<br>       HAProxy Cluster   HAProxy Cluster   HAProxy Cluster<br>       ┌────┼────┐       ┌────┼────┐       ┌────┼────┐<br>       ↓    ↓    ↓       ↓    ↓    ↓       ↓    ↓    ↓<br>      HA   HA   HA      HA   HA   HA      HA   HA   HA<br>       └────┼────┘       └────┼────┘       └────┼────┘<br>            |                 |                 |<br>       Kubernetes        Kubernetes        Kubernetes<br>            |                 |                 |<br>         Apps              Apps              Apps<br>            |                 |                 |<br>        Cache/DB          Cache/DB          Cache/DB</pre><p>This architecture allows independent scaling at every layer.</p><h3>How I Would Approach 10 Billion Requests</h3><p>If the requirement is:</p><h4>10B requests/month</h4><p>That’s approximately:</p><pre>3,858 RPS average</pre><p>A practical architecture could be:</p><pre>                 CDN/WAF<br>                    |<br>              Load Balancer<br>                    |<br>           ┌────────┴────────┐<br>           ↓                 ↓<br>       HAProxy-01        HAProxy-02<br>           |                 |<br>           └────────┬────────┘<br>                    |<br>                Kubernetes<br>                    |<br>                App Pods</pre><p>This is not an enormous HAProxy problem.</p><p>If the requirement is:</p><h4>10B requests/day</h4><p>That’s approximately:</p><pre>115,741 RPS average</pre><p>Now you should start thinking about:</p><pre>CDN<br>+<br>WAF<br>+<br>Multiple HAProxy nodes<br>+<br>High-speed networking<br>+<br>Application autoscaling<br>+<br>Caching<br>+<br>Database scaling<br>+<br>Load testing</pre><p>For example:</p><pre>                     CDN/WAF<br>                        |<br>                   Global LB<br>                        |<br>          ┌─────────────┼─────────────┐<br>          ↓             ↓             ↓<br>      HAProxy-1     HAProxy-2     HAProxy-3<br>          ↓             ↓             ↓<br>          └─────────────┼─────────────┘<br>                        |<br>                  Kubernetes<br>                        |<br>                 Application<br>                        |<br>                     Cache<br>                        |<br>                    Database</pre><h3>The Most Important Lesson</h3><p>When scaling HAProxy, don’t think:</p><blockquote><em>“How can I make one HAProxy handle more requests?”</em></blockquote><p>Instead, think:</p><blockquote><em>“How can I distribute the workload across multiple independent layers?”</em></blockquote><p>The progression looks like:</p><pre>Level 1<br>Single HAProxy<br>     ↓<br>Level 2<br>Bigger HAProxy<br>     ↓<br>Level 3<br>Multiple HAProxy nodes<br>     ↓<br>Level 4<br>CDN + WAF + HAProxy cluster<br>     ↓<br>Level 5<br>Multi-region HAProxy<br>     ↓<br>Level 6<br>Anycast + global infrastructure</pre><p>And the complete scaling strategy becomes:</p><pre>                 GLOBAL USERS<br>                      |<br>                  CDN / WAF<br>                      |<br>               Global Routing<br>                      |<br>        ┌─────────────┼─────────────┐<br>        ↓             ↓             ↓<br>      Region A      Region B      Region C<br>        ↓             ↓             ↓<br>   HAProxy × N    HAProxy × N    HAProxy × N<br>        ↓             ↓             ↓<br>   Kubernetes     Kubernetes     Kubernetes<br>        ↓             ↓             ↓<br>      Cache         Cache         Cache<br>        ↓             ↓             ↓<br>    Database       Database       Database</pre><p><strong>HAProxy is highly scalable, but HAProxy should be treated as one layer of a distributed system — not the entire scaling solution.</strong></p><p>For billions of requests, the biggest wins usually come from <strong>traffic distribution, caching, connection reuse, horizontal scaling, and eliminating unnecessary requests before they reach the origin.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=eee00b4ba786" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How Large Language Models (LLMs) Actually Work: From User Prompts to Reasoning and GPU Computation]]></title>
            <link>https://arn-ob.medium.com/how-large-language-models-llms-actually-work-from-user-prompts-to-reasoning-and-gpu-computation-dabecec242a3?source=rss-c7040ca07397------2</link>
            <guid isPermaLink="false">https://medium.com/p/dabecec242a3</guid>
            <category><![CDATA[prompt-engineering]]></category>
            <category><![CDATA[chatgpt]]></category>
            <category><![CDATA[ai-assistant]]></category>
            <category><![CDATA[reasoning]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Arnob]]></dc:creator>
            <pubDate>Thu, 06 Aug 2026 07:18:02 GMT</pubDate>
            <atom:updated>2026-08-06T07:18:02.231Z</atom:updated>
            <content:encoded><![CDATA[<h3>How Large Language Models Actually Work: From User Prompts to Reasoning and GPU Computation</h3><p>When we ask ChatGPT or any AI assistant to do something, it often feels like magic:</p><blockquote><em>“Deploy an Nginx container on my laptop.”</em></blockquote><p>Within seconds, the AI understands the request and generates a Docker command. But what actually happens behind the scenes? How does a simple sentence become meaningful instructions that a machine can process?</p><p>In this article, we’ll explore the internal architecture of Large Language Models (LLMs), from user input to reasoning and GPU execution.</p><h3>The Journey of a User Prompt</h3><p>Let’s start with a simple example:</p><pre>Deploy an Nginx container on my laptop.</pre><p>Humans immediately understand the meaning:</p><ul><li>Action: Deploy</li><li>Application: Nginx</li><li>Type: Container</li><li>Target: Local machine</li></ul><p>However, an LLM doesn’t understand English in the same way humans do. Instead, it transforms text into mathematical representations and predicts the most likely response.</p><p>The complete pipeline looks like this:</p><pre>User Prompt<br>     ↓<br>Tokenization<br>     ↓<br>Embeddings<br>     ↓<br>Attention Mechanism<br>     ↓<br>Pattern Recognition<br>     ↓<br>Hidden Reasoning<br>     ↓<br>Probability Calculation<br>     ↓<br>Next Token Prediction<br>     ↓<br>Final Response</pre><p>Let’s dive deeper into each stage.</p><h3>Step 1: Tokenization</h3><p>The first thing an LLM does is break the sentence into smaller pieces called <strong>tokens</strong>.</p><p>Input:</p><pre>Deploy an Nginx container on my laptop</pre><p>Tokens:</p><pre>[&quot;Deploy&quot;, &quot;an&quot;, &quot;Nginx&quot;, &quot;container&quot;, &quot;on&quot;, &quot;my&quot;, &quot;laptop&quot;]</pre><p>Each token is converted into a unique numerical ID:</p><pre>Deploy    → 4821<br>Nginx     → 8734<br>container → 5532<br>laptop    → 9101</pre><p>At this point, the model no longer sees words — it only sees numbers.</p><h3>Step 2: Embeddings — Converting Words into Mathematics</h3><p>The model converts each token into a vector containing thousands of numbers.</p><p>For example:</p><pre>Deploy → [0.23, -0.11, 0.87, ...]<br>Nginx  → [0.91, 0.34, -0.45, ...]<br>Laptop → [-0.32, 0.77, 0.22, ...]</pre><p>These vectors represent semantic meaning.</p><p>Interestingly, words with similar meanings tend to have similar vectors:</p><pre>deploy ≈ run ≈ launch</pre><pre>docker ≈ container runtime</pre><pre>laptop ≈ local machine</pre><p>This is how the model begins to understand relationships between concepts.</p><h3>Step 3: Pattern Matching Isn’t Traditional Matching</h3><p>Many people think that LLMs use simple string matching:</p><pre>deploy == deploy</pre><p>But that’s not how it works.</p><p>Instead, LLMs learn statistical relationships from billions of examples.</p><p>Imagine the model has seen these sentences during training:</p><pre>Deploy Nginx on Docker.</pre><pre>Run an Nginx container.</pre><pre>Launch a web server locally.</pre><pre>Start Redis on my machine.</pre><p>Over time, the model learns patterns such as:</p><pre>deploy ≈ run ≈ launch ≈ start</pre><pre>nginx ≈ web server</pre><pre>locally ≈ my laptop</pre><p>As a result, these prompts are interpreted similarly:</p><pre>Deploy Nginx locally.</pre><pre>Run Nginx on my machine.</pre><pre>Launch a web server in Docker.</pre><p>Even though the wording is different, the underlying meaning is nearly identical.</p><h3>Step 4: The Attention Mechanism</h3><p>The most important innovation behind modern LLMs is the Transformer architecture and its attention mechanism.</p><p>Attention allows every word to interact with every other word.</p><p>For example:</p><pre>Deploy Redis on my laptop.</pre><p>Internally, the model creates relationships such as:</p><pre>Deploy  ─────► Redis<br>Deploy  ─────► laptop<br>Redis   ─────► laptop</pre><p>From these relationships, the model infers:</p><ul><li>Action: Deploy</li><li>Service: Redis</li><li>Environment: Local machine</li></ul><p>Context is critical.</p><p>Consider the word “Apple”:</p><p>Example 1:</p><pre>I ate an apple.</pre><p>Example 2:</p><pre>Apple released a new MacBook.</pre><p>The same word has completely different meanings depending on its surrounding context.</p><h3>Step 5: The Hidden Reasoning Process</h3><p>People often ask:</p><p>“Where is the reasoning engine inside an LLM?”</p><p>The answer is surprising:</p><p>There is no separate reasoning module.</p><p>Reasoning emerges from multiple transformer layers working together.</p><p>Suppose the user writes:</p><pre>Deploy an Nginx container and expose port 8080.</pre><p>Conceptually, the model understands:</p><pre>{<br>  &quot;intent&quot;: &quot;deploy&quot;,<br>  &quot;service&quot;: &quot;nginx&quot;,<br>  &quot;target&quot;: &quot;local_machine&quot;,<br>  &quot;port&quot;: 8080<br>}</pre><p>However, this JSON doesn’t actually exist inside the model. Instead, the information is encoded in hidden vectors spread across many neural network layers.</p><p>A simplified view:</p><pre>Layer 1  → Understand words</pre><pre>Layer 5  → Learn grammar</pre><pre>Layer 15 → Detect actions</pre><pre>Layer 30 → Connect relationships</pre><pre>Layer 50 → Generate response</pre><p>Each layer transforms the input representation into something richer and more meaningful.</p><h3>Step 6: Next-Token Prediction</h3><p>LLMs don’t generate complete paragraphs at once.</p><p>They generate one token at a time.</p><p>Suppose the model has already produced:</p><pre>docker run -d</pre><p>Now it calculates probabilities:</p><pre>nginx    → 82%<br></pre><p>The most probable token is selected:</p><pre>docker run -d nginx</pre><p>The process repeats until the entire response is generated.</p><p>Every answer you receive is simply a sequence of probability calculations.</p><h3>Step 7: The Role of the GPU</h3><p>At this point, we’ve discussed language and reasoning. But where does the GPU fit into all of this?</p><p>The GPU doesn’t understand language.</p><p>It doesn’t know what “Docker” or “Nginx” means.</p><p>Its job is to perform mathematical operations extremely quickly.</p><p>The computational pipeline looks like this:</p><pre>User Prompt<br>     ↓<br>Tokenization (CPU)<br>     ↓<br>Embeddings<br>     ↓<br>GPU Computation<br>     ↓<br>Attention Layers<br>     ↓<br>Feed-Forward Networks<br>     ↓<br>Probability Calculation<br>     ↓<br>Response</pre><h3>Matrix Multiplication: The Heart of LLMs</h3><p>Most of the computation inside an LLM comes from matrix multiplication.</p><p>For example:</p><pre>Input Matrix (5 × 4096)<br>×<br>Weight Matrix (4096 × 4096)<br>=<br>Output Matrix (5 × 4096)</pre><p>This single operation may involve tens of millions of calculations.</p><p>Now multiply that across dozens of transformer layers.</p><p>Modern LLMs perform billions of calculations every second.</p><h3>Why GPUs Instead of CPUs?</h3><p>CPUs are designed for sequential tasks.</p><p>GPUs are designed for parallel computation.</p><p>A CPU may have:</p><ul><li>8 cores</li><li>16 cores</li><li>32 cores</li></ul><p>A GPU may have:</p><ul><li>5,000 cores</li><li>10,000 cores</li><li>20,000 cores</li></ul><p>Instead of processing operations one by one, GPUs process thousands of calculations simultaneously.</p><p>This makes them ideal for neural networks.</p><h3>Model Memory and VRAM</h3><p>Large models require enormous amounts of memory.</p><p>Approximate memory requirements:</p><p>ModelFP16 Memory7B14 GB13B26 GB70B140 GB</p><p>With quantization:</p><p>Model4-bit Memory7B4 GB13B8 GB70B35 GB</p><p>The model weights are stored in GPU memory (VRAM), allowing fast access during inference.</p><h3>How AI Agents Extend LLMs</h3><p>Modern AI agents add an extra layer on top of the LLM.</p><p>Consider this prompt:</p><pre>Deploy PostgreSQL on Kubernetes.</pre><p>An AI agent might internally convert it into:</p><pre>{<br>  &quot;intent&quot;: &quot;deploy&quot;,<br>  &quot;service&quot;: &quot;postgresql&quot;,<br>  &quot;platform&quot;: &quot;kubernetes&quot;,<br>  &quot;steps&quot;: [<br>    &quot;check cluster&quot;,<br>    &quot;generate YAML&quot;,<br>    &quot;apply manifest&quot;,<br>    &quot;verify pods&quot;<br>  ]<br>}</pre><p>The agent workflow looks like this:</p><pre>User Prompt<br>     ↓<br>Intent Detection<br>     ↓<br>Entity Extraction<br>     ↓<br>Planning<br>     ↓<br>Tool Selection<br>     ↓<br>Execution<br>     ↓<br>Validation<br>     ↓<br>Final Response</pre><p>The LLM provides language understanding, while the agent framework provides execution and orchestration.</p><h3>Final Thoughts</h3><p>Large Language Models don’t truly “understand” language in the human sense.</p><p>Internally, they operate on:</p><ul><li>Tokens</li><li>Embeddings</li><li>Attention matrices</li><li>Neural network layers</li><li>Probability distributions</li><li>GPU computations</li></ul><p>What we perceive as intelligence emerges from billions of mathematical operations performed in parallel.</p><p>The next time you ask an AI to deploy a server, write code, or explain a concept, remember that behind that simple interaction lies an enormous pipeline of mathematics, probability, and computation.</p><p>The magic of AI isn’t magic at all — it’s mathematics at an incredible scale.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dabecec242a3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a DevOps Agent: From User Prompt to Kubernetes and Docker Execution]]></title>
            <link>https://arn-ob.medium.com/building-a-devops-agent-from-user-prompt-to-kubernetes-and-docker-execution-112a263880cd?source=rss-c7040ca07397------2</link>
            <guid isPermaLink="false">https://medium.com/p/112a263880cd</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[infrastructure]]></category>
            <category><![CDATA[execution]]></category>
            <dc:creator><![CDATA[Arnob]]></dc:creator>
            <pubDate>Wed, 05 Aug 2026 16:02:13 GMT</pubDate>
            <atom:updated>2026-08-05T16:02:13.645Z</atom:updated>
            <content:encoded><![CDATA[<p>Modern infrastructure teams increasingly rely on AI agents to automate operational tasks. But an operations agent is not just a chatbot — it must understand user intent, validate requests, enforce security boundaries, and execute commands safely on real machines.</p><p>The uploaded document describes the complete architecture of <strong>ops-agent</strong>, an AI-powered infrastructure assistant that manages Kubernetes and Docker workloads while maintaining strict security controls.</p><h3>The Big Picture</h3><p>At a high level, <strong>ops-agent</strong> acts as a bridge between users and infrastructure. Users interact through either a command-line interface or a web dashboard, while the agent processes requests and executes operations locally.</p><pre>User (CLI / Browser)<br>        │<br>        ▼<br>   Ops-Agent Runtime<br>        │<br> ┌──────┴──────┐<br> │             │<br>CLI          GUI<br> │             │<br> └──────┬──────┘<br>        ▼<br>   Agent Engine<br>        │<br>        ▼<br> Tool Execution Layer<br>        │<br> ┌──────┼──────────┐<br> │      │          │<br>Docker Kubernetes Config</pre><p>One important design principle stands out:</p><blockquote><em>The language model may run remotely, but every tool executes on the machine where the agent is running.</em></blockquote><p>This separation ensures that the AI itself never directly controls infrastructure.</p><h3>Startup Flow</h3><p>When the application starts, it follows a well-defined initialization process:</p><ol><li>Load environment variables and configuration files.</li><li>Parse command-line flags.</li><li>Create a runtime context.</li><li>Start either CLI mode or GUI mode.</li><li>Initialize the encrypted configuration store.</li><li>Register tools and start the agent.</li></ol><pre>make build<br><br>./bin/ops-agent<br>./bin/ops-agent --provider ollama --model qwen2.5:7b<br>./bin/ops-agent --gui --port 8080</pre><p>The runtime supports multiple execution modes, including:</p><ul><li>Local rule-based mode</li><li>Ollama</li><li>OpenAI</li><li>LangGraph</li></ul><p>Each provider is selected dynamically based on configuration and environment variables.</p><h3>User Interaction Paths</h3><p>The agent supports two interfaces.</p><h3>CLI Mode</h3><p>In terminal mode, users communicate through a REPL interface:</p><pre>you&gt; deploy nginx<br>agent&gt; Kubernetes plan created.</pre><p>The CLI also supports multi-line Kubernetes configurations, making cluster onboarding easier.</p><h3>GUI Mode</h3><p>The web interface exposes REST endpoints:</p><p>EndpointPurpose/api/chatSend messages/api/statusGet system status/api/manifest/validateValidate Kubernetes YAML/api/manifest/applyApply manifests</p><p>All chat requests are serialized to prevent concurrent state corruption.</p><h3>The Agent Execution Pipeline</h3><p>Every request goes through a deterministic pipeline before reaching the LLM.</p><pre>User Request<br>     │<br>     ▼<br>Pre-Handlers<br>     │<br>     ▼<br>Validation<br>     │<br>     ▼<br>LLM Tool Loop<br>     │<br>     ▼<br>Tool Execution<br>     │<br>     ▼<br>Infrastructure</pre><p>The pre-handlers are responsible for:</p><ul><li>Machine information requests</li><li>Kubeconfig validation</li><li>Local Docker operations</li><li>Deployment confirmation</li><li>Watch operations</li></ul><p>These checks prevent the model from hallucinating infrastructure actions.</p><h3>Kubernetes Deployment Flow</h3><p>A deployment request such as:</p><pre>deploy nginx</pre><p>follows a strict approval process.</p><pre>User Request<br>      │<br>      ▼<br>Check kubeconfig<br>      │<br>      ▼<br>Validate cluster<br>      │<br>      ▼<br>Generate deployment plan<br>      │<br>      ▼<br>User approval<br>      │<br>      ▼<br>Apply deployment</pre><p>The system never deploys resources without explicit approval.</p><p>This approval-based workflow ensures:</p><ul><li>No accidental deployments</li><li>Cluster validation before execution</li><li>Human oversight for all write operations</li></ul><h3>Local Docker Deployment</h3><p>Docker deployments follow a similar process.</p><p>For example:</p><pre>deploy mysql in docker</pre><p>The flow becomes:</p><pre>Check Docker<br>      │<br>      ▼<br>Search image<br>      │<br>      ▼<br>Generate plan<br>      │<br>      ▼<br>User approval<br>      │<br>      ▼<br>docker pull<br>docker run</pre><p>The agent maintains deployment templates for services such as:</p><ul><li>NGINX</li><li>MySQL</li><li>PostgreSQL</li><li>Redis</li><li>MongoDB</li><li>RabbitMQ</li><li>HAProxy</li><li>Caddy</li></ul><p>The deployment is executed only after the user confirms the generated plan.</p><h3>Tool System</h3><p>The tool layer separates read-only operations from write operations.</p><h3>Read-only tools</h3><p>Examples:</p><ul><li>List pods</li><li>Get logs</li><li>List deployments</li><li>Validate cluster</li><li>Check Docker status</li></ul><h3>Write tools</h3><p>Examples:</p><ul><li>Save kubeconfig</li><li>Deploy NGINX</li><li>Apply Docker deployments</li><li>Delete resources</li></ul><p>Every write tool requires approval before execution.</p><h3>Security Principles</h3><p>The document emphasizes several security guarantees:</p><ol><li>Never invent kubeconfig files.</li><li>Never deploy without approval.</li><li>Never expose secrets in logs.</li><li>Never allow unrestricted shell execution.</li><li>Always validate clusters before applying changes.</li><li>All tools run locally.</li><li>Never fabricate Docker logs.</li></ol><p>These rules protect both infrastructure and users from accidental or malicious actions.</p><h3>Package Structure</h3><p>The project is divided into clear modules:</p><pre>cmd/ops-agent        → Entry point<br>internal/agent       → Agent engine<br>internal/docker      → Docker operations<br>internal/k8s         → Kubernetes operations<br>internal/gui         → Web interface<br>internal/cli         → Command-line interface<br>internal/config      → Encrypted storage<br>internal/local       → Machine information</pre><p>This separation keeps the architecture modular and easier to maintain.</p><h3>Final Thoughts</h3><p>Ops-agent demonstrates how an AI-powered infrastructure assistant can combine large language models with deterministic execution and strong security guarantees. Instead of allowing the model to execute arbitrary commands, the architecture enforces validation, planning, approval, and controlled tool execution.</p><p>The result is an operations platform that can automate Kubernetes and Docker tasks while preserving safety, transparency, and human control.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=112a263880cd" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Effective Context Engineering for AI Agents: Beyond Prompt Engineering]]></title>
            <link>https://arn-ob.medium.com/effective-context-engineering-for-ai-agents-beyond-prompt-engineering-7d322f557a95?source=rss-c7040ca07397------2</link>
            <guid isPermaLink="false">https://medium.com/p/7d322f557a95</guid>
            <category><![CDATA[context-engineering]]></category>
            <category><![CDATA[prompt-engineering]]></category>
            <dc:creator><![CDATA[Arnob]]></dc:creator>
            <pubDate>Tue, 04 Aug 2026 06:53:45 GMT</pubDate>
            <atom:updated>2026-08-04T06:53:45.022Z</atom:updated>
            <content:encoded><![CDATA[<p>Artificial intelligence is no longer just about writing clever prompts. Modern AI agents interact with tools, databases, APIs, files, and cloud infrastructure. The real challenge is no longer <em>what to ask</em> the model — it’s <em>what information the model should see at each step</em>.</p><p>This discipline is called <strong>context engineering</strong>.</p><p>Anthropic describes context engineering as the process of selecting, organizing, retrieving, compressing, and managing information throughout an agent’s lifecycle. Context is not infinite; it is a limited resource that must be used wisely.</p><h3>Prompt Engineering vs Context Engineering</h3><p>Prompt engineering focuses on instructions:</p><pre>You are a DevOps engineer.<br>Deploy Redis on my laptop.</pre><p>Context engineering focuses on the entire information environment:</p><pre>{<br>  &quot;system_prompt&quot;: &quot;You are a DevOps engineer.&quot;,<br>  &quot;environment&quot;: {<br>    &quot;os&quot;: &quot;Ubuntu 24.04&quot;,<br>    &quot;docker&quot;: true,<br>    &quot;kubernetes&quot;: false<br>  },<br>  &quot;memory&quot;: {<br>    &quot;preferred_runtime&quot;: &quot;docker&quot;<br>  },<br>  &quot;available_tools&quot;: [<br>    &quot;docker&quot;,<br>    &quot;ssh&quot;<br>  ],<br>  &quot;user_request&quot;: &quot;Deploy Redis&quot;<br>}</pre><p>The prompt tells the model <strong>how to behave</strong>.</p><p>The context tells the model <strong>how the world actually works</strong>.</p><h3>Context Is a Finite Resource</h3><p>One of the biggest mistakes in agent design is assuming that more context always produces better results.</p><p>It doesn’t.</p><p>Adding unnecessary logs, documents, tool outputs, and chat history creates:</p><ul><li>Context pollution</li><li>Higher token costs</li><li>Slower responses</li><li>Tool confusion</li><li>Hallucinations</li></ul><p>Good agents optimize context instead of maximizing it.</p><h3>The Four Pillars of Context Engineering</h3><h3>1. Write Context</h3><p>Agents generate new information while they work:</p><ul><li>Plans</li><li>Notes</li><li>Summaries</li><li>Intermediate results</li><li>Decisions</li></ul><p>Instead of keeping the entire conversation, agents should store only important facts.</p><h3>2. Select Context</h3><p>Every task requires different information.</p><p>For Docker operations:</p><pre>Load:<br>✓ Docker tools<br>✓ Container state<br>✓ Network configuration<br><br>Ignore:<br>✗ Kubernetes manifests<br>✗ Terraform state<br>✗ AWS credentials</pre><p>Only relevant context should enter the context window.</p><h3>3. Compress Context</h3><p>Long-running agents accumulate enormous amounts of data.</p><p>Instead of storing:</p><pre>Container restarted 20 times</pre><p>Store:</p><pre>{<br>  &quot;service&quot;: &quot;api-server&quot;,<br>  &quot;restart_count&quot;: 20,<br>  &quot;last_restart&quot;: &quot;2026-08-04&quot;<br>}</pre><p>Compression preserves meaning while reducing token usage.</p><h3>4. Isolate Context</h3><p>Different tasks require different workspaces.</p><p>A Docker agent should not automatically load Kubernetes data.</p><p>An AWS deployment task should not include PostgreSQL backups.</p><p>Context isolation reduces confusion and improves reasoning.</p><h3>The Context Lifecycle</h3><p>Modern agents constantly update their context.</p><pre>User Request<br>     │<br>     ▼<br>Intent Detection<br>     │<br>     ▼<br>Retrieve Relevant Data<br>     │<br>     ▼<br>Filter Context<br>     │<br>     ▼<br>Execute Tools<br>     │<br>     ▼<br>Compress Results<br>     │<br>     ▼<br>Store Memory<br>     │<br>     ▼<br>Generate Response</pre><p>The context window is dynamic, not static.</p><h3>Context for a DevOps Agent</h3><p>Suppose a user says:</p><blockquote><em>Deploy Redis on my laptop.</em></blockquote><p>The agent builds context like this:</p><pre>{<br>  &quot;request&quot;: &quot;Deploy Redis&quot;,<br>  &quot;environment&quot;: {<br>    &quot;os&quot;: &quot;Ubuntu 24.04&quot;,<br>    &quot;docker&quot;: true<br>  },<br>  &quot;memory&quot;: {<br>    &quot;preferred_runtime&quot;: &quot;docker&quot;<br>  },<br>  &quot;tools&quot;: [<br>    &quot;docker&quot;,<br>    &quot;ssh&quot;<br>  ]<br>}</pre><p>The model generates:</p><pre>docker pull redis:latest<br><br>docker run -d \<br>  --name redis \<br>  -p 6379:6379 \<br>  redis:latest</pre><p>The next user request is:</p><blockquote><em>Back it up.</em></blockquote><p>The agent retrieves memory:</p><pre>{<br>  &quot;last_service&quot;: &quot;redis&quot;<br>}</pre><p>The model now understands the meaning of “it”.</p><h3>Just-in-Time Context</h3><p>Anthropic emphasizes that agents should load context only when needed.</p><p>Instead of loading every available tool:</p><pre>500 tool definitions<br>200 MB documentation<br>10,000 log lines</pre><p>The agent loads only:</p><pre>docker_logs<br>docker_restart<br>docker_inspect</pre><p>When the user switches to Kubernetes, Docker tools can be removed and Kubernetes tools loaded dynamically.</p><p>This approach reduces cost and improves accuracy.</p><h3>The Golden Rule</h3><p>The quality of an AI agent is determined by:</p><pre>Agent Quality = <br><br>Model<br>× Context<br>× Memory<br>× Tools<br>× Retrieval</pre><p>A powerful model with poor context will fail.</p><p>A smaller model with excellent context engineering can outperform a larger model with bad context.</p><p>The future of AI agents is not just better models — it’s better context.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7d322f557a95" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Context Engineering: The Secret Behind Powerful AI Agents]]></title>
            <link>https://arn-ob.medium.com/context-engineering-the-secret-behind-powerful-ai-agents-2cbec3cdb1fa?source=rss-c7040ca07397------2</link>
            <guid isPermaLink="false">https://medium.com/p/2cbec3cdb1fa</guid>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[context-engineering]]></category>
            <dc:creator><![CDATA[Arnob]]></dc:creator>
            <pubDate>Tue, 04 Aug 2026 06:49:41 GMT</pubDate>
            <atom:updated>2026-08-04T06:49:41.853Z</atom:updated>
            <content:encoded><![CDATA[<p>Artificial intelligence models are becoming increasingly powerful, but even the most advanced language model cannot magically understand your infrastructure, applications, or intentions. The quality of an AI system depends not only on the model itself but also on the information provided to it.</p><p>This is where <strong>context engineering</strong> comes into play.</p><h3>What Is Context Engineering?</h3><p>Context engineering is the process of designing, organizing, and managing all the information that an AI model receives before generating a response.</p><p>Most people think that an AI only sees the user’s message, but in reality, modern AI agents rely on many additional sources of information:</p><ul><li>System prompts</li><li>User prompts</li><li>Conversation history</li><li>Long-term memory</li><li>Tool outputs</li><li>Database records</li><li>API responses</li><li>RAG (Retrieval-Augmented Generation)</li><li>Environment information such as Docker, Kubernetes, and AWS</li></ul><p>The goal of context engineering is simple: provide the right information at the right time so the AI can make better decisions.</p><h3>Why Context Engineering Matters</h3><p>Without proper context, AI systems can easily make mistakes.</p><p>For example, imagine giving an AI agent the following instruction:</p><blockquote><em>Deploy Nginx.</em></blockquote><p>The model immediately faces several questions:</p><ul><li>Should it use Docker or Kubernetes?</li><li>Is the target machine local or cloud-based?</li><li>Which operating system is running?</li><li>Is this for development or production?</li><li>Which ports should be exposed?</li></ul><p>Without context, the model will guess. Sometimes it will guess correctly, but often it will not.</p><p>With context engineering, the same request becomes much clearer:</p><pre>{<br>  &quot;user_request&quot;: &quot;Deploy nginx&quot;,<br>  &quot;environment&quot;: {<br>    &quot;os&quot;: &quot;Ubuntu 24.04&quot;,<br>    &quot;docker_installed&quot;: true,<br>    &quot;kubernetes_installed&quot;: false<br>  },<br>  &quot;preferences&quot;: {<br>    &quot;runtime&quot;: &quot;docker&quot;<br>  }<br>}</pre><p>Now the model can confidently generate:</p><pre>docker run -d \<br>  --name nginx \<br>  -p 80:80 \<br>  nginx:latest</pre><h3>The Architecture of Context Engineering</h3><p>A modern AI agent typically follows this pipeline:</p><pre>User Input<br>    │<br>    ▼<br>Intent Detection<br>    │<br>    ▼<br>Context Builder<br>    ├── System Prompt<br>    ├── User History<br>    ├── Memory<br>    ├── Tool Outputs<br>    ├── Environment State<br>    └── RAG Data<br>    │<br>    ▼<br>   LLM<br>    │<br>    ▼<br>Action Planner<br>    │<br>    ▼<br>Command Execution</pre><p>The context builder is the brain behind the system. Its responsibility is to collect only the information relevant to the current task.</p><h3>The Five Layers of Context</h3><h3>1. Static Context</h3><p>Static context rarely changes.</p><p>Examples:</p><ul><li>Operating system</li><li>Installed software</li><li>Security policies</li><li>System prompts</li></ul><pre>{<br>  &quot;os&quot;: &quot;Ubuntu 24.04&quot;,<br>  &quot;docker&quot;: true,<br>  &quot;kubernetes&quot;: true,<br>  &quot;cloud&quot;: &quot;AWS&quot;<br>}</pre><h3>2. Dynamic Context</h3><p>Dynamic context represents the current state of the machine.</p><p>Examples:</p><ul><li>Running containers</li><li>CPU usage</li><li>Disk usage</li><li>Memory usage</li></ul><pre>{<br>  &quot;running_containers&quot;: [<br>    &quot;nginx&quot;,<br>    &quot;redis&quot;<br>  ],<br>  &quot;disk_usage&quot;: &quot;85%&quot;<br>}</pre><h3>3. User Context</h3><p>User context explains what the user actually wants.</p><pre>{<br>  &quot;request&quot;: &quot;Deploy Redis&quot;,<br>  &quot;environment&quot;: &quot;local machine&quot;,<br>  &quot;port&quot;: 6379<br>}</pre><h3>4. Memory Context</h3><p>Memory allows AI systems to remember previous actions.</p><pre>{<br>  &quot;last_deployment&quot;: &quot;postgres&quot;,<br>  &quot;preferred_runtime&quot;: &quot;docker&quot;<br>}</pre><p>Without memory:</p><p><strong>User:</strong> Deploy MySQL.</p><p>Later:</p><p><strong>User:</strong> Back it up.</p><p>The AI may not know what “it” refers to.</p><p>With memory:</p><pre>{<br>  &quot;last_service&quot;: &quot;mysql&quot;<br>}</pre><p>The AI understands that the user means:</p><blockquote><em>Back up the MySQL database.</em></blockquote><h3>5. Retrieved Context (RAG)</h3><p>Instead of stuffing huge amounts of information into prompts, AI agents retrieve only relevant documents.</p><p>For example:</p><p>User request:</p><blockquote><em>Deploy PostgreSQL.</em></blockquote><p>Retrieved data:</p><ul><li>PostgreSQL deployment guide</li><li>Existing Docker Compose files</li><li>Network configuration</li><li>Previous deployment history</li></ul><p>This keeps the context lightweight and accurate.</p><h3>How Context Engineering Improves AI Systems</h3><h3>Reduces hallucinations</h3><p>Good context prevents the model from inventing commands or making assumptions.</p><h3>Improves intent understanding</h3><p>User input:</p><blockquote><em>Run Redis locally.</em></blockquote><p>Structured interpretation:</p><pre>{<br>  &quot;intent&quot;: &quot;deploy_container&quot;,<br>  &quot;target&quot;: &quot;local_machine&quot;,<br>  &quot;service&quot;: &quot;redis&quot;<br>}</pre><h3>Improves tool selection</h3><p>If the AI knows which tools are available:</p><pre>{<br>  &quot;available_tools&quot;: [<br>    &quot;docker&quot;,<br>    &quot;kubectl&quot;,<br>    &quot;ssh&quot;<br>  ]<br>}</pre><p>Then:</p><p><strong>User:</strong> Show all running containers.</p><p>The AI chooses:</p><pre>docker ps</pre><p>instead of:</p><pre>kubectl get pods</pre><h3>Provides real-time awareness</h3><pre>{<br>  &quot;cpu_usage&quot;: &quot;90%&quot;,<br>  &quot;free_memory&quot;: &quot;512MB&quot;<br>}</pre><p>The AI can avoid dangerous actions that may overload the system.</p><h3>Context Engineering for DevOps Agents</h3><p>For infrastructure agents, context is often more important than the model itself.</p><p>A complete context package might look like this:</p><pre>{<br>  &quot;system&quot;: {<br>    &quot;role&quot;: &quot;DevOps engineer&quot;<br>  },<br>  &quot;environment&quot;: {<br>    &quot;os&quot;: &quot;Ubuntu 24.04&quot;,<br>    &quot;docker&quot;: true<br>  },<br>  &quot;memory&quot;: {<br>    &quot;preferred_runtime&quot;: &quot;docker&quot;<br>  },<br>  &quot;tools&quot;: [<br>    &quot;docker&quot;,<br>    &quot;kubectl&quot;,<br>    &quot;terraform&quot;<br>  ],<br>  &quot;request&quot;: &quot;Deploy Redis&quot;<br>}</pre><p>The AI processes the context and generates:</p><pre>{<br>  &quot;intent&quot;: &quot;deploy_service&quot;,<br>  &quot;commands&quot;: [<br>    &quot;docker pull redis:latest&quot;,<br>    &quot;docker run -d --name redis -p 6379:6379 redis:latest&quot;<br>  ]<br>}</pre><h3>The Rise of Just-in-Time Context</h3><p>Modern AI systems are moving toward a strategy called <strong>Just-in-Time Context Enrichment</strong>.</p><p>Instead of loading everything into memory, the system:</p><ol><li>Detects user intent.</li><li>Retrieves only relevant tools.</li><li>Executes operations.</li><li>Filters unnecessary data.</li><li>Keeps the context lightweight.</li></ol><p>For example:</p><p>User:</p><blockquote><em>Fix my Docker container.</em></blockquote><p>The system dynamically loads Docker tools only.</p><p>Later:</p><blockquote><em>Deploy to Kubernetes.</em></blockquote><p>The Docker context is removed, and Kubernetes tools are loaded instead.</p><p>This reduces token usage and significantly improves accuracy.</p><h3>Best Practices for Context Engineering</h3><p>To build reliable AI agents:</p><ul><li>Keep system prompts concise.</li><li>Separate instructions from data.</li><li>Store memory outside the LLM.</li><li>Use RAG instead of huge prompts.</li><li>Send only relevant context.</li><li>Validate model outputs.</li><li>Add safety checks.</li><li>Summarize old conversations.</li><li>Keep environment state separate.</li><li>Limit unnecessary tokens.</li></ul><h3>The Golden Formula</h3><p>The quality of an AI system can be summarized by one equation:</p><pre>Final Quality =<br>    Model Quality<br>    × Context Quality<br>    × Tool Quality</pre><p>A powerful model with poor context will still produce poor results.</p><p>However, a smaller model with excellent context engineering can often outperform a larger model that lacks the right information.</p><p>In the future of AI agents, context engineering will not be optional, it will be the foundation that makes intelligent systems truly useful.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2cbec3cdb1fa" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>