Virtual Address Resolution: From TLB Hits to Demand Paging

How Virtual address generated by CPU are resolved by MMU to fetch data from hierarchy of CPU caches and computer memory

I have been trying to understand (once again) how cpu requests are served from memory and to build mental map of things happening at low level(Or at least low level for me as application developer). Adding my notes here from what I understood about topic from books, blogs and with help of AI tools which helped me draw flow diagrams and answering my queries.

Modern runtimes handle object allocation and garbage collection, the CPU hardware must physically route every memory read and write. Let us understand each component while we walk through the request flow. Quick note on virtual memory before we start. If you wish, skim thorugh it to next parts of article.

Virtual Memory

Why do we need virtual memory when physical memory is already there. There would be several issues if we use physical memory directly

• Security: Since multiple programs share the same memory space and run in parallel, there is nothing to prevent one program from accessing or modifying the memory of another program.
• Fragmentation : In a system without virtual memory, programs must be loaded as contiguous blocks of memory, each with a defined size. As programs are loaded and released, gaps between allocated memory regions begin to form as shown below. These gaps remain unused and waste memory
• Limited memory: Directly accessing RAM means that the total available memory is limited to the installed physical memory. Multiple processes might not be able to run due to limited memory.

Virtual memory solves this constraint. Each process is assigned an independent, isolated virtual address space. The CPU execution core operates strictly within this sandbox, completely oblivious to the underlying physical RAM layout. This isolation ensures stability and enables advanced OS features like memory mapping and demand paging. The illusion of this contiguous space is maintained entirely by the Memory Management Unit (MMU).

CPU always deal with virtual address of process not physical memory(RAM)

Diagnosing tail latencies and performance bottlenecks requires understanding the microarchitecture of this address resolution. Here is the mechanical breakdown of how the CPU resolves virtual addresses, transitioning from the nanosecond scale of hardware caches to the millisecond scale of disk I/O.

Read this article for CPU caches and how data flows between them and main memory :https://rednirus.medium.com/how-data-actually-travels-from-ram-to-your-cpu-c28b285c1779

1. The Frontline: VIPT and the L1 TLB

When an instruction requests memory, the pipeline’s Address Generation Unit (AGU) computes the Virtual Address (VA). The hardware splits this address into a Virtual Page Number (VPN) and a page offset. To mask translation latency, the CPU employs a Virtually Indexed, Physically Tagged (VIPT) architecture, executing two operations simultaneously:

• Cache Indexing: The page offset is routed to the L1 Data Cache as the Virtual Index, initiating the fetch of the specific cache line.
• Address Translation: The VPN is routed to the L1 Translation Lookaside Buffer (TLB), an SRAM structure physically adjacent to the core, acting as the MMU’s frontline cache.

If the L1 TLB holds the mapping, it immediately returns the Physical Frame Number (PFN). This physical tag is compared against the tag retrieved from the L1 Cache. A match confirms the data is correct, and the payload is delivered to the registers in just a few clock cycles.

2. Hardware Translation Caches and the Page Table Walk

The L1 TLB is small by design to maintain single-cycle latency. When a memory request misses the L1 TLB, the hardware falls back to the larger L2 TLB.

If the L2 TLB also misses, the hardware translation caches are exhausted. The MMU then activates the Hardware Page Table Walker (PTW). The PTW halts the requesting CPU thread, reads a dedicated control register (such as the CR3 register on x86 architectures) to locate the root of the page directory in physical memory, and issues sequential memory read requests to traverse the hierarchical page table structure.

Once the PTW locates the valid Page Table Entry (PTE), it reads the physical address, updates both the L1 and L2 TLBs, and resumes the stalled instruction.

3. The Slow Path: Interrupt Vectors and Page Faults

The most severe performance penalty occurs when the PTW successfully navigates the page tables but discovers the “Present” bit on the final PTE is set to 0. The memory page exists within the virtual address space, but it is not currently resident in physical RAM.

Get Surinder’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

This is a Page Fault. The hardware cannot proceed and raises a synchronous exception. The routing of this interrupt follows a strict hardware-defined path:

1. The CPU immediately suspends the current execution context.
2. It uses the specific fault number (e.g., Exception 14 on x86) as an index into the Interrupt Descriptor Table (IDT). CPU register contain the base address of IDT for quick lookups. Interrupt Handlers are a separate topic to discuss how ISR works and IDTs are used.
3. The IDT provides the memory address of the specific Interrupt Service Routine (ISR) designed to handle page faults.
4. The CPU vectors execution to this ISR, elevating privileges and transitioning from user mode to kernel mode.
5. The ISR reads the exact faulting virtual address from a specialized control register (the CR2 register on x86) and hands control to the OS Page Fault Handler.

Page fault and IDTR table

4. Disk I/O, DMA, and Hardware Communication

The OS Page Fault Handler determines where the missing page resides on the backing store (disk or swap file). To fetch this data, the kernel must communicate with the hardware storage controller using Memory-Mapped I/O (MMIO).

With MMIO, device control registers are mapped directly into the standard physical address space. The CPU executes standard memory store instructions to write command bits to these specific physical addresses. The disk controller detects these writes and initiates the read operation.

To move the data, modern systems rely on Direct Memory Access (DMA). The kernel allocates a free physical frame in DRAM and configures the DMA controller. The disk controller then writes the fetched data directly into this physical RAM frame over the system bus, completely bypassing the CPU. The CPU is free to schedule and execute other threads during this transfer.

DMA is another separate topic under I/O Communication and Addressing Schemes like port-mapped io/memory mapped IO. We will have another article for it

5. State Reconciliation and Instruction Restart

Once the DMA transfer completes, the disk controller fires an asynchronous hardware interrupt to signal the CPU. The ISR for the disk controller executes, notifying the OS that the page is ready in physical RAM.

The OS Kernel handles the final state update. This requires three distinct commands to reconcile the software state with the hardware caches:

1. PTE Update (Memory Store): The OS calculates the memory address of the specific PTE. The kernel executes standard memory store instructions to write the new Physical Frame Number (PFN) into the PTE and flip the “Present” bit to 1. The MMU is completely bypassed for this step; to the hardware, the kernel is simply writing data to RAM.
2. TLB Invalidation (Hardware Instruction): The MMU likely still holds a cached record that this specific virtual address is invalid. The OS executes a specific architecture-level instruction (such as INVLPG on x86 or TLBI on ARM). This command is routed directly to the MMU hardware, forcing it to flush the stale entry from the L1 and L2 TLBs.
3. Context Restoration (Interrupt Return): The OS executes a return-from-interrupt instruction (e.g., IRET on x86). This restores the CPU execution context and points the instruction pointer back to the exact instruction that originally triggered the page fault.

6. The Second Attempt

The CPU executes the restarted instruction as if the fault never happened. The AGU computes the Virtual Address. The L1 TLB checks its cache, but because the OS explicitly invalidated the entry using INVLPG, it results in a cache miss.

The request falls back to the L2 TLB (also a miss), activating the PTW. The PTW walks the page tables in DRAM. This time, it encounters the newly updated PTE, reads the valid PFN, caches the new translation in the TLBs, and returns the physical tag to the CPU core. The tag matches the cache, and the instruction finally completes.

I asked Gemini to generate the diagram from paper draft sketch. Here is the end to end flow

Hope you found it useful. Until next time, good bytes!

Source : https://www.youtube.com/watch?v=B6tJxvYBNrU
Some of the pictures used are from his books I purchased online