Translation lookaside buffer and huge pages

A translation lookaside buffer (TLB) is a small hardware cache of virtual-to-physical address translations. Reusing a translation can spare the processor a lookup through the page tables. A huge page is a memory page larger than the platform's base page. It can let the TLB cover more memory with fewer translations. In a low-latency trading system, that matters when, for example, a feed handler jumps among order-book records scattered over many pages. The bytes it reads per update may be small while the number of pages it repeatedly touches is large. Linux kernel documentation on page tables

In this toy example, every page holds 4 KiB, or $2^{12}$ bytes. The low 12 bits (three hexadecimal digits) specify the byte offset within the page. The higher bits specify the virtual page number (VPN). Suppose the current address space maps virtual page 0x12 to page frame number (PFN) 0x9A. The physical frame starts at 0x9A000.

For virtual address 0x12345, the byte offset is 0x345. Add it to the frame start to get physical address 0x9A345. Virtual address 0x12380 has the same VPN and a different offset, so it maps to 0x9A380. The second access can reuse the same translation if the entry still resides in the applicable TLB, is valid for that address space and permits the access. Intel System Programming Guide, §4.10.2

The same virtual address can refer to a different frame in another process. For ordinary process-private mappings, the processor must distinguish address spaces when reusing entries, for example through a process-context identifier on supported Intel processors. When the kernel changes a mapping, it may need to invalidate a stale entry so the next access sees the new mapping. Intel System Programming Guide, §4.10.3.2 · Linux kernel TLB flushing interfaces

A TLB miss means the translation was absent from the TLB level checked. A later TLB level can supply it on processors with several levels. If the translation is absent from the relevant levels, a processor with a hardware page walker can traverse the page tables and cache the result. A present, permitted mapping can therefore complete normally after a miss. A page fault is an exception, for example when a mapping is absent or when a write violates a page's permissions. On Linux, first access to lazily allocated memory can fault and be resolved by the kernel without disk I/O. Intel System Programming Guide, §4.10.3.2 · Linux kernel documentation on page faults

On a system with 4 KiB base pages and 2 MiB huge pages, a suitably aligned 2 MiB region can use one huge-page mapping instead of 512 base-page mappings, since $2\,\text{MiB}/4\,\text{KiB}=512$. An entry that covers the whole huge page covers 512 times as many bytes as a base-page entry. Actual TLB reach depends on how the processor caches each page size and how many entries it has for each. Sparse access can also waste memory. On Linux, allocating or clearing a larger page or compacting memory to make room can add latency, depending on the huge-page policy. Intel System Programming Guide, §4.10.2.3 · Linux transparent huge-page documentation

In a replay benchmark, keep the book-update sequence and measured work fixed across page-size settings. Separate startup faults from steady-state updates, verify how the buffer is mapped, and compare page-walk counts alongside the latency distribution. Linux's per-mapping AnonHugePages field in /proc/<pid>/smaps can confirm the amount of anonymous memory backed by PMD-level transparent huge pages. Counter meanings also depend on the processor. On Intel Meteor Lake P-cores, for example, a second-level TLB hit and a completed data-load page walk are distinct events, and a completed walk can end with or without a fault. Linux kernel proc filesystem documentation · Intel Meteor Lake P-core performance events