Memory prefetching requests data ahead of a program’s use so it can arrive in cache while the processor is still doing other work. A demand load is the instruction that reads the value for the program. If a prefetch has already brought the relevant cache line close to the core, the demand load can complete sooner. In a market-data replay, this can reduce the time a scanning loop spends waiting for the next records. GCC’s prefetch documentation
Eight reads, one or eight lines
Suppose a contiguous array of at least 57 eight-byte values starts at address 0x1000, aligned to a 64-byte cache line. Number the lines from zero at the array base. Element starts bytes from that base, so it lies on relative line . Each element fits inside a single line under this alignment.
- Reading indices 0 through 7 advances eight bytes per read. All eight values come from one line, line 0.
- Reading indices 0, 8, 16, …, 56 advances 64 bytes per read. Each value comes from a different line, for eight distinct lines.
Each blue mark is an element requested by the program. The outlines group elements into cache lines.
The adjacent scan uses every byte of its line, an example of spatial cache locality. Both scans have a regular stride. Even the second scan visits consecutive line numbers, despite skipping seven values between reads. A line count comes from the addresses. A cache miss depends on whether the requested line is present at the cache level being measured when the load arrives. With the relevant lines already there, either scan could record zero misses at that level.
Which reads can arrive early?
A hardware prefetcher looks for patterns in recent accesses. AMD’s EPYC 9005 guide, for example, describes an L1 stream prefetcher that looks for sequential lines and an L1 stride prefetcher that tracks constant address steps for individual instructions. A software prefetch gives the processor a hint for a future address. GCC’s built-in can emit such an instruction on supported targets. The address pattern above shows what a prefetcher could exploit. Whether it detects the pattern and gets ahead in a short trace depends on the processor and workload. AMD’s cache prefetcher guide, GCC’s prefetch documentation
A useful prefetch overlaps the transfer with earlier work and leaves the line available when needed. The bytes still have to travel if the line is absent. Fetching unused lines consumes bandwidth and can evict useful data. Fetching a line already in the relevant cache offers no latency benefit. On a bandwidth-limited risk run, extra traffic can slow the job even if some demand loads hit sooner. To compare two versions, keep the input and outputs fixed and time the same workload under representative cache conditions and thread counts. GNU Project’s prefetching guidelines, SelfMadeQuant’s Profiling and Optimisation lesson (public preview)
Sources
- AMD, EPYC 9005 BIOS & Workload Tuning Guide, May 2025. §3.1.2, Cache Prefetchers.
- GNU Compiler Collection manual. __builtin_prefetch.
- GNU Project, Data Prefetch Support. Guidelines for Prefetching Data.