A system call is a request from a user-space program for the operating-system kernel to perform an operation, such as reading from a file descriptor. A context switch, in the scheduler sense, occurs when the CPU stops running one thread and starts or resumes another, saving and restoring execution state as needed. A thread can enter and leave kernel mode while remaining the current thread throughout. In a low-latency trading system, this distinction helps explain whether a delayed feed handler was executing, waiting for an event, or ready but waiting for a CPU. Linux kernel documentation on syscall entry and exit, Linux scheduler tracepoint definitions.
Two reads on one core
Suppose feed-handler thread A runs on a single core alongside runnable thread B. Use a Linux event notification descriptor, created with
eventfd, to model A waiting for a notification. With an empty nonblocking event descriptor, A's first read enters the kernel and fails with EAGAIN without waiting for a notification. In this trace, A remains on the CPU through the return, so the first call incurs zero task switches. Linux man-pages, eventfd(2).For the second
read, A uses a separate, empty descriptor in blocking mode. A enters the kernel, finds no notification, and blocks. The scheduler selects B, the first task switch. When the notification arrives, the kernel wakes A. A is now runnable, meaning eligible to run, while B still has the CPU in this example. B later yields and the scheduler selects A, the second task switch. A resumes the pending read in kernel mode and then returns to user space. The choice of A after B yields is part of this schedule. On Linux, the outcome of yielding depends on scheduling policy and which threads are runnable. Linux man-pages, sched_yield(2).
The CPU lane changes only when the scheduler hands the core to a different task. A's wakeup and subsequent return to user mode appear separately. Horizontal spacing shows event order.
Nonblocking I/O lets the thread continue when the resource is unavailable. The scheduler can still preempt that thread, whether or not a system call is in progress. The switch counts above follow from the stipulated order of events. Linux man-pages,
sched(7).Separate waiting from running in a trace
On Linux, pair syscall entry and exit for the thread of interest, then look for its scheduler events. A long entry-to-exit interval is elapsed time between tracepoints. When the call blocks, this can include both time spent waiting for the notification and time spent runnable before the scheduler selects the thread. Linux kernel documentation on syscall entry and exit, Linux ftrace documentation on wakeup latency.
- Use
sched_switchto identify the outgoing and incoming thread IDs and inspect the outgoing state to distinguish a blocking switch-out from a thread that is still runnable. - Find
sched_wakeupfor the blocked thread and the latersched_switchthat selects it. The interval between them estimates wakeup-to-dispatch delay, one possible contributor to latency jitter. The switch tracepoint marks the scheduling handoff before the actual switch completes. The syscall return can occur later. - When events come from different CPUs, use a trace clock comparable across those CPUs. Check for dropped or overwritten records before drawing conclusions from missing events.