Separate connection and worker liveness from instrument freshness and feed completeness, using explicit clocks, deadlines and recovery evidence.
Heartbeats and staleness detection
Heartbeats and staleness detection are checks that ask whether a system is still communicating or making progress, and whether its information is recent enough for a specified use. A heartbeat is an expected liveness signal; a staleness check compares an observation's age with a declared freshness rule. In trading, a connection can keep answering while the strategy's price input stops updating. Treating those as one “healthy” indicator can let decisions consume old market state.
Which part is alive?
A heartbeat only speaks for the component and path that produces it. A transport response, a feed-server message and a strategy worker's progress report carry different evidence.
Check
Evidence it can provide
What remains unproved
Connection ping and matching reply
The remote endpoint responded over that connection
The intended subscription is active or market data are being processed
Application heartbeat
The component generating the message was able to send it
A separate pricing worker, book builder or downstream consumer is progressing
Processing-progress marker
A named worker advanced its validated sequence or completed work
The input represents a current, complete market state
Instrument-data freshness
The instrument's last accepted update or confirmation satisfies the age rule
The absence of gaps, correct prices or future execution availability
For a concrete protocol distinction, RFC 6455 §§5.5.2–5.5.3 defines WebSocket Ping/Pong control frames: a Ping can test endpoint responsiveness, and an unsolicited Pong can serve as a one-way heartbeat. These are different from an exchange's application messages carrying product identifiers and market metadata.
For example, Coinbase Exchange's heartbeat channel, checked on 31 October 2026, documents per-product messages every second, with sequence numbers and last trade IDs. Its matches-channel documentation explains how the last trade ID helps detect missed trades. That documented interval belongs to this channel; it is not a general trading-system timeout.
A heartbeat emitted by an independent background thread can continue while the actual data worker is stuck. If the objective is to monitor that worker, include a meaningful progress marker and identify where it is updated. Conversely, a worker need not manufacture progress when there is legitimately no work. Check expected activity as well as elapsed time.
One monitor, three ages
Consider an original fictional capture, using UTC clocks assumed perfectly synchronised. Times below are seconds after 10:00:00. The market event occurs at 0.00 s, reaches the consumer at 0.05 s, and is immediately validated. A heartbeat is received at 5.00 s. The monitor checks at 5.10 s.
For check time t, latest relevant market-event time e, its local receipt time r, and latest heartbeat receipt time h, define
Aevent=t−e,Areceipt=t−r,Aheartbeat=t−h.
The three ages are 5.10 s, 5.05 s and 0.10 s, respectively. The first asks how old the event is; the second asks how long it has been since this market message arrived; the third asks how recently the liveness signal arrived. Receipt age here tracks the instrument's market message, not every byte received on the connection.
Declare two toy rules: flag data as old when event age is strictly greater than 2 s, and let the heartbeat freshness check pass when heartbeat age is at most 1 s. At the check, the data-old flag is set while the heartbeat check passes. These thresholds demonstrate the logic; they are not calibrated service requirements or venue rules.
Fictional zero-error UTC capture checked at 10:00:05.100: market event age 5.10 seconds exceeds the toy 2-second limit; local receipt age is 5.05 seconds; heartbeat receipt age 0.10 seconds meets the toy 1-second limit.
Figure 1. Independent ages at the same check. The recent heartbeat does not refresh the market event or establish that the feed is complete.
The event-to-receipt delay was 0.05 s. That delay and the current event age are different measurements: a message delivered promptly can subsequently sit unused or unchanged for a long time. Equally, a newly received message can already be old because of upstream buffering. Preserve source-event, local-receipt and post-validation availability times rather than relabelling arrival as a new market event. The Observation, Event and Availability Timestamps lesson develops these timing boundaries; its preview is public and the full lesson requires membership.
Old does not automatically mean missing
A stale quote concerns whether a quote remains suitable for use. An old last-trade timestamp can instead describe a genuinely quiet instrument. A valid standing quote may remain unchanged, and a refresh can confirm the same price. Price movement is therefore a poor substitute for an update or confirmation timestamp.
Before interpreting silence, identify the instrument, venue, channel and subscription state. Use the relevant trading calendar, session phase, halt status and expected message behaviour. A closed session can explain no trades without explaining the disappearance of a heartbeat promised throughout a connected session. Traffic in another instrument cannot refresh this instrument's record. Missing timestamps or an uninitialised subscription should be represented as unknown, rather than assigned age zero.
A market-data gap asks a separate question: were required messages missed? Coinbase Exchange's sequence-number documentation explicitly warns that upstream delivery can leave gaps even over a WebSocket connection using TCP. Apply each channel's sequencing and recovery contract. A stream filtered to selected message types need not expose a contiguous sequence of every underlying event.
In a fictional stream that explicitly promises consecutive sequence numbers for every delivered event, observing 102 followed by 104 leaves one required event unaccounted for. A fresh heartbeat does not supply event 103. Nor should an out-of-sequence message blindly replace newer validated state merely because it arrived most recently. Detect gaps and reordering, then follow the feed's documented reconciliation or snapshot procedure before trusting an incremental order book.
Make the deadline and response meaningful
Use a compatible local monotonic clock for elapsed time since receipt: Python's time.monotonic() documentation specifies a clock unaffected by system-clock updates, with an arbitrary origin whose differences are meaningful. Do not subtract a monotonic counter from an exchange UTC timestamp. Comparing source-event age across machines requires timestamp synchronisation, known timestamp semantics and an allowance for clock uncertainty. A negative apparent delay is a reason to investigate the clocks or record, not evidence of instantaneous delivery.
Choose deadlines from the promised cadence, processing and network variability, market state, decision horizon and consequences of using uncertain information. A timeout establishes that expected evidence was not observed by the deadline. It cannot by itself distinguish a failed process from a delayed network or an overloaded monitor. An independent supervisor should also detect when the monitor itself stops running.
An actionable alert identifies the affected worker or instrument, observed ages and sequence state, the rule breached, its owner, and the evidence required to recover. The response depends on what the strategy needs: mark an input unavailable, suppress dependent signals, restrict affected new orders, or escalate for reconciliation. Restoring a connection alone does not establish that subscriptions, books and consumers have caught up. Preserve the incident history and verify the required state before clearing the alert.
The Live Execution Monitoring lesson connects data-validity alerts with outstanding-order reservations and controlled recovery. Its preview is public; the full lesson requires membership. That distinction matters during a stale-data incident: stopping new submissions does not cancel existing orders or remove positions.
