Polars is a DataFrame library with a column-oriented engine written in Rust. A lazy dataframe, called a LazyFrame in Polars, represents a query plan. If you need to attach reference quotes to millions of trades, you can describe the file scans, column selections and join together so Polars can plan the work before it builds the result. Polars user guide
Which quote was available at the trade?
Suppose one instrument has trades at 10 and 14 seconds from the session start and quotes at 9, 11 and 15 seconds. For this example the quote times are when the prices became usable. Sort both inputs by time, then use a backward as-of join. Keep each trade and attach the last quote whose time is at or before the trade time. With multiple instruments, group the match by instrument and keep the time keys sorted within each group. An unmatched trade keeps a null quote, and a tolerance can rule out stale matches. Polars join_asof reference
| Trade time (s) | Matched quote time (s) | Quote price (USD) |
|---|---|---|
| 10 | 9 | 100 |
| 14 | 11 | 102 |
A backward join uses the latest eligible quote for each trade. The quote at 15 seconds is closest to the second trade but arrives too late.
On a real feed, a quote stamped by the exchange before a trade could arrive after it. Matching on the exchange timestamp alone can introduce look-ahead bias. Use the time the quote became available to the process you are reconstructing, and check event order when timestamps tie. The Point-in-Time Data lesson works through availability rules and missing matches. The full lesson requires membership.
What the lazy plan changes
With eager operations, each call produces its result immediately. In the lazy version, the scans, sorts,
join_asof and final column selection form one plan. explain shows the plan so you can inspect where a filter or column selection lands, and collect executes the query and returns the result as a DataFrame. Calling lazy on an in-memory DataFrame defers the downstream operations. Starting with scan_parquet also gives the optimizer control over the file read. Polars lazy API guideProjection pushdown means requesting only the needed columns at the scan, including instrument and time keys even if the final output drops them. Predicate pushdown means moving an eligible row filter into the scan. A Parquet reader can use file statistics to skip data that cannot pass the filter, so savings depend on the query and the file layout. When restricting the trade window, retain the quote history needed for the first trade. Cutting both inputs at 10 seconds would leave the first trade unmatched. Polars Parquet scan reference