A data catalogue is a searchable, curated register of metadata describing datasets: what they contain, where to obtain them, who is responsible for them and under what conditions they can be used. For a quant researcher, it helps answer a question that a filename cannot: does this particular history support the decision I want to test?
The catalogue describes the data. Storage holds the observations; a query engine retrieves them; access controls enforce permissions. A catalogue can connect these services without replacing them. For example, AWS Glue describes its Data Catalog as a metadata repository containing source locations, schemas and properties. A model inventory registers models and their governance state; a dataset catalogue registers inputs that several models may share.
What a useful quote record says
Consider a fictional catalogue entry for venue-level equity quotes. These are illustrative metadata values and requirements, not a description of a deployed service or a real vendor agreement.
| Record field | Fictional entry and the research question it answers |
|---|---|
| Identity and location | Dataset venue-quotes; immutable snapshot quotes-v7; authenticated storage reference. Which exact observations should the run load? |
| Responsible owner | Market-data steward, with a maintained contact and escalation route. Who resolves a missing session or disputed field definition? |
| Licence and use restrictions | Internal research permitted for the licensed team; raw redistribution prohibited; external derived-output use unconfirmed. Link the applicable agreement and its effective period. |
| Coverage | Top-of-book quotes from fictional venue VX, instrument list attached to this snapshot, UTC dates 1–5 October 2026. No claim of consolidated-market coverage. |
| Clock meanings | event_time is the venue event timestamp; usable_time is when this research system has received and completed required processing of that record. Both use UTC. |
| Schema and meaning | schema-v3: stable instrument ID, venue, sequence, bid/ask prices in USD per share, sizes in shares, and both clocks. Document nulls, corrections and sequence resets. |
| Lineage | Raw delivery ID → normalisation rule normalise-v3 → snapshot quotes-v7, with links to transformation and input records. |
| Versions | Catalogue entry metadata-revision-4; schema schema-v3; data snapshot quotes-v7. A metadata edit does not imply that observations changed. |
| Quality evidence | Link checks for duplicate keys, sequence gaps, invalid sizes and crossed quotes, with results, run time and snapshot ID. Exceptions retain their reasons. |
| Retention and expiry | Link the applicable retention/deletion schedule, contract-expiry review and treatment of backups. Separately record whether a retained copy may be reused. |
The licence entry is a summary linked to evidence, not a new grant of rights. A successful download does not settle financial data licensing, and an audit-retention permission does not automatically authorise another backtest. Restrict access to sensitive catalogue metadata too: locations and coverage descriptions can themselves reveal confidential research activity.
The structure is an original research-oriented example. For interoperable descriptions, W3C's DCAT 3 distinguishes datasets, their distributions and catalogue records, and supplies properties for rights and versions. Its catalogue-record dates describe the registration, rather than necessarily the underlying dataset. DCAT 3 is a W3C Recommendation dated 22 August 2024; a team can use these distinctions without claiming that a simple table is a conforming DCAT implementation.
A catalogue cannot manufacture historical availability
Suppose one quote in the fictional snapshot has an event timestamp of 09:30:00 UTC and a usable timestamp of 09:30:02 UTC. The strategy decides at 09:30:01 UTC, on the same day. Receipt and required processing take two seconds in this example; the row becomes usable one second after the decision.
An event-time filter would include it. A replay of this system must exclude it. Using the earlier clock would introduce look-ahead bias, even if the catalogue entry was complete and the schema was correct.
Original fictional record and timing example. A catalogue locates and describes a snapshot; the row-level usable timestamp determines historical eligibility.
For a point-in-time dataset, the register should identify how historical availability, revisions and instrument membership are preserved. Calling a field “timestamp” or tagging the dataset “PIT” supplies no such evidence. If the source overwrote an earlier value, adding a catalogue description today cannot recover the missing vintage. A schema crawler can discover column names and types; the venue clock's economic meaning and the team's actual historical access need separate verification.
From search result to reproducible input
Start with the intended use: venue, instruments, period, fields, decision deadline and permitted purpose. Use the catalogue to find candidates, then inspect the referenced contract, sample rows, timestamp rules and quality reports. Coverage is a claim to reconcile against actual sessions and identifiers. A passed duplicate check does not establish that every expected instrument arrived, and a crossed quote may need contextual investigation rather than automatic deletion.
Bind a research run to the resolved snapshot, transformation versions and applicable permission record. Preserve the relevant catalogue revision so a later reader can reconstruct what the entry said. A pointer named
latest is convenient for discovery but insufficient to identify an old experiment. A checksum can establish byte identity against a recorded digest; it cannot establish that those bytes describe the right market or information set.Existing teaching takes the next step: Data lineage follows vendor inputs through features to weights, while Point-in-Time Data develops historical version selection and snapshot audits. Both have public previews; the full lessons require membership.
Sources
- W3C, Data Catalog Vocabulary (DCAT) — Version 3, Recommendation, 22 August 2024; §§6.3, 6.5, 6.8.5–6.8.7 and 11. Checked the distinctions between catalogue, catalogue record and dataset, distribution rights, and version relationships.
- AWS Glue, Data discovery and cataloging, checked 29 October 2026; introduction, “Metadata repository” and “Schema management.” Used for the concrete separation between source data and catalogue metadata. The quote record and timing diagram above are original fictional examples.