Weak supervision learns from incomplete, coarse or noisy labels. Labeling functions scale annotation, but votes and label-model probabilities still need independent validation.
Weak supervision
Weak supervision is learning from labels that are incomplete, coarser than the prediction task, or potentially wrong. In financial-text research, it can turn a small set of analyst rules, dictionaries or existing records into training labels for a large filing corpus. The gain is cheaper supervision at scale; the remaining question is whether those labels capture the intended financial fact.
Three ways a label can be weak
The term is broader than combining heuristic votes. Zhou's survey distinguishes:
Weakness
Financial-text example
What is missing
Incomplete
Analysts label only a subset of disclosures
Labels for the remaining documents
Inexact, or coarse
An entire filing is marked as describing distress, while the task asks which sentences describe it
Location of the relevant evidence
Inaccurate, or noisy
A word rule mistakes “no material deterioration” for an adverse development
A reliable class assignment
These weaknesses can coexist. Semi-supervised learning addresses the particular setting of labelled and unlabelled data; it is one part of this wider landscape.
Write the labelling rule once
Programmatic supervision supplies labels through reusable procedures rather than annotating every item separately. In data programming, a labeling function (LF) takes an item and emits a class or abstains. Functions can use text patterns, external records or another classifier. They may overlap and disagree. Ratner et al. (2016) formalise this approach.
Suppose the task is to classify a sentence as reporting a realised adverse operating development. Use +1 for that class, −1 for other, and a separate abstention value when a rule offers no opinion. Here +1 means the target class, not positive sentiment.
A financial-word rule might flag “deterioration”; a phrase rule might recognise a reported plant closure; a context rule might reject purely hypothetical risk language. Their specifications must identify the issuer, event and relevant time. A sentence about a competitor's past closure should not automatically become evidence of this issuer's current deterioration. A financial lexicon is a possible ingredient, not a gold label. The Finance-Specific Lexicons lesson develops the document and horizon choices behind dictionary measurements (public preview; full lesson requires membership).
For two fictional sentences, the functions return:
Item
LF 1
LF 2
LF 3
Unweighted majority
A
+1
+1
−1
+1
B
+1
−1
Abstain
Unresolved tie
Count only non-abstaining votes. A has two target votes against one other vote. B has one of each; abstention is neither a negative vote nor evidence for a third class. A tie needs a declared policy, such as leaving the item unlabelled or sending it for review. The winning count is not an accuracy estimate.
Fictional votes: A has +1, +1, −1, giving majority +1. B has +1, −1 and abstention, giving an unresolved tie. Separately, +1, −1, −1 gives −1; replacing the first function with ten identical copies produces ten +1 votes versus two −1 votes and flips the majority without adding independent information.
Original fictional label votes. Abstention is separate from either class. A copied rule can dominate an unweighted vote even though its copies contain exactly the same source information.
Majority vote is a baseline; a label model is an inference
The original Snorkel paper separates three stages: apply LFs, fit a generative model of their outputs, then train a discriminative classifier using the resulting probabilistic labels. The label model treats the true class as latent and estimates source reliability from agreement patterns. The end classifier learns from the text or other features and may generalise beyond the explicit rules. That generalisation needs testing.
Unlabelled agreement does not identify arbitrary error mechanisms. In a binary latent-class model without an orientation constraint, swapping the hidden class meanings and the corresponding source behaviour can leave the observed votes unchanged. Estimation therefore needs assumptions that identify the intended solution: sufficient informative source overlap, restrictions on source quality or orientation, and an appropriate dependence structure. The data-programming paper's guarantees apply under specified modelling and learning conditions; they are not a guarantee for any collection of rules.
Dependencies are particularly consequential. Ten exact copies of one LF remain one information source. In a separate toy row, the votes +1,−1,−1 favour the other class. Replacing the first function with ten identical copies creates ten target votes against two other votes and reverses the majority. Nothing new was observed. Shared dictionaries, prompts or vendor records can create less obvious versions of this problem. Snorkel's original formulation explicitly models selected dependencies.
Check the implementation's assumptions. As checked on 1 November 2026, the official open-source LabelModel documentation, labelled version 0.9.7, states that its implemented model assumes LFs are conditionally independent given the true class. Do not transfer the original paper's dependency-handling claims to every current interface. Its API also reserves -1 for abstention and uses nonnegative class IDs; the toy class signs above are explanatory notation, not drop-in API values.
A fitted probability is an estimate conditional on the label model and its assumptions. It is not the truth, and apparent confidence can be driven by a shared mistake. Compare the label model and the final classifier with majority vote on independent reference labels; complexity earns its place through that comparison.
The financial validation file
Before generating a corpus of weak labels, define the unit and codebook: sentence or document, realised or hypothetical event, issuer attribution, negation, ambiguity and the permitted source evidence. Keep a human-coded development sample for rule revisions and a separate adjudicated gold test set that is not used to choose rules, prompts or thresholds. Preserve initial reviewer disagreements as well as the final adjudication. The LLM Annotation lesson explains the shared codebook and human-comparison discipline (public preview; full lesson requires membership).
For financial research, retain the original document version, release and ingestion timestamps, LF code, dictionary or database vintage, and model/prompt identity where applicable. An as-of reconstruction must use inputs available at the claimed decision time. A later restatement or a subsequently revised vendor label cannot silently become an earlier investor's information. Freeze rule development before the later test period and keep duplicates or closely related issuer disclosures from leaking across the chosen evaluation boundary.
Audit coverage, conflicts, unresolved items and per-class errors, including by period, industry and document source. Report probability calibration if probabilities are used quantitatively. A rule tuned on 10-Ks can fail on earnings-call questions; later language changes create concept drift. Neither majority agreement nor agreement with humans establishes that the codebook measures the intended construct. The Measurement Validity lesson develops that distinction (public preview; full lesson requires membership). A validated text label still requires a separate time-respecting economic test before supporting a return-prediction claim.
