A concept bottleneck model predicts through named intermediate concepts. Trace an editable two-concept forecast and distinguish semantic leakage, intervention compatibility and real-world causation.
Concept bottleneck model
A concept bottleneck model predicts an outcome by first estimating named, human-understandable concepts, then using those estimates as the sole inputs to a target predictor. For a quant reviewing a filing-based risk classifier, the useful interface is a set of inspectable concepts—such as refinancing pressure and liquidity resilience—that can be corrected before recomputing the prediction. Those financial concepts are illustrative, not evidence of a successful deployed model.
There is no direct input-to-target bypass. Merely adding a concept-prediction head beside an ordinary predictor does not create this bottleneck. The classical supervised setup supplies concept annotations as well as target labels during training. See Koh et al., §§2–3.
Correct one concept, recompute the output
Consider a fictional binary classifier for a specified future credit-stress event. Its concepts are soft scores between zero and one: c1 represents refinancing pressure and c2 represents liquidity resilience. Their values and all weights below are stipulated; no model is fitted.
The target predictor has no intercept and uses
z=2c1−c2,p=σ(z)=1+e−z1.
Suppose the concept predictor returns (c^1,c^2)=(0.8,0.2). Then
z=2(0.8)−0.2=1.4,p≈0.80218.
An analyst identifies stronger liquidity support in the filing and replaces only the second score with 0.9. Holding the first score and target-predictor weights fixed gives
z′=2(0.8)−0.9=0.7,p′≈0.66819.
The negative coefficient makes the direction transparent: increasing the liquidity-resilience score reduces this model's stress probability. Whether the analyst's replacement is correct, and whether the resulting probability is empirically reliable, are separate questions. A sigmoid map alone does not establish probability calibration.
Fictional concept-only model routes raw input through two concepts with no bypass. Concepts 0.8 and 0.2 give logit 1.4 and probability 0.80218. Editing only the second concept to 0.9 gives logit 0.7 and probability 0.66819.
Original fictional concept-only predictor. The edited concept changes the model output while the first concept and target-predictor weights remain fixed; this is neither a fitted financial forecast nor a real-world causal effect.
A named coordinate can still hide information
A soft concept value contains more information than a binary label. Two observations assigned “concept absent” can receive different small scores, and a downstream predictor can exploit that variation for reasons unrelated to the named concept. This is concept information leakage, not necessarily future-data leakage. Mahinpei et al. show that it can occur even when the concept predictor is trained separately from the target predictor. Accurate concept classification therefore does not establish that every useful variation in its scores has the intended meaning. See Mahinpei et al., §3.
An incomplete concept vocabulary can impair prediction. Adding an unrestricted residual channel weakens the claim that named concepts explain the whole prediction.
Editing requires a compatible interface
Validate edited predictions: replacing soft estimates with hard labels or unusual combinations can shift the target predictor's input distribution. Respect its scale—probabilities, logits or continuous measurements. Related concepts may need coordinated edits. Concept accuracy alone does not guarantee useful interventions; see Koh et al., §6.
The edit changes a value inside the model. Concluding that increasing real liquidity reduces actual stress risk requires causal inference, beyond this predictive calculation.
For the numerical probability map, Logistic Regression and Maximum Likelihood develops the sigmoid and Bernoulli likelihood. The preview is public; the full lesson requires membership.
