Probit regression models the probability of a binary outcome by passing a linear index through the standard normal cumulative distribution function. A credit analyst can use it to estimate the probability of a covenant breach over a fixed horizon while keeping predictions between zero and one.
For and predictors , including an intercept,
Here is the CDF of the standard normal distribution. The probit link is the inverse relationship, ; is the inverse link that converts the index into a probability. A zero index gives 50%. Finite indices give probabilities strictly inside . This is a Bernoulli generalised linear model with a probit link. The statsmodels link documentation states this inverse-CDF convention.
A coefficient moves the index, not the probability by a fixed amount
Consider a fictional lender's one-month covenant-breach model. Let mean at least one breach during the next month and be a stress score, measured in arbitrary score units and available when the forecast is made. For illustration, stipulate
These coefficients are teaching inputs, not estimates from borrower data. The same breach definition and horizon apply to both profiles:
| Stress score | Linear index | Predicted breach probability |
|---|---|---|
The one-unit increase changes the index by , but changes the probability by
or 9.1848 percentage points, not 50 percentage points.
Figure: Original fictional scenario. The upper panel shows probabilities; the lower panel shows their local rate of change. The constant coefficient 0.5 produces neither a constant slope nor a constant finite probability change.
For a continuous predictor entering the index linearly without interactions, the marginal effect is
where is the standard normal density. At , this is : approximately 12.10 percentage points per score unit locally. The slope falls as moves from 4 to 5, so multiplying the starting slope by one overstates the exact change. A derivative describes an arbitrarily small movement; a finite change requires evaluating both probabilities. Stata's marginal-effects explanation makes this tangent-versus-curve distinction explicit.
For a binary predictor, compare its predicted probabilities at 1 and 0, holding the other predictors fixed. Differentiating through a nonexistent intermediate category is not the relevant comparison. With squares or interactions, differentiate the whole index. An average marginal effect averages observation-specific slopes; evaluating the slope at average predictors generally gives a different answer. These distinctions are set out in SAS's marginal-effects note.
Why the latent noise has variance one
An equivalent representation introduces an unobserved variable:
Only the sign is observed. If instead the conditional noise standard deviation were , the probability would be . Multiplying both and by a positive constant leaves probabilities unchanged. Only their ratio is identified; setting fixes the scale. The zero threshold is absorbed into the intercept. This is a normalization, not evidence that an economic shock has variance one. It also does not require normally distributed predictors. Schmidheiny's binary-response notes, §3 derive this identification issue.
Fitting and checking the model
For conditionally independent observations, the Bernoulli likelihood is
Maximum likelihood estimation chooses coefficients to maximize
Numerical optimization fits this objective; ordinary least squares is not the probit fitting rule. Separation can prevent a finite optimum: a predictor or combination of predictors may perfectly distinguish outcomes, driving some coefficients toward infinity. Inspect convergence and identification warnings. Repeated borrower observations also require inference that accounts for dependence. The Stata manual's model-identification and methods sections discuss these issues.
A normal CDF guarantees valid probability bounds, not probability calibration. On held-out observations, compare predicted probabilities with realized event frequencies; use log loss or Brier score alongside a calibration curve. These scores assess more than calibration alone, as the scikit-learn calibration guide explains. In the lender example, test on later forecast dates using only information actually available at each date. A score incorporating a subsequently reported breach would leak the target into the predictors.
Finally, a positive fitted coefficient describes a conditional association. It does not establish that intervening to raise the stress score causes breaches: the score may proxy for omitted financial distress. Causal inference needs an identification argument beyond the probability link.
Probit or logit?
Logistic regression uses the logistic CDF instead of the normal CDF. Both give smooth binary probabilities, but their tails and coefficient scales differ. Compare fitted probabilities and probability changes rather than raw coefficients; a probit coefficient has no constant odds-ratio interpretation. The two links need not give identical predictions, especially in the tails. Schmidheiny's §2 comparison shows the scaling difference.
For a worked derivation of the shared Bernoulli-likelihood logic using the other link, see Logistic Regression and Maximum Likelihood (public preview; full lesson requires membership).
Practice: average the slopes or the predictors?
In the fictional model above, a two-profile sample has stress scores 2 and 4 with equal weight. Is its average marginal effect equal to the marginal effect at the mean score of 3?
Answer. No. The two slopes are and . Their average is , or 16.02 percentage points per score unit. At the mean score, the slope is , or 17.60 percentage points per score unit. Averaging inputs before a nonlinear transformation does not generally reproduce the average transformed value. Neither quantity is an exact one-unit probability change.
Sources
- statsmodels developers. Probit link, inverse-CDF link convention.
- StataCorp. Probit regression, description, model identification, and methods and formulas.
- Kurt Schmidheiny, University of Basel. Binary Response Models, §§2–5, Fall 2025.
- StataCorp. Marginal effects of probabilities greater than 1, derivative interpretation and finite-change caveat.
- SAS. Marginal effect estimation for predictors in logistic and probit models, continuous, categorical, average and higher-order effects.
- scikit-learn developers. Probability calibration, calibration curves and scoring-rule interpretation.