Stabilise Gaussian class covariance estimates with pooled and spherical shrinkage, follow a checked example, and distinguish parameter conventions and probability calibration.
Regularised discriminant analysis
Regularised discriminant analysis (RDA) is a Gaussian classification method that stabilises class covariance estimates by shrinking them toward a pooled covariance and toward a spherical matrix. It gives a quant researcher a way to retain some class-specific feature geometry without trusting every covariance estimated from a small class sample.
For example, a classifier for liquidity conditions might use spread, depth and volatility features. Separate covariance matrices can capture different relationships in calm and stressed observations, but the stressed class may contain relatively few examples. RDA trades some of that flexibility for a more stable estimate. It does not establish that those labels are economically useful or that the Gaussian model fits market tails.
Two controls, with an explicit convention
Let there be K classes, p features and n=∑k=1Knk training observations. For class k, estimate its mean μk and sample covariance using the divisor nk−1:
Sk=nk−11i:yi=k∑(xi−μk)(xi−μk)⊤,nk>1.
The pooled within-class covariance is
Spool=n−K∑k=1K(nk−1)Sk.
It centres each observation around its own class mean. Taking the covariance around the overall mean would also incorporate differences between class means and would be a different matrix.
This article uses direct covariance mixing, followed by scaled-identity shrinkage:
Increasing λ makes the classes share more of their covariance structure. Increasing γ makes each resulting matrix more spherical: off-diagonal covariances move toward zero and variances move toward their average. This is covariance shrinkage used inside a classifier. The direct-mixing form is documented in the R klaR reference, More details.
Check the parameterisation before copying settings. Friedman's original formulation first mixes class and pooled scatter matrices and their sample weights, then divides. Its λ therefore need not equal the direct covariance weight above. With two equally sized classes and consistent divisors, original scatter-mixing λ=1/3 corresponds to direct covariance weight 1/2. The methods share endpoints, but an identical parameter label is insufficient to reproduce an intermediate fit. Friedman (1989), equations (16)–(18).
A checked two-feature example
Consider fictional training covariance summaries for two equally sized classes. Features have already been expressed in comparable units:
S1=(1009),S2=(9001),Spool=5I.
At direct covariance weight λ=0.5,
A1=0.5S1+0.5(5I)=(3007).
Its average variance is still (3+7)/2=5. Taking γ=0.2 gives
Σ1=0.8(3007)+0.2(5I)=(3.4006.6).
For class 2 the diagonal entries are reversed: 6.6 and 3.4. The class 1 spectral condition number falls from 9 to 6.6/3.4≈1.94, while its trace stays at 10. These are consequences of the stipulated blend, not evidence that these hyperparameters minimise classification error.
Three unit Mahalanobis contours for fictional class 1: diagonal variances 1 and 9, then 3 and 7 at lambda 0.5, then 3.4 and 6.6 at gamma 0.2. A dashed circle shows pooled variances 5 and 5; class 2 swaps the feature axes.
Original fictional calculation. Solid contours show the two shrinkage stages; the dashed circle is the pooled reference. Contours have squared Mahalanobis distance one and are not labelled probability coverage regions.
What the endpoints mean
The table holds class means and priors fixed and assumes the matrices needed for scoring are invertible.
Spherical covariance for each class; scales can still differ between classes
λ=1,γ=1
Common tr(Spool)I/p
Nearest Euclidean class mean when priors are equal
A spherical matrix has equal variances across features within that class. A merely diagonal matrix can retain different feature variances. Neither phrase should be substituted for the other. When the spherical scale differs across classes, the boundary can remain quadratic.
Software may use another shrinkage target. For example, scikit-learn's QDA reg_param is documented as shrinking its class scaling toward an unscaled identity, without this separate pooling control. It is not a drop-in name for γ in the formula above. Official QDA parameter reference.
The covariance is only part of the classification rule
For positive prior probabilitiesπk summing to one, RDA chooses the class with the largest Gaussian log score:
The final term measures distance relative to class geometry. The log determinant accounts for density being spread over a larger volume; the prior accounts for baseline class prevalence. Omitting either can change the predicted class. This score is equivalent to minimising Friedman's equation (19), after multiplying by −2. Original classification rule.
Under the fitted Gaussian model, class probabilities follow by normalising exp(δk). Compute this with a stable log-sum-exp calculation. Training class proportions are a possible prior estimate, but deliberately oversampling stress observations does not make their training frequency the deployment prevalence. Record the prior choice separately from the covariance settings.
Stability needs an honest evaluation
A class covariance has p(p+1)/2 distinct entries. After estimating the class mean, its rank is at most nk−1, so it is necessarily singular when nk≤p. Even when invertible, small estimated eigenvalues can make inverse-covariance distances unstable. Pooling helps only if the pooled estimate contains the missing directions; its rank is at most n−K.
For a positive-semidefinite Ak with positive trace, any γ>0 makes the stated spherical-shrinkage estimate positive definite: every eigenvalue receives a positive contribution γtr(Ak)/p. If the trace is zero, this target provides no repair. Numerical invertibility also does not guarantee useful class separation or protection from outliers.
Select λ, γ and any feature processing using cross-validation within the development data, keeping the final test untouched. For nested cross-validation, repeat tuning separately within each outer training split and assess only on its held-out outer fold. Within each training fold, refit scaling, imputation, class means, covariances and data-estimated priors. Spherical shrinkage depends on feature units: changing one feature from decimals to basis points changes the target geometry unless the preprocessing convention compensates. Apply each fitted transformation unchanged to its held-out fold. The official preprocessing guidance explains this leakage control.
For market time series, use splits that respect information availability and overlapping labels. A random split can let future observations influence a historical forecast. The SelfMadeQuant lesson Time-Ordered Train, Validation and Test Design develops this workflow; its preview is public and the full teaching requires membership.
Finally, classification accuracy and probability calibration are different checks. Gaussian model probabilities can be overconfident under misspecified distributions or changing regimes. Assess reliability on held-out predictions; if recalibration is needed, fit it on predictions from observations excluded from the underlying classifier's fit, and reserve a further evaluation set. The official calibration guide explains the separation and reliability diagrams.
Two classes have the same zero mean and equal priors in two dimensions. Their fitted covariances are I and 4I. For x=(1,0)⊤, which class has the higher Gaussian score? Both covariances are already spherical.
Answer. Class 1. After dropping the common log prior, its score is −1/2. Class 2 has squared Mahalanobis distance 1/4, but determinant 16, so its score is −21log16−81≈−1.5113. Choosing the smaller Mahalanobis distance alone would incorrectly select class 2. Spherical shrinkage does not remove the determinant term when class scales differ.
