Joint GLS estimation for equations with correlated errors: the system covariance, feasible estimation, and why identical regressors leave OLS coefficients unchanged.
Seemingly unrelated regressions
Seemingly unrelated regressions (SUR) jointly estimate several linear equations whose errors may be correlated for the same observation. A quant might model returns on two strategies using different exposures while allowing a shared unobserved shock. SUR applies generalised least squares to that system. It can improve estimation precision under the right covariance model, but correlated errors alone do not guarantee a gain.
The revealing exception is a pair of equations with exactly the same regressors: unrestricted SUR produces the same coefficients as separate ordinary least squares, even when their errors are strongly correlated. Joint inference can still matter. Stata’s examples demonstrate this distinction.
What connects the equations?
Write equation j, for j=1,…,m, as
yj=Xjβj+uj.
All equations here use the same n matched observations; each Xj has full column rank. Assume exogeneity with respect to the entire system’s regressors:
E[uj∣X1,…,Xm]=0.
For this standard SUR specification, the row error vectors εt=(u1t,…,umt)⊤ are conditionally independent across rows, with a common positive-definite covariance matrix Σ. Thus errors may move together across equations at row t, while there is no error covariance between different rows. Normality is unnecessary for the known-covariance linear unbiased efficiency result. MathWorks’ model description distinguishes within-row correlation from independence across rows.
Stack by equation: all observations for equation 1, then all for equation 2, and so on. With Z=diag(X1,…,Xm), the system becomes
y=Zβ+u,Ω=Var(u∣X1,…,Xm)=Σ⊗In.
The Kronecker product means block (i,j) of Ω is σijIn. Stacking by observation instead would change the matrix ordering. The linearmodels mathematical documentation uses the equation-stacked formulation.
With this correctly specified covariance, exogeneity and full rank, GLS has the smallest covariance among linear unbiased estimators of the system coefficients, in the positive-semidefinite ordering. This is an estimator comparison under repeated sampling, not a promise that each realised estimate is closer to its true value.
A correlated system that leaves the fits unchanged
Consider fictional responses in arbitrary units and a shared predictor t:
t
y1t
y2t
0
1
0
1
2
1
2
2
0
3
4
2
Both equations include an intercept and t. Stipulate independent row errors with known covariance
Σ=(10.80.81).
This is an assumed error covariance, not an estimate from these four rows. Separate OLS gives
y^1t=0.9+0.9t,y^2t=0+0.5t.
Solving the eight-observation stacked GLS problem gives exactly the same two fitted equations.
Fictional four-row example: OLS and SUR share intercepts 0.9 and 0, and slopes 0.9 and 0.5, despite known error correlation 0.8. Their slope-difference variance is 0.08; ignoring cross-equation covariance gives 0.40.
Identical regressors leave unrestricted SUR and OLS coefficients equal. Cross-equation covariance still matters for comparing their slopes; all variances here use the stipulated, known Σ.
Why the equality is exact. Put the response columns in Y and coefficient columns in B, so Y=XB+U. The GLS first-order condition is
X⊤(Y−XB^)Σ−1=0.
Multiplying by the nonsingular Σ leaves the ordinary least-squares normal equations, matching the common-design cancellation in Sørensen’s lecture notes (pp.3–4). Therefore
B^=(X⊤X)−1X⊤Y.
This requires a common full-rank design and the same matched sample, with no restrictions tying coefficients across equations. Merely calling two regressions a “system” does not add information about their separate slopes.
What joint inference retains. Here ∑t(t−tˉ)2=5. Under the stipulated covariance, each slope estimate has variance 1/5=0.20, and their covariance is 0.8/5=0.16. Consequently,
The estimated difference is 0.4, with standard error0.08≈0.283. Ignoring their covariance would instead give 0.40≈0.632. These are known-covariance calculations, not residual-based significance tests on four observations. Separate OLS plus a correctly assembled joint covariance can recover the same comparison; SUR is not uniquely required.
Estimated covariance and the limits of pooling
In applications Σ is usually unknown. A common two-step feasible GLS (FGLS) procedure fits each equation by OLS, collects row residual vectors ε^t, then sets
Σ^=n1t=1∑nε^tε^t⊤.
Replace Ω by Σ^⊗In in the GLS formula. This divisor-n convention differs from small-sample adjustments. The fitted covariance must be invertible. Consistent covariance estimation and suitable regularity conditions can give FGLS the same asymptotic efficiency as known-covariance GLS; they do not confer its finite-sample BLUE guarantee. The common-design algebra above also applies to any invertible Σ^, so estimating the covariance does not break that coefficient equality.
Different designs can allow cross-equation error correlation to improve precision, but they do not ensure a strict gain for every coefficient. If Σ is diagonal, GLS separates into OLS equation by equation in this model, even with different regressors. Near-singular estimated covariances can make feasible estimates unstable.
For financial time series, serial dependence or changing conditional variances invalidate the simple Σ⊗In covariance. Appropriate covariance modelling or robust system inference is then needed. Endogeneity is a different problem: weighting correlated errors does not make regressors exogenous or identify causal effects.
For the underlying transformation argument and the limits of feasible weighting, see Generalised Least Squares (public preview; full lesson requires membership).
MathWorks, Implement Seemingly Unrelated Regression, opening model description and common-predictor example. Exogenous predictors and the distinction between within-period correlation and across-period independence.
