Multivariate linear regression: Definition & Examples

Multivariate linear regression models several response variables using a common set of predictors. A quant might model two portfolios' excess returns against the same factors, retaining how their unexplained returns move together. Each response has its own coefficients; the joint model also describes dependence between their errors.

“Multivariate” counts responses. Multiple linear regression counts predictors: one return regressed on market, size and value is multiple regression. Several returns regressed on those factors are both multiple and multivariate. Here, even one nonconstant predictor will suffice to illustrate the multivariate distinction.

One design, several coefficient columns

Stack $n$ observations of $q$ responses into $Y$ and the common predictors into $X$ :

Y_{n\times q}=X_{n\times k}B_{k\times q}+U_{n\times q}.

Here

k

counts all design columns, including an intercept if present. A column of

B

contains one response's coefficients; a row contains one predictor's coefficients across responses. Responses and predictors must refer to the same observational units and aligned dates.

With full column rank in

X

, ordinary least squares gives

\hat B=(X^\top X)^{-1}X^\top Y.

This is the collection of separate OLS fits, one per response column. R's lm documentation explicitly describes this matrix-response fitting rule. The inverse is a mathematical expression; use QR or SVD for computation. Full rank ensures unique coefficients, while near dependence between columns can still make them sensitive to small data changes.

Four observations, two portfolios

Use these fictional monthly excess returns, entered as percent numbers, against a dimensionless score

x

. A return of

1.1

means

1.1\%

; differences in these numbers are percentage points (pp).

Observation	Score $x$	Portfolio A (%)	Portfolio B (%)
1	0	1.1	2.2
2	1	2.9	0.4
3	2	4.9	0.6
4	3	7.1	−1.2

The design has rows $(1,x_i)$ , so $n=4$ , $k=2$ , $q=2$ . The coefficient matrix and fitted equations are

\hat B=\begin{pmatrix}1&2\\2&-1\end{pmatrix}, \qquad \hat y_A=1+2x,\qquad \hat y_B=2-x.

The first row contains intercepts; the second contains score slopes. Increasing the score by one unit changes fitted A return by $2$ pp and fitted B return by $-1$ pp. These are fitted associations in a constructed sample, not causal or forecasting results.

Two responses fitted to the same score values 0, 1, 2 and 3. Portfolio A observations are 1.1, 2.9, 4.9 and 7.1 percent with fitted line 1 + 2x. Portfolio B observations are 2.2, 0.4, 0.6 and −1.2 percent with fitted line 2 − x. Residual SSCP is diagonal with entries 0.04 and 0.80 squared percentage points.

Fictional monthly data. The shared design produces a separate fitted line for each response; vertical segments show residuals. Both panels use the same return scale.

The residual columns, observed minus fitted, are

e_A=\begin{pmatrix}0.1\\-0.1\\-0.1\\0.1\end{pmatrix}, \qquad e_B=\begin{pmatrix}0.2\\-0.6\\0.6\\-0.2\end{pmatrix}.

Each column has zero sum and zero inner product with the score, so $X^\top E=0$ for $E=Y-X\hat B$ . Their sums of squares and cross-products give

E^\top E=\begin{pmatrix}0.04&0\\0&0.80\end{pmatrix}\text{ pp}^2, \qquad S=\frac{E^\top E}{n-k} =\begin{pmatrix}0.02&0\\0&0.40\end{pmatrix}\text{ pp}^2.

There are two residual degrees of freedom per response, not four from counting the two responses twice. The off-diagonal zero describes these deliberately chosen residuals; it does not establish independent portfolio shocks. Four artificial rows support an arithmetic demonstration, not a significance claim.

What the joint model adds

Condition on $X$ . Suppose $\mathbb EU\mid X]=0$ , error vectors from different observations are uncorrelated, and each has the same response [covariance matrix $\Sigma$ . Then $\hat B$ is unbiased and $S$ estimates $\Sigma$ without bias when $n>k$ . For coefficient columns $\hat{\mathbf b}_a$ and $\hat{\mathbf b}_b$ ,

\text{Cov}(\hat{\mathbf b}_a,\hat{\mathbf b}_b\mid X) =\Sigma_{ab}(X^\top X)^{-1}.

Cross-response covariance therefore enters uncertainty about coefficient differences and combinations. Joint hypotheses, such as whether a factor's slopes are zero across all responses, use that dependence; separate unadjusted tests do not answer the same question. The statsmodels multivariate-model example explains the common-design coefficient equivalence and joint hypothesis tests.

With this same $X$ and same rows for every response, and a positive-definite common

\Sigma

, allowing contemporaneously correlated errors in GLS still gives the OLS coefficient matrix. Correlation alone does not improve these unrestricted point estimates. Different regressors, missing-row patterns or cross-equation coefficient restrictions require a separate analysis; the equivalence cannot simply be carried over.

For classical exact multivariate tests, assume independent multivariate normal error vectors and positive-definite

\Sigma

, with enough residual rank for the required inverses. Stanford's notes derive the estimator, covariance estimates and Gaussian reference results. Normality is unnecessary for computing OLS or the moment results above. Serial dependence or changing error covariance invalidates those classical covariance formulas and calls for an inference method suited to the data.

For the scalar foundation, Multiple Linear Regression develops the matrix estimator and its assumptions (public preview; full lesson requires membership). For a joint-testing continuation, see MANOVA and Wilks lambda.

Sources

Jonathan Taylor, Stanford STATS305C: Multivariate regression — model, coefficient estimation, unbiased error covariance and multivariate inference.
statsmodels: Multivariate Linear Model — MultivariateLS — common-design separate-fit equivalence and joint coefficient hypotheses.
R stats: lm — Fitting Linear Models — matrix-response least squares and QR fitting.

One design, several coefficient columns

Four observations, two portfolios

What the joint model adds

Sources

Related terms

MANOVA and Wilks lambda

Multiple linear regression

Multivariate normal distribution

Ordinary least squares

Residual