Generalised linear model: Definition & Examples

A generalised linear model (GLM) models an outcome's conditional distribution using an exponential-family response and connects its conditional mean to a linear predictor through a link function. It lets a quant model trade counts as positive expected counts or defaults as probabilities, while interpreting coefficients on a chosen link scale.

The defining equation is

\mu_i=\mathbb{E}[Y_i\mid\mathbf{x}_i],\qquad \eta_i=\beta_0+\mathbf{x}_i^\top\boldsymbol\beta,\qquad g(\mu_i)=\eta_i.

Here

\mu_i

is the conditional expectation,

\eta_i

is the linear predictor, and

g

is the link. Predictions on the outcome scale use the inverse link:

\mu_i=g^{-1}(\eta_i)

. “Linear” refers to the coefficients in

\eta_i

; fixed predictors such as squared leverage or interactions can still appear in a GLM. Minnesota's theory notes describe the mean/link construction.

A family controls more than the mean

In the usual independent-observation GLM, each conditional response has an exponential-family density or probability mass function. For unit observation weights, a common form is

f(y_i\mid\theta_i,\phi) =\exp\left\{\frac{y_i\theta_i-b(\theta_i)}{\phi} +c(y_i,\phi)\right\}.

The natural parameter $\theta_i$ determines the mean; $b$ is the cumulant function; $c$ supplies the remaining normalisation. Under the standard regularity conditions,

\mu_i=b'(\theta_i),\qquad \text{Var}(Y_i\mid\mathbf{x}_i)=\phi b''(\theta_i) =\phi V(\mu_i).

The variance function

V

describes how conditional variance changes with the mean. The dispersion

\phi

scales it. Gaussian responses have

V(\mu)=1

and

\phi=\sigma^2

; Bernoulli responses have

V(\mu)=\mu(1-\mu)

and

\phi=1

; Poisson responses have

V(\mu)=\mu

and

\phi=1

. These are distributional assumptions, rather than consequences of choosing a link. Minnesota: exponential-family properties.

A canonical link sets the linear predictor equal to the natural parameter:

\eta_i=\theta_i

. Other links are valid GLM choices too. The CMU notes, §2.4, explicitly distinguish these cases.

Response	Link $g(\mu)$	Inverse link	Status
Gaussian	$\mu$	$\eta$	Canonical identity
Bernoulli	$\log\{\mu/(1-\mu)\}$	$1/(1+e^{-\eta})$	Canonical logit
Bernoulli	$\Phi^{-1}(\mu)$	$\Phi(\eta)$	Noncanonical probit
Poisson	$\log\mu$	$e^\eta$	Canonical log
Poisson	$\mu$	$\eta$	Noncanonical identity; requires positive fitted means

Here

\Phi

is the standard normal cumulative distribution function. Bernoulli logit and probit both produce probabilities in

(0,1)

, but have different coefficient scales. A Poisson identity link needs restrictions to keep means positive; a log link enforces positivity for every real score. The official R family documentation lists these supported family/link combinations.

One hour, two counts, and a log link

Consider a fictional trade-count model. $Y$ counts trades in a fixed one-hour window. The dimensionless predictor $x$ is observed at the start of that window. Specify

Y\mid x\sim\text{Poisson}(\mu(x)),\qquad \log\mu(x)=\log 2+0.3x.

At $x=0$ , the expected count is $2$ . At $x=1$ ,

\mu(1)=2e^{0.3}\approx2.6997, \qquad \frac{\mu(1)}{\mu(0)}=e^{0.3}\approx1.3499.

A one-unit increase in $x$ multiplies the expected count by about $1.35$ , holding other predictors and exposure fixed. It does not add $0.3$ trades. A fractional expected count is valid even though each realised count is an integer.

Fictional Poisson GLM: the one-hour linear predictor is log(2) plus 0.3x, while the count mean is 2 exp(0.3x). One-hour means at x equal to 0 and 1 are 2.00 and 2.70; the dashed two-hour mean curve is twice the solid one-hour curve.

Original specified model, not fitted market data. The link makes the one-hour score linear; the inverse link makes the count mean positive. The second curve changes exposure while preserving the hourly rate.

Under this Poisson distribution assumption, conditional variance also equals

2.6997

x=1

. The log link alone does not imply that equality. Nor does it imply that the variance of counts pooled across different predictor values equals their pooled mean.

When observation windows vary, let $e_i>0$ be exposure measured as the number of hours. A constant-rate-within-window specification uses

\log\mu_i=\underbrace{\log e_i}_{\text{offset}} +\log 2+0.3x_i, \qquad \mu_i=e_i\,2e^{0.3x_i}.

The offset has coefficient fixed at one. At

x=1

, two hours gives an expected

5.3994

trades, while the modelled rate remains

2.6997

trades per hour. Counts and rates have different units; replacing counts by noninteger counts-per-hour and fitting an ordinary Poisson response discards this observation structure. Proportional exposure scaling assumes the same modelled rate applies throughout the longer window. Penn State's rate-model treatment derives the offset construction.

The link acts on the mean

A Poisson log-link GLM fits $\log\mathbb{E}[Y\mid x]$ . Regressing $\log Y$ on $x$ by ordinary least squares instead targets a different quantity, encounters undefined values when $Y=0$ , and generally does not recover the same conditional mean by exponentiation. In general, $\mathbb{E}[\log Y\mid x]\ne\log\mathbb{E}[Y\mid x]$ when the former exists.

Standard GLM fitting uses maximum likelihood estimation, usually with an iterative algorithm such as iteratively reweighted least squares. Its changing weights come from the family and link; this is not a single least-squares regression on transformed observed responses. See R's fitting documentation.

Before treating a fitted count or default model as a forecast, check what the specification leaves unresolved:

Overdispersion: conditional count variation exceeding the Poisson mean can reflect omitted heterogeneity or a wrong distribution. Quasi-Poisson allows $\text{Var}(Y\mid x)=\phi\mu$ without specifying a full Poisson likelihood; a negative-binomial model supplies a different count distribution. Changing dispersion does not repair the conditional mean automatically. R family documentation.
Dependence: adjacent trade windows or repeated observations on a borrower can remain correlated after conditioning. Ordinary independent-observation likelihood standard errors then need reconsideration; changing the link does not remove dependence. The statsmodels GEE overview describes a framework for correlated observations.
Calibration and timing: a probability in $(0,1)$ need not match observed default frequencies. Check held-out calibration at the intended horizon, and use only predictor values available at the forecast timestamp. An end-of-window trade statistic cannot support a start-of-window forecast without information leakage. Probability calibration documentation explains the comparison between predicted probabilities and observed frequencies.

For worked Bernoulli and Gaussian canonical-link derivations, the owned lesson The Exponential Family and Generalized Linear Models has a public preview; the full lesson requires membership.

Practice: distinguish a count from a rate

A different specified model is $\log\mu=\log e+\log 3-0.2x$ , with exposure $e$ measured in hours. At $x=1$ , what are the expected count for a half-hour window and the expected hourly rate? Is $\log e$ a coefficient to estimate?

Answer. The rate is

3e^{-0.2}\approx2.4562

trades per hour. The expected half-hour count is

0.5\times2.4562\approx1.2281

trades. The exposure offset has coefficient fixed at one; estimating that coefficient would change the proportional-exposure assumption.

A family controls more than the mean

One hour, two counts, and a log link

The link acts on the mean

Practice: distinguish a count from a rate

Sources

Related terms

Conditional expectation

Maximum likelihood estimation

Poisson distribution