A positive-data power family that includes natural logs. See the square-root case, fit lambda with its Jacobian, and avoid confusing inverse forecasts with means.
Box–Cox transformation
A Box–Cox transformation is a one-parameter family of monotone transformations of strictly positive data, combining power functions with a logarithmic limiting case. It can make level-dependent variability or a nonlinear response relationship easier to model. For a quant forecasting positive trading volume or volatility, that means choosing a modelling scale while keeping track of how predictions return to the original units.
A power family with a log at zero
For a positive numerical input y and real parameter λ, the basic family is
gλ(y)=⎩⎨⎧λyλ−1,lny,λ=0,λ=0,y>0.
The zero case is the natural logarithm, obtained continuously as λ→0. All members preserve ordering because gλ′(y)=yλ−1>0. SciPy's definition uses this positive-data convention.
Three landmarks help interpret the parameter:
Parameter
Transformation
What changes
λ=1
y−1
A shift; distributional shape is unchanged
λ=1/2
2(y−1)
A scaled, shifted square root
λ=0
lny
Multiplicative ratios become additive differences
Calling λ=1 the “identity case” refers to unchanged shape: the formula returns y−1, not exactly y.
Take synthetic positive inputs 1,4,9, expressed as dimensionless ratios to a fixed reference level. At λ=1/2,
g1/2(1)=0,g1/2(4)=2,g1/2(9)=4.
The original gaps are 3 and 5; the transformed gaps are both 2. With λ=0, the outputs instead are 0,ln4,ln9, approximately 0,1.386294,2.197225. Equal spacing in this tiny example is arithmetic, not evidence that the population is Gaussian.
Three Box–Cox curves for positive dimensionless inputs. At y equal to 1, 4 and 9, lambda 0.5 gives 0, 2 and 4; lambda 0 gives 0, 1.386 and 2.197; lambda 1 gives 0, 3 and 8. The log and square-root members compress larger inputs more than lambda 1.
Original synthetic scale comparison. The square-root member makes these particular three inputs equally spaced; three points establish neither a distribution nor a fitted transformation parameter.
Zeros and negative observations fall outside this basic family. Transforming y+c is a different specification: state the shift and ensure positivity for every observation to which it will be applied. A shift can change the selected power and the interpretation. Changing units changes transformed values and coefficient units; declaring a fixed reference scale makes the input convention explicit. A pure unit conversion can leave the fitted λ unchanged when the regression includes an intercept and the complete likelihood is used; the numerical coefficients still refer to the declared transformed scale. An added shift has no such general invariance.
Fit the model, not just the transformed variance
The classical motivation is a response scale on which a linear model with constant error variance and approximately Gaussian errors is useful. Box and Cox's original summary distinguishes normality, constant variance and additivity as separate concerns.
For a fixed design matrix X, write the working model as gλ(yi)=xi⊤β+εi, with independent errors of common variance σ2. When comparing λ by Gaussian maximum likelihood, the density must be expressed on the original observed-data scale. Its change-of-variable factor is
Jλ=i=1∏nyiλ−1.
After fitting β and profiling out σ2, the usual working profile log-likelihood is
ℓp(λ)=−2nln(nSSEλ)+(λ−1)i=1∑nlnyi+C,
where SSEλ is the residual sum of squares from refitting the same regression specification at that λ, and C is independent of λ. SciPy documents the intercept-only version, where residuals are deviations from the transformed sample mean. Minimising transformed variance alone omits the Jacobian term and compares incompatible scales.
For the three illustrative inputs, an intercept-only calculation gives:
Candidate λ
Residual sum of squares
ℓp(λ)−C
0
2.469072
−3.291364
0.5
8
−3.263003
The log has the smaller residual variance, yet the square-root candidate has the larger complete criterion. This compares two stipulated candidates; it does not estimate a reliable power from three observations or establish the global optimum.
The expression is a working Gaussian criterion. For nonzero λ, the transformed positive response has a bound, so an unrestricted normal model is not literally supported everywhere; a model that explicitly truncates the normal distribution needs its corresponding normalisation. More generally, the transformation supplies no guarantee of Gaussian tails, constant variance, independence or forecast accuracy. Inspect model residuals, rather than judging success only from a histogram of the response.
Residual Diagnostics and Model Checking develops residual-versus-fitted plots and QQ plots for these checks (public preview; full lesson requires membership).
Returning to the original scale
For a transformed value z, the inverse is
gλ−1(z)={(1+λz)1/λ,ez,λ=0,1+λz>0,λ=0.
Inverting a conditional expectation generally does not recover the original-scale conditional mean:
gλ−1(E[gλ(Y)∣x])=E[Y∣x].
For a concrete assumed lognormal model, let lnY∣x∼N(ln100,0.16), with positive Y measured on a fixed numerical scale. Exponentiating the log-scale mean gives 100, the conditional median. The conditional mean instead is
E[Y∣x]=eln100+0.16/2=100e0.08≈108.328707.
This exact correction uses the conditional lognormal assumption and its variance, not just the fitted log mean. The NIST lognormal formulas identify the distinction between median and mean; Forecasting: Principles and Practice, §5.6 explains the forecasting consequence. Other powers or residual distributions require an appropriate distributional calculation or integration. A transformation is part of the model, including its return journey to the quantity being forecast.
