ANOVA Tables Partition Variation in Regression

An analysis of variance table—an ANOVA table—organizes a regression into variation explained by the model and variation left in the residuals. It turns one fitted equation into an accounting system for variability.

Partitioning the sums of squares

For observations $y_i$, fitted values $\hat y_i$, and mean $\bar y$,

$$\mathrm{SST}=\sum_i(y_i-\bar y)^2$$

is total variation,

$$\mathrm{SSR}=\sum_i(\hat y_i-\bar y)^2$$

is variation explained by the regression, and

$$\mathrm{SSE}=\sum_i(y_i-\hat y_i)^2$$

is residual variation. For ordinary least squares with an intercept,

$$\mathrm{SST}=\mathrm{SSR}+\mathrm{SSE}.$$

Degrees of freedom and mean squares

If a model contains $p$ estimated coefficients including the intercept, the common regression table uses:

Source	Degrees of freedom	Sum of squares	Mean square
Regression	$p-1$	SSR	$\mathrm{MSR}=\mathrm{SSR}/(p-1)$
Error	$n-p$	SSE	$\mathrm{MSE}=\mathrm{SSE}/(n-p)$
Total	$n-1$	SST	—

The residual mean square estimates the error variance when the model assumptions are appropriate.

Interpreting the F test

The classical null hypothesis says that all non-intercept regression coefficients are zero. A large $F$ is evidence that the model explains more variation than expected from noise alone, under the stated assumptions.

Sequential and partial tests

In models with several terms, sums of squares can depend on the order in which terms enter. Partial F tests compare nested models and ask whether added terms reduce error enough to justify their degrees of freedom.

Build a table from numbers

Suppose $n=20$, the model has an intercept and two predictors ($p=3$), SST is $200$, and SSE is $80$. Then SSR is $120$. The degrees of freedom are $2$, $17$, and $19$.

$\mathrm{MSR}=120/2=60,\qquad \mathrm{MSE}=80/17\approx4.706,$

so $F\approx12.75$. The reference distribution is $F_{2,17}$ under the null and standard regression assumptions.

Partial F tests

For nested reduced and full models,

$F=\frac{(\mathrm{SSE}_R-\mathrm{SSE}_F)/(p_F-p_R)}{\mathrm{SSE}_F/(n-p_F)}.$

This asks whether a block of added terms reduces error enough to justify its parameter cost.

Check your understanding

If SSR rises after adding predictors, must the partial F test be significant?

Show the reasoning

No. SSR always rises or stays fixed in nested ordinary least squares. Significance depends on the size of the improvement relative to the added degrees of freedom and residual variation.