ANOVA versus Regression

4.7. ANOVA versus Regression#

There are several similarities between ANOVA and regression analysis:

  • They both partition the variance using sum of squares i.e., they both depend on variance in their estimates, although in different ways:

    • In ANOVA the SS ratio is \(F = \frac{BMS}{EMS} \)

    • In regression the SS ratio is \(R^2 = \frac{TSS -SSE}{TSS} \)

  • They can both include more than one \(x\)-variable, and test for interactions.

  • And, they could both be used to answer the same research question about differences between groups, but with differences in their approach…

The differences between ANOVA and regression analysis are:

  • ANOVA and Regression differ in their aims. ANOVA is a way of determining whether groups are different to each other, with a YES or NO answer. E.g., “Is personality trait the same or different among different age groups?” “Were the outcomes the same across treatment groups and control?” Regression, on the other hand, is a statistical model that approximates the relationship between x and y, telling us about the strength of the relationship.

  • Traditionally, ANOVA was often used to analyse designed experiments while regression was used for inferential statistics.

  • ANOVA is used for categorical \(x\) variables and continuous \(y\) variables, while linear regression is applied to continuous \(y\) variables and \(x\) variables of all types (binary, categorical, continuous). NB. Remember that regression would use ‘dummy’ variables to analyse categorical \(x\) variables (taking values 0 and 1).