Let’s do the ANOVA step by step. We use the F-statistic to accept or reject the null hypothesis by comparing it with the critical F value. Once you get the F-value, you can calculate the p-value based on a significance level.
The definition of F-statistic is

F = Between groups variance / Within-group variance

Between groups variance

Here, you are estimating the variation of the group statistic from the global statistic. In other words, you determine the means of each group and the global mean (of all data or the mean of means). The estimate the difference, square, add up and divide by the degree of freedom like you do standard variance.

Recall the previous example (strength of materials by four vendors). So you have four groups, each containing ten samples. First, estimate four means and the global mean. They are:

The numerator (mean squares of factor) is calculated by dividing the sum square of factor with the degrees of freedom, i.e., 43.62/3 = 14.54.

Within-group variance

Here, you add up all the variations inside the groups. Add them up and then divide by the sum of the degrees of freedom of each group.

The denominator (mean squares of error) is calculated by dividing the sum within group squares for error with the total degrees of freedom, i.e., 158.466/36 = 4.402.

F – Statistics = 14.54 / 4.402 = 3.30

The 3.30 is then compared with the critical F-value corresponding to a set significance level, 0.05, in the present case. You can either look up at the F distribution table or use the R function.

qf(0.05, 3,36, lower.tail=FALSE)

The critical value is 2.87. Since the F-statistics in our case is larger than 2.87, we reject the null hypothesis. The p-value turned out to be 0.031.

pf(3.303, 3, 36, lower.tail = FALSE)

We know how to use a 1-sample t-test to perform a hypothesis test on the mean of a single group and a 2-sample t-test to compare two groups. The scope of t-tests ends here as two is the limit. What happens when you have three groups of data? If the number is two or more, you will use the analysis of variance or ANOVA. As we have done before, we will do an ANOVA using R programming.

Comparing four vendors

ANOVA requires an independent variable (categorical factor) and a dependent variable (continuous). The following table will tell you what I meant. The data used in the analysis is taken from https://statisticsbyjim.com/. The strength of certain materials from four vendors is available, and we are determining if there is a statistically significant difference between the mean strengths. Here is how a few selected entries appear (out of 40).

Note that the categorical factor, vendor names (1, 2, 3 and 4), divided the continuous data into four groups. Before performing ANOVA, we plot the data and check how they look.

par(bg = "antiquewhite")
boxplot(AN_data$Strength ~ factor(AN_data$Vendor), xlab = "Vendor", ylab = "Strength of Material")

The null and alternate hypotheses are:

N₀ = four mean strengths are equal
N_A = four mean strengths are not equal

Doing ANOVA is pretty easy; the following commands will do the job.

sum_aov <- aov(AN_data$Strength ~ factor(AN_data$Vendor))
summary(sum_aov)

                      Df Sum Sq Mean Sq F value Pr(>F)  
factor(AN_data$Vendor)  3  43.62  14.540   3.303 0.0311 *
Residuals              36 158.47   4.402                 
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

We are testing the statistical significance by the F-test. And here is the most important thing, the p-value is 0.03, less than the 0.05 we chose. So the null hypothesis is rejected. We will see what they all mean, in the next post.

Hypothesis Testing: An Intuitive Guide: Jim Frost