November 2023

In the last post, we have seen the power of the hypothesis test for a mean, which is away from the null hypothesis. What happens if the true mean is further away, say 110 instead of 108? Let’s run the following code and you get it.

power_calc <- function(alt){

Null_mu <- 100
sigma <- 16
N_sample <- 16
alpha <- 0.05
Alt_mu <- alt

z <- qnorm(alpha, 0, 1, lower.tail = FALSE)
X_05 <- z * sigma /sqrt(N_sample) + Null_mu
Z_cum <- (X_05 - Alt_mu) / (sigma/sqrt(N_sample))
pnorm(Z_cum, 0, 1, lower.tail = FALSE)  

}

power_calc(110)

You see, the power is increased from 0.64 to 0.8. The whole spectrum of power values from the null hypothesis mean (100) all the way to 120 is shown below.

When alpha is reduced, the power is also reduced (type II error increases).

Reference

Power Functions: PennState

Let’s assume the IQ of a population is normally distributed with a standard deviation of 16. A hypothesis test collected 16 samples for the null hypothesis of mean IQ = 100 for a significance level of 5%. What is the power of the hypothesis test if the true population mean is 108?

Definition: The power of a test is the probability that it will correctly reject the null hypothesis when the null hypothesis is false.

Step 1: Estimate the Z-score for the alpha (significance level)

qnorm(0.05, 0, 1, lower.tail = FALSE)

1.645

Step 2: Estimate IQ corresponds to Z = 1.645

1.645 * 16 / sqrt(16) + 100 =  106.58

Above the IQ average of 106.58, the Null hypothesis (that the mean = 100) will be rejected.

Step 3: Estimate Z-score at X = 106.58 for mean = 108

The entire area above Z = -0.355 is included in the power region (the area below Z = -0.355 will be the false negative part as the null hypothesis will not be rejected).

Step 4: Estimate the cumulative probability > Z = -0.355

pnorm(-0.355, 0, 1, lower.tail = FALSE)

 0.639

The power is 0.639 or 64%

Reference

Power Functions: PennState

Alan knows the average fuel bill of families in his town last year was $260, which followed a normal distribution with a standard deviation of $50. He estimated this year’s value to be $278.3 by sampling 20 people. He then rejected last year’s average (the null hypothesis) in favour of an alternate of average > $260.

1. What is the probability that he wrongly rejected the Null hypothesis?

The probability is estimated by transforming the normal distribution with mean = 260 and standard deviation = 50 to a standard normal distribution (Z):

The ‘pnormGC’ function of the ‘tigerstats’ package makes it easy to depict the distribution and the region of importance.

library(tigerstats)

sample <- 20
null_mean <- 260
ssd <- 50
data <- 278.3

zz <- round((data - null_mean)/(ssd/sqrt(sample)), 2)
pnormGC(zz, region="above", mean=0, sd=1, graph=TRUE)

The probability of incorrectly rejecting the null hypothesis is 0.0505 (5.05%). It is the probability of Alan committing a Type I error.

If Alan accepts a 5% Type I error, he will reject the null hypothesis for every value > 278.3 and accept all values < 278.3.

2. If so, what is the probability of wrongly rejecting the alternate hypothesis that the true population mean for this year = $290?

sample <- 20
alt_mean <- 290
ssd <- 50
data <- 278.3

zz1 <- round((data - alt_mean)/(ssd/sqrt(sample)), 2)
pnormGC(zz1, region="below", mean=0, sd=1, graph=TRUE)

The shaded area represents all the values < 278.3. And it is the probability of Type II error.