I recommend you read the recent post on p-value first. In short, if the investigator rejects the null hypothesis based on evidence, it may be called a discovery. Then what is a false discovery rate (FDR)?
FDR is the proportion of tests in which the null hypothesis is true out of all cases where it is rejected. In probability notation, FDR = P(H0 is true | reject H0).
At first glance, it may resemble the significance level or alpha. But alpha is the probability of rejecting the null hypothesis when it is true; it is P(reject H0 | H0 is true). So, to get the FDR, we need to use Bayes’ theorem.
FDR = P(H0 is true | reject H0) = P(reject H0 | H0 is true) x P(H0 is true) /(P(reject H0 | H0 is true) x P(H0 is true) + P(reject H0 | H0 is not true) x P(H0 is not true))
The first term, P(reject H0 | H0 is true), as we know, is alpha. The next one, P(H0 is true), is the prior probability for the null hypothesis to be true that we need to find out. P(H0 is not true) = 1 – P(H0 is true). That leaves the last term, P(reject H0 | H0 is not true). We know the chance of not rejecting if H0 is not true is beta (false-negative or type II error). So, P(reject H0 | H0 is not true) = 1 – beta.
Let’s assume alpha = 0.05, the prior probability of the null hypothesis is 0.25, beta = 0.2,
