P-Hacking

P-hacking is an often malicious practice in which the analysis is chosen based on what makes the p-value significant. Before going into detail, let’s recall the definition of p-value. It is the probability that an effect is seen purely by chance. In other words, if we chose 5% as the critical p-value to reject (or fail to reject) a null hypothesis, 1 in 20 tests will result in a spectacular finding even when there was none. 

So what happens if the researcher carries out several tests and reports only the one with the ‘shock value’ without mentioning the context of the other non-significant tests? It becomes an example of a p-hacking. 

References

P-Hacking: Crash Course Statistics: CrashCourse
Data dredging: Wiki
The method that can “prove” almost anything: TED-Ed