As the power level increases, the percentage of detections increases and the exaggeration of the effect size decreases. Both are good things and have a common cause. The graph below displays the exaggeration factor (mean significant effect / actual effect) by power. No exaggeration occurs at a value of one.

Statistical power is the opposite of Type II errors, both mathematically (1 – β) and conceptually. Power is the ability of the test to detect an effect that exists in the population. In other words, the test correctly rejects a false null hypothesis.

To represent a significance level of 0.05, I’ll shade 5% of the distribution furthest from the null value.

Parametric statistics is a branch of statistics that assumes sample data come from populations that are adequately modeled by probability distributions with a set of parameters.

The three factors that affect power are sample size, variability in the population, and the effect size.

If your experimental design is sound, you collect representative data, and the data satisfy the hypothesis test’s assumptions, the Type I error rate equals the significance level that you specify. However, if there is a problem in one of those areas, it can affect the false positive rate.

Inferential statistics takes data from a sample and makes inferences about the larger population from which the sample was drawn. Consequently, we need to have confidence that our sample accurately reflects the population.

On the probability distribution plot, the significance level defines how far the sample value must be from the null value before we can reject the null hypothesis.

When your study does everything correctly, sampling error is the one thing that causes Type I errors.