Power of Hypothesis Test

Effect Size, Cohen’s d, and Power

In statistical analysis, effect size quantifies the magnitude of a phenomenon, independent of sample size.

Cohen’s d Calculation:

For paired samples like the sleep dataset, Cohen’s \(d\) is calculated using the standard deviation of the differences. This very large effect size (\(d = 1.28\)) indicates Drug 2 provides 1.58 more hours of sleep than Drug 1 relative to variability (SD of differences = 1.23). Cohen’s thresholds (0.2=small, 0.5=medium, 0.8=large) show this is an exceptionally strong effect.

---

Power Analysis:

With \(n=10\) pairs and \(d=1.28\), power = 95%. The formula for paired tests uses the non-central t-distribution:

\[ \text{Power} = 1 - \beta = P\left(t_{ncp} > t_{critical}\right) \] where \(ncp = d\sqrt{n}\)

---

Type I/II Error Trade-off:

Reducing α from 0.05 to 0.01 decreases power by 18.5 percentage points (95% → 76.5%), demonstrating the sensitivity of power to α-level choices.

---

Sample Size Impact:

Doubling \(n\) to 20 yields 97.7% power, showing how increased sample size boosts sensitivity even with large effects.