🟢 Stats: Type I and Type II Errors
| Hâ‚€ True (no effect) | Hâ‚€ False (real effect) | |
|---|---|---|
| Reject H₀ | Type I (α) — false positive, "crying wolf" | ✅ Correct! (Power = 1-β) |
| Fail to reject H₀ | ✅ Correct! | Type II (β) — false negative, "missing the wolf" |
Power = 1 - β = probability of detecting a real effect when it exists. Benchmark: ≥ 0.80.
What increases power? Larger sample size, larger effect size, higher α, lower variance.
The √n relationship: Doubling your sample size does NOT double precision. It multiplies precision by √2 ≈ 1.41. To halve your confidence interval, you need 4× the sample.
Practice Questions
Q: Your boss says "we can only detect a 5% lift — the actual lift is probably 2%. What do we need?"