🟠ML: The Bias-Variance Tradeoff
Total Error = Bias² + Variance + Irreducible Error
Bias = systematic error from oversimplified model. "A straight line trying to fit a curve." Variance = sensitivity to training data. "The model memorized the noise."
| Symptom | Diagnosis | Fix |
|---|---|---|
| Both train AND test error high | High bias (underfitting) | More complex model, add features, less regularization |
| Low train error, HIGH test error | High variance (overfitting) | More data, regularization, simpler model, dropout |
Practice Questions
Q: Your model gets 95% train accuracy and 72% test accuracy. What's the problem and what do you try first?