🟠ML: Random Forest vs Gradient Boosting
| Aspect | Random Forest | Gradient Boosting (XGBoost) |
|---|---|---|
| How trees are built | Independently, in parallel | Sequentially, each corrects errors |
| Individual trees | Deep (strong learners) | Shallow (weak learners, 3-6 levels) |
| Primary effect | Reduces variance | Reduces bias |
| Overfitting risk | Lower | Higher |
| Tuning difficulty | Easy (good defaults) | Hard (learning rate, depth, subsample) |
| When to use | Quick baseline, noisy data, limited tuning time | Max accuracy, clean data, time to tune |
Interview question: "When would you pick Random Forest over XGBoost?"
Answer: "When I have limited tuning time, noisy data, or need a reliable baseline fast. RF is harder to screw up — works well with defaults. XGBoost can outperform with careful tuning but can also overfit badly with wrong hyperparameters."