Three models of fitting. Consider trying to fit some dataset \(|D|= n\) that’s roughly quadratic with…
- a linear model: underfit, high bias (i.e. “model imposes bias of linearity on data”)
- a nth order polynomial: overfit, high variance (i.e. “a small perturbation of data brings lots of change”)
Its important to pay attention if you are having high bias of high variance—solutions of each is different from each other.
intuition of overfitting
See overfit
diagnosing bias variance tradeoff problems
Let’s consider:
| Reference | Training | Test | Judgment |
|---|---|---|---|
| 2% | 2% | 2% | High Bias |
| 10% | 2.5% | 10% | High Variance |
| 10% | 10% | 20% | Both High Bias and Variance |