Interview Q&A / Machine Learning

1. What is overfitting and underfitting, and what is the bias-variance trade-off?

   Overfitting means the model memorises the training data, including noise, so it does well in training but badly on new data. Underfitting means the model is too simple to learn the pattern at all. The bias-variance trade-off is the balance between these two errors.

   BeginnerIn-depth4 min read
2. What is the difference between L1 and L2 regularisation?

   Both add a penalty on weight size to the loss to reduce overfitting. L1 (Lasso) adds the sum of absolute weights and pushes many weights to exactly zero, so it also selects features. L2 (Ridge) adds the sum of squared weights and shrinks all weights smoothly towards zero without removing them.

   AdvancedIn-depth5 min read
3. How do you stop a decision tree from overfitting?

   Limit its growth (max_depth, min_samples_leaf, min_samples_split, max_leaf_nodes) or prune it afterwards with cost-complexity pruning (ccp_alpha). Choose the settings by cross-validation. Or use an ensemble such as a random forest.

   BeginnerQuick answer1 min read