Which hyperparameters matter most in a random forest?
n_estimators (more trees gives stable results with diminishing returns), max_features (how many features each split considers), max_depth, min_samples_leaf and min_samples_split. max_features and min_samples_leaf usually matter most.
Explain it simply
A forest is a committee. How many members (n_estimators), how much each member is allowed to see at one time (max_features), and how detailed each member's rulebook may be (max_depth, min_samples_leaf).
Example
n_estimators 100-500 typical; more trees never overfit more,
they only cost time
max_features "sqrt" is default for classification
with 16 features, each split looks at sqrt(16) = 4
max_depth limits tree depth; None grows until pure
min_samples_leaf larger values smooth the model (1 is default)
class_weight "balanced" for imbalanced dataSmaller max_features makes trees more different from each other, which lowers correlation and variance, but each tree is individually weaker.
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import RandomizedSearchCV
params = {"max_features": ["sqrt", 0.3, 0.5],
"min_samples_leaf": [1, 3, 5, 10]}
search = RandomizedSearchCV(RandomForestClassifier(n_estimators=300, random_state=0),
params, n_iter=8, cv=5)Random forests are fairly forgiving: defaults often give a decent baseline.
What to say in the interview
"The main levers are max_features, min_samples_leaf and max_depth, with n_estimators set high enough for stable scores. I tune them using randomised search with cross-validation, and use class_weight for imbalance."
Common mistakes
- Believing more trees cause overfitting.
- Tuning on the test set.