Interview Q&A / Machine Learning

1. How does a decision tree work, and why does a random forest usually perform better?

   A decision tree splits data with a series of yes/no questions, choosing each split to make the groups as pure as possible. A single deep tree overfits easily. A random forest trains many trees on random samples of rows and features and averages their votes, which reduces variance and usually improves accuracy.

   IntermediateIn-depth4 min read
2. How do you stop a decision tree from overfitting?

   Limit its growth (max_depth, min_samples_leaf, min_samples_split, max_leaf_nodes) or prune it afterwards with cost-complexity pruning (ccp_alpha). Choose the settings by cross-validation. Or use an ensemble such as a random forest.

   BeginnerQuick answer1 min read
3. How does a decision tree predict a number in regression?

   It splits the data to reduce squared error and each leaf predicts the average target of the rows in it. The output is therefore piecewise constant, a staircase, not a smooth line.

   IntermediateQuick answer1 min read
4. What are entropy and information gain in a decision tree?

   Entropy measures how mixed a group is: 0 when pure, 1 for a 50-50 split of two classes. Information gain is the entropy before a split minus the weighted entropy after. The tree picks the split with the highest gain.

   IntermediateQuick answer1 min read
5. Why can't a random forest or decision tree extrapolate beyond the training range?

   Tree leaves hold averages of training targets, so a prediction can never exceed the largest or fall below the smallest value seen in training. For trends that continue upward, such as sales growth, linear models or feature engineering work better.

   AdvancedQuick answer1 min read