Why can't a random forest or decision tree extrapolate beyond the training range?
Tree leaves hold averages of training targets, so a prediction can never exceed the largest or fall below the smallest value seen in training. For trends that continue upward, such as sales growth, linear models or feature engineering work better.
Explain it simply
A shopkeeper who has only ever sold between 10 and 100 packets a day, and decides by looking at similar past days, will never quote 500 packets, even if the festival season clearly points there. A tree does exactly that: it answers from similar past cases.
Example
Train on x = 1, 2, ..., 10 with y = 2x (so y runs from 2 to 20). A fully grown regression tree stores leaves with y = 2, 4, ..., 20.
Predict x = 5 -> 10 (correct)
Predict x = 10 -> 20 (correct)
Predict x = 20 -> 20 (true value is 40)
Predict x = 100 -> 20 (true value is 200)Everything beyond x = 10 falls into the last leaf. A linear regression would predict 40 and 200.
from sklearn.tree import DecisionTreeRegressor
import numpy as np
X = np.arange(1, 11).reshape(-1, 1)
y = 2 * X.ravel()
print(DecisionTreeRegressor().fit(X, y).predict([[20]])) # [20.]Remedies: model the trend with a linear component and let trees fit the residuals, predict differences or growth rates instead of absolute values, or use a model that extrapolates.
What to say in the interview
"Trees predict by averaging training targets in a leaf, so predictions stay within the training range; random forests and gradient boosting inherit that. For trending data I detrend or difference the target, or combine a linear model with a tree on the residual."
Common mistakes
- Using a random forest for long-horizon forecasting of a rising trend.
- Evaluating on a random split that never tests extrapolation.