Interview Q&A / Machine Learning

1. How do you choose k in KNN, and what happens when k is very small or very large?

   Small k (like 1) follows noise and overfits; very large k smooths too much and underfits, at the extreme predicting the majority class for everything. Choose k by cross-validation, and use an odd k for two-class problems to avoid ties.

   BeginnerQuick answer1 min read
2. How does the K-Nearest Neighbours algorithm make a prediction?

   KNN stores all training data. To predict, it finds the k training points closest to the new point and takes a majority vote (classification) or an average (regression) of their targets.

   BeginnerQuick answer1 min read
3. What is the difference between Euclidean and Manhattan distance, and which should KNN use?

   Euclidean is the straight-line distance, sqrt of the sum of squared differences. Manhattan is the sum of absolute differences, like walking along city blocks. Euclidean is the default; Manhattan can be better in high dimensions or with many outliers.

   BeginnerQuick answer1 min read
4. Why do we scale features, and which algorithms need it?

   Scaling puts features on comparable ranges. Distance-based and gradient-based models (KNN, SVM, k-means, PCA, neural networks, regularised regression) need it. Tree-based models do not.

   BeginnerQuick answer1 min read
5. Why is KNN slow at prediction time and poor with many features?

   KNN must compare a query against every stored training row, so prediction cost grows with data size. With many features, distances between points become almost equal, so 'nearest' stops meaning much. This is the curse of dimensionality.

   IntermediateQuick answer1 min read