Recall K-NN algorithms are lazy learners!

Regression: considers the VALUE in majority of k neighbours

Classification: considers the CLASS in majority of k neighbours

Manhattan distance is the L1-norm and therefore cannot be used for categorical variables. Hamming distance would be a good choice for categorical variables.

Recall when K=1 the decision boundaries perfectly match the data and its noise. This will result in overfitting. Increasing K makes K-NN more resilient to overfitting.

Think about what happens when K=1. Increasing K results in K-NN being less effected by noise as a large number of neighbours are considered when making a decision.

Eager learners are desirable as all the heavy computation occurs at training time. Therefore the algorithm is time + compute efficient at inference time.

Recall that the weight you assign is inversely correlated with the distance metric. Both '-d' and 'log(min(0.25*d, 1)' yield negative distances which does not make sense. exp(-d) acts as a good distance metric as it increases exponentially with distance. This favours points close by and quickly ignores points far away.

Too many points to address here. Please refer to lecture slides if confused.

The curse of dimensionality plagues all machine learning algorithms. Hence the need for feature extractors and engineered priors.

The choice of distance metric in k-NN directly influences how distances between data points are calculated, impacting both the accuracy of classifications and the computational speed of the algorithm.