Hey guys, So here comes the eighth blog of the Handwritten notes series which we started. We will be talking about K-Nearest Neighbors (KNN) in this blog. I have uploaded my handwritten notes below and tried to explain them in the shortest and best way possible.
Let’s go through the K-Nearest Neighbors notes…
- In the first diagram, I have plotted dog and horse classes based on weight and height.
- Given this plot, if asked to predict the class of + points, would you be able to classify them correctly? And if yes, what approach will you use to do so?
- The most naive approach to do so will be to see some of the nearest points and make a prediction based on that.
- This is called the K-Nearest Neighbors algorithm. Based on the K-nearest data points, we predict the class of the data point under test.
- In the training phase, KNN does not make any type of discriminator function or anything else. It just stores all the data points.
- In the testing phase, it just calculates the euclidean distance of all points from the test point and sorts them in increasing order. Then it takes the first k-nearest points and sees which class is the majority class. And this majority class is allotted to the test point.
- This diagram shows how the KNN classification algorithm works.
- For different k’s, it will give different answers.
- Very Simple.
- Training is trivial.
- Easy to add more data.
- Works with any no. of classes.
- Very few parameters (k and distance metric).
- Worse for large datasets.
- Not good for high-dimensional data.
- Categorical features don’t work well.
- One thing to note while using KNN is that it is a distance-based algorithm and it expects everything to be on the same scale.
- That’s why we need to use Standard Scaler or Min-Max Normalization in this case.
Do let me know if there’s any query regarding K-Nearest Neighbors (KNN) by contacting me on email or LinkedIn.
So this is all for this blog folks, thanks for reading it and I hope you are taking something with you after reading this and till the next time ?…
READ MY PREVIOUS BLOG: LOGISTIC REGRESSION – LAST-MINUTE NOTES