So I need to find nearest neighbors of a given row in pyspark DF using euclidean distance or anything. the data that I have 20+ columns, more than thousand rows and all the values are numbers.
I am trying to oversample some data in pyspark, as mllib doesn't have inbuilt support for it, i decided to create it myself using smote.
my approach till now has been to convert all the categorical distance into index using stringtoindex so that i can find the euclidean distance and neighbors and hence perform smote.
I am fairly new to spark and ml. Any help would be appreciated.