Each piece of this code runs great by itself on a single case...There are 450 timestamps each with +60k agents, and if I run one all alone (outside of the for loop) it finishes in about 2 seconds. Why does running them in the for loop take so long? Shouldn't it take 450*2 seconds? little.my.df has 50k rows and eligible.df has about 6300 rows.
libary(SearchTrees)
### Make a column to put my result
eligible.df$withinradius <- vector(length = dim(eligible.df)[1])
### For loop selects which rows from little.my.df are in the same
### timestamp [i,1] and are not the same agent [i,3].
### There are 450 timestamps.
for (i in 1:dim(eligible.df)[1]){
timestamp.select <- little.my.df[
which(
little.my.df[,1] == eligible.df[i,1] &
little.my.df != eligible.df[i,3]),
c(5,4)]
### Create a tree from timestamp.select and find the first NN from i
test.tree <- createTree(timestamp.select,
treeType = 'quad',
dataType = 'point',
maxDepth = 1)
test.lookup <- knnLookup(test.tree,
newdat = eligible.df[i,c(5,4)],
k = 1)
### Calculate the euclidian distance from the first NN and record it in the
### blank column on the original dataframe.
eligible.df[i,(dim(eligible.df))[2]] <- dist(matrix(
data = c(eligible.df[i,c(5,4)],
timestamp.select[test.lookup[1,1],]),
ncol = 2, nrow = 2, byrow = TRUE))
}
For each row in eligible.df I want to find the first nearest neighbor in little.my.df (50000 rows). The ACTUAL my.df has over 1 million rows so I am trying to speed this up, but I can't even get it to work with 50k rows.