1

I came across a surprising result with data.table. Here is a really simple example :

library(data.table)
df <- data.table(x = 1:10)
df[,x[x>3][.N]]
[1] NA

This syntax gives NA, but this work:

df[,x[x>3][1]]
[1] 4

and of course this

df[,x[.N]]
[1] 10

I know that in this simple example case you can do

df[x>3,x[.N]]

but I wanted to use the df[,x[x>3][.N]] syntax while using lapply on .SD to avoid a loop on the i selection, so something like

df2 <- data.table(x = rep(1:10,2), y = rep(2:11,2),ID = rep(c("A","B"),each = 10))
cols = c("x","y")
df2[,lapply(.SD,function(x){x[x>3][.N]}),.SDcols = cols, by = ID]

But this fail, same as in my simple example. Is it because .N is not implemented in this case, or am I doing something wrong ?

My actual work around:

Reduce(merge,lapply(cols,function(col){df2[col>3,setNames(list( get(col)[.N]),col),by = ID]}))

   ID  x  y
1:  A 10 11
2:  B 10 11

but I am not fully happy with it, I find it less readable. Has anyone an explanation and a better work around ? Thank you !!

2
  • @dww Thanks, it makes sense. But how can I do something similar then? df[,x[x>3][sum(x>3)]] ?
    – denis
    May 29, 2018 at 17:08
  • @dww Perfect! thank you again
    – denis
    May 29, 2018 at 17:24

1 Answer 1

2

df[,x[x>3]] has 7 elements. .N is 10. You are trying to subset a vector out of range.

So you can access the last element of the vector in lapply using:

df2[, lapply(.SD, function(x) tail(x[x>3], 1) ), .SDcols = c('x','y'), by = ID]

Or more idiomatic for data.table we can use

df2[, lapply(.SD, function(x) last(x[x>3]) ), .SDcols = c('x','y'), by = ID]

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.