2

I'd like to transpose a time series dataset to feed into some machine learning algorithms. Here's an example of what I'd like to do, except the number of lags is large and I'm looking for a more elegant way to do it:

    set.seed(42)
data <- data.frame(time = 1:5, value = rnorm(5))
data
# time      value
# 1    1  1.3709584
# 2    2 -0.5646982
# 3    3  0.3631284
# 4    4  0.6328626
# 5    5  0.4042683
data %>%
  mutate(lag_1 = lag(value),
         lag_2 = lag(value, 2),
         lag_3 = lag(value, 3),
         lag_4 = lag(value, 4),
         lag_5 = lag(value, 5))
# time      value      lag_1      lag_2      lag_3    lag_4 lag_5
# 1    1  1.3709584         NA         NA         NA       NA    NA
# 2    2 -0.5646982  1.3709584         NA         NA       NA    NA
# 3    3  0.3631284 -0.5646982  1.3709584         NA       NA    NA
# 4    4  0.6328626  0.3631284 -0.5646982  1.3709584       NA    NA
# 5    5  0.4042683  0.6328626  0.3631284 -0.5646982 1.370958    NA

4 Answers 4

4

You can transform using data.table more conveniently since shift from data.table allows the parameter n to be a vector while the lag function in dplyr does not.

library(data.table)
> setDT(data)[, paste("lag", 1:5, sep = "_") := shift(value, 1:5)]
> data
   time      value      lag_1      lag_2      lag_3     lag_4 lag_5
1:    1 -1.4162466         NA         NA         NA        NA    NA
2:    2 -0.2366333 -1.4162466         NA         NA        NA    NA
3:    3  0.5146632 -0.2366333 -1.4162466         NA        NA    NA
4:    4  1.9243923  0.5146632 -0.2366333 -1.4162466        NA    NA
5:    5  1.6161165  1.9243923  0.5146632 -0.2366333 -1.416247    NA

To be more specific about the shift and lag, here is an example of how the lag function does not allow you to do what shift does.

> vec <- 1:10
> shift(vec, 1:2)
[[1]]
 [1] NA  1  2  3  4  5  6  7  8  9

[[2]]
 [1] NA NA  1  2  3  4  5  6  7  8

> lag(vec, 1:2)
Error in lag(vec, 1:2) : n must be a single positive integer
In addition: Warning message:
In if (n == 0) return(x) :
  the condition has length > 1 and only the first element will be used
0
1

bind_cols may be more convenient than mutate, as it's easy to generate a data.frame of lags, either with a vectorized version of dplyr::lag or data.table::shift:

data %>% bind_cols(setNames(data.frame(Vectorize(lag, 'n')(.$value, 1:5)), 
                            paste0('lag_', 1:5)))
# Source: local data frame [5 x 7]
# 
#    time      value      lag_1      lag_2      lag_3    lag_4 lag_5
#   (int)      (dbl)      (dbl)      (dbl)      (dbl)    (dbl) (dbl)
# 1     1  1.3709584         NA         NA         NA       NA    NA
# 2     2 -0.5646982  1.3709584         NA         NA       NA    NA
# 3     3  0.3631284 -0.5646982  1.3709584         NA       NA    NA
# 4     4  0.6328626  0.3631284 -0.5646982  1.3709584       NA    NA
# 5     5  0.4042683  0.6328626  0.3631284 -0.5646982 1.370958    NA

data %>% bind_cols(data.frame(shift(.$value, 1:5, give.names = TRUE)))
# Source: local data frame [5 x 7]
# 
#    time      value   V1_lag_1   V1_lag_2   V1_lag_3 V1_lag_4 V1_lag_5
#   (int)      (dbl)      (dbl)      (dbl)      (dbl)    (dbl)    (dbl)
# 1     1  1.3709584         NA         NA         NA       NA       NA
# 2     2 -0.5646982  1.3709584         NA         NA       NA       NA
# 3     3  0.3631284 -0.5646982  1.3709584         NA       NA       NA
# 4     4  0.6328626  0.3631284 -0.5646982  1.3709584       NA       NA
# 5     5  0.4042683  0.6328626  0.3631284 -0.5646982 1.370958       NA

Alternately, you can use plain old data.frame instead:

data.frame(data, shift(data$value, 1:5, give.names = TRUE))
2
  • I think this is closest in spirit (without using reference semantics) to what I was looking for... Is there a way to generalize to a "grouped mutate" where the lag depends on the grouping?
    – kevinykuo
    Commented May 10, 2016 at 13:48
  • That sounds like another question, and is not very clear without an example. If I understand you correctly, you can lag groups, but lag can't really calculate anything in its second parameter (even 2-1 fails). shift is a little better, but still can't handle n(), which would be useful, but you could likely make the whole thing work in pure data.table.
    – alistaire
    Commented May 10, 2016 at 15:44
0

This can be done with base R by using lapply. We loop through the k using lapply and assign the output to new columns in 'data'.

data[paste("lag", 1:5, sep="_")] <- lapply(1:5, function(i) lag(data$value, i))
data
#  time      value      lag_1      lag_2      lag_3    lag_4 lag_5
#1    1  1.3709584         NA         NA         NA       NA    NA
#2    2 -0.5646982  1.3709584         NA         NA       NA    NA
#3    3  0.3631284 -0.5646982  1.3709584         NA       NA    NA
#4    4  0.6328626  0.3631284 -0.5646982  1.3709584       NA    NA
#5    5  0.4042683  0.6328626  0.3631284 -0.5646982 1.370958    NA
0

I had the same issue. This is how I did it:

set.seed(42)

data <- data.frame(time = 1:5, value = rnorm(5))

lags<-5

lags.tmp <-funs_(sapply(1:lags, function(x) paste0("lag(.,",x,")")))
names(lags.tmp)<-sapply(1:lags, function(x) paste0("lag_",x))

data %>%
  mutate_at(vars(value),.funs=lags.tmp)
#  time      value      lag_1      lag_2      lag_3    lag_4 lag_5
#1    1  1.3709584         NA         NA         NA       NA    NA
#2    2 -0.5646982  1.3709584         NA         NA       NA    NA
#3    3  0.3631284 -0.5646982  1.3709584         NA       NA    NA
#4    4  0.6328626  0.3631284 -0.5646982  1.3709584       NA    NA
#5    5  0.4042683  0.6328626  0.3631284 -0.5646982 1.370958    NA

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.