28

This problem is also known as 'transforming a "start-end" dataset to a panel dataset'

I have a data frame containing "name" of U.S. Presidents, the years when they start and end in office, ("from" and "to" columns). Here is a sample:

presidents <- data.frame(
  name = c("Bill Clinton", "George W. Bush", "Barack Obama"),
  from = c(1993, 2001, 2009),
  to = c(2001, 2009, 2012)
)
presidents
#>             name from   to
#> 1   Bill Clinton 1993 2001
#> 2 George W. Bush 2001 2009
#> 3   Barack Obama 2009 2012

I want to create data frame with two columns ("name" and "year"), with a row for each year that a president was in office. Thus, I need to create a regular sequence with each year from "from", to "to". Here's my expected out:

name           year
Bill Clinton   1993
Bill Clinton   1994
...
Bill Clinton   2000
Bill Clinton   2001
George W. Bush 2001
George W. Bush 2002
... 
George W. Bush 2008
George W. Bush 2009
Barack Obama   2009
Barack Obama   2010
Barack Obama   2011
Barack Obama   2012

I know that I can use data.frame(name = "Bill Clinton", year = seq(1993, 2001)) to expand things for a single president, but I can't figure out how to iterate for each president.

How do I do this? I feel that I should know this, but I'm drawing a blank.

Update 1

OK, I've tried both solutions, and I'm getting an error:

foo<-structure(list(name = c("Grover Cleveland", "Benjamin Harrison", "Grover Cleveland"), from = c(1885, 1889, 1893), to = c(1889, 1893, 1897)), .Names = c("name", "from", "to"), row.names = 22:24, class = "data.frame")
ddply(foo, "name", summarise, year = seq(from, to))
Error in seq.default(from, to) : 'from' must be of length 1
0

11 Answers 11

22

Here's a data.table solution. It has the nice (if minor) feature of leaving the presidents in their supplied order:

library(data.table)
dt <- data.table(presidents)
dt[, list(year = seq(from, to)), by = name]
#               name year
#  1:   Bill Clinton 1993
#  2:   Bill Clinton 1994
#  ...
#  ...
# 21:   Barack Obama 2011
# 22:   Barack Obama 2012

Edit: To handle presidents with non-consecutive terms, use this instead:

dt[, list(year = seq(from, to)), by = c("name", "from")]
3
  • Alas, if you have more data columns than just name they won't be present in the result. (I know the question's example doesn't require that, but many use cases do.)
    – Esteis
    Commented Jan 6, 2023 at 20:24
  • 1
    @Esteis -- If you have other columns that you'd like to keep -- say birthdate and state -- you can keep them like so: dt[, list(birthdate, state, year = seq(from, to)), by = name]. Commented Jan 7, 2023 at 18:12
  • 1
    @Esteis -- Or, if you'd like to keep all columns except from and to, you could do this: merge(dt[, !c("from", "to")], dt[, list(year = seq(from, to)), by = name], by="name"). Commented Jan 7, 2023 at 18:17
19

You can use the plyr package:

library(plyr)
ddply(presidents, "name", summarise, year = seq(from, to))
#              name year
# 1    Barack Obama 2009
# 2    Barack Obama 2010
# 3    Barack Obama 2011
# 4    Barack Obama 2012
# 5    Bill Clinton 1993
# 6    Bill Clinton 1994
# [...]

and if it is important that the data be sorted by year, you can use the arrange function:

df <- ddply(presidents, "name", summarise, year = seq(from, to))
arrange(df, df$year)
#              name year
# 1    Bill Clinton 1993
# 2    Bill Clinton 1994
# 3    Bill Clinton 1995
# [...]
# 21   Barack Obama 2011
# 22   Barack Obama 2012

Edit 1: Following's @edgester's "Update 1", a more appropriate approach is to use adply to account for presidents with non-consecutive terms:

adply(foo, 1, summarise, year = seq(from, to))[c("name", "year")]
4
  • You're solution works for most of the data. Please see my update.
    – edgester
    Commented Jul 16, 2012 at 1:04
  • 1
    The adply solution was the only one that worked without the error "Error in seq.default(from, to) : 'from' must be of length 1". Thanks for providing a working solution. Can you explain why I'm getting the "must be of length 1" errors for the other solutions?
    – edgester
    Commented Aug 7, 2012 at 0:45
  • 1
    Both @JoshOBrien's and mine work on your example data, so it is hard to say without looking at your full data. Maybe you can trim your data down to a subset that reproduces the error you see? Then we may be able to help.
    – flodel
    Commented Aug 7, 2012 at 17:30
  • Alas, if you have more data columns than just name they won't be present in the result. (I know the question's example doesn't require that, but many use cases do.)
    – Esteis
    Commented Jan 6, 2023 at 20:29
15

Some alternate tidyverse approaches:

Using reframe() and mapply():

library(tidyverse)

presidents %>%
  reframe(year = mapply(seq, from, to), .by = -c(from, to))

#              name  year
# 1    Bill Clinton  1993
# 2    Bill Clinton  1994
...
# 21   Barack Obama  2011
# 22   Barack Obama  2012

Using map2() and unnest():

presidents %>%
  mutate(year = map2(from, to, seq), .keep = "unused") %>%
  unnest(year)

#              name  year
# 1    Bill Clinton  1993
# 2    Bill Clinton  1994
...
# 21   Barack Obama  2011
# 22   Barack Obama  2012
1
  • 2
    To avoid the select, one can use .keep = "unused": presidents %>% mutate(year = map2(from, to, seq), .keep = "unused") %>% unnest(year)
    – Maël
    Commented Jan 5, 2023 at 12:57
8

Here's a dplyr solution:

library(dplyr)

# the data
presidents <- 
structure(list(name = c("Bill Clinton", "George W. Bush", "Barack Obama"
), from = c(1993, 2001, 2009), to = c(2001, 2009, 2012)), .Names = c("name", 
"from", "to"), row.names = 42:44, class = "data.frame")

# the expansion of the table
presidents %>%
    rowwise() %>%
    do(data.frame(name = .$name, year = seq(.$from, .$to, by = 1)))

# the output
Source: local data frame [22 x 2]
Groups: <by row>

             name  year
            (chr) (dbl)
1    Bill Clinton  1993
2    Bill Clinton  1994
3    Bill Clinton  1995
4    Bill Clinton  1996
5    Bill Clinton  1997
6    Bill Clinton  1998
7    Bill Clinton  1999
8    Bill Clinton  2000
9    Bill Clinton  2001
10 George W. Bush  2001
..            ...   ...

h/t: https://stackoverflow.com/a/24804470/1036500

8

Two base solutions.

Using sequence:

len = d$to - d$from + 1
data.frame(name = d$name[rep(1:nrow(d), len)], year = sequence(len, d$from))

Using mapply:

l <- mapply(`:`, d$from, d$to) 
data.frame(name = d$name[rep(1:nrow(d), lengths(l))], year = unlist(l))

#              name year
# 1    Bill Clinton 1993
# 2    Bill Clinton 1994
# ...snip
# 8    Bill Clinton 2000
# 9    Bill Clinton 2001
# 10 George W. Bush 2001
# 11 George W. Bush 2002
# ...snip
# 17 George W. Bush 2008
# 18 George W. Bush 2009
# 19   Barack Obama 2009
# 20   Barack Obama 2010
# 21   Barack Obama 2011
# 22   Barack Obama 2012

As noted by @Esteis in comment, there may well be several columns that needs to be repeated following the expansion of the ranges (not only 'name', like in OP). In such case, instead of repeating values of a single column, simply repeat the rows of the entire data frame, except the 'from' & 'to' columns. A simple example:

d = data.frame(x = 1:2, y = 3:4, names = c("a", "b"),
               from = c(2001, 2011), to = c(2003, 2012))
#   x y names from   to
# 1 1 3     a 2001 2003
# 2 2 4     b 2011 2012

len = d$to - d$from + 1

cbind(d[rep(1:nrow(d), len), setdiff(names(d), c("from", "to"))],
      year = sequence(len, d$from))

    x y names year
1   1 3     a 2001
1.1 1 3     a 2002
1.2 1 3     a 2003
2   2 4     b 2011
2.1 2 4     b 2012
2
  • 1
    Alas, if you have more data columns than just name they won't be present in the result. (I know the question's example doesn't require that, but many use cases do.)
    – Esteis
    Commented Jan 6, 2023 at 20:27
  • 1
    @ Henrik, thank you very much for your quick reply. I have proposed an edit that incorporates your elegant general solution. If the edit is unwelcome, and you prefer to keep the addendum in the comments, please feel free to roll it back. P.s. my comments on this old question+answers were prompted by stackoverflow.com/q/75018688, which explicitly needs a solution that handles multiple comments.
    – Esteis
    Commented Jan 6, 2023 at 21:07
3

Here is a quick base-R solution, where Df is your data.frame:

do.call(rbind, apply(Df, 1, function(x) {
  data.frame(name=x[1], year=seq(x[2], x[3]))}))

It gives some warnings about row names, but appears to return the correct data.frame.

2
  • Alas, if you have more data columns than just name they won't be present in the result. (I know the question's example doesn't require that, but many use cases do.)
    – Esteis
    Commented Jan 6, 2023 at 20:28
  • Warnings of mean slow code so this helps get rid of that: do.call(rbind, apply(presidents, 1, function(x) { data.frame(name=rep(x[1],as.numeric(x[3])-as.numeric(x[2])+1), year=x[2]:x[3])}))
    – M.L.
    Commented May 18, 2023 at 17:59
3

Another option using tidyverse could be to gather data into long format, group_by name and create a sequence between from and to date.

library(tidyverse)

presidents %>%
  gather(key, date, -name) %>%
  group_by(name) %>%
  complete(date = seq(date[1], date[2]))%>%
  select(-key) 

# A tibble: 22 x 2
# Groups:   name [3]
#   name          date
#   <chr>        <dbl>
# 1 Barack Obama  2009
# 2 Barack Obama  2010
# 3 Barack Obama  2011
# 4 Barack Obama  2012
# 5 Bill Clinton  1993
# 6 Bill Clinton  1994
# 7 Bill Clinton  1995
# 8 Bill Clinton  1996
# 9 Bill Clinton  1997
#10 Bill Clinton  1998
# … with 12 more rows
1
  • Alas, if you have more data columns than just name they won't be present in the result. (I know the question's example doesn't require that, but many use cases do.)
    – Esteis
    Commented Jan 6, 2023 at 20:28
2

Another solution using dplyr and tidyr. It correctly preserves any data columns you have.

library(magrittr) # for pipes

df <- data.frame(
    tata = c('toto1', 'toto2'),
    from = c(2000, 2004),
    to = c(2001, 2009),
    measure1 = rnorm(2),
    measure2 = 10 * rnorm(2)
)

   tata from   to measure1 measure2
1 toto1 2000 2001   -0.575   -10.13
2 toto2 2004 2009   -0.258    17.37

df %>% 
  dplyr::rowwise() %>%
  dplyr::mutate(year = list(seq(from, to))) %>%
  dplyr::select(-from, -to) %>%
  tidyr::unnest(c(year))

# A tibble: 8 x 4
  tata  measure1 measure2  year
  <chr>    <dbl>    <dbl> <int>
1 toto1   -0.575    -10.1  2000
2 toto1   -0.575    -10.1  2001
3 toto2   -0.258     17.4  2004
4 toto2   -0.258     17.4  2005
5 toto2   -0.258     17.4  2006
6 toto2   -0.258     17.4  2007
7 toto2   -0.258     17.4  2008
8 toto2   -0.258     17.4  2009
0
1

Use by to create a by list L of data.frames, one data.frame per president, and then rbind them together. No packages are used.

L <- by(presidents, presidents$name, with, data.frame(name, year = from:to))
do.call("rbind", setNames(L, NULL))

If you don't mind row names then the last line could be reduced to just:

do.call("rbind", L)
2
  • Alas, if you have more data columns than just name they won't be present in the result. (I know the question's example doesn't require that, but many use cases do.)
    – Esteis
    Commented Jan 6, 2023 at 20:28
  • It would be trivial to add them if that were the question. Commented Jan 6, 2023 at 21:23
1

An addition to the tidyverse solutions can be:

df %>%
 uncount(to - from + 1) %>%
 group_by(name) %>%
 transmute(year = seq(first(from), first(to)))

   name            year
   <chr>          <dbl>
 1 Bill Clinton    1993
 2 Bill Clinton    1994
 3 Bill Clinton    1995
 4 Bill Clinton    1996
 5 Bill Clinton    1997
 6 Bill Clinton    1998
 7 Bill Clinton    1999
 8 Bill Clinton    2000
 9 Bill Clinton    2001
10 George W. Bush  2001
1
  • Alas, if you have more data columns than just name they won't be present in the result. (I know the question's example doesn't require that, but many use cases do.)
    – Esteis
    Commented Jan 6, 2023 at 20:28
0

Here's another base R solution that should be fairly speedy:

a <- lapply(1:nrow(presidents),function(a){
  data.frame(
    name=rep(presidents$name[a],presidents$to[a]-presidents$from[a]+1),
    year=presidents$from[a]:presidents$to[a]
    )
  })
do.call('rbind',a)

and any data you want to keep can be handled in the same way as the name column.

benchmark against the other Base R solution by Jason Morgan above, as I was curious:

f_max <- function(x){
  a <- lapply(1:nrow(x),function(a){
    data.frame(
      name=rep(x$name[a],x$to[a]-x$from[a]+1),
      year=x$from[a]:x$to[a]
    )
  })
  do.call(rbind,a)
}

f_jason <- function(y){
  do.call(rbind, apply(y, 1, function(x) {
    data.frame(name=x[1], year=seq(x[2], x[3]))}))
}

f_combined <- function(y){
  do.call(rbind, apply(y, 1, function(x) {
    data.frame(name=rep(x[1],as.numeric(x[3])-as.numeric(x[2])+1), year=x[2]:x[3])}))
}

r <- f_jason(presidents)
all(r==f_max(presidents))
all(r==f_combined(presidents))
res <- microbenchmark(f_jason(presidents),f_combined(presidents),f_max(presidents))
print(res, order="mean")

                   expr   min     lq    mean median      uq    max neval cld
      f_max(presidents) 436.9 462.75 561.074 482.30  521.25 5601.0   100  a 
 f_combined(presidents) 566.5 605.95 796.029 639.70  723.60 7548.7   100   b
    f_jason(presidents) 770.2 829.70 998.108 906.15 1011.85 4891.0   100   b

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.