0

I have a data frame where each row is a record with its id, start and end dates. I would like to create another data frame that contains every calendar month's start dates (eg "2020-01-01" is January), and a second column counting how many unique records are open (for any/all portion of) that month.

I could create new columns for each calendar month and generate dummies for whether a record is open that month, then add up each column. What's a more efficient way of doing this?

ds <- data.frame(record_id = c("00a", "00b", "00c"),
                 record_start_date = as.Date(c("2020-01-16", "2020-03-25", "2020-02-22")),
                 record_end_date = as.Date(c("2020-12-05", "2020-06-21", "2020-11-12")))
3
  • 1
    Look at the tidyr::complete function
    – Julien
    Aug 31, 2022 at 21:16
  • 1
    Can you spell out more specifically what "how many records are open that month" means? Unique records (can an id count more than once in a month)? As of the start/end of the month, or open at any point in the month?
    – Jon Spring
    Aug 31, 2022 at 22:00
  • Thanks I edited my question to make it clearer. In this case I'm looking for a unique count of all records that may be open for some/all of a given month
    – RV702
    Aug 31, 2022 at 22:39

2 Answers 2

1

The ivs package was created for working with intervals like this. iv_count_between() is perfect for this problem.

library(ivs)
library(dplyr)
library(clock)

ds <- data.frame(
  record_id = c("00a", "00b", "00c"),
  record_start_date = as.Date(c("2020-01-16", "2020-03-25", "2020-02-22")),
  record_end_date = as.Date(c("2020-12-05", "2020-06-21", "2020-11-12"))
)

# Record the start and end months to generate the counts for
start <- date_start(min(ds$record_start_date), "year")
end <- date_end(max(ds$record_end_date), "year") + 1L

# Construct an interval vector
ds <- ds %>%
  mutate(
    record_range = iv(record_start_date, record_end_date), 
    .keep = "unused"
  )

ds
#>   record_id             record_range
#> 1       00a [2020-01-16, 2020-12-05)
#> 2       00b [2020-03-25, 2020-06-21)
#> 3       00c [2020-02-22, 2020-11-12)

# Generate the months sequence to count along
result <- tibble(
  month = date_seq(
    from = start, 
    to = end, 
    by = duration_months(1)
  )
)

# Count the number of times `month[[i]]` is between any of the
# ranges in `ds$record_range`
result %>%
  mutate(
    count = iv_count_between(month, ds$record_range)
  )
#> # A tibble: 13 × 2
#>    month      count
#>    <date>     <int>
#>  1 2020-01-01     0
#>  2 2020-02-01     1
#>  3 2020-03-01     2
#>  4 2020-04-01     3
#>  5 2020-05-01     3
#>  6 2020-06-01     3
#>  7 2020-07-01     2
#>  8 2020-08-01     2
#>  9 2020-09-01     2
#> 10 2020-10-01     2
#> 11 2020-11-01     2
#> 12 2020-12-01     1
#> 13 2021-01-01     0

Created on 2022-09-01 with reprex v2.0.2

1

Here's an approach where we reshape the data and add rows for each month start. Then it can be a very efficient vectorized cumulative count to figure out the active records as of the end of the 1st of each month. If you want to count a record that ends on the 1st (or one that ends the same day it began) toward the count, you could add a line to shift end dates one day later.

library(tidyverse); library(lubridate)
ds %>%
  pivot_longer(-record_id) %>%
  mutate(change = if_else(name == "record_start_date", 1, -1)) %>%
  # mutate(value = value + if_else(name == "record_end_date", 1, 0)) %>%
  add_row(name = "month_start", 
          value = seq.Date(floor_date(min(ds$record_start_date), "month"),
                           floor_date(max(ds$record_end_date), "month"),
                           by = "month"),
          change = 0) %>%
  arrange(value, desc(name)) %>%
  mutate(count = cumsum(change)) %>%
  filter(name == "month_start") %>% 
  select(value, count)

Result:

# A tibble: 12 × 2
   value      count
   <date>     <dbl>
 1 2020-01-01     0
 2 2020-02-01     1
 3 2020-03-01     2
 4 2020-04-01     3
 5 2020-05-01     3
 6 2020-06-01     3
 7 2020-07-01     2
 8 2020-08-01     2
 9 2020-09-01     2
10 2020-10-01     2
11 2020-11-01     2
12 2020-12-01     1

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.