2

I'm migrating from SAS to R. I need help figuring out how to sum up weather data for date ranges. In SAS, I take the date ranges, use a data step to create a record for every date (with startdate, enddate, date) in the range, merge with weather and then summarize (VAR hdd cdd; CLASS=startdate enddate sum=) to sum up the values for the date range.

R code:

startdate <- c(100,103,107)
enddate <- c(105,104,110)
billperiods <-data.frame(startdate,enddate);

to get:

> billperiods
startdate enddate
1       100     105
2       103     104
3       107     110

R code:

weatherdate <- c(100:103,105:110)
hdd <- c(0,0,4,5,0,0,3,1,9,0)
cdd <- c(4,1,0,0,5,6,0,0,0,10)
weather <- data.frame(weatherdate,hdd,cdd)

to get:

> weather
   weatherdate hdd cdd
1          100   0   4
2          101   0   1
3          102   4   0
4          103   5   0
5          105   0   5
6          106   0   6
7          107   3   0
8          108   1   0
9          109   9   0
10         110   0  10

Note: weatherdate = 104 is missing. I may not have weather for a day.

I can't figure out how to get to:

> billweather
  startdate enddate sumhdd sumcdd
1       100     105      9     10
2       103     104      5      0
3       107     110     13     10

where sumhdd is the sum of the hdd's from startdate to enddate in the weather data.frame.

Any ideas?

3 Answers 3

3

Here's a method using IRanges and data.table. Seemingly, for this question, this answer may seem kind of an overkill. But in general, I find it convenient to use IRanges to deal with intervals, how simple they may be.

# load packages
require(IRanges)
require(data.table)

# convert data.frames to data.tables
dt1 <- data.table(billperiods)
dt2 <- data.table(weather)

# construct Ranges to get overlaps
ir1 <- IRanges(dt1$startdate, dt1$enddate)
ir2 <- IRanges(dt2$weatherdate, width=1) # start = end

# find Overlaps
olaps <- findOverlaps(ir1, ir2)

# Hits of length 10
# queryLength: 3
# subjectLength: 10
#    queryHits subjectHits 
#     <integer>   <integer> 
#  1          1           1 
#  2          1           2 
#  3          1           3 
#  4          1           4 
#  5          1           5 
#  6          2           4 
#  7          3           7 
#  8          3           8 
#  9          3           9 
#  10         3          10 

# get billweather (final output)
billweather <- cbind(dt1[queryHits(olaps)], 
                dt2[subjectHits(olaps), 
                list(hdd, cdd)])[, list(sumhdd = sum(hdd), 
                sumcdd = sum(cdd)), 
                by=list(startdate, enddate)]

#    startdate enddate sumhdd sumcdd
# 1:       100     105      9     10
# 2:       103     104      5      0
# 3:       107     110     13     10

Code breakdown for last line: First I construct using queryHits, subjectHits and cbind a mid-way data.table from which then, I group by startdate, enddate and get the sum of hdd and sum of cdd. It is easier to look at the line separately as shown below for better understanding.

# split for easier understanding
billweather <- cbind(dt1[queryHits(olaps)], 
            dt2[subjectHits(olaps), 
            list(hdd, cdd)])
billweather <- billweather[, list(sumhdd = sum(hdd), 
            sumcdd = sum(cdd)), 
            by=list(startdate, enddate)]
1
 cbind(billperiods, t(sapply(apply(billperiods, 1, function(x) 
     weather[weather$weatherdate >= x[1] & 
             weather$weatherdate <= x[2], c("hdd", "cdd")]), colSums)))

  startdate enddate hdd cdd
1       100     105   9  10
2       103     104   5   0
3       107     110  13  10
1
  • Thanks for the quick response! I tried it against the bigger data frame (12,356 rows) and it took 6.75 seconds and the results are good! Mar 25, 2013 at 22:08
1
billweather <- cbind(billperiods, 
                 t(apply(billperiods, 1, function(x) { 
                   colSums(weather[weather[, 1] %in% c(x[1]:x[2]), 2:3])
               })))
1
  • Thanks for the quick response! I tried it against the bigger data frame (12,356 rows) and it took 7.89 seconds and the results are good! I'm surprised how fast people responded. This is the first time I asked a question in here. Mar 25, 2013 at 22:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.