24

I have a sample dataframe "data" as follows:

X            Y  Month   Year    income
2281205 228120  3   2011    1000
2281212 228121  9   2010    1100
2281213 228121  12  2010    900
2281214 228121  3   2011    9000
2281222 228122  6   2010    1111
2281223 228122  9   2010    3000
2281224 228122  12  2010    1889
2281225 228122  3   2011    778
2281243 228124  12  2010    1111
2281244 228124  3   2011    200
2281282 228128  9   2010    7889
2281283 228128  12  2010    2900
2281284 228128  3   2011    3400
2281302 228130  9   2010    1200
2281303 228130  12  2010    2000
2281304 228130  3   2011    1900
2281352 228135  9   2010    2300
2281353 228135  12  2010    1333
2281354 228135  3   2011    2340

I want to use the ddply to compute the income for each Y(not X), if I have four observations for each Y (for example for 2281223 with months 6,9,12 of 2010 and month 3 of 2011). If I have less than four observations (for example for Y =228130), I want to simply ignore it. I use the following commands in R for the above purpose:

require(plyr)
     # the data are in the data csv file
    data<-read.csv("data.csv")
    # convert Y (integers) into factors
    y<-as.factor(y)
    # get the count of each unique Y
    count<-ddply(data,.(Y), summarize, freq=length(Y))
    # get the sum of each unique Y 
    sum<-ddply(data,.(Y),summarize,tot=sum(income))
    # show the sum if number of observations for each Y is less than 4
    colbind<-cbind(count,sum)
    finalsum<-subset(colbind,freq>3)

My output are as follows:

>colbind
       Y freq      Y   tot
1 228120    1 228120  1000
2 228121    3 228121 11000
3 228122    4 228122  6778
4 228124    2 228124  1311
5 228128    3 228128 14189
6 228130    3 228130  5100
7 228135    3 228135  5973
>finalsum
       Y freq    Y.1  tot
3 228122    4 228122 6778

The above code works, but requires many steps. So,I would like to know whether there is a simple way of performing the above task (using the plyr package).

1
  • 1
    you can create both freq and tot variables in one go with summarise, and probably don't need to convert Y to factor.
    – baptiste
    Commented Dec 26, 2012 at 4:09

2 Answers 2

35

As pointed out in a comment, you can do multiple operations inside the summarize.

This reduces your code to one line of ddply() and one line of subsetting, which is easy enough with the [ operator:

x <- ddply(data, .(Y), summarize, freq=length(Y), tot=sum(income))
x[x$freq > 3, ]

       Y freq  tot
3 228122    4 6778

This is also exceptionally easy with the data.table package:

library(data.table)
data.table(data)[, list(freq=length(income), tot=sum(income)), by=Y][freq > 3]
        Y freq  tot
1: 228122    4 6778

In fact, the operation to calculate the length of a vector has its own shortcut in data.table - use the .N shortcut:

data.table(data)[, list(freq=.N, tot=sum(income)), by=Y][freq > 3]
        Y freq  tot
1: 228122    4 6778
2
  • Thanks. I used my and your codes to my extended sample with N (number of observation) around 35000. It took around 200 secs to execute both the codes. Is this normal in ddply function?
    – Metrics
    Commented Dec 26, 2012 at 14:50
  • 3
    Yes. plyr is extremely convenient but can be slow, especially compared to data.table.
    – Andrie
    Commented Dec 26, 2012 at 18:53
21

I think the package dplyr is faster than plyr::ddply and more elegant.

testData <- read.table(file = "clipboard",header = TRUE)
require(dplyr)
testData %>%
  group_by(Y) %>%
  summarise(total = sum(income),freq = n()) %>%
  filter(freq > 3)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.