4

Is there a more efficient way to use pandas groupby or pandas.core.groupby.DataFrameGroupBy object to create a unique list, series or dataframe, where I want unique combinations of 2 of N columns. E.g., if I have columns: Date, Name, Item Purchased and I just want to know unique Name and Date combination this works fine:

y = x.groupby(['Date','Name']).count()
y = y.reset_index()[['Date', 'Name']]

but I feel like there should be a cleaner way using

y = x.groupby(['Date','Name'])

but y.index gives me an error, although y.keys works. This actually leads me to ask the general question as what are pandas.core.groupby.DataFrameGroupBy objects convenient for?

Thanks!

2
  • What do you need that y.keys doesn't give you?
    – BrenBarn
    Aug 27, 2014 at 19:49
  • "y.index gives me an error," please can you show the error? ideally with an example DataFrame which demonstrates it. Aug 27, 2014 at 19:54

1 Answer 1

6

You don't need to use -- and in fact shouldn't use -- groupby here. You could use drop_duplicates to get unique rows instead:

x.drop_duplicates(['Date','Name'])

Demo:

In [156]: x = pd.DataFrame({'Date':[0,1,2]*2, 'Name':list('ABC')*2})

In [158]: x
Out[158]: 
   Date Name
0     0    A
1     1    B
2     2    C
3     0    A
4     1    B
5     2    C

In [160]: x.drop_duplicates(['Date','Name'])
Out[160]: 
   Date Name
0     0    A
1     1    B
2     2    C

You shouldn't use groupby because

  1. x.groupby(['Date','Name']).count() performs a count of the number of elements in each group, but the count is not used -- it's a wasted computation.
  2. x.groupby(['Date','Name']).count() raises an AttributeError if x has only Date and Name columns.
  3. drop_duplicates is much much faster for this purpose.

Use groupby when you want to perform some operation on each group, such as counting the number of elements in each group, or computing some statistic (e.g. a sum or mean, etc.) per group.

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.