I'm trying to apply simple functions to groups in pandas. I have this dataframe which I can group by type
:
df = pandas.DataFrame({"id": ["a", "b", "c", "d"], "v": [1,2,3,4], "type": ["X", "Y", "Y", "Y"]}).set_index("id")
df.groupby("type").mean() # gets the mean per type
I want to apply a function like np.log2
only to the groups before taking the mean of each group. This does not work since apply
is element wise and type
(as well as potentially other columns in df
in a real scenario) is not numeric:
# fails
df.apply(np.log2).groupby("type").mean()
is there a way to apply np.log2
only to the groups prior to taking the mean? I thought transform
would be the answer but the problem is that it returns a dataframe that does not have the original type
columns:
df.groupby("type").transform(np.log2)
v
id
a 0.000000
b 1.000000
c 1.584963
d 2.000000
Variants like grouping and then applying do not work: df.groupby("type").apply(np.log2)
. What is the correct way to do this?
applymap
on a frame for an element-wise applyer (before grouping); if you want to do:df.groupby('type').apply(lambda x: x.applymap(np.log2))
; better yetnp.log2(df._get_numeric_data())
('Not implemented for this type', u'occurred at index type')
. The second one works but it drops thetype
, so you can't group afterwards._get_numeric_data()
can't be used with groups I believe. So can't think of how to use the second one to applynp.log2
to numeric data only and then group or group first and then apply only to groups