Suppose we have such df (user may have multiple rows on the same date):
df = pd.DataFrame({"user_id" : ["A"] * 5 + ["B"] * 5,
"hour" : [10] * 10,
"date" : ["2018-01-16", "2018-01-16","2018-01-18","2018-01-19","2018-02-16","2018-01-16", "2018-01-16","2018-01-18","2018-01-19","2018-02-16"], "amount" : [1] * 10})
df['date'] = pd.to_datetime(df['date'])
Output:
amount date hour user_id
0 1 2018-01-16 10 A
1 1 2018-01-16 10 A
2 1 2018-01-18 10 A
3 1 2018-01-19 10 A
4 1 2018-02-16 10 A
5 1 2018-01-16 10 B
6 1 2018-01-16 10 B
7 1 2018-01-18 10 B
8 1 2018-01-19 10 B
9 1 2018-02-16 10 B
I want to get agg rolling stats
for amount
by each user_id
and hour
. Currently I did it like that:
def get_rolling_stats(df, rolling_interval = 7) :
index_cols = ['user_id', 'hour', 'date']
grp = df.groupby(by = ['user_id', 'hour'], as_index = True, group_keys = False).rolling(window='%sD'%rolling_interval, on = 'date')
def agg_grp(grp, func):
res = grp.agg({'amount' : func})
res = res.reset_index()
res.drop_duplicates(index_cols, inplace = True, keep = 'last')
res.rename(columns = {'amount' : "amount_" + func}, inplace = True)
return res
grp1 = agg_grp(grp, "mean")
grp2 = agg_grp(grp, "count")
grp = grp1.merge(grp2, on = index_cols)
return grp
So it outputs:
user_id hour date amount_mean amount_count
0 A 10 2018-01-16 1.0 1.0
1 A 10 2018-01-18 1.0 3.0
2 A 10 2018-01-19 1.0 4.0
3 A 10 2018-02-16 1.0 1.0
4 B 10 2018-01-16 1.0 1.0
5 B 10 2018-01-18 1.0 3.0
6 B 10 2018-01-19 1.0 4.0
7 B 10 2018-02-16 1.0 1.0
But I want to exclude the current date from rolling window. So I want output like that:
user_id hour date amount_mean amount_count
0 A 10 2018-01-16 nan 0.0
1 A 10 2018-01-18 1.0 2.0
2 A 10 2018-01-19 1.0 3.0
3 A 10 2018-02-16 nan 0.0
4 B 10 2018-01-16 nan 0.0
5 B 10 2018-01-18 1.0 2.0
6 B 10 2018-01-19 1.0 3.0
7 B 10 2018-02-16 nan 0.0
I've read that rolling
method has arg closed
. But if I use it - it raises error : ValueError: closed only implemented for datetimelike and offset based windows
. I haven't found any example how to use it.Can someone shed some light how to properly implement get_rolling_stats
function ?