2

Suppose we have such df (user may have multiple rows on the same date):

df = pd.DataFrame({"user_id" : ["A"] * 5 + ["B"] * 5,
               "hour" : [10] * 10,
               "date" : ["2018-01-16", "2018-01-16","2018-01-18","2018-01-19","2018-02-16","2018-01-16", "2018-01-16","2018-01-18","2018-01-19","2018-02-16"], "amount" : [1] * 10})  
df['date'] = pd.to_datetime(df['date'])

Output:

amount  date    hour    user_id
0   1   2018-01-16  10  A
1   1   2018-01-16  10  A
2   1   2018-01-18  10  A
3   1   2018-01-19  10  A
4   1   2018-02-16  10  A
5   1   2018-01-16  10  B
6   1   2018-01-16  10  B
7   1   2018-01-18  10  B
8   1   2018-01-19  10  B
9   1   2018-02-16  10  B

I want to get agg rolling stats for amount by each user_id and hour. Currently I did it like that:

def get_rolling_stats(df, rolling_interval = 7) : 
    index_cols = ['user_id', 'hour', 'date']
    grp = df.groupby(by = ['user_id', 'hour'], as_index = True, group_keys = False).rolling(window='%sD'%rolling_interval, on = 'date')
    def agg_grp(grp, func):
        res = grp.agg({'amount' : func})

        res = res.reset_index()
        res.drop_duplicates(index_cols, inplace = True, keep = 'last')
        res.rename(columns = {'amount' : "amount_" + func}, inplace = True)
       return res

    grp1 = agg_grp(grp, "mean")
    grp2 = agg_grp(grp, "count")

    grp = grp1.merge(grp2, on = index_cols)
    return grp

So it outputs:

user_id hour    date    amount_mean amount_count
0   A   10  2018-01-16  1.0 1.0
1   A   10  2018-01-18  1.0 3.0
2   A   10  2018-01-19  1.0 4.0
3   A   10  2018-02-16  1.0 1.0
4   B   10  2018-01-16  1.0 1.0
5   B   10  2018-01-18  1.0 3.0
6   B   10  2018-01-19  1.0 4.0
7   B   10  2018-02-16  1.0 1.0

But I want to exclude the current date from rolling window. So I want output like that:

user_id hour    date    amount_mean amount_count
0   A   10  2018-01-16  nan 0.0
1   A   10  2018-01-18  1.0 2.0
2   A   10  2018-01-19  1.0 3.0
3   A   10  2018-02-16  nan 0.0
4   B   10  2018-01-16  nan 0.0
5   B   10  2018-01-18  1.0 2.0
6   B   10  2018-01-19  1.0 3.0
7   B   10  2018-02-16  nan 0.0

I've read that rolling method has arg closed. But if I use it - it raises error : ValueError: closed only implemented for datetimelike and offset based windows. I haven't found any example how to use it.Can someone shed some light how to properly implement get_rolling_stats function ?

1 Answer 1

1

Seems like I found example - https://pandas.pydata.org/pandas-docs/stable/computation.html#rolling-window-endpoints. And all I had to do is to replace:

grp = df.groupby(by = ['user_id', 'hour'], as_index = True, group_keys = False).rolling(window='%sD'%rolling_interval, on = 'date')

by

grp = df.set_index('date').groupby(by = ['user_id', 'hour'], as_index = True, group_keys = False).\
                   rolling(window='%sD'%rolling_interval, closed = 'neither')

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.