4

I would like to calculate Rolling Mean of dataframe groupby second level (Key2 in following code sample).

import pandas as pd
d = {'Key1':[1,1,1,2,2,2,3,3,3,4,4,4,5,5,5,6,6,6], 'Key2':[2,7,8,5,3,2,7,5,8,7,2,9,8,3,9,2,7,9],'Value':[1,2,3,1,2,3,1,2,3,1,2,3,1,2,3,1,2,3]}
df = pd.DataFrame(d)
df = df.set_index(['Key1', 'Key2'])
df['MA'] = (df.groupby('Key2')['Value']
                .rolling(window=3)
                .mean()
                .reset_index(level=0, drop=True))

print(df)

Expected output:

           Value        MA
Key1 Key2                 
1    2         1       NaN
     7         2       NaN
     8         3       NaN
2    5         1       NaN
     3         2       NaN
     2         3       NaN
3    7         1       NaN
     5         2       NaN
     8         3       NaN
4    7         1  1.333333
     2         2  2.000000
     9         3       NaN
5    8         1  2.333333
     3         2       NaN
     9         3       NaN
6    2         1  2.000000
     7         2  1.333333
     9         3  3.000000

But the actual output is NaN. It seems something wrong with the assignment. Actual output:

           Value        MA
Key1 Key2                 
1    2         1       NaN
     7         2       NaN
     8         3       NaN
2    5         1       NaN
     3         2       NaN
     2         3       NaN
3    7         1       NaN
     5         2       NaN
     8         3       NaN
4    7         1      NaN
     2         2       NaN
     9         3       NaN
5    8         1      NaN
     3         2       NaN
     9         3       NaN
6    2         1      NaN
     7         2       NaN
     9         3       NaN

Python 3.8 + Pandas 1.2.1. (Also tried on Python 3.7.9 + Pandas 1.1.5)

6
  • Yes, that's the expected behavior. By rolling(3), you only get non-nan values for Key2 with >=3 rows. You can pass min_periods=1 to rolling(3, min_periods=1). Is it what you expect? Feb 15, 2021 at 3:50
  • @Quang Hoang, I didn’t get the expected output. Please see the updated actual output.
    – Yoh
    Feb 15, 2021 at 4:29
  • 1
    The code returns the expected output on my system. You may have different Pandas version. You can try to print the groupby().rolling().mean() series to see if reset_index is needed. Feb 15, 2021 at 4:43
  • Could you please advise the version of python&pandas in your system?
    – Yoh
    Feb 15, 2021 at 5:13
  • 1
    Python 3.7 and Pandas 1.1.4. Feb 15, 2021 at 5:14

1 Answer 1

2

Use lambda function for avoid lost MultiIndex, so assign working well:

df['MA'] = df.groupby('Key2')['Value'].apply(lambda x: x.rolling(window=3).mean())
print(df)
           Value        MA
Key1 Key2                 
1    2         1       NaN
     7         2       NaN
     8         3       NaN
2    5         1       NaN
     3         2       NaN
     2         3       NaN
3    7         1       NaN
     5         2       NaN
     8         3       NaN
4    7         1  1.333333
     2         2  2.000000
     9         3       NaN
5    8         1  2.333333
     3         2       NaN
     9         3       NaN
6    2         1  2.000000
     7         2  1.333333
     9         3  3.000000
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.