1

this question is very similar to the one here:

Sum duplicated rows on a multi-index pandas dataframe

Except it is for a Pandas Series, not a Pandas DataFrame and the answers given and accepted for a DataFrame are not working on my Series.

Say I have a multi index pd.Series, called s, like so:

                  volume1  
year   product
2010   A          10         
       A          7          
       B          7          
2011   A          10         
       B          7          
       C          5     

Expected output : if there are duplicated products for a given year then we sum them. But for missing categories per year, I would like to record the sm as "0". So a Pandas Series like the following is something like I want the output to look like:

              volume1
year product         
2010 A             17
     B              7
     C              0
2011 A             10
     B              7
     C              5

I tried all the answers on the question I linked to that explain how to do this for a pd.DataFrame, such as:

s = s.sum(level=[0,1]).unstack(fill_value=0).stack()

and

s = s.sum(level=[0,1]).unstack().stack(dropna=False)

But none of these work and seemingly just fill the whole Series with NaN values. This is incredibly frustrating and there must be a quick fix I just cannot find. Thanks.

1
  • If you are trying to add missing categories as 0, that is an important detail and you should make sure to highlight it appropriately, because it is easy to miss otherwise. Right now you have one answer that handles it and another that doesn't. What is important to you?
    – cs95
    Dec 23, 2018 at 18:26

2 Answers 2

1

I think you're looking to unstack on the penultimate level.

s.sum(level=[0, 1]).unstack(1, fill_value=0).stack()

              volume1
year product         
2010 A             17
     B              7
     C              0
2011 A             10
     B              7
     C              5

Another option is to convert the first level to categorical, then unstacking is not needed (it is inefficient).

df.index = df.index.set_levels(pd.Categorical(df.index.levels[1]), level=1)
df.sum(level=[0, 1]).fillna(0, downcast='infer')

              volume1
year product         
2010 A             17
     B              7
     C              0
2011 A             10
     B              7
     C              5
0

You could groupby the index itself and sum inside the groups, something like this:

import pandas as pd

# create example series
index = pd.MultiIndex.from_tuples(tuples=[(2010, 'A'), (2010, 'A'), (2010, 'B'), (2011, 'A'), (2011, 'B'), (2011, 'C')],
                                  names=['year', 'product'])
s = pd.Series(data=[10, 7, 7, 10, 7, 5], index=index)

# group by index and sum
result = s.groupby(index).sum()

# re-index the resulting pd.Series
result = result.reindex(pd.MultiIndex.from_tuples(result.index, names=s.index.names))

print(result)

Output

year  product
2010  A          17
      B           7
2011  A          10
      B           7
      C           5
dtype: int64
4
  • Based on the code it seems they want to fill missing categories with 0.
    – cs95
    Dec 23, 2018 at 18:16
  • @coldspeed - Well the question, is then pretty confusing: he says and I quote: "Expected output : if there are duplicated products for a given year then we sum them.", and then shows an expected output, like the one would get from grouping by index. Dec 23, 2018 at 18:23
  • Indeed. Code never lies though. Wonder what they are trying to do... especially with that .stack(dropna=False) call... ;-)
    – cs95
    Dec 23, 2018 at 18:25
  • 1
    To complicate things further, if the point of the question was to just sum along duplicated indices, then the solution would've been s.sum(level=[0,1]), which they seem to already have figured out. It's the stuff that comes after that that has me stumped.
    – cs95
    Dec 23, 2018 at 18:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.