31

I've spent hours browsing everywhere now to try to create a multiindex from dataframe in pandas. This is the dataframe I have (posting excel sheet mockup. I do have this in pandas dataframe):

have

And this is what I want:

want

I have tried

newmulti = currentDataFrame.set_index(['user_id','account_num'])

But it returns a dataframe, not a multiindex. Also, I could not figure out how to make 'user_id' level 0 and 'account_num' level 1. I think this must be trivial but I've read so many posts, tutorials, etc. and still could not figure it out. Partly because I'm a very visual person and most posts are not. Please help!

3
  • For processing purposes, both the tables are the same. But for display purposes, I suggest you to refer to: stackoverflow.com/a/25127764/2306662
    – nikpod
    Commented Jun 8, 2017 at 18:30
  • But I thought I need multi-index if, say I want to plot total sales (of all account) vs. dates?
    – puifais
    Commented Jun 8, 2017 at 18:32
  • @puifais why can't you plot the second dataframe you've put together?
    – Andrew L
    Commented Jun 8, 2017 at 18:51

6 Answers 6

37

You could simply use groupby in this case, which will create the multi-index automatically when it sums the sales along the requested columns.

df.groupby(['user_id', 'account_num', 'dates']).sales.sum().to_frame()

You should also be able to simply do this:

df.set_index(['user_id', 'account_num', 'dates'])

Although you probably want to avoid any duplicates (e.g. two or more rows with identical user_id, account_num and date values but different sales figures) by summing them, which is why I recommended using groupby.

If you need the multi-index, you can simply access viat new_df.index where new_df is the new dataframe created from either of the two operations above.

And user_id will be level 0 and account_num will be level 1.

3
  • So this means, group by user_id, account_num, and dates and pull out sales data. if sales data so happen to have the same user_id, account_num, and dates, then sum them. is that right?
    – puifais
    Commented Jun 9, 2017 at 20:54
  • 1
    Sort of... It means you are aggregating the sales data via sum. If the column was not numeric, you wouldn't be able to sum it. You would have to use something like first, last or unique with a lambda function.
    – Alexander
    Commented Jun 9, 2017 at 21:13
  • What if I have more columns, other than sales? Commented Mar 20, 2020 at 3:15
8

For clarification of future users I would like to add the following:

As said by Alexander,

df.set_index(['user_id', 'account_num', 'dates'])

with a possible inplace=True does the job.

The type(df) gives

pandas.core.frame.DataFrame

whereas type(df.index) is indeed the expected

pandas.core.indexes.multi.MultiIndex
3

Use pd.MultiIndex.from_arrays

lvl0 = currentDataFrame.user_id.values
lvl1 = currentDataFrame.account_num.values

midx = pd.MultiIndex.from_arrays([lvl0, lvl1], names=['level 0', 'level 1'])
2

There are two ways to do it, albeit not exactly like you have shown, but it works.
Say you have the following df:

      A   B    C      D
0   nil one    1    NaN
1   bar one    5    5.0
2   foo two    3    8.0
3   bar three  2    1.0
4   foo two    4    2.0
5   bar two    6    NaN

1. Workaround 1:

df.set_index('A', append = True, drop = False).reorder_levels(order = [1,0]).sort_index()

This will return:

enter image description here

2. Workaround 2:

df.set_index(['A', 'B']).sort_index()

This will return:
enter image description here

1

The DataFrame returned by currentDataFrame.set_index(['user_id','account_num']) has it's index set to ['user_id','account_num']

newmulti.index will return the MultiIndex object.

2
  • 1
    Um...I don't understand. So do I do newmulti = currentDataFrame.set_index(['user_id','account_num']) and then newmultiReal = newmulti.index? Would you please clarify? I'm totally new at pandas.
    – puifais
    Commented Jun 8, 2017 at 18:33
  • What is your end goal? If you want the original dataframe with a multiindex, you've already got it.
    – user2047399
    Commented Jun 8, 2017 at 18:46
0

You can do the following to get your desired result:

newmulti = currentDataFrame.set_index(['user_id','account_num'])
newmulti.sort_index(level=[0,1], axis=0, inplace=True)
1
  • Thank you for your interest in contributing to the Stack Overflow community. This question already has quite a few answers—including one that has been extensively validated by the community. Are you certain your approach hasn’t been given previously? If so, it would be useful to explain how your approach is different, under what circumstances your approach might be preferred, and/or why you think the previous answers aren’t sufficient. Can you kindly edit your answer to offer an explanation? Commented Jul 15 at 1:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.