10

My data-structure is:

ds = [{
    "name": "groupA",
    "subGroups": [123,456]
},
{
    "name": "groupB",
    "subGroups": ['aaa', 'bbb' , 'ccc']
}]

This gives the following dataframe

df = pd.DataFrame(ds)

    name    subGroups
0   groupA  [123, 456]
1   groupB  [aaa, bbb, ccc]   

I want:

    name    subGroupsFlattend
0   groupA  123
1   groupA  456
2   groupB  aaa
3   groupB  bbb
4   groupB  ccc

Any ideas?

4 Answers 4

10

Use explode:

df = df.explode('subGroups')
5

You can fix your output by following :

pd.DataFrame({'name':df.name.repeat(df.subGroups.str.len()),'subGroup':df.subGroups.sum()})
Out[364]: 
     name subGroup
0  groupA      123
0  groupA      456
1  groupB      aaa
1  groupB      bbb
1  groupB      ccc
3
  • @daiyue flatten the list
    – BENY
    Mar 30, 2018 at 14:24
  • what if the dtype of the series is not string, and couldn't use str.len, what should I use?
    – daiyue
    Mar 30, 2018 at 14:26
  • @daiyue what you mean is not string , in this question it is not string but object (list)
    – BENY
    Mar 30, 2018 at 14:36
3

You can use json_normalize:

from pandas.io.json import json_normalize

df = json_normalize(ds,  ['subGroups'], 'name').rename(columns={0:'subGroupsFlattend'})
print (df)
  subGroupsFlattend    name
0               123  groupA
1               456  groupA
2               aaa  groupB
3               bbb  groupB
4               ccc  groupB

Alternative solution with flattening dictionaries:

L = [y for x in ds for y in zip(x["subGroups"], [x["name"]] * len(x["subGroups"]))]
print (L)
[(123, 'groupA'), (456, 'groupA'), ('aaa', 'groupB'), ('bbb', 'groupB'), ('ccc', 'groupB')]

df = pd.DataFrame(L, columns=['subGroupsFlattend','name'])
print (df)
  subGroupsFlattend    name
0               123  groupA
1               456  groupA
2               aaa  groupB
3               bbb  groupB
4               ccc  groupB

EDIT:

from itertools import chain
df = pd.DataFrame(ds)

df1 = pd.DataFrame({
    'subGroups' : list(chain.from_iterable(df['subGroups'].tolist())), 
    'name' : df['name'].values.repeat(df['subGroups'].str.len())
})
print (df1)
     name subGroups
0  groupA       123
1  groupA       456
2  groupB       aaa
3  groupB       bbb
4  groupB       ccc
2
  • Thanks for this. What if I already have the DataFrame as described in question and I have no access to the DataStructure. Should I make the DataFrame into DataStructure first? Mar 23, 2018 at 15:52
  • 1
    @MoreThanFive - If already DataFrame is created, then use edited answer with flatenning lists.
    – jezrael
    Mar 23, 2018 at 15:57
0

YOBEN_S solution, but much more efficient for big dataframes.

from itertools import chain
pd.DataFrame({'name':df.name.repeat(df.subGroups.str.len()),
              'subGroup':list(chain.from_iterable(df.subGroups.to_list()))})

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.