3

I have a list where the number of items between a 'SUMMARY' element and the next one is not fixed

list = ['SUMMARY - Dec 2013', 'Person1', 'None', 'None', '10', 'SUMMARY - Dec 2013', 'Person2', '20', 'SUMMARY - Jan 2014', 'Person3', 'None']

What I'm trying to achieve is to transform the list to a dataframe where each row starts with a 'SUMMARY' element.

I've used

match = []
match.append([n for n, l in enumerate(list) if l.startswith('SUMMARY')])

with output [[0, 5, 8]] to get the indexes of the items that contain 'SUMMARY', and I would like each row of my dataframe to start with the corresponding items whose indexes are included in match. In this case, match has 3 elements, so I would like my dataframe to have 3 rows and the following structure:

'SUMMARY - Dec 2013', 'Person1', 'None', 'None', '10'
'SUMMARY - Dec 2013', 'Person2', '20',   NA,    NA
'SUMMARY - Jan 2014', 'Person3', 'None', NA,    NA

Basically, when the number of items for a specific row is smaller than the number of max columns, the rest gets filled with NA/NaN.

0

1 Answer 1

3

Thanks for the sample data. It is easier to approach this if you start with a single Series. You can group rows of data based on whether "Summary" is present and then use this to aggregate and re-explode your data across columns.

s = pd.Series(your_list)
pd.DataFrame(s.groupby(s.str.contains('summary', case=False).cumsum())
              .agg(list)
              .tolist())                                                   

                    0        1     2     3     4
0  SUMMARY - Dec 2013  Person1  None  None    10
1  SUMMARY - Dec 2013  Person2    20  None  None
2  SUMMARY - Jan 2014  Person3  None  None  None

Thanks @Shubham Sharma for the suggestion of iterating over groups:

pd.DataFrame([g.tolist() for k, g in s.groupby(
    s.str.contains('summary', case=False).cumsum())])

                    0        1     2     3     4
0  SUMMARY - Dec 2013  Person1  None  None    10
1  SUMMARY - Dec 2013  Person2    20  None  None
2  SUMMARY - Jan 2014  Person3  None  None  None
2
  • 1
    pd.DataFrame([grp.tolist() for k, grp in s.groupby(s.str.contains('summary', case=False).cumsum()) might be faster, what do you think? Jul 14, 2020 at 10:58
  • @ShubhamSharma that would work as well, why not post an answer?
    – cs95
    Jul 14, 2020 at 10:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.