1

How can insert rows in a pandas dataframe? I have a large dataframe and I am trying to identify specific values to repeat rows and inserting in the dataframe. For example:

df1 = pd.DataFrame([[1, 2], [3, 4],[1, 6],[2,3],[1,5]], columns=["a", "b"])
   a  b
0  1  2
1  3  4
2  1  6
3  2  3
4  1  5

Column "a" represent the number of row repetitions to inset in the dataframe, So I would like to get the following

   a  b
0  1  2
1  1  4
2  1  4
3  1  4
4  1  6
5  1  3
6  1  3
7  1  5

I tried to use append but the results is not what I expected. Here is what I have so far. I'll appreciate any insight.

df2 = df1[df1.a > 1]               # To select rows with values more than 1
repeats = (df2.iloc[0]["a"] - 1)   # number of repetitions -1
r2 = pd.concat([df2]*repeats, ignore_index=True)
df_modified = df1.append(r2, ignore_index=True)

3 Answers 3

1

Using reindex with repeat

df1.reindex(df1.index.repeat(df1.a)).assign(a=1).reset_index(drop=True)
Out[1266]: 
   a  b
0  1  2
1  1  4
2  1  4
3  1  4
4  1  6
5  1  3
6  1  3
7  1  5
4
  • Thank you, for your help. But what If I have some empty or NaN values in column "a" ?. I guess I'll get an error: 'Cannot cast array data from dtype('float64') to dtype('int64') according to the rule 'safe'. Any thoughts?
    – Raul
    Commented Sep 2, 2018 at 19:58
  • @Raul you can add two more steps , like df1=df.loc[df.a=='',], df2=df.loc[df.a!='',], Then do df2 for df1 , and concat the result of df2 , and df1
    – BENY
    Commented Sep 2, 2018 at 20:01
  • Thanks, Wen, but still the same error, also, I tried to clean it first using: df2=df1.dropna(subset = ['a']) and then use your code but it is not working. I'll appreciate any insights
    – Raul
    Commented Sep 2, 2018 at 20:59
  • @Raul that may not a nan try df2=df1.loc[df1.a!=‘ ‘].astype(int)
    – BENY
    Commented Sep 2, 2018 at 21:41
1

You can use numpy.repeat:

import numpy as np

res = pd.DataFrame({'a': 1, 'b': np.repeat(df1['b'].values, df1['a'].values)})

print(res)

   a  b
0  1  2
1  1  4
2  1  4
3  1  4
4  1  6
5  1  3
6  1  3
7  1  5
2
  • 1
    You may wish to reset your index to match the OP's target result.
    – Alexander
    Commented Aug 31, 2018 at 23:11
  • @Alexander, Good point. Or more efficiently (as per my update), just use NumPy all the way :)
    – jpp
    Commented Aug 31, 2018 at 23:12
1

You could use a nested list comprehension:

df2 = pd.DataFrame({
    'a': 1, 
    'b': [b for a, b in df1[['a', 'b']].values for _ in range(a)]})

>>> df2
   a  b
0  1  2
1  1  4
2  1  4
3  1  4
4  1  6
5  1  3
6  1  3
7  1  5

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.