0

I've got two dataframes. First one is empty but with columns defined:

Empty DataFrame
Columns: [ID, 3120, 3121, 3122, 3123, 3124, 3125, 3126, 3127, 3128, 3129, 3130, 3131, 3146, 3147, 3148, 3149, 3150, 3151, 3152, 3153, 3154, 3155, 3156, 3157]
Index: []

Second dataframe is:

    3123    3124    3125    3126    3127
0   A       B       C       D       

Later, I will have another dataframe that will be:

    3146    3147    3148    3149    3150
0   X       Y       Z           

And so on. What I want is to put all this little dataframes in the first one to get something like:

ID  3120    3121    3122    3123    3124    3125    3126    3127    3128    3129    3130    3131    3146    3147    3148    3149    3150    3151    3152    3153    3154    3155    3156    3157
1                           A       B       C       D                                               X       Y       Z

So what I am doing in my loop is:

df_main.merge(df_i, how='inner', on=df_i.columns)

Where, when i=1:

df_main.columns:

Index(['ID', '3120', '3121', '3122', '3123', '3124', '3125', '3126',
       '3127', '3128', '3129', '3130', '3131', '3146', '3147', '3148', '3149',
       '3150', '3151', '3152', '3153', '3154', '3155', '3156', '3157'],
      dtype='object')


df_i.columns:

Index(['3123', '3124', '3125', '3126', '3127'], dtype='object')

And code is raising this KeyError:

    raise KeyError(key)
KeyError: Index(['3123', '3124', '3125', '3126', '3127'], dtype='object')

How is this possible? df_i.columns is contained and exist in df_main.columns

Thank you in advance!

3
  • I believe this error means that pandas is trying to use the entire list ['3123', '3124', '3125', '3126', '3127'] as one column name.
    – IanS
    Jun 17, 2019 at 8:02
  • Try this: df_main.merge(df_i, how='inner', on=df_i.columns.tolist()). There are a few cases where columns cannot be used directly as an iterable, this could be one of them.
    – IanS
    Jun 17, 2019 at 8:10
  • Hi, yes, that way works partially... I dont get the KeyError but an Empty Dataframe as result :/ Maybe merge is not what I am looking for? :/ Thanks
    – Solar
    Jun 17, 2019 at 8:16

1 Answer 1

1

okay one way to do this

df1

  3123 3124 3125 3126  3127
0    A    B    C    D   NaN

df2

  3146 3147 3148  3149  3150
0    X    Y    Z   NaN   NaN

using pd.concat

df = pd.concat([df.drop(df1.columns.append(df2.columns),axis=1),df2,df3], sort=True, axis=1)
df = df[['ID', 3120, 3121, 3122, 3123, 3124, 3125, 3126, 3127, 3128, 3129, 3130, 3131, 3146, 3147, 3148, 3149, 3150, 3151, 3152, 3153, 3154, 3155, 3156, 3157]] # for reordering
df.fillna('', inplace=True)

Output

    ID 3120 3121 3122 3123 3124 3125 3126 3127 3128 ...  3148 3149 3150 3151  \
0                      A    B    C    D           ...     Z                  

  3152 3153 3154 3155 3156 3157  
0                                

[1 rows x 25 columns]
1
  • this solution is almost fine but is not keeping the df columns order :/ thank you!
    – Solar
    Jun 17, 2019 at 8:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.