2

I have the following dataframes:

df1: dataframe with patient critical notes

AREA                      DATE_TIME                 CRITICAL ISSUE NOTES
0013                      11/6/2017 2:25:00 P.M     Nurse attended to the patient 
1121                      10/23/2017 6:43:00 A.M    Completed an ER
1121                      10/2/2017 9:30:00 P.M     Admitted 

df2: Patient other details

ZIP                TIME_NOTED   NAME    OCCUPIED    STATE
4568    10/1/2017 10:04:00 A.M  Chris          Y    NORMAL
1121    10/23/2017 6:43:00 A.M  Nancy          Y    CRITICAL
1121    10/2/2017 9:30:00 P.M   Derek          N    CRITICAL

I have to map the records in df2 using DATE_TIME and AREA code from df1 and also retain all other columns in both dataframes. I tried merging on multiple columns but didnt work as expected.

new_df = pd.merge(df1, df2,  how='right', left_on=['Date_Time','AREA'], right_on = ['ZIP','TIME_NOTED'])
14
  • 2
    What is the error you get when you try the merge. And what is ENS?
    – kdheepak
    Jan 19, 2018 at 16:43
  • 1
    Looks like right_on should be ['TIME_NOTED', 'ZIP'] to stand a chance of matching the left_on... Jan 19, 2018 at 16:51
  • What @JonClements is saying is that you need to check to order of your right_on and left_on list. Either put the date column first in both list or second in both list. Currently, right_on doesn't match left_on and the merge will not work. Jan 19, 2018 at 17:01
  • @ScottBoston Tried changing the order. It shows empty values for all columns in df1 after the merge
    – Hackerds
    Jan 19, 2018 at 17:19
  • Please add the outputs of df1.to_dict() and df2.to_dict() in your question. Jan 19, 2018 at 17:20

1 Answer 1

1

If you put the columns in the same order for both left/right_on (area/zip then date time/time noted) it should work. I also changed the merge to an inner, so you just get records with the same zip/area and date time/time noted.

new_df = pd.merge(df1, df2,  how='inner', left_on = ['AREA','DATE_TIME'], right_on = ['ZIP','TIME_NOTED'])

Another potential solution would be creating an "ID" column and merging on that.

df1['ID'] = df1['AREA'].astype(str) + '_' + df1['DATE_TIME'].astype(str)
df2['ID'] = df2['ZIP'].astype(str) + '_' + df2['TIME_NOTED'].astype(str)

Now merge on the IDs

new_df = pd.merge(df1, df2, how = 'inner',left_on = ['ID'], right_on = ['ID'])

This should yield the same table (with the addition of an ID column).

4
  • how = 'inner' resulted in Zero records in the result. And for some reason the after re ordering the columns, it still doesnt work with how ='right'. Even after all this discussion, I'm just curious how simple the fix would be.
    – Hackerds
    Jan 19, 2018 at 19:25
  • How are you creating the dataframes? Because the merge worked properly for me after copying your df examples above and using pd.read_clipboard(sep='\s\s+') to recreate the dataframes.
    – Brian
    Jan 19, 2018 at 19:29
  • Reading from a csv file. I have checked for the datatypes of all the columns on which we need to merge and they are equal. int and datetime64[ns]
    – Hackerds
    Jan 19, 2018 at 20:26
  • Strange, it must be a dtype problem. The dates do not appear to be in a typical date time format. Have you tried the ID work around? Or possibly trying to do an .astype() to the columns individually to specify the desired dtype explicitly.
    – Brian
    Jan 19, 2018 at 21:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.