0

I have large df with datettime index with hourly time step and precipitation values in several columns. My precipitation valuesare a cumulative total during the day (from 1:00 am to 0:00 am of the next day) and are reset after every day, example:

datetime                      S1                                                                        
2000-01-01 00:00:00          4.5  ...  
2000-01-01 01:00:00            0  ...  
2000-01-01 02:00:00            0  ...  
2000-01-01 03:00:00            0  ...  
2000-01-01 04:00:00            0
2000-01-01 05:00:00            0
2000-01-01 06:00:00            0
2000-01-01 07:00:00            0
2000-01-01 08:00:00            0
2000-01-01 09:00:00            0
2000-01-01 10:00:00            0
2000-01-01 11:00:00          6.5
2000-01-01 12:00:00          7.5
2000-01-01 13:00:00          8.7
2000-01-01 14:00:00          8.7
...
2000-01-01 22:00:00          8.7
2000-01-01 23:00:00          8.7
2000-01-02 00:00:00          8.7
2000-01-02 01:00:00            0

I am trying to go from this to the actual hourly values, so the value for 1:00 am for every day is fine and then I want to substract the value from the timestep before. Can I somehow use if statement inside of df.apply? I thought of smth like:

df_copy = df.copy()
df = df.apply(lambda x: if df.hour !=1: era5_T[x]=era5_T[x]-era5_T_copy[x-1])

But this is not working since I'm not calling a function? I could work with a for loop but that doesn't seem like the most efficient way as I'm working with a big dataset.

1 Answer 1

0

You can use numpy.where and pd.Series.shift to acheive the result

import numpy as np
df['hourly_S1'] = np.where(df.hour ==1, df.S1, df.S1-df.S1.shift())
1
  • I used your solution with a loop over all columns (there are more than S1). Works fine, thank you!
    – PDistl
    Mar 9, 2021 at 12:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.