6

I have a pandas DataFrame with a colum containing string timestamps in the form:

31 Jan 2020 17:29:37 CET

09 Apr 2021 15:34:53 CEST

Converting the column to timestamps returns a warning:

df['timestamp'] = pd.to_datetime(df["timestamp"])
c:\Users\user\Miniconda3\lib\site-packages\dateutil\parser\_parser.py:1213: UnknownTimezoneWarning: tzname CET identified but not understood.  Pass `tzinfos` argument in order to correctly return a timezone-aware datetime.  In a future version, this will raise an exception.
  warnings.warn("tzname {tzname} identified but not understood.  "
c:\Users\user\Miniconda3\lib\site-packages\dateutil\parser\_parser.py:1213: UnknownTimezoneWarning: tzname CEST identified but not understood.  Pass `tzinfos` argument in order to correctly return a timezone-aware datetime.  In a future version, this will raise an exception.
  warnings.warn("tzname {tzname} identified but not understood.  "

I've seen lots of discussions about this warning, but couldn't find a solution in the context of pandas' to_datetime() method. Can anyone help? It's critical that the timestamps are normaized accurately as I then want to sort my dataframe with this column.

Thanks!

0

1 Answer 1

6

Probably most efficient would be to just strip the time zone abbreviation, parse to datetime and localize to the correct time zone:

import pandas as pd
df = pd.DataFrame({'datetime': ["31 Jan 2020 17:29:37 CET",
                                "09 Apr 2021 15:34:53 CEST"]})

df['datetime'] = df['datetime'].str.replace('CET|CEST', '', regex=True)
df['datetime'] = pd.to_datetime(df['datetime']).dt.tz_localize('Europe/Berlin')

df['datetime']
0   2020-01-31 17:29:37+01:00
1   2021-04-09 15:34:53+02:00
Name: datetime, dtype: datetime64[ns, Europe/Berlin]

Or you can make use of dateutil's parser here as well, but only using an apply. This is especially useful if you have data from multiple time zones (not only e.g. CET/CEST):

import pandas as pd
import dateutil

df = pd.DataFrame({'datetime': ["31 Jan 2020 17:29:37 CET",
                                "09 Apr 2021 15:34:53 CEST"]})

# define a 'real' time zone for each abbreviation:
tzmapping = {'CET': dateutil.tz.gettz('Europe/Berlin'),
             'CEST': dateutil.tz.gettz('Europe/Berlin')}

df['datetime'] = df['datetime'].apply(dateutil.parser.parse, tzinfos=tzmapping)

df['datetime']
0   2020-01-31 17:29:37+01:00
1   2021-04-09 15:34:53+02:00
Name: datetime, dtype: datetime64[ns, tzfile('/usr/share/zoneinfo/Europe/Berlin')]
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.