9

I'm trying to convert an xarray data array to a pandas dataframe for a machine learning project, but the time data appears to be in a cftime.DatetimeJulian format, which is not convertible using the pandas to_datetime() method. Suggestions? Thanks.

nor_xr.time

<xarray.DataArray 'time' (time: 1372)>
array([cftime.DatetimeJulian(2015, 3, 31, 0, 0, 0, 0, 0, 90),
       cftime.DatetimeJulian(2018, 12, 31, 0, 0, 0, 0, 6, 365)], dtype=object)
Coordinates:
  * time     (time) object 2015-03-31 00:00:00 ... 2018-12-31 00:00:00
Attributes:
    standard_name:  time
    axis:           T

nor_df = nor_xr.to_dataframe().reset_index()
nor_df.head()

    time
0   2015-03-31 00:00:00
1   2015-04-01 00:00:00

pd.to_datetime(nor_df.time)

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-104-1f0fc00ad825> in <module>
      2 
      3 #|nor_df.time.unique()
----> 4 pd.to_datetime(nor_df.time)

~\AppData\Local\Continuum\anaconda3A\lib\site-packages\pandas\core\tools\datetimes.py in to_datetime(arg, errors, dayfirst, yearfirst, utc, box, format, exact, unit, infer_datetime_format, origin, cache)
    449         else:
    450             from pandas import Series
--> 451             values = _convert_listlike(arg._values, True, format)
    452             result = Series(values, index=arg.index, name=arg.name)
    453     elif isinstance(arg, (ABCDataFrame, MutableMapping)):

~\AppData\Local\Continuum\anaconda3A\lib\site-packages\pandas\core\tools\datetimes.py in _convert_listlike(arg, box, format, name, tz)
    366                     dayfirst=dayfirst,
    367                     yearfirst=yearfirst,
--> 368                     require_iso8601=require_iso8601
    369                 )
    370 

pandas\_libs\tslib.pyx in pandas._libs.tslib.array_to_datetime()

pandas\_libs\tslib.pyx in pandas._libs.tslib.array_to_datetime()

pandas\_libs\tslib.pyx in pandas._libs.tslib.array_to_datetime()

TypeError: <class 'cftime._cftime.DatetimeJulian'> is not convertible to datetime

3 Answers 3

17

Indeed this is a common request. The recommended way of doing it is to pull out the index (a CFTimeIndex) and use its built-in to_datetimeindex method:

datetimeindex = nor_xr.indexes['time'].to_datetimeindex()

You can then replace the CFTimeIndex with its DatetimeIndex counterpart:

nor_xr['time'] = datetimeindex
1
  • Confirming that this solution works for a similar issue with cftime.DatetimeNoLeap as well. Aug 13, 2020 at 16:08
1

I found a simple way to convert from cftime.DatetimeNoLeap to datetime object. Say for eg datetimes is a list of cftime.DatetimeNoLeap objects with datetimes[0] = cftime.DatetimeNoLeap(1970, 1, 1, 12, 0, 0, 0, has_year_zero=True)

datetime.strptime(str(datetimes[0]),'%Y-%m-%d %H:%M:%S')

This will give you datetime.datetime(1970, 1, 1, 12, 0).

0

A simple and clean solution using iso format

from datetime import datetime

datetime(cfdatetime.isoformat())

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.