0

I've been working to_csv()/read_csv() to read/write a data frame a user works with in an applet, where one of the columns is a datetime.datetime object, and it seems like to_csv automatically converts the datetimes to strings. Is this correct? If so, is there a way to "preserve" the dates as datetime rather than them being converted to strings? I've read through the documentation, and I can't seem to find the answer. Thank you.

3
  • Can you explain why you are trying to do this? What information is being lost when it's converted to a string? What would you prefer it be written as? Apr 10, 2018 at 3:42
  • So, I'd like to add an option to graph date vs. value for any read .csv file, and according to my professor you can only do that with datetimes, not strings. Is this not true?
    – Axioms
    Apr 10, 2018 at 3:45
  • The parse_dates argument in read_csv should allow you to read dates back into python objects. Or you can do as the answer suggests if you are only using the csv for serialization. Apr 10, 2018 at 3:54

1 Answer 1

2

To preserve the exact structure of a DataFrame, complete with data types, check out the pickle module, which "serializes" any python object to disk and reloads it back into a python environment.

Use pd.to_pickle instead of pd.to_csv, optionally with a compression argument (see docs):

# Save to pickle
df.to_pickle('pickle-file.pkl')
# Pickle with compression
df.to_pickle('pickle-file.pkl.gz', compression='gzip')

# Load pickle from disk
df = pd.read_pickle('pickle-file.pkl')
# or...
df = pd.read_pickle('pickle-file.pkl.gz', compression='gzip')
2

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.