30

While using:

with open("data_file.pickle", "rb") as pfile:
     raw_data = pickle.load(pfile)  

I get the error:

AttributeError: Can't get attribute '_unpickle_block' on <module 'pandas._libs.internals' from '/opt/conda/lib/python3.8/site-packages/pandas/_libs/internals.cpython-38-x86_64-linux-gnu.so'>

Another answer to a similar question suggests checking the version of pickle I am using. It is the same on my machine, where I developed the code and on server, where I am running the code. I have searched everywhere with no answers. Please help.

6 Answers 6

26

I don't think the problem is pickle module but Pandas version. Your file was probably created with an older version of Pandas. Now you use a newer version, pickle can't "deserialize" the object because the API change.

Try to downgrade your Pandas version and reload file. You can also try to use pd.read_pickle.

2
  • I don't know if this is the way to proceed. However, this is what I did: I read data using pandas v1.4.0 and serialized it as hdf5 file. Then downgraded pandas to pandas v1.1.5 serialized again using pickle. @Corralien was right, the issue was with pandas and not pickle. Commented Feb 12, 2022 at 10:21
  • 1
    The inverse can also be true. Creating a pickle with a later version, and trying to deserialise with an older version. For example: creating with 1.5.3 and deserialising with 1.2.4.
    – s3dev
    Commented Jul 4, 2023 at 14:51
11

In my case I had to upgrade instead of downgrade the Pandas version. Just make sure they match. Some tips for future readers:

Ask the version with:

import pandas as pd
pd.__version__

And change the version with (replace with your own version)

%pip install pandas==1.4.1
2
  • another tip is to iterate minor version, that is something like pip install pandas==1.x.0 nad changes the x one by one. Also, now pandas go pandas 2, so you might be checking pandas==2.x.0 as well. Commented May 7, 2023 at 9:30
  • Can also check pandas version with pip list | grep pandas. In my case I saved the data using 1.5.2 and could not open with pandas 1.3.0. Commented Aug 29, 2023 at 8:02
2

I got this error when using vscode and .net interactive notebooks extension. It got resolved when I updated pandas AND restarted the notebook.

0

Just change the read function from pickle.load(f) to pd.read_pickle(f) without changing anything in the versions.

# from this :
with open('Data.pkl', 'rb') as f:
     t = pickle.load(f)

# to this:
import pandas as pd 
with open('Data.pkl', 'rb') as f:
      t = pd.read_pickle(f)

This solves the problem for me.

0
-1

Faced same issue. Reading pickle in new version of pandas (the one that it WA originally saved in), saving file as parquet, then reading in older version and saving as pickle helped me.

-7

all you have to do is downgrade your sklearn version to 1.0.2

doesn't work? make sure the version you had used in your code and env as same.

that'll work.

happy coding!!

1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.