Questions tagged [pickle]
Pickle is an object serialization module for Python. Use this tag together with the Python tag for questions related to storing or loading objects with Pickle.
4,948
questions
0
votes
0
answers
8
views
torch.load() with error _pickle.UnpicklingError: could not find MARK
in my code,the sentence
state = torch.load(best_model_file)
reports the following error
File "/home/usr/anaconda3/envs/env1/lib/python3.7/site-packages/torch/serialization.py", line 367, in ...
0
votes
0
answers
15
views
I am trying to setup pickle for the first time ever and keep getting errors, or the program does not save/load the variables
I keep getting the error (load() missing 1 required positional argument: 'player') when I try to load.
I have kinda given up on this project for a bit until I can fix this, so I finally got back to it ...
0
votes
0
answers
40
views
Django API attribute not found
I am building an API using django and the django rest framework that uses models from tensorflow and scikit-learn. I have ran into an error where my CountVectorizer that I fitted and used in another ...
0
votes
0
answers
46
views
Python multiprocessing and M1 - can't pickle function defined at the top level
In the toy code below, I am trying to apply the multiprocessing map function to a list, but I'm getting the following error:
_pickle.PicklingError: Can't pickle <function add_function at ...
1
vote
2
answers
37
views
Cannot unpickle an instance of a class which inherits from `time`
Consider the following class, which inherits from datetime.time:
class TimeWithAddSub(time):
_date_base = date(2000, 1, 1)
def __new__(cls, hour=0, minute=0, second=0, microsecond=0, tzinfo=...
0
votes
0
answers
17
views
Machine Learning integration with FLASK
I have a model trained and saved in a folder. I want to load it to integrate its result within a Flask web application.
However, I have the error :
Traceback (most recent call last):
File "C:\...
0
votes
1
answer
36
views
Issue with Pickling in Top2Vec v1.0.34
I recently upgraded to Top2Vec version 1.0.34 and encountered an issue with pickling the model.
Specifically, I'm getting following error:
--------------------------------------------------------------...
-1
votes
0
answers
31
views
Couldn't unpickle a Python object when making predictions in a gunicorn application
I tried the solution from this StackOverflow link: Unable to unpickle an object while predicting using gunicorn
However, I ran into some problems getting it to run. When attempting to start Gunicorn, ...
0
votes
1
answer
21
views
pickle fails to load data: TypeError: _unpickle_timestamp() takes exactly 3 positional arguments (4 given)
I have a dataset, stored in a pandas dataframe, that I have built and pickled in one environment. The pickle file loads fine in that environment (environment 1). However, when I try to load it in a ...
0
votes
0
answers
15
views
ModuleError with pyranges when reading .pkl file in Jupyter Notebook
I need to read a .pkl file in JupyterLab. The code I have is:
import dill
# Loading scplus_obj
scplus_obj = dill.load(open('/path-to-my-file/scplus_obj.pkl', 'rb'))
The error get is :
--------------...
-3
votes
0
answers
18
views
EOFError Pickle
I am making a password encrypting / storing program that use pickle to store to names and passwords.
Whenever I run the code it give me the following error:
EOFError: Ran out of input
I know this was ...
-1
votes
2
answers
89
views
I keep getting "ValueError: Length of values (1) does not match length of index (11)" even though I am following an example problem
I am attempting to couple a pymoo optimization algorithm with a regression model. I found this fantastic example on github, and am attempting to follow the steps starting from In[16].
The example ...
0
votes
0
answers
34
views
Tkinter - export attachment
So I have created a user form with Tkinter that has a button that allows you to attach the folder path of a file. The problem is I want to press a button to get the folder path instead of opening the ...
0
votes
0
answers
42
views
Can't read in pickeld dataframe with either pd.read_pickle or pickle.load
I’m trying to read in a dataframe I pickled. It worked well untill last week, but today I got an error that says ‘ValueError: unsupported pickle protocol: 5’.
I found this page unsupported-pickle-...
1
vote
1
answer
57
views
How to write to Python dictionary without loading the dictionary into memory?
I have a large table that I want to convert to a Python dictionary but I don't want to load all of the data into memory.
Is it possible to actively write to a pickle dump without building the object ...
0
votes
1
answer
40
views
AttributeError: Can't pickle local object. How to fix this issue?
I am trying to save a BERT model and am reaching the following error. I already tried using model.save(), but that gave me another error so I am trying to find a workaround.
Here is the code I used to ...
0
votes
0
answers
10
views
Store queryset in local storage
I want to store a queryset result from a form submission to restore its state once it is accesed again. I have tried using,localStorage,cache but both of them are giving problems that instead of ...
0
votes
0
answers
16
views
understanding python stacktrace in mailman3 log
I just upgraded a postorius/mailman3 instance.
Now the webfrontend shows HTTP Error 500: {"title": "500 Internal Server Error"} and the mailman3 log contains this message:
File &...
0
votes
1
answer
71
views
Transfer format for Durable functions
I am trying to implement data analysis by function using Durable Functions.
When implementing by function, I need to transfer data frames, etc., so I am trying to exchange data by making it pickle. ...
0
votes
1
answer
23
views
How do you register dill in copyreg to pickle a non pickable module?
I have a setup where a python package I'm using is trying to pickle my code. A reduced example looks like
constraint.py
manifest = []
constraint_loader.py
import importlib
class constraint_loader:
...
0
votes
1
answer
116
views
Python type hint pickable argument
I want to communicate to users of my code that some function requires a pickable object. Here is an example that abstracts from any project details:
import pickle
from typing import Callable
def ...
0
votes
0
answers
14
views
Python joblib with pickle error: Can't pickle <class '__main__.Foo'>: it's not found as __main__.Foo
I have some code where a function returns a class Foo, which i want to save to a file. This function takes quite a while, so I wanted to speed it up with joblib. However, I now get the following error:...
0
votes
1
answer
35
views
airflow DAG run: unpickling with dill gives NameError: name 'timedelta' is not defined
I have a simple DAG (with render_template_as_native_obj option enabled). I'm trying to pass a {{ dag_run }} to a PythonVirtualenvOperator, but when it starts to execute it fails with NameError: name '...
0
votes
0
answers
37
views
Resumable AES GCM in Python 3?
Is there an implementation of AES GCM for Python 3 which not only accepts piecewise messages, but also allows the working state of the cipher to be serialized, stored, and later resumed?
(I have an ...
1
vote
1
answer
44
views
Different code behavior using VSC Debugger (out of bounds on iterable)
I have an iterable with appropriate len and getitem methods implemented.
I pickle this object to a "train_file.pth" pickle file.
Then, when running the following code snippet :
with open(...
0
votes
0
answers
25
views
pickle gives me an EOFError: Ran out of input error, and the file is not empty
this is the code that im working on. it basically stores the user ip address in a pickle file after encoding it and then tries the decode it. pretty simple. what i tried to find out (from the many ...
-1
votes
0
answers
20
views
PicklingError Can't pickle <function Mixture._expectation_list at 0x000Ef>: it's not the same object as sidekit.mixture.Mixture._expectation_list
I am trying to train Universal Background Model(ubm) using EM split of sidekit. But when I run
ubm.EM_split(
features_server=server, #sidekit.FeaturesServer used to load data
...
0
votes
0
answers
32
views
Performance of very large dictionaries when storing/loading via pickle
I have a very large dictionary (multiple millions) which will also be accesses multiple millions times.
I was wondering about the low-level performances of such dicts because if they need to be ...
0
votes
1
answer
32
views
How is a multiprocessing.Event passed to a child process under the hood?
According to this SO answer, when a multiprocessing.Queue is passed to a child process, what is actually sent is a file descriptor (or handle) obtained from pipe, instead of pickling the Queue object ...
0
votes
0
answers
63
views
pikle (pkl) file for AI image generator
I'm trying to generate pictures form (StyleGAN2-ADA-PyTorch) (https://colab.research.google.com/github/dvschultz/stylegan2-ada-pytorch/blob/main/SG2_ADA_PyTorch.ipynb#scrollTo=jG7ZEc_982io).
I'm stuck ...
0
votes
0
answers
14
views
Can't find attribute of type `MyData` in serialized file even though it requires `MyData`
I'm struggling with fixing a serialized pickle file right now in py2 (can't use py3).
I have a file called data.pickle that I'm trying to load with:
import cPickle as CP
with open('data.pickle', 'rb')...
4
votes
1
answer
140
views
PythonVirtualenvOperator gives error: No module named unusual_prefix_***_dag
I'm using airflow 2.5.3 with Kubernetes executor and Python 3.7.
I've tried to make a simple DAG with only one PythonVirtualnvOperator and two context variables ({{ ts }} and {{ dag }}) passed into it....
1
vote
0
answers
36
views
How to locate object that cannot be pickled inside complex object
Related to my earlier question: How to I change import statements in a pickle file?
I am trying to change an import statement inside a serialized file. It ended up causing some unexpected issues when ...
1
vote
1
answer
40
views
How to I change import statements in a pickle file?
I recently removed a module xyz and replace it with another one of a different name abc. In some pickle files, it still imports the xyz module. I want to change it to import the abc module. How can I ...
0
votes
0
answers
20
views
creating a function to return average daily pm2.5 values from data stored in a dictionary with timestamps and pm2.5 values
Disclaimer I just started learning python, so I have very little clue what I am doing. This is my code so far, my average day function is overly complicated and does not work. It should return the ...
0
votes
0
answers
29
views
_pickle.PicklingError: Can't pickle <function _lambdifygenerated at 0x0000026D008DFF70>: attribute lookup _lambdifygenerated on __main__ failed
For some reason I can't save my newly created object with pickle.
This is my Python code:
import pickle
from distutils.log import warn
from ikpy.chain import Chain
import numpy as np
import casadi as ...
0
votes
0
answers
13
views
Use DataLoader Tensorflow for a large pickle size
I have pickle 2 pickle for trainSet and testSet such as data.plk , lables.plk
for example for train data pickle have size 2GB.
I want to use this pickle file for fit model and get error OOM (out of ...
0
votes
0
answers
23
views
Celery is refusing to deserialize content of type pickle
Recently I upgraded packages celery, amqp, kombu and redis of our Django project to their later versions but for some reason I had to downgrade again. The problem is that after downgrading, celery app ...
0
votes
0
answers
47
views
python check if type is serializable
Trying to use a distributed cluster computing framework, i want to pre-validate a graph before running it for non serializable types.
Presume i have the following class:
class Human:
def __init__(...
-1
votes
1
answer
73
views
Trying to unpickle LogisticRegression estimator from version 1.3.0 when using version 0.24.2. What does this warning mean and how do I fix it?
I'm trying to deploy a LogisticRegression model on an active cloud shell that was trained using scikit-learn version 1.3.0, but I'm getting the following warning message:
/usr/local/lib/python3.9/site-...
2
votes
3
answers
62
views
Do numpy.savez and numpy.savez_compressed use pickle?
I recently encountered numpy.savez and numpy.savez_compressed. Both seem to work well with arrays of differing types, including object arrays. However, numpy.load does not work well with object type ...
0
votes
2
answers
52
views
How to load pickle files independent from any current implementation?
Remade Post:
Let's say I have a pickle file with an instance of a custom class inside. However, I do not have the source code of that class anymore (or only a modified one, which will not match).
...
0
votes
0
answers
20
views
ModuleNotFoundError: No module named 'sklearn.linear_model.base' while running util.py file
I encountered a ModuleNotFoundError while running the server. The error occurs when the script attempts to load the model using pickle. It specifically states that no module is named 'sklearn....
1
vote
0
answers
34
views
Save data from cpp into bytes that can be pickle loaded by Python and seen as numpy ndarray
PROBLEM
Let's say I have a python script that works with a numpy ndarray and then pickle.dumps it into bytes. Another python script would then pickle.loads those bytes and see the object as a numpy ...
0
votes
0
answers
20
views
Pickle - Trying to read from a text file - UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 37: character maps to <undefined>
with open ("Save.txt" , "r+") as f:
f.seek(0)
if not f.read():
print("No Data")
while True:
starter = int(input('...
4
votes
1
answer
94
views
how to read corrupted pickle file
I have a binary file written using pickle.dump containing logs from an app (a set of tuples of floats and strings), it worked great for 24h but now when trying to read it using
import pickle
path = &...
0
votes
1
answer
88
views
Error of saving json during converting openvino model to tensorflow with parameter --output_weight_and_json
I want convert my openvino-model to tf saved_model according to the documentation of https://github.com/PINTO0309/openvino2tensorflow repository.
But when I use command: openvino2tensorflow --...
0
votes
2
answers
81
views
Understanding the pickled object size
I execute the following line in Python interpreter:
len(pickle.dumps([numpy.random.random(384).tolist()]*55))
It gives 3584.
Given that numpy.random.random produces 8-byte floats, the data being ...
0
votes
0
answers
6
views
EOF about spec = torch.load(spec_filename)
I have the same problem as follows:
spec = torch.load(spec_filename)
magic_number = pickle_module.load(f, **pickle_load_args)
return _legacy_load(opened_file, map_location, pickle_module, **...
1
vote
1
answer
19
views
Poor performance of unpickling files with uWSGI
I have to unpickle around 200 files, each containing a custom class containing a numpu.ndarray (shape 500x32) and some metadata (few int and str).
Each file's size is around 50ko.
Here is the ...