Questions tagged [pickle]

Pickle is an object serialization module for Python. Use this tag together with the Python tag for questions related to storing or loading objects with Pickle.

Filter by
Sorted by
Tagged with
0 votes
0 answers
8 views

torch.load() with error _pickle.UnpicklingError: could not find MARK

in my code,the sentence state = torch.load(best_model_file) reports the following error File "/home/usr/anaconda3/envs/env1/lib/python3.7/site-packages/torch/serialization.py", line 367, in ...
HJLQZ's user avatar
  • 1
0 votes
0 answers
15 views

I am trying to setup pickle for the first time ever and keep getting errors, or the program does not save/load the variables

I keep getting the error (load() missing 1 required positional argument: 'player') when I try to load. I have kinda given up on this project for a bit until I can fix this, so I finally got back to it ...
Ben's user avatar
  • 13
0 votes
0 answers
40 views

Django API attribute not found

I am building an API using django and the django rest framework that uses models from tensorflow and scikit-learn. I have ran into an error where my CountVectorizer that I fitted and used in another ...
vmmgame's user avatar
0 votes
0 answers
46 views

Python multiprocessing and M1 - can't pickle function defined at the top level

In the toy code below, I am trying to apply the multiprocessing map function to a list, but I'm getting the following error: _pickle.PicklingError: Can't pickle <function add_function at ...
jvitalis's user avatar
1 vote
2 answers
37 views

Cannot unpickle an instance of a class which inherits from `time`

Consider the following class, which inherits from datetime.time: class TimeWithAddSub(time): _date_base = date(2000, 1, 1) def __new__(cls, hour=0, minute=0, second=0, microsecond=0, tzinfo=...
bavaza's user avatar
  • 10.6k
0 votes
0 answers
17 views

Machine Learning integration with FLASK

I have a model trained and saved in a folder. I want to load it to integrate its result within a Flask web application. However, I have the error : Traceback (most recent call last): File "C:\...
TheArcturus's user avatar
0 votes
1 answer
36 views

Issue with Pickling in Top2Vec v1.0.34

I recently upgraded to Top2Vec version 1.0.34 and encountered an issue with pickling the model. Specifically, I'm getting following error: --------------------------------------------------------------...
Davood's user avatar
  • 395
-1 votes
0 answers
31 views

Couldn't unpickle a Python object when making predictions in a gunicorn application

I tried the solution from this StackOverflow link: Unable to unpickle an object while predicting using gunicorn However, I ran into some problems getting it to run. When attempting to start Gunicorn, ...
Hồ Đạt Tấn's user avatar
0 votes
1 answer
21 views

pickle fails to load data: TypeError: _unpickle_timestamp() takes exactly 3 positional arguments (4 given)

I have a dataset, stored in a pandas dataframe, that I have built and pickled in one environment. The pickle file loads fine in that environment (environment 1). However, when I try to load it in a ...
Jeremy Matt's user avatar
0 votes
0 answers
15 views

ModuleError with pyranges when reading .pkl file in Jupyter Notebook

I need to read a .pkl file in JupyterLab. The code I have is: import dill # Loading scplus_obj scplus_obj = dill.load(open('/path-to-my-file/scplus_obj.pkl', 'rb')) The error get is : --------------...
Arnoneel Sinha's user avatar
-3 votes
0 answers
18 views

EOFError Pickle

I am making a password encrypting / storing program that use pickle to store to names and passwords. Whenever I run the code it give me the following error: EOFError: Ran out of input I know this was ...
basinj's user avatar
  • 1
-1 votes
2 answers
89 views

I keep getting "ValueError: Length of values (1) does not match length of index (11)" even though I am following an example problem

I am attempting to couple a pymoo optimization algorithm with a regression model. I found this fantastic example on github, and am attempting to follow the steps starting from In[16]. The example ...
ali bab613's user avatar
0 votes
0 answers
34 views

Tkinter - export attachment

So I have created a user form with Tkinter that has a button that allows you to attach the folder path of a file. The problem is I want to press a button to get the folder path instead of opening the ...
Billy's user avatar
  • 9
0 votes
0 answers
42 views

Can't read in pickeld dataframe with either pd.read_pickle or pickle.load

I’m trying to read in a dataframe I pickled. It worked well untill last week, but today I got an error that says ‘ValueError: unsupported pickle protocol: 5’. I found this page unsupported-pickle-...
Cici Xu's user avatar
1 vote
1 answer
57 views

How to write to Python dictionary without loading the dictionary into memory?

I have a large table that I want to convert to a Python dictionary but I don't want to load all of the data into memory. Is it possible to actively write to a pickle dump without building the object ...
O.rka's user avatar
  • 30.2k
0 votes
1 answer
40 views

AttributeError: Can't pickle local object. How to fix this issue?

I am trying to save a BERT model and am reaching the following error. I already tried using model.save(), but that gave me another error so I am trying to find a workaround. Here is the code I used to ...
Sid R.'s user avatar
  • 19
0 votes
0 answers
10 views

Store queryset in local storage

I want to store a queryset result from a form submission to restore its state once it is accesed again. I have tried using,localStorage,cache but both of them are giving problems that instead of ...
Varad Shinde's user avatar
0 votes
0 answers
16 views

understanding python stacktrace in mailman3 log

I just upgraded a postorius/mailman3 instance. Now the webfrontend shows HTTP Error 500: {"title": "500 Internal Server Error"} and the mailman3 log contains this message: File &...
greyjedi's user avatar
0 votes
1 answer
71 views

Transfer format for Durable functions

I am trying to implement data analysis by function using Durable Functions. When implementing by function, I need to transfer data frames, etc., so I am trying to exchange data by making it pickle. ...
TY00's user avatar
  • 33
0 votes
1 answer
23 views

How do you register dill in copyreg to pickle a non pickable module?

I have a setup where a python package I'm using is trying to pickle my code. A reduced example looks like constraint.py manifest = [] constraint_loader.py import importlib class constraint_loader: ...
user1139069's user avatar
  • 1,515
0 votes
1 answer
116 views

Python type hint pickable argument

I want to communicate to users of my code that some function requires a pickable object. Here is an example that abstracts from any project details: import pickle from typing import Callable def ...
Izaak Cornelis's user avatar
0 votes
0 answers
14 views

Python joblib with pickle error: Can't pickle <class '__main__.Foo'>: it's not found as __main__.Foo

I have some code where a function returns a class Foo, which i want to save to a file. This function takes quite a while, so I wanted to speed it up with joblib. However, I now get the following error:...
Bob's user avatar
  • 23
0 votes
1 answer
35 views

airflow DAG run: unpickling with dill gives NameError: name 'timedelta' is not defined

I have a simple DAG (with render_template_as_native_obj option enabled). I'm trying to pass a {{ dag_run }} to a PythonVirtualenvOperator, but when it starts to execute it fails with NameError: name '...
Felix's user avatar
  • 3,361
0 votes
0 answers
37 views

Resumable AES GCM in Python 3?

Is there an implementation of AES GCM for Python 3 which not only accepts piecewise messages, but also allows the working state of the cipher to be serialized, stored, and later resumed? (I have an ...
Craig Iannello's user avatar
1 vote
1 answer
44 views

Different code behavior using VSC Debugger (out of bounds on iterable)

I have an iterable with appropriate len and getitem methods implemented. I pickle this object to a "train_file.pth" pickle file. Then, when running the following code snippet : with open(...
ewenmichel's user avatar
0 votes
0 answers
25 views

pickle gives me an EOFError: Ran out of input error, and the file is not empty

this is the code that im working on. it basically stores the user ip address in a pickle file after encoding it and then tries the decode it. pretty simple. what i tried to find out (from the many ...
Metaverse09's user avatar
-1 votes
0 answers
20 views

PicklingError Can't pickle <function Mixture._expectation_list at 0x000Ef>: it's not the same object as sidekit.mixture.Mixture._expectation_list

I am trying to train Universal Background Model(ubm) using EM split of sidekit. But when I run ubm.EM_split( features_server=server, #sidekit.FeaturesServer used to load data ...
0 votes
0 answers
32 views

Performance of very large dictionaries when storing/loading via pickle

I have a very large dictionary (multiple millions) which will also be accesses multiple millions times. I was wondering about the low-level performances of such dicts because if they need to be ...
Daraan's user avatar
  • 1,805
0 votes
1 answer
32 views

How is a multiprocessing.Event passed to a child process under the hood?

According to this SO answer, when a multiprocessing.Queue is passed to a child process, what is actually sent is a file descriptor (or handle) obtained from pipe, instead of pickling the Queue object ...
Joerg's user avatar
  • 812
0 votes
0 answers
63 views

pikle (pkl) file for AI image generator

I'm trying to generate pictures form (StyleGAN2-ADA-PyTorch) (https://colab.research.google.com/github/dvschultz/stylegan2-ada-pytorch/blob/main/SG2_ADA_PyTorch.ipynb#scrollTo=jG7ZEc_982io). I'm stuck ...
Yiorgos Paraskeva's user avatar
0 votes
0 answers
14 views

Can't find attribute of type `MyData` in serialized file even though it requires `MyData`

I'm struggling with fixing a serialized pickle file right now in py2 (can't use py3). I have a file called data.pickle that I'm trying to load with: import cPickle as CP with open('data.pickle', 'rb')...
roulette01's user avatar
  • 2,054
4 votes
1 answer
140 views

PythonVirtualenvOperator gives error: No module named unusual_prefix_***_dag

I'm using airflow 2.5.3 with Kubernetes executor and Python 3.7. I've tried to make a simple DAG with only one PythonVirtualnvOperator and two context variables ({{ ts }} and {{ dag }}) passed into it....
Felix's user avatar
  • 3,361
1 vote
0 answers
36 views

How to locate object that cannot be pickled inside complex object

Related to my earlier question: How to I change import statements in a pickle file? I am trying to change an import statement inside a serialized file. It ended up causing some unexpected issues when ...
roulette01's user avatar
  • 2,054
1 vote
1 answer
40 views

How to I change import statements in a pickle file?

I recently removed a module xyz and replace it with another one of a different name abc. In some pickle files, it still imports the xyz module. I want to change it to import the abc module. How can I ...
roulette01's user avatar
  • 2,054
0 votes
0 answers
20 views

creating a function to return average daily pm2.5 values from data stored in a dictionary with timestamps and pm2.5 values

Disclaimer I just started learning python, so I have very little clue what I am doing. This is my code so far, my average day function is overly complicated and does not work. It should return the ...
bigburd's user avatar
0 votes
0 answers
29 views

_pickle.PicklingError: Can't pickle <function _lambdifygenerated at 0x0000026D008DFF70>: attribute lookup _lambdifygenerated on __main__ failed

For some reason I can't save my newly created object with pickle. This is my Python code: import pickle from distutils.log import warn from ikpy.chain import Chain import numpy as np import casadi as ...
openapple's user avatar
0 votes
0 answers
13 views

Use DataLoader Tensorflow for a large pickle size

I have pickle 2 pickle for trainSet and testSet such as data.plk , lables.plk for example for train data pickle have size 2GB. I want to use this pickle file for fit model and get error OOM (out of ...
Amir Ahmadi's user avatar
0 votes
0 answers
23 views

Celery is refusing to deserialize content of type pickle

Recently I upgraded packages celery, amqp, kombu and redis of our Django project to their later versions but for some reason I had to downgrade again. The problem is that after downgrading, celery app ...
Sarah A's user avatar
0 votes
0 answers
47 views

python check if type is serializable

Trying to use a distributed cluster computing framework, i want to pre-validate a graph before running it for non serializable types. Presume i have the following class: class Human: def __init__(...
user22734546's user avatar
-1 votes
1 answer
73 views

Trying to unpickle LogisticRegression estimator from version 1.3.0 when using version 0.24.2. What does this warning mean and how do I fix it?

I'm trying to deploy a LogisticRegression model on an active cloud shell that was trained using scikit-learn version 1.3.0, but I'm getting the following warning message: /usr/local/lib/python3.9/site-...
Yashdeep Khare's user avatar
2 votes
3 answers
62 views

Do numpy.savez and numpy.savez_compressed use pickle?

I recently encountered numpy.savez and numpy.savez_compressed. Both seem to work well with arrays of differing types, including object arrays. However, numpy.load does not work well with object type ...
tnknepp's user avatar
  • 6,050
0 votes
2 answers
52 views

How to load pickle files independent from any current implementation?

Remade Post: Let's say I have a pickle file with an instance of a custom class inside. However, I do not have the source code of that class anymore (or only a modified one, which will not match). ...
N. Jonas Figge's user avatar
0 votes
0 answers
20 views

ModuleNotFoundError: No module named 'sklearn.linear_model.base' while running util.py file

I encountered a ModuleNotFoundError while running the server. The error occurs when the script attempts to load the model using pickle. It specifically states that no module is named 'sklearn....
G Rishith's user avatar
1 vote
0 answers
34 views

Save data from cpp into bytes that can be pickle loaded by Python and seen as numpy ndarray

PROBLEM Let's say I have a python script that works with a numpy ndarray and then pickle.dumps it into bytes. Another python script would then pickle.loads those bytes and see the object as a numpy ...
FV_at_Alst's user avatar
0 votes
0 answers
20 views

Pickle - Trying to read from a text file - UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 37: character maps to <undefined>

with open ("Save.txt" , "r+") as f: f.seek(0) if not f.read(): print("No Data") while True: starter = int(input('...
Anonymous's user avatar
4 votes
1 answer
94 views

how to read corrupted pickle file

I have a binary file written using pickle.dump containing logs from an app (a set of tuples of floats and strings), it worked great for 24h but now when trying to read it using import pickle path = &...
Avo K's user avatar
  • 41
0 votes
1 answer
88 views

Error of saving json during converting openvino model to tensorflow with parameter --output_weight_and_json

I want convert my openvino-model to tf saved_model according to the documentation of https://github.com/PINTO0309/openvino2tensorflow repository. But when I use command: openvino2tensorflow --...
Alimagadov K.'s user avatar
0 votes
2 answers
81 views

Understanding the pickled object size

I execute the following line in Python interpreter: len(pickle.dumps([numpy.random.random(384).tolist()]*55)) It gives 3584. Given that numpy.random.random produces 8-byte floats, the data being ...
AlwaysLearning's user avatar
0 votes
0 answers
6 views

EOF about spec = torch.load(spec_filename)

I have the same problem as follows: spec = torch.load(spec_filename) magic_number = pickle_module.load(f, **pickle_load_args) return _legacy_load(opened_file, map_location, pickle_module, **...
QinHsiu's user avatar
1 vote
1 answer
19 views

Poor performance of unpickling files with uWSGI

I have to unpickle around 200 files, each containing a custom class containing a numpu.ndarray (shape 500x32) and some metadata (few int and str). Each file's size is around 50ko. Here is the ...
Laurent Claessens's user avatar

1
2 3 4 5
99