5

Instance methods can not automatically be pickled in both Python 2 or Python 3.

I need to pickle instance methods with Python 3 and I ported example code of Steven Bethard to Python 3:

import copyreg
import types

def _pickle_method(method):
    func_name = method.__func__.__name__
    obj = method.__self__
    cls = method.__self__.__class__
    return _unpickle_method, (func_name, obj, cls)

def _unpickle_method(func_name, obj, cls):
    for cls in cls.mro():
        try:
            func = cls.__dict__[func_name]
        except KeyError:
            pass
        else:
            break
    return func.__get__(obj, cls)

copyreg.pickle(types.MethodType, _pickle_method, _unpickle_method)

Is this method fool proof for pickling instance methods? Or can some things go horribly wrong? I have tested it with some mock up classes and everything seem to work.

If nothing can go wrong, why isn't it possible in Python 3 to standard pickle instance methods?

2
  • Because all references must be globally accessible. velocityreviews.com/forums/… Jul 5, 2011 at 20:48
  • @Lennart If you look to the link, you'll see that I found the same thread myself. It however does not very well explain what the real difference would be between pickling a method and function. I know that also some functions, like a lambda function can not be pickled, but in that case an error is thrown when it is not possible. Why not the same for methods?
    – Peter Smit
    Jul 5, 2011 at 20:54

2 Answers 2

0

I can't actually reproduce the original issue on python 3.5.0, see the following example. It might be worth checking on the latest version to see if it just works out-of-the-box :-)

import pickle
import sys

class Foo:

    @staticmethod
    def my_func():
        return 'Foo.my_func'


class Bar:
    pass

if __name__ == '__main__':
    if len(sys.argv) > 1 and sys.argv[1] == 'restore':
        print('loading pickle')
        with open('test.pickle', 'rb') as fp:
            restored = pickle.load(fp)
            print(restored.baz())
    else:
        print('saving pickle')
        b = Bar()
        b.baz = Foo.my_func

        with open('test.pickle', 'w+b') as fp:
            p = pickle.dump(b, fp)

(test)~/test$ python test.py

saving pickle

(test)~/test$ python test.py restore

loading pickle

Foo.my_func

0

If you want to pickle class instances (and instance methods), just use dill...

>>> import dill
>>> 
>>> class Foo:
...   def bar(self, x):
...     return self.y + x
...   def zap(self, y):
...     self.y = y
...   y = 1
... 
>>> f = Foo()
>>> f.zap(4)
>>> f.monty = 'python'
>>> 
>>> _f = dill.dumps(f)
>>> del Foo
>>> del f
>>> f = dill.loads(_f)
>>> f.monty
'python'
>>> f.y
4
>>> 
>>> _b = dill.dumps(f.bar)
>>> del f
>>> bar = dill.loads(_b)
>>> bar(4)
8
>>> 

dill works when you delete the class object, as seen above… so it also works if you start a fresh python session without the class defined, or if you change the class definition. dill even works when you don't have an instance of the class object, or a class instance available. If you want to see how to do it, look at the dill source code: https://github.com/uqfoundation/dill

Also see related: https://stackoverflow.com/a/21345273/2379433

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.