5

I am wondering what this error might mean:

PicklingError: Can't pickle <type 'function'>: attribute lookup __builtin__.function failed

I understand that it has something to do with using multiple cores. I am running my program on a cluster and using 15 threads in this line of my code:

gauss2 = PTSampler(ntemps, renwalkers, rendim, lnlike, lnprior, threads=15)

The sampler in question is the one documented for the Parallel Tempering sampler at http://dan.iel.fm/emcee/current/user/pt/

Any idea what this error might mean?

4

1 Answer 1

5

The error means you are trying to pickle a builtin FunctionType… not the function itself. It's likely do to a coding error somewhere picking up the class of the function instead of the function itself.

>>> import sys
>>> import pickle
>>> import types
>>> types.FunctionType
<type 'function'>
>>> try:
...     pickle.dumps(types.FunctionType)
... except:
...     print sys.exc_info()[1]
... 
Can't pickle <type 'function'>: it's not found as __builtin__.function
>>> def foo(x):
...   return x
... 
>>> try:
...     pickle.dumps(type(foo))
... except:
...     print sys.exc_info()[1]
... 
Can't pickle <type 'function'>: it's not found as __builtin__.function
>>> try:
...     pickle.dumps(foo.__class__)
... except:
...     print sys.exc_info()[1]
... 
Can't pickle <type 'function'>: it's not found as __builtin__.function
>>> pickle.dumps(foo)
'c__main__\nfoo\np0\n.'
>>> pickle.dumps(foo, -1)
'\x80\x02c__main__\nfoo\nq\x00.'

If you have a FunctionType object, then all you need to do is get one of the instances of that class -- i.e. a function like foo.

6
  • What if you want to pickle a class?
    – salotz
    Commented Aug 11, 2016 at 21:25
  • Classes tend to work if they are user built, however they pickle by reference, so there are some limitations (e.g. they are best imported). You can try dill, which makes pickling classes much easier. Commented Aug 12, 2016 at 13:15
  • I tried dill and I couldn't get it to work for dynamically created classes, although elsewhere I know you said it is possible. I decided that having these complicated objects is probably not the best anyhow.
    – salotz
    Commented Aug 12, 2016 at 13:49
  • dill should work for dynamically generated classes, in general, so if you have found cases that it doesn't please ask a SO question or post it on the dill github issues. Commented Aug 12, 2016 at 14:40
  • All your exceptions above say it's not found as __builtin__.function but the exception from the original question says attribute lookup __builtin__.function failed...
    – djvg
    Commented Jul 9, 2020 at 9:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.