Something like this exists in dill
. Let's look at a list of objects, and see what we can do:
>>> import dill
>>> f = open('whatever', 'w')
>>> f.close()
>>>
>>> l = [iter([1,2,3]), xrange(5), open('whatever', 'r'), lambda x:x]
>>> dill.detect.trace(False)
>>> dill.pickles(l)
False
Ok, dill
fails to pickle the list. So what's the problem?
>>> dill.detect.trace(True)
>>> dill.pickles(l)
T4: <type 'listiterator'>
False
Ok, the first item in the list fails to pickle. What about the rest?
>>> map(dill.pickles, l)
T4: <type 'listiterator'>
Si: xrange(5)
F2: <function _eval_repr at 0x106991cf8>
Fi: <open file 'whatever', mode 'r' at 0x10699c810>
F2: <function _create_filehandle at 0x106991848>
B2: <built-in function open>
F1: <function <lambda> at 0x1069f6848>
F2: <function _create_function at 0x1069916e0>
Co: <code object <lambda> at 0x105a0acb0, file "<stdin>", line 1>
F2: <function _unmarshal at 0x106991578>
D1: <dict object at 0x10591d168>
D2: <dict object at 0x1069b1050>
[False, True, True, True]
Hm. The other objects pickle just fine. So, let's replace the first object.
>>> dill.detect.trace(False)
>>> l[0] = xrange(1,4)
>>> dill.pickles(l)
True
>>> _l = dill.loads(dill.dumps(l))
Now our object pickles. Well, we could be taking advantage of some built-in object sharing that happens for pickling on linux/unix/mac… so what about a stronger check, like actually pickling across a sub-process (like happens on windows)?
>>> dill.check(l)
[xrange(1, 4), xrange(5), <open file 'whatever', mode 'r' at 0x107998810>, <function <lambda> at 0x1079ec410>]
>>>
Nope, the list still works… so this is an object that could be sent to another process successfully.
Now, with regard to your error, which everyone seemed to ignore…
The ModuleType
object is not pickleable, and that's causing your error.
>>> import types
>>> types.ModuleType
<type 'module'>
>>>
>>> import pickle
>>> pickle.dumps(types.ModuleType)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 1374, in dumps
Pickler(file, protocol).dump(obj)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 224, in dump
self.save(obj)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 748, in save_global
(obj, module, name))
pickle.PicklingError: Can't pickle <type 'module'>: it's not found as __builtin__.module
However, if we import dill
, it magically works.
>>> import dill
>>> pickle.dumps(types.ModuleType)
"cdill.dill\n_load_type\np0\n(S'ModuleType'\np1\ntp2\nRp3\n."
>>>