11

When I put an object in Queue, Is it necessary to create deep copy of object and then put in Queue?

2
  • 2
    I believe the object is pickled before it is transferred to the receiving process (ie you're not working on the same copy)
    – thebjorn
    Jan 16, 2017 at 10:01
  • @Majid please accept an answer and close the question.
    – ppasler
    Jan 17, 2017 at 12:35

1 Answer 1

2

If you can ensure that the Object is only processed in one Thread, this is not a problem. But if you can't, it is recommended to use a deep copy.

The Queue object doesn't do this autmatically if you put the object into it.

See Refs

Keep in mind that the object need to be able to be pickled (Multiprocessing Basics)

It usually more useful to be able to spawn a process with arguments to tell it what work to do. Unlike with threading, to pass arguments to a multiprocessing Process the argument must be able to be serialized using pickle. This example passes each worker a number so the output is a little more interesting.

6
  • 1
    Is this relevant when the OP is using multiprocessing?
    – thebjorn
    Jan 16, 2017 at 10:00
  • What do you mean? The blog article and the book page refer to the question of deep copy, not only multiprocessing in common.
    – ppasler
    Jan 16, 2017 at 10:08
  • 1
    Sorry, I got confused by the first link's interchangeable use of multi threading and processing..
    – thebjorn
    Jan 16, 2017 at 10:15
  • Thread isn't directly relevant to multiprocessing. multiprocessing.Queue (at least in python2) makes a copy of the object, as it serializes it, though beware race conditions (see link in other answer). Finally, the book link is broken, as it doesn't contain the referenced page. Apr 18, 2018 at 15:12
  • @user1071847 which link is broken? Ich can access all of them.
    – ppasler
    Apr 26, 2018 at 12:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.