1

This may be trivial, but for me takes long time to solve in Python. I tried to implement multiprocessing and I used pool. I need to pass multiple arguments. Then I implement like

def doWork(worker_args):
bbox_list, conf_list, all_rects, w, h, pix_per_w, n = worker_args

sys.stdout.write("w: %d%%   \n" % n )
sys.stdout.flush()

for k in range(h * w):
    conf = conf_list[n][k,1].flatten()[0]
    if conf > 0.75:
       y = int(k / w)
       x = int(k % w)
       bbox = bbox_list[n][k]
       abs_cx = pix_per_w/2 + pix_per_w*x + int(bbox[0,0,0])
       abs_cy = pix_per_h/2 + pix_per_h*y+int(bbox[1,0,0])
       w = bbox[2,0,0]
       h = bbox[3,0,0]
       all_rects[y][x].append(Rect(abs_cx,abs_cy,w,h,conf))

Then from main, I sent as

         all_rects = [[[] for x in range(net_config["grid_width"])] for y in range(net_config["grid_height"])]
         pool = multiprocessing.Pool(len(bbox_list))
         worker_args = [(bbox_list, conf_list, all_rects, net_config["grid_width"], net_config["grid_height"], pix_per_w, range(len(bbox_list)))]
         pool.map(doWork, worker_args)

Then error in printing n in doWork function is

TypeError: %d format: a number is required, not list

If I implement like

def doWork(n): and 
in main
pool.map(doWork, range(len(bbox_list)))

It works. Send data from 0 to 5.

Then I implement like

def doWork(worker_args): and 
in main
worker_args = [(bbox_list, conf_list, all_rects, net_config["grid_width"], net_config["grid_height"], pix_per_w)]
pool.map(doWork, worker_args)

It works also. But if I combine two as in the first example, then I got error at n. How to send n in single digit, not in list? Thanks

5
  • Could the answers in this question help? E.g., one answer mentions a small library, parmap that extends the Python map to handle multiple arguments.
    – Hans
    Oct 21, 2015 at 10:50
  • firstly, I don't think map from multiprocessing works as you're expecting. it will call you doWork function with each element of the worker_args. If you want to pass multiple parameters they need to be zipped up, or an alternative could be to pass an initializer to the pool and pass in the shared state there.
    – Sam Mason
    Oct 21, 2015 at 11:40
  • @SamMason, thanks. My another issue is the way I implement, it has memory issue. I am expecting send by reference. Actually it is not. How to implement send by reference? (stackoverflow.com/questions/986006/…) this discussion says about byvalue or byreference.
    – batuman
    Oct 22, 2015 at 5:12
  • This article (docs.python.org/3/faq/…) passed by assignment. Since assignment just creates references to objects, there’s no alias between an argument name in the caller and callee, why I have memory issue? All memories are used and program hanged after running sometime.
    – batuman
    Oct 22, 2015 at 5:17
  • I'd post a separate question about variable references. Judging from your other questions you seem trying to learn a lot all at once, try slowing down and learning each concept in greater detail before trying to combine them with other new ideas
    – Sam Mason
    Oct 23, 2015 at 19:34

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.