0

I'm trying to create a python script that copies files from a remote server to my local drive. The problem is that I noticed that it doesn't spawn a thread to copy other files in parallel.

import shutil
import threading

LocalPath = "C:\\folder1"
RemotePath = "X:\\folder1"

# downloader/copier
def monitorCopy (Filename) :
    print("Copying of "+Filename+" started.")
    shutil.copy(RemotePath+"\\"+Filename+"\\"+Filename+".zip",LocalPath)
    print("Copying of "+Filename+" finished.")

# main function
if __name__ == "__main__" :
    filelist = ["file1","file2","file3"]
    print(range(len(filelist)))
    p = []
    for i in range(len(filelist)) :
        p.append(i)
        p[i] = threading.Thread(target=monitorCopy,args=(filelist[i],))
        p[i].daemon = True
        p[i].start()
        p[i].join() 

Folders tree looks like this:

Remote Repository
     [+] --- Filename
                [+] --- Filename.zip

Local Repository
     [+] --- Filename.zip

can someone please help me on this?

1 Answer 1

1

You're calling join on each thread object right after you create it, which is preventing more than one thread from running at a time. A thread_object.join() call tells Python to block until thread_object has completed execution, which in your case means you're starting one thread for an item in filelist, blocking until the thread finishes, then moving on to the next item in filelist and doing the same thing. What you really want to do is create all your threads in parallel, and only join them after they're all created:

if __name__ == "__main__" :
    filelist = ["file1","file2","file3"]
    print(range(len(filelist)))
    p = []
    for f in filelist:
        t = threading.Thread(target=monitorCopy,args=(f,))
        p.append(t)
        t.daemon = True
        t.start()

    for t in p:
        t.join() 
7
  • I see. It works. I have one more question, if a thread's task ends, does it automatically take itself out of the list?
    – jezztify
    Jul 8, 2014 at 2:16
  • @user3814548 No, the completed Thread object will remain in the list.
    – dano
    Jul 8, 2014 at 2:17
  • hmmmm. Do you have any suggestion on how to do this? I'm thinking of having "t" to become global and pass "f" to monitorCopy and just add t[f].stop().
    – jezztify
    Jul 8, 2014 at 2:33
  • @user3814548 Can you explain a bit about your motivation for deleting the thread objects from the list? If you're calling thread.join() on each thread in the list, once you're done, every thread will be finished, and the whole list can just be thrown away; there's no reason to bother deleting individual entries.
    – dano
    Jul 8, 2014 at 2:38
  • Oh, sorry I didn't know about that. I'm new to python programming. My motivation is that the in the future I will replace the filelist into a dynamic list which parses a text file which contains filenames. So when the main function parses the text file and finds a new filename from it, it will create a thread to download it. The code up there is for python practice.
    – jezztify
    Jul 8, 2014 at 2:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.