211

I am learning how to use the threading and the multiprocessing modules in Python to run certain operations in parallel and speed up my code.

I am finding it hard (maybe because I don't have any theoretical background about it) to understand what the difference is between a threading.Thread() object and a multiprocessing.Process() one.

Also, it is not entirely clear to me how to instantiate a queue of jobs and have only 4 (for example) of them running in parallel, while the others wait for resources to free before being executed.

I find the examples in the documentation clear, but not very exhaustive; as soon as I try to complicate things a bit, I receive a lot of weird errors (like a method that can't be pickled, and so on).

So, when should I use the threading and multiprocessing modules?

Can you link me to some resources that explain the concepts behind these two modules and how to use them properly for complex tasks?

3
  • There's more, there's also the Thread module (called _thread in python 3.x). To be honest, I've never understood the differences myself...
    – Dunno
    Aug 7, 2013 at 21:42
  • 3
    @Dunno: As the Thread/_thread documentation explicitly says, it's "low-level primitives". You might use it to build custom synchronization objects, to control the join order of a tree of threads, etc. If you can't imagine why you'd need to use it, don't use it, and stick with threading.
    – abarnert
    Aug 7, 2013 at 22:36
  • The @abarnert answer, which is also the accepted answer, is wonderful. I just want to share an excellent link as a good supplement to GIL: eli.thegreenplace.net/2012/01/16/…
    – Bruce
    Jun 24, 2021 at 18:41

6 Answers 6

347

What Giulio Franco says is true for multithreading vs. multiprocessing in general.

However, Python* has an added issue: There's a Global Interpreter Lock that prevents two threads in the same process from running Python code at the same time. This means that if you have 8 cores, and change your code to use 8 threads, it won't be able to use 800% CPU and run 8x faster; it'll use the same 100% CPU and run at the same speed. (In reality, it'll run a little slower, because there's extra overhead from threading, even if you don't have any shared data, but ignore that for now.)

There are exceptions to this. If your code's heavy computation doesn't actually happen in Python, but in some library with custom C code that does proper GIL handling, like a numpy app, you will get the expected performance benefit from threading. The same is true if the heavy computation is done by some subprocess that you run and wait on.

More importantly, there are cases where this doesn't matter. For example, a network server spends most of its time reading packets off the network, and a GUI app spends most of its time waiting for user events. One reason to use threads in a network server or GUI app is to allow you to do long-running "background tasks" without stopping the main thread from continuing to service network packets or GUI events. And that works just fine with Python threads. (In technical terms, this means Python threads give you concurrency, even though they don't give you core-parallelism.)

But if you're writing a CPU-bound program in pure Python, using more threads is generally not helpful.

Using separate processes has no such problems with the GIL, because each process has its own separate GIL. Of course you still have all the same tradeoffs between threads and processes as in any other languages—it's more difficult and more expensive to share data between processes than between threads, it can be costly to run a huge number of processes or to create and destroy them frequently, etc. But the GIL weighs heavily on the balance toward processes, in a way that isn't true for, say, C or Java. So, you will find yourself using multiprocessing a lot more often in Python than you would in C or Java.


Meanwhile, Python's "batteries included" philosophy brings some good news: It's very easy to write code that can be switched back and forth between threads and processes with a one-liner change.

If you design your code in terms of self-contained "jobs" that don't share anything with other jobs (or the main program) except input and output, you can use the concurrent.futures library to write your code around a thread pool like this:

with concurrent.futures.ThreadPoolExecutor(max_workers=4) as executor:
    executor.submit(job, argument)
    executor.map(some_function, collection_of_independent_things)
    # ...

You can even get the results of those jobs and pass them on to further jobs, wait for things in order of execution or in order of completion, etc.; read the section on Future objects for details.

Now, if it turns out that your program is constantly using 100% CPU, and adding more threads just makes it slower, then you're running into the GIL problem, so you need to switch to processes. All you have to do is change that first line:

with concurrent.futures.ProcessPoolExecutor(max_workers=4) as executor:

The only real caveat is that your jobs' arguments and return values have to be pickleable (and not take too much time or memory to pickle) to be usable cross-process. Usually this isn't a problem, but sometimes it is.


But what if your jobs can't be self-contained? If you can design your code in terms of jobs that pass messages from one to another, it's still pretty easy. You may have to use threading.Thread or multiprocessing.Process instead of relying on pools. And you will have to create queue.Queue or multiprocessing.Queue objects explicitly. (There are plenty of other options—pipes, sockets, files with flocks, … but the point is, you have to do something manually if the automatic magic of an Executor is insufficient.)

But what if you can't even rely on message passing? What if you need two jobs to both mutate the same structure, and see each others' changes? In that case, you will need to do manual synchronization (locks, semaphores, conditions, etc.) and, if you want to use processes, explicit shared-memory objects to boot. This is when multithreading (or multiprocessing) gets difficult. If you can avoid it, great; if you can't, you will need to read more than someone can put into an SO answer.


From a comment, you wanted to know what's different between threads and processes in Python. Really, if you read Giulio Franco's answer and mine and all of our links, that should cover everything… but a summary would definitely be useful, so here goes:

  1. Threads share data by default; processes do not.
  2. As a consequence of (1), sending data between processes generally requires pickling and unpickling it.**
  3. As another consequence of (1), directly sharing data between processes generally requires putting it into low-level formats like Value, Array, and ctypes types.
  4. Processes are not subject to the GIL.
  5. On some platforms (mainly Windows), processes are much more expensive to create and destroy.
  6. There are some extra restrictions on processes, some of which are different on different platforms. See Programming guidelines for details.
  7. The threading module doesn't have some of the features of the multiprocessing module. (You can use multiprocessing.dummy to get most of the missing API on top of threads, or you can use higher-level modules like concurrent.futures and not worry about it.)

* It's not actually Python, the language, that has this issue, but CPython, the "standard" implementation of that language. Some other implementations don't have a GIL, like Jython.

** If you're using the fork start method for multiprocessing—which you can on most non-Windows platforms—each child process gets any resources the parent had when the child was started, which can be another way to pass data to children.

11
  • thanks, but I am not sure I understood everything. Anyway I am trying to do it a bit for learning purposes, and a bit because with a naive use of thread I halved the speed of my code (starting more than 1000 threads at the same time, each calling an external app.. this saturates the cpu, yet there is a x2 increase in speed). I think managing the thread smartly might really improve the speed of my code..
    – lucacerone
    Aug 7, 2013 at 22:32
  • 4
    @LucaCerone: Ah, if your code spends most of its time waiting on external programs, then yes, it will benefit from threading. Good point. Let me edit the answer to explain that.
    – abarnert
    Aug 7, 2013 at 22:37
  • 3
    @LucaCerone: Meanwhile, what parts do you not understand? Without knowing the level of knowledge you're starting with, it's hard to write a good answer… but with some feedback, maybe we can come up with something that's helpful to you and to future readers as well.
    – abarnert
    Aug 7, 2013 at 22:39
  • 6
    @LucaCerone You should read the PEP for multiprocessing here. It gives timings and examples of threads vs multiprocessing.
    – mr2ert
    Aug 7, 2013 at 23:15
  • 1
    @LucaCerone: If the object that the method is bound to doesn't have any complex state, the simplest workaround for the pickling issue is to write a stupid wrapper function that generates the object and calls its method. If it does have complex state, then you probably need to make it picklable (which is pretty easy; the pickle docs explain it), and then at worst your stupid wrapper is def wrapper(obj, *args): return obj.wrapper(*args).
    – abarnert
    Aug 7, 2013 at 23:48
51

Multiple threads can exist in a single process. The threads that belong to the same process share the same memory area (can read from and write to the very same variables, and can interfere with one another). On the contrary, different processes live in different memory areas, and each of them has its own variables. In order to communicate, processes have to use other channels (files, pipes or sockets).

If you want to parallelize a computation, you're probably going to need multithreading, because you probably want the threads to cooperate on the same memory.

Speaking about performance, threads are faster to create and manage than processes (because the OS doesn't need to allocate a whole new virtual memory area), and inter-thread communication is usually faster than inter-process communication. But threads are harder to program. Threads can interfere with one another, and can write to each other's memory, but the way this happens is not always obvious (due to several factors, mainly instruction reordering and memory caching), and so you are going to need synchronization primitives to control access to your variables.

4
  • 16
    This is missing some very important information about the GIL, which makes it misleading.
    – abarnert
    Aug 7, 2013 at 22:17
  • 1
    @mr2ert: Yes, that's the very important information in a nutshell. :) But it's a bit more complicated than that, which is why I wrote a separate answer.
    – abarnert
    Aug 7, 2013 at 23:01
  • 3
    I thought I commented saying that @abarnert is right, and I forgot about the GIL in answering here. So this answer is wrong, you should not upvote it. Jan 6, 2015 at 11:24
  • 9
    I downvoted this answer because it still does not answer at all what is the difference between Python threading and multiprocessing. Feb 7, 2016 at 11:48
15

Python documentation quotes

I've highlighted the key Python documentation quotes about Process vs Threads and the GIL at: What is the global interpreter lock (GIL) in CPython?

Process vs thread experiments

I did a bit of benchmarking in order to show the difference more concretely.

In the benchmark, I timed CPU and IO bound work for various numbers of threads on an 8 hyperthread CPU. The work supplied per thread is always the same, such that more threads means more total work supplied.

The results were:

enter image description here

Plot data.

Meaning of each section:

  • "CPU Bound": how long it takes for all threads/processes to finish a fixed amount of CPU-bound work for each thread. The amount of work per thread is fixed, so e.g. when 2 threads were used, 2x total work is done.

    Interpretation: threads were always slower because they were fighting for CPU lock

  • "CPU bound / threads": the above graph divided by the number of threads. This gives the average time it took to finish each unit of work. We observe that:

    • for threads, this is constant: 2x work takes 2x time to finish. Therefore, it didn't parallelize at all.

    • for processes, this decreases until 4x, and then remains constant. Therefore it parallelized well up to 4x and was able to run things faster, but didn't scale beyond that.

      I would have expected scaling up to 8x since I'm on a 4 core 8 hyperthread machine.

      Contrast that with a C POSIX CPU-bound work which reaches the expected 8x speedup: What do 'real', 'user' and 'sys' mean in the output of time(1)?

      TODO: I don't know the reason for this, there must be other Python inefficiencies coming into play.

  • "Thread / Process ratio": ratio of the two above lines. This shows us the 4x speedup limit very clearly.

  • "IO bound": same as "CPU bound" but with an IO bound task

Conclusions:

  • for CPU bound work, multiprocessing is always faster, presumably due to the GIL
  • for IO bound work. both are exactly the same speed

Test code:

#!/usr/bin/env python3

import multiprocessing
import threading
import time
import sys

def cpu_func(result, niters):
    '''
    A useless CPU bound function.
    '''
    for i in range(niters):
        result = (result * result * i + 2 * result * i * i + 3) % 10000000
    return result

class CpuThread(threading.Thread):
    def __init__(self, niters):
        super().__init__()
        self.niters = niters
        self.result = 1
    def run(self):
        self.result = cpu_func(self.result, self.niters)

class CpuProcess(multiprocessing.Process):
    def __init__(self, niters):
        super().__init__()
        self.niters = niters
        self.result = 1
    def run(self):
        self.result = cpu_func(self.result, self.niters)

class IoThread(threading.Thread):
    def __init__(self, sleep):
        super().__init__()
        self.sleep = sleep
        self.result = self.sleep
    def run(self):
        time.sleep(self.sleep)

class IoProcess(multiprocessing.Process):
    def __init__(self, sleep):
        super().__init__()
        self.sleep = sleep
        self.result = self.sleep
    def run(self):
        time.sleep(self.sleep)

if __name__ == '__main__':
    cpu_n_iters = int(sys.argv[1])
    sleep = 1
    cpu_count = multiprocessing.cpu_count()
    input_params = [
        (CpuThread, cpu_n_iters),
        (CpuProcess, cpu_n_iters),
        (IoThread, sleep),
        (IoProcess, sleep),
    ]
    header = ['nthreads']
    for thread_class, _ in input_params:
        header.append(thread_class.__name__)
    print(' '.join(header))
    for nthreads in range(1, 2 * cpu_count):
        results = [nthreads]
        for thread_class, work_size in input_params:
            start_time = time.time()
            threads = []
            for i in range(nthreads):
                thread = thread_class(work_size)
                threads.append(thread)
                thread.start()
            for i, thread in enumerate(threads):
                thread.join()
            results.append(time.time() - start_time)
        print(' '.join('{:.6e}'.format(result) for result in results))

GitHub upstream + plotting code on same directory.

Tested on Ubuntu 18.10, Python 3.6.7, in a Lenovo ThinkPad P51 laptop with CPU: Intel Core i7-7820HQ CPU (4 cores / 8 threads), RAM: 2x Samsung M471A2K43BB1-CRC (2x 16GiB), SSD: Samsung MZVLB512HAJQ-000L7 (3,000 MB/s).

Visualize which threads are running at a given time

This post https://rohanvarma.me/GIL/ taught me that you can run a callback whenever a thread is scheduled with the target= argument of threading.Thread and the same for multiprocessing.Process.

This allows us to view exactly which thread runs at each time. When this is done, we would see something like (I made this particular graph up):

            +--------------------------------------+
            + Active threads / processes           +
+-----------+--------------------------------------+
|Thread   1 |********     ************             |
|         2 |        *****            *************|
+-----------+--------------------------------------+
|Process  1 |***  ************** ******  ****      |
|         2 |** **** ****** ** ********* **********|
+-----------+--------------------------------------+
            + Time -->                             +
            +--------------------------------------+

which would show that:

  • threads are fully serialized by the GIL
  • processes can run in parallel
9
  • It's not clear what is being shown in the different figures. Can you add some explanation? Jan 13 at 17:09
  • Is the purple line always 2 threads, and the green line always single thread? Jan 14 at 11:11
  • @FreelanceConsultant no, the number of threads or processes is the x axis. One color is thread the other processes. Jan 14 at 11:14
  • 1
    "...would have expected scaling up to 8x since I'm on a 4 core 8 hyperthread machine" <- Bad expectation for CPU-bound work, I think. A 4 core / 8 HT ("vcore") CPU only has 4 real hardware cores, and thus can only actively run 4 threads/processes at a time. The hyperthreads just make it faster to switch between threads on a single core. For CPU-bound work, 4x speedup is realistic. That "8x speedup on C POSIX" - is that actually measuring computational work done, or just metrics for how long a process was scheduled ("swapped in"), which could be on a virtual core doing no real work? Apr 2 at 0:21
  • 1
    Ah, yeah, it looks like I oversimplified things there. And I've got some reading to do. Will follow up on that. Thanks! Apr 8 at 15:58
5

I believe this link answers your question in an elegant way.

To be short, if one of your sub-problems has to wait while another finishes, multithreading is good (in I/O heavy operations, for example); by contrast, if your sub-problems could really happen at the same time, multiprocessing is suggested. However, you won't create more processes than your number of cores.

2

Here's some performance data for python 2.6.x that calls to question the notion that threading is more performant than multiprocessing in IO-bound scenarios. These results are from a 40-processor IBM System x3650 M4 BD.

IO-Bound Processing : Process Pool performed better than Thread Pool

>>> do_work(50, 300, 'thread','fileio')
do_work function took 455.752 ms

>>> do_work(50, 300, 'process','fileio')
do_work function took 319.279 ms

CPU-Bound Processing : Process Pool performed better than Thread Pool

>>> do_work(50, 2000, 'thread','square')
do_work function took 338.309 ms

>>> do_work(50, 2000, 'process','square')
do_work function took 287.488 ms

These aren't rigorous tests, but they tell me that multiprocessing isn't entirely unperformant in comparison to threading.

Code used in the interactive python console for the above tests

from multiprocessing import Pool
from multiprocessing.pool import ThreadPool
import time
import sys
import os
from glob import glob

text_for_test = str(range(1,100000))

def fileio(i):
 try :
  os.remove(glob('./test/test-*'))
 except : 
  pass
 f=open('./test/test-'+str(i),'a')
 f.write(text_for_test)
 f.close()
 f=open('./test/test-'+str(i),'r')
 text = f.read()
 f.close()


def square(i):
 return i*i

def timing(f):
 def wrap(*args):
  time1 = time.time()
  ret = f(*args)
  time2 = time.time()
  print '%s function took %0.3f ms' % (f.func_name, (time2-time1)*1000.0)
  return ret
 return wrap

result = None

@timing
def do_work(process_count, items, process_type, method) :
 pool = None
 if process_type == 'process' :
  pool = Pool(processes=process_count)
 else :
  pool = ThreadPool(processes=process_count)
 if method == 'square' : 
  multiple_results = [pool.apply_async(square,(a,)) for a in range(1,items)]
  result = [res.get()  for res in multiple_results]
 else :
  multiple_results = [pool.apply_async(fileio,(a,)) for a in range(1,items)]
  result = [res.get()  for res in multiple_results]


do_work(50, 300, 'thread','fileio')
do_work(50, 300, 'process','fileio')

do_work(50, 2000, 'thread','square')
do_work(50, 2000, 'process','square')
1
  • 1
    I have used your code (removed the glob part) and have found this interesting results with Python 2.6.6: >>> do_work(50, 300, 'thread', 'fileio') --> 237.557 ms >>> do_work(50, 300, 'process', 'fileio') --> 323.963 ms >>> do_work(50, 2000, 'thread', 'square') --> 232.082 ms >>> do_work(50, 2000, 'process', 'square') --> 282.785 ms Nov 8, 2017 at 19:49
-5

Well, most of the question is answered by Giulio Franco. I will further elaborate on the consumer-producer problem, which I suppose will put you on the right track for your solution to using a multithreaded app.

fill_count = Semaphore(0) # items produced
empty_count = Semaphore(BUFFER_SIZE) # remaining space
buffer = Buffer()

def producer(fill_count, empty_count, buffer):
    while True:
        item = produceItem()
        empty_count.down();
        buffer.push(item)
        fill_count.up()

def consumer(fill_count, empty_count, buffer):
    while True:
        fill_count.down()
        item = buffer.pop()
        empty_count.up()
        consume_item(item)

You could read more on the synchronization primitives from:

 http://linux.die.net/man/7/sem_overview
 http://docs.python.org/2/library/threading.html

The pseudocode is above. I suppose you should search the producer-consumer-problem to get more references.

5
  • sorry innosam, but this seems C++ to me? thanks for the links :)
    – lucacerone
    Aug 7, 2013 at 22:07
  • Actually, ideas behind multiprocessing and multithreading are language independent. The solution would be similar to the above code.
    – innosam
    Aug 7, 2013 at 22:10
  • 2
    This isn't C++; it's pseudocode (or it's code for a mostly-dynamically-typed language with a C-like syntax. That being said, I think it's more useful to write Python-like pseudocode for teaching Python users. (Especially since the Python-like psuedocode often turns out to be runnable code, or at least close to it, which is rarely true for C-like pseudocode…)
    – abarnert
    Aug 7, 2013 at 22:53
  • I've rewritten it as Python-like pseudocode (also using OO and passing parameters instead of using global objects); feel free to revert if you think that makes things less clear.
    – abarnert
    Aug 7, 2013 at 22:57
  • Also, it's worth noting that the Python stdlib has a synchronized queue built in that wraps up all of these details, and its thread and process pool APIs abstract things even further. It's definitely worth understanding how synchronized queues work under the covers, but you will rarely need to write one yourself.
    – abarnert
    Aug 7, 2013 at 23:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.