48

I have two integers in my program; let's call them "a" and "b". I would like to add them together and get another integer as a result. These are regular Python int objects. I'm wondering; how do I add them together with Twisted? Is there a special performAsynchronousAddition function somewhere? Do I need a Deferred? What about the reactor? Is the reactor involved?

6
  • 4
    Why wouldn't you use a + b?
    – Gabe
    Commented Sep 6, 2011 at 0:14
  • 15
    Won't "a + b" block?
    – Glyph
    Commented Sep 6, 2011 at 4:11
  • 4
    Since a + b is not I/O, it will not block. It will take some time to execute, but so will any other operation that you do.
    – Gabe
    Commented Sep 6, 2011 at 4:12
  • 4
    To be clear a+b does block. What if a and b are sufficiently huge integers ? Keep in mind that non-blocking very often means "blocking for a very very small time".
    – Jeremy
    Commented Oct 9, 2013 at 10:20
  • 2
    I think your use of "block" here is inconsistent with general usage. Normally, a process is described as "blocked" when it becomes non-runnable, waiting for some external event (such as the end of a network transfer). a+b as a normal arithmetic operation will never block - it will just use CPU until it's completed, so the process/thread that runs it will remain runnable throughout.
    – holdenweb
    Commented May 24, 2017 at 10:34

3 Answers 3

43

OK, to be clear.

Twisted doesn't do anything about cpu bound tasks and for good reason. there's no way to make a compute bound job go any quicker by reordering subtasks; the only thing you could possibly do is add more compute resources; and even that wouldn't work out in python because of a subtlety of its implementation.

Twisted offers special semantics and event loop handling in case the program would become "stuck" waiting for something outside if its control; most normally a process running on another machine and communicating with your twisted process over a network connection. Since you would be waiting anyways, twisted gives you a mechanism to get more things done in the meantime. That is to say, twisted provides concurrency for I/O Bound tasks

tl;dr: twisted is for network code. Everything else is just normal python.

4
  • 5
    (Worth noting though is that with spawnProcess, Twisted does provide a little support for CPU-bound tasks, by converting them into I/O bound tasks and then communicating with them the same way. But this is a good explanation nonetheless.)
    – Glyph
    Commented Sep 6, 2011 at 4:16
  • 5
    As a friend and neighbor, I'm begging you: Please do not spawn a process for the purpose of adding two integers. Commented Sep 6, 2011 at 4:49
  • 26
    Tempting as it is to troll you further, I should note at this point that I'm the original author of Twisted and that this question was written as a sort of experiment. We get a lot of questions of the flavor: "How do I call function X, with Twisted?", where the answer ("X()") is so obvious that we have trouble communicating it. Many programmers are too used to using idioms like "time.sleep()" to mean "... and then do a bunch of I/O and/or computation ... " that they can't or won't flesh out their examples further, when the distinction between I/O and computation is really critical.
    – Glyph
    Commented Sep 6, 2011 at 7:45
  • 2
    I actually quite liked your answer and will probably use elements from it in the future. I was surprised to get such useful feedback so quickly ;).
    – Glyph
    Commented Sep 6, 2011 at 7:48
9

How about this:

c = a + b

That should work, and it doesn't need to be done asynchronously (it's pretty fast).

2
  • 1
    You are forgetting that Python integers are unbound. It is very easy to build numbers for which any operation takes minutes. Commented Aug 26, 2015 at 8:48
  • See the comments to the original question Commented Apr 2, 2021 at 9:44
-9

Good question, and Twisted (or Python) should have a way to at least spawn "a + b" of to several cores (on my 8 core i7).

Unfortunately the Python GIL prevents this from happening, meaning that you will have to wait, not only for the CPU bound task, but for one core doing the job, while the seven others core are doing nothing.

Note: Maybe a better example would be "a() + b()", or even "fact(sqrt(a()**b())" etc. but the important fact is that above operation will lock one core and the GIL pretty much prevents Python for doing anything else during that operation, which could be several ms...

5
  • -1: whether or not a + b runs on a single or multiple CPUs depends on what a and b are e.g., it is possible to release GIL if a and b are numpy arrays. The task may be embarrassingly parallel i.e., it is trivial to distribute it among several CPUs. The result can be slower than sequential variant but it can be done easily.
    – jfs
    Commented Dec 21, 2013 at 7:39
  • It is not possible to distribute processing across several cores in the same CPU and process, because of the GIL. So even if I have 8 cores in my brand new MBP and I started 8 threads to distribute a calculation, the performance will degrade a factor 1000 or more.
    – renejsum
    Commented Dec 28, 2013 at 22:38
  • it is possible to distribute processing across several CPU cores in the same process because GIL can be released in C extensions such as numpy, lxml, regex, etc.
    – jfs
    Commented Dec 29, 2013 at 5:26
  • I understand that there are workarounds, but if you just wanted to start 8 threads in "pure" Python and have them crunch ahead one on each core, possible coordinating progress through some locks you are out of luck...
    – renejsum
    Commented Dec 30, 2013 at 1:11
  • It's probably a bit late, but have a look at Pony language, it has an actor keyword, which means async class/object. Pony will start a thread for each core on CPU, potentially making the same program running on a 16 core CPU (almost) twice as fast as running on a 8-core. I wish I could do the same in Python, but the GIL prevent's it. (It does work in GIL'less Jython and IronPython though)
    – renejsum
    Commented Jan 11, 2018 at 6:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.