62

How can I translate the following code from Java to Python?

AtomicInteger cont = new AtomicInteger(0);

int value = cont.getAndIncrement();
2
  • 2
    Are you actually using threading in your Python app?
    – Martijn Pieters
    May 8, 2014 at 16:43
  • 1
    Yes. I have a pool of 100 threads and I need to increment a variable in each one. May 8, 2014 at 18:39

6 Answers 6

52

Most likely with an threading.Lock around any usage of that value. There's no atomic modification in Python unless you use pypy (if you do, have a look at __pypy__.thread.atomic in stm version).

4
  • 2
    I use acquire() and release() of lock an it work, but I think that is not efficient like atomic class in Java. May 8, 2014 at 18:42
  • 22
    with your_lock: variable += 1 is probably shorter.
    – viraptor
    May 8, 2014 at 21:08
  • 4
    @viraptor, don't forget to take value from variable while the lock is still held Jun 17, 2016 at 12:33
  • 1
    FYI: suspending and resuming threads is expensive, hence the implementation of atomics in java, which do a repeated compare and swap operation instead of locking: baeldung.com/java-atomic-variables
    – lyjackal
    Apr 13, 2019 at 14:42
34

itertools.count returns an iterator which will perform the equivalent to getAndIncrement() on each iteration.

Example:

import itertools
cont = itertools.count()
value = next(cont)
10
  • 11
    At first glance this doesn't appear to be thread safe, is it?
    – Collin
    Nov 21, 2014 at 14:35
  • 33
    Ignoring for the moment my hesitation in using the ticking time-bomb that is relying on the GIL for synchronization, what makes you say that is definitively a single operation? It's a single function call, but is it a single bytecode operation?
    – Collin
    Nov 21, 2014 at 16:11
  • 23
    Yes Nov 24, 2014 at 11:29
  • 16
    @JianggeZhang I still agree with Colin's hesitation. In many contexts, it's not wise to rely on a contingency for synchronization: i.e., implicit details of how it's currently implemented that go beyond the "contract" and intent of the method. It's tantamount to relying on a side effect. For some contexts, this might be fine, but imo it's still iffy in general
    – Turix
    Jul 8, 2016 at 23:18
  • 24
    @WillManley it relies on an implementation detail of CPython (and PyPy), it's not a good practice to rely on implementation details. Moreover it would not be safe on Jython, see: jython.org/jythonbook/en/1.0/… and: bitbucket.org/jython/jython/src/… Please update the post. Jan 19, 2017 at 20:33
21

This will perform the same function, although its not lockless as the name 'AtomicInteger' would imply.

Note other methods are also not strictly lockless -- they rely on the GIL and are not portable between python interpreters.

class AtomicInteger():
    def __init__(self, value=0):
        self._value = int(value)
        self._lock = threading.Lock()
        
    def inc(self, d=1):
        with self._lock:
            self._value += int(d)
            return self._value

    def dec(self, d=1):
        return self.inc(-d)    

    @property
    def value(self):
        with self._lock:
            return self._value

    @value.setter
    def value(self, v):
        with self._lock:
            self._value = int(v)
            return self._value
7
  • Why lock in value()... isnt retrieval an atomic op by itself??
    – Tomer W
    Jun 7, 2020 at 12:30
  • That sounds right. Although, you’d also need to check that value remains an int throughout, and not promoted to a long, imaginary or some kind of non atomic object. (E.g, what would happen if you supplied MyLoggingInt to the constructor)
    – user48956
    Jun 7, 2020 at 14:36
  • Isnt all those just Read reference? You eitger get the old value, or the newer. (If not, python is really messed up)... is it?
    – Tomer W
    Jun 8, 2020 at 5:49
  • Actually, I'm not sure you can assume that. Is self.x. atomic. It certainly not always atomic (beceause __getattr__, __getattribute__ may be implemented in subclasses. (Also, I've added int(x) to protect cases where the input is not an immutable, threadsafe object).
    – user48956
    Nov 6, 2020 at 19:59
  • 1
    @user48956 After another test, I found out it's working.. it was merely the platform I'm working on sluggish. +1 from me!
    – Yahya
    Sep 8, 2022 at 16:43
15

Using the atomics library, the same could would be written in Python as:

import atomics


a = atomics.atomic(width=4, atype=atomics.INT)

value = a.fetch_inc()

This method is strictly lock-free.

Note: I am the author of this library

9
  • It looks like you are recommending your own library. Please be aware that in case of self-promotion, "you must disclose your affiliation in your answers". Nov 10, 2021 at 17:03
  • My apologies, I'll do that now (including other comments)
    – doodspav
    Nov 10, 2021 at 17:06
  • 1
    No worries, just wanted to let you know. Nice work, by the way. Nov 10, 2021 at 17:06
  • Hi. I am just a passerby here, and luckly just found your answer. It is great that it was posted just a few hours ago. So, let me ask you: What is width=4? It was not clear for me by reading the docs. Also, why should I care about the width at all? Further, the docs should include a GitHub link for contributing and also present simple cases understandable by newbies like the one in this answer instead of jumping right to complicated stuff like testing its correctness in multiple threads with heavy concurrency or memory mapping access. Nov 11, 2021 at 8:12
  • 1
    Anyway, congratulations for providing such library. Nov 11, 2021 at 8:17
1

8 years and still no full example code for the threading.Lock option without using any external library... Here it comes:

import threading

i = 0
lock = threading.Lock()

# Worker thread for increasing count
class CounterThread(threading.Thread):
    def __init__(self):
        super(CounterThread, self).__init__()
        
    def run(self):
        lock.acquire()
        global i
        i = i + 1
        lock.release()


threads = []
for a in range(0, 10000):
    th = CounterThread()
    th.start()
    threads.append(th)

for thread in threads:
    thread.join()

global i
print(i)
1
0

Python atomic for shared data types.

https://sharedatomic.top

The module can be used for atomic operations under multiple processs and multiple threads conditions. High performance python! High concurrency, High performance!

atomic api Example with multiprocessing and multiple threads:

You need the following steps to utilize the module:

  1. create function used by child processes, refer to UIntAPIs, IntAPIs, BytearrayAPIs, StringAPIs, SetAPIs, ListAPIs, in each process, you can create multiple threads.

     def process_run(a):
       def subthread_run(a):
         a.array_sub_and_fetch(b'\x0F')
    
       threadlist = []
       for t in range(5000):
           threadlist.append(Thread(target=subthread_run, args=(a,)))
    
       for t in range(5000):
           threadlist[t].start()
    
       for t in range(5000):
           threadlist[t].join()
    
  2. create the shared bytearray

    a = atomic_bytearray(b'ab', length=7, paddingdirection='r', paddingbytes=b'012', mode='m')
    
  3. start processes / threads to utilize the shared bytearray

     processlist = []
    
     for p in range(2):
    
       processlist.append(Process(target=process_run, args=(a,)))
    
     for p in range(2):
    
       processlist[p].start()
    
     for p in range(2):
    
       processlist[p].join()
    
     assert a.value == int.to_bytes(27411031864108609, length=8, byteorder='big')
    

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.