22

If I have a single int which I want to write to from one thread and read from on another, I need to use std::atomic, to ensure that its value is consistent across cores, regardless of whether or not the instructions that read from and write to it are conceptually atomic. If I don't, it may be that the reading core has an old value in its cache, and will not see the new value. This makes sense to me.

If I have some complex data type that cannot be read/written to atomically, I need to guard access to it using some synchronisation primitive, such as std::mutex. This will prevent the object getting into (or being read from) an inconsistent state. This makes sense to me.

What doesn't make sense to me is how mutexes help with the caching problem that atomics solve. They seem to exist solely to prevent concurrent access to some resource, but not to propagate any values contained within that resource to other cores' caches. Is there some part of their semantics I've missed which deals with this?

3
  • I don't get what you don't get. Atomics have the semantics [lock global mutex; do op; unlock global mutex]. Just some have intrinsic support so done implicitly and fast.
    – Balog Pal
    Jun 11, 2013 at 14:35
  • Just FYI - std::atomic works with all data types. Its amusingly just not lock free if its too complicated - it falls back to locks in that case. See the member function 'is_lock_free()'. However, you still have to be careful if you do this instead of locks.
    – Mike Vine
    Jun 11, 2013 at 14:55
  • 1
    Balog, I understand the semantics of mutexes, and I understand how atomics do what they do, and I understand that in C++11 they officially have extra semantics that prevent different cores keeping old values in cache. What I don't understand is how mutexes do that. They prevent concurrent access to a thing, but I've not read anything that states that thing will have a consistent value in the caches of different cores.
    – Ben Hymers
    Jun 14, 2013 at 10:37

3 Answers 3

6

The right answer to this is magic pixies - e.g. It Just Works. The implementation of std::atomic for each platform must do the right thing.

The right thing is a combination of 3 parts.

Firstly, the compiler needs to know that it can't move instructions across boundaries [in fact it can in some cases, but assume that it doesn't].

Secondly, the cache/memory subsystem needs to know - this is generally done using memory barriers, although x86/x64 generally have such strong memory guarantees that this isn't necessary in the vast majority of cases (which is a big shame as its nice for wrong code to actually go wrong).

Finally the CPU needs to know it cannot reorder instructions. Modern CPUs are massively aggressive at reordering operations and making sure in the single threaded case that this is unnoticeable. They may need more hints that this cannot happen in certain places.

For most CPUs part 2 and 3 come down to the same thing - a memory barrier implies both. Part 1 is totally inside the compiler, and is down to the compiler writers to get right.

See Herb Sutters talk 'Atomic Weapons' for a lot more interesting info.

3
  • This seems like the most complete answer, thanks! Unfortunately it now means I need to ask the follow-up question "how do memory barriers enforce consistency" since I don't see how preventing reordering can ensure CPUs/cores make sure the values they have cached are synchronised... do they just flush everything when they encounter a barrier? Perhaps just things accessed a few instructions around the barrier? I imagine the answer again is "magic pixies" :)
    – Ben Hymers
    Jun 12, 2013 at 11:15
  • Also, hello Mike! I think we were at Rare together for a little while after I joined!
    – Ben Hymers
    Jun 12, 2013 at 11:17
  • @BenHymers Caches synchronize all the time. There is no special instruction to keep caches in sync. It's done automatically and at all times.
    – curiousguy
    Jun 25, 2013 at 2:41
6

The consistency across cores is ensured by memory barriers (which also prevents instructions reordering). When you use std::atomic, not only do you access the data atomically, but the compiler (and library) also insert the relevant memory barriers.

Mutexes work the same way: the mutex implementations (eg. pthreads or WinAPI or what not) internally also insert memory barriers.

5
  • 1
    What exactly is a memory barrier? How does it work? Or should that be a separate question?
    – SirGuy
    Jun 11, 2013 at 14:30
  • 1
    That should be a Google search ;) Jun 11, 2013 at 14:33
  • 3
    @GuyGreer: In my opinion it should be a separate question, but I'm not even sure it is fit for SO since the subject is so broad (typical "we need a whole book to answer that"). An online search is indeed a good start to get the basics, then precise questions can be asked.
    – syam
    Jun 11, 2013 at 14:34
  • @syam Fair enough, I think the link that was added in the answer is a good compromise
    – SirGuy
    Jun 11, 2013 at 14:46
  • @GuyGreer: Yes, I was trying to write a short explanation but it was no good. Drew adding the Wikipedia link is much better than what I could have come up with.
    – syam
    Jun 11, 2013 at 14:48
5

Most modern multicore processors (including x86 and x64) are cache coherent. If two cores hold the same memory location in cache and one of them updates the value, the change is automatically propagated to other cores' caches. It's inefficient (writing to the same cache line at the same time from two cores is really slow) but without cache coherence it would be very difficult to write multithreaded software.

And like syam said, memory barriers are also required. They prevent the compiler or processor from reordering memory accesses, and also force the write into memory (or at least into cache), when for example a variable is held in a register because of compiler optizations.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.