53

I have concurrent goroutines which want to append a (pointer to a) struct to the same slice. How do you write that in Go to make it concurrency-safe?

This would be my concurrency-unsafe code, using a wait group:

var wg sync.WaitGroup
MySlice = make([]*MyStruct)
for _, param := range params {
    wg.Add(1)
    go func(param string) {
        defer wg.Done()
        OneOfMyStructs := getMyStruct(param)
        MySlice = append(MySlice, &OneOfMyStructs)
    }(param)
}
wg.Wait()

I guess you would need to use go channels for concurrency-safety. Can anyone contribute with an example?

1

3 Answers 3

58

There is nothing wrong with guarding the MySlice = append(MySlice, &OneOfMyStructs) with a sync.Mutex. But of course you can have a result channel with buffer size len(params) all goroutines send their answers and once your work is finished you collect from this result channel.

If your params has a fixed size:

MySlice = make([]*MyStruct, len(params))
for i, param := range params {
    wg.Add(1)
    go func(i int, param string) {
         defer wg.Done()
         OneOfMyStructs := getMyStruct(param)
         MySlice[i] = &OneOfMyStructs
     }(i, param)
}

As all goroutines write to different memory this isn't racy.

10
  • 8
    It's very interesting your last consideration: in case the size of the slice is known and you are just dealing with pointers to the objects, you don't need to use a concurrency mechanism at all
    – Daniele B
    Commented Aug 28, 2013 at 23:23
  • 1
    This does not depend on "slice of pointers": It would work also for "slice of MyStruct". Again the code never writes to the same memory.
    – Volker
    Commented Aug 28, 2013 at 23:35
  • I was assuming that the memory allocation for a pointer is fixed, while the memory allocation for a struct is not fixed. I suppose I am wrong then.
    – Daniele B
    Commented Aug 29, 2013 at 1:10
  • Hu? What is "fixed"? Any type in Go has a certain memory layout which is determined completely a compile time. No difference between a pointer and something else.
    – Volker
    Commented Aug 29, 2013 at 4:59
  • I've read that you should usually not have buffering in channels, unless there's a specific need. For your suggestion about the goroutines writing to a shared channel, isn't that a case where you would just deliver synchronously (unbuffered)? Commented Feb 26, 2016 at 13:02
28

The answer posted by @jimt is not quite right, in that it misses the last value sent in the channel and the last defer wg.Done() is never called. The snippet below has the corrections.

https://play.golang.org/p/7N4sxD-Bai

package main

import "fmt"
import "sync"

type T int

func main() {
    var slice []T
    var wg sync.WaitGroup

    queue := make(chan T, 1)

    // Create our data and send it into the queue.
    wg.Add(100)
    for i := 0; i < 100; i++ {
        go func(i int) {
            // defer wg.Done()  <- will result in the last int to be missed in the receiving channel
            queue <- T(i)
        }(i)
    }

    go func() {
        // defer wg.Done() <- Never gets called since the 100 `Done()` calls are made above, resulting in the `Wait()` to continue on before this is executed
        for t := range queue {
            slice = append(slice, t)
            wg.Done()   // ** move the `Done()` call here
        }
    }()

    wg.Wait()

    // now prints off all 100 int values
    fmt.Println(slice)
}
3
  • 1
    Coming from the future: why does the @jimt solution not work? The wg.Done() is deferred, so it is only called after the value is sent through the channel. What am I missing?
    – ineiti
    Commented Mar 2, 2021 at 15:27
  • @chris Can I use an unbuffered channel? queue := make(chan T)
    – Pioz
    Commented Mar 16, 2021 at 15:47
  • using two different sync primitives is overkill, keeping only channel will do the job
    – valu
    Commented Aug 7, 2022 at 9:55
0

I wanted to add that since you know how many values you are expecting from the channel, you may not need to make use of any synchronization primitives. Just read from the channel as much data as you are expecting and leave it alone:

borrowing @chris' answer

package main

import "fmt"

type T int

func main() {
    var slice []T

    queue := make(chan T)

    // Create our data and send it into the queue.
    for i := 0; i < 100; i++ {
        go func(i int) {
            queue <- T(i)
        }(i)
    }

    for i := 0; i < 100; i++ {
        select {
        case t := <-queue:
            slice = append(slice, t)
        }
    }

    // now prints off all 100 int values
    fmt.Println(slice)
}

The select will block until the channels receives some data, so we can rely on this behaviour to just read from the channel 100 times before exiting.

In your case, you can just do:

package main

func main() {
    MySlice = []*MyStruct{}
    queue := make(chan *MyStruct)

    for _, param := range params {
        go func(param string) {
            OneOfMyStructs := getMyStruct(param)
            queue <- &OneOfMyStructs
        }(param)
    }

    for _ := range params {
        select {
        case OneOfMyStructs := <-queue:
            MySlice = append(MySlice, OneOfMyStructs)
        }
    }
}

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.