I have a slice of integers, which are manipulated concurrently:

ints := []int{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}

I'm using a buffered channel as semaphore in order to have a an upper bound of concurrently running go routines:

sem := make(chan struct{}, 2)

for _, i := range ints {
  // acquire semaphore
  sem <- struct{}{}

  // start long running go routine
  go func(id int, sem chan struct{}) {
    // do something

    // release semaphore
    <- sem
  }(i, sem)
}

The code above works pretty well until the last or last two integers are reached, because the program ends before those last go routines are finished.

Question: how do I wait for the buffered channel to drain?

  • You gotta use a mutex or something. The buffered channel blocks when it's full, but there is no language feature to block until it's empty. – evanmcdonnal Sep 29 '16 at 17:29
up vote 9 down vote accepted
+100

You can't use a semaphore (channel in this case) in that manner. There's no guarantee it won't be empty any point while you are processing values and dispatching more goroutines. That's not a concern in this case specifically since you're dispatching work synchronously, but because there's no race-free way to check a channel's length, there's no primitive to wait for a channel's length to reach 0.

Use a sync.WaitGroup to wait for all goroutines to complete

sem := make(chan struct{}, 2)

var wg sync.WaitGroup

for _, i := range ints {
    wg.Add(1)
    // acquire semaphore
    sem <- struct{}{}
    // start long running go routine
    go func(id int) {
        defer wg.Done()
        // do something
        // release semaphore
        <-sem
    }(i)
}

wg.Wait()
  • 1
    Thanks, I also thought about using a WaitGroup. This feels like the right way! – Kiril Sep 29 '16 at 17:36

Use "worker pool" to process you data. It is cheeper than run goroutine for each int, allocate memory for variables inside it and so on...

ints := []int{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}

ch := make(chan int)

var wg sync.WaitGroup

// run worker pool
for i := 2; i > 0; i-- {
    wg.Add(1)

    go func() {
        defer wg.Done()

        for id := range ch {
            // do something
            fmt.Println(id)
        }
    }()
}

// send ints to workers
for _, i := range ints {
    ch <- i
}

close(ch)

wg.Wait()
  • what if "ints" is reading from a file instead of a slice? – Jau L May 14 at 1:55

Clearly there is no one waiting for your go-routines to complete. Thus the program ends before the last 2 go-routines are completed. You may use a workgroup to wait for all your go-routines complete before the program ends. This tells it better - https://nathanleclaire.com/blog/2014/02/15/how-to-wait-for-all-goroutines-to-finish-executing-before-continuing/

  • Thank you, I was looking for how to avoid that, and a WorkGroup works perfectly. JimB's answers clarified this – Kiril Sep 29 '16 at 17:37

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.