16

I implemented an HTTP server in Go.

For each request, I need to create hundreds of objects for a particular struct, and I have ~10 structs like that. So after the request is finished as per Go implementation it will be garbage collected.

So for each request this much amount of memory will be allocated and deallocated.

Instead I wanted to implement memory pooling to improve performance from allocation side as well as GC side also

In the beginning of request, I will take from pool and put them back after the request is served

From the pool implementation side

  1. How to allocate and deallocate memory of a particular type of struct?
  2. How keep track of information this memory got assigned and other is not?

Any other suggestions to improve performance in case of memory allocation and deallocation?

2

2 Answers 2

34

Note beforehand:

Many suggest to use sync.Pool which is a fast, good implementation for temporary objects. But note that sync.Pool does not guarantee that pooled objects are retained. Quoting from its doc:

Any item stored in the Pool may be removed automatically at any time without notification. If the Pool holds the only reference when this happens, the item might be deallocated.

So if you don't want your objects in the Pool to get garbage collected (which depending on your case might result in more allocations), the solution presented below is better, as values in the channel's buffer are not garbage collected. If your objects are really that big that memory pool is justified, the overhead of the pool-channel will be amortized.

Moreover, sync.Pool does not allow you to limit the number of pooled objects, while the presented solution below naturally does.


The simplest memory pool "implementation" is a buffered channel.

Let's say you want a memory pool of some big objects. Create a buffered channel holding pointers to values of such expensive objects, and whenever you need one, receive one from the pool (channel). When you're done using it, put it back to the pool (send on the channel). To avoid accidentally losing the objects (e.g. in case of a panic), use defer statement when putting them back.

Let's use this as the type of our big objects:

type BigObject struct {
    Id        int
    Something string
}

Creating a pool is:

pool := make(chan *BigObject, 10)

The size of the pool is simply the size of the channel's buffer.

Filling the pool with pointers of expensive objects (this is optional, see notes at the end):

for i := 0; i < cap(pool); i++ {
    bo := &BigObject{Id: i}
    pool <- bo
}

Using the pool by many goroutines:

wg := sync.WaitGroup{}
for i := 0; i < 100; i++ {
    wg.Add(1)
    go func() {
        defer wg.Done()
        bo := <-pool
        defer func() { pool <- bo }()
        fmt.Println("Using", bo.Id)
        fmt.Println("Releasing", bo.Id)
    }()
}

wg.Wait()

Try it on the Go Playground.

Note that this implementation blocks if all the "pooled" objects are in use. If you don't want this, you may use select to force creating new objects if all are in use:

var bo *BigObject
select {
case bo = <-pool: // Try to get one from the pool
default: // All in use, create a new, temporary:
    bo = &BigObject{Id:-1}
}

And in this case you don't need to put it back into the pool. Or you may choose to try to put all back into the pool if there's room in the pool, without blocking, again with select:

select {
case pool <- bo: // Try to put back into the pool
default: // Pool is full, will be garbage collected
}

Notes:

Filling the pool prior is optional. If you use select to try to get / put back values from / to the pool, the pool may initially be empty.

You have to make sure you're not leaking information between requests, e.g. make sure you don't use fields and values in your shared objects that were set and belong to other requests.

11
  • 6
    Note this method is considerably slower than sync.Pool.
    – OneOfOne
    Jul 21, 2016 at 14:15
  • 8
    @OneOfOne Yes, but sync.Pool does not guarantee retention of pooled values. See edited answer.
    – icza
    Jul 21, 2016 at 14:31
  • How frequent GC in Golang will run? What are the factors does it depend on?
    – Tej
    Jul 22, 2016 at 3:27
  • 1
    @Raghu GC is a huge topic. You may check out the package doc of runtime which details env variables which control the GC. Also read the blog post: Go GC: Prioritizing low latency and simplicity
    – icza
    Jul 22, 2016 at 7:16
  • 1
    Also, sync.Pool doesn't provide any form of bound, say i want to pool just 5 resources and use them in different goroutines. Calling .Get() in all instances would mean it would keep creating new instances of the same object.
    – Subomi
    Jan 24, 2018 at 13:46
14

This is the sync.Pool implementation mentioned by @JimB. Mind the usage of defer to return object to pool.

package main

import "sync"

type Something struct {
    Name string
}

var pool = sync.Pool{
    New: func() interface{} {
        return &Something{}
    },
}

func main() {
    s := pool.Get().(*Something)
    defer pool.Put(s)
    s.Name = "hello"
    // use the object
}
1
  • 4
    This doesn't create a limit on object instance creation.
    – Subomi
    Jan 24, 2018 at 13:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.