I'm trying to build an asynchronous codec. I have implemented a job dispatcher that has access to a buffered channel of jobs
var JobChannel chan Job = make(chan Job, 100000)
the dispatcher takes as input the number of workers and assigns work to them
func StartDispacher(numberOfWorkers int){
// start workers
wg := &sync.WaitGroup{}
wg.Add(numberOfWorkers)
for i := int(1); i <= numberOfWorkers; i++ {
go func(i int) {
defer wg.Done()
for j := range JobChannel {
doWork(i, j)
}
}(i)
}
}
my main function starts the dispatcher and keeps giving it jobs to do (in this case 200000 jobs)
workDispatcher.StartDispacher(2*runtime.NumCPU())
for i := 0; i < 200000; i++ {
j := workDispatcher.Job{
BytePacket: d,
JobType: workDispatcher.DECODE_JOB,
}
workDispatcher.JobChannel <- j
}
after experimenting: turns out there are 2 factors that affect the performance of this code
- the size of the buffered channel
JobChannel
- the number of workers there are
func StartDispacher(numberOfWorkers int)
Is there a standard way to find the optimal values for these parameters, and is it possible to make these values independent from the physical set-up of the machine running the code?