Explain to me Go Concurrency Worker Pool Pattern like I’m five
When I first came into the Go language I was a bit reluctant to its syntax and verbosity. After a couple of months, I slowly began to fall in love with its simplicity, readability, performance superiority, and small memory footprints over other languages.
I especially had some particular interest in the language for its rich concurrency model. But despite the “new language honeymoon” stuff, I also struggled a bit trying to figure out how does this model work.
Because I was running behind a deadline, I went straight to the pattern implementation example instead of understanding the bases of concurrency first. Wrong!So I highly recommend you guys to go and have a look at the docs as a starting point. Even though Go handles concurrency with simplicity does not mean that programming concurrent code is that straightforward to grasp.
Nonetheless, I will do my best to be the more colloquial as possible here. I also provide the GitHub repo with an implementation of this pattern and tests for example usage.Having said that, the below diagram depicts the generalities of this pattern. A deeper and detailed description of what is happening behind the curtain is provided further below. (You can also check a bigger diagram version here. I made this on my own for you with so much love for better understanding!).
1. Jobs Batch
I created a minimal work unit called Job, composed of an ExecutionFn that would let us write custom logic for the Job to returning a Result. The latter could be either a value or an error.As the second step, I used the generator concurrency pattern to stream all the Jobs into the WorkerPool. What is this about? Generating a stream from ranging over some client’s defined Jobs slice pushing each of them into a channel, the Jobs channel. Which will be used to feed concurrently theAs the second step, I used the generator concurrency pattern to stream all the Jobs into the WorkerPool. What is this about? Generating a stream from ranging over some client’s defined Jobs slice pushing each of them into a channel, the Jobs channel. Which will be used to feed concurrently the
0 Comments