In a previous post I wrote about simple job queue in Redis. In this post I like to write a bit about the design choices. I think these choices illustrate a point about design.

Job handle

The job handle in the code is create by the client (with help from the Redis server). This job handle is passed to the queue in Redis and picked up on the other side by the worker.

The worker doesn’t need to know the content of the key. The key is opaque. This means that the key could be changed to be something else. In the current case the handle is the build up out of the queue name and a unique id.

The worker only needs to know which commands it can use with that job handle. In this case the commands are HGETALL and DEL. When the job handle changes, because it uses a hash type value, then the workers don’t need to change.

After the worker got the hash value with the job from Redis, it can perform it’s job. The value is specific to the worker and this shouldn’t change. In the content of the value uses the keys and value types then the workers keep working. Upgrading the content can be done separately for the client and the worker, if you keep the same keys in the job and only add keys. The workers can get upgraded later with the newer commands. This could also be done the other way around as long as the new worker understands the old job data.

The keys here are similar to URI in HTTP (and REST). The value of the URI is opaque. As a client you shouldn’t create URI yourself, but you should the follow the URI from the server according to its media type.

Incrementing the id of the job

Handle of the job consists of the name of the queue and a unique id. We get this id from Redis using the INCR command. The command returns the current value + 1. We use this exact value in the handle. In a way we use the value in pre increment way. We increment and then use the value. This means that we can use the exact value that we got back from Redis.

Another way is to say that the value in Redis is id of the next job. By incrementing the value we say that we used this value. The only problem is that the returned value is one higher than the value we would like to use. So we need to decrement the returned value with 1.

The design choice here was to use the exact value from Redis. In a way this is the same as the job handle example. There the worker uses the exact value that it got. This choice makes it easier, because there is no calculation needed

Redis is a flexible solution to a lot of problems, but as always you need to make choices to get a solution the works for your problem.