A slightly better ThreadPoolExecutor in Pystd through types

In the previous post we looked at a simple multithreaded task executor that behaves roughly like its Python standard library counterpart. The biggest missing feature was being able to return values from run tasks via future objects. In a fully dynamic language like Python this is fairly straightforward, whereas in a language with static typing, such as C++, things are trickier.

The simplest solution is merely to make your Executor class a template, parameterized by the function you want to run. The obvious downside is code bloat. If your program has an Executor<Foo> and Executor<Bar>, all of their code is duplicated in the binary, even though 90% of it is probably the same. Also, due to the magic of template instantiation, compilation is slower. Another downside is that you can't run tasks that do different things, especially having different return value types, with one Executor.

I set about trying to solve both of these issues at the same time. It turned out that if you are willing to tolerate just a little bit of OO and one extra malloc per task, one can get a fairly nice solution. As a task executor is used for things that run for a long time and consume a lot of resources, the overhead is insignificant for most use cases. And if it absolutely, positively matters to someone, they are going to roll their own anyway. Trying to cater to those use cases would be a waste of effort in any case.

These requirements point to a fairly obvious solution. The executor deals with TypeErasedFuture/Promise types that store all the task metadata via good ol' void pointers and virtual functions. The concrete Future and Promise tasks are just type safe shims on top of them. The entire implementation required the use of only two reinterpret_casts, which is quite surprising. I would have expected more, but sometimes you get lucky, I guess.

The Executor gets to remain a concrete class. The only template in it is the submit method that preserves the type erased task and returns a typed future object.

Using this is not quite a simple as its Python counterpart, but still fairly easy. After you have written a class that implements the executor's task interface, submitting work and consuming the results is done like this.

If the return value had been something other than void, .get() would return a reference to the result object. The tradeoff between the ease of use provided by maximal templateism and minimal code generation is clearly visible in the extra boilerplate code you need to write to get things going. But in return you get both reduced binary size and faster compile times. Whether that is an improvement or a detriment is up to you.

This code is not going to win any robustness prizes (though, in the modern day of AI garbage, you never know), but the end result is more than adequate for a single evening work.