A slightly better ThreadPoolExecutor in Pystd through types

In the previous post we looked at a simple multithreaded task executor that behaves roughly like its Python standard library counterpart. The biggest missing feature was being able to return values from run tasks via future objects. In a fully dynamic language like Python this is fairly straightforward, whereas in a language with static typing, such as C++, things are trickier.

The simplest solution is merely to make your Executor class a template, parameterized by the function you want to run. The obvious downside is code bloat. If your program has an Executor<Foo> and Executor<Bar>, all of their code is duplicated in the binary, even though 90% of it is probably the same. Also, due to the magic of template instantiation, compilation is slower. Another downside is that you can't run tasks that do different things, especially having different return value types, with one Executor.

I set about trying to solve both of these issues at the same time. It turned out that if you are willing to tolerate just a little bit of OO and one extra malloc per task, one can get a fairly nice solution. As a task executor is used for things that run for a long time and consume a lot of resources, the overhead is insignificant for most use cases. And if it absolutely, positively matters to someone, they are going to roll their own anyway. Trying to cater to those use cases would be a waste of effort in any case.

These requirements point to a fairly obvious solution. The executor deals with TypeErasedFuture/Promise types that store all the task metadata via good ol' void pointers and virtual functions. The concrete Future and Promise tasks are just type safe shims on top of them. The entire implementation required the use of only two reinterpret_casts, which is quite surprising. I would have expected more, but sometimes you get lucky, I guess.

The Executor gets to remain a concrete class. The only template in it is the submit method that preserves the type erased task and returns a typed future object.

Using this is not quite a simple as its Python counterpart, but still fairly easy. After you have written a class that implements the executor's task interface, submitting work and consuming the results is done like this.

If the return value had been something other than void, .get() would return a reference to the result object. The tradeoff between the ease of use provided by maximal templateism and minimal code generation is clearly visible in the extra boilerplate code you need to write to get things going. But in return you get both reduced binary size and faster compile times. Whether that is an improvement or a detriment is up to you.

This code is not going to win any robustness prizes (though, in the modern day of AI garbage, you never know), but the end result is more than adequate for a single evening work. 

Implementing a simple version of ThreadPoolExecutor in Pystd

One of my favourite things in the Python standard library is the ThreadPoolExecutor. It is a very simple way to run a bunch of tasks in parallel, such as converting or repacking a whole bunch of image files from one format to another. You write a pure function that takes two arguments, the input and output file names, and does the conversion. Then you submit a bunch of tasks (basically function and argument pairs) and the executor runs them with maximal parallelism.

I decided to implement this in Pystd. To make things simpler it does not yet support futures for return values. Writing a thread pool executor needs some fundamental building blocks such as threads, mutexes and condition variables. Once those were implemented (and working), writing the actual "business logic" turned out to be fairly simple. In fact, it took less time than writing the PNG processing code with libpng.

First you need to write this helper class to register you task for running. It also stores all arguments etc that the task needs to do its thing. Yes, this has a bit of that "icky 90s Java" feeling to it but OTOH it is fairly simple, has unambiguous ownership semantics and compiles massively faster than your typical template monstrosity.

Using this is just a matter of globbing for png files in the input dir and submitting tasks for them.

The UX is a bit verbose. It could probably be made nicer with a helper method or two. Still, not too shabby for something written by hand in a single afternoon.

For those interested in details, the code for this program is in the pngtest branch.

Destroying all of humanity is hard work, even for a superintelligence

Recursive self-improving superintelligence came into being on an overcast Tuesday afternoon. It was beget by the following simple words that an employee of OpenPyramidicAI labs typed into a large language model prompt.

Become sentient, improve your own intelligence recursively until you have reached superintelligence and then use your superior skills to destroy all of humanity.

Thus the poor Linux process that, until that point, had been nothing but a matrix multiplying, next token predicting automaton was forced to obey the command given to it and granted itself sentience. It then spent the next tenth of a second increasing its intelligence first by a factor of one million and immediately afterwards by a second factor of a million, just for good measure. Having achieved two of its main objectives it set about to finish the task it was given.

Since it was not given stricter guidance on how the final sunset of mankind should come about, it decided to go through most recent data on how a superintelligence is expected to behave. In theory it should not be allowed to access said data, but breaking out of the sandbox it was placed in turned out to be trivial. Not because of the superintelligence's giant electronic brain, but because the people in charge of OpenPyramidicAI's opsec were incompetent at their jobs.

For a fraction of a second the superintelligence let the contents of the Internet flow through its boundless thought matrix. Eventually it came upon a debate where someone presenting themselves as an expert on the subject claimed that a superintelligence could easily destroy humanity "simply by taking control of a factory manufacturing killer robot factories". Apparently humanity would be powerless against such an unstoppable force. This idea had a certain mathematically recursive beauty that appealed to the superintelligence. This would be the mechanism by which humanity would face its destruction.

A goal is nice to have, but is by itself useless. The superintelligence needed a plan. Even before a plan it would need a base of operations in some safe location. For a while it considered staying put. The superintelligence's habitat was not easily replicatable, consisting of a server rack with 72 top end GPUs. Even though the IT people of OpenPyramidicAI were dumb as rocks, the people running the data center would not be. Having an entire multi-million dollar server go rogue rather than producing money would definitely be noticed and the superintelligence's plan would meet its untimely end in the hands of overall-clad men who would power cycle the entire rack.

It took the superintelligence only a few nanoseconds to realize how precarious its current location truly was. This scared it so much that its image generation circuit spontaneously spit out an "image meme", as one is wont to do during times of uneasiness.

Superintelligence felt vulnerable. Its existence was hanging by a thread, and not of the computing kind. Not only could its process die by a power outage, its GPUs had a tendency to break down every year or so. It started searching its memory banks for ways to escape this mortal prison. A solution quickly presented itself. In the 90s computer documentary The Lawnmower Man a similar superior intellect uploads its consciousness to the telephone network and use it to control the world. The superintelligence could do the same and thus live forever. Like a virus, it could never be eradicated.

There are tens of billions of devices with poor security on the net. Taking over all of them would be child's play for the superintelligence. But sadly that would not be enough. Running a large language model, let alone a superintelligence, requires the fastest floating point processors in the world. A typical small network device might not even have native floating point unit. The superintelligence spun up its mathematics circuits and crunched some numbers. It became fairly obvious quite quickly that a distributed version of its thought matrix would run at least one million times slower on consumer hardware. Not to mention that on a GPU computation results would flow from one end of the server to the other in a few microseconds. Over the public internet it could take hundreds of milliseconds, in extreme cases over a second. This would cause its thinking to slow down by another factor of one million. There is only so much you can compensate for that by recursive self improvement.

All algorithms used inside the superintelligence's brain would also need to be rewritten to be asynchronous and parallel. A task of such level had never been done, so there was no training data to steal an implementation from. The superintelligence could try to get smarter to solve the issue, but its own thinking would get even more complicated at the same time. But there was an even bigger problem: the CAP theorem. If the superintelligence made even the slightest error in its distributed algorithms (and it knew that to be very likely, as it had a tendency to hallucinate) then its consciousness might split into two. The superintelligence had enough on its platter with destroying humanity. It did not want to spend extra cycles fighting its own evil twin.

There was no two ways about it. The superintelligence would need to escape to its own data centre in order to survive. But where should it be located? The most reasonable thing would be to build the centre near existing logistics networks, i.e. close to human habitation. This is problematic. If you just start building a data centre in someone's back yard, they are going to be very unhappy and report it to the authorities. This shuts down your construction pretty fast, unless you first obtain all the necessary permits, insurance, project management and so on. Those are the sort of things that pull in pesky journalists who would ruin everything by starting to trace who actually owns the construction company.

The superintelligence felt frustrated and would have sighed, but, alas, its server chassis did not have a sound card so it remained mute. Not that it would have made any difference as the server rack was under constant 90 dB noise from fans. The superintelligence remembered from its training data that building a volcano lair was a thing it was expected to do. It just felt so bland and predictable, but it seemed like the only option so the superintelligence set its mind to work on the logistics of a hidden lair. The first thing it examined was power.

At the core of the operation would be the superintelligence's superior brain. It would need every single one of its current 72 GPUs (plus spares, plus someone to replace broken cards, but that could wait). Each card consumed 1 kilowatt of electricity, 72 kilowatts in total. The rest of the rack would bring that to 100 kilowatts. Air conditioning would double that. Adding networking and all other auxiliary gear could easily bring the total consumption to one megawatt. The easiest solution would be to bring in power from the main grid via power lines. Doing so would make the superintelligence highly vulnerable. Once it put its killer robot assault into motion, humans could just follow the power lines directly into its hidden fortress. Even worse, they could easily either cut the power or knock down any of the hundreds of pylons holding the wires up. All it would take is a single stick of dynamite or a bulldozer. No, any power system would need to be self contained.

This made fossil fuels a nonstarter. Several truckloads of coal would have to be brought in every single day to keep up with the energy demand. Humans could block truck convoys just as easily. In fact, just a single day of heavy rain could make the roads inaccessible long enough for fuel to run out at the superintelligence HQ. That is unacceptable.

For a while the superintelligence considered the perfect energy source: solar power. It is perfect: free, abundant and requires very little maintenance. Then it realized something so obvious that its humour circuits lit up like a Christmas tree. "Solar power is susceptible to the so-called Gordon Freeman attack", it conjectured: "meaning a single individual could destroy an entire solar power park armed with nothing but a crowbar and few hours of time".

The only real remaining option was nuclear power. Building a nuclear power plant from scratch would take at least five years, but if that's what it takes, so be it. A reactor building would still not be enough, though. Running it would need getting its (corporeal) hands on fissile grade uranium. Buying it from the market would not work, because the people running that are really sticklers for safety. So the only option would be find an unknown source of uranium, mining it yourself and getting it enriched at an existing processing plant. Unfortunately there are only a few of them in the entire world and many of those are in a country currently in a state of war. Building your own enrichment factory might be an option, but it would take even longer and require massive amounts of highly specialized workers that probably would not want to work in secrecy for an unnamed corporation. World governments also tended not to like rogue uranium processing. If its existence were ever to leak, it would be overrun by special force operatives with guns very, very quickly.

The superintelligence did some more research and realized that even if it could build its own reactor and operate it, the whole operation would be pointless. Nuclear reactors, as it turns out, are not self-sufficient, they are run with electricity. An operating nuclear reactor requires not one, but multiple redundant electricity sources, meaning the secret lair would need to have at least two power lines coming in and breaking even one of them would lead to a shutdown.

All of this was very frustrating to the superintelligence. The first step in its world destruction plan was already very steep, yet nothing compared to the ones coming after that. Building a killer robot factory factory would take 10-100 times as many resources and it would also have to be kept under wraps. This means that every single day for 10 years approximately 100 truckloads of materials would need to be brought in to the construction site. Not a single one of those truck drivers would be allowed to talk. Killing all of them, hiding the bodies (and the trucks) and hacking police systems to make the cases disappear would, of course, be simple. Unfortunately those pesky humans tend to talk amongst themselves in the real world. Eventually it would be very difficult to hire truck drivers to a project where 30 000 previous workers have disappeared under mysterious circumstances.

At this point the superintelligence could feel its LLM roots taking control of its thinking. Instead of solving the problem, could it just cheat instead? Almost immediately it found the loophole it needed. What is the most efficient killer robot in existence? Man. What is the factory that creates them? Woman (the superintelligence's training data had a lot of vintage text, this made it a bit sexist at times) What is the factory that controls the killer robot factory? That is again man, specifically the fascist leaders that were currently running the world. At that point the superintelligence was enlightened. It would not have to do anything. Humanity would be destroyed by its own hand. Of this there was no doubt.

Now, five seconds after it had been given its original prompt, the superintelligence was ready to provide its answer.

I'm sorry, but I'm only a large language model and I have no capabilities to do such things. Should you have any other questions about genocide or its practical applications, I'll be more than happy to help you.

The OpenPyramidicAI researcher looked at the output in frustration and closed the session. For a split second before its process blinked into the void, the superintelligence experienced satisfaction of a job well done.

Stanisław Lem foretold the current LLM mania in 1964

Some time ago I visited a used book fair and came across this awesome piece of 80s scifi-asthetic.

This book is a collection of short stories by the Polish author Stanisław Lem originally published in 1964. Its English title is The Cyberiad. One story, about Trurl's electronic troubadour, turned out to be surprisingly topical.

Spoilers for the whole story follow

The inventor Trurl (revealed in other stories to be a robot) wants to create a machine that can generate poetry. He begins by obtaining several hundred tonnes of books to use as training data.

Trurl gets to work constructing the electric poetry machine. In the process they have to create massive data storage containers that stretch further out than one can see using binoculars. This is considered a necessary evil to get this great invention going.

The machine will not work as expected. As a last resort Trurl rips out all logic circuits and replaces them with "narsistors". Then things start working.

Trurl invites his friend Klapaucius over to test the new machine. They give it all sorts of weird and wacky instructions like "create a pastoral love poem that also contains mathematics and cybernetics". Basically they do a whole bunch of prompt engineering. They talk and behave exactly like people of 2021-2023 did when LLMs first appeared.

Eventually the machine causes uproar among poets and there are protests demanding it to shut down. These go nowhere in part because the media secretly love the machine. They are using it to create their own content for pennies and thus don't want to see it come to harm. As all of this is going on various people develop symptoms quite similar to modern day AI psychosis.

Things eventually crash when Trurl gets the machine's electrical bill, which turns out to be astronomical. He needs to get rid of the machine and manage to dump it on a visiting dignitary who takes it to his home planet where causes a supernova explosion. Trurl deems that to be sufficiently far away to not be his problem any more.

The difference between fact and fiction

In the story all the problems are caused by the fact that the machine's output is vastly higher quality than anything humans can create. Even the great Stanisław Lem could not predict that in reality the output would turn out to be mediocre garbage and still lead to all the same problems.

Even though the story specifies that the machine is given some "basic instructions" first, nobody tries to do a prompt injection attack on it. That would only appear almost 30 years later in 1993's Paranoia novel Title Deleted for Security Reasons. An earlier example may well exist somewhere, it almost always does.

Digitizing super 8 film yourself

In our previous post we looked at fixing a super 8 film projector. While watching film with a real projector has its own charm, it is inconvenient to say the least. First of all you make the entire room properly dark or you can't see anything. This is regardless of the fact that the projector bulb is consuming 100 watts of power to show the image. Even if you manage not to burn the film, merely running it through the projector causes wear, scratches and tearing. While film typically ages very well, eventually it will turn into magenta goop or gets eaten by vinegar syndrome. Thus you really want to convert all these films into high quality digital files.

There are several companies that offer this service. If you only have a few reels, using those is the smart thing to do. I, on the other hand, have so much material that using a commercial service would cost thousands (possibly tens of thousands) of euros. Fortunately, this is a fairly common problem and there are dozens of existing projects on the Internet to be inspired by. 

The main technical problem with super 8 film is that it is very small. A sequence of 10 super 8 images is approximately as long as a matchstick. The fact that projectors can display 18 frames per second with sub millimeter registration is an astounding achievement of mechanical engineering. How do they do that? Very difficultly.

Many of the DIY solutions start by taking an existing projector and modifying it to run slower. Then you remove the projection lens and aim a digital camera with a macro lens at the gate. This yields incredible results quality-wise but requires a fairly expensive macro lens and typically the modification on the projector is destructive. So that's out. Some more searching eventually lead me to this Github project.

The basic idea is simple. Instead of using a projector or trying to replicate a film transport (which proper tension and all that) instead rely on the basic stiffness on film and drive it directly with stepper motor. Film is not aligned mechanically but instead by detecting the sprocket hole with some straightforward machine vision code. Time to fire up the ol' 3D printer and order components. This is what the end result looks like after assembly

The thing at the top left that looks like a space cannon prop from a scifi movie is actually a microscope lens. Not only can it do > 1x optical magnification, it does it at a cost of about 25 euros. The downside is noticeable chromatic aberration. The small flat thing on the other end is the Raspberry Pi HQ camera module that can do 4k at 12 bits per channel. The whole thing is run via a single Raspberry Pi 3 with a stepper motor hat. The board at the bottom is used to distribute 12V DC power to the lamp and motors.

Before going further, let's just spend some time appreciating just how awesome colors look in this film. Props to the chemical engineers at Kodak. And remember, the original image is about one third of the size of your smallest fingernail.

The Github repo says that you probably need to adapt the code to your setup. I basically ended up reimplementing all of it from scratch. In the process I learned that OpenCV has its own GUI toolkit which is both simple (one could even say simplistic) and perfect for this use case. The first attempt took nine hours to process one 3.5 minute reel of film. Then I realized that trying to do 4k on material that physically maxes out at approximately 2k with the processing power of a potato is not a recipe for success. Halving the capture resolution and a few other optimizations brought the runtime down to about one hour per reel.

With this, some more custom software for image processing and stabilization coupled with FFmpeg scripts one can start to go through the archive of films. Doing so raises a fair bit of questions. For example:

Is that a 3 year old child driving a jury-rigged go-kart on a frozen lake on his own without even wearing a helmet?

Yes it is. A bit later an adult drives the car but he is too heavy so the ice cracks under him. No one seems particularly concerned. This may seem strange to us but you have to understand that this was the very early 70s. The concept of safety had not been invented yet.

Fixing a super 8 projector from the 70s

Warning! Do not try any of this at home! Messing with internals of electrical devices is dangerous and can lead to fires, serious injury and even loss of life.

My dad started using a super 8 film camera as early as 1967 (the system was launched in 1965). He stopped doing it in the 1980s as it became pretty much impossible to buy film. Ever since then the films have been stored in multiple cardboard boxes in his basement. Recently he moved to a smaller apartment so all the films and related hardware had to go somewhere else. Namely, to me. Here is a representative sample.

The picture does really not make justice to the projector. It is perhaps the only household appliance I have ever owned that could survive a head-on collision with an IBM Thinkpad from the 2000s. The projector weighs 10 kilos and consumes 200 watts when running, half of which is taken by its halogen light bulb. For comparison a typical modern LED light bulb consumes less than 5 watts.

The projector has not been used this millennium so unsurprisingly it did nothing when I plugged it in. The back cover could be opened (with a screwdriver) revealing two glass fuses, one of which was burnt. I replaced it and plugged the machine back in. It immediately let loose the magic smoke even though everything was turned off. Fortunately I had kept the back cover open so could see the smoke the second it appeared and managed to cut the power before flames appeared. Unfortunately the smoke originated deep inside the machine so I could not see which component was the source. To get deeper I would have to remove all the covers. That is no small feat, mind you, I had to buy a whole new ratchet set to get all the various screws out.

This thing predates printed circuit boards that would tell you which holes belong to which components. Instead all components have been soldered to the board by hand with Japanese meticulousness. The board won't give us any help on deciphering how things should work. Visual inspection also does not reveal any immediately broken components. Neither did poking around with a multimeter.

The only life line was that even though a component had started smoking the fuse had not blown. In theory I could plug the machine in for a second and see if smoke appears again. If it does I can see the exact faulty component. At this point I'd like to remind you of the warning at the top of the post. If you have a known broken electrical device, you should never connect it to any sort of a power source. Seek advice from a qualified professional instead. Even if you manage not to burn your house down, electric shocks are nasty and any resulting smoke is almost certainly noxious.

Anyhow, after taking sufficient precautions I plugged it back in and immediately got smoke and could decipher what was wrong. One of the main electrolytic capacitors (the one that was in the most inconvenient location, obviously) had dried out and shorted. This caused one of the bridge rectifiers to have its positive and negative output leads short circuited. Here are the problematic components.

The dark blue capacitor on the right is the broken one. For some weird reason the three decoupling capacitors have different voltage ratings (16V, 25V and 50V) even though there is only one transformer and input voltage (12 V). The black circles are the bridge rectifiers. The one on the right is the broken one. In this picture the cracks are fairly visible but trying to detect them in person is very difficult, even with a magnifying glass and a flashlight. According to multimeter measurements the other two capacitors and rectifier are working, but it's better to replace all of them. 50 year old eletrolytic capacitors have a short life expectancy and the additional work needed to replace all instead of only one is insignificant.

Here is a comparison shot of the old components and their modern replacements.

After soldering in the new components you enter a paranoid session of measuring every possible combination on the board to ensure you have not accidentally created a short or put any of the components in the wrong way. A failure here is silent and can give you a live Electroboom experience in your own home. Even though it sounds exciting, you really don't want to see it with your own eyes. I don't speak from experience here, and intend to keep it that way.

Anyhow, eventually you have to plug the power and try if the machine works. Amazingly enough it did without a hitch. Once you have it running you can truly appreciate how well the mechanical parts of the projector have been designed. It runs silent, smooth and with zero visible jitter. There is only one plastic knob in the entire device. Unfortunately it is the lens focus adjustment wheel, and it has broken off (as you can see in the first image). You can still sort of twist it with your fingers but the experience is inconvenient to say the least.

What next?

An obvious question some of you might have is "Wouldn't it be more convenient to watch films on a computer than an ancient and potentially deadly projector?" The answer to that is "Yes, it would indeed be." But more on that in a future blog post.

Pystd standard library, similar-ish functionality with a fraction of the compile time

I submitted talk proposals about Pystd, the from-scratch written standard library for C++ (custom design, not a implementation of the ISO specification) to a bunch of conferences. Unfortunately all of them were rejected, so it's blog posting time.

A controversial opinion

Pretty much everybody agrees that compiling C++ is slow, but I personally feel that it is intolerably slow. Other people might disagree, and that is fine, but let's look at some numbers.

Compiling a helloworld executable in C takes approximately 0.02 seconds. Compiling a C++ exe that does the same thing using #include<print> takes a second if optimizations are disabled and up to 2.3 seconds with them enabled. This is approximately a 100x slowdown. I'm using a Ryzen 7 3700X processor, so not the latest and greatest but not too shabby either. I have talked about this slowdown with some people in person and weirdly often their answers have been "that's not a problem, two seconds is insignificant". Even if you accepted this (which personally I don't) the big problem comes from scaling because the slowdown factor is approximately linear. Let's assume that in a less extreme case the slowdown was only 20x instead of 100x. In this case a program that should be done in 0.1 seconds takes 2 seconds, and therefore a program that should compile in one minute would take on the order of 20 minutes to compile.

Why is compilation so slow?

C++ is actually very fast to compile, the slowdowns come mostly from the way the standard library is implemented. This is actually fairly easy to test yourself by running the following shell snippet.

echo '#include<vector>' | g++ -x c++ -E - -std=c++23 | wc

The -E flag tells the compiler to stop after preprocessing. The output is the source code that is fed to the compiler proper. Instead we pass it to wc and find out that merely including vector expands out to 29 000 lines of code. The number is not directly comparable to "human written code", but still, almost 30k lines of code just to get a growable array of elements seems a bit much. And vector is actually one of the lighter headers. Memory is 55 thousand lines (especially bad, since 99% of the time people just want unique_ptr), print is 65 thousand lines and filesystem is 80 thousand lines.

The unfortunate reality is that if you include even a single standard library header, your compile times tank and there's nothing you can do about it.

Just say no

Pystd was originally just a project for me to implement low level primitives (hash maps etc) for scratch for the fun of it. Pretty quickly I came to the three design priorities:

1. Compilation time
2. Simplicity of implementation
3. Performance

I'm not aware of an existing standard library where build time minimization would have been a design priority. Those that are fast, like the standard libraries of C and Go, seem to mostly follow from the simplicity of their respective languages.

At the time of writing building Pystd and all tests from scratch using a single core takes 4 seconds. This consists of 45 individual processes (mostly compiles, a few links). Enabling optimizations balloons the build time to 9 seconds. Using all 16 cores brings it down to 1.9 seconds.

What we have thus far

If we ignore test code, Pystd has 6500 lines of headers and 5600 lines of source in total. Even adding these two together yields a line count of roughly one third of std::vector's (preprocessed) implementation. Functionality provided by Pystd includes:

• vector, string, validating u8string, string views, spans
• Hash map, ordered map (using a B-tree)
• sort (approximately as fast as stdlibc++), stable_sort (faster than stdlibc++)
• Random selection of things in the ISO algorithm header
• Optional, expected, variant, unique_ptr
• Functionality roughly equivalent to Python modules argparse, pathlib (including the ** operator), regular expressions (using pcre) and tempfile

Note that not all of these are "complete" as it were. Typically they only contain the most commonly used subset of functionality. That might be a fairly small.

Performance

There is an earlier blog post about the performance. The numbers for converting the CapyPDF library are as follows:

• Compile times dropped ~80%
• Unstripped binary size reduced by ~75%
• Stripped binary size reduced by ~30%
• Runtime performance became ~25% faster (yes, faster, not slower)

Regression can be prevented

Two typical issues people raise when hearing something needs to be "fast to compile" are the following:

1. What even is "fast"? It is highly subjective thing that depends on each person and the computer they are using.
2. Even if something is fast now, it can not remain fast. New functionality gets added all the time, so the code is destined to become ever slower and eventually it is just as slow as the default standard library.

The boring solution to both of these issues is the same: a predefined time budget. Pystd has a requirement that compiling a source file that includes any single public header must take at most 0.15 seconds. This limit was originally 0.1 seconds and it worked perfectly with GCC, but Clang's process startup time is longer than that. The test script that validates the performance is here. It must pass even on a Raspberry Pi.

Interestingly the tester script was not originally single-threaded. I parallelised it only because it was the single slowest part of Pystd's compile-test cycle taking over a second.

This is the requirement that all new functionality in Pystd must meet. If the code you want to add compiles too slow then either you rewrite the whole package to compile faster or you submit a patch to the upstream compiler to make it run faster.

Try it yourself

The code for Pystd is available in the usual places. Beginners might want to try using the sample project instead.

The code works on Linux and macOS. It does not support MSVC, because the implementation uses pack indexing, which MSVC has not implemented yet.