How much effort would it take to convert OpenSSL's Perl source code generators to Python?

There is an ongoing discussion to write Meson build definitions to OpenSSL so it can be added to the WrapDB and built transparently as a subproject. One major issue is that OpenSSL generates a lot of assembly during build time with Perl. Having a Perl dependency would be bad, but shipping pregenerated source files would also be bad. Having "some pregenerated asm" that comes from "somewhere" would understandably be bad in a crypto library.

The obvious third option would be to convert the generator script from Perl to Python. This is not the first time this has been proposed and the counterargument has always been that said conversion would take an unreasonable amount of effort and could never be done. Since nobody has tried to do the conversion we don't really know whether that claim is accurate or not. I converted the x86_64 AES asm generator to see how much work it would actually take. The code is here.

The code setup

The asm generator has two different levels of generators. First a Perl script generates asm in a specific format and after that a second Perl script converts it to a different asm type if needed (Nasm, At&T, Intel etc). The first script can be found here and the second one is here. In this test only the first script was converted.

At first sight the task seems formidable. The generator script is 2927 lines of code. Interestingly the asm file it outputs is only 2679 lines. Thus the script is not only larger than its output, it is also less understandable than the thing it generates both because it is mishmash of text processing operations and because it is written in Perl.

Once you get over the original hump, things do get easier. Basically you have a lot of strings with statements that look like this:

movzb `&lo("$s2")`,$acc0

This means that the string needs to be evaluated so that $s2 and $acc0 are replaced with the values of variables s2 and acc0. The backticks mean that you have to then call the lo function with the given value and substitute that in the output string. This is very perlistic and until recently would not have been straightforward to do in Python. Fortunately now we have f-strings so that becomes simply:

f'movzb {lo(s2)},{acc0}'

With that worked out the actual problem is no longer hard, only laborious. First you replace all the Perl multiline strings with Python equivalents, then change the function declarations from something like this:

sub enctransform_ref()
{ my $sn = shift;

to this:

def enctransform_ref(sn):

and then it's a matter of repeated search-and-replaces for all the replacement variables.

The absolute best thing about this is that it is almost trivial to verify that the conversion has been done without errors. First you run the original script and store the generated assembly. Then you run your own script and compare the output. If they are not identical then you know exactly where the bug is. It's like having a unit tests for every print statement for the program.

How long did it take?

From scratch the conversion took less than a day. Once you know how it's done a similar conversion would take maybe a few hours. The asm type converter script seems more complicated so would probably take longer.

A reasonable port would contain these conversions for the most popular algorithms to the most popular CPU architectures (x86, x86_64, arm, aarch64). It would require a notable amount of work but it should be measured in days rather than months or years. I did browse through some of the other asm files and it seems that they have generators that work in quite different ways. Converting them might take more or less work, but probably it would still be within an order of magnitude.