Saturday, January 8, 2022

Portability is not sufficient for portability

A forenote

This blog post has some examples of questionable quality. This should not be meant as an attack on those projects. The issues listed here are fairly widespread, these are just the examples I ran into while doing other work.

What is meant by portability?

Before looking into portable software, let's first examine portability from a hardware perspective. When you ask most people what they consider a "portable computer", they'll probably think of laptops or possibly even a modern smartphone. But what about this:

This is most definitely a computer (the one I'm using to write this blog post, in fact), but not portable. It weighs something on the order of 10 kilos and it is too big to comfortably wrap your hands around.

And yet, I have carried this computer from one end of the Helsinki metropolitan region to another. It took over an hour on a train and a subway. When I finally got it home my arms were so exhausted that for a while I could not even lift them up and all muscles in them were sore for several days. I did use a helper carry strap, but it did not help much.

So in a way, yes, this computer is portable. It's not really designed for it, the actual transport process is a long painful slog and if you make an accidental misstep and bump it against a wall you run the risk of breaking everything inside. But it is a "portable computer" as a single person can carry it from one place to another using nothing but their own muscles.

Tying this to software portability

There is a lot of software out there that claims to be "portable" but can only be said to be that in the same way as the computer shown above is "portable". For the rest of the post we're only going to focus on portability to Windows.

Let's say a project has a parser that is built with Lex and Bison. Thus you need to have those programs during compilation. Building them from source is problematic on Windows (because of, among other things, Autotools) so it would be nice to get some prebuilt binaries for them. After a bit of googling you might find this page which provides Windows binaries. That has last been updated in 2004. So no.

You could also install Msys2 and get the binaries with Pacman. If you are using Visual Studio and just want to build the thing, installing a whole separate userspace system and package manager just to get two executables seems like bit of an overkill. Thinking about it further you might realize that you could install Msys on some other machine, get the executables, copy them and their direct dependency DLLs to your machine and put them in PATH. If you try this, the binaries segfault on run, probably because they can't access their localisation files that are "somewhere". 

Is this piece of software portable to Windows? Yes it is, in the "desktop PC is portable" sense, but definitely not in the "a laptop is portable" sense.

As an another example let's look at the ICU project. It claims to be highly portable, and it kind of is, here is a random snippet from their highly portable Makefile.

I don't know about you, but just looking at that … thing gives me a headache. If you ever need to do something the existing functionality does not provide, then trying to decipher what that thing is doing is an exercise in masochism. For Windows this is relevant, because Visual Studio only ships with nmake, which is not at all compatible with Make so you need to decrypt absolutely everything.

Again, this is portable to Windows, you just need to prebuild it with MinGW or using the provided VS solution files, copying the libraries from one place to another and using them. This is very much "desktop PC portable" again.

Sometimes you don't get even that. Take for example the liblangtag project. It is a fairly typical dependency library that provides a single shared library. It even hides its symbols and only exports those belonging to the public API. Sadly it does this using Libtool magic postprocessing. On Windows you have to annotate exported symbols with magic markers. Thus it is actually impossible to build a shared library properly on VS without making source code changes[1]. Thus you have to go the Mingw build route here. But that is again "portable" as in if you spend a ton of time and effort then you can sorta kinda make it work in a rubegoldbergesque way.

Being more specific

Due to various reason I have had to deal with the innards of libraries of different vintage. Fairly often the experience has been similar to dragging my desktop computer across town: arduous, miserable and exhausting. It is something I would wish upon my worst enemy and also upon most of my lesser enemies. It would serve them right.

In my personal opinion saying that some piece of code is portable should imply some basic ease of use. If you need to spend time fighting with it to make it work on an uncommon platform, toolchain or configuration then the thing is not really portable. It also blocks adoption, because if some library is a massive pain to use, people will prefer to reimplement the functionality or use some other library, just to get away from the pain.

Since changing the generally accepted meanings of words is unlikely to work, this won't happen. So in the mean time when you are talking about portability with someone else, do be sure to specify whether you mean "portable as in a desktop gaming PC" or "portable as in a laptop".

How this blog post came about

Some years ago I ported a sizable fraction of LibreOffice to build with Meson. It worked only on Linux as it used system dependencies. I rebased it to current trunk and tried to see if it could be built using nothing but Visual Studio by getting dependencies via the WrapDB. This repo contains the code, which now actually does build some code including dependencies like libxml, zlib and icu.

The code that is there is portable in the laptop sense. You only need to do a git checkout and start the build in a VS x64 dev tools prompt. It does cheat in some points, such as using pregenerated flex + bison sources, but it's not meant to be production quality, just an experiment.

[1] The project in question seems to have more preprocessor macro magic definitions than actual code, so it is possible there is some combination of defines that makes this work. If so, I did not manage to find it. This is typical in many old school C projects.

Friday, December 24, 2021

This year receive the gift of a free Meson manual

About two years ago, the Meson manual was published and made available for purchase. The sales were not particularly stellar and the bureaucracy needed to keep the sales channel going took a noticeable amount of time and effort. The same goes for keeping the book continually up to date.

Thus it came to pass that sales were shut down a year ago. At the time there were some questions on whether the book could be made freely available. This was not done, as it would not really have been fair to all the people who paid actual money to get it. So the book has been unavailable since.

However since an entire year has passed since then, the time has come. I'm making the full PDF manual available for personal use. You can download your own copy via this link. The contents have not been updated in more than a year, so it's not really up to date on details but the fundamentals are still valid.

Enjoy, and have a happy whatever-it-is-that-you-call-the-holiday-at-this-time-of-year.

The boring small print

Even though the book is freely downloadable it is not under any sort of an open license. You can download it and read it for personal use, but redistribution of any kind is not permitted.

Tuesday, November 9, 2021

Typesetting a whole book part III, the analog edition

In earlier editions (part 1, part 2) we looked at typesetting a full book to a PDF file. This is fun and all, but until you actually hold a physical copy in your hands you don't really know how good the end result is. Puddings, eatings and all that.

So I decided to examine how would you go about printing and binding an entire book. For text I used P. G. Wodehouse's The Inimitable Jeeves. It has roughly 220 pages which is a good amount for perfect binding. Typesetting it in LibreOffice only took a few hours. To make things even simpler I used only one font, the Palatino lookalike P052 that comes packaged with Ghostscript. As the Jeeves stories take place in the 1920s something like Century would have been more period accurate but we'll have to work with what we got.

The only printer I had access to was an A4 laser printer that could only print on one side of the page. Thus to keep things as simple as possible the page size became A5, which is easy to obtain by folding A4 paper in half. None of the printer dialogs seemed to do the imposition I needed (single page saddle fold, basically) so I had to convert the A5 originals to A4 printable sheets with a custom Python script (using PyPDF2)

Printing turned out to have its complications as it always does. The printer did not have Cups drivers so I had to print using Windows. I used Acrobat Reader as it should be the gold standard of PDF manipulation. The original documents were in A4 and the printer only had A4 paper capability. And yet, Acrobat insisted on compositing the pages to 8.5x11 and then printing them distorted because AMERICA! After going through every menu (especially the hidden ones) I managed to make it stop doing that. Then I printed the pages by first printing the odd pages, taking the output, adding it back in the paper tray and then printing the even pages. If you want to try the same, note that for the latter batch you need to print the pages in reverse order.

Do this in small batches. I did and was happy that I did, because every now and then I botched it up and had to redo the pages. But eventually I did prevail.

Then you fold each page in half and stack them to get the text block.

Fasten them together using clips and cut out a piece of cardboard for the cover.

Then glue the spine together, first the pages and then bookbinder's mull on top of that. This step is a bit involved, but Youtube has several videos that show you how to do it.

Then glue on the cover and place the whole thing into a press. Low tech solutions work surprisingly well. (Trivia: this box contains 25 Meson mugs.)

Leave like this overnight. In the morning the book is ready for trimming.

Sadly I don't have a heavy duty paper guillotine so the edges remained a bit rough. Other than that the end result looks pretty nice.

All in all this was a surprisingly simple and fun project. The binding process was so simple that even a craft-noob like myself could get it done without too much trouble. I did make a practice run with some empty pages, though. The clips were a bit fiddly, having a proper press or a woodworking bench would have made the end result smoother.

Monday, October 25, 2021

A call for more downstream testing of Meson

As Meson gets more and more popular, the number of regressions also grows. This is an unvoidable fact of life. To minimize this effort we publish release candidates before the actual releases. Unfortunately not many people use these so many issues are not found until after the release (as happened with 0.60.0).

For this reason we'd like to ask more people to test these rcs on their systems. It's fairly straightforward.

Testing individual projects

If you have a CI that installs Meson using pip, this is easy. You can tell Pip to use prerelease versions with the --pre flag.

pip install --pre meson

If you use prebuilt images rather than reinstalling on every build, do update your images once a week. Meson releases happen traditionally on Sunday evenings European time.

Testing if you are a distro or similar

The release candidates are packaged and uploaded to Debian experimental, so if you can use those, it is the simplest solution. They are not uploaded to unstable as I was instructed not to do so because of breakage potential. If you are a Debian person and know that the above explanation is incorrect and that I should be doing something else, let me know so I can change how that is done.

If you have some different setup that has a full CI run (hopefully something smaller than a full Debian archive rebuild) then doing that with the rc version would be the best test.

If you don't have such a test suite, you'll probably want to set one up for other reasons as well. :)

Monday, September 20, 2021

Glyphtracer 2.0

Ages ago I wrote a simple GUI app called Glyphtracer to simplify the task of creating fonts from scanned images. It seems people are still using it. The app is written in Python 2 and Qt 4, so getting it running becomes harder and harder as time goes by.

Thus I spent a few hours porting it to Python3 and PyQt 5 and bumped the major version to 2.0. The program can be obtained from this Github repo.

Saturday, September 11, 2021

Swappable faceplates for laptops

On the whole, laptops are boring. There are three different types.

  1. Apple aluminium laptops that all look the same.
  2. Other manufacturer's laptops that try to look like Apple laptops that look all the same.
  3. RGB led studded gamer laptops that all look the same.
There is a cottage industry that manufactures stickers that are the size of the entire laptop but those are inconvenient and not particularly slick. You only have one chance to apply and any misalignments or air bubbles are going to be there forever.

But what if some laptop manufacturer chose to do things differently and designed their laptops so that the entire face plate were detachable and replaceable? This would allow everybody to customize their own laptops in exactly the way they want to (this should naturally extend to the inner keyboard plate, but we'll ignore that for now). For example you could create a laptop with the exact Pantone color that you want rather than what the laptop manufacturer saw fit to give you.

Or maybe you'd like to have a laptop cover which lights up not the manufacturer's branding but instead the logo of your favourite band?

Or perhaps you are a fan of classic muscle cars?

The thing that separates these covers from what we currently have is that they would not be pictures of things or stickers. The racing stripe cover could be manufactured in exactly the same way as real car hoods. It would shine like real metal. It would feel different when touched. You could, one imagines, polish it with real car wax. It would be the real thing. Being able to create experimental things in isolation makes all sorts of experimentation possible so you could try holograms, laser engravings, embossings, unusual metals and all sorts of funky stuff easily. In case of failure the outcome is just a lost cover plate you can toss into the recycling bin rather than a very expensive laptop that is either unusably ugly or flat out broken.

But wait, there's more. Corporate PR departments should be delighted with this. Currently whenever people do presentations in conferences their laptops are clearly visible and advertise the manufacturer. The same goes for sales people visiting customers (well, eventually, once Covid 19 passes) and so on. Suppose you could have this instead:

Suddenly you have reclaimed a ton of prime advertising real estate that can be used for brand promotion (or something like that, not really a PR person so I'm not intimately familiar with the terminology). If you are a sufficiently large customer and order thousands of laptops at a time, it might be worth it to get all the custom plates fitted at the main factory during assembly. Providing this service would also increase the manufacturer's profit margin.

Tuesday, August 10, 2021

Hercule Poirot and the Mystery Box, as written by JJ Abrams

Outside, the scorching heat of the British summer could not have made a starker contrast with the icy cold atmosphere of the main living room of Lord Ellington's mansion. All fifteen people sitting around it were staring at the plump Belgian private inspector, who seemed to take great pleasure in waxing his mustache and letting the audience simmer in their own sweat. Finally he spoke.

"You may have wondered why I have called all you here. It has to do with the untimely death of Lady Sybill. We have reason to believe that the three dozen stab wounds in her back were not an accident but that she was in fact ..."

He paused for effect.

"... murdered."

Madame Smith shrieked in terror and dropped her tea cup, which shattered into a million trillion pieces, each of which glittered with the sparkle of a thousand suns. Which was strange, since the curtains were closed and there were no sources of direct light in the room.

"Exactly, mon frére" he said to her, even though his mother tongue was French and he should have known not to call a woman "my brother". And also to spell his sentences properly. But who cares about trivial details such as these when there is a mystery afoot?

"And furthermore, I'm happy to tell you all that our investigation on this has finally come to a conclusion."

Poirot took off his monocle, polished it and then put it back on.

"Oui, we can finally reveal that the identity of the murderer ... will remain unknown forever. Even with our plethora of clues, we could not determine what actually happened. Whoever the murderer was, they got away scott free as we are shutting down the investigation. Mon dieu, that makes this our seventy-sixth unsuccessful murder investigation in a row!"

The audience in the salon seemed perplexed, so captain Hastings chose to interject.

"It could also be space aliens!"

"My money is on an interdimensional time travel accident" countered Inspector Japp with the calm voice of an experienced police officer.

For a while, the room was enveloped in silence. Then the house was blown up by nazis.