Going inside Cairo to add color management

Before going further you might want to read the previous blog post. Also, this:

I don't really have prior experience with color management, Cairo internals or the like. I did not even look at the existing patchsets for this. They are fairly old so they might have bitrotted and debugging that is not particularly fun. This is more of a "the fun is in the doing" kind of thing. What follows is just a description of things tried, I don't know if any of it would be feasible for real world use.

Basically this is an experiment. Sometimes experiments fail. That is totally fine.

Main goals

There are two things that I personally care about: creating fully color managed PDFs (in grayscale and CMYK) and making the image backend support images in colorspaces other than sRGB (or, more specifically, "uncalibrated RGB which most of the time is sRGB but sometimes isn't"). The first of these two is simpler as you don't need to actually do any graphics manipulations, just specify and serialize the color data out to the PDF file. Rendering it is the PDF viewer's job. So that's what we are going to focus on.

Color specification

Colors in Cairo are specified with the following struct:

struct _cairo_color {

    double red;

    double green;

    double blue;

    double alpha;

    unsigned short red_short;

    unsigned short green_short;

    unsigned short blue_short;

    unsigned short alpha_short;

};

As you can probably tell it is tied very tightly to the fact that internally everything works with (uncalibrated) RGB, the latter four elements are used for premultiplied alpha computations. It also has a depressingly amusing comment above it:

Fortunately this struct is never exposed to the end users. If it were it could never be changed without breaking backwards compatibility. Somehow we need to change this so that it supports other color models. This struct is used a lot throughout the code base so changing it has the potential to break many things. The minimally invasive change I could come up with was the following:

struct _comac_color {

    comac_colorspace_t colorspace;

    union {

        struct _comac_rgb_color rgb;

        struct _comac_gray_color gray;

        struct _comac_cmyk_color cmyk;

    } c;

};

The definition of rbg color is the original color struct. With this change every usage of this struct inside the code base becomes a compile error which is is exactly what we want. At every point the code is changed so that it first asserts that colorspace is RGB and then accesses the rgb part of the union. In theory this change should be a no-op and unless someone has done memcpy/memset magic, it is also a no-op in practice. After some debugging (surprisingly little, in fact) this change seemed to work just fine.

Color conversion

Ideally you'd want to use LittleCMS but making it a hard dependency seems a bit suspicious. There are also use cases where people would like to use other color management engines and even select between them at run time. So a callback functions it is:

typedef void (*comac_color_convert_cb) (comac_colorspace_t,
                                        const double *,
                                        comac_colorspace_t,
                                        double *,
                                        comac_rendering_intent_t,
                                        void *);

This only converts a single color element. A better version would probably need to take a count so it could do batch operations. I had to put this in the surface struct, since they are standalone objects that can be manipulated without a drawing context.

Generating a CMYK PDF stream is actually fairly simple for solid colors. There are only two color setting operators, one for stroking and one for nonstroking color (there may be others in e.g. gradient batch definitions, I did not do an exhaustive search). That code needs to be changed to convert the color to match the format of the output PDF and then serialize it out.

CMYK PDF output

With these changes creating simple CMYK PDF files becomes fairly straightforward. All you need to do as the end user is to specify color management details on surface creation:

comac_pdf_surface_create2 (

    "cmyktest.pdf",

    COMAC_COLORSPACE_CMYK,

    COMAC_RENDERING_INTENT_RELATIVE_COLORIMETRIC,

    comac_default_color_convert_func,

    NULL,

    595.28,

    841.89);

and then enjoy the CMYK goodness:

What next?

Probably handling images with ICC color profiles.

Experimenting on how to add CMYK and color management to Cairo

Cairo is an amazing piece of tech that powers a lot of stuff, like all of GTK. Unfortunately it is not without its problems. The biggest one being that it was designed almost 20 years ago with the main use case of dealing with "good old" 8 bit uncalibrated RGB images. There has been a lot of interest in adding native support for things like CMYK documents, linear RGB, color calibration, wide gamuts and all of that good stuff. Sadly it has not come to be.

The reasons are mostly the same as always. The project is sadly understaffed and there does not seem to be a corporate sponsor to really drive the development forward. What makes things extra difficult is that Cairo supports a lot of different platforms like Postscript, Win32, Quartz and SVG. So if someone wants to add new features in Cairo, not only do they need to understand how color math works and how to do C, they would also need to handle all the various backends. That is a rare combination of skills. In any case the patchset needed to make all that happen would be enormous and thus hard to get reviewed and merged.

As an exercise I thought I'd see if I could change the landscape somewhat to make it easier to experiment with the code base. Out of this came this project repo called Comac (for, obviously, COlor MAnaged Cairo). The creation process was fairly straightforward:

• Take the latest Cairo trunk
• Delete every backend except image and PDF
• Rename all files and symbols from cairo-something to comac-something
• Add minimal code for generating grayscale and CMYK PDFs
• Create a demo app that creates test PDFs

The output files are very simple, but the whole thing actually works. Both Okular and Acrobat Reader will happily display the documents without errors. Thus people who are interested in color work can now actually look into it without having to understand how Xlib works. Since Comac does not need to follow all of Cairo's backwards compatibility guarantees, experimenters can play a bit more fast & loose.

Want to contribute?

Merge requests welcome, I guess? I have not thought too deeply about governance and all that. If there is a lot of interest, this could be moved to its own top level project rather than living in my Github pages. Just note that the goal is not to fork things and create yet another graphics library. In the best possible case this project is only used to discover a good API and an idea on how it could best be implemented. These changes could then be put into Cairo core.

If you don't tolerate it in new code you should not tolerate it in old code either

Let's assume that you are working on a code base and notice that it has some minor issue. For argument's sake we'll say that it has some self written functionality and that the language's standard library has added identical functionality recently. Let's further assume that that said implementation behaves exactly the same as the self written one. At this point you might decide to clean up the code base, make it use the stdlib implementation and delete the custom code. This seems like a nice cleanup so you then file merge request to get the thing changed.

It might be accepted and merged without problems. On the other hand it might be blocked, either temporarily or permanently. There are several valid reasons for not merging the change. For example:

1. That part of the code does not have sufficient tests so we don't know if the change is safe.
2. A release is coming up soon, so we are minimizing all changes that might cause regressions, no matter how minor. The merge can only be done after the release is out.
3. We need to support old platform X, whose stdlib does not have that functionality.
4. It's not immediately obvious that the two implementations behave identically (for example, because the old implementation has load-bearing bugs) so a change might introduce bugs.

Getting blocked like this is a bit unfortunate, but these things happen. The important thing, however, is that all these are solid technical reason for not doing the cleanup (or at least not do it immediately). Things get bad when you get blocked by some other reason, such by your reviewer asking "why are you even doing this at all". This is a reasonable question, so let's examine it in detail.

Suppose that instead of submitting a cleanup commit you are instead submitting a piece of completely new functionality. In this MR you have chosen to reimplement a piece of standard library code (for no actual gain, just because you were not aware of its existence). The review comment that you should get is "You are reimplementing stdlib functionality, delete that code here and use the standard library instead". This is a valid review comment and something that should be heeded.

The weird thing here is that this is in a way the exact same change, but it is either not acceptable or absolutely necessary depending on whether parts of the code are already inside your repo or not. This is weird and should not be the case, but human beings are strangely irrational and their "value functions" are highly asymmetric. This can lead to lots of review fighting if one person really wants to fix the issue whereas some other one does not see the value in it. The only real solution is to have a policy on this, specifically that submitting a change that fixes an issue that would be unacceptable in new code is, by itself, a sufficient reason to do the work but not to merge it without technical review. This shifts the discussion from "should this be done at all" to "what are the technical risks and merits of this change", which is the way reviews should be done.

I should emphasize that this does not mean that you should go and immediately spend 100% of your time fixing all these existing issues in your code base. You could, but probably it is not worth it. What this guideline merely says that if you happen to come across an issue like this and if you feel like fixing it then you can do it and reviewers can't block you merely by "I personally don't like this" line of reasoning.

A nastier version

Suppose again you are a person who cares about the quality of your work. That you want to write code that is of sufficiently good quality and actually care about things like usability, code understandability and long term maintainability. For you people there exists a blocker comment that is much worse than the one above. In fact is the single biggest red flag for working conditions I have ever encountered:

But we already have [functionality X] and it works.

This question does have a grain of truth in it, existing code does have value. Sadly it is most commonly used as a convenient way to hardblock the change without needing to actually think about it or having to explain your reasoning.

Several times I've had to make some change into existing code and hours and days of debugging later found that the existing self written code has several bugs that a stdlib implementation definitely would not have. After reporting my findings and suggesting a cleanup the proposal has been knocked out with reasoning above. As a consultant I have to remain calm and respect the client's decision but the answer I have wanted to shout every time is: "No it's not! It's completely broken. I just spent [a large amount of time] fixing it because it was buggy and looking at the code makes me suspect that there are a ton of similar bugs in it. Your entire premise is wrong and thus every conclusion drawn from it is incorrect."

And, to reiterate, even this is not a reason by itself to actually go and change the code. It might get changed, it might not. The point is the prevailing attitude around minor fixups and how it matches your personal desire of getting things done. If you are a "fixing things proactively makes sense" kind of person you are probably not going to be happy in a "let's hide under the bed and pretend everything is fine" kind of organization and vice versa.

Making Visual Studio compilers directly runnable from any shell (yes, even plain cmd.exe)

The Visual Studio compiler toolchain behaves in peculiar ways One of the weirdest is that you can't run the compiler from any shell. Instead you have to run the compiler either from a special, blessed shell that comes with VS (the most common are "x86 native tools shell" and "x64 native tools shell", there are also ARM shells as well as cross compilation shells) or by running a special bat file inside a pristine shell that sets things up. A commonly held misbelief is that using the VS compiler only requires setting PATH correctly. That is not true, it requires a bunch of other stuff as well (I'm not sure if all of that is even documented).

To anyone who has used unixy toolchains, this is maddening. The classic Unix approach is to have compiler binaries with unique names like a hypothetical armhf-linux-gcc-11 from any shell. Sadly this VS setup has been the status quo for decades now and it is unlikely to change. In fact, some times ago I had a discussion with a person from Microsoft where I told them about this problem and the response I got back was, effectively: "I don't understand what the problem is" followed by "just run the compiles from the correct shell".

So why is this a bad state of things then? There are two major issues. The first one is that you have to remember how every one of your build trees has been set up. If you accidentally run a compilation command using the wrong shell, the outcome is very undefined. This is the sort of things that happens all the time because human beings are terrible at remembering the states of complicated systems and specific actions that need to be taken depending on their state (as opposed to computers, which are exceptionally good at those things). The second niggle is that you can't have two different compilers active in the same shell at the same time. So if, for example, you are cross compiling and you need to build and run a tool as part of that compilation (e.g. Protobuf) then you can't do that with the command line VS tools. Dunno if it can be with solution files either.

Rolling up them sleeves

The best possible solution would be for Microsoft to provide compiler binaries that are standalone and parallel runnable with unique names like cl-14-x64.exe. This seems unlikely to happen in the near future so the only remaining option is to create them ourselves. At first this might seem infeasible but the problem breaks neatly down into two pieces:

• Introspect all changes that the vsenv setup bat file performs on the system.
• Generate a simple executable that does the same setup and then invokes cl.exe with the same command line arguments as were given to it.

The code that implements this can be found in this repository. Most of it was swiped from Meson VS autoactivator. When you run the script in a VS dev tools shell (it needs access to VS to compile the exe) you get a cl-x64.exe that you can then use from any shell. Here we use it to compile itself for the second time:

Downsides

Process invocation on Windows is not particularly fast and with this approach every compiler invocation becomes two process invocations. I don't know enough about Windows to know whether one could avoid that with dlopen trickery or the like.

For actual use you'd probably need to generate these wrappers for VS linkers too.

You have to regenerate the wrapper binary every time VS updates (at least for major releases, not sure about minor ones).

The end results has not been tested apart from simple tests. It is a PoC after all.

Using cppfront with Meson

Recently Herb Sutter published cppfront, which is an attempt to create C++ a new syntax to fix many issues that can't be changed in existing C++ because of backwards compatibility. Like with the original cfront compiler, cppfront works by parsing the "new syntax" C++ and transpiling it to "classic" C++, which is then compiled in the usual way. These kinds of source generators are fairly common (it is basically how Protobuf et al work) so let's look at how to add support for this in Meson. We are also going to download and build the cppfront compiler transparently.

Building the compiler

The first thing we need to do is to add Meson build definitions for cppfront. It's basically this one file:

project('cppfront', 'cpp', default_options: ['cpp_std=c++20'])

cppfront = executable('cppfront', 'source/cppfront.cpp',
  override_options: ['optimization=2'])

meson.override_find_program('cppfront', cppfront)
cpp2_dep = declare_dependency(include_directories: 'include')

The compiler itself is in a single source file so building it is simple. The only thing to note is that we override settings so it is always built with optimizations enabled. This is acceptable for this particular case because the end result is not used for development, only consumption. The more important bits for integration purposes are the last two lines where we define that from now on whenever someone does a find_program('cppfront') Meson does not do a system lookup for the binary but instead returns the just-built executable object instead. Code generated by cppfront requires a small amount of helper functionality, which is provided as a header-only library. The last line defines a dependency object that carries this information (basically just the include directory).

Building the program

The actual program is just a helloworld. The Meson definition needed to build it is this:

project('cpp2hello', 'cpp',

    default_options: ['cpp_std=c++20'])

cpp2_dep = dependency('cpp2')

cppfront = find_program('cppfront')

g = generator(cppfront,

  output: '@BASENAME@.cpp',

  arguments: ['@INPUT@', '-o', '@OUTPUT@']

  )

sources = g.process('sampleprog.cpp2')

executable('sampleprog', sources,

   dependencies: [cpp2_dep])

That's a bit more code but still fairly straightforward. First we get the cppfront program and the corresponding dependency object. Then we create a generator that translates cpp2 files to cpp files, give it some input and compile the result.

Gluing it all together

Each one of these is its own isolated repo (available here and here respectively). The simple thing would have been to put both of these in the same repository but that is very inconvenient. Instead we want to write the compiler setup once and use it from any other project. Thus we need some way of telling our app repository where to get the compiler. This is achieved with a wrap file:

[wrap-git]
directory=cppfront
url=https://github.com/jpakkane/cppfront
revision=main

[provide]
cpp2 = cpp2_dep
program_names = cppfront

Placing this in the consuming project's subprojects directory is all it takes. When you start the build and try to look up either the dependency or the executable name, Meson will see that they are provided by the referenced repo and will clone, configure and build it automatically:

The Meson build system

Version: 0.63.99

Source dir: /home/jpakkane/src/cpp2meson

Build dir: /home/jpakkane/src/cpp2meson/build

Build type: native build

Project name: cpp2hello

Project version: undefined

C++ compiler for the host machine: ccache c++ (gcc 11.2.0 "c++ (Ubuntu 11.2.0-19ubuntu1) 11.2.0")

C++ linker for the host machine: c++ ld.bfd 2.38

Host machine cpu family: x86_64

Host machine cpu: x86_64

Found pkg-config: /usr/bin/pkg-config (0.29.2)

Found CMake: /usr/bin/cmake (3.22.1)

Run-time dependency cpp2 found: NO (tried pkgconfig and cmake)

Looking for a fallback subproject for the dependency cpp2

Executing subproject cppfront 

cppfront| Project name: cppfront

cppfront| Project version: undefined

cppfront| C++ compiler for the host machine: ccache c++ (gcc 11.2.0 "c++ (Ubuntu 11.2.0-19ubuntu1) 11.2.0")

cppfront| C++ linker for the host machine: c++ ld.bfd 2.38

cppfront| Build targets in project: 1

cppfront| Subproject cppfront finished.

Dependency cpp2 from subproject subprojects/cppfront found: YES undefined

Program cppfront found: YES (overridden)

Build targets in project: 2

As you can tell from the logs, Meson first tries to find the dependencies from the system and only after it fails does it try to download them from the net. (This behaviour can be altered.) Now the code can be built and the end result run:

$ build/sampleprog

Cpp2 compilation is working.

The code has only been tested with GCC but in theory it should work with Clang and VS too.

"Why is it that package managers are unnecessarily hard?" — or are they?

At the moment the top rated post in In the C++ subreddit is Why is it that package managers are unnecessarily hard?. The poster wants to create an application that uses fmt and SDL2. After writing a lengthy and complicated (for the task) build file, installing a package manager, integrating the two and then trying to build their code the end result fails leaving only incomprehensible error messages in its wake.

The poster is understandably frustrated about all this and asks a reasonable question about the state of package management. The obvious follow-up question, then, would be whether they need to be hard. Let's try to answer that by implementing the thing they were trying to do from absolute scratch using Meson. For extra challenge we'll do it on Windows to be entirely sure we are not using any external dependency providers.

Prerequisites

• A fresh Windows install with Visual Studio
• No vcpkg, Conan or any other third party package manager installed (more strictly, they can be installed, just ensure that they are not used)
• Meson installed so that you can run it just by typing meson from a VS dev tools command prompt (if you set it up so that you run python meson.py or meson.py, adjust the commands below accordingly)
• Ninja installed in the same way (you can also use the VS solution generator if you prefer in which case this is not needed)

The steps required

Create a subdirectory to hold source files.

Create a meson.build file in said dir with the following contents.

project('deptest', 'cpp',
    default_options: ['default_library=static',
                      'cpp_std=c++latest'])
fmt_dep = dependency('fmt')
sdl2_dep = dependency('sdl2')
executable('deptest', 'deptest.cpp',
   dependencies: [sdl2_dep, fmt_dep])

Create a deptest.cpp file in the same dir with the following contents:

#include<fmt/core.h>
#include<SDL.h>

int main(int, char**) {
    if (SDL_Init(SDL_INIT_VIDEO|SDL_INIT_AUDIO) != 0) {
        fmt::print("Unable to initialize SDL: {}", SDL_GetError());
        return 1;
    }
    SDL_version sdlver;
    SDL_GetVersion(&sdlver);
    fmt::print("Currently using SDL version {}.{}.{}.",
               sdlver.major, sdlver.minor, sdlver.patch);
    return 0;
}

Start a Visual Studio x64 dev tools command prompt, cd into the source directory and run the following commands.

mkdir subprojects
meson wrap install fmt
meson wrap install sdl2
meson build
ninja -C build

build\deptest

This is all you need to do to get the following output:

Currently using SDL version 2.24.0.

Most people would probably agree that this is not "unnecessarily hard". Some might even call it easy.

Diving deeper into custom PDF and epub generation

In a previous blog post I looked into converting a custom markup text format into "proper" PDF and epub documents. The format at the time was very simple and could not do even simple things like italic text. At the time it was ok, but as time went on it seemed a bit unsatisfactory.

Ergo, here is a sample input document:

# Demonstration document

This document is a collection of sample paragraphs that demonstrate

the different features available, like /italic text/, *bold text* and

even |Small Caps text|. All of Unicode is supported: ", », “, ”.

The previous paragraph was not indented as it is the first one following a section title. This one is indented. Immediately after this paragraph the input document will have a scene break token. It is not printed, but will cause vertical white space to be added. The

paragraph following this one will also not be indented.

#s

A new scene has now started. To finish things off, here is a

standalone code block:

```code

#include<something.h>

/* Cool stuff here */

```

This is "Markdown-like" but specifically not Markdown because novel typesetting has requirements that can't easily be retrofit in Markdown. When processed this will yield the following output:

Links to generated documents: PDF, epub. The code can be found on Github.

A look in the code

An old saying goes that that the natural data structure for any problem is an array and if it is not, then change the problem so that it is. This turned out very much to be the case in this problem. The document is an array of variants (section, paragraph, scene change etc). Text is an array of words (split at whitespace) which get processed into output, which is an array of formatted lines. Each line is an array of formatted words.

For computing the global chapter justification and final PDF it turned out that we need to be able to render each word in its final formatted form, and also hyphenated sub-forms, in isolation. This means that the elementary data structure is this:

struct EnrichedWord {
    std::string text;
    std::vector<HyphenPoint> hyphen_points;
    std::vector<FormattingChange> format;
    StyleStack start_style;
};

This is "pure data" and fully self-contained. The fields are obvious: text has the actual text in UTF-8. hyphen_points lists all points where the word can be hyphenated and how. For example if you split the word "monotonic" in the middle you'd need to add a hyphen to the output but if you split the hypothetical combination word "meta–avatar" in the middle you should not add a hyphen, because there is already an en-dash at the end. format contains all points within the word where styling changes (e.g. italic starts or ends). start_style is the only trickier one. It lists all styles (italic, bold, etc) that are "active" at the start of the word and the order in which they have been declared. Since formatting tags can't be nested, this is needed to compute and validate style changes within the word.

Given an array of these enriched words the code computes another array of all possible points where the text stream can be split, both within and between words. The output of this algorithm is then yet another array. It contains all the split points. With this the final output can be created fairly easily: each output line is the text between split points n and n+1.

The one major missing typographical feature missing is widow and orphan control. The code merely splits the page whenever it is full. Interestingly it turns out that doing this properly is done with the same algorithm as paragraph justification. The difference is that the penalty terms are things like "widow existence" and "adjacent page height imbalance".

But that, as they say, is another story. Which I have not written yet and might not do for a while because there are other fruit to fry.