Solving the C/C++ dependency problem through build system composability

The perennial problem in C and C++ development is dealing with dependencies. It is relatively straightforward on Linux where you have package managers (assuming your dependency is packaged in the distro you are using) but becomes a whole lot of work the second you step outside this zone. To develop native Windows applications, for example, the best solution people have come up with is copying the source code of dependencies to your own project and hoping for the best. Unfortunately wishful thinking is not a particularly efficient engineering method.

The main reason why dependency management is simple on Linux is a program called pkg-config. It is really simple, as most good solutions are, and basically just says that "to use dependency foo, use these flags to compile and these other flags to link". This makes all libraries that provide pkg-config definitions composable. That is, you can use them in a very simple way without knowing anything about their internals and you can mix and match them in any combination with any other library that provides a pkg-config definition.

It would be really nice if you could use pkg-config everywhere but unfortunately this is not possible. In order to do pkg-config really well, the support for it must come from the basic platform. That is, from the Visual Studios and XCodes out there. They have not provided such support at the moment and thus support is unlikely to appear in the near future. There are also some technical hurdles, especially on Windows with its multitude of incompatible binary ABIs.

Since we can't have pkg-config proper, is it possible to achieve the same result with a different kind of mechanism? It turns out that this is indeed possible and has been under development for quite a while in the Meson build system and its Wrap dependency system. The implementation details are slightly involved so for the purposes of this article the approach taken has been reduced to two main points.

The first one is that you can take any project that uses the Meson build system and then run it inside another Meson project. The subproject runs in a sandbox but the master project can use artifacts from it as if they were a native part of the master project. And yes, this can even be done recursively so subprojects can also use subprojects, but it goes even deeper than that. If two projects use the same subproject, the system guarantees that they both use the same subproject instance so it is compiled only once.

The second point is what you might call an "internal pkg-config". It encapsulates the same information as pkg-config proper, that is, what compiler flags to use and what libraries to link against.

With these two piece we have achieved the same level of composability as pkg-config proper without using pkg-config itself. We can now take our dependency projects and use them inside any other project on any platform because we do not depend on any piece of system infrastructure any more. We can choose to build and link all our dependencies statically for platforms such as iOS that require this. There is even a repository of Meson build definitions for open source libraries that you can easily use on your projects. There are not a whole lot of projects available yet, but anyone is free to submit more.

Talk is cheap, show me the code!

As a sample project we created a simple app using the SDL 2 library. It animates some images on the screen and plays sound effects when keys are pressed. It embeds all its images and sounds and statically links all its dependencies producing a single standalone exe file. Here is a screen shot of it running on OSX.

This project is available on github. The most interesting bit about it is probably the build definition file. The whole definition, which takes care of embedding SDL2, converting resources into C++ source, setting up all build flags for OSX, Windows, Linux etc, contains 33 lines of code, of which only three lines are dedicated to setting up the subproject (see lines 6-8).

If you were to build the same application with currently established methods, you would need to spend a fair bit of time downloading dependencies for each target platform, installing them, setting up build flags, debugging weird issues since you got something wrong and all that jazz. Using Meson you can just write a few declarations and let the build system deal with all the boring legwork including downloading and extracting all dependency projects.

That is the power of composability.