Sunday, February 23, 2020

Open source does not have a reward mechanism for tedious

A common talking point about everyday broken things is "I can't believe we put a man on the moon, but we still don't have <something seemingly straightforward>". The phrase is thankfully going out of style (maybe people are starting to forget that we did, in fact, put a man on the moon) but the sentiment still holds true. There are many things in our every day life that should be a lot better but aren't. This is also the case for open source development.

Since about the year 2000, when the term "open source" came into common usage, tens of thousands of software projects have sprung up, seemingly from thin air. The projects range from simple scripts to incredibly large and complex pieces of software. Some are purely volunteer run such as the Mame and Dolphin emulators whereas others have corporate support such as the Webkit browser engine. Yet, many seemingly simpler things are broken. Connecting projectors to laptops is still a bit of a gamble, touchpads behave wonkily almost any open source project's documentation is either thin or nonexisting, many dev tools break when faced with nonstandard setups and so on.

This does not seem to make sense. Writing a full HTML rendering engine from scratch is a lot harder than, say, writing documentation for the classes of a widget toolkit. Like most real world problems there are many causes and effects, but in this post we are going to examine the fact that the word "hard" has two different meanings. Things can be either be hard because they are difficult or they can be hard because they are tedious.

Many software developers are creators and builders. They are drawn to problems of the first type. The fact that they are difficult is not a downside, it is a challenge to be overcome. It can even be a badge of merit which you can wave around your fellow developers. These projects include things like writing your own operating system or 3D game engine, writing device drivers that saturate the fastest of transfer links, lock free atomic parallelism, distributed file systems that store exabytes of data as well as embedded firmware that has less than 1 kilobyte of RAM. Working on these kinds of problems is rewarding on its own, even if the actual product never finishes or fails horribly when eventually launched. They are, in a single word, sexy.

Most problems are not like that, but are instead the programming equivalent of ditch digging. They consist of a lot of hard work, which is not very exciting on its own but it still needs to be done. It is difficult to get volunteers to work on these kinds of problems and this is where the problem gets amplified in open source. Corporations have a very strong way to motivate people to work on tedious problems and it is called a paycheck. Volunteer driven open source development does not have a way to incentivise people in the same way. This is a shame, because the chances of success for any given software project (and startup) is directly proportional to the amount of tedious work people working on it are willing to do.

1 comment:

  1. The thing is, even the HTML rendering engine was not done that often.

    FIrefox is descended from Mozilla, that's descended from Netscape, which is one of the first rendering engines ever.

    Chrome is descended from WebKit, which is descended from KHTML, which is from 1999. (I am not sure if chrome nowadays has any original KHTML code, but who knows.)

    There aren't that many rendering engines around either, that's what I try to say.