Measuring code size and performance

Are exceptions faster and/or bloatier than using error codes? Well...

The traditional wisdom is that exceptions are faster when not taken, slower when taken and lead to more bloated code. On the other hand there are cases where using exceptions makes code a lot smaller. In embedded development, even, where code size is often the limiting factor.

Artificial benchmarks aside, measuring the effect on real world code is fairly difficult. Basically you'd need to implement the exact same, nontrivial piece of code twice. One implementation would use exceptions, the other would use error codes but they should be otherwise identical. No-one is going to do that for fun or even idle curiosity.

CapyPDF has been written exclusively using C++ 23's new expected object for error handling. As every Go programmer knows, typing error checks over and over again is super annoying. Very early on I wrote macros to autopropagate errors. That props up an interesting question, namely could you commit horrible macro crimes to make the error handling either use error objects or exceptions?

It tuns out that yes you can. After a thorough scrubbing of the ensuring shame from your body and soul you can start doing measurements. To get started I built and ran CapyPDF's benchmark application with the following option combinations:

• Optimization -O1, -O2, -O3, -Os
• LTO enabled and disabled
• Exceptions enabled and disabled
• RTTI enabled and disabled
• NDEBUG enabled and disabled

The measurements are the stripped size of the resulting shared library and runtime of the test executable. The code and full measurement data can be found in this repo. The code size breakdown looks like this:

Performance goes like this:

Some interesting things to note:

• The fastest runtime of 0.92 seconds with O3-lto-rtti-noexc-ndbg
• The slowest is 1.2s with Os-noltto-rtti-noexc-ndbg
• If we ignore Os the slowes is 1.07s O1-noltto-rtti-noexc-ndbg
• The largest code is 724 kb with O3-nolto-nortt-exc-nondbg
• The smallest is 335 kB with Os-lto-nortti-noexc-ndbg
• Ignoring Os the smallest is 470 kB with O1-lto-nortti-noexc-ndbg

Things noticed via eyeballing

• Os leads to noticeably smaller binaries at the cost of performance
• O3 makes binaries a lot bigger in exchange for a fairly modest performance gain
• NDEBUG makes programs both smaller and faster, as one would expect
• LTO typically improves both speed and code size
• The fastest times for O1, O2 and O3 are within a few percent points of each other with 0.95, 094 and 0.92 seconds, respectively

Caveats

At the time of writing the upstream code uses error objects even when exceptions are enabled. To replicate these results you need to edit the source code.

The benchmark does not actually raise any errors. This test only measures the golden path.

The tests were run on GCC 14.2 on x86_64 Ubuntu 10/24.