One of the greatest strengths of the classical C compiler model is that all compile jobs are fully isolated. This means that they can be run perfectly in parallel. This same feature is also one of its greatest weaknesses. There are no ways for individual compile jobs to communicate with each other even if they wanted to. This could be useful for things like caching. As an example a compiler might transparently create "precompiled headers" of sorts during one compilation and use them in other compilations if needed. This might also be useful for languages that require scanning steps before building such as Fortran and C++ using modules.
This is not idle speculation. Clang's thinLTO does use caching to speed up incremental builds. Since there is no existing standard for this, they did the usual thing and created a new compiler flag for specifying the location of the cache directory. Or, to be precise, they created four of them:
- gold (as of LLVM 4.0): -Wl,-plugin-opt,cache-dir=/path/to/cache
- ld64 (support in clang 3.9 and Xcode 8): -Wl,-cache_path_lto,/path/to/cache
- ELF lld (as of LLVM 5.0): -Wl,--thinlto-cache-dir=/path/to/cache
- COFF lld-link (as of LLVM 6.0): /lldltocache:/path/to/cache
For one option this is tedious but for many it becomes completely unworkable. Clearly something better is needed.
The basic idea: each build target gets its own private directory
Basically what one should be able to do is this:
gcc -c -o foo/bar.o bar.c -fprivate-dir=some_path
The private directory would have the following properties:
- The build system guarantees that it is set to the same value for all compilations of a single target (executable, shared library, static library, etc)
- Every build target gets its own unique private directory
- The contents of the directory may persist over successive invocations (i.e. its contents may be deleted at any time, but most of the time won't be)
- The compiler can create anything it wants in the private dir but should also tolerate other usages (typically you'd also want to put the target's object files in this dir)
- The contents of the private dir are transitory, they have no backwards or forwards compatibility guarantees. Any compiler update would invalidate all files.
Post a Comment