Earlier today I livestreamed what it would take to build a small part of the recently open sourced O3DE game engine. The attempt did not get very far, so here is a followup. It should not be considered exhaustive in any way, it is literally just me poking the code for a few hours and writing down what was discovered.
Size
Name prefixes
Compiling
The code can be obtained from the upstream repo. There's not much more you can do with it since it does not actually build on Linux (tested on latest Fedora) but instead errors out with a non-helpful CMake error message.
When I finally managed to compile something, the console practically drowned in compiler warnings. These ranged from the common (missing virtual destructors in interface classes) to the bizarre (superfluous pragma pops that come from "somewhere" due to macros).
Compiler support
The code explicitly only supports Visual Studio and Clang. It errors out when trying to build with GCC. Looking through the code it seems like it is mostly a case of adding some defines. I tried that but pretty quickly ran into page-long error messages. A person with more knowledge of the inner workings of GCC could probably make it work with moderate effort.
Stdlib reimplementations
O3DE is based on CryEngine, which predates C++ 11. One place where this shows up is that rather than using threading functionality in the standard library they have their own thread, mutex, etc classes that are implemented with either pthread or Windows threads. There may be some specific use cases (thread affinity?) why you'd need to scoop to plain primitives but otherwise this seem like legacy stuff nobody has gotten around to cleaning.
Yes, there is a string class. Several, in fact. But you already knew that.
Dependencies
This is where things get weird. The code uses, for example, RapidXML and RapidJSON. For some reason I could not get them to work even though I used the exact same versions that are listed in the CMake definition. After a fair bit of head scratching things eventually became clear. For example the system has its own header for RapidXML called rapidxml.h whose contents are roughly the following:
Upstream does not provide its header in a rapidxml subdirectory, it is under include. The same happens when the header is installed on the system. Thus the include as given can not work. Even more importantly, the upstream header is not named rapidxml.h but instead rapidxml.hpp.
It turns out that O3DE has its own dependency system which takes upstream source, makes arbitrary changes to it, builds it and then is provided as a "package" which is basically a zip file with headers and prebuilt static libs. These are downloaded from not-at-all-suspicous-looking URLs like this one. What changes are done to these packages is not readily apparent. There are two different repos with info but they don't seem to have everything.
When using external libraries like this there are two typically two choices. Either you patch the original to "fit in" with the rest of your code or you can write a very small adapter wrapper. This project does both and with preprocessor macros no less.
The whole dependency system seems to basically be a reimplementation of Conan using pure CMake. If that sentence on its own does not make cold sweat run down your back then let it be noted that one of the dependencies obtained in this way is OpenSSL.
The way the system is set up prevents you from building the project using system dependencies. This includes Qt as the editor GUI is written with it. Neither can you build the entire project from source yourself because the existing source only works with its own special prebuilt libraries and the changes applied do not seem to be readily available as patches.
Most of this is probably because CryEngine was originally written for internal use on Windows only. This sort of an approach works for that use case but not all that well for a multiplatform open source project.
This sounds a lot like how CMake setup for Tensorflow works, including the reimplementation of standard library.
ReplyDeleteHi Jussi,
ReplyDeleteFirst of all, thank you for giving it a try and experimenting with O3DE. I may be able to give some context here...
Looks like you have a typo "PLY", I imagine you meant "PAL", yes, it stands for "Platform Abstraction Layer". It is a set of patterns and ways to handle different platforms. Basically, we place things in a `Platform/` folder and include it depending on the platform. We do link time bindings, compilation traits, etc that way. We use a similar pattern for the build system. So you will see a lot of `include(Platform/${PAL_PLATFORM_NAME}/something.cmake)`.
PAL is a bit complex because for legal reasons some platform's source code cannot be publicly available. So it has to be able to be hooked with a different file on a different path.
Linux is in the works, 3rdParty libraries are being compiled and code is being fixed to launch the Editor. At the state it is right now, it can only build "server targets". Keep an eye on https://github.com/o3de/o3de/issues/745 and https://github.com/o3de/o3de/issues/746
I missed the livestream, if you can post some of the compiler warnings, it will help to understand why you are getting them. We run CI on every PR and have warnings as errors so they should not be creeping in.
ReplyDeleteIf you tried to run it with GCC and are getting warnings from there, we dont support GCC. The reason is that we already have a lot of permutations/configurations and platforms. Adding compilers and generators create a huge list of things that "have to be working". This creates some friction in development (works with this compiler, not in this compiler, has to be tested in both, etc). So, we choose one "golden path" on each platform and stick to that. For Linux is Ninja+Clang, for macOS/iOS is XCode+Clang, for Windows is MSBuild+CL, for Android is Graddle+Ninja+Clang, etc.
That will be reflected in the documentation once that platform becomes available.
We are open to add more compilers/generators, however, they may be "not validated" continously. It will be up to the community to push for those.
To add GCC, you will need to create a file like cmake\Platform\Common\Clang\Configurations_clang.cmake and handle it in the platform that will support it, e.g. cmake\Platform\Linux\Configurations_linux.cmake
You can use clang as an example and disable the same warnings. Those warnings are going to be eventually cleaned (I am going through a lot of MSVC first and then I will go over the clang ones).
We do total control of the compilation/linker options so by default adding a new compiler needs the basic options. We do this because is problematic to replace compiler/linker flags in CMake (e.g. opposing flags wont be cancelled).
You can see the warnings that are not yet fixed in e.g. cmake\Platform\Common\Clang\Configurations_clang.cmake.
We set the current standard to C++17. O3DE evolved from CryEngine, but its not based on Cry. We actually removed most of CryEngine. E.g. the renderer and physics were removed. Is really hard to remove a whole engine, we need to untangle dependencies and replace functionality. Takes time, we will get there.
Most AAA game engines have some sort of STD/STL implementation. There are several reasons for this, for O3DE some reasons are:
1) behavior consistency across platforms (specially on consoles, some of them have very old versions of the standard and sometimes dont fully implement it);
2) hook to a different memory system (O3DE uses its own memory system, the OS memory system is expensive, specially on some consoles, common in AAA engines);
3) we can implement optimizations earlier or that the std doesnt have (e.g. short string optimization was implemented before it was available in some platforms, vector::resize_no_construct is not part of the std)
The "several string classes" are going to be cleaned up, its mostly old Cry stuff. The main string class is "AZStd::string". Of course, there are fixed string class, a wstring class, etc...
ReplyDeleteFor Dependencies, yes... there is a lot going on here.
There was an investigation done to go with our own 3p system. For Conan these were the problems:
* we have our own python version. So we cannot use a python-based package system to download python.
* you need to run an artifactory server in order to host packages. Artifactory is a JFROG product and costs licenses/seats to actually use it effectively. Customers that want to replicate infrastructure would be tied to that.
* Conan is not designed to deal with platforms that need to be in a different path. Because of legal reasons, code for some platforms in O3DE are kept in a different (private) repository. We have a special way to link those together.
The 3p system that was implemented is really simple and supports multiple backends (it can even use a fileserver). That url is just Amazon's CDN, the packages are stored in S3. Package source scripts are versioned here: https://github.com/o3de/3p-package-source
There are ways to bypass it. The 3p system just adds a "download this" and "add it to the module path". Each package that is downloaded has a "Find" file.
Modifications that have been done to 3rdParty packages are a hassle so we do them when absolutely needed, some examples:
* security fixes that are not yet done in the public source code
* compilation fixes for other platforms that the original source code does not support
* small fixes that the original source code has not accepted a PR yet
* modification/trimming to the specific needs of the engine (e.g. in Qt we remove a lot of modules)
If you have more questions, let me know I would be happy to help.