Compiler Technology
Returning to the history of C++Builder as it originally was and as we know it today, its compiler, known today as the ‘classic’ compiler, was famous for being faster than other compilers in the industry. One reason it was fast was its architecture: today, it’s normal for compilers to parse an entire translation unit of source code, build an AST (an abstract syntax tree, basically an in-memory representation of what the source represents), generate an intermediate representation (IR) likely performing many transformations on that, and give that to a backend that itself may do some transformations. This is a three-phase (frontend, intermediate representation, backend) and multi-pass model, the model Clang and LLVM implements.
But the classic compiler was different: it would read a block of source at a time – a method, or maybe even just an if-block or for-loop – generate enough of an AST for just that block, and hand that over to the backend which wrote to the object file: effectively a two-phase (frontend, backend) single-pass (hand-waving at the optimisations the backend performed here) model, using a backend originally shared with but now separate to Delphi’s. This was immensely fast, but limited the understanding of the compiler of the entire source, including for compiler optimisations.
That classic compiler grew out of Turbo C and Turbo C++, and was one of the first to implement templates: in fact, in the 2000s, there were two template models: Cfront and Borland. Over time more language features were added to the classic compiler, including some from what would become C++11, and the compiler is still shipped today for those who want to use it, even if it’s not the default. Yet eventually C++Builder switched to using Clang. Why? There were many reasons: an ageing codebase coming from Turbo C and written mostly in C; an older architecture hard to adapt to modern needs as noted above; but fundamentally the item that really caused Embarcadero to switch was templates.
C++11, the second major version of C++ (C++98 being the first) added more powerful template technology, especially variadic templates. The compiler architecture was showing its age, and was going to be very hard to rework to add these and some other key C++11 features. In 2011 Clang and LLVM were rapidly-moving projects, and it was clear that they provided a well-architected path forward to which it would be easier to add custom behaviour. While macOS (then OS X) support was added to C++Builder in 2011 using the classic compiler, in parallel an internal effort was under way to add C++Builder’s language extensions to Clang.
XE2, the same version that shipped OS X support with the classic compiler, shipped C++ Audits (a now-removed feature.) The audits feature was based on Clang 3.1, and using it for audits in one version let the team test their customised Clang on real-world source code using C++Builder’s syntax and extensions before shipping a compiler that supported those extensions. The compiler shipped in Update 1 to XE3, and the Embarcadero team led Clang history here as the first to bring Clang to Windows, followed later by Google and Microsoft.
Clang 3.1 was rapidly updated to Clang 3.3, and then later to Clang 5 in 2018 when C++Builder added C++17 support.