C++ has enormous low-level power, but lacks a standardized meta-DSL-layer: portable reflection-ish primitives for defining and “compiling” DSLs at runtime, and an ergonomic runtime compile/link/load facility. Even though highly performant and power-efficient code is incredibly important in this brave new world, most modern code (AI or otherwise) is often written in Python. As C++ programmers, we often forget how little can be done at compile time in real-world systems. Programs increasingly need to build programs at runtime - query engines, DSLs, data pipelines, ML graphs, packet filters, SIMD kernels. Those systems in other languages don’t always “JIT compile C++”; they often “compile” by lowering a runtime graph into an executable plan, which may be (1) dynamic dispatch over precompiled variants, (2) runtime specialization over embedded IR/bitcode, or (3) true in-process compilation and loading.
In this talk, we will go on a journey to build a real numerical (but toy) DSL which can do GPU and CPU operations, with multiple source languages and multiple backends, supporting host bindings.
To do that, we discuss the philosophy of why we need graph based DSLs, and what they can cover. We will then distill the mechanisms other ecosystems rely on - DSL graph building, specialization, caching, and JITing. Then we shall build a numerical DSL language in C++ from scratch, and in that process, try to generalize three concrete subsystems for use in C++ that can make DSLs vastly more ergnonomic in C++:
- A Graph DSL-IR pattern for representing programmatic computation as runtime graphs in any syntax (including C++) and “serializing” these program into dispatch plans, agnostic of what "dialect" of computation we're representing, whether it be closer to LLVM IR, Triton, or CUDA. This would include safe serialization of types as well.
- A compiler interface to compile programs represented in the DSL-IR to machine code. The toolchain used would depend on the dialect of the graph DSL. This allows us to build a simple JIT layer to compile a standard serialized IR to code, without binding directly to Clang/LLVM or platform-specific loaders.
- Small tooling primitives such as a standard dynamic library loader, and type safety (runtime type checking) to make loading and workflows safe and ergnomic.
We’ll show how these three items compose into a real DSL IR, which can build specialized kernels, plugin architectures, and useful runtime programs.
Saksham Sharma is a Director of Quantitative Research Technology at Tower Research Capital LLC, a high frequency trading firm based out of New York. He develops low latency and high throughput trading systems and strategies used for the firm's global quantitative trading. In addition, he also helps design and improve big data research infrastructure used for trading research using a combination of C++ and Python. Thanks to an urge to optimize his daily workflow, he also owns the build systems, tooling, and package managers for C++ within his team.In the past, he has worked in the fields of program analysis research, functional programming, and systems security. He is a reformed geek (exemplified by a switch to Cursor after almost a decade with Emacs), and now enjoys driving, guitar, badminton, and snowboarding.