Research / Why Rust
Why Rust
Transient is Rust-first, not Rust-only. Rust is the default for everything we own, and any layer can use something else when a measurement says it is better. The project is the idea, not the language.
The case for it
- The cost is around the kernels. In our regime, small systems taking many small steps, the interpreter, allocation, tokenising, process start-up and glue code are a larger share of the total than they are for giant GPU-bound models. A language with no runtime and predictable memory attacks that share directly.
- One static binary. Nothing to install on the target machine, no Python environment, no framework version to match. That matters if "anyone can run it" is to be literal.
- Predictable memory. Light on RAM is a design goal, and we want to know where every byte goes. Peak memory of the whole teach-from-nothing session is about 12 MB, of which about 3 MB is the program itself.
- Zero dependencies.
babybuilds offline with the standard library alone, about 11,000 lines. Every dependency would have to earn its place with a measurement. It also means the whole thing can be read, audited and rebuilt without trusting a supply chain. - Safe concurrency and cross-compilation. We expect a multi-step system with background learning, and eventually phones. Both are things Rust handles well.
- Room to build from the bottom. Frameworks assume dense matrix multiplication, attention and a KV cache. Starting from scratch lets bandwidth, sparse reads and per-answer compute budgets sit in the foundations.
What we measured
Not much about the language itself, and we would rather say so. What we have is small and concrete:
- A dot product written as eight independent accumulators let the compiler vectorise a loop in safe code, with no flags: 32 µs to 13 µs per question, same answers. That is an implementation choice that the language made easy to reach, not a property of Rust.
- The release build is one 2 MB binary; the test suite (92 tests, including batteries of questions that must be refused) runs in about 13 seconds; teaching from nothing takes about a third of a second.
What Rust does not change
Caveat
The inner loop of bandwidth-bound work compiles through LLVM in Rust, C++ and Zig alike. The language does not make DRAM faster. Gains show up around the kernels and in the confidence to build more aggressive designs, and we do not want to claim more than that.
- The ecosystem is young. Most research ships PyTorch or CUDA reference code, so ideas from papers have to be ported by hand. Where a quick PyTorch run answers a question faster than a port would, we will use it and say so. A Python or C reference may serve as a test oracle that produces golden outputs the Rust version must match, never as the runtime.
- Some hardware features lag. At the time we last checked, the ARM instructions that fast quantised matrix multiplication uses were still unstable in Rust's standard library. The options are a nightly toolchain, inline assembly, or calling existing C kernels at first.
How we choose per layer
| Layer | Default | Note |
|---|---|---|
| System runtime: routing, stores, scheduling | Rust | Where baby lives today |
| CPU kernels | Rust with std::arch | Possibly C over FFI first for the hottest loop |
| Portable GPU compute | CubeCL / wgpu | Unverified on phones |
| Training and architecture research | Burn; PyTorch as a conscious exception | The planned transformer comparison is to be written in Rust so operations and joules count the same way |
| Plots, notebooks, one-off analysis | Whatever is fastest | Never shipped |
Alternatives we would take up if they measured better: Mojo, Zig, and compiler stacks such as MLIR that generate kernels per target. The rule is to choose per layer from a micro-benchmark, not from allegiance. The longer version is docs/Direction.md.