Sep 4, 2025, 9:30 AM ETInterview

Why ML Needs a New Programming Language with Chris Lattner

Chris Lattner began in computing by manually typing code from books and hobbyist magazines, fostering a deep understanding of debugging and hardware limits.
His academic path was shaped by Professor Steven Vegdal, leading him to the University of Illinois to build LLVM, which he describes as a "layered" systems engineering experience superior to typical academic assignments.
Lattner views his career as a non-static pursuit of the "hardware-software boundary," moving between compiler engineering and language design to solve distinct, high-value problems.
He identifies a structural flaw in current language design: many academics focus on mathematical theory over utility, whereas he prioritizes features that make developers' lives easier and code more expressive.

Swift originated in 2010 as a "nights and weekends" project by Lattner, born from burnout after implementing C++ in Clang and a desire to create a language with modern features like pattern matching.
Lattner argues that pattern matching (algebraic data types) is a foundational, utility-first feature that is standard in ML/Haskell but absent in mainstream languages like C++, Java, and C# at the time.
He critiques the academic tendency to prioritize type theory over practical utility, noting that features which compose well mathematically are often the most robust in practice.
A key lesson from Swift is the danger of feature creep; the "Grain of sand" problem where small, unnecessary complexities in C++ or Swift compound to make the entire system unwieldy.
Lattner advocates for a design approach that prioritizes simplicity and composability, viewing mathematical simplicity as a proxy for truth in engineering.

The AI industry suffers from extreme fragmentation where every hardware vendor (NVIDIA, AMD, Google) builds incompatible, proprietary software stacks (CUDA, ROCm, XLA) because no neutral entity is incentivized to unify them.
This fragmentation forces software platforms like PyTorch to "glue" together incompatible worlds, resulting in leaky abstractions that break when performance optimization is required.
Hardware companies focus on their specific chips with no incentive to support competitors' hardware, leading to a lack of portable software that works across heterogeneous accelerators (GPUs, TPUs, FPGAs, ASICs).
Previous attempts to fix this via high-level abstractions (MLOps, Python layers) failed because they could not hide the underlying performance trade-offs and incompatibilities of the hardware.
Modular was founded to build a foundational software platform that unifies this compute, aiming to beat hardware vendors at their own game on performance while providing portability.

Modular's business model does not sell the programming language (Mojo) or charge for the core compiler; it sells enterprise solutions for managing large-scale AI infrastructure (platform engineering, observability, and scalability).
The company addresses the "platform team" pain point: hundreds of GPUs rented from clouds require maintenance, and the current chaotic software ecosystem makes production workloads unstable.
The goal is to reduce complexity multiplicatively by providing a single release train that supports both AMD and NVIDIA GPUs, solving the fragmentation issue at the source.

Mojo is a member of the Python family, adopting Python's syntax to lower the barrier to entry for the AI community while diverging to add performance-critical features.
The language was designed from first principles to support modern heterogeneous hardware (Tensor Cores, SIMD) where existing languages like C++ and CUDA fail due to lack of portability and performance.
Metaprogramming: Mojo unifies compile-time and runtime code in a single language, allowing developers to build powerful, high-level abstractions that compile away control structures, unlike C++ templates which are difficult to debug and reason about.
Type System: Mojo utilizes a strong, explicit type system with traits (similar to Rust protocols or Swift protocols) to manage complexity and prevent the "template explosion" seen in C++.
Portability Strategy: Mojo compiles to an intermediate representation (IR) that retains programmer intent and type information, allowing a single source file to be specialized for different hardware targets (CPU/GPU) without "if-def" hell.
The language allows for "layout tensors," a compile-time abstraction that maps memory layouts declaratively to specific hardware architectures (e.g., NVIDIA vs. AMD tensor cores).

Mojo avoids the "sufficiently smart compiler" trap by requiring explicit code for performance tuning, ensuring predictability and control over the hardware rather than relying on hidden optimizations that break easily.
The team is addressing the massive configuration space of hardware by using traits and type-level constraints to allow modular, testable abstractions that the compiler can verify at compile time.
Backward compatibility is managed via semantic versioning (1.0, 2.0), with a vision to allow different versions of packages to interoperate, avoiding the Python 2-to-3 migration nightmare.
Agentic AI: The open-source nature of Mojo and its strong type system are leveraged to train AI coding agents, enabling them to learn the language from public repositories and generate high-quality GPU kernels.
Short-term Utility: Currently, Mojo is the best way to extend Python for performance (replacing C/C++ extensions) or to write full-stack GPU kernels that are portable across vendors.
Long-term Vision: Lattner projects that within 18 months, Mojo will evolve to include classes and dynamic features, potentially becoming a "Python 4" superset that is fully compatible and superior to the Python ecosystem while replacing Rust in many systems contexts.
The immediate focus remains on "democratizing AI compute" by making it possible for developers to achieve full hardware performance without the complexity of CUDA.