AI Engineer18 min
All Code Is Slop (Except Mine) — Lauren Tan, SpaceXSI
Conference Presentation, Keynote
Productivity Shift & Thesis
- The speaker (Potato/Lauren) at SpaceX SI shifted from viewing volume as an outlier concern to a core metric, enabling agents to function as "coding machines" for any engineer.
- The core thesis is that the software engineer's primary role is now maintaining or improving code quality while drastically increasing productivity through agent trust.
- A "software factory" model was adopted, prioritizing trust in the agent environment over blind trust.
- The team achieved the ability to ship thousands of pull requests automatically, including during sleep, after six months of effort to establish high-trust environments.
The "Software Garden" Architecture
- The codebase is treated as the largest input for agents; poor inputs (slop) are replicated automatically, necessitating a pristine foundation.
- The volume of code arriving shifted from ~3 PRs per engineer previously to ~300 PRs, requiring a transition to "organic architecture" similar to gardening.
- Code constraints and guardrails act as concrete foundations allowing new agent-generated code to grow correctly without human oversight.
Deterministic Code Constraints (The First Two Layers)
- API & Architecture Design: The team created "Dune," a client framework for Grokbot analogous to Next.js for Electron, to encode lessons learned in desktop app development.
- Banning Nuance-Heavy APIs:
useEffectin React was categorically banned in product code due to its nuance; it is now only allowed in specific internal Dune APIs to prevent agent misuse. - Structural Context Enforcement: Imports between
ElectronMainandElectronRendererare forbidden via dependency graph analyzers and Lint rules to prevent main-thread code from polluting the render thread. - Comment Reduction: 99% of code comments were banned because agents use them as "escape hatches" to justify incorrect fixes, and comments often go stale or are ignored by humans.
- Error-Driven Correction: Lint rules now provide specific error messages forcing agents to fix root causes rather than applying patchy workarounds like imperative timers.
Verification & Testing Strategy (The Third Layer)
- Code review focus shifted from human line-by-line reading to reviewing "verification artifacts" such as videos, screenshots, and formal proofs.
- Runtime Verification: Agents utilize the Chrome DevTools Protocol and custom "feature maps" (site maps of user-facing features) to navigate and test applications autonomously.
- Tooling Efficiency: Custom CLIs and drivers were integrated into agent skills to reduce token usage and increase testability.
- Formal Verification: The team employs TLA+ for model checking major user flows (e.g., cross-device notifications) and Lean for proving algorithm correctness in custom virtualization libraries.
- Property-Based Testing: The library "Bombadil" was adopted to catch UI bugs through automated property testing.
- State Visualization: A custom skill called "state chart" was built to visualize state machines, aiding in the review of agent logic.
Outcomes & Metrics
- The team shipped 3,000 pull requests in a single month and 11,000 pull requests in the most recent month.
- A new stable client version is released approximately once a day.
- Contributor volume expanded significantly, with 227 unique contributors in October, including casual "drive-by" contributors.
- Quality Control: The revert rate remained steady at 0.5% despite the massive increase in velocity.
- Performance: Application performance (startup, typing, switching chats) improved relative to launch, despite the high velocity and the inherent constraints of the Electron framework.
- Token Allocation: Verification tasks now consume approximately 70% of daily tokens, reflecting a shift from writing code to verifying it.
Strategic Implementation & Automation
- The workflow requires a "low-to-high trust" progression: correct the agent once, then convert that correction into a deterministic Lint rule or architectural constraint.
- Automating a messy codebase (inner loop) results in exponential "slop" generation; therefore, static analysis, runtime verification, and formal verification are prerequisites for automation.
- Automated Release Workflow: Grokbot now automatically spawns Cursor projects to fuzz the entire application and build upon every release cut.
- Auto-Merge Capability: Agents identify bugs via fuzzing, open PRs with fixes, and auto-merge them due to the high quality of the underlying codebase, reducing the need for deep human review.
- The speaker argues that agents enable a "golden age" of software testing by making comprehensive verification economically viable.