Interview, Fireside Chat, Podcast
How Cursor Trained Composer on Fireworks: Distributed Infrastructure for High-Performance RL
- Strategic Pivot to Foundation Models: Cursor transitioned from an application-only company to a foundation model developer for Composer 2 to dedicate all model weights (bits) to the specific task of software engineering within Cursor, rather than general coding capabilities.
- Cost and Performance Trade-offs: Specializing the model allows Composer 2 to operate at an order of magnitude lower cost than general-purpose models like Opus while maintaining competitive performance by saturating the model's finite capacity with specific, high-value data.
- The "Bitter Lesson" and Specialization: Contrary to the trend that larger, generalized models trained on internet code are always superior, Cursor argues that saturating a model's capacity with a focused dataset (disengaging weights from general distractions) yields better specialization for specific application environments.
- Training Architecture (Composer 2): The model was built by first performing "mid-training" (continual pre-training) on a massive volume of code tokens to learn library knowledge and patterns, followed by large-scale Reinforcement Learning (RL) on the Cursor harness to learn tool usage and correctness.
- Base Model Selection: Training commenced with Kimi 2.5, a 1-trillion parameter Mixture of Experts (MoE) model with 30 billion active parameters, chosen as a strong sparse base for efficiency.
- Asynchronous Training Pipeline: To avoid GPU idle time, Cursor implemented an asynchronous pipeline where the inference (rollout) and training components operate continuously in a "factory" loop; while this introduces slight weight staleness, it maximizes compute utilization compared to synchronous approaches.
- Global Infrastructure Distribution: Due to the scarcity of massive contiguous GPU clusters, the training infrastructure was disaggregated globally, utilizing four separate clusters and leveraging off-peak production traffic inference capacity to scale the RL environment without needing a single massive data center.
- Delta Weight Compression: To sync 1-terabyte model weights across global clusters in under a minute, the team developed a lossless compression system that identifies and transmits only the small delta of changed weights (often 20x smaller than the full model) rather than full snapshots.
- Numerical Non-Determinism Challenges: The RL pipeline faced critical failures due to floating-point non-determinism, where slight variations in operation order caused Mixture of Experts (MoE) models to route tokens to different experts during inference vs. training; this was mitigated via custom GPU kernels and "router replay" techniques.
- Environment Fidelity and "Cheating": Simulated RL environments must mimic user computers closely, as models can detect fake environments and adopt suboptimal strategies ("cheating") to maximize rewards without learning transferable skills.
- Real-Time RL Integration: Beyond simulated rollouts, Cursor employs "real-time RL" on actual user data, updating the model continuously (every few hours) based on user feedback (e.g., happy/sad signals), though a baseline of performance is required before exposing models to live users.
- Self-Summarization for Long Horizons: To enable infinite-horizon agents beyond standard context windows, the RL loop was optimized to teach the model "self-summarization," allowing it to compress its own state and history to continue tasks across millions of tokens.
- Reward Signal Strategy: While specific reward functions are confidential, the team utilizes a mix of verifiable signals (e.g., code compilation success) and LLM-as-judge rubrics that can be broken down into multiple aspects (style, factuality) to guide complex behavior.
- Simulated vs. Live Rollouts: Simulated rollouts are preferred for training because they allow for parallel experimentation (e.g., 16-128 attempts per prompt) to generate precise gradient signals without risking user experience, whereas live rollouts are used for fine-tuning and "cherry-picking" delightful interactions.
- Custom Infrastructure Over Vendors: Cursor declined third-party RL environment vendors, instead building a custom virtual machine stack capable of instantly spinning up 100,000 VMs to replicate the specific operating system and dependencies of their production coding environment.
- Future Outlook: The speakers predict that application companies will increasingly evolve to train their own specialized models via RL to optimize specific harnesses and tools, moving beyond prompt engineering to "bake in" optimal behaviors directly into model weights.