AI Engineer14 min
40x More Merged PRs With 2.7x the Engineers — Walden Yan, Cognition
Conference Presentation, Product Demonstration
- Cognition engineers are projected to achieve over 10x productivity gains while maintaining or exceeding current quality levels in merged pull requests.
- The organization aims to replicate its own 40x throughput growth across other large entities by adopting similar software factory and pod structures.
- Future workflows will default to Devon executing end-to-end software development processes, reserving human involvement exclusively for escalations.
- Engineering roles are expected to evolve away from ticket assignment, as any describable and implementable task will be delegated to agents.
- Team structures are predicted to shift so that only two to three engineers per team handle non-engineering functions such as bug reporting, QA, and product management.
- The development lifecycle is anticipated to reach a state where coding can occur without opening an IDE or launching a coding agent.
- Infrastructure plans include a single virtual machine environment for all services with mocked third-party versions, requiring agents to avoid live service access.
- The entire code base is scheduled for rebuilding every hour inside VMs, with Devon acting as the primary responder to resolve build failures.
- Agents are expected to self-maintain training loops and update skills automatically as the code base evolves, enabling testing without human intervention.
- Some engineers will transition to using iPads for development due to the implementation of a cloud-based, secure VM architecture.
- Testing capabilities will expand to include advanced multiplayer setups, such as generating multiple accounts within one VM to verify voice functionality on shared calls.
- Systems are designed to enable recursive self-improvement, allowing agents to analyze code and decision signals to reduce CI time by a minute or catch 20% more bugs before AI code review.
- A dedicated channel will receive daily optimization ideas from Devon, with agents possessing the capability to autonomously optimize the software factory.
- The industry is predicted to shift from task-based to job-based agent interactions, treating agents as co-workers managing long-duration goals without constant prompting.
- Organizations are advised to quantify non-measurable processes, such as latency or layout shifts, to enable agents to optimize these metrics continuously.
- Cognition intends to leverage agents to maximize throughput on Vera Rubin GPUs, building on prior findings that agents discover significant performance strategies.
- The company plans to launch an Android application developed over 44 days, with agents managing permanent hooks, live QA, and iOS parity syncing.
- During the Android maintenance loop, agents are expected to merge 40 to 50 pull requests daily with minimal human messaging input.
- Most engineering time during the maintenance loop will be spent monitoring Slack recordings rather than directly interacting with Devon.
- Organizational ambition is identified as a primary limitation to be overcome by establishing correct agent loops to drive further efficiency.