newsfilter
← Interviews
AI Engineer14 min

40x More Merged PRs With 2.7x the Engineers — Walden Yan, Cognition

Walden Yan

Conference Presentation, Product Demonstration
  • Cognition engineers are projected to achieve over 10x productivity gains while maintaining or exceeding current quality levels in merged pull requests.
  • The organization aims to replicate its own 40x throughput growth across other large entities by adopting similar software factory and pod structures.
  • Future workflows will default to Devon executing end-to-end software development processes, reserving human involvement exclusively for escalations.
  • Engineering roles are expected to evolve away from ticket assignment, as any describable and implementable task will be delegated to agents.
  • Team structures are predicted to shift so that only two to three engineers per team handle non-engineering functions such as bug reporting, QA, and product management.
  • The development lifecycle is anticipated to reach a state where coding can occur without opening an IDE or launching a coding agent.
  • Infrastructure plans include a single virtual machine environment for all services with mocked third-party versions, requiring agents to avoid live service access.
  • The entire code base is scheduled for rebuilding every hour inside VMs, with Devon acting as the primary responder to resolve build failures.
  • Agents are expected to self-maintain training loops and update skills automatically as the code base evolves, enabling testing without human intervention.
  • Some engineers will transition to using iPads for development due to the implementation of a cloud-based, secure VM architecture.
  • Testing capabilities will expand to include advanced multiplayer setups, such as generating multiple accounts within one VM to verify voice functionality on shared calls.
  • Systems are designed to enable recursive self-improvement, allowing agents to analyze code and decision signals to reduce CI time by a minute or catch 20% more bugs before AI code review.
  • A dedicated channel will receive daily optimization ideas from Devon, with agents possessing the capability to autonomously optimize the software factory.
  • The industry is predicted to shift from task-based to job-based agent interactions, treating agents as co-workers managing long-duration goals without constant prompting.
  • Organizations are advised to quantify non-measurable processes, such as latency or layout shifts, to enable agents to optimize these metrics continuously.
  • Cognition intends to leverage agents to maximize throughput on Vera Rubin GPUs, building on prior findings that agents discover significant performance strategies.
  • The company plans to launch an Android application developed over 44 days, with agents managing permanent hooks, live QA, and iOS parity syncing.
  • During the Android maintenance loop, agents are expected to merge 40 to 50 pull requests daily with minimal human messaging input.
  • Most engineering time during the maintenance loop will be spent monitoring Slack recordings rather than directly interacting with Devon.
  • Organizational ambition is identified as a primary limitation to be overcome by establishing correct agent loops to drive further efficiency.