AI Engineer13 min
The Agents Were Fine. Everything Around Them Broke. - Joey Wang, Harvey
Conference Presentation
- The outlook anticipates that the existence of a reliable harness will introduce unplanned work volumes, requiring a transition from local laptop agents—which realistically support only 5 to 10 concurrent instances—to cloud-based infrastructure.
- Plans include a unified workflow allowing agents to initiate sessions locally, migrate to the cloud, and return, with a critical focus on a follow-up loop where 4 out of 5 agent pull requests merged into the backend during early October passed this stage.
- Operational standards are expected to remain consistent across agents regarding audit trails and checks, while the command line interface is projected to handle approximately 6,500 tasks monthly by October, with some users executing hundreds of tasks monthly without app interaction.
- Recent performance metrics show an engineer completed roughly 950 tasks via the command line in three hours by late October, and the platform successfully sustained increasing task bursts despite unspecified technical fixes.
- Infrastructure reliability improved significantly by November 4th, with task completion rates exceeding twice that of August, where specific failure rates included two stalled tasks and a 10% non-completion rate for others.
- Despite these gains, the system retains capacity to grow, though PR volume surged approximately 60% in late October, causing merge times to extend from 15 to 30 minutes to over an hour.
- A known bottleneck persists regarding PR rebasing and merge queues, which remains unresolved, and last month approximately 110 individuals had automation actively running.
- Automation safety protocols are designed to self-disable if an agent fails five consecutive times, while future goals include eliminating lock-offs for agents in every Slack channel and establishing a proof-of-work standard before human review.
- Current proof-of-work adoption stands at one merge in 20, and the speaker projects that specific system failures will become apparent within six months.
- The overarching conclusion identifies the loss of work as stemming from the absence of a harness capable of running, checking, and merging tasks automatically.