Conference Presentation, Keynote
Continual Learning: How AI Agents Get Better With Every Use | Arjun Karanam, Trajectory
- Arjun and Ronak plan to build a continual learning platform to capture the "100 trillion" tokens of real AI agent work currently discarded, aiming to close the "experience gap" and prevent models from acting as if it is their "first day on the job."
- The team intends to capture interactions for traceability as a primary step, developing a "model spec" to define agent requirements and researching algorithms like SDPO and reinforcement learning to utilize full long traces for improvement.
- A beta product demonstration is planned where training a model takes "probably like 15 minutes" of actual work, excluding model training time, to facilitate rapid iteration on difficult tasks.
- The strategy involves splitting feedback between models and harnesses, keeping factual knowledge like delistings in the harness while training specific errors or globally accurate information into the model to optimize across the system.
- Arjun predicts that correcting behavior requires eliciting and capturing user edits, undos, and retries, alongside solving the problem of noisy signals by securing the "right amount of feedback."
- The company plans to develop product interfaces where every UI action is replicable via agent tool calls, with tool responses explicitly stating written or read content rather than returning generic status messages.
- Future expectations include a "wishes" framework that traces the entire tree of events including sub-agents, and an evolution of harnesses from enforcing specific flows to defining primitives for tool orchestration and private information access.
- The firm aims to enable companies to own their own weights and models for continual learning on "open-weights," reducing reliance on consulting and allowing customers to build internal expertise.
- Research will address the lack of initial evals or traceability by assuming these do not exist, focusing on frontier customers pushing production boundaries and tasks where models "barely can do" what is asked.
- The speakers plan to overcome differential privacy constraints by sampling distributions from customer data and synthetically generating data rather than training directly on raw customer data.
- Arjun predicts that "model routers" will play a large role in directing intelligence to specific task capabilities, while grading should occur on "real harnesses that people use in production."
- The team expects to classify signals into those indicating failure without a correct answer and those with high confidence rewards, with feedback pertinence likely occurring on a per-organization or recursive per-customer basis.
- Audits will be performed to determine current standing on traceability, evals, and harnesses, acknowledging "a lot of work to be done" to reach the ideal state where the product, evaluation, and training environments are identical.
- The company envisions a future where every task becomes "rolloutable" by replaying user actions to solve infrastructure challenges, creating a cycle where agents fail on difficult tasks, train on those failures, and push the frontier of possibilities.