Interview, Fireside Chat
LangChain’s Harrison Chase on Building the Orchestration Layer for AI Agents | Training Data
- Definition of Agents: Harrison Chase defines an agent as an application where the LLM decides the control flow, rather than following a pre-determined, hard-coded sequence of steps (as seen in traditional RAG chains).
- Spectrum of Autonomy: Agent capabilities exist on a spectrum ranging from simple routers (classification steps) to fully autonomous agents that operate without human intervention.
- Langchain's Strategic Focus: Langchain is positioning itself as an orchestration layer specifically for the "middle ground" of this spectrum—building agents that are more flexible than chains but more constrained and reliable than fully autonomous prototypes.
- Hype Cycle Context:
- The initial hype for autonomous agents (e.g., Baby AGI, AutoGPT) peaked in Spring–Summer 2023.
- A "lull" period occurred from late 2023 to early 2024.
- Realistic, production-grade agents began emerging in 2024, including the Elastic assistant, Klarna customer support bot, and tools from companies like Devin and Sierra.
- Cognitive Architectures:
- Defined as the specific data flow and LLM call structure within an application (e.g., loops, branching, cycles).
- Production deployments increasingly rely on bespoke, custom cognitive architectures rather than generic, one-size-fits-all loops.
- Generic planning and reflection patterns (e.g., "Tree of Thoughts," React) are expected to be trained into foundation models over time, reducing the need for complex external logic for general tasks.
- Reliability and Production:
- General-purpose agents (e.g., solving GitHub issues) currently achieve roughly 12.5% success rates (citing Princeton SWE agent), which is insufficient for replacing human roles but shows progress.
- Custom, domain-specific agents (e.g., Vercel's v0, Elastic) are achieving sufficient reliability for customer-facing production deployment, though not necessarily "five nines" uptime.
- Future Industry Dynamics:
- The industry is shifting from "co-pilots" (human-in-the-loop) to "agents" (doing work independently, checking in only when necessary).
- Companies will likely retain control over "custom" cognitive architectures (codifying specific business logic) to differentiate their products, while ceding "generic" planning capabilities to model providers.
- Impact Areas:
- Customer Support: Currently a primary traction area where agents are automating routine inquiries.
- Software Development: Agents are closing the gap between idea and execution, enabling non-coders to build products (e.g., "idea to reality") and allowing engineers to focus on higher-level strategy.
- Cognitive Architecture Patterns:
- Planning: Explicit steps to force the LLM to generate a long-term plan before execution to prevent "hallucinated" completion.
- Reflection: Explicit steps added after actions to verify correctness and decide if re-try or correction is needed.
- UX Evolution:
- Chat remains the dominant UX but is limited by requiring constant human input.
- Emerging UX patterns for autonomous agents include:
- Rewind/Edit: Allowing users to backtrace the agent's decision tree to correct mistakes.
- Agent Inbox: Agents pausing execution to request human clarification via an inbox system.
- Collaborative Review: Asynchronous review of agent output (e.g., GPT Researcher drafts) where humans provide feedback loops.
- Product Strategy (Langchain):
- LangGraph: Solved orchestration for cyclical, long-running agents with persistent state, enabling async operations and human-in-the-loop workflows.
- LangSmith: Provides observability and testing for non-deterministic LLM applications, featuring:
- Pairwise comparisons for evaluation (LLM-as-a-judge).
- Human-in-the-loop annotation queues.
- Tracking of regression/improvement over time rather than binary pass/fail.
- Out of Scope: Langchain is intentionally avoiding building foundation models, vector databases, or fine-tuning infrastructure.
- Testing Heuristics:
- Traditional software testing (binary pass/fail) does not adequately apply to LLMs due to non-determinism.
- New testing paradigms require pairwise comparison, continuous regression tracking, and human verification.
- Leadership and Inspiration:
- Influences: Cites OpenAI (Sam Altman, Ilya Sutskever), Meta (Mark Zuckerberg's open-source Llama push), and Airbnb (Brian Chesky's product-centric leadership) as key inspirations.
- Core Advice: Emphasizes immediate building and experimentation over waiting for maturity, noting that while GPT-5 may render current implementations irrelevant, the learning process is the primary value for founders.