Interview, Fireside Chat
Sierra co-founder Clay Bavor on Making Customer-Facing AI Agents Delightful
- Large language models are currently more effective at detecting errors in their own output than preventing them, with the industry unable to claim victory over the unsolved issues of hallucination regarding trust, safety, and reliability.
- Approximately 80% of customer service inquiries occur via phone, driving the expectation that multimodal models and voice capabilities will become central to AI agent interactions.
- Over the next few years, AI agents are projected to evolve from handling simple queries to managing the entire customer journey, from pre-purchase consideration to troubleshooting, while resolving issues faster than current methods.
- Companies are expected to require their own AI agents, with some envisioning agents that perfectly replicate organizational voice, values, and "vibe" to deepen customer connections.
- Future model architectures will likely involve swapping in next-generation frontier models for "IQ upgrades" while utilizing smaller, specialized models for specific tasks like triaging to improve speed and reduce costs.
- Metrics for conversion and retention are predicted to shift significantly if companies provide five to ten times the current amount of fluent, conversation-based support.
- Customer satisfaction scores for AI agents, currently in the mid-4s, are expected to improve over time, potentially surpassing human performance on complex issues and reducing frustration-driven churn.
- The average cost of a customer service call, currently $12 to $13, is anticipated to drop to a fraction of that amount when resolved by AI agents, resulting in unequivocally positive ROI.
- Organizations plan to capture institutional wisdom through "experience manager" tools that continuously coach agents and correct errors, ensuring iterative improvement.
- Without sophisticated agent architectures, relying solely on LLMs risks high failure rates in large-scale interactions, where a "pass at K" probability of 0.61 raised to the eighth power yields approximately 25% success.
- The "agent development life cycle" is forecast to evolve to include specialized testing simulators and quality assurance processes distinct from traditional software development.
- Companies aim to distill best practices, such as sales forecasts and support workflows, to run them consistently across all regions and calls, thereby increasing organizational capability.
- AI agents are expected to perform value discovery and revenue preservation by understanding customer intent to offer appropriate plans and prevent churn.
- A feature-length film entirely "filmed" with AI is predicted within the next couple of years, alongside the ability to generate entire worlds from brief text descriptions, changing computer graphics and rendering landscapes.
- Within the next five years, AI is expected to act as a massive creative force multiplier for individuals, accelerating the transition from idea to manifestation.
- Solutions to many AI problems are anticipated to rely on "more AI," leveraging techniques such as chain of thought and specific prompting strategies to elicit better reasoning and results.
- Tooling and documentation are expected to automate deployment processes, making the deployment of successful AI agents 10 times faster and more impactful.
- Agents are predicted to eventually handle processes currently deemed too complex to model and scale using existing architectures.