Pilot to Production: The Valley of Death and How to Cross It | RAISE Summit 2026

• Venkat Pulela (CTO, AI Infrastructure, Keysight), Yvonne (Co-founder, Daytona), Juro (Chief AI Officer, Prozus), Jonathan Corbin (CEO, Maven AGI), Ariel Schulman (CPO, Bright Data), and Barack (Chief Strategy Officer, Wonderful) formed a panel discussing the transition from AI pilots to enterprise production.
• The panel identified the primary challenge as scaling Proof of Concept (PoC) successes into secure, reliable enterprise deployments rather than initial model creation.

Data Infrastructure and Reliability

• Ariel Schulman (Bright Data) noted that AI agents fail at scale because the web, built for humans, often blocks or lies to agents, causing catastrophic decision errors.
• Bright Data provides uninterrupted, structured web access ensuring AI agents see identical information to humans, contrasting "quantum mechanics in reverse" where home demos work but production fails due to environmental variables.
• Ariel emphasized that reliable data access is critical to prevent agents from acting on hallucinated or incorrect web data, such as falsely reporting a 50% price increase and sending thousands of erroneous emails.
• Ariel Schulman added that enterprises are increasingly reluctant to share proprietary data with frontier LLMs due to security concerns, driving a shift toward on-premises deployments.

Security and Guardrails

• Jonathan Corbin (Maven AGI) cited that early AI agent deployments in customer service were initially limited by fear of brand damage, noting that human agents were actually wrong 40% of the time in similar contexts.
• Jonathan stated that despite initial reluctance, deploying AI agents in customer support has led to improved CSAT scores, reduced time-to-first-response, and increased customer happiness due to 24/7 availability.
• Yvonne (Daytona) explained that providing agents with full computer access (credentials, firewalls, secret management) via sandboxes is necessary for end-to-end task execution while maintaining safety guardrails.
• Yvonne highlighted that "computer use" capabilities allow agents to interact with legacy desktop applications (e.g., Siebel CRM) via virtual machines when APIs are unavailable.
• Barack (Wonderful) clarified that agent security largely mirrors human security protocols; existing observability, logging, and sandboxing for humans are generally sufficient for agents.
• Yvonne announced the launch of "Windows Sandboxes" to enable agents to automate interactions with legacy systems requiring headless GUI interaction.
• Juro (Prozus) emphasized that for e-commerce scales, commercial LLMs are too expensive, necessitating proprietary "Large Commerce Models" trained on internal data to achieve 200x cost reduction.
• Ariel warned that relying on a single model provider creates lock-in risks, urging enterprises to adopt model-agnostic architectures to navigate rapid model evolution.

Enterprise Adoption and Organizational Strategy

• Juro (Prozus) described a bottom-up strategy where 40,000 employees were incentivized to build agents, resulting in 60,000 agents created against a goal of 30,000.
• Prozus reported that 5,000 agents are used daily by non-engineers (lawyers, accountants), with 500-600 agents covering the entire operation's profitability.
• Juro observed that while AI is not a technology problem anymore, the primary hurdle is organizational change management and rethinking workflows rather than just process automation.
• Barack noted a shift from "copilot 2024" to "function-oriented work" and "end-to-end workflow rethinking" in 2025, focusing on revenue gains rather than just cost savings.
• Barack stated that enterprises are calculating agent ROI through revenue generation and throughput optimization, moving beyond simple token maximization to value maximization strategies.
• The panel suggested using open-source models with continual learning (post-training/RL on proprietary data) to create specialized, state-of-the-art models at lower costs.

Legacy Systems Integration

• The panel identified legacy systems (e.g., 30-year-old CRM systems) as a major bottleneck, where manual onboarding can take over two hours due to slow screen loads.
• Jonathan Corbin described how AI agents bypass legacy limitations by automating GUI interactions (computer use), allowing enterprises to leapfrog modernization without replacing core infrastructure immediately.
• Jonathan noted that the ability of agents to bypass rigid legacy systems is putting pressure on incumbents to modernize their software stacks.
• The panel predicted a significant shift in legacy system renewal decisions within the next one to two years as organizations begin to decommission systems that agents can now bypass.

Advice for Startup Founders

• Barack advised founders to avoid consensus strategies, noting that "non-consensus beliefs" are required to differentiate in a market with infinite competitors.
• Barack encouraged founders to develop an enduring first-principle strategy regarding where AI is headed rather than chasing short-term trends.
• Ariel recommended that startups focus on agents as a growth engine (targeting 20-30% growth) rather than solely a cost-cutting tool.
• Jonathan advised a methodical approach to deployment: identify specific pain points, define success metrics (efficiency, CSAT, revenue), and iterate rather than deploying agents broadly.
• The panel recommended that startups and enterprises alike assume production will have problems and implement rigorous data validation and mentorship-like guardrails for AI agents.