Interview, Fireside Chat
State-Of-The-Art Prompting For AI Agents
- Metaprompting is emerging as a widely adopted, powerful tool where the current landscape remains an unrefined frontier with tools that are not yet fully developed.
- Best practices for prompt engineering involve defining the LLM's role, decomposing tasks into high-level plans, specifying output structures (including XML tags for compatibility), and eventually integrating specific scenario-based examples.
- Parahelp currently powers customer support for major AI companies including Perplexity, Replit, and Bolt, and has agreed to open source its vertical AI agent prompts.
- A significant industry challenge involves balancing the need for general-purpose products with the requirement to accommodate unique customer "flavors," a problem potentially addressed through the emerging practice of forking and merging prompts.
- Startup opportunities are anticipated in tooling that automates the ingestion of worked examples from customer datasets, alongside developments like Tropir's "prompt folding" for dynamic self-generation of improved prompts and Jasberry's automatic code bug detection via hard error examples.
- Strategies for handling complex tasks include using examples over prose when tasks are difficult to articulate, providing LLMs with an "escape hatch" to request missing information to prevent hallucination, and utilizing debug parameters to generate developer to-do lists.
- Companies plan to use larger models (e.g., GPT-4o, GPT-o3) for meta prompting to refine prompts before distilling them into smaller, faster models for latency-sensitive applications like voice AI.
- Advanced model features, such as Gemini 2.5 Pro's thinking traces and long context windows, are expected to enable real-time reasoning trace analysis and prompt steering.
- The "forward-deployed engineer" model positions founders as ethnographers and designers who close six-to-seven-figure deals quickly by providing immediate, working software demos, a strategy exemplified by companies like Happy Robot and GigaML.
- Competitive advantages against incumbents like Salesforce are expected to come from slightly better UIs and the ability to tweak and demo products during initial meetings based on forward-deployed engineering.
- Future AI models are predicted to have distinct characteristics: Claude as a more human-steerable model, Llama 4 requiring significant steering with less RLHF, O3 being rigid regarding rubrics, and Gemini 2.5 Pro excelling at flexibility while applying rubrics.
- Founders are expected to leverage LLMs to score investors on a zero-to-100 rubric, while the core philosophy of meta prompting aligns with Kaizen principles where the practitioners driving the process are best equipped to improve it.
- The current state of AI tooling is described as comparable to coding in 1995, characterized by an unspecified frontier where many elements remain undefined.