What's next for AI agentic workflows ft. Andrew Ng of AI Fund

Core Thesis: AI agents represent a significant shift from non-agentic, single-step prompting to iterative workflows that yield remarkably better results, even when using older or smaller language models.

• Non-agentic workflow analogy: Comparable to asking a human to write an essay from start to finish without using backspace or revision.
• Agentic workflow structure: Involves a loop of generating an outline, conducting research, drafting, self-critiquing, revising, and repeating the process multiple times.
• Empirical evidence: In the HumanEval coding benchmark, an agentic workflow wrapping GPT-3.5 outperformed GPT-4 running in zero-shot mode (which scored 67%).
• Baseline comparison: GPT-3.5 in zero-shot mode scored only 48% on the same benchmark.

Four Agentic Design Patterns:

• Reflection: A robust, widely applicable technique where an LLM critiques and revises its own output.
  • Mechanism: The same LLM generates code, then is prompted to check for bugs, efficiency, and algorithmic soundness before regenerating a corrected version (v2).
  • Advanced application: Systems can run unit tests; if a test fails, the agent analyzes the error log, adjusts the code, and attempts a third iteration (v3).
• Multi-Agent Collaboration (Specialization): Evolution from single-agent reflection to distinct roles within a single workflow.
  • Implementation: Assigns different prompts to the same base LLM to act as separate entities, such as a "coder" and a "critic" or "code reviewer."
  • Example: "Devon" (ChatDev), an open-source project where agents assume roles like CEO, designer, product manager, and tester to collaboratively build complex programs like a "Gomoki" game.
• Tool Use: Expanding LLM capabilities by enabling function calls for external actions.
  • Origin: Much of this work originated in computer vision to manipulate images, as early LLMs were "blind" to visual data.
  • Current application: LLMs generate code to execute web searches, run analyses, or perform actions via tools like Copilot.
• Planning: Breaking complex tasks into sequential steps and allowing agents to recover from failures autonomously.
  • Live demo example: Generating an image of a girl with a specific pose involves steps to determine pose, extract data from a reference image, synthesize the new image, convert to text, and finally generate speech.
  • Reliability: Currently "finicky" and not always reliable, but capable of rerouting around failures when successful.

Forward-Looking Statements & Strategic Shifts:

• Timeline: The set of tasks AI can perform is expected to expand dramatically this year due to the adoption of agentic workflows.
• User Behavior Change: Users must adapt to a "delegate and wait" paradigm, accepting latency of minutes or hours rather than demanding instant sub-second responses.
• Hardware & Speed: Fast token generation is critical for agent loops; generating tokens rapidly with a slightly lower-quality model can outperform slower, higher-quality models by enabling more iterations.
• Model Comparison: Early models utilizing agentic reasoning may achieve performance levels comparable to future "zero-shot" generations of next-gen models (e.g., GPT-5, Gemini 2.0, Cloud 4).
• Long-term Outlook: The path to Artificial General Intelligence (AGI) is viewed as a journey where agentic workflows serve as a crucial incremental step.

Current Status of Technologies:

• Reflection: Described as "pretty robust" with high reliability for immediate integration.
• Planning & Multi-Agent: Described as "emerging" and "messy," with performance that can be "amazing" but is not yet fully reliable for all use cases.
• Debate Pattern: Having distinct agents (e.g., ChatGPT vs. Gemini) debate each other has been observed to improve overall performance.