Y Combinator37 min
Now Anyone Can Code: How AI Agents Can Build Your Whole App
Amjad Masad, Jared Harge, Diana, Mark Mandelmann, Zahid Ali Jeelani, Francesc Campoy Flores, Franc Espinosa, Mark Manderson, Mark Mandelbacher, Mark Mandalbach, Amjad Ali Khani
Interview, Fireside Chat
- Users are predicted to eventually "orchestrate a giant army of agents" as a new capability, evolving from simple prompting to building complex systems without manual coding of every line.
- The current "team version" of the agent plans to expand to support background execution, allowing autonomous forking of projects to work independently before submitting pull requests or reporting issues.
- A market mechanism is envisioned where users can financially incentivize human experts, described as "bounty hunters," to solve problems the agent cannot resolve.
- Non-programmers who initially only prompt the agent are expected to gradually learn to read and edit the generated code, effectively transitioning into programmers.
- The return on learning to code is projected to double every six months, moving from limited leverage in 2023 with ChatGPT to massive leverage in 2024 with agents.
- Within the next six months, programmers are expected to gain even more power as the trajectory of increased capability using tools like Replit Agent continues.
- Retrieval systems will expand to quickly index any codebase and provide intelligence on existing projects, including LLM-generated summaries of files and project scope.
- Agents will evolve to accept user-specified technology stacks rather than currently pushing back on preferences like Python.
- Interaction methods will evolve beyond text to include UI drawing and voice, allowing expressive commands like "move this here" or "refactor this file" on a canvas.
- Advanced users will gain access to simpler agentic tools that provide a "dry run" showing diffs and required package installations before changes are accepted.
- Users who do not know how to code will incrementally learn programming skills by observing the agent's work, mimicking traditional learning processes.
- True AGI is not yet achieved because current LLMs lack efficient learning capabilities in unstructured environments and require symbolic representations like ASTs.
- The history of machine learning may see models eventually consuming specialized systems to create end-to-end solutions similar to Tesla's V13, though this does not guarantee true AGI.
- Functional AGI involving economically useful task automation is considered "fairly within reach," whereas true AGI requires capabilities beyond current LLM limitations.
- Users will still need to read and debug code because agents can get stuck or enter endless loops, making coding knowledge crucial despite reduced manual effort.
- The shift from pure RAG to custom orchestration is identified as critical, with determining what to edit being the most important factor for agent success.
- Relying solely on large context windows will not eliminate the need for specialized function and symbol lookups to prevent model bias toward end-of-context information.
- Future agentic tools must become multimodal and visualize agent trace graphs, addressing the current difficulty of debugging Directed Acyclic Graphs (DAGs).
- Retrieval systems will evolve to become "neuro-symbolic," combining RAG-style embeddings with direct lookups of functions and symbols within the code.
- Memory management will require agents to pick appropriate memories, augment them with bug fixes, or remove memories of resolved issues to prevent persistent errors.
- Scaling via larger context windows and GPUs is not a guaranteed path to success; specialized intermediate representations remain necessary for complex programming tasks.
- The "agent task force" structure will continue generating progress through weekly "run and review" meetings to identify failures and rapidly reprioritize work.
- Early access software is expected to move past the "barely beta" stage to become more reliable and capable of handling larger stacks.
- The "bounties program" may evolve into a real-time market where problems are solved for set prices, such as $50, creating a hybrid AI-human workforce.
- Interaction models will likely shift to include drawing in the UI and talking, making the experience more creative on devices like the iPad.
- Advanced users may prefer "single step or single action agents" that provide detailed plans before execution to maintain control over the code.
- The "no code tools" industry faces challenges as agents allow users to escape platform limits by editing generated code, potentially converting users into programmers.
- Agents will be able to work autonomously in the background for teams, forking projects and working independently until a problem arises or the task is complete.
- Retrieval systems will incorporate summaries of files and projects created by LLMs to help the agent understand project scope before indexing.
- Agents will continue improving their ability to follow orders, which is currently a larger issue than hallucination in these systems.
- Human-machine symbiosis is expected to prevail, with computers acting as extensions of users rather than competitors.
- The return on learning to code is expected to increase rapidly, with 2024 improvements providing massive leverage compared to 2020.
- Agents will improve at handling distribution shifts, though true AGI still requires efficient learning in zero-information environments which current models lack.
- Agents will move from pushing back on user preferences to fully accepting user requirements regarding technology stacks.
- Agents will function as development partners that ask questions, get stuck, and collaborate to mimic a human developer's workflow.
- Agents will eventually be able to deploy applications directly after building them from a prompt.
- Agents will be capable of creating data for testing and visualization, such as generating data to populate graphs.
- Agents will autonomously fix bugs but require a "reflection loop" to prevent entering endless loops and ensure correctness.
- Agents will orchestrate a "giant army of agents," allowing users to build complex systems without manually coding every line.
- Agents will summon human experts to the system via a market where users bid on solving specific problems, creating a hybrid workforce.