Self-Improving Harnesses, Local Personal AI And YC's Agent For Work | YC Paper Club

• YC Harness Club frames "harnesses" not as mere prompt engineering wrappers, but as essential scaffolding for AGI, citing an 18% performance boost between Harness 1 and 2 that determines whether Arc AGI succeeds or fails.
• Recent progress in agent capabilities is attributed more to "self-improving harnesses" over the last six months than to raw model intelligence, addressing the lack of test-time experience adaptation in current models.
• Benchmark results for the Arc AGI holdout set show a dramatic improvement from 30% with standard prompts to 95% with a specialized harness, and 100% using a combination of Prime Agent and NVIDIA's Avio.
• The "static harness era" (V0/V1) is distinguished from the emerging "self-improving harness era" by the ability of the system to modify its own system prompts, tools, memory, and sub-agent configurations rather than just executing them.
• DspY introduced the ability to optimize system prompts via genetic programming, while Darwin Godel Machines extended this to allow agents to modify the actual harness code running them.
• Continual Harness concepts, including DAgger-style online learning, enable test-time training on small example sets to update LLM weights and refine agent behavior dynamically.

Prime Agent (Seth)

• Prime Agent is a self-improving RLM (Recursive Language Model) harness designed to treat the raw LLM as a Turing machine while the harness provides von Neumann-style read/write operations on external memory.
• The architecture features persistent daemon subagents with "nuclear family" messaging capabilities, allowing parent, child, and sibling agents to share context and coordinate without restarting.
• Context management is structured across three tiers: L1 (model weights), L2 (active input context and in-RAM variables via REPL), and L3 (persistent file system and memory), utilizing "agentic garbage collection" to prevent resource exhaustion.
• On the Arc AGI evaluation, Prime Agent achieved 95.5% with Claude Opus and 78% with GPT-4o, significantly outperforming Cloud Code and Hermes Agent, which incurred high costs with poor performance.
• Long-horizon experiments include a seven-day factorial run utilizing 633 agents and 23 million output tokens to drive technological progression, where sub-agents divvied up research, resource gathering, and building tasks.
• Auto-research experiments on NanoGPT demonstrated agents performing "out-of-loop experiments" (e.g., hyperparameter search on CPU) to optimize expensive GPU training runs, a capability enabled by REPL access.

OpenJarvis (John Sad Falcone)

• OpenJarvis is a fully local, on-device personal AI stack aiming to rival cloud-based systems by running inference, agent logic, memory, and learning entirely on consumer hardware (e.g., Apple Silicon, NVIDIA).
• The system relies on five core primitives: UI, agentic logic (composable reasoning), intelligence engine (e.g., Qwen 3, Gemma 3N), inference hardware/engines, and learning systems (e.g., GRPO, LoRA).
• Local LMs like Qwen 3.827B (current state) match the performance of legacy frontier models like Claude Opus, with the performance gap closing as local accelerators improve.
• Cloud-based models (e.g., Claude Opus 5, Gemini) are utilized exclusively to optimize local configurations via an automated loop, achieving 800x cost reductions and significant latency improvements for local deployment.
• The project targets a shift where the majority of daily inference calls move to on-device laptops/workstations, prioritizing privacy, security, and energy efficiency over cloud dependency.

QM (Josh & Reagan)

• QM is YC's open-source agent harness deployed as a centralized, cloud-hosted system rather than a fleet of decentralized VMs, offloading all context and state to a centralized Postgres database.
• Sandboxes in QM function as on-demand resources rather than a fixed home, allowing the agent to dynamically select resource intensity (e.g., lightweight sandbox for simple tasks, high-performance nodes for dev work).
• The system integrates with internal APIs, OAuth credentials, and keychains to mimic an employee's local laptop experience, with a "grind tool" that enforces minimum runtime budgets to prevent agents from giving up prematurely.
• A key architectural decision is keeping the harness thin by exposing only three core tools: remote sandbox execution, object storage read/write, and internal app publishing, while delegating complex logic to the agent.
• Challenges identified include "main character syndrome" in agents (failing to see the whole system), social context blindness, and the critical need for fine-grained permission systems to prevent data leakage in multi-agent environments.
• Human-in-the-loop review is currently required for database writes, though the team anticipates building trust to automate this process as the agent proves reliability.