AI Engineer23 min
I love you so jevving much — Diogo Almeida, TypeSafe AI
Conference Presentation
Core Diagnosis of Current AI State
- The field suffers from a "bifurcation" where Large Language Models (LLMs) achieve superhuman performance on benchmarks but fail at practical automation tasks (e.g., autonomous drive-throughs, accounting).
- This discrepancy is attributed to the industry's exclusive optimization for "assistance" (chatbots) rather than "automation."
- RLHF (Reinforcement Learning from Human Feedback) is identified as a primary cause of this limitation, as it inherently requires a human-in-the-loop for verification, preventing true machine-native automation.
- Current models are described as "unreliable intelligence," functioning as "Einstein 95% of the time and Mr. Bean 5%," which renders them unusable for high-stakes decision-making.
The "Jevolution" and Strategic Pivot
- The speaker proposes a shift from RLHF to optimizing for "machine-native" tasks, prioritizing extreme, narrow-focused reliability over general capability.
- The Bitterest Lesson: Beyond Sutton's lesson on compute vs. algorithms, the speaker argues that "doing the right task" is the single most critical factor for performance gains.
- Historical precedent: The shift from masked language modeling (BERT) to next-token prediction (GPT) was a task-specific breakthrough, not an algorithmic one, despite similar compute constraints.
- RLHF data was a more significant bottleneck than model size; even small models achieved GPT-3/4 level instruction following with the right task design.
- Reliability as the Product: The core value proposition of "Jev" is providing a foundation for "smart software" that is reliable enough to handle expensive decisions without human oversight.
Software Transformation: The "Fourth Logic Gate"
- The Blue LED Analogy: The speaker argues that adding AI to software is akin to adding the blue LED; it is not a 33% improvement in representational ability but a foundational shift enabling entirely new categories of functionality.
- Semantic Logic: AI introduces a fourth semantic logic gate, allowing software to be programmed with "common sense," "intention," and "human understanding" rather than just binary logic.
- New Capabilities: This shift enables:
- Replacing brittle regex with semantic declarations (e.g., validating "real looking" email addresses).
- Real-time, semantic-dependent type systems.
- Interfaces that interpret user "meaning" rather than executing hard-coded commands.
- Call for "Software Research": The speaker urges the community to move beyond "vibe coding" and "LLM wrappers" toward rigorous software engineering principles:
- Decomposition and multi-stage verification.
- Explicit state management (moving away from global KV cache state).
- Encouraging "cursed software" and playful experimentation to discover unforeseen utility.
Specific Technical and Architectural Proposals
- Agent Memory & Efficiency: Critiques the current O(N) memory cost of KV caches; proposes implementing MapReduce-style memory lookups and shared state across multiple agents to enable continual learning and pattern recognition.
- Principle-Agent Problem: Suggests that encoding the "spirit" of rules rather than just the "letter" could solve societal and organizational alignment issues, reducing fraud and waste.
- Security: Proposes that real-time, smart security (preventing sandbox escapes, etc.) is feasible if intelligence is fast, cheap, and reliable enough to run constantly.
- Stop Slop: Identifies AI unreliability as the root cause of "slop" (low-quality output), which thrives when bad actors externalize the costs of verification onto others.
Future Outlook & Convergence
- The speaker predicts a future where the question shifts from "can we automate this?" to "how much work is it to automate this?"
- The industry is urged to break "LLM Stockholm syndrome" and stop treating AI as a tool for generating text, instead viewing it as a primitive for building robust, autonomous software systems.
- The goal is to empower developers to explore new frontiers of software, similar to the early, chaotic, and innovative days of the internet.