newsfilter
← Interviews
AI Engineer23 min

I love you so jevving much — Diogo Almeida, TypeSafe AI

Diogo Almeida, Jev

Conference Presentation
  • Core Diagnosis of Current AI State

    • The field suffers from a "bifurcation" where Large Language Models (LLMs) achieve superhuman performance on benchmarks but fail at practical automation tasks (e.g., autonomous drive-throughs, accounting).
    • This discrepancy is attributed to the industry's exclusive optimization for "assistance" (chatbots) rather than "automation."
    • RLHF (Reinforcement Learning from Human Feedback) is identified as a primary cause of this limitation, as it inherently requires a human-in-the-loop for verification, preventing true machine-native automation.
    • Current models are described as "unreliable intelligence," functioning as "Einstein 95% of the time and Mr. Bean 5%," which renders them unusable for high-stakes decision-making.
  • The "Jevolution" and Strategic Pivot

    • The speaker proposes a shift from RLHF to optimizing for "machine-native" tasks, prioritizing extreme, narrow-focused reliability over general capability.
    • The Bitterest Lesson: Beyond Sutton's lesson on compute vs. algorithms, the speaker argues that "doing the right task" is the single most critical factor for performance gains.
      • Historical precedent: The shift from masked language modeling (BERT) to next-token prediction (GPT) was a task-specific breakthrough, not an algorithmic one, despite similar compute constraints.
      • RLHF data was a more significant bottleneck than model size; even small models achieved GPT-3/4 level instruction following with the right task design.
    • Reliability as the Product: The core value proposition of "Jev" is providing a foundation for "smart software" that is reliable enough to handle expensive decisions without human oversight.
  • Software Transformation: The "Fourth Logic Gate"

    • The Blue LED Analogy: The speaker argues that adding AI to software is akin to adding the blue LED; it is not a 33% improvement in representational ability but a foundational shift enabling entirely new categories of functionality.
    • Semantic Logic: AI introduces a fourth semantic logic gate, allowing software to be programmed with "common sense," "intention," and "human understanding" rather than just binary logic.
    • New Capabilities: This shift enables:
      • Replacing brittle regex with semantic declarations (e.g., validating "real looking" email addresses).
      • Real-time, semantic-dependent type systems.
      • Interfaces that interpret user "meaning" rather than executing hard-coded commands.
    • Call for "Software Research": The speaker urges the community to move beyond "vibe coding" and "LLM wrappers" toward rigorous software engineering principles:
      • Decomposition and multi-stage verification.
      • Explicit state management (moving away from global KV cache state).
      • Encouraging "cursed software" and playful experimentation to discover unforeseen utility.
  • Specific Technical and Architectural Proposals

    • Agent Memory & Efficiency: Critiques the current O(N) memory cost of KV caches; proposes implementing MapReduce-style memory lookups and shared state across multiple agents to enable continual learning and pattern recognition.
    • Principle-Agent Problem: Suggests that encoding the "spirit" of rules rather than just the "letter" could solve societal and organizational alignment issues, reducing fraud and waste.
    • Security: Proposes that real-time, smart security (preventing sandbox escapes, etc.) is feasible if intelligence is fast, cheap, and reliable enough to run constantly.
    • Stop Slop: Identifies AI unreliability as the root cause of "slop" (low-quality output), which thrives when bad actors externalize the costs of verification onto others.
  • Future Outlook & Convergence

    • The speaker predicts a future where the question shifts from "can we automate this?" to "how much work is it to automate this?"
    • The industry is urged to break "LLM Stockholm syndrome" and stop treating AI as a tool for generating text, instead viewing it as a primitive for building robust, autonomous software systems.
    • The goal is to empower developers to explore new frontiers of software, similar to the early, chaotic, and innovative days of the internet.