Dr. Walden C. Rhines, Cornami: Scalable Secure AI Deploying LLMs with Fully Homomorphic Encryption

Quantum Encryption Threat and Solution

• Quantum computers with 10,000+ sustainable qubits are predicted to break current internet encryption.
• Fully Homomorphic Encryption (FHE), pioneered at Stanford by Craig Gentry, is identified by the U.S. Department of Defense as the "holy grail of encryption."
• Unlike standard cloud encryption, FHE allows computation on data while it remains encrypted, preventing exposure to plain text.
• FHE enables secure data monetization by allowing data owners to rent encrypted databases without revealing the underlying information.
• The technology eliminates GDPR liability and fines (exceeding €6 billion in Europe) by ensuring data is never stored in plain text at data centers.
• FHE protects both users and model owners in AI by securing encrypted queries and results against open-source model theft.

Barriers to FHE Adoption

• Traditional computing imposes prohibitive overhead: encrypting 4 bytes of data expands to 1.7 megabytes.
• Computational complexity results in a 1:12 million CPU operation ratio; tasks taking 1 second on plain text require 11 days using standard encryption methods.
• Von Neumann architectures suffer from memory bottlenecks and contention, rendering them incapable of scaling FHE workloads.
• Amdahl's Law limits traditional core scaling; adding more cores eventually degrades performance due to synchronization overhead.

Kornami Architecture and Software Strategy

• Kornami prioritized a seven-year software and compiler optimization phase (TrueStream) before implementing hardware, reversing the industry norm of hardware-first development.
• The TrueStream compiler vectorizes applications into parallel streams, exploiting 55 distinct forms of parallelism often unused by competitors.
• The architecture utilizes a "sea of cores" (up to 64 million) that are software-reconfigurable and linearly scalable without memory contention.
• Data flow architecture eliminates repeated memory access, achieving one output per cycle compared to NVIDIA's one output per 16 million cycles for specific workloads.
• The system operates as a "share-nothing" architecture where each memory module operates independently per processor.
• Existing software can run unmodified on Kornami chips via standard interfaces (e.g., PyTorch), with optimization occurring at the primitive layer.

Performance and Efficiency Benchmarks

• Kornami hardware reduced a banking security benchmark from 24 hours (traditional hardware) to 3.7 seconds (equivalent to plain text speed) on four servers.
• Current 16-nanometer Kornami chips deliver approximately 3 orders of magnitude lower cost per FHE operation compared to NVIDIA H100s.
• In multi-user scenarios, Kornami achieves a 10–20x advantage in tokens per second over H100s with significant power savings.
• Current 125-watt chips offer 235x higher performance per token compared to a single NVIDIA core using equivalent silicon area.
• A 128,000-core server configuration demonstrates 20x faster inference and 17x faster zero-knowledge proofs for blockchain applications.
• The architecture supports independent core precision settings ranging from 1 to 128 bits, optimized dynamically by the compiler.

Future Projections and Power Scalability

• Porting the Kornami architecture to 4-nanometer technology (currently unused) is projected to yield a 900x performance increase over current generation H100s.
• Future 4-nanometer implementations are expected to consume less than one-fourth of the power of comparable AI data centers.
• Hypothetical comparison of Elon Musk's Colossus data center (25,000 H100 servers, 150 MW) shows Kornami 4-nm could achieve equivalent performance with only 28 servers (130 kW).
• The technology challenges fears regarding global power capacity constraints for AI, suggesting architectural innovation alone can resolve energy limits.

Commercial Roadmap and Q&A

• The company is currently building and testing second-generation servers with no reported chip defects.
• Beta units are scheduled for customer deployment in the current quarter.
• Volume production and order shipping are targeted for the fourth quarter or potentially September.
• The speakers view FHE as the ultimate privacy solution, rendering other privacy-enhancing technologies unnecessary if FHE is implemented cost-effectively.