AI Engineer7 min
Your KV Cache Can Spot Attacks on Your AI Agents - Ari Azarafrooz, Intrinsec AI
Conference Presentation, Keynote
- Ari Azafrouz, founder of Intrinsic AI, proposes treating the KV cache as a source of security signals rather than solely a throughput optimization mechanism for AI agents.
- The approach addresses growing agent vulnerabilities, including prompt injection, jailbreaks, broken tool calls, sensitive data exposure, hidden backdoors, and model drift.
- Current edge-guarding methods (judge models or classifiers) have blind spots that KV cache analysis can fill by inspecting the internal computation state before token generation.
- The proposed methodology involves three steps: establishing a baseline for normal AI inference traffic, identifying rare KV pages as anomalies against that baseline, and classifying/explaining those anomalies.
- Integration points include:
- Embedding detection logic directly inside inference engines (e.g., via LLMCache) for real-time alerts that can harden future policies or tool calls.
- Integrating with KV cache layers to read stored pages and detect intrusions that do not belong to normal traffic patterns.
- A voice agent case study using Gemma-4-7B (72 baseline clips across four voices, 16 token pages per clip) successfully detected hidden instructions embedded in audio.
- In the voice study, an untrained model using KV pages as features correctly identified personal information, home addresses, location, and travel destinations from the anomaly.
- A text agent case study using Qwen-3-14B (Chat) with Hugging Face ATBench demonstrated that anomaly detection flags meaningful alerts in 7 to 8 out of 10 cases involving attacks ranging from tool description injection to misinformation.
- A key architectural challenge is managing diverse model baselines; fine-tuned models, new releases, and multi-model router systems create incompatible coordinate systems for KV pages.
- Moving to a shared representation space is necessary to avoid the computational prohibitions of training individual classifiers for every model variant.
- Real-time analysis faces performance bottlenecks; for example, compressing a 1,256 token chunk from Qwen-3-14B results in approximately 40MB of data, making streaming and immediate action slow.
- Intrinsic AI is targeting teams building custom inference stacks to treat security as a foundational design decision rather than a post-deployment add-on.