'Production Ready RAG' by David Leconte from DataStax | RAISE Summit 2024 | Paris

Speaker & Context

• David Leconte, Solution Engineer at Datastack, presented on "production-ready RAG" at a summit.
• Presentation focused on transitioning RAG applications from lab experiments to stable, supported production environments.

Core Challenges in RAG Production

• Library Fragmentation: Implementations rely on hundreds of asynchronous open-source libraries that are not designed to work together, leading to incompatibility and breakage.
• Support & Liability: Open-source frameworks often lack enterprise-grade support, forcing teams to rely on community forums (e.g., Stack Overflow) for critical production fixes.
• Compliance Risks: Regulated industries (e.g., banking) struggle to validate and certify open-source stacks for security and audit requirements.
• Operational Blindspots: Enterprises face difficulty predicting, detecting, and preventing application failures caused by dependency updates or API changes.

Datastack Solution: RAG Stack

• Curated Integration: Bundles essential frameworks (LangChain, LlamaIndex) into a pre-tested, compatible set of libraries to eliminate "dependency hell."
• Daily Validation: Runs thousands of automated tests daily, including security scans for SQL injection and other cyber risks.
• Enterprise Support: Provides a single point of contact with 24/7, follow-the-sun support for production issues.
• Simplified Deployment: Enables one-command installation (pip install ragstack) to deploy verified configurations.
• Industry Validation: The framework is used by Harrison Chase (creator of LangChain) for internal testing due to its reliability.
• Innovation vs. Stability: Allows organizations to stay current with open-source innovation without the risks associated forking codebases or reinventing the wheel.

Database Performance: AstraDB

• Architecture: Powered by Apache Cassandra, offering a serverless, multi-cloud, multi-region vector database.
• Latency: Achieves 10-millisecond latency for queries, ensuring the database is not a bottleneck during LLM inference.
• Benchmarking: Claims to be 74% faster than dedicated competitor vector databases in retrieval speed.
• Accuracy: Maintains superior F1 scores (relevancy) compared to competitors like Pinecone, balancing speed with retrieval quality.
• Cost Efficiency: Demonstrates better Total Cost of Ownership (TCO) than specialized vector stores.
• NVIDIA Integration: Optimized for NVIDIA Nemo microservices to accelerate embedding generation at lower costs.

Development Tooling: Longflow

• Acquisition: Datastack recently acquired Longflow to enhance its offering.
• Functionality: Provides a UI and workflow engine that automates code generation for RAG systems.
• Outcome: Reduces development time by automating code correctness and enabling API-driven deployment in a single click.

Strategic Conclusion

• Datastack positions itself as a de-risking partner for enterprises requiring low latency, high throughput, and strict compliance in GenAI deployments.
• The company leverages 15+ years of experience with enterprise data challenges to solve the specific complexity of generative AI production.