newsfilter.io
Conference Presentation

Jason Mars

a16zJason Mars, Lingjia Tang, Johann Haswell, Mike
Watch on YouTube →

Clarity Lab & Sirius Overview

  • Jason Mars, an assistant professor at the University of Michigan, co-runs the Clarity lab with Lingjia Tang, managing a team of 18 PhD students and postdocs.
  • The lab has achieved high research productivity, publishing at top-tier venues and exploring unconventional hardware cooling methods like embedding candle wax in servers.
  • The lab developed Sirius (later renamed Lucida by Apple), an end-to-end voice and vision personal assistant designed to democratize intelligent systems previously monopolized by major tech giants.
  • Apple requested the renaming from "Sirius" to "Lucida" after the project gained traction and coverage in the open-source community.

System Architecture & Capabilities

  • Lucida supports three distinct query types:
    • Voice Commands: Actions executed directly on the mobile device (e.g., "take a picture").
    • Voice Queries: Questions answered by a QA system drawing from a full Wikipedia corpus.
    • Voice-Vision Queries: Multimodal inputs combining image analysis and voice (e.g., asking "When does this bar close?" while pointing a camera at a bar).
  • The infrastructure is built as a microservice architecture integrating:
    • Speech Recognition: Supporting both legacy Gaussian mixture models and modern deep neural networks.
    • Image Matching: Utilizing the SURF algorithm for feature extraction and knowledge base matching.
    • Question Answering: Built upon the foundation of OpenEFR (a precursor to IBM Watson).
  • A live demo verified the system's ability to answer historical facts, identify building heights via image, and retrieve vehicle specifications through natural language.

Scalability Challenges & Research Findings

  • Transitioning from text-based search to voice/vision personal assistants creates a massive "scalability gap," requiring a potential two-order-of-magnitude increase in data center capacity to handle the workload at scale.
  • 92% of the system's execution time is consumed by compute-intensive machine learning algorithms, including:
    • Speech recognition (Gaussian models/Deep Neural Networks).
    • Natural language processing (Stemmer, Conditional Random Fields).
    • Computer vision (feature extraction and description).
  • The team created a benchmark suite from these algorithms to study accelerator platforms, porting workloads to CPUs, GPUs, Intel Phi, and FPGAs.
  • Leveraging GPUs and FPGAs demonstrated the potential to reduce the latency and resource gap by approximately 10x, though further optimization remains necessary.

Commercialization: Clarity Lab Inc.

  • The research lab spun out Clarity Lab Inc. to transition academic technology into commercial products, with co-founders including Prof. Lingjia Tang and lead students Mike and Johann Haswell.
  • The company offers a "highly intelligent experiences" platform, moving beyond algorithm-as-a-service to provide full end-to-end AI solutions for enterprise partners.
  • Product Portfolio & Partnerships:
    • Gin: A deep learning appliance (GPU box) with a custom software stack, shown to outperform off-the-shelf solutions; published at ISCA.
    • Notion: A sentiment analysis engine integrated into the Notion platform to personalize email interactions by analyzing user history.
    • Lucida for Banking: A domain-specific personal assistant developed for BankJoy (a Y Combinator-backed startup), enabling natural language banking queries like checking daily spending limits or transferring funds via voice.
  • The company secured a license agreement covering four years of future technology improvements exclusively owned by Clarity Lab Inc., ensuring a continuous pipeline of R&D for partners.