Safety in Numbers: Keeping AI Open

Foundational Scaling Law Breakthroughs

• The 2022 "Chinchilla" paper co-authored by Arthur Mensch (DeepMind), Guillaume Lampel, and Timothée Lacroix (Meta) debunked the consensus that model size alone dictated performance, establishing that optimal scaling requires balancing compute between model parameters and dataset size.
  • The paper demonstrated that the previously dominant "Gopher" scaling law (scaling model size 3.5x for a 4x compute increase) was mathematically suboptimal.
  • The new empirical consensus dictates a 2x increase in both model size and dataset size for every 4x increase in compute capacity.
• The "Llama" project at Meta built upon these laws by prioritizing inference efficiency over pure training optimality, effectively "overtraining" models to compress them into more efficient inference states.
  • This approach proved that spending more compute during training reduces latency and cost during inference for a given performance level.

Mistral and the "Mixtral" Architecture

• Former DeepMind and Meta researchers founded Mistral AI in June 2023 to develop small, high-performance teams focused on open-source distribution, releasing the Mistral 7B model in September 2023.
• The newly released "Mixtral" model utilizes a Sparse Mixture of Experts (MoE) architecture to decouple model capacity from inference cost.
  • The model contains 46 billion total parameters but executes only 12 billion parameters per token by routing tokens to a subset of 2 experts out of 8 available per layer.
  • This architecture delivers performance parity with Llama 2 70B while being approximately six times cheaper or faster to run.
• Implementation challenges for MoE architectures included solving mathematical training stability and optimizing hardware communication constraints to manage tokens moving between experts.

Strategic Positioning on Open Source vs. Closed Models

• Mistral operates on the philosophy that pre-trained models should be neutral mathematical tools, similar to programming languages, rather than the regulated entities themselves.
  • The company argues that safety and bias should be regulated at the application level (e.g., healthcare diagnostics) rather than the foundational model level.
  • They contend that open-sourcing models accelerates safety by allowing the global community to "red team" systems, a process more robust than relying on a single company's internal security.
• The company forecasts a closing performance gap between open-source and proprietary models, estimating the current gap is approximately six months and shrinking faster due to community-driven modifications.
  • Community contributions, such as extending context windows to 128k and adding visual capabilities, have already been integrated into the Mistral ecosystem.
• Mistral maintains that while training costs remain high, the field will converge to a state where open-source models match proprietary performance, citing Linux as a precedent for open-source dominance in infrastructure.

Future Trends in AI Development

• The primary bottleneck for the next generation of AI is identified as data efficiency and the ability of models to perform complex, multi-step reasoning.
  • Current training processes are estimated to be roughly 100,000 times less efficient than biological brains.
  • Future innovation will focus on adaptive compute strategies where models allocate processing power dynamically based on problem complexity.
• Long-term predictions suggest a shift toward specialized, task-specific small models embedded within larger application systems to minimize latency and cost.
  • Developers will increasingly deploy swarms of smaller, fine-tuned models rather than single massive models for specific sub-tasks within an application.
• The company urges developers to utilize open-source stacks on local hardware (e.g., MacBook Pro) to build specialized applications, emphasizing that the tooling stack is now mature enough for this shift.

Regulatory and Industry Perspective

• Arthur Mensch advises policymakers against regulating "flops" (compute metrics) as a proxy for model performance, advocating instead for independent evaluation frameworks and application-level oversight.
• The team distinguishes between the "math" (the neutral model) and the "system" (the application), arguing that regulation should target the latter to ensure safety without stifling fundamental research.
• They note that closed-source APIs are paradoxically more vulnerable to adversarial exploitation via fine-tuning because the community lacks the transparency to audit internal breaches.