Interview
SB 1047: California's AI bill & its potential to shape US AI policy | Nathan Calvin
- The proposed legislation targets AI models requiring $100 million or more in compute for training, or those fine-tuned from open-source models of that size with an additional $10 million in compute, though no current models meet these thresholds.
- Developers of advanced models must submit comprehensive safety plans, conduct annual audits, publish safety test results, and commit to guardrails preventing harmful use, with liability and fines possible if their model causes or facilitates critical harm.
- Critical harm is legally defined as incidents resulting in mass casualties or damages exceeding $500 million, with liability applying to developers who fail to exercise reasonable care under existing tort negligence standards.
- Safety and compliance costs are estimated at single-digit percentages of model training costs, with fine-tuners spending under $10 million exempt from direct responsibilities while retaining the original developer's liability.
- The bill introduces protections for whistleblowers and includes exemptions for open-source models where the developer lacks control over the model's shutdown capabilities.
- Legislative timing anticipates a vote following the August 31st deadline for bills to pass both houses of the California State Assembly, with the possibility of a governor signing it followed by a delayed effective date.
- Future risk scenarios include the emergence of models meeting the 10^26 FLOPs threshold around 2025, where the bill may be insufficient if algorithmic improvements allow for cheaper training of highly capable systems.
- The statute is designed with flexibility by referencing NIST standards and the concept of "reasonable care" to allow for future interpretation, avoiding strict liability regimes while establishing a legal framework for existing tort standards.
- There is a strategic expectation that similar regulations in large markets like New York, Texas, or Florida could create a "Brussels effect," prompting global process changes if companies are unwilling to lose access to major jurisdictions.
- Long-term legal risks involve potential federal preemption if Congress passes nationwide AI regulations, though state legislation is viewed as having more longevity than executive orders subject to Supreme Court limitations.
- Support for the bill is reported at three-quarters among Californians, with proponents arguing it serves as a necessary, modest step to prevent more severe and less nuanced regulation following a catastrophic event.