Interview, Fireside Chat
Anthropic’s CEO explains why he took on the Pentagon
- The speaker distinguishes their hesitation regarding fully autonomous weapons from "democratic values," attributing the position primarily to the current technological readiness of the models.
- The technology is characterized as "not safe" for specific operational parameters, similar to an aircraft that exceeds its designed altitude or turning speed limits, rather than being universally unsuitable.
- Disputes are not driven by specific past incidents in Venezuela or Iran, which were compliant with the original, "substantially more limited" contract that remained in force for a long period.
- The conflict centers on negotiating a new, broader contract, as the supplier cannot dictate the specific geographic deployment of its services once provided.
- The speaker notes that the Department of War (Pentagon) views such constraints as capricious, arguing that military suppliers must provide trusted tools with "full operational ability" without private sector interference.
- Despite the public friction, the speaker asserts that there is "99% agreement" between the company and the government on broader topics, including areas not yet addressed.
- The potential termination of the contract raises a concern about the "effective nationalization" of AI companies working with the Pentagon if suppliers are granted no operational oversight.
- The issue is framed as a long-term, multi-administration challenge involving Congress and shifting geopolitical contexts, rather than a dispute with a specific current administration.
- A "dual dilemma" is identified where AI empowers private actors to exceed government influence, while simultaneously granting governments (both democratic and autocratic) unprecedented power to vest in the technology.
- The speaker raises two specific objections to fully autonomous weapons:
- Reliability: The technology is not yet ready for autonomous use cases.
- Oversight: The replacement of human soldiers with drone armies creates a normative vacuum where a single individual could control "10 million drones" with no internal military norms to prevent abuse.
- The speaker concludes that the technology is currently "too powerful" to reside solely in the hands of private companies, yet equally too powerful to be entirely controlled by governments without checks.