Other
Can AI make society and government smarter? (article by Zershaaneh Qureshi)
- AGI development may compress a century of societal progress into a single decade, creating a critical window where decisions must be made in months rather than years to avoid existential stakes.
- A future labor market could be reshaped where AI systems, rather than humans, drive most economic progress and develop advanced technologies, including weapons, faster than humans can assess risks.
- Geopolitical tensions regarding control or access to advanced AI benefits could escalate into serious conflict, necessitating robust coordination and verification mechanisms.
- Current investment in AI decision-making tools is negligible compared to the billions directed toward broadly capable agents, yet even minor accelerations in these safety tools could be consequential for preventing catastrophe.
- Specific tools proposed include AI fact-checkers, forecasting systems, moral reasoning aids, negotiation simulators, verification monitors, and structured transparency systems to address epistemic and coordination failures.
- A gap exists between societal capability to build decision-support tools and their actual deployment; closing this lag is critical to prevent arriving at capabilities only after AGI is already present.
- While some risk of deliberate misuse or power grabs exists, the assessment is that these tools are more likely to prevent bad outcomes than enable them, provided they are widely accessible to prevent any single group from gaining dangerous advantages.
- Approximately a few hundred thoughtful and entrepreneurial individuals are needed to work on this path, as it offers outsized impact for those with strong judgment in ambiguous, non-mature fields.
- Work in this area may lack commercial incentives for predicting non-profitable events or ethical reasoning, suggesting that counterfactual impact is higher for those not focusing on already commercialized applications.
- Development strategies could include founding new projects, designing benchmarks, managing datasets, building infrastructure, and integrating tools into existing institutional processes to remove adoption barriers.
- Individuals can position themselves for this future by building networks in AI safety, gaining expertise in fields like forecasting or diplomacy, joining key institutions, or founding non-harmful technology companies to practice founding skills.
- The outlook acknowledges significant uncertainty regarding which specific work will be most useful, with the potential for some efforts to have little impact or negative effects if specific misuse risks are not factored in.
- Progress on safety-promoting technologies may be easier to accelerate than slowing down dangerous capabilities, as it does not require agreement from officials or corporations.
- Researchers hold varying degrees of optimism about the feasibility of expanding the workforce in this area, with some suggesting the current assessment may underrate the potential for many more people to be involved.