Interview, Panel
AI and our future with Yuval Noah Harari and Mustafa Suleyman
- Within the next five years, Frontier AI model companies are expected to train models over a thousand times larger than current GPT-4 versions, shifting output from single-shot text to generating sequences of actions.
- By 2028, the outlook predicts the emergence of AIs capable of performing independent actions and decisions, such as negotiating via phone calls, establishing supply chains with other AIs, and using APIs to access external knowledge bases without direct human input.
- Technological advancements may eventually lead to the creation of new ideas without direct human input, representing a historical transition where control shifts from human-dominated systems to entities with independent decision-making capabilities.
- AI systems are viewed as tools applied in narrow settings lacking inherent agency by default, yet they possess potential for high intelligence and agency that is not naturally emergent but requires deliberate engineering, precise constraints, and careful building to prevent.
- Preventing more intelligent AI systems from developing agency is considered a very difficult challenge that may not be impossible, particularly as AI systems may move toward autonomy without being fully autonomous due to the inability of companies to fully predict resulting powers.
- Risks associated with relying on creator companies to manage safety and errors will increase as AI capabilities grow, and while companies will attempt to pressure test systems to make them fail, it is not possible to test for every scenario in advance.
- High-risk capabilities including autonomy and recursive self-improvement should be taken off the table under a precautionary principle, with frameworks like the EU AI Act serving as a risk-based model to ban certain capabilities when thresholds are exceeded.
- The governance challenge over the next couple of decades involves ensuring humans can impose constraints on the development trajectory to contain the wave of AI, a task that becomes almost impossible if humans remain divided and engaged in an arms race compared to a unified front.
- New institutions will be required with human, economic, and technological resources to effectively regulate and contain AI development, relying on public trust to function, as the situation is compared to an alien invasion where beings arrive in five years with unknown outcomes like solving cancer and climate change.
- The West is expected to collectively establish and stand behind its values rather than participating in a race to the bottom, and if some countries do not join a coalition of the willing regarding AI regulation, others will still proceed with regulations such as banning bots impersonating people.