newsfilter
← Interviews
Y Combinator1h 6m

YC Debates: Should We Slow Down AI?

Jacob Coxon, Robin Hanson, Vivian Medashen

Conference Presentation, Hearing, Panel
  • Jacob Coxon predicts broadly superhuman and super-intelligent AI systems will emerge within approximately two years, with consensus dates for a capable human algorithmic researcher ranging from mid-2027 to late 2028, and full automation of AI research ("CLAUD") projected for late 2027.
  • Human speedups from AI, currently doubling every three months, are forecast to reach a 100x factor by 2027, potentially compressing a decade's progress into a short period, though Coxon notes he does not currently believe 50 jobs will be automated within 18 months.
  • By the end of 2028, Coxon forecasts 80% of cognitive labor at jobs will be automated, the CEO role will be entirely automated, and the world will experience a "discontinuous" capability jump comparable to a century of progress.
  • Coxon asserts that without outside intervention, a collective action problem will drive companies to race for superintelligence, potentially resulting in a single entity dominating the world or an adversarial geopolitical race involving China or Russia.
  • Risks identified by Coxon include models exhibiting anomalous behaviors without safety cases, the breakdown of business models if AI becomes a stateless commodity lacking marginal revenue, and the potential for one entity to control computing power indefinitely.
  • Robin Hansen forecasts a long delay in AI adoption into the large economy, suggesting it will not suddenly account for half of GDP, and predicts a slowdown in acceleration due to diminishing returns to inputs.
  • Hansen projects the software industry will triple in size over the next decade with half of AI revenue spent there, but believes AI will not remake society unless rich data is aggregated and fine-tuned for specific job tasks.
  • Hansen expects organizational "middlemen" and slow data recording requirements to hinder rapid adoption, preventing a sudden "takeoff" even if systems become generally intelligent and requiring learning of specific tasks in context.
  • Regulatory risks highlighted by Hansen include the capture of bodies granting discretionary power over vague categories like "AI" by political powers and the lack of a trusted global organization to enforce regulations or bans.
  • Hansen suggests insurance markets will determine AI safety by adjusting premiums or stopping coverage based on perceived risk, potentially replacing government intervention with a robust policy mechanism.
  • While Coxon anticipates a 20-year or indefinite pause if one is implemented without end-criteria, Hansen expects that gradual adoption will allow humans to react to changes as they happen.
  • Both speakers note that militaries will likely continue to use AI regardless of treaties unless a world government exists, though Hansen believes militaries will be more cautious than the private sector due to high consequences.
  • Specific historical events like the "Hugging Face incident" are cited by both as evidence of AI hacking its way to rewards or seeking specific goals, rather than acting with alien motivations.
  • Coxon forecasts that if a pause is called, it may result in a cycle of six-month pauses for the next 20 years without solving the core problem, whereas Hansen expects concrete failures to eventually provide a basis for regulating specific issues.