Worldview diversification and how big the future could be | Ajeya Cotra

Organization & Role Adjustments at 80,000 Hours

• Host Role Shift: Rob Wiblin will shift focus to produce more interviews after Arden Kaler assumes responsibility for written work and the website.
• Audience Growth Strategy: The organization aims to reach 10 to 100 times its current subscriber base, relying on listener referrals via messaging apps to identify new potential listeners.
• Survey Insights: A user survey revealed the podcast is a primary life-changing resource for many, prompting the decision to double down on content production.

Open Philanthropy's Worldview Diversification Framework

• Three Core Worldviews: Open Philanthropy currently allocates funds across three primary philosophical "worldviews":
  • Long-termism: Focuses on preserving the option for a vast future (e.g., space colonization) and reducing existential risks.
  • Near-termism (Human-centric): Prioritizes helping people alive today (e.g., poverty reduction) while caring about the future in a "commonsensical" rather than philosophical way.
  • Near-termism (Animal-inclusive): Prioritizes reducing animal suffering on factory farms, arguing that the sheer scale of animals creates a larger "moral pie" than the human-centric view.
• Divergence on Strategy:
  • Long-termists prioritize philosophical rigor and are willing to accept "weird" conclusions (e.g., astronomical waste) to maximize moral value.
  • Near-termists prioritize quantitative rigor, empirical feedback loops, and avoiding "fanaticism" or high-risk bets where the return on investment is hard to verify.
• Decision-Making Mechanisms:
  • Credence Allocation: Allocating funds proportional to the probability assigned to each worldview (e.g., if long-termism is 10% likely, it gets 10% of the budget).
  • Fairness Agreements (Veil of Ignorance): Simulating a scenario where funders do not know which worldview they represent to negotiate a fair split of resources before observing the world, preventing a "winning" worldview from taking all capital if it turns out to be correct.
  • Outlier Opportunities: Allocating bonus funds to worldviews that demonstrate unexpectedly high cost-effectiveness in the empirical data.
• Excluded Worldviews: The organization excludes "charity starts at home" or nationalism, prioritizing impartiality and global scale, though "economic growth" and "civic institution improvement" are under consideration.
• Science & Policy: A portion of funding supports basic science and policy not based on a specific philosophical "bet" but on the historical track record of these fields driving human progress (e.g., Progress Studies).

Empirical Research on Space Colonization & The Future

• Astronomical Waste Argument: The long-termist case relies on the possibility of colonizing space to create a robust, low-risk future with a vastly larger population than Earth can support.
• Biological vs. Digital Colonization:
  • Robust space colonization likely requires uploading humans/computers to silicon-based media rather than sending biological bodies, as biological bodies require massive ships and are fragile against space debris.
  • Self-replicating computer probes (e.g., "Eternity in Six Hours" paper) are seen as the most technically feasible path to spreading life across the observable universe.
• Risk Factors:
  • Biological colonization of other star systems is deemed "dicey" due to the energy requirements for self-sustaining ships and the risk of disintegration by interstellar dust at relativistic speeds.
  • Software and AI capabilities are identified as the primary bottleneck for self-replicating probes, rather than hardware or propulsion.
• Philosophical Challenges to the Future:
  • The Doomsday Argument: Suggests that finding oneself early in human history implies a higher probability of a short future (doom) rather than a long one, based on self-sampling assumptions.
  • The Simulation Argument: Suggests that if future civilizations run many simulations, the probability of us being in a simulation is high; this would cap the value of the "real" future relative to the present.
  • Current Stance: Open Philanthropy acknowledges these arguments but does not let them prevent funding existential risk reduction; they view the "crazy town" of extreme philosophical implications as a reason to hedge rather than abandon long-termism.

AI Timelines Research (Forecasting Transformative AI)

• Methodology: The report uses "biological anchors," comparing the computational efficiency of the human brain to artificial intelligence systems to estimate training requirements.
• Four Key Variables:
  1. 2020 Training Compute Requirements: Estimated to be roughly $10^{16}$ to $10^{18}$ FLOPS for a transformative model, based on the brain's estimated power and scaling laws.
  2. Algorithmic Progress: Estimated to halve compute requirements roughly every 18–24 months for proxy tasks.
  3. Hardware Efficiency: Projected to follow a trend of halving costs every 2.5 years, slower than historical Moore's Law.
  4. Investment Scaling: Assumed to ramp up rapidly until transformative AI yields economic value, potentially reaching 1% of global GDP.
• Timelines Forecast:
  • Median Estimate: Transformative AI (defined as causing 10x faster economic growth) is expected by 2055.
  • 2036 Probability: 12–15% chance of transformative AI by 2036 (slightly lower than previous 10% "conservative" baseline, but consistent).
  • Century Probability: 70–80% chance of transformative AI occurring this century.
• Shift in Perspective: The research moved from viewing AI risk as a "sudden event" (like a biological pandemic) to a "gradual onrushing tide" of capabilities, with early failures likely to occur before full transformation.
• Scaling Laws: Empirical data suggests data requirements scale sub-linearly (approx. $N^{0.75}$) with model size, meaning larger models are more data-efficient than previously thought.

Financial Strategy: The "Last Dollar" Project

• Core Principle: Capital should be spent where the marginal value of the last dollar spent today equals the expected value of the last dollar spent in the future.
• Near-Termist Allocation:
  • Driven by the "GiveWell Top Charities" benchmark, which offers scalable, high-return interventions (e.g., cash transfers, bed nets).
  • Models suggest a constant percentage of the budget should be given away annually to match the declining returns of opportunities against market returns.
• Long-Termist Allocation:
  • Lacks a "GiveWell equivalent" (a single scalable intervention); instead, funding is directed toward AI and biosecurity.
  • Biosecurity Lower Bound: Open Philanthropy modeled "meta-R&D" for pandemic response (e.g., stockpiling vaccines, rapid detection) as a "GiveWell for long-termism," estimating a cost-effectiveness of $200 trillion per life saved.
  • Conclusion: Long-termism can absorb billions without hitting diminishing returns immediately, but the primary bottleneck remains finding qualified grantees.
• Spending Pace:
  • Near-termists are expected to give away a significant fraction annually (potentially 10%+).
  • Long-termists may concentrate spending in a specific "window" (5–10 years) prior to the development of transformative AI to maximize leverage.

Organizational Culture & Career Advice

• Siloed Structure: Open Philanthropy is organized into small, specialized teams, leading to isolation where colleagues cannot easily read or collaborate on each other's deep-dive reports.
• Workload Stress: Staff face high psychological pressure from writing definitive reports on ambiguous topics without clear "publishable units" or peer feedback on the most critical, messy parts of the research.
• Hiring Outlook:
  • The organization receives thousands of applications for a handful of generalist research roles.
  • Entry Strategy: Candidates are encouraged to publish open-ended research on the Effective Altruism Forum or LessWrong to demonstrate fit, rather than waiting for formal job openings.
  • Skillset: Success requires a balance of "arrogance" (to tackle big questions) and "finicky rigor" (to make small improvements in epistemic clarity).