#21 - Holden Karnofsky on times philanthropy transformed the world & Open Phil’s plan to do the same

Organizational Structure and Mission

• The Open Philanthropy Project (Open Phil) is a collaboration between the Good Ventures foundation and the Open Philanthropy organization, with Good Ventures as its primary funder.
• Open Phil operates on a "hits-based" philosophy, similar to venture capital, aiming to fund high-risk, high-reward projects where a few massive successes justify many failures.
• Current annual giving ranges between $100 million and $200 million, with a long-term goal to give away the vast majority of donors' wealth within their lifetimes rather than maintaining an endowment.
• The organization focuses on causes that are important, neglected, and tractable, deliberately excluding global health and development (relying instead on GiveWell's recommendations) to explore other high-impact avenues.

Core Focus Areas

• U.S. Criminal Justice Reform: Driven by evidence that the U.S. over-incarcerates with little public safety benefit; viewed as politically tractable due to bipartisan support for cost reduction.
• Farm Animal Welfare: Focused on reducing suffering in factory farming through corporate campaigns (e.g., fast food and grocery chains), leveraging high cost-effectiveness and neglectedness.
• Global Catastrophic Risks: Includes biosecurity, pandemic preparedness (specifically regarding synthetic biology), and risks associated with advanced artificial intelligence.
• Scientific Research: Supports "moonshot" science, such as gene drives for malaria eradication, and aims to build academic fields (e.g., AI alignment) that lack sufficient scholarly attention.
• Effective Altruism Community: Supports the broader movement, including funding 80,000 Hours, to improve the quality of global priorities research and grant-making.

Methodological Approaches and Decision Frameworks

• Cause Prioritization: Uses the criteria of importance, neglectedness, and tractability, though acknowledges the difficulty of quantifying value across different worldviews (e.g., human welfare vs. animal welfare vs. long-term future).
• The "Last-Dollar" Question: Evaluates grants by comparing the value of a specific grant to the value of the last dollar of funding that would otherwise be spent, though this is complicated by philosophical disagreements on moral weighting.
• 50-40-10 Rule: A grant approval framework where 50% of the portfolio requires full internal buy-in, 40% is "okay" if the reasoning is visible but not fully agreed upon, and 10% is "discretionary" (fast-tracked with minimal review).
• Field Building: Prioritizes funding the creation of expert communities in nascent fields (e.g., AI safety) rather than just funding specific interventions, betting that a robust field will navigate risks better.
• Critique of Academia: Open Phil argues academia is optimized for frontier knowledge creation rather than synthesizing existing evidence for policy or pragmatic decision-making, leading to gaps in areas like critical evidence reviews for policy.

Artificial Intelligence Safety Strategy

• Risk Assessment: Views the development of transformative AI within the next 20 years as having at least a 10% probability, posing risks of "misuse" (concentration of power) and "accident" (misaligned objectives).
• Intervention Focus: Funds field building to create a large community of researchers working on AI alignment, reward learning, and adversarial robustness.
• Board Engagement: Holden Karnofsky serves on the OpenAI board to influence safety practices from within, supporting the lab's efforts while maintaining an arm's-length evaluation of their commitment to safety.
• Hiring Specialization: Plans to hire experts to map AI technical agendas, develop AI timelines/probabilities, and analyze geopolitical implications of AI capabilities.

Hiring and Organizational Culture

• Recruitment Process: Relies heavily on work trials and "simulated" work periods to assess candidates, viewing standard interviews as unreliable.
• Target Roles: Currently hiring for generalist research analysts, three specialist AI risk researchers (technical, policy/strategy, and timelines), an operations associate, a grants associate, a general counsel, and a director of operations.
• Culture Values: Emphasizes "truth-seeking" over agreeableness; staff are expected to push back on management, rigorously critique their own reasoning, and accept that projects may be dropped if they fail to prove valuable.
• Compensation Philosophy: Salaries are competitive with the nonprofit sector and designed to cover a good standard of living in San Francisco, but significantly lower than for-profit alternatives to ensure candidates are motivated by the mission rather than money.
• Career Trajectory: The research analyst role serves as a training ground for future program officers and leaders, with many current staff starting as interns or analysts before advancing.

Historical Lessons and Philanthropic Theory

• Historical Precedents: Cites the Green Revolution and the invention of the birth control pill as major philanthropic "hits" that occurred because governments were not prioritized or capable of funding such neglected or edgy research.
• Lessons from Failure: Notes that fields like nanotechnology and cryonics were sometimes stifled by philanthropic hype that alienated the scientific community, a pitfall Open Phil seeks to avoid in AI research.
• Radical Empathy: A guiding principle involves expanding the circle of moral concern to include marginalized populations, such as people of color in the justice system, the global poor, and farm animals.
• Expert Consensus vs. Contrarian Bets: The organization seeks to be contrarian only when the most knowledgeable experts in a field support the contrarian view, rather than going against consensus without expert backing.

Utopia Survey and Future Vision

• Empirical Study: Karnofsky conducted a survey on various literary and constructed utopias, finding that abstract descriptions (e.g., "no hunger") score well, but specific societal structures often trigger concerns about totalitarianism.
• Key Finding: "Freedom" was the most highly rated trait across political spectrums, while detailed government control schemes were the least favored, suggesting a difficulty in articulating a desirable future without it sounding oppressive.
• Conclusion: The survey highlighted the challenge of having coherent, long-term conversations about the future without triggering political or cultural objections to specific implementations of a "perfect" world.