#17 - Prof Will MacAskill on moral uncertainty, utilitarianism & how to avoid being a moral monster

Community Origins and Growth

• Effective Altruism (EA) emerged from the convergence of three movements: GiveWell (founded by Ellie Hassenfeld and Holden Karnofsky), Less Wrong (Bay Area-based), and Giving What We Can (co-founded by Will McCaskill and Toby Ord).
• Giving What We Can initially encouraged members to pledge at least 10% of income to high-impact charities, with McCaskill and Ord providing early recommendations.
• The movement has grown from a small academic project to a global community with approximately 100 local groups and thousands of members.
• 80,000 Hours was launched in early 2011 to expand the "do the most good" framework beyond charity to career choice and other cause areas, including existential risk and animal welfare.

Organizational Leadership Changes

• McCaskill stepped down as CEO of the Center for Effective Altruism (CEA) after an interim period intended to consolidate projects and update the organization's mission.
• He handed the CEO role to Tara McCauley, citing her competence and his own comparative advantage in academic research over organizational management.
• McCaskill remains a trustee of the CEA, Giving What We Can, and the Center for Effective Altruism.

Current Academic and Research Focus

• McCaskill is finishing a book co-authored with Toby Ord and Krista Bickfist titled "Moral Uncertainty," currently projected for release in 1–1.5 years.
• He is co-founding the Global Priorities Institute at Oxford University, led by Hilary Graves and Michelle Hutchison, to address theoretical questions in EA.
• He teaches a course on utilitarianism within the Oxford undergraduate curriculum.
• His research prioritizes "model uncertainty" (uncertainty about which moral theory is correct) and the "Long Reflection," a proposed era where humanity dedicates tens of thousands of years to resolving fundamental questions of value.

Theoretical Framework: Moral Uncertainty

• Standard decision-making under empirical uncertainty uses expected value reasoning; McCaskill argues this should apply analogously to moral uncertainty.
• Current philosophical practice often treats moral views as fixed "football fan" identities rather than probabilistic credences, which McCaskill deems irrational.
• Key Problems Identified:
  • Intertheoretic Value Comparisons: Difficulty comparing the "weight" of wrongness or goodness across different moral frameworks (e.g., comparing the severity of killing one person vs. failing to save five).
  • Fanaticism Problem: Small credences in views with infinite value (e.g., absolutist rules) can dominate expected value calculations, leading to absurd conclusions.
  • Infectiousness Problems: The presence of incomparability or undefined values in even one moral theory can render the entire expected value calculation undefined.
• Structural vs. Non-Structural Views: McCaskill argues against "structuralist" views that normalize value functions (e.g., standardizing means and variances), which he claims erases differences in the "right-makers" or metaphysical grounds of value between theories.
• Amplified Theories: He proposes that some moral theories are identical in ranking but differ in the magnitude of importance, suggesting that intertheoretic comparison is a question of credence assignment rather than formal normalization.

Practical Decision-Making Implications

• Consequentialism + Rights: Under moral uncertainty, the dominant view suggests a "consequentialism plus rights" framework: maximize good while respecting side constraints (rights) against violations like murder or exploitation.
• Population Ethics: One should adopt a broad view of value, including bringing future people into existence, due to the vast number of potential future lives (potentially trillions).
• Wild Animal Suffering: Even if views on wild animal suffering are uncertain, the sheer scale of suffering suggests it is a high-priority cause area, though less so than existential risk for humanity.
• Value of Moral Information: Because history shows humans have held abominable moral views with high confidence (e.g., Aristotle on slavery), there is a massive "option value" to delaying irreversible actions until more moral truth is known.
• The Long Reflection: Humanity should aim to reduce existential risks to near zero and then engage in a prolonged period of reflection to converge on a shared definition of value, rather than committing to one specific trajectory immediately.
• Nihilism: Even a 90% credence in nihilism does not preclude action, as a 10% credence in moral realism creates a dominance argument to act as if moral facts exist (no downside if false, massive upside if true).

Arguments for Classical Utilitarianism

• Historical Track Record: Early utilitarians (Bentham, Mill) made bold predictions against common sense (e.g., decriminalizing homosexuality, animal welfare) that later became accepted, whereas non-consequentialist contemporaries (e.g., Kant) held regressive views (e.g., white supremacy, opposition to organ donation) that were rejected.
• Harsanyi's Veil of Ignorance: John Harsanyi proved that rational agents behind a veil of ignorance (assuming equal probability of being anyone) would choose utilitarianism to maximize expected utility, contrasting with Rawls' modification which leads to the "maximin" rule.
• Rejection of Personhood: Drawing on Derek Parfit, McCaskill argues that personal identity is a fiction; once the moral distinction between "within a life" and "across lives" is rejected, it becomes permissible to make trade-offs across persons similar to those made within a single life.

Arguments Against Classical Utilitarianism

• Conflicts with Common Sense: Utilitarianism conflicts with intuitions regarding side constraints (e.g., killing one to save five) and the "permissible" realm (it demands extreme beneficence, prohibiting self-spending).
• Intrinsic Value: Utilitarianism does not treat equality, justice, or environmental preservation as intrinsically valuable, only instrumentally so.
• Demandingness: It imposes extremely high obligations on affluent individuals to donate resources or dedicate careers, far exceeding common moral expectations.

Critiques of the Academic Profession

• Publication Delays: The peer review process can take up to five years, hindering the rapid dissemination of urgent moral insights.
• Incentive Misalignment: Academics are incentivized to pursue novelty and "intellectual chops" (often in abstract decision theory) rather than practical, high-impact applied ethics.
• Admissions Bias: Oxford University's admissions interviews are less predictive of success than standardized tests (an 8:1 ratio), yet decisions are based heavily on interviews, perpetuating class and educational bias.
• Bureaucracy: A significant portion of academic time is wasted on administrative tasks and low-efficiency meetings.

Community Dynamics and EA Evolution

• Core vs. Public Perception: There is a significant lag between the "core" EA view (long-termism, existential risk, political action) and media/wide public perception (short-term charity, earning to give, apolitical).
• Marketing Mistakes: McCaskill admits that early "aggressive marketing" of earning to give created a lasting stereotype that 80,000 Hours is solely about fundraising, despite later clarifications.
• Diversity: The community is urged to improve racial, gender, and epistemic diversity to avoid groupthink and ensure broader appeal, particularly by limiting jargon and fostering a welcoming culture.
• Controversial Topics: The community must balance the responsibility to explore controversial ideas (e.g., wild animal suffering) with the need to avoid gratuitous offense or misrepresentation.

Best Arguments Against Effective Altruism

• Justice vs. Beneficence: The strongest theoretical critique is that wealthy nations or individuals have prior obligations to rectify historical injustices (e.g., colonialism) rather than maximizing generic "goodness" through charity.
• Institutional Focus: Critics argue the community initially over-emphasized individual charity over high-impact political and institutional change.
• Cause Selection Risk: The possibility of "Cause X," a currently unknown cause that is vastly superior to current priorities, represents a fundamental uncertainty in the movement's strategy.

Advice on Careers

• Philosophy vs. Economics: McCaskill recommends economics over philosophy for those seeking impact, citing better job prospects, higher demand in policy/think tanks, and more versatile skills.
• Niche for Philosophers: Philosophy remains valuable for those with a specific passion for the field or those targeting theoretical questions in global priorities research where high-quality academic work is available.
• Risk of "Success Bias": Advice from successful academics often ignores the high failure rate and luck involved, leading to skewed recommendations.