#24 - Stefan Schubert on why it’s a bad idea to break the rules, even if it’s for a good cause

Epistemic Modesty and Culture

• Stefan Schubert identifies a systemic tendency in Effective Altruism (EA) toward overconfidence, advocating for "epistemic modesty" to compensate for human cognitive biases.
• He proposes a distinction between "impressions" (personal evidence-weighted views) and "beliefs" (the position held after deferring to others), suggesting individuals act on the latter while retaining the former.
• Schubert acknowledges a tension in his own practice: while he supports the theoretical "monk-like" approach of total deference, he finds it difficult to implement without abandoning all personal agency in real-time decision-making.
• A significant cultural risk identified is the "tribalism" inherent in politics, which can erode the rational, accurate-belief culture EA aims to cultivate; the movement should prioritize long-term visions that unite rather than divide.
• Schubert predicts a continued shift away from EA being characterized as a "youth movement" toward one defined by deep specialization and subject-matter expertise in fields like AI safety and biosecurity.

Strategic Shifts and Organizational Trends

• The movement's focus has pivoted from addressing funding gaps to addressing talent gaps; Ben Todd previously predicted this shift would make funding less of a constraint than talent acquisition.
• Schubert observes a slowdown in the creation of new EA organizations, potentially leading to a "monopoly" dynamic similar to late 19th-century US capitalism, where dominant players like 80,000 Hours and Giving What We Can outgrow new entrants.
• He notes that the "creative destruction" of old projects (e.g., 80,000 Hours ending its membership/pledge program) is occurring too slowly in EA compared to market economies, partly because poor impact is less immediately visible than financial loss.
• Schubert admits past over-optimism regarding the speed and ease of institutional reform and broad policy changes, attributing this to the "typical mind fallacy" and an underestimation of how difficult these specific domains are.
• There is a notable gap in EA's organizational growth in the developing world; Schubert suggests that while interventions exist (e.g., Charity Science Health), the path to scaling is difficult, and the reasons for the lack of local organizations require further investigation.

Research on Norms, Considerateness, and Reversibility

• In the paper "Considering Considerateness," Schubert argues that EA members should strictly follow social norms (honesty, friendliness, loyalty) because reputational damage to the community outweighs the immediate marginal gains of breaking them.
• The paper posits that in a tight-knit community, the negative externalities of dishonesty scale with the number of members, making the "cost" of violating norms significantly higher for larger groups than for small ones.
• A tension exists between "niceness" and "honesty"; Schubert argues that in cases of conflict, honesty should generally trump niceness, as misleading someone about the efficacy of their work is a form of inconsiderateness that harms their long-term interests.
• The essay "Hard to Reverse Decisions Destroy Option Value" highlights that EA must avoid irreversible commitments, such as deep entanglement in partisan politics, which could permanently damage the movement's reputation and flexibility.
• Schubert cites the analogy that reputation (like creditworthiness) is "easy to lose and very hard to win back," making the preservation of a norm of integrity a high-priority strategic goal.

Cause Neutrality and Future Directions

• Schubert clarifies misconceptions about "cause neutrality," distinguishing it from "cause divergence" (working on many causes); one can be neutral to new evidence while still focusing on a single best cause.
• He distinguishes between being cause neutral (open to changing one's mind) and being "cause agnostic" (uncertain which cause is best), arguing that high confidence in a specific cause (like existential risk) is compatible with neutrality if one remains open to updating.
• The concept of "Cause X" (a currently unknown, highly effective cause) has failed to gain traction as a focal point for discovery, leading to a consensus that known existential risks may currently represent the most impactful frontier.
• Schubert rejects the conflation of "existential risk" solely with technical safety (AI, biosecurity), noting it also encompasses broader social and institutional resilience strategies.
• He maintains a stance of deep epistemic humility, suggesting he rarely assigns more than 80% confidence to complex practical questions, yet acknowledges the necessity of acting decisively despite this uncertainty.