Interview
#9 - Christine Peterson on how insecure computers could lead to global disaster, and how to fix it
- Christine Peterson, co-founder of the Foresight Institute and coiner of the term "open-source software," discusses the evolution of the effective altruism (EA) movement by comparing it to the idealistic, young space settlement communities of the 1980s.
- Early EA activists, like those in the space settlement movement, prioritized environmental concerns and resource scarcity, but the movement has shifted focus toward existential risks from new technologies like AI, nanotechnology, and longevity.
- Peterson advises that the most intelligent and ambitious EA practitioners should not focus exclusively on easily measurable, near-term goals like bed nets, but should instead target hard, abstract, high-leverage problems with long time horizons (20–60 years) that offer the potential for civilization-scale impact.
- Foresight Institute recently convened a satellite workshop for AI researchers to address the trend of shortening AGI timelines, with some experts now estimating a non-trivial chance of AGI emerging in under 10 years.
- The consensus among AI researchers attending the workshop is that AI development is accelerating due to massive for-profit investment and the immediate profitability of early-stage AI applications, rather than purely because the problem is easier than anticipated.
- Foresight Institute focuses on minimizing downsides of advanced technologies (nanotech, AI, longevity) because the for-profit sector prioritizes benefits and government action is currently too slow to address these emerging threats.
- Atomic precision manufacturing (nanotechnology) is technically feasible but faces significant skepticism; the debate was historically fueled by Richard Smalley's concern that public discussion of "gray goo" risks would stifle progress, though the scientific community now generally accepts the physical possibility of the technology.
- Computer security is identified as a critical, immediate existential risk because modern civilization relies on insecure software that is vulnerable to automated AI probing and potential catastrophic failure of essential infrastructure like electricity and water.
- Secure software is achievable through paradigms like the formally verified SEL4 operating system or blockchain technologies, but it requires building security into the design phase rather than patching insecure legacy systems like Windows or macOS.
- Peterson advocates for a 10-year global deadline for releasing all known vulnerabilities held by the NSA to force a paradigm shift in the software industry toward building fundamentally secure systems from the ground up.
- Aging is described as a global priority that accounts for more lost human life years than any other disease, with the added leverage that it preserves intellectual capital within the EA movement and disproportionately burdens developing nations that cannot afford high healthcare costs for the elderly.
- The strategy for longevity research favors a "repair" approach (fixing accumulated cellular damage) over prevention, potentially yielding results within 20–30 years, with cryonics serving as a backup "worst-case" scenario for those who die before these cures arrive.
- Cryonics is noted as a low-cost insurance strategy (a few hundred dollars/year for young people) with high technical probability of eventual success, though societal disruption remains a primary risk factor for preserving suspended bodies.
- Peterson argues that while general technological acceleration is beneficial, the primary goal should be to accelerate defensive technologies (security, longevity) relative to offensive ones, as offense is often easier to develop than defense.
- Personal sustainability is a key theme for long-term impact; altruists are advised to prioritize physical health, financial stability, and finding a supportive life partner to prevent burnout and maintain decades of productive work.
- The advice for life planning includes taking regular vacations (2–3 weeks annually) to recharge, avoiding overworking in one's 20s to the detriment of long-term stamina, and recognizing that a balanced life supports greater cumulative good.
- Starting a small, nimble nonprofit is encouraged for those passionate about neglected problems, as it avoids the political inertia of large organizations, provided the group maintains high behavioral standards and stays small enough to move quickly.
- For aspiring EAs, computer science and physical sciences (chemistry, biology) are recommended as high-leverage academic paths that provide high earning potential for "earning-to-give" models while building expertise in critical fields.
- Fundraising is highlighted as a critical, high-impact skill for EAs, offering a way to connect with idealistic high-tech workers and cryptocurrency holders who can support causes while having personal constraints on full-time altruism.
- Networking and mentorship are best achieved through volunteering to gain trust, offering reciprocal help (e.g., transcribing interviews) to build strong professional relationships with experienced leaders in the field.