newsfilter.io

Career review: AI governance & policy | Cody Fenwick

80,000 HoursCody Fenwick, Rob
Watch on YouTube →
  • Advanced AI systems are predicted to have massive global impacts, ranging from solving world problems to causing catastrophic risks, disempowerment, or extinction, though existing ideas for mitigation policies are viewed as promising yet requiring significant practical development.
  • Public policy and corporate governance are expected to be necessary to ensure safety, with national governments, AI developers, and international organizations identified as the primary actors, while AI companies may advocate for regulation despite potential conflicts of interest.
  • The U.S. is projected to remain the pivotal actor in AI regulation with China as a major caveat, while the EU's regulatory efforts may extend globally and UK officials have expressed interest in policy development and extreme risk reduction.
  • Future governance may involve legally required evaluations, government-employed auditors for compute and security, and potentially international treaties, with the specific impactful bodies and agencies remaining uncertain over a five-year horizon.
  • Career opportunities in AI governance require a mix of technical and policy expertise, with entry points available through government, military service, or non-specialized degrees, though military routes initially demand time on non-AI work.
  • Policy roles are described as potentially meaningful but characterized by fast pace, high stress, long hours, slow progress, and pay that is typically lower than comparable private sector positions.
  • Career capital is expected to accumulate through working effectively in government, though burnout is a risk for some, and positions with autonomy or decision-making authority may take a long time to attain.
  • Congressional staff roles are noted as more competitive and influential than general positions, particularly within committees, though the landscape of impactful bodies is expected to shift over time.
  • Significant room remains for work shaping AI policy, as increased attention makes interventions more tractable, even if it is unclear whether focus will align with the most important risks.
  • Any relatively wealthy country could fund AI safety research, which may inform powerful models and complement U.S. efforts, provided that major conflicts between great powers are avoided.