Interview, Fireside Chat
Fast paths into high-impact ML engineering roles | Catherine Olsson & Daniel Ziegler
- Powerful AI systems exceeding human cognitive performance are expected to emerge soon, creating significant economic and strategic incentives for deployment that could lead to a vacuum of uncontrolled systems without intervention.
- The AI safety field is predicted to evolve into a diverse community with distinct skill requirements rather than a monolithic discipline, ranging from theoretical research to practical engineering, with open positions anticipated to require specialized mixes of software engineering, ML implementation, research direction, and theory.
- Effective Altruism Grants are expected to potentially fund retraining for listeners seeking ML engineering roles, specifically addressing financial gaps during the transition period, while the host believes the step-by-step guide for transferring into ML engineering may be generalized beyond AI alignment.
- Career transitions into ML engineering are expected to be feasible for software developers and researchers within a few weeks of intense work if they follow a critical path focused on practical skills and specific targets rather than broad, nebulous reading.
- OpenAI plans to rapidly expand its research engineering team and hire general software engineers for data collection and workflow optimization, though current growth is bottlenecked by the availability of skilled mentors to train new staff.
- The current paradigm of adversarial examples is expected to show cracks due to an unrealistic threat model regarding small pixel perturbations, necessitating a transition to a better threat model and a next paradigm that relates more closely to deployed systems.
- Success in ML engineering and research is expected to depend heavily on high frustration tolerance for mysterious debugging processes, the ability to learn from GitHub implementations to uncover undocumented tricks, and the flexibility to work on specific open problems rather than relying solely on formal degrees.
- Various pathways to contributing to AI safety are expected to remain viable, including PhD programs in related fields like theoretical computer science, residencies or fellowships for short-term mentorship, and learning on the job, provided individuals can identify specific projects ready for software engineers to tackle.
- The distinction between research scientist and research engineer roles is expected to remain fluid, with specific skills being more important than titles, while the "negative space" of avoiding harmful outcomes is expected to be vast compared to the positive space of defining what to build.
- Organizational decision-making expertise is identified as a rare and valuable skill set for steering organizations toward safety, alongside verification of neural nets which is expected to become a critical tool if scalable, while the "Agent Foundations" agenda is viewed as useful for modeling intelligence but may or may not be on the critical path depending on development trajectories.