Interview, Fireside Chat
Forecasting & the drivers of AI progress | Danny Hernandez (2020)
- Breakthroughs in thinking machines are expected to profoundly influence society over coming decades and centuries, making precise forecasting highly valuable despite the challenge of measuring AI progress which is described as "fuzzier" than most problems.
- Forecasting expertise is characterized as distinct from science, requiring the interpretation of weak evidence, with reliable methods often relying on extending historical lines or curves while acknowledging the difficulty of verifying expert track records over long horizons.
- Research efforts are planned to focus on highly uncertain, tractable, and contentious problems, including defining decision boundaries for low-probability, high-impact events and establishing personal probability thresholds for transformative science capability.
- Calibration training is noted to convert uninterested individuals into competent forecasters, with 97% of Twitch attendees recommending the program; however, generalization across domains and from past to future predictions requires further research to achieve certification.
- Compute growth trends show a 10x annual increase in training computation between 2012 and 2018, totaling a 300,000x rise, driven by the AlexNet breakthrough and continuing under the influence of AI-specific hardware startups, though Moore's law governed growth from the 1960s to 2012.
- Algorithmic efficiency has improved such that models now require approximately 25 times less compute than at the AlexNet level, with translation domains seeing 60x gains, suggesting the 25x figure is a conservative floor for total effective compute growth when multiplied by raw scaling.
- A divergence is expected between industrial and academic sectors as the compute gap widens, prompting suggestions for government funding to help academics verify research, while the marginal returns on spending for advanced AI models currently justify the costs due to better efficiency than the expenses incurred.
- Transitioning from consulting at Open Philanthropy to AI forecasting at OpenAI is planned to access rare expert time, with the Foresight team comprising ex-physicists aiming to understand underlying science and macro trends to inform government tracking and organizational rigor.
- Organizational efficiency is expected to vary significantly, with non-numerate entities facing 70-80% political overhead compared to startups with clear metrics, while the user experience of forecasting remains challenging for those viewing probabilities as abstract or fearing error.
- The speaker anticipates that compute scaling alone could drive rapid AI capabilities, such as creating great scientists, potentially 4x beyond current human-like systems, though risks include insufficient time for safety research if hardware progress accelerates unchecked.
- Hiring plans include research engineers for OpenAI and experts in AI hardware, particularly secure hardware to mitigate vulnerabilities like Spectre, recognizing that hardware companies could capture significant value or pre-commit to sharing windfalls in profitable AI worlds.
- Policy and forecasting roles are expected to benefit from specialized hardware expertise, which is currently scarce, alongside the recommendation that individuals define personal triggers and evidence thresholds for major AI decisions, viewing such commitments as flexible rather than absolute.