Interview
More accurately predicting the future | Philip Tetlock
- Research sponsored by IARPA and the Civilization V counterfactual tournament seeks steeper learning curves with fewer participants, restricting analysis to correlations rather than counterfactual thought experiments while expecting human performance to surpass machine learning in environments lacking data-rich conditions; methods are anticipated to transfer to the real world given the game's complexity, path dependency, and stochasticity.
- Machine learning is projected to succeed in data-rich environments like OECD macroeconomic statistics but is expected to fail in areas with elusive base rates such as the Syrian civil war or China relations, whereas human analysts with statistical reasoning and strategic savvy are viewed as essential for geopolitical forecasting.
- Forecasting accuracy is predicted to decay significantly beyond five to ten years due to compounding randomness, with superforecasters becoming ineffective against historical discontinuities and unable to accurately predict events a century out, though they remain valuable for short-term generation-level forecasts.
- Performance in forecasting tournaments is driven three times more by the ability to update beliefs (perpetual beta) than by raw intelligence, with calibration training tools expected to improve performance by 6% to 12% over four years, while extreme probability estimates of 0% or 100% are considered high-risk to credibility.
- The academic labor market is expected to become increasingly stratified, with a precipitous drop in research-focused economics positions for graduates outside the top 10 or 20 programs, and fluid intelligence peaking around ages 25 to 30 making it difficult for those over 50 to compete in research roles.
- Venture capital strategies rely on low funding thresholds to find rare high-growth companies like Facebook or Google, while the startup base rate for a one-in-three hit rate is suspected to be low and kept confidential by VCs.
- Effective Altruism activities include the creation of a calibration training tool by the Open Philanthropy Project and upcoming global conferences in London (October 18–20) and Sydney (September 28–29), alongside a trajectory toward integrating statistical and narrativist modes of understanding uncertainty.
- Forecasting tournaments face resistance from senior intelligence community members viewing them as status challenges, risk disappearing if reliant on top management funding, and may face political hurdles from leaders like President Trump who are less open to updating views than predecessors.
- Prediction markets are expected to lose to strong forecasting tournaments unless they are deep and liquid, while professional sectors like banking and law may resist probabilistic judgments due to fears of misuse or a reliance on the attribution-substitution heuristic.
- Geopolitical analysts are expected to have longer job security than loan officers in profit-driven environments, while loan officers serving political functions remain secure, and organizations like Bridgewater may face high turnover due to the perceived oppressiveness of their required rigor.
- Forecasting tournaments are expected to appeal more to younger, upwardly mobile, male, and lower-status individuals, with future generations incentivized to generate creative questions alongside accurate answers.
- Long-term risks include the sun going supernova in three to four billion years, the potential for leaders with accurate estimates to become reckless in the short term, and the collective intelligence of the online community being lowered by specific behaviors.