OpenAI’s IMO Team on Why Models Are Finally Solving Elite-Level Math

Key Milestones and Performance Metrics

• IMO Gold Achievement: The trio of Alex Wei, Cheryl Hsu, and Noam Brown achieved gold medal performance at the International Math Olympiad (IMO), representing the first time a model has reached this level.
• Rapid Benchmark Progression: The models progressed through math benchmarks at an astonishing rate: from struggling with grade school math a few years ago, to mastering GSM8K and "Math" in 2024, to AIME, USAMO, and finally IMO within a few years.
• Self-Verification Capabilities: The system demonstrated significant self-awareness by refusing to hallucinate a solution for Problem 6, instead explicitly stating it could not solve the problem rather than providing a convincing but incorrect answer.
• Grading Protocol: To verify correctness, proofs were evaluated by three external former IMO medalists per problem, who reached unanimous consensus on the validity of the model's output.
• Reasoning Duration: The models now operate with reasoning capabilities on the order of hundreds of minutes (100+ minutes), a massive increase from the ~10 seconds of reasoning capability seen just a year ago.
• Problem 6 Exclusion: The model did not attempt Problem 6, which is traditionally the hardest problem (often a combinatorics task) and remains out of reach for current state-of-the-art models.

Team Dynamics and Development Strategy

• Small Core Team: The specific effort to achieve IMO gold was driven by a "super small, scrappy" core team of just three individuals (Wei, Hsu, Brown).
• Timeline: While the team had considered the goal for years, the focused sprint to prepare for the current year's IMO began only two months prior to the competition.
• Internal Skepticism: Early on, there was significant internal skepticism; a researcher reportedly declined to bet against the team's success despite offering 2-to-1 odds, viewing the outcome as a low-probability event at the time.
• Empowerment Model: The project originated from the team self-directing a high-risk research pitch, with OpenAI providing the freedom to explore despite initial doubt.
• Non-Bespoke Infrastructure: The techniques used were deliberately general-purpose rather than bespoke; the team prioritized scaling test-time compute and handling hard-to-verify tasks to benefit broader systems like ChatGPT and agents, rather than building a narrow math-only tool.

Technical Architecture and Methodology

• Scaling Test-Time Compute: The breakthrough relies on general-purpose techniques for scaling inference time, allowing the model to "think" for extended periods rather than generating immediate responses.
• Multi-Agent Systems: The team utilized multi-agent systems to scale parallel compute, enabling the model to explore multiple reasoning paths simultaneously, though specific architectural details were withheld.
• Informal vs. Formal Reasoning: The team chose to use natural language (informal reasoning) rather than Lean (formal verification), prioritizing the development of broad, general-purpose reasoning capabilities over narrow formalizability.
• Output Readability: The raw model outputs were described as "atrocious" or "alien-like" in style; while they could be rewritten for readability by other AI models (like ChatGPT), the team chose to publish the original proofs for transparency.
• Combinatorics Challenges: The team identified combinatorics as particularly difficult for current models due to the abstract, high-dimensional nature of the problems requiring "leaps of faith" or insight, unlike the step-by-step nature of geometry problems where models perform better.

Future Outlook and Limitations

• Millennium Prizes: Researchers noted that while progress is rapid, solving Millennium Prize problems remains far away, requiring reasoning durations on the order of 1,500 hours per problem compared to the 4.5-hour limit of the IMO.
• Evaluation Bottlenecks: A significant hurdle in scaling reasoning to thousands of hours is that evaluating a model's output takes the same amount of time the model took to generate it, creating a speed bump for iterative progress.
• Beyond Competition Math: The immediate frontier is shifting from time-boxed competition problems (like Putnam) to real-world scientific research, where the "next hurdle" is having models generate novel problems and questions rather than just solving existing ones.
• Science and Physics: The team noted that while math performance is strong, the Physics Olympiad remains harder due to the inclusion of experimental sections requiring robotics and world models, which are not yet integrated.
• Deployment Strategy: The specific model is not being released immediately to the general public; instead, the focus is on integrating these general-purpose reasoning techniques into future OpenAI models to benefit everyday usage and collaborate with mathematicians on unsolved problems.
• Human Collaboration: The team is currently engaging with academic mathematicians (e.g., from Stanford) to test the model on real research problems, noting that while the model still fails on the hardest tasks, it has improved from hallucinating to correctly identifying when it lacks a solution.