Interview
Jeremy Howard: fast.ai Deep Learning Courses and Research | Lex Fridman Podcast #35
- The speaker hopes to resume playing music contingent on resolving current Repetitive Strain Injury (RSI) issues affecting their fingers.
- Swift is predicted to potentially mature into a programming environment comparable to Delphi, dependent on the evolution of cross-platform development and UI toolkits like SwiftUI.
- Deep learning libraries based on Swift are expected to become practical for daily use and replace Python reliance in TensorFlow within approximately three years.
- The speaker anticipates that major AI breakthroughs over the next twenty years will be achievable on a single GPU, noting no significant breakthroughs in the preceding two decades required multiple GPUs.
- MLIR is expected to facilitate domain-specific languages for tensor computations targeting diverse backends such as AMD GPUs, GraphCore, and Vertex AI, which could increase market competition.
- Google is unlikely to make TPUs programmable in the near future due to intellectual property concerns regarding their proprietary technology.
- Regulatory and legal interpretations are expected to continue hindering data sharing in healthcare longer than the regulations themselves, driven by a risk-averse legal culture.
- Most organizations are expected to discover that state-of-the-art results can be achieved with significantly less data than currently assumed, reducing the necessity for mass data aggregation.
- Academic research is expected to eventually prioritize practical techniques like active learning and transfer learning over incremental improvements on familiar problems.
- AI innovation in the next twenty years is predicted to be driven by domain experts applying deep learning tools to specific fields such as medicine, language, and environmental analysis.
- Deep learning optimization is expected to evolve toward fully automated hyperparameter selection, rendering human intervention regarding parameters like learning rates largely unnecessary.
- Swift for TensorFlow could eventually enable the writing of GPU kernels in a concise manner similar to J or APL, though this requires significant foundational retooling.
- The current reliance on massive datasets and computation is feared to be stifling creativity and discouraging new practitioners from entering the field.
- The speaker believes widespread adoption of multi-GPU or multi-machine training is largely a waste of time unless it significantly improves iteration speed, predicting most research will remain viable on single machines.
- The "superconvergence" phenomenon, where models train ten times faster using higher learning rates, is expected to lead to more widespread adoption and a better understanding of optimizer mechanics despite current academic resistance.
- Computational methods for audio and photography, including multi-mic synthesis and night mode, are expected to become standard in consumer devices.
- Deep learning frameworks are expected to move away from Python-based solutions if they fail to resolve performance and programmability issues, potentially favoring languages like Swift.
- The narrative that only tech giants can perform deep learning is expected to fade as individuals demonstrate the ability to achieve world-class results with consumer hardware.
- The future of AI ethics is expected to require data scientists to actively consider societal consequences, including labor displacement and the need for human-in-the-loop safeguards.
- The problem of labor force displacement caused by AI is expected to grow significantly in the coming years, potentially leading to a hollowing out of the middle class and increased social anxiety.