Tutorial, Lecture, Keynote
Deep Learning for Speech Recognition (Adam Coates, Baidu)
Lex FridmanAdam Coates, Hugo Larochelle, Andrej Karpathy, Richard Socher, Sherry Moore, Ruslan Salakhutdinov, Andrew Ng, John Schulman, Pascal Lamblin, Alex Wiltschko, Quoc Le, Yoshua Bengio, Shubho Sengupta
- Speech recognition technology is expected to mature into enabling exciting applications, replacing human labor for video captioning and lectures, and making mobile and home interfaces safer and more efficient through deep learning.
- Voice recognition systems are predicted to enable text entry three times faster than typing, with future deep learning models leveraging increased data and computing power to raise performance ceilings and potentially solve speech recognition for everyone.
- Deep learning acoustic models are anticipated to drive speech engine performance toward below 6% character error rates, with systems becoming highly competitive with human committees in tasks including Mandarin transcription.
- Future acoustic models may process audio waves directly without traditional pre-processing steps like spectrogram conversion, while neural language models are expected to improve spelling correction and phrase resolution compared to traditional N-gram approaches.
- Production constraints will prioritize latency and economic efficiency alongside accuracy, necessitating trade-offs such as using forward-only recurrent networks to enable on-the-fly responses, potentially at a cost to some accuracy.
- System efficiency will be improved through batching audio packets to allow single GPUs to support 10 to 30 users in parallel, utilizing 64 GPUs for linear speedups, and grouping similar-length utterances to reduce padding waste.
- Training robustness will be enhanced by synthesizing noise and reverberation on clean audio and eliciting the Lombard effect via loud headphones during data collection, aiming to build engines that can handle noisy environments.
- Current formalisms will likely result in undefined outputs or the transcription of only the dominant speaker when multiple voices are present simultaneously, though systems are expected to filter out background speakers in practical scenarios.
- Customization to specific speaker characteristics via embeddings or i-vectors is expected for future systems, while proper names and foreign words will be handled correctly if included in training data or fused via language models.
- Technology is expected to reach a stage where end-to-end solutions are mature for production, with users able to reach performance thresholds where the remaining challenges are primarily related to scaling data and computing resources.