Feb 20, 2020, 12:16 PM ETInterview

Andrew Ng: Deep Learning, Education, and Real-World AI | Lex Fridman Podcast #73

Ng began coding at age 5 or 6 in Hong Kong and Singapore, initially motivated by the ability to implement simple code to play video games.
A high school internship involving repetitive photocopying sparked his enduring theme of automation, leading to the philosophy of using AI to automate human tasks.
His early work on Massively Open Online Courses (MOOCs) was driven by a desire to automate the teaching process to impact more students, moving beyond the limitation of teaching 400 students in person at Stanford.
The first MOOC videos were filmed between 10:00 PM and 3:00 AM due to the high pressure of 100,000 sign-ups and the necessity of producing content before filming.
The core guiding principle established for Coursera and Deep Learning AI was to prioritize "what is best for learners" over personal research fame or citation counts.

Ng observed that interest in machine learning extends far beyond academic researchers (NeurIPS attendees) to include developers, biologists, and professionals globally, with Russia, India, China, and South America showing high engagement.
He projects that in the future, nearly 100% of software engineers will need some understanding of AI, similar to how literacy became ubiquitous in society.
Data science and machine learning are becoming accessible entry points for non-traditional developers (e.g., accountants, factory workers) who possess domain data but previously lacked coding skills.
Ng advocates for using whiteboards in teaching to enforce minimalism and focus on first principles, noting that the slowness of writing forces the reduction of complex concepts to their basics.
He advises students to build habits of regular, short-duration study (e.g., reading two papers a week for a year) rather than relying on unsustainable bursts of effort.
Handwritten notes are recommended over typing for learning deep learning concepts, as the slower pace of handwriting forces the brain to recode information, improving long-term retention.

Ng admits the early Google Brain team wrongly prioritized unsupervised learning, a view changed by Jeff Hinton's "napkin argument" regarding the sheer volume of data the human brain absorbs unconsciously versus the limited capacity for supervised labeling.
The conviction to pursue scale in deep learning came from experiments by Adam Coates showing that model performance directly correlated with the size of the training data, a controversial idea at the time.
While acknowledging that novel architectures (like Transformers) are necessary for some breakthroughs, Ng maintains that for many problems, simply using larger datasets with current algorithms will continue to yield performance gains.
He identifies "small data" as a critical challenge outside the consumer internet sector, where labeling errors can comprise a significant percentage of the dataset (e.g., 10% error rate in a 100-example set), requiring robust data cleaning and management.
Self-supervised learning is highlighted as a promising area for the long term, using techniques like rotating images or jigsaw puzzles to generate infinite labels from unlabeled data.
Despite its educational value, Ng notes that real-world applications of reinforcement learning remain limited compared to its success in games, suggesting it is currently more of a "toy domain" than a primary industrial tool.

AI Fund: A startup studio focused on systematically creating new companies from scratch to explore AI opportunities in fields like healthcare and education, emphasizing that the founder must be supported by a team to avoid the loneliness of entrepreneurship.
Landing AI: An initiative helping established companies integrate AI by addressing practical deployment challenges, such as small data, environmental changes (e.g., lighting shifts, birds on cameras), and the need for system-level robustness.
Ng advises companies to start with small-scale projects to build internal faith and learn, citing Google's early speech and Maps projects as successful precedents before tackling larger initiatives like Google Ads.
A significant gap exists between models working in a Jupyter notebook and those running in production; success requires addressing software engineering, maintenance (MLOps), and change management within workflows.
Ng rejects building products solely for profit if they do not create social good, citing his decision to kill a project that would have increased video consumption without educational value.
He emphasizes that for startups and job seekers, the quality of the immediate team and manager matters more than the company brand or logo.

Ng believes AGI (Artificial General Intelligence) is likely but estimates the timeline as unknown (100 to 5,000 years), expressing concern for long-term existential issues like overpopulation on Mars.
He argues that current discussions on AGI alignment and "paperclip maxims" distract from more pressing immediate problems like bias, wealth inequality, and the concentration of power in the AI industry.
Practical safety challenges (e.g., self-driving cars failing to brake for large trucks due to sensor issues) are more immediate and solvable than hypothetical moral dilemmas.
Ng warns against the "win and take all" dynamics spreading from the internet sector into traditional industries like transportation and manufacturing.

Ng identifies his proudest moments as those where he was able to help others achieve their dreams, stating that the meaning of life is "helping others achieve whatever are their dreams."
He views regrets as a natural part of the discovery process, noting that many solutions seem obvious in hindsight but were only found after years of experimentation.
His career path includes founding Coursera, Google Brain, Deep Learning AI, Landing AI, and the AI Fund, alongside a role as Chief Scientist at Baidu.
He emphasizes that students and professionals should not feel compelled to pursue a PhD to be successful in AI, as impactful careers can be built through coursework, projects, and industry experience.