a16z Podcast | Putting AI in Medicine, in Practice

Historical Context & Early Success

• AI concepts in healthcare date to the 1960s, utilizing expert systems and ontologies to generate diagnosis probabilities from verbal inputs, occasionally outperforming average physicians.
• The 1978 "Mycin" expert system trained at Stanford successfully predicted pathology results better than five human pathologists but failed to deploy due to lack of reimbursement models and risk allocation.
• Current adoption is hindered by fee-for-service incentives that financially reward misdiagnosis (via follow-up testing) rather than accurate diagnosis.
• Shifts toward "fee-for-value" reimbursement models, which reward accurate diagnoses and reduced hospitalizations, create the necessary financial incentives for AI deployment.

Technical Application & Deployment Strategies

• AI is most immediately effective in "substituting" tasks with clean data structures, such as radiology (imaging) and EKG analysis, where human error patterns are recapitulated by algorithms.
• A core thesis for AI trust is that algorithms need not be perfect; they must be "liable" by making errors similar to human consensus, allowing for safe scaling.
• Novel AI applications will involve "complementing" physicians by processing data streams (e.g., continuous wearable data) too dense for human interpretation.
• Predictive AI faces significant challenges in stochastic healthcare environments, where missing data (e.g., patients too sick to wear sensors) is itself a strong predictor of illness.
• Future high-fidelity diagnosis may involve fusing image data with blood work and other streams to predict outcomes like biopsy results without invasive procedures.
• The industry is moving toward a tiered autonomy model (similar to self-driving cars) where AI acts assistively before society decides on full autonomy.
• Liability frameworks will likely focus on the implementing hospital or physician rather than the algorithm itself, as courts cannot sue convolutional neural networks.

Data Challenges & Model Integrity

• Machine learning models suffer from overfitting when trained on limited or non-representative labeled data, failing to generalize to different populations or demographics.
• Population shifts (e.g., regional differences in obesity or disease prevalence) create calibration issues where models trained in one locale fail in another.
• Wearable data offers a solution to labeling scarcity because devices provide consistent, globally standardized inputs, unlike variable clinical documentation.
• Deep learning models require high-quality, limited-label data; the "kitchen sink" approach of including every variable often leads to statistical overfitting without proper regularization.
• Continuous learning models face versioning challenges where "learning on the fly" from bad inputs can cause harm; solutions involve batch training, strict validation on held-out data, and timestamped versioning.

Systemic Gaps & Operational Efficiency

• Current medical screening is reactive and sparse; for example, EKGs are often not performed until age 65, leaving conditions like sleep apnea (80% undiagnosed) and hypertension (20% undiagnosed) untreated.
• AI can shift healthcare from reactive symptom treatment to proactive monitoring, potentially serving populations lacking primary care access (approx. 50% of the US).
• Operational inefficiencies in hospitals, such as OR scheduling reliant on human negotiation and whiteboards, represent low-regulatory-risk areas for AI optimization.
• Scheduling AI can optimize resource allocation by factoring in physician fatigue (via wearable data), case risk, and historical speed to improve efficiency and safety.

Adoption Models & Commercialization

• Early market entry is expected via "full-stack" startups (e.g., Omada, Virta) that act as registered providers, bypassing the need for hospital-specific adoption.
• These startups can sell directly to self-insured employers, simplifying the payer decision process compared to convincing individual hospitals.
• Long-term industry restructuring is predicted to shift from geographic provider models to vertical data networks specialized in specific domains (e.g., radiology, pathology).
• Clinical research is evolving to require mobile design expertise to combat high user drop-off rates in digital health trials.
• Incentives for adoption rely on proving cost reduction and better outcomes, driving early adoption by accountable care organizations and insurers willing to take financial risk.

Regulatory & Quality Improvement

• The FDA's Office of Digital Health is adopting a forward-thinking regulatory stance, addressing risks associated with AI updates and continuous learning.
• AI can enhance Quality Improvement (QI) processes by identifying physicians with rising error rates through standardized metric tracking and algorithmic comparison.
• Versioning strategies will involve strict testing on novel datasets to ensure new models do not inherit biases or degrade performance over time.
• Successful interdisciplinary teams must integrate clinical expertise, AI engineering, and mobile design (gamification/engagement) to ensure study retention and efficacy.