Conference Presentation, Keynote, Lecture
Explaining AI
- Microsoft has achieved human parity in key AI perception tasks, including computer vision, speech recognition, and natural language processing.
- Computer Vision: ResNet, developed in Microsoft Research Beijing with 152 layers, achieved 96% accuracy on the ImageNet dataset.
- Speech Recognition: Microsoft reduced error rates on the Switchboard dataset to 5.1%, matching the 5.2% error rate of professional human transcribers.
- Machine Translation: The English-Chinese and Chinese-English translation model achieved a BLEU score of 69.9% on news corpora.
- Machine Reading Comprehension: Significant progress was made on Stanford's SQuAD dataset.
- The speaker highlighted the development of "Xiaoice," a social chatbot with an average conversation length of 23 turns per session, compared to the typical digital assistant's few turns.
- Xiaoice has 120 million monthly active users in China and appeared on 60 TV programs and radio shows.
- The longest recorded conversation between a human and Xiaoice exceeded 29 hours and 7,000 turns.
- The system incorporates Emotional Intelligence (EQ) alongside IQ, enabling capabilities such as poetry writing, music composition, painting, and book authorship.
- Commercial applications of Xiaoice demonstrated significant efficacy in the Japanese retail sector through collaboration with Lawson.
- The AI-powered persona "Akiko" handled product recommendations, surveys, and coupon distribution.
- In an experiment, one million coupons were distributed in 13 hours, resulting in a 40% installation conversion rate within four days.
- Xiaoice now powers summary generation for 90% of quarterly earnings reports for Chinese companies.
- The presentation identified "AI bias" as a critical challenge stemming from the inherent biases in large training datasets used for deep learning models.
- Gender Bias in Job Classification: A system trained on bio data incorrectly classified a philanthropist as a "teacher" but assigned different roles when pronouns were switched.
- Word Embedding Analysis: NLP vector analysis revealed that word associations in training data (including Google's data) correlated "she" with "nurse" or "homemaker" and "he" with "computer programmer."
- Abstract Bias: Data analysis showed "genius" and "brilliance" were spatially clustered closer to male-associated terms in vector space.
- Mitigation Strategy: Microsoft developed methods to "squash" the gender axis in word embeddings, ensuring terms like "brilliant" apply equally across genders.
- The speaker emphasized the necessity of "Explainable AI" to transform complex deep learning models from "black boxes" into transparent systems.
- Rationale for Transparency: Trust is required for high-stakes decisions in medical diagnosis, military applications, and political advertising, unlike low-stakes movie recommendations.
- Operational Improvement: Explainability allows developers to pinpoint error sources, identify bias in specific data segments, and improve model accuracy.
- Market Trade-off: There is a fundamental tension between model accuracy (favored by complex, large models) and explainability (favored by simple, logical models).
- Two primary methodologies for achieving explainability are currently being pursued in the industry.
- Bottom-Up Approach: Starting with simple, explainable models (e.g., logic regression) and iteratively improving accuracy, such as the proposed Generalized Additive Model (GAMM).
- Top-Down Approach: Starting with highly accurate, complex models and applying model-agnostic explanation techniques to interpret their decisions.
- The speaker concluded with a forward-looking statement regarding the societal responsibility of the first generation to live with AI.
- Microsoft acknowledges the "tremendous social responsibilities" to ensure AI decisions are explainable and understandable.
- The organization rejects the prospect of a future where AI makes decisions that cannot be explained to humans.