Interview, Other
How to make computers less biased
The EconomistPa Manjang, Parr, Rochelle Farool, Meredith Broussard, Thomas Adams, Tamara Gilkes-Boer
- Uber driver Pa Manjang had his account permanently deactivated in April 2023 after failing an automated selfie verification system introduced in 2020 to check driver identities.
- Manjang alleges the algorithm failed to match his selfies due to racial bias; he is joining other minority drivers in suing Uber for unfair dismissal.
- Uber states that 52% of its UK drivers are from ethnic minorities and maintains that human oversight prevents algorithmic discrimination.
- Historical examples of embedded racial bias include Kodak Color Film's inability to capture dark skin tones until chocolate manufacturers complained in the 1970s.
- Major tech firms including Microsoft, Facebook, and Google have faced scrutiny for facial recognition errors, such as labeling Black individuals as "primates."
- Biased AI training datasets, often lacking sufficient data from ethnic minorities, are a primary driver of these systemic errors.
- Researchers label the assumption that technology is neutral as "techno-chauvinism," noting that light-skinned developers often test tools solely on themselves.
- Medical devices like pulse oximeters have been shown to provide flawed oxygen readings for Black patients, potentially leading to dangerous discharges.
- US mortgage lending algorithms have historically rejected Black applicants at a rate 80% higher than white applicants with similar financial profiles.
- Data journalist Meredith Broussard and Thomas Adams are developing software tools to detect and identify embedded racial bias within decision-making algorithms.
- O'Neill Risk Consulting is creating a "regulatory sandbox" system to allow companies to test algorithms for bias before deployment.
- Critics argue that while reducing bias may align with profits for some, strict government regulation is necessary when it does not.
- The European Union is leading regulatory efforts with a draft AI white paper proposing stricter oversight proportional to the risk an AI system poses to fundamental rights.
- Concerns remain regarding the EU framework's reliance on a shared, precise definition of "risk" to implement effective calculations.
- Without concerted self-policing or government intervention, systemic racism risks becoming a foundational element of the digital future.