NLM Science, Technology, and Society Lecture - Confronting Race, Gender, & Ability Bias in Tech

The Fourth Annual NLM Lecture is organized to raise awareness about societal and ethical implications in research. The guest lecturer, Meredith Broussard, an Associate Professor at NYU and author, is introduced. Her expertise lies in AI ethics, investigative reporting, and using data analysis for social good. The lecture aims to discuss strategies to mitigate bias in AI technologies and is set against the backdrop of increasing discussions around trustworthy AI and recent guiding documents from the U.S. government.

Meredith Broussard emphasizes that AI is not magic and dispels Hollywood's portrayal of AI. She discusses the importance of understanding AI's limitations and potential biases. Using the concept of 'Technochauvinism,' she explains that AI systems can default to discrimination because they learn from historical data, which contains patterns of past discrimination. The Markup's investigation into mortgage-approval algorithms is highlighted as an example of this bias.

The paragraph discusses the need to adjust algorithms to ensure fairness, particularly in lending practices. It also touches on the issues with facial recognition systems, which often exhibit bias based on gender and skin color. The importance of diverse training data is stressed, and the use of AI in high-stakes contexts, such as policing, is questioned. The potential regulation of such technologies is suggested to prevent misuse and discrimination.

Meredith shares her personal experience with breast cancer during the pandemic and her subsequent exploration of AI's role in medical diagnosis. She recounts her attempt to use an open-source AI to analyze her own medical scans, which led to a series of challenges and misconceptions about AI capabilities in medical imaging and diagnosis.

The speaker clarifies misconceptions about AI in cancer detection, explaining that AI typically identifies areas of concern in scans and assigns a score rather than providing a definitive diagnosis. She also discusses the complexities of racial disparities in medical statistics and the limitations of AI in accounting for these factors.

Meredith Broussard addresses the issue of race in medical algorithms and AI systems, noting that race is a social construct rather than a biological reality. She provides examples where race has been incorrectly used in medical calculations, such as kidney disease diagnosis and concussion assessments, and the recent changes made to remove race as a variable in these algorithms.

The paragraph discusses the issues with large language models (LLMs) and their propagation of race-based medicine. It highlights the biases in ML systems and the need for more thoughtful and less discriminatory AI applications in healthcare and other fields.

Broussard expresses skepticism about the timeline for improving AI, suggesting that many easy problems have already been solved by computers and what remains are complex sociotechnical problems. She references the AI Democracy Project by ProPublica, which benchmarked leading AI models for their potential to generate misinformation, especially in the context of U.S. elections.

The speaker advocates for a collaborative approach to AI development, involving humanists, social scientists, and biomedical researchers. She emphasizes the need for algorithmic auditing and benchmarking to ensure AI technologies are accessible and unbiased. Broussard suggests that AI is better suited for low-stakes, mundane tasks rather than high-stakes decisions.

Meredith provides a list of resources for further learning about racial disparities in medicine and AI, including books, social media accounts, and organizations focused on algorithmic accountability and responsible AI practices.

The session concludes with a Q&A where the audience asks about the specifics of harm in the context of election information, the role of algorithmic auditing, and the importance of considering social determinants of health when evaluating AI systems. The speaker emphasizes the need for granular, context-specific evaluations of AI and the importance of consulting with affected communities in technology design.