Plenary Talk: Designing for Equity

Sharad Goel

Machine learning algorithms are now used to automate routine tasks and to guide high-stakes decisions. In the first part of this talk, I'll describe an evaluation of automated speech recognition (ASR) tools, which convert spoken language to text, and have become increasingly widespread, powering popular virtual assistants, facilitating automated closed captioning, and enabling digital dictation platforms for health care. Over the last several years, the quality of these systems has dramatically improved, due both to advances in deep learning and to the collection of large-scale datasets used to train the systems. There is concern, however, that these tools do not work equally well for all subgroups of the population. Indeed, I'll show that five state-of-the-art ASR systems — developed by Amazon, Apple, Google, IBM, and Microsoft — exhibited substantial racial disparities, making twice as many errors for Black speakers compared to white speakers. In the second part of the talk, I'll describe a general, consequentialist paradigm for designing equitable decision-making algorithms. Stakeholders first specify preferences over the possible outcomes of an algorithmically informed decision-making process. For example, lenders may prefer extending credit to those most likely to repay a loan, while also preferring similar lending rates across neighborhoods. One then searches the space of decision policies to maximize the specified utility. I'll describe a method for efficiently learning these optimal policies from data for a large family of expressive utility functions, facilitating a more holistic approach to equitable decision making. Papers: - https://5harad.com/papers/asr-disparities.pdf - https://5harad.com/papers/learning-to-be-fair.pdf

https://www.msri.org/workshops/1012/schedules/29783

MSRI- [Online] Workshop on Mathematics and Racial Justice