How can we guard against AI-generated discrimination?

One of the most commonly expressed concerns about AI systems is their capacity to produce unfairly discriminatory outputs and decisions. Well-known examples abound: for instance, in healthcare, AI-enabled decision support systems have been shown to produce less accurate predictions for women relative to men, and for Black patients relative to white patients. In AI-based recruitment systems, natural language processing algorithms can produce discriminatory results. For example, Amazon found that its AI-based hiring algorithm favoured those who used words like “executed” or “captured,” which were more often found in men’s resumes rather than those submitted by women. Similarly, researchers have found that generative AI systems, when asked to create images of people in specialised professions, depicted both younger and older people, but the older people were always men, reinforcing biases concerning women in the workplace.