EEOC Enters into First Settlement Involving AI Software Discrimination

Artificial Intelligence (AI) software has become increasingly popular in corporate America as a tool for HR to use in recruitment and talent acquisition. Recent studies show that 85% of U.S. companies are using AI software in some capacity, with nearly 80% of those companies using AI in hiring and recruitment. As a result, the EEOC created a task force in 2021 through the Artificial Intelligence and Algorithmic Fairness Initiative to address potential discrimination in hiring practices and application filtering that may arise from the use of such software.

In 2022, the EEOC filed its first lawsuit involving alleged discrimination by a company through its use of AI software. Chinese tutoring company iTutorGroup Inc., which provides English language tutoring to Chinese students through online classes, allegedly used software that discriminated against female applicants aged 55 and older, as well as male applicants 60 or older by automatically ejecting these applicants from the pool. iTutor agreed to pay out $365,000 to more than 200 applicants in a settlement but denied wrongdoing as part of the settlement agreement that was filed in federal court for the Eastern District of New York on August 9, 2023. As of this writing the settlement was still pending with the court.

An additional case has been filed in the Northern District of California, alleging that WorkDay, Inc., an employment agency that creates AI products used in employment screening, disseminated AI software and other products that result in discriminatory treatment of African Americans, individuals with disabilities, and individuals 40 and older. The plaintiff further alleges that his applications to employers that used WorkDay software numbered between 80 and 100, yet resulted in no employment offers. WorkDay has denied the allegations and the case is currently pending.

The EEOC has published guidance on its website for companies employing AI software in their hiring practices. The EEOC notes that the Americans with Disabilities Act (ADA) is often violated when AI software does not take into account reasonable accommodations that could be offered to applicants or does not allow for disability related inquiries and medical examinations.

Additionally, the EEOC has published guidance on potential application of Title VII to hiring and selection practices that utilize AI. The guidance confirms that unlawful “disparate impact” may exist if AI tools used in hiring and promotion decisions result in people from protected classes being disproportionately excluded from opportunities, even where there is no discriminatory intent.

The guidance also suggests that the EEOC’s decades-old “four-fifths rule,” which is designed to measure whether the selection rate for another group is “substantially” different than the selection rate for another group, can provide a rule-of-thumb way of analyzing potential discrimination in AI-based decision making. The rule states that one rate is substantially different than another if their ratio is less than four-fifths (or 80%.) For example, consider a test scored by an algorithm, in which the selection rate for Black applicants was 20% and the selection rate for White applicants was 50%. The ratio of the two rates is 20/50 (or 40%). Because 20/50 (or 40%) is less than 4/5 (or 80%), the four-fifths rule says that the selection rate for Black applicants is substantially different than the selection rate for White applicants in this test, which could be evidence of discrimination against Black applicants and may support an allegation of discriminatory hiring practices under Title VII.