How Artificial Intelligence Bias Affects Women and People of Color

Bias, or prejudice for or against a thing, person, or group, is traditionally thought of as part of human decision-making. But when left unchecked, bias can extend beyond individual actions and infiltrate the systems people create to protect everyone.

Even computers, which think in purely mathematical terms, take on the human trait of prejudice.

Artificial intelligence (AI) is a computational problem-solving mechanism; data teaches a machine how to make specific decisions and tests performance against an answer key. Eventually, the training wheels are removed, and the computer can complete the decision-making process independently on new data. However, the computer can also learn to prefer certain demographics and discriminatory outcomes.

Over the past decade, researchers, lawmakers, and engineers have debated not just the technical, but the moral and ethical codes of AI.

The Stanford Encyclopedia of Philosophy outlines some of AI’s ethical issues, like opacity, when information about algorithmic processes and results is kept secret, and surveillance, when the data used to train computers is collected without consent or at risk of personal privacy.

These issues contribute to a system of algorithmic bias which “often reflects or reinforces existing socioeconomic, racial and gender biases.” Members of marginalized groups, such as women and people of color, are often those adversely affected by erroneous algorithms. There are real-world consequences: minorities pay $765 million more interest annually on existing mortgage stocks due to bias in AI used by FinTech leaders, according to a study from the Mitigating Bias in Artificial Intelligence Playbook (PDF, 7.7 MB).

Read on to learn more about how bias enters the algorithmic process, and how it disproportionately affects marginalized groups.

Table of Contents

How Do Algorithms Become Biased?

How AI Bias Affects Marginalized Groups

5 Women and BIPOC Leaders in AI Ethics

How to Support Women and People of Color in the AI Community

Resources for Women and People of Color in AI

(Lack of) Representation in AI, at a Glance

• A peer-reviewed study analyzing 72 leading computer science conferences found that women represent only about 10% of systems researchers, according to the National Library of Medicine.

• A separate study of 4,948 editorial board members across leading AI and computer science journals found women were underrepresented across every position — making up just 14% of editors-in-chief, 18% of senior editors, and 17% of associate editors.

• Among 2023–24 PhD recipients aggregated across CS, CE, and I, 26% identified as female, up slightly from 24% in 2022–23. CS-only rose from 23% to 25%, according to a 2025 survey by the Computing Research Association.

AI is created using a feedback loop. Real-world experiences shape data, which is used to build algorithms. Those algorithms drive decisions affecting real-world experiences. This kind of circular reasoning means that bias can infiltrate the AI process in many ways.

Recent research in The Lancet Digital Health confirms that bias in healthcare arises from social, data-related, methodological, and algorithmic factors—echoing an earlier framework from the British Medical Journal for how bias and discrimination can corrupt the algorithmic process at all stages:

World

---

Inequality affects outcomes:
Things like institutional racism, discriminatory laws, and inequitable access to resources lead to poorer health outcomes for members of minority groups and women.

Data

---

Discriminatory data:
Collected data can lack fair representation and include sampling biases. Also, the real-world patterns of inequality affect data distribution.

Design

---

Human bias in AI development:
Developer bias infiltrates algorithmic design, testing, and deployment; demographic imbalances in the field contribute to unfair process selection and methodology.

Use

---

Application injustices:
Algorithms are applied to the real world in a way that exacerbates inequality and deepens discriminatory gaps.

[Back to World]

Examine the case of an algorithm meant to predict mortgage defaults. Financial institutions automated the risk evaluation portion of the loan application process. Instead of leaving it to an individual, a computer looked at applicants’ credit histories and determined the likelihood of default.

The outcome, according to 2021 research on the default algorithm, is that the computer contributed to unequal credit market outcomes for historically underserved groups. Why?

World

---

People of color are given fewer loans and have lower home ownership rates.

Data

---

People of color have less data in their credit history. The resulting statistical noise creates an information disparity that increases uncertainty.

Design

---

Algorithm creators do not perform an analytical study on the data to search for biases.

Use

---

Loans for members of minority groups are evaluated as higher risk and affect the likelihood that a bank will approve them.

[Back to World]

These interacting systems — real-world discrimination, industry demographics, and technical processes — each have the potential to drive or mitigate unfair algorithms.

Back to top

Generative AI doesn’t just replicate historical bias: it produces it at scale, in real time, on demand. Multimodal LLMs like ChatGPT and other conversational chatbots can perpetuate gender biases due to inherent flaws in unfiltered web-scraped training data, algorithms, and user feedback loops — manifesting as unequal treatment in hiring decisions, academic recommendations, or healthcare diagnostics, systematically disadvantageous to women.

Facial recognition software, used in services like biometric security at airports and on phones,  performs better on lighter skin tones, according to the Gender Shades project. The study also found that these algorithms, from Microsoft, IBM, and Face++, performed the worst on darker-skinned females.

In a 2024 Brookings study on the use of LLMs for resume screenings, researchers found men’s and women’s names were selected at equal rates in only 37% of cases. In the rest, resumes with men’s names were favored 52% of the time, while women’s names were favored just 11% of the time.

A 2025 peer-reviewed study found that AI-generated portraits of people in STEM professions were almost exclusively depicting male, white, and older individuals — even when prompts did not specify gender or race. Even when specifications were made, the resulting pictures were affected by stereotypes associated with females.

A 2025 study published in Nature Medicine tested nine AI programs using 1,000 emergency room cases. For each file, it kept the medical symptoms identical but changed patient details like race, gender, sexuality, income, or housing status. After running over 1.7 million AI responses, researchers found the recommendations frequently changed based on these personal characteristics, not the actual health condition.

Deepfake sexual images made without women’s consent now make up the majority of all deepfake content online, according to UN Women. 98% of all deepfakes are made with non-consensual, pornographic images depicting women nude or sexualized. Tools like Grok AI rose in popularity as deepfakes flooded the Internet; despite public scrutiny and federal investigations, Grok still allows users to create and post deepfakes.

The online sexual abuse caused by deepfakes can have dire consequences: the trauma, depression, and isolation many survivors experience due to their abuse can lead to suicidal ideation and, in some cases, suicide attempts, according to one study on image-based sexual abuse.

Back to top

A 2024 UNESCO analysis of major LLMs found they associate women with “home” and “family” four times more often than men, while disproportionately linking male-sounding names to “business,” “career,” and “executive” roles. This bias has real-world consequences, as it can influence automated hiring tools, career advisory chatbots, and educational AI, limiting perceived opportunities for women and perpetuating gender inequality.

A 2024 University College London study found AI not only learns human biases but exacerbates them, creating a dangerous feedback loop where users of biased AI can become more biased themselves. The data these systems continue to learn from becomes more biased as a result.

Biased AI models produce more negative real-world outcomes for certain groups, and those outcomes then become the data that trains the next generation of models, creating a self-reinforcing feedback loop where each new iteration of AI doesn’t just reflect existing bias, it amplifies it. For women and people of color, this means the discrimination baked into today’s hiring tools, healthcare systems, and credit algorithms can compound and accumulate in ways that are nearly impossible to untangle.

One of the most insidious long-term consequences of AI bias is that it gives discrimination somewhere to hide. A 2025 peer-reviewed study from the Analyses of Social Issues and Public Policy found that organizations can use AI to shield themselves from moral scrutiny. Unlike explicit discrimination, AI-based discrimination allows decision-makers to believe they’re being objective.

Organizations with strong diversity commitments can still implement biased algorithms because the tool appears more neutral than human judgment, and because the psychological mechanisms that allow bias to thrive are operating beneath the surface of many AI tools. Researchers call this non-intentional dehumanization, where AI systems cause harm not through malicious intent, but by optimizing for efficiency while producing discriminatory outcomes no one anticipated or chose to address. For women and people of color, this leaves them particularly vulnerable.

The people building AI matter as much as the technology itself. When the field is overwhelmingly white and male, the blind spots are structural — and the data proves it:

44%

of AI experts believe the people who build AI systems take women’s perspectives into account. 75% of them believe the same about men’s perspectives, according to a 2025 Pew Research study.

84%

of global models never disclosed whether race or ethnicity was used in training data, according to a 2025 study of 390 clinical AI/ML models used in healthcare and medicine. Nearly a third (31%) never disclosed whether the data included men, women, or both.

18%

of researchers at leading AI conferences are women, according to the Mitigating Bias in Artificial Intelligence Playbook (PDF, 7.7 MB) from the Berkeley Haas Center for Equity, Gender, and Leadership.

22%

of AI professionals are female as of 2025, according to the World Economic Forum.

Data from Stanford shows people of color are also underrepresented among the highest levels of artificial intelligence education. In the 2024 AI Index Report (PDF, 1.5MB), research found that the proportion of female computer science master’s graduates has grown only slightly over the last decade, increasing from 25% in 2011 to 26% in 2022. For female PhD graduates in computer science, the percentage slightly decreased to 22%.

Back to top

danah boyd

Data & Society founder, advisor, and Professor of Communication at Cornell University

boyd completed her Ph.D. from the UC Berkeley School of Information (I School) in 2008, where her research focused on teenage sociality in networked public spaces like Facebook. In 2014, she founded Data & Society, which studies the societal implications of AI and automation on inequality. boyd was awarded the MIT Morison Prize for Science, Technology, and Society in 2023 and is the author of Data Are Made, Not Found: A Story of Politics, Power, and the Civil Servants Who Saved the US Census. Image via danah boyd’s website.

Joy Buolamwini

Algorithmic Justice League founder and author

Buolamwini’s MIT thesis, Gender Shades, evaluated gender classification of facial recognition models and uncovered large racial and gender biases in some of the industry’s most prominent services. In 2016, she founded the Algorithmic Justice League (AJL), which focuses on the societal implications and harms of AI. Buolamwini’s TED Talk “How I’m Fighting Bias in Algorithms” helped launch the AJL. She is also the author of Unmasking AI: My Mission to Protect What Is Human in a World of Machines. Image via Joy Buolamwini’s website.

Kay Firth-Butterfield 

CEO of Good Tech Advisory, author, and TIME 100 Impact Awardee

Firth-Butterfield was the world’s first chief AI ethics officer and creator of the #AIEthics hashtag on Twitter and is considered one of the leading experts on the governance of artificial intelligence. She was also the former head of AI at the World Economic Forum and is the CEO of Good Tech Advisory. Firth-Butterfield is also the author of Coexisting with AI: Work, Love, and Play in a Changing World. Image via Kay Firth-Butterfield’s LinkedIn profile.

Timnit Gebru

Founder and Executive Director, The Distributed AI Research Institute (DAIR)

Gebru founded and leads DAIR as its executive director. She came to the role after being fired by Google in December 2020, where she was co-leading the Ethical AI research team when she spoke out about workplace discrimination. Gebru also co-founded Black in AI, a nonprofit focused on growing the presence, inclusion, visibility, and health of Black people in artificial intelligence, and serves on the board of AddisCoder, which teaches algorithms and computer programming to high school students in Ethiopia and Jamaica. Her honors include recognition as one of Nature’s Ten people who helped shape science and a spot on the TIME 100 list of most influential people. She is currently at work on The View from Somewhere, a memoir and manifesto making the case for a technological future that serves communities rather than one used for surveillance, warfare, and the concentration of power in Silicon Valley. Image via DAIR.

Arati Prabhakar

Executive Fellow in Applied Technology Policy

Prabhakar was President Biden’s science and technology advisor and led the White House Office of Science and Technology Policy 2022–25. Her four decades of professional contributions span the public and private sectors. She has led DARPA and NIST, been a partner at an early-stage VC firm and a senior corporate executive, and started an innovation nonprofit. Today, she advocates for science, technology, and innovation for our future: speaking for publicly supported R&D, encouraging bigger and better ambitions for AI, and developing a project to tell the 1000-hero stories of great American innovations. Image via UC Berkeley School of Information.

Back to top

Tech leaders, managers, and colleagues who want to be allies to those in marginalized communities can help foster a more inclusive and welcoming environment. Some of the ways to encourage women and people of color to join, and happily stay, in the tech industry include:

Assess diversity in your own organization to look for opportunities for improvement and learn about employees’ experiences. Evaluate diversity at different levels throughout your company and have conversations with your team about what a culturally competent environment looks like to them.

Invest in employees’ professional development and leadership skills. This can help improve employee retention and build a stronger organization.

Provide employees with access to external communities and time for mentorship opportunities. Some examples include:

• Women in AI Ethics mentorship program
  SheCanCode community
  Black in AI
  Blacks in Technology
  Society of Hispanic Professional Engineers (SHPE)

Back to top

Fixing AI bias requires intervention at every stage of model development and deployment. Researchers have identified three main points of intervention for addressing bias in AI systems:

However, technical fixes alone aren’t enough. Addressing bias in AI requires a holistic approach: one that combines diverse and representative datasets with more transparency and accountability, and draws on expertise across disciplines outside of engineering.

That means involving ethicists, social scientists, legal experts, and most importantly, the communities most affected by these systems in the design process from the start. Independent audits, bias bounty programs, and ongoing real-world performance monitoring are among the tools researchers say must become standard practice.

Here are some additional resources for people looking for professional research, projects in progress, and learning tools.

Back to top

Created by the online Master of Information and Data Science from UC Berkeley.