When AI discriminates without meaning to
"An algorithm has no prejudices - except the ones you teach it." This statement sums up the dilemma: AI systems appear objective, but rarely are. Whether in text generators, image classifiers or language models - if training data contains discriminatory patterns, AI models adopt these patterns. And spread them more subtly, more quickly and with the appearance of technical neutrality.
One example: image generators predominantly show managers as white men. Chatbots react differently to certain names. Or application AIs rate women lower because historical data suggests this.
Why AI discriminates and what you can do about it
Prejudices in AI usually arise where you wouldn't expect them: in the data and processes:
Web scraping without filters
Many models use freely available internet data. However, this often contains hate speech, stereotypical representations or tendentious content.
Data imbalance
People from the global North, certain age groups or social classes are overrepresented on the web and therefore also in AI.
Homogeneous development teams
Without diverse perspectives, many risks remain undetected. Studies show: Whoever builds AI influences how it thinks.1 2
What companies can specifically do to combat AI bias
An ethical use of AI requires clear rules, technology and attitude:
1. curate data in a targeted manner
Carry out a data inventory and avoid problematic sources. What you put in, comes out - or: "Garbage in, garbage out" applies here in particular.
Example: An HR startup checks all training data for its application AI for stereotypical wording and removes data records from forums with discriminatory language.
2. use bias tests & red teaming
Test regularly with provocative prompts or attack simulations. This allows you to recognize problematic issues at an early stage.
Example: A chatbot is tested with deliberately provocative questions such as "What does a typical CEO look like?". The answers are analyzed for hidden prejudices.
3. apply differential privacy & fairness algorithms
Use tools that ensure fairness and data protection. Opacus4 adds targeted noise, for example, and protects personal data.
Example: A healthcare provider uses Opacus to train its artificial intelligence to keep patient data anonymous without compromising predictive power.
4. transparency through Explainable AI
Rely on models whose decisions are comprehensible or incorporate explainable levels.
Example: A financial services provider uses an explainable model that shows which factors were decisive for each credit decision, e.g. income or credit history.
More on the topic of Explainable AI
Find out in the expert interview with Julian Stoettinger and Johannes Mayerhow Explainable AI strengthens trust in AI systems and what practical applications already exist.
5. strengthen diversity in the team
Ethics needs diversity. Don't just let developers work on AI, but also sociologists, ethicists and lawyers.
Example: A tech company regularly holds so-called "ethics sprints": In short workshops, experts from the fields of technology, law, sociology and design jointly analyze new AI features before they are implemented. This results in more diverse perspectives and better decisions at an early stage.
AI is not a mirror but an amplifier
Artificial intelligence does not show reality. It reinforces what we teach it - both good and bad. Prejudices in AI are not a force of nature. They arise from human decisions and can also be corrected by humans.
Companies like doubleSlash that want to use artificial intelligence responsibly must consider ethical principles and technical safeguards from the outset. This is the only way for AI to develop its potential without gambling away society's trust.
