Why AI Ethics Matters

Why AI ethics matters

Artificial intelligence is changing our world at a breathtaking pace. It diagnoses diseases, steers vehicles, decides on loans, and influences which news you see. But with this power come fundamental questions: Who decides what AI is allowed to do? According to which values does it act? And who bears responsibility when something goes wrong?

AI ethics is not a side issue for philosophers; it is one of the most pressing challenges of our time. In this lesson, you'll learn why this is so, what real harm has already been done, and why you, as an AI user, are affected.

Did you know? In 2021, all 193 member states of UNESCO adopted the first global recommendation on the ethics of AI. However, it is not legally binding. The first binding international agreement is the Council of Europe's AI Convention (2024), which Switzerland also signed in 2025. The EU AI Act (in force since August 2024, applicable in stages) is the most comprehensive law so far, but it does not cover all ethical questions either. AI ethics is a constantly evolving field.

Ethics is not a nice-to-have

Some see ethics as a brake on innovation. The opposite is true: a lack of ethics is a risk, for companies, for society, and for individuals. AI systems without ethical guardrails have already caused real harm. The following examples show what happens when ethics is considered as an afterthought instead of from the start.

Historical examples of ethical failure

The history of AI is full of warning signs. Three particularly striking cases:

The COMPAS system (2016)

The COMPAS system (Correctional Offender Management Profiling for Alternative Sanctions) was used in US courts to predict defendants' likelihood of reoffending. Judges used these scores for decisions on bail and sentencing.

The problem: an investigation by ProPublica showed that the system systematically rated Black defendants as more dangerous than white ones, even with comparable backgrounds. The share of people wrongly rated as high-risk was almost twice as high among African Americans. Nevertheless, the algorithm was used for years without the people affected being able to question the system.

Ethical problem: Discrimination, lack of transparency, no possibility of appeal.

Amazon's recruiting AI (2018)

Amazon developed an AI system that was supposed to evaluate job applications automatically. The system was trained with historical hiring data, and since Amazon had hired mainly men in the past, the AI learned to prefer male applicants.

The AI systematically downgraded résumés containing the word "women's" (e.g., "women's chess club"). Amazon discontinued the project, but the case became a prime example of how historical prejudices live on in AI systems.

Ethical problem: Gender discrimination due to biased training data.

Microsoft Tay (2016)

Microsoft launched the chatbot "Tay" on Twitter, which was supposed to learn through interactions with users. Within 16 hours, it had to be shut down: trolls had taught it racist, sexist, and antisemitic statements, which it cheerfully spread further.

Tay showed a fundamental problem: when an AI system learns from user behavior without filters, it also adopts the worst aspects of human behavior. Without safety mechanisms, AI becomes an amplifier of hate.

Ethical problem: Missing safety precautions, no alignment with human values.

The trolley problem for AI

Imagine a self-driving car recognizes that an accident is unavoidable. It can swerve to the left and endanger a pedestrian, or keep going straight and put its occupants at risk. How should the AI decide?

Philosophy's classic "trolley problem" suddenly becomes real with autonomous driving. And it gets even more complex: should the age of the people affected play a role? Their number? What if a child is standing in the road?

Example: The MIT research project "Moral Machine" collected more than 40 million decisions from people around the world. The results showed major cultural differences: in almost all countries, younger people were more likely to be spared, but in many East Asian countries this preference was considerably weaker than in Western countries (Awad et al., Nature 2018). How do you program a "global ethics" into a car that drives in different countries? The answer: there is no simple solution.

The trolley problem is only the tip of the iceberg. Similar dilemmas arise wherever AI makes decisions: should a medical AI give preference to a young or an older patient when resources are scarce? Should a credit AI take the neighborhood into account, even if that leads to discrimination?

Who is responsible when AI makes mistakes?

When a human doctor makes a mistake, the question of liability is clear. But what if an AI makes a wrong diagnosis? Who is liable?

Caution: The question of responsibility for AI errors has not been resolved anywhere in the world. Neither the EU AI Act nor other sets of rules offer a complete answer so far. This means that if you use AI systems professionally, you may bear a responsibility whose scope has not yet been legally defined. Never rely blindly on AI decisions in critical areas.

The stakeholders, meaning those involved and those responsible, form a complex web:

Developers and researchers: They design the models, choose the training data, and define the optimization goals. Their decisions in the development phase have far-reaching ethical consequences.
Companies: They deploy AI systems and bear responsibility for their impact on customers, employees, and society. The pursuit of profit must not trump ethical principles.
Legislators and regulators: They create the legal framework, but technology develops faster than laws. The gap between technical reality and regulation is a permanent problem.
Society and individuals: Each of us is affected, and each of us has a voice. Informed citizens can put pressure on companies and politicians and demand ethical standards.

Quote: "Technology is neither good nor bad; nor is it neutral" (Melvin Kranzberg, historian of technology, 1986). This quote applies to AI in particular: it is a powerful tool whose effect depends entirely on how we design and use it.

Practical tip: You don't have to be a lawyer or a philosopher to understand AI ethics. Start with one simple question for every AI system you use: "Who could be disadvantaged by this system?" This question alone sharpens your awareness enormously and helps you recognize potential problems early on.

Why was the COMPAS system criticized on ethical grounds?

Correct! The ProPublica investigation showed that, with comparable backgrounds, COMPAS wrongly rated Black defendants as "high-risk" almost twice as often. The system reinforced existing racist prejudices instead of correcting them, a prime example of algorithmic discrimination.

Not quite. The main problem with COMPAS was neither technical nor a question of authorization. The ProPublica investigation revealed that the system systematically rated Black defendants as more dangerous, a clear case of algorithmic discrimination with real consequences for the people affected.

Key Takeaways:

AI ethics is not a side issue; it affects real people with real consequences, from criminal sentences to credit decisions.
Historical cases such as COMPAS, Amazon's recruiting AI, and Tay show that without ethical guardrails, AI causes measurable harm.
The trolley problem illustrates that many ethical questions about AI have no simple answers, but the questions must be asked anyway.
Responsibility for AI ethics lies with all stakeholders: developers, companies, legislators, and society.
As an AI user, you can make a contribution by critically questioning AI systems and demanding transparency.

This lesson is part of a course with final exam and certificate. View the course "AI & Ethics"