Existential risk from AI - between justified concern and exaggerated panic?

The paper clip myth: what's behind it?

"Imagine an AI is given the task of producing as many paper clips as possible - and turns the entire earth into wire to do so."

This disturbing thought experiment comes from the philosopher Nick Bostrom and was formulated in a specialist article back in 2003¹. Today, it has long been part of a serious debate: Could AI one day take control - with existential consequences for humanity?

In this article, we take a closer look: How realistic are such scenarios really - and what belongs more in the realm of science fiction? 

AI: hype, hope, horror? 

Artificial intelligence has made rapid progress in recent years. Today, it writes texts, generates images and automates entire business processes. With this development comes a growing concern: what will happen if one day these systems are no longer controllable?

It's not just science fiction authors who warn of a loss of control. Researchers and companies also see risks. At the same time, other voices emphasize that many of these scenarios are greatly exaggerated - and reflect psychological rather than technical fears.

Focus on two real AI risks 

The debate about existential AI risks focuses primarily on two scenarios - one spectacular, the other gradual:

1. the loss of control over a super AI

As described in the introduction - in the thought experiment of Nick Bostrom's Paperclip Maximizer - it is about the idea that an AI with increasing intelligence pursues goals that are incompatible with human values and undermines all regulation in the process.

Such scenarios seem futuristic, but are also being discussed in research. In their Preparedness Framework, for example, companies such as OpenAI are investigating whether AI models are prone to self-preservation strategies or could replicate themselves autonomously².

2. cumulative risks - many small mistakes with a big impact

Less in focus, but at least as dangerous: the scenario of gradual escalation. AI-supported deepfakes, automated cyberattacks or incorrect decisions in security-critical areas can add up - and ultimately have equally existential consequences.

One example is the Microsoft AutoAttacker project, which demonstrates how AI can already (semi-)autonomously exploit vulnerabilities today³.

Researchers such as Bucknall et al.⁴ show how many smaller AI malfunctions can cumulatively generate a system overload. The risk is real and affects us all. 

Reality vs. panic: Which is true?

Many warnings about existential AI risks sound dramatic - and they are. But how realistic are they?

The fact is that technological development is still a long way from an autonomous superintelligence. The loss-of-control scenarios are based on assumptions about future developments that are still speculative.

The situation is different when it comes to cumulative risks: These can already be observed today - for example in AI-generated deepfakes or cyberattacks with LLM support. One example: a deepfake song with deceptively real voices of two pop stars went viral on TikTok - and shows how real this threat is⁵.

So instead of getting lost in horror stories, it's about acting with foresight: Companies such as OpenAI or Microsoft are already relying on risk analyses, audits and technical guard rails to identify dangerous developments - before they escalate. A good example of this is the Preparedness Framework, which systematically assesses four central risk categories, including "loss of control" and "cybersecurity".