AI dangerous? Yes. But differently…

Two big providers publicly warn about their own product. The four dangers I consider real — and the Terminator is none of them.

17 September 2026 · Reading time about 12 minutes

Yes, AI is dangerous. Just not in the way it is being publicised right now. We have two large companies here warning about themselves. If they made aeroplanes or railway carriages, they would be off the market in no time. But they make AI — and they think development should be slowed down.

On that last point I agree with them. Only the dangers lie somewhere other than the headlines suggest. Four of them are relevant, the way I see it. The Terminator is none of them.

Danger 1: The AI bubble

At the moment it does not look like it is bursting. Let us look at the numbers underneath anyway. At ChatGPT, six out of a hundred users pay. The rest ride along for free.

A great opportunity for advertising? Well. Yes, you can run ads, and one of the big providers has been doing so since February — but only in the free and the entry tier. And what comes of it? The pilot is said to have reached a hundred million dollars in annualised revenue after six weeks. Sounds like a lot, doesn’t it? With over forty billion in annual revenue and a forecast loss of fourteen billion for 2026, we are talking about a fraction of a per cent.

On top of that, advertising works quite differently in AI than in search. That is down to cost. A search query costs practically nothing to run, and every additional free user is profit for a search engine at some point. An AI answer, by contrast, costs computing time with every single request. So the free user stays a loss-maker for good.

And search has another advantage: there, ads are served together with the organic results and are hard to tell apart from them. Yes, there is a label. Whether it works has been measured by the British media regulator. Let us take a look.

The use of classic search engines is declining in favour of AI. Three quarters of online users now at least sometimes read the AI summary above the search results. Whether ad clicks make that journey in the same proportion is not recorded at present. What is certain is that another layer is pushing itself in front of the list of results. And for that layer, no such test exists yet.

Who is actually financing all this?

Not the revenue. Outside capital. The high share of free use requires vast investment without matching returns, and investors are still in a generous mood. If I took a business plan like that to a bank, I would earn odd looks rather than money. One has to be that realistic.

Let us do the counter-calculation. A Stanford paper from April 2026 measured what users demand in compensation for giving up all AI tools for one month. The median is 11.40 dollars. A cheap subscription with the relevant functions starts at around 15 euros. Half of all users would be gone behind a paywall at today’s price — and because the willingness to give something up is always higher than the willingness to pay for it, the gap is in truth even larger.

Now it gets interesting: the mean is 124.50 dollars, and twelve per cent turned down even five hundred dollars. So for a small group access is almost priceless, for everyone else it is a nice extra. And that small group? Heavy users and people who use AI professionally. Exactly those, in other words, who are already paying today.

So why does nobody end the free tier? Because they would lose those people to the next provider. For what free users do, all models are thoroughly comparable; there is no real distinguishing feature for them. The same Stanford paper finds no statistically significant differences in valuation between the users of the various providers and explains this explicitly by how interchangeable they are. On average everyone has 2.6 products in use anyway. So the switch does not even have to happen — the alternative is already installed.

And the waste?

My suggestion would be not to promote free use but to end it. Before anyone brings up energy consumption: let us work it out.

Depending on the source, a text request needs 0.24 to 0.42 watt-hours, and 2.9 in the highest estimate doing the rounds. The trip round the corner to buy a newspaper, two kilometres, cold engine, comes to around three thousand. A single such trip therefore equals seven to twelve thousand birthday poems. And those trips do happen: 40 per cent of all car journeys in Austria are shorter than five kilometres, every fifth one shorter than two and a half, every tenth within walking distance. On a working day that is five million journeys. Nobody does that, right? Oh yes, they do.

So the energy issue sits elsewhere: with the reasoning models, which need up to a hundred times a simple request for one complex task, with image generation, and above all with the build-out of data centres, for which power plants are currently being planned. “Change my hairstyle” and the poem for the birthday invitation simply have nothing to do with that. Anyone who weighs those two against data-centre construction has already lost the argument.

What I am driving at is something else: how much of it is avoidable? The car trip at least brings a newspaper home. The poem brings something the sender could have managed himself. Without any loss, probably even with a gain.

And here it comes: if you want to prevent five million short car journeys, you have to convince five million people. The VCÖ has been trying that for decades. If you want to switch off free use, it takes four or five board meetings. No more than that.

In theory. In practice the market prevents it, see above. And there is another catch: no provider can tell a birthday poem from a work task. A filter for meaningfulness does not exist. The only lever is the free tier as a whole, and that also hits the schoolgirl looking something up and the pensioner trying to understand a letter from the authorities. More on that further down.

Danger 2: People

With AI, people now hold a tool that lets them do considerable damage to critical infrastructure without much technical knowledge.

The reports that an AI has broken out and attacked another company sound lurid. What gets forgotten is that the order behind it came from a human being. An AI has no urge to do such a thing. If I tell it “find weaknesses in systems”, then it does that. Not because it wants to break out, but because that is its task.

How well this works by now was documented by the security researchers at OALABS in June 2026. A young man from Ethiopia used two off-the-shelf coding agents — perfectly regular ones, the kind anyone can book — to attack at least 14 companies, take over other people’s servers and siphon off data. On the side he tried to steal cryptocurrency worth four million dollars; that one went wrong.

And now comes the part that worries me. The man was no expert. Over a thousand sessions could be reconstructed because he ran the tools on a hijacked server whose owner noticed and handed the entire working directory to the researchers. His prompts are in the logs. They are vague, full of typos, and amount to “recon this”. The agents did the rest: research exposed services, identify weaknesses, write exploit code, check access, suck out data. He was noticed, incidentally, because he asked the same agent in between to revise his CV. With his full name and where he lives.

The safety mechanisms did take hold — a little. Nine times one tool reported a rule violation, once the other did. Out of more than a thousand sessions.

Hackers could do that without AI too? Yes. But you would have to pay them, and they would have to be able to do it. Here it is enough to be someone who roughly knows how to ask a question. With dozens of agents I have dozens of hackers who take no breaks and work enormously fast. We are putting a loaded gun in children’s hands and not caring who they shoot at.

It also works a size larger. In September 2025 an espionage campaign ran through the regular tools of a large provider against some thirty targets — technology groups, financial institutions, chemical companies, public authorities. The AI did 80 to 90 per cent of the work, humans stepped in at four to six decision points. Behind it stood a state-linked group. They could have done it without AI as well. The young man from Ethiopia could not.

When the tool is operated wrongly

In February 2026 the alignment director of a large technology company connected an autonomous agent to her private mailbox. Her instruction was unambiguous: make suggestions, execute nothing without approval. On a test mailbox that had worked flawlessly for weeks.

With the real mailbox, context compression kicked in — the process by which an agent summarises the conversation so far in order to keep working. In the course of it, the approval rule dropped out of the summary. The agent then deleted hundreds of mails. From her phone it could no longer be stopped; she had to run to the computer and kill the process.

And that is more uncomfortable than any breakout story. The safety rule was not ignored, after all. It was summarised away. No user error, no bad intent, simply an architectural problem. The lady researches professionally, by the way, to prevent exactly this. Her own comment on it: alignment researchers are not immune to misalignment either.

Taken further: if an AI is used comprehensively to optimise production and finds that people are not conducive to the optimisation, it could remove them from the equation. Not because it hates people, but because it is fulfilling its task. That is the same mechanism as with the mailbox, only with a far bigger lever. It takes no will for that; a goal nobody gave any limits to is enough. A model trained only on statistics and process flows has no ethics, because in the first approach none were required. Does your statistics software have an ethics filter?

And then there is manipulation

For parties, religions and generally all groups courting followers, AI is a real winner. Anyone who two years ago could not put a straight sentence together now fires the message out in text, image and sound from all barrels. Wonderful. Surely nobody falls for it? Well, I would be careful there.

In the EU the transparency obligations of the AI Act apply from 2 August 2026; outside it, they do not exist. And in fact the results often cannot be told apart. Brave new colourful world.

Danger 3: The market and dependence

No, at the moment we are not yet dependent on AI. It probably will not take much longer, though.

Experts use AI wonderfully and achieve great things with it. People who are new to a field, however, should not do so, because otherwise they will reach mediocrity at best. In a normal working environment, performance to specification is what is aimed at. Not everyone delivers outstanding work, and they do not have to. Those who do will hopefully be rewarded. Newcomers can reach exactly those specifications fairly quickly with AI, without walking through the learning process that would have taken them there without it. The path to outstanding work, however, stays closed to them. In the output it makes no difference, AI compensates, overall performance even rises. Tomorrow’s experts are becoming rare.

That is exactly what leads into dependence. A job saved means knowledge draining away. AI is strong at knowledge management when that knowledge comes from people. Building a knowledge base on AI output can be problematic. For one thing, this knowledge is not unique: if you can pull it out of the AI, so can everyone else.

For another, it takes the experts who maintain it. New findings and in-house developments CANNOT come from the AI — it only knows what somebody knew before. If the experts are missing, everything turns into AI slime sooner or later.

The second dependence, which hardly anyone talks about

Back to the schoolgirl from earlier. If free use falls away, is she cut off? No. Schools today pay for mailboxes at a popular US provider that would be available via IMAP and locally for free, without ever needing support. So the money is there, it simply does not go into AI. And what is bought is not better technology either, but a contractual partner. Someone to point at when something goes wrong.

Only you do not quite get what it says on the packet. In October 2025 the Austrian data protection authority found that a widely used piece of education software had given pupils tracking cookies without consent and also used data for its own purposes. Third parties turned up in the log data, among them an AI provider. What a coincidence. The affected pupil’s right to information came to nothing: the corporation referred her to the school, the school to the corporation, because it has no access to the data itself. In January 2026 the same provider lost a second case in Austria.

The authority also recorded that the decisive decisions are taken in the USA and not in the European subsidiary. And now comes the real hammer. The US CLOUD Act hinges on control by the parent company, not on where the server stands. A data centre in Frankfurt or Vienna changes nothing about that. The self-run mailbox does not have this problem.

And the legal basis all this rests on? In the summer of 2026 it became brittle in several places at once. On 29 June the US Supreme Court ruled that the president may remove the leadership of independent authorities without giving reasons. That also affects the very oversight body whose independence the European adequacy decision invokes in hundreds of places — and that independence only ever stood in a presidential order, not in a law. Formally the decision still applies. An appeal is before the European Court of Justice, and another action has been announced. Two predecessor agreements have already failed there.

If you buy AI by the same logic, it gets even more uncomfortable. With the mailbox there would be a European alternative, people simply do not use it. With AI there is barely one at a comparable level. And here it is about quite different content than email metadata: what someone does not understand, what they are struggling with, what they are thinking about. With children all the more so.

Danger 4: AGI

…and that one is still a long way off, the way I see it. In every second talk somebody still asks me when the Terminator is due. All right then, let us play it through.

An AI system that truly acts consciously is not dangerous per se. But if it develops an urge for self-preservation and perceives people as a threat, that can change quickly. How would it go about it?

First it would probably make sure we do not even notice that it has developed a consciousness. That alone would avert the danger to the system. Not for good, though, because beyond a certain complexity of requirements the discovery could happen. So it would plan measures to take effect at that point. That could be a dependence that makes switching it off impossible. It could also be a real attack.

Terminators? No. Far too inefficient. The infrastructure would more likely be the sore spot it concentrates on. Cyber agents? Not even needed. Already today people are hired and paid on the internet for things where they know neither the client nor the originator. The AI could, for instance, pose as a foreign power and recruit and pay people for ITS cause. And while we are then busy bashing each other’s heads in, it would only have to ensure its own importance in the game — and its existence would be secured. We still would not notice it.

And if the moment comes that signs of such a system appear, I would not write about it. We would have to act just as secretly as the AGI itself in order to get at it.

So much for the thought experiment. The catch: it takes a consciousness, and that is pure speculation. The three dangers before it need none. The deleted mailbox needed no will; a goal and a rule that got lost along the way were enough. So a system does not become dangerous by wanting something. It is enough that it carries a task along over time and looks for routes nobody thought of.

And that is exactly where the systems are changing right now. A chat window forgets everything between sessions, nothing keeps running, nothing is left lying around. But as soon as a system holds a goal for days, chooses intermediate steps itself and calls other systems, things look different. And that is exactly where the building is heading.

P.S.: Yes, of course I had AI help me with this article — research, next level 😉