The Right Problem

A sk most people what frightens them about AI and nuclear weapons, and they will describe the same thing: a machine deciding, on its own, to launch. It has been the image in films for fifty years, and in policy debates for the last five years. In September, it got its most authoritative backing yet. “I won’t lie to you – there are real dangers,” Anthropic’s chief executive Dario Amodei told CBS News, adding that “for too long the industry lied to people about the fact that this technology had risks.”

He is right to be worried. The danger he has in mind, though, is machines slipping out of human control. No nuclear-armed state is building a system where an algorithm makes the launch decision. All of them have said so, repeatedly. The danger is real. It just sits somewhere else, and because we keep looking in the wrong place, we keep missing it. The safeguards being debated now guard against the wrong failure. The ones we actually need are cheaper, faster, and almost entirely absent from the conversation.

It sits with the human. In a 2010 review in the journal Human Factors, Raja Parasuraman and Dietrich Manzey set out what they called automation bias: the tendency to trust a machine’s output more than the evidence justifies, and to stop checking once the system has given an answer. It is not a failing of careless people. It is what capable people do once a tool has earned their trust. Questioning it starts to feel like wasted effort. Drivers have followed satnav into rivers. Pilots have trusted an autopilot reading over the horizon in front of them. Radiologists shown a computer’s suggested diagnosis find the problems it flagged and miss the ones it did not. In every case, a human was in the loop. In every case, the machine had already done the deciding.

Put that inside a nuclear crisis. Early-warning and command systems now use AI to process sensor data, fuse intelligence, assess threats and propose responses. The safeguard every government points to is that a human keeps the final decision. That is true, and it is hollow. A commander looking at a system-generated recommendation, a confidence score and a shrinking clock is not weighing the evidence from scratch. They are deciding whether to overrule a machine that, as far as they know, has never been wrong, at a moment when hesitating has its own catastrophic cost. The final decision is human. Nothing that shaped it was: which signals were flagged as threats, which were dismissed as noise, which three options reached the screen and which thirty never did. Control of the last step is not control of the outcome.

Two developments make this more dangerous, and neither is slowing down. The first is time. AI is adopted because it is faster than people, and speed in one part of a crisis eats the time available in every other part. Diplomats once had hours to talk, and commanders hours to wait for a second confirmation. Those hours are disappearing, not because anyone chose to remove human judgment but because many small choices made each step quicker. The second is entanglement. The same AI-enabled surveillance, targeting and air defence networks now serve conventional and nuclear forces alike. Ambassador Zamir Akram of Pakistan’s Strategic Plans Division has said that new technologies have complicated the entanglement of conventional and strategic weapons, making de-escalation harder and escalation quicker. A system optimising a conventional strike can select a target beside nuclear command infrastructure and present it to a human with ninety seconds and no reason for doubt.