The Conference on Sure Standard Weapons, a global discussion board that focuses on deadly autonomous programs, is hosted twice a yr on the United Nations in Geneva. When Branka Marijan attended in November 2017, she thought the five-day classes — which dealt largely in hypotheticals, speculating on a world the place warfare was fought with killer robots — can be enterprise as standard. In spite of everything, this was expertise some thought would possibly by no means be developed, and sure by no means deployed. That yr, she shortly realized, was totally different. That distant, imagined future was abruptly nearer and realer than ever.
On the primary day, some attendees watched a short film referred to as Slaughterbots, put collectively by the Way forward for Life Institute. The video featured a fictional protection contractor pitching an AI-powered drone that might kill unassisted with precision strikes. “They used to say weapons don’t kill individuals, individuals do,” its CEO tells the viewers. “However individuals don’t. They get emotional, disobey orders, intention excessive. Let’s watch the weapons make the choices.” The temper within the room, Marijan recollects, abruptly turned apprehensive. Essentially the most horrifying half wasn’t the premise — it was that the Pentagon was already growing a model of this expertise.
That assembly was the primary one held after the beginning of Mission Maven, a US Division of Protection initiative utilizing AI to investigate drone surveillance footage. And by late 2017, Maven had a serious tech firm on board: Google. “The programs we have been speaking about weren’t futuristic,” mentioned Marijan, who’s a senior researcher at Mission Ploughshares, a peace-focused unbiased analysis institute. “They have been present platforms that had levels of autonomy in them, or the aptitude to pick and interact targets primarily based on sensor knowledge and sensor enter.”
The world had already seen drone warfare — lethal machines directed by people. Now, it was a future the place people could also be faraway from the loop completely. “These weren’t these Terminator-like figures that we have been involved about, however actually what was taking place with the enablement of autonomy,” mentioned Marijan.
The US army has backed AI improvement for many years, and in flip, AI has reworked warfare
Practically a decade later, militaries haven’t but developed absolutely autonomous deadly weapons. However these programs sit squarely within the heart of a current high-stakes battle between the US authorities and AI startup Anthropic. Anthropic is searching for to protect two “crimson strains”: bans on home mass surveillance and on weapons that may determine, observe, and kill targets with zero human involvement. For the reason that begin of the yr, it’s emerged as the one army AI contractor to position significant limits on what specialists name one of many remaining frontiers of AI warfare.
However amid shifting alliances, lawsuits, and melodrama, it’s simple to lose sight of the bigger context — that AI is, and lengthy has been, deeply embedded within the army. Seventy years in the past, a summer season meeting between scientists in New Hampshire made the Division of Protection sit up and take discover of AI’s potential for conflict. Since then, its affect has grown exponentially each decade. Lately specifically, AI has enabled extra and sooner killings than ever earlier than.
Even Anthropic appears to assume its crimson strains received’t maintain for lengthy. In spite of everything, historical past has confirmed in any other case.
The US army has backed AI improvement for many years, and in flip, AI has reworked warfare. Within the 2000s, the expertise turned able to parsing unprecedented quantities of information collected worldwide, making a surveillance revolution. And the late 2010s noticed the event of superior facial recognition and different subtle machine imaginative and prescient programs.
The struggle between Anthropic and the Pentagon has drawn consideration to the rising energy of those programs. It started in January of 2026, when Protection Secretary Pete Hegseth demanded a renegotiation of the DOD’s present AI contracts. The phrases did away with any grey areas or beforehand agreed-upon phrases, permitting the Pentagon to make use of the businesses’ expertise throughout the imprecise and expansive limits of “any lawful use.” Anthropic — the one AI firm authorised to deploy its tech on the Pentagon’s labeled networks at that time — objected.
Even Anthropic appears to assume its crimson strains received’t maintain for lengthy
A contractor like Anthropic setting limits on particular makes use of of its tech is uncommon. “It’s not government-created expertise in the way in which that the Manhattan Mission was,” nor a standard army provider like Northrop Grumman, mentioned Andrew Reddie, an affiliate analysis professor of public coverage on the College of California, Berkeley. “This is likely one of the ache factors that’s made clear once you’ve bought this startup ecosystem partaking with the Pentagon immediately.” Even inside Silicon Valley, he mentioned, there’s “a variety of disagreement” over when limits ought to be set.
It’s additionally unclear who will win. As a negotiation tactic, the DOD designated Anthropic a army provide chain danger in March, and President Donald Trump declared he was banning all authorities businesses from utilizing its Claude system. The connection has apparently warmed somewhat since then, with the discharge of Anthropic’s cybersecurity-focused mannequin Mythos, however a courtroom battle continues to be taking part in out. Anthropic declined to supply a remark for this story.
Both method, the controversy has introduced “absolutely autonomous weapons” into the general public lexicon in a brand new method. However AI’s creeping affect in army operations hasn’t slowed down in many years.
“We’ve type of crossed the rubicon whereas we faux that we haven’t,” Reddie mentioned.
On the heart of the debates is DOD Directive 3000.09, one of many solely insurance policies governing using deadly autonomous weapons. Initially written in 2012, it defines such a system as one which, “as soon as activated, can choose and interact targets with out additional intervention by an operator.” And it decrees that each absolutely autonomous and semi-autonomous weapons be designed to permit people to “train acceptable ranges” of judgment over using pressure.
The directive arrange the “first coverage on using autonomy in warfare,” mentioned Hamza Chaudhry, who leads AI and nationwide safety on the Way forward for Life Institute.
“Even with out full autonomy, AI compresses kill chains to mere seconds.”
Relying on the way you interpret the definition, nonetheless, sure missile protection applications could have crossed that line many years in the past. Take the Phalanx CIWS, for example. It’s an automatic weapon system resembling a really giant gun, constructed to defend naval vessels from incoming missile assaults. That kind of system wouldn’t work if there have been a human within the loop, because it has to reply in milliseconds.
The distinction, some specialists say, is that these programs function solely in a defense-only, fastened setting. They’re partaking, this interpretation goes, however not deciding — simply reacting to an incoming risk. “The ‘and’ is doing a variety of work inside that statute — now we have programs that may determine and programs that may have interaction however you’ll be able to’t have a system that does each,” Reddie mentioned.
“Even with out full autonomy, AI compresses kill chains to mere seconds in order that people aren’t truly making the assessments that worldwide humanitarian legislation requires to forestall civilian hurt,” mentioned Maddy Batt, authorized fellow at Tech Justice Regulation. “When people’ failure to try this leads to civilian demise, that may be a conflict crime.”
The definition between offensive and defensive, too, is fuzzy. “One among my favourite workout routines with my college students is placing up a army expertise after which asking, ‘Is that this defensive or offensive?’” Reddie mentioned, pointing to the instance of a nuclear weapon in a silo, which some would argue is protection as a result of it’s getting used to discourage, and a few would argue is offense as a result of it’s designed to hit international targets. “Simply because its major perform … is defensive in nature doesn’t imply the expertise itself is defensive in nature.”
Sure missile protection programs could have crossed the road of autonomous response many years in the past
Sorin Adam Matei, a professor at Purdue College, was blunter: “You can’t struggle a conflict solely in protection.”
In 2023, the federal government’s Chief Digital and Synthetic Intelligence Workplace (CDAO) — which is the central hub of all of the DOD’s AI operations — issued an update to DOD Directive 3000.09. Nevertheless it didn’t resolve the doc’s core ambiguities. In 2024, the Biden administration revealed a memorandum on AI and nationwide safety, establishing guidelines for the way AI can be utilized in sure nationwide safety situations — and for now, even below Trump, that coverage is in pressure. However the Pentagon has seen important upheaval. The CDAO is presently present process a big restructuring that makes it extra remoted from the remainder of the DOD, and the workplace now studies to Emil Michael, who’s each the DOD’s undersecretary of analysis and engineering and the division’s CTO.
Worldwide efforts, such because the Conference on Sure Standard Weapons and the Marketing campaign to Cease Killer Robots, haven’t been capable of make a lot progress both. Marijan advised The Verge that although the CCW has significantly helped smaller nations to know the panorama round AI warfare, progress total has “been very gradual and we haven’t seen concrete settlement, notably among the many main nations and the extra subtle militaries.” Although some nations have expressed curiosity in a ban on deadly autonomous weapons, there’s not even an official worldwide definition of the time period. The state of affairs “usually leads to lots of people speaking previous one another … and a few nations discover the dearth of a binding instrument to be to their benefit,” mentioned Sarah Shoker, a senior analysis scholar on the College of California, Berkeley and former lead of OpenAI’s geopolitics workforce.
Some nations have expressed curiosity in a ban on deadly autonomous weapons, however there’s not even an official worldwide definition
“I believe most individuals — policymakers, civil society members … who attend these conferences are possible drained,” Shoker added. “It’s been over a decade, and there’s actually no settlement.”
Regardless of the standing of autonomous weapons earlier than 2017, Maven accelerated their timeline dramatically — and adjusted the phrases of debate. This system marked the primary time the idyllic, “don’t be evil” Silicon Valley of the 2010s was clearly concerned in warfare, forcing Google workers and the general public to react. “That was the primary nationwide dialog we had in regards to the position of AI applied sciences in army operations,” mentioned Matei.
Mission Maven started with a memo from Robert Work, then deputy protection secretary. He established an “Algorithmic Warfare Cross-Useful Crew” tasked with coaching AI algorithms that might be deployed to war zones, possibly even by year-end. Google was quickly awarded the contract.
Inner resistance grew shortly. In April 2018, about 4,000 workers signed a petition demanding that Google take away itself from the “enterprise of conflict.” The letter mentioned a Google government had assured Googlers their tech “is not going to ‘function or fly drones’ and ‘is not going to be used to launch weapons.’” However “whereas this eliminates a slim set of direct functions, the expertise is being constructed for the army, and as soon as it’s delivered it might simply be used to help in these duties,” the staff mentioned. “We can’t outsource the ethical accountability of our applied sciences to 3rd events.” Google didn’t reply to a request for remark for this story.
Just lately, MSS has reportedly been integral to each the seize of Venezuelan President Nicolás Maduro and the US strikes on Iran, which have reportedly killed 1000’s so far within the area.
“You can’t struggle a conflict solely in protection.”
Google workers argued their firm ought to take a stand — and it did, selecting to not renew its contract amid the controversy in mid-2018. However Amazon and Microsoft shortly swooped in to choose up tens of thousands and thousands of {dollars} in contracts for a similar work. Palantir quickly took over, and Mission Maven turned the Maven Sensible System (MSS), which not solely permits for object detection and monitoring but in addition analyzing surveillance knowledge on a big scale.
Claude is intertwined with MSS too, albeit in a extra restricted capability. After Anthropic started permitting army use of its expertise in 2024, its Claude chatbot was added to the person interface, serving to analysts question the system about sure geographical areas and sorts of intelligence or targets. Even this comparatively minor addition, some specialists say, has made the system extra environment friendly. Because of this, Claude enabled concentrating on, doubtlessly putting many extra people.
The sheer quantity of targets might make any significant human supervision troublesome, mentioned Shoker. “What we find out about MSS is that it reduces the variety of human beings within the concentrating on cycle — and that’s truly by design.”
Whereas Anthropic might need been all proper lowering human intervention, it’s pushed again towards setting it to zero. As Google discovered with Mission Maven, although, opponents are greater than keen to fill the hole.
Different AI firms have jumped on the likelihood to work with the Pentagon. Even earlier than its struggle with Anthropic started, a number of main AI labs had loosened their operating guidelines or tweaked their mission statements to permit for army offers — together with Google, OpenAI, and arguably Anthropic itself. OpenAI shortly signed onto the terms Anthropic had spurned. And within the months after snubbing Anthropic, the Division of Protection signed offers with eight firms to deploy their AI on labeled networks: Google, Microsoft, Amazon Internet Providers, Nvidia, OpenAI, Reflection, Oracle, and SpaceX.
AI firms have jumped on the likelihood to work with the Pentagon
Shoker mentioned that whereas working at OpenAI — one of many sector’s largest gamers — she didn’t work together with many individuals learning the dangers of decision-support programs or autonomous weapons programs. The analysis group in these areas continues to be “extraordinarily siloed,” she mentioned. “Individuals on this area have been warning and researching about (these dangers) for some time,” Shoker mentioned, including, “The issue … is that these individuals weren’t within the labs.” OpenAI didn’t reply to a request for remark.
Internally, workers have voiced dissent in ways in which strongly echo the pushback towards Mission Maven virtually a decade in the past. Externally, OpenAI’s Pentagon deal sparked widespread controversy, main CEO Sam Altman to announce he was making an attempt to renegotiate the deal. However Silicon Valley executives are aggressively pushing again towards worker organizing and talking out, together with by using AI to identify leakers. And lots of tech staff already worry for his or her jobs in an period when AI is ready to switch entry-level roles at their very own companies.
In the meantime, by its courtroom struggle with the Trump administration, Anthropic continues to be attempting to regain its place as a army and intelligence contractor. Its expertise has been deeply enmeshed within the DOD’s operations since 2024, and final summer season it launched Claude Gov, a product designed particularly to be used by nationwide safety businesses with loosened guardrails round labeled supplies.
Anthropic is risking a high-stakes loss because it reportedly prepares to IPO this yr. It’s reportedly in talks with traders to boost funding at a $900 billion valuation. There’s extra investor stress for the lab to show a revenue than ever before.
In current months, Anthropic CEO Dario Amodei has held agency on mass surveillance for Individuals, however he’s demonstrated no drawback with — and actually expressed his support for — such surveillance for everybody else.
Anthropic’s “very slim” crimson strains “don’t go far sufficient to guard human rights or to adjust to worldwide legislation,” mentioned Tech Justice Regulation’s Batt. “Anthropic particularly talks about mass home surveillance of US individuals as posing grave civil liberties considerations, however the identical civil liberties considerations apply with equal pressure to non-US individuals,” she added. “The truth is, we should always in all probability be particularly involved, given the brutal immigration crackdown, about superior capabilities for mass surveillance of non-citizens.”
Anthropic CEO Dario Amodei has decried mass surveillance for Individuals however demonstrated no drawback with such surveillance for everybody else
That doesn’t even cowl the ethics of mass surveillance overseas, a difficulty that’s strikingly absent from most ethics conversations taking place stateside.
And whereas Amodei believes Anthropic’s programs aren’t able to assist absolutely autonomous weapons, he’s not conceptually or morally against them. In a blog post, he mentioned that “absolutely autonomous weapons (those who take people out of the loop completely and automate deciding on and interesting targets) could show essential for our nationwide protection.” Amodei even mentioned he was glad to “work immediately with the Division of Conflict on R&D to enhance the reliability of those programs” and velocity up the timeline for the corporate’s assist in deploying them.
That place, Tech Justice Regulation’s Batt mentioned, is “essentially in rigidity” with worldwide humanitarian legislation, which requires case-by-case assessments of any given assault. “The adoption of AI into these concentrating on processes is basically primarily based on a ‘velocity wins’ ethos,” she mentioned. “No matter when within the course of people are looped in, if the objective of implementing these programs is to remodel one thing that took weeks or days into seconds, people aren’t meaningfully making that evaluation for themselves.”
When Hegseth introduced his intent to redraft the Pentagon’s AI contracts, he explicitly said that the DOD “should settle for that the dangers of not transferring quick sufficient outweigh the dangers of imperfect alignment” — and {that a} important quantity of testing and analysis have to be achieved away with.
“The adoption of AI into these concentrating on processes is basically primarily based on a ‘velocity wins’ ethos.”
Previous administrations have sought to take away “purely bureaucratic” layers of warfare whereas preserving security, Chaudhry mentioned, however the Hegseth memo “explicitly units up a tradeoff and says that we favor velocity.” This isn’t usually the very best blanket method relating to army operations, specialists say, however it’s usually the deadliest. AI has already enabled that lethality — and it’s poised to go even additional.
For the reason that Hegseth memo, says Marijan, extra nations are listening to the dangers of autonomous warfare. However towards the American army and its companions in Silicon Valley, it’s not clear how a lot they’ll do. Proper now, they’re merely attempting to know its implications. If people are “simply rubber-stamping choices,” Marijan mentioned, “what are the strains of accountability there — and what does the legal responsibility appear to be?”
Throughout a Protection One Tech summit in July 2017, the place Maven was centered as the way forward for the division, the chief of the Algorithmic Warfare Cross-Useful Crew presented to an viewers.
AI, he mentioned, wouldn’t be deciding on a goal in fight “anytime quickly.” However now, with out intervention, that second isn’t far off.
