Anthropic’s fight with the Pentagon highlights the risks of autonomous warfare — but obscures just how close it is.
The Convention on Certain Conventional Weapons, an international forum that focuses on lethal autonomous systems, is hosted twice a year at the United Nations in Geneva. When Branka Marijan attended in November 2017, she thought the five-day sessions — which dealt largely in hypotheticals, speculating on a world where warfare was fought with killer robots — would be business as usual. After all, this was technology some thought might never be developed, and likely never deployed. That year, she quickly realized, was different. That distant, imagined future was suddenly closer and realer than ever.
On the first day, some attendees watched ashort filmcalledSlaughterbots, put together by the Future of Life Institute. The video featured a fictional defense contractor pitching an AI-powered drone that could kill unassisted with precision strikes. “They used to say guns don’t kill people, people do,” its CEO tells the audience. “But people don’t. They get emotional, disobey orders, aim high. Let’s watch the weapons make the decisions.” The mood in the room, Marijan recalls, suddenly turned apprehensive. The most frightening part wasn’t the premise — it was that the Pentagon was already developing a version of this technology.
That meeting was the first one held after the start of Project Maven, a US Department of Defense initiative using AI to analyze drone surveillance footage. And by late 2017, Maven had a major tech company on board: Google. “The systems we were talking about were not futuristic,” said Marijan, who is a senior researcher at Project Ploughshares, a peace-focused independent research institute. “They were existing platforms that had degrees of autonomy in them, or the capability to select and engage targets based on sensor data and sensor input.”
The world had already seen drone warfare — deadly machines directed by humans. Now, it was looking at a future where humans may be removed from the loop entirely. “These were not these Terminator-like figures that we were concerned about, but really what was happening with the enablement of autonomy,” said Marijan.
The US military has backed AI development for decades, and in turn, AI has transformed warfare
Nearly a decade later, militaries haven’t yet developed fully autonomous lethal weapons. But these systems sit squarely in the center of a recent high-stakes battle between the US government and AI startup Anthropic. Anthropic is seeking to preserve two “red lines”: bans on domestic mass surveillance and on weapons that can identify, track, and kill targets with zero human involvement. Since the start of the year, it’s emerged as the only military AI contractor to place meaningful limits on what experts call one of the final frontiers of AI warfare.
But amid shifting alliances, lawsuits, and melodrama, it’s easy to lose sight of the larger context — that AI is, and long has been, deeply embedded in the military. Seventy years ago, a summermeetingbetween scientists in New Hampshire made the Department of Defense sit up and take notice of AI’s potential for war. Since then, its influence has grown exponentially every decade. In recent years in particular, AI has enabled more and faster killings than ever before.
Even Anthropic seems to think its red lines won’t hold for long. After all, history has proven otherwise.
The US military has backed AI development for decades, and in turn, AI has transformed warfare. In the 2000s, the technology became capable of parsing unprecedented amounts of data collected worldwide, creating a surveillance revolution. And the late 2010s saw the development of advanced facial recognition and other sophisticated machine vision systems.
The fight between Anthropic and the Pentagon has drawn attention to the growing power of these systems. It began in January of 2026, when Defense Secretary Pete Hegsethdemanded a renegotiationof the DOD’s existing AI contracts. The terms did away with any gray areas or previously agreed-upon terms, allowing the Pentagon to use the companies’ technology within the vague and expansive limits of “any lawful use.” Anthropic — the only AI company approved to deploy its tech on the Pentagon’s classified networks at that point — objected.
Even Anthropic seems to think its red lines won’t hold for long
A contractor like Anthropic setting limits on specific uses of its tech is unusual. “It’s not government-created technology in the way that the Manhattan Project was,” nor a conventional military supplier like Northrop Grumman, said Andrew Reddie, an associate research professor of public policy at the University of California, Berkeley. “This is one of the pain points that’s made clear when you’ve got this startup ecosystem engaging with the Pentagon directly.” Even within Silicon Valley, he said, there’s “a lot of disagreement” over when limits should be set.
It’s also unclear who will win. As a negotiation tactic, the DOD designated Anthropic a military supply chain risk in March, and President Donald Trump declared he was banning all government agencies from using its Claude system. The relationship has apparentlywarmed somewhatsince then, with the release of Anthropic’s cybersecurity-focused model Mythos, but a court battle is still playing out. Anthropic declined to provide a comment for this story.
Either way, the debate has brought “fully autonomous weapons” into the public lexicon in a new way. But AI’s creeping influence in military operations hasn’t slowed down in decades.
“We’ve kind of crossed the rubicon while we pretend that we haven’t,” Reddie said.
At the center of the debates is DOD Directive 3000.09, one of the only policies governing the use of lethal autonomous weapons. Originally writtenin 2012, it defines such a system as one that, “once activated, can select and engage targets without further intervention by an operator.” And it decrees that both fully autonomous and semi-autonomous weapons be designed to allow humans to “exercise appropriate levels” of judgment over the use of force.
The directive set up the “first policy on the use of autonomy in warfare,” said Hamza Chaudhry, who leads AI and national security at the Future of Life Institute.
“Even without full autonomy, AI compresses kill chains to mere seconds.”
Depending on how you interpret the definition, however, certain missile defense programs may have crossed that line decades ago. Take the Phalanx CIWS, for instance. It’s an automated weapon system resembling a very large gun, built to defend naval vessels from incoming missile attacks. That type of system wouldn’t work if there were a human in the loop, since it has to respond in milliseconds.
The difference, some experts say, is that these systems operate solely in a defense-only, fixed environment. They’reengaging, this interpretation goes, but notdeciding— just reacting to an incoming threat. “The ‘and’ is doing a lot of work inside of that statute — we have systems that can decide and systems that can engage but you can’t have a system that does both,” Reddie said.
“Even without full autonomy, AI compresses kill chains to mere seconds so that humans are not actually making the assessments that international humanitarian law requires to prevent civilian harm,” said Maddy Batt, legal fellow at Tech Justice Law. “When humans’ failure to do that results in civilian death, that is a war crime.”
The definition between offensive and defensive, too, is fuzzy. “One of my favorite exercises with my students is putting up a military technology and then asking, ‘Is this defensive or offensive?’” Reddie said, pointing to the example of a nuclear weapon in a silo, which some would argue is defense because it’s being used to deter, and some would argue is offense because it’s designed to hit foreign targets. “Just because its primary function … is defensive in nature doesn’t mean the technology itself is defensive in nature.”
Certain missile defense systems may have crossed the line of autonomous response decades ago
Sorin Adam Matei, a professor at Purdue University, was blunter: “You cannot fight a war only in defense.”
In 2023, the government’s Chief Digital and Artificial Intelligence Office (CDAO) — which is the central hub of all the DOD’s AI operations — issued anupdateto DOD Directive 3000.09. But it didn’t resolve the document’s core ambiguities. In 2024, the Biden administration published amemorandumon AI and national security, setting up rules for how AI can be used in certain national security scenarios — and for now, even under Trump, that policy is in force. But the Pentagon has seen significant upheaval. The CDAO is currently undergoing a significant restructuring that makes it more isolated from the rest of the DOD, and the office now reports to Emil Michael, who is both the DOD’s undersecretary of research and engineering and the department’s CTO.
International efforts, such as the Convention on Certain Conventional Weapons and the Campaign to Stop Killer Robots, haven’t been able to make much progress either. Marijan toldThe Vergethat though the CCW has greatly helped smaller countries to understand the landscape around AI warfare, progress overall has “been very slow and we haven’t seen concrete agreement, particularly among the major countries and the more sophisticated militaries.” Though some countries have expressed interest in a ban on lethal autonomous weapons, there’s not even an official international definition of the term. The situation “often results in a lot of people talking past each other … and some countries find the lack of a binding instrument to be to their advantage,” said Sarah Shoker, a senior research scholar at the University of California, Berkeley and former lead of OpenAI’s geopolitics team.
Some countries have expressed interest in a ban on lethal autonomous weapons, but there’s not even an official international definition
“I think most people — policymakers, civil society members … who attend these meetings are likely tired,” Shoker added. “It’s been over a decade, and there is really no agreement.”