SUBSCRIBE TO OUR FREE NEWSLETTER
Daily news & progressive opinion—funded by the people, not the corporations—delivered straight to your inbox.
5
#000000
#FFFFFF
5
#000000
#FFFFFF
autonomous weapons
Pentagon Urged to Just Say No to AI-Powered Killer Robots
"The Department of Defense should declare its opposition to the development and deployment of autonomous weapons."
Mar 26, 2024
The watchdog group Public Citizen on Tuesday led a letter urging Pentagon leaders "to clarify that the Replicator Initiative will not involve the development and deployment of autonomous weapons systems," also known as "killer robots."
Last September, U.S. Deputy Secretary of Defense Kathleen Hicks "asserted that the development of all-domain, attributable autonomy systems (ADA2) is an essential way for the Pentagon to maintain its comparative cutting-edge and keep up with the technological advancements of other states," notes the letter, which was addressed to her and Defense Secretary Lloyd Austin.
"However, those comments failed to specify whether or not supporting autonomous weapons systems is one of the key focuses of this initiative," the letter stresses. "When addressing whether or not 'ADA2 means weapons systems,' Secretary Hicks stated: 'That's a serious question to be sure. They are not synonymous. There are many applications for ADA2 systems beyond delivering weapons effects.'"
"Autonomous weapons are inherently dehumanizing and unethical, no matter whether a human is 'ultimately' responsible for the use of force or not."
Public Citizen and the 13 other organizations argued that "this is no place for strategic ambiguity. Autonomous weapons are inherently dehumanizing and unethical, no matter whether a human is 'ultimately' responsible for the use of force or not."
Deploying lethal weapons that rely on artificial intelligence (AI) "in battlefield conditions necessarily means inserting them into novel conditions for which they have not been programmed, an invitation for disastrous outcomes," the groups warned. "'Swarms' of the sort envisioned by Replicator pose even heightened risks, because of the unpredictability of how autonomous systems will function in a network. And the mere ambiguity of the U.S. position on autonomous weapons risks spurring a catastrophic arms race."
"We believe the Department of Defense should declare its opposition to the development and deployment of autonomous weapons," the coalition concluded. "However, even if you are not prepared to make that declaration, we strongly urge you to clarify that the Replicator Initiative will not employ autonomous weapons."
In addition to Public Citizen, the coalition included the American Friends Service Committee, Autistic Women & Nonbinary Network, Backbone Campaign, Demand Progress Education Fund, Fight for the Future, Future of Life, National Priorities Project at the Institute for Policy Studies, RootsAction.org, United Church of Christ, the Value Alliance, Women's International League for Peace and Freedom U.S., Win Without War, and World Beyond War.
The letter comes on the heels of Public Citizen releasing a report about the rise of killer robots, AI Joe: The Dangers of Artificial Intelligence and the Military.
The February report addresses the Pentagon's AI policy, the dangers of killer robots, the need to ensure decisions about nuclear weapons aren't made by automated systems, how artificial intelligence can increase not diminish the use of violence, risks of using deepfakes on the battlefield, and how AI startups are seeking government contracts.
The publication concludes with recommendations that Public Citizen president Robert Weissman echoed in a statement Tuesday.
"The United States should state plainly that it will not create or deploy killer robots and should work to advance global treaty negotiations to ban such weapons," Weissman said. "At minimum, the United States should commit that the Replicator Initiative will not involve the use of autonomous weapons."
"Ambiguity about the Replicator program essentially ensures a catastrophic arms race over autonomous weapons," he added. "That's a race in which all of humanity is the loser."
Keep ReadingShow Less
New Report Warns of the Rise of Killer Robots
Public Citizen says autonomous weapons "inherently dehumanize the people targeted and make it easier to tolerate widespread killing."
Feb 29, 2024
A Public Citizen report released on Thursday outlines the risks of the Pentagon adopting the use of autonomous weapons on the battlefield and how this could occur in the not-too-distant future.
The report focuses on the Pentagon's efforts to utilize artificial intelligence in various ways and the American military contractors that are developing weaponry that utilizes AI, including autonomous weapons.
"The single greatest concern involving AI and the Pentagon is the integration of AI into weapons systems such that they can function autonomously, delivering lethal force without intervention or meaningful human control," the report state. "The most serious worry involving autonomous weapons is that they inherently dehumanize the people targeted and make it easier to tolerate widespread killing, including in violation of international human rights law."
"Autonomous weapons are already in development around the world and racing forward."
Though the Pentagon has instituted some policies to ensure AI will be used ethically by the military, the report states, these policies don't go as far as to ban the use of autonomous weapons—often referred to as "killer robots."
Furthermore, the report notes that military contractors like General Dynamics, Vigor Industrial, Anduril Industries, and others are currently developing unmanned tanks, submarines, and drones.
"As a discrete weapons technology, autonomous weapons deployment is nearly certain to create an AI weapons arms race. That is the logic of international military strategy. In the United States, a geopolitical rivalry-driven autonomous weapons arms race will be spurred further by the military-industrial complex and corporate contractors," the report states. "Autonomous weapons are already in development around the world and racing forward."
Israel has purchased and at times deployed self-piloting, lethal drones. Russia has utilized autonomous drones in Ukraine. The age of the killer robot is here, and Public Citizen warns that the U.S. needs to help prevent the deployment of killer robots from being normalized.
"The United States should pledge not to develop or deploy autonomous weapons, and should support a global treaty banning such weapons," the report states.
Public Citizen also recommends the U.S. adopt policies that prevent using AI to launch nuclear weapons or using deepfakes on the battlefield, which could be used for "influence operations."
Keep ReadingShow Less
When the AI-Powered Killer Drones Start Talking to Each Other
It should be obvious that humanity could face a significant risk to its existence, should killing machines acquire the ability to think on their own.
Feb 20, 2024
Yes, it’s already time to be worried — very worried. As the wars in Ukraine and Gaza have shown, the earliest drone equivalents of “killer robots” have made it onto the battlefield and proved to be devastating weapons. But at least they remain largely under human control. Imagine, for a moment, a world of war in which those aerial drones (or their ground and sea equivalents) controlled us, rather than vice-versa. Then we would be on a destructively different planet in a fashion that might seem almost unimaginable today. Sadly, though, it’s anything but unimaginable, given the work on artificial intelligence (AI) and robot weaponry that the major powers have already begun. Now, let me take you into that arcane world and try to envision what the future of warfare might mean for the rest of us.
By combining AI with advanced robotics, the U.S. military and those of other advanced powers are already hard at work creating an array of self-guided “autonomous” weapons systems — combat drones that can employ lethal force independently of any human officers meant to command them. Called “killer robots” by critics, such devices include a variety of uncrewed or “unmanned” planes, tanks, ships, and submarines capable of autonomous operation. The U.S. Air Force, for example, is developing its “collaborative combat aircraft,” an unmanned aerial vehicle (UAV) intended to join piloted aircraft on high-risk missions. The Army is similarly testing a variety of autonomous unmanned ground vehicles (UGVs), while the Navy is experimenting with both unmanned surface vessels (USVs) and unmanned undersea vessels (UUVs, or drone submarines). China, Russia, Australia, and Israel are also working on such weaponry for the battlefields of the future.
The imminent appearance of those killing machines has generated concern and controversy globally, with some countries already seeking a total ban on them and others, including the U.S., planning to authorize their use only under human-supervised conditions. In Geneva, a group of states has even sought to prohibit the deployment and use of fully autonomous weapons, citing a 1980 U.N. treaty, the Convention on Certain Conventional Weapons, that aims to curb or outlaw non-nuclear munitions believed to be especially harmful to civilians. Meanwhile, in New York, the U.N. General Assembly held its first discussion of autonomous weapons last October and is planning a full-scale review of the topic this coming fall.
Some American strategists have championed an alternative approach to the use of autonomous weapons on future battlefields in which they would serve not as junior colleagues in human-led teams but as coequal members of self-directed robot swarms.
For the most part, debate over the battlefield use of such devices hinges on whether they will be empowered to take human lives without human oversight. Many religious and civil society organizations argue that such systems will be unable to distinguish between combatants and civilians on the battlefield and so should be banned in order to protect noncombatants from death or injury, as is required by international humanitarian law. American officials, on the other hand, contend that such weaponry can be designed to operate perfectly well within legal constraints.
However, neither side in this debate has addressed the most potentially unnerving aspect of using them in battle: the likelihood that, sooner or later, they’ll be able to communicate with each other without human intervention and, being “intelligent,” will be able to come up with their own unscripted tactics for defeating an enemy — or something else entirely. Such computer-driven groupthink, labeled “emergent behavior” by computer scientists, opens up a host of dangers not yet being considered by officials in Geneva, Washington, or at the U.N.
For the time being, most of the autonomous weaponry being developed by the American military will be unmanned (or, as they sometimes say, “uninhabited”) versions of existing combat platforms and will be designed to operate in conjunction with their crewed counterparts. While they might also have some capacity to communicate with each other, they’ll be part of a “networked” combat team whose mission will be dictated and overseen by human commanders. The Collaborative Combat Aircraft, for instance, is expected to serve as a “loyal wingman” for the manned F-35 stealth fighter, while conducting high-risk missions in contested airspace. The Army and Navy have largely followed a similar trajectory in their approach to the development of autonomous weaponry.
The Appeal of Robot “Swarms”
However, some American strategists have championed an alternative approach to the use of autonomous weapons on future battlefields in which they would serve not as junior colleagues in human-led teams but as coequal members of self-directed robot swarms. Such formations would consist of scores or even hundreds of AI-enabled UAVs, USVs, or UGVs — all able to communicate with one another, share data on changing battlefield conditions, and collectively alter their combat tactics as the group-mind deems necessary.
“Emerging robotic technologies will allow tomorrow’s forces to fight as a swarm, with greater mass, coordination, intelligence and speed than today’s networked forces,” predicted Paul Scharre, an early enthusiast of the concept, in a 2014 report for the Center for a New American Security (CNAS). “Networked, cooperative autonomous systems,” he wrote then, “will be capable of true swarming — cooperative behavior among distributed elements that gives rise to a coherent, intelligent whole.”
As Scharre made clear in his prophetic report, any full realization of the swarm concept would require the development of advanced algorithms that would enable autonomous combat systems to communicate with each other and “vote” on preferred modes of attack. This, he noted, would involve creating software capable of mimicking ants, bees, wolves, and other creatures that exhibit “swarm” behavior in nature. As Scharre put it, “Just like wolves in a pack present their enemy with an ever-shifting blur of threats from all directions, uninhabited vehicles that can coordinate maneuver and attack could be significantly more effective than uncoordinated systems operating en masse.”
In 2014, however, the technology needed to make such machine behavior possible was still in its infancy. To address that critical deficiency, the Department of Defense proceeded to fund research in the AI and robotics field, even as it also acquired such technology from private firms like Google and Microsoft. A key figure in that drive was Robert Work, a former colleague of Paul Scharre’s at CNAS and an early enthusiast of swarm warfare. Work served from 2014 to 2017 as deputy secretary of defense, a position that enabled him to steer ever-increasing sums of money to the development of high-tech weaponry, especially unmanned and autonomous systems.
From Mosaic to Replicator
Much of this effort was delegated to the Defense Advanced Research Projects Agency (DARPA), the Pentagon’s in-house high-tech research organization. As part of a drive to develop AI for such collaborative swarm operations, DARPA initiated its “Mosaic” program, a series of projects intended to perfect the algorithms and other technologies needed to coordinate the activities of manned and unmanned combat systems in future high-intensity combat with Russia and/or China.
“Applying the great flexibility of the mosaic concept to warfare,” explained Dan Patt, deputy director of DARPA’s Strategic Technology Office, “lower-cost, less complex systems may be linked together in a vast number of ways to create desired, interwoven effects tailored to any scenario. The individual parts of a mosaic are attritable [dispensable], but together are invaluable for how they contribute to the whole.”
This concept of warfare apparently undergirds the new “Replicator” strategy announced by Deputy Secretary of Defense Kathleen Hicks just last summer. “Replicator is meant to help us overcome [China’s] biggest advantage, which is mass. More ships. More missiles. More people,” she told arms industry officials last August. By deploying thousands of autonomous UAVs, USVs, UUVs, and UGVs, she suggested, the U.S. military would be able to outwit, outmaneuver, and overpower China’s military, the People’s Liberation Army (PLA). “To stay ahead, we’re going to create a new state of the art… We’ll counter the PLA’s mass with mass of our own, but ours will be harder to plan for, harder to hit, harder to beat.”
To obtain both the hardware and software needed to implement such an ambitious program, the Department of Defense is now seeking proposals from traditional defense contractors like Boeing and Raytheon as well as AI startups like Anduril and Shield AI. While large-scale devices like the Air Force’s Collaborative Combat Aircraft and the Navy’s Orca Extra-Large UUV may be included in this drive, the emphasis is on the rapid production of smaller, less complex systems like AeroVironment’s Switchblade attack drone, now used by Ukrainian troops to take out Russian tanks and armored vehicles behind enemy lines.
At the same time, the Pentagon is already calling on tech startups to develop the necessary software to facilitate communication and coordination among such disparate robotic units and their associated manned platforms. To facilitate this, the Air Force asked Congress for $50 million in its fiscal year 2024 budget to underwrite what it ominously enough calls Project VENOM, or “Viper Experimentation and Next-generation Operations Model.” Under VENOM, the Air Force will convert existing fighter aircraft into AI-governed UAVs and use them to test advanced autonomous software in multi-drone operations. The Army and Navy are testing similar systems.
When Swarms Choose Their Own Path
In other words, it’s only a matter of time before the U.S. military (and presumably China’s, Russia’s, and perhaps those of a few other powers) will be able to deploy swarms of autonomous weapons systems equipped with algorithms that allow them to communicate with each other and jointly choose novel, unpredictable combat maneuvers while in motion. Any participating robotic member of such swarms would be given a mission objective (“seek out and destroy all enemy radars and anti-aircraft missile batteries located within these [specified] geographical coordinates”) but not be given precise instructions on how to do so. That would allow them to select their own battle tactics in consultation with one another. If the limited test data we have is anything to go by, this could mean employing highly unconventional tactics never conceived for (and impossible to replicate by) human pilots and commanders.
The propensity for such interconnected AI systems to engage in novel, unplanned outcomes is what computer experts call “emergent behavior.” As ScienceDirect, a digest of scientific journals, explains it, “An emergent behavior can be described as a process whereby larger patterns arise through interactions among smaller or simpler entities that themselves do not exhibit such properties.” In military terms, this means that a swarm of autonomous weapons might jointly elect to adopt combat tactics none of the individual devices were programmed to perform — possibly achieving astounding results on the battlefield, but also conceivably engaging in escalatory acts unintended and unforeseen by their human commanders, including the destruction of critical civilian infrastructure or communications facilities used for nuclear as well as conventional operations.
At this point, of course, it’s almost impossible to predict what an alien group-mind might choose to do if armed with multiple weapons and cut off from human oversight. Supposedly, such systems would be outfitted with failsafe mechanisms requiring that they return to base if communications with their human supervisors were lost, whether due to enemy jamming or for any other reason. Who knows, however, how such thinking machines would function in demanding real-world conditions or if, in fact, the group-mind would prove capable of overriding such directives and striking out on its own.
What then? Might they choose to keep fighting beyond their preprogrammed limits, provoking unintended escalation — even, conceivably, of a nuclear kind? Or would they choose to stop their attacks on enemy forces and instead interfere with the operations of friendly ones, perhaps firing on and devastating them (as Skynet does in the classic science fiction Terminator movie series)? Or might they engage in behaviors that, for better or infinitely worse, are entirely beyond our imagination?
Top U.S. military and diplomatic officials insist that AI can indeed be used without incurring such future risks and that this country will only employ devices that incorporate thoroughly adequate safeguards against any future dangerous misbehavior. That is, in fact, the essential point made in the “Political Declaration on Responsible Military Use of Artificial Intelligence and Autonomy” issued by the State Department in February 2023. Many prominent security and technology officials are, however, all too aware of the potential risks of emergent behavior in future robotic weaponry and continue to issue warnings against the rapid utilization of AI in warfare.
Of particular note is the final report that the National Security Commission on Artificial Intelligence issued in February 2021. Co-chaired by Robert Work (back at CNAS after his stint at the Pentagon) and Eric Schmidt, former CEO of Google, the commission recommended the rapid utilization of AI by the U.S. military to ensure victory in any future conflict with China and/or Russia. However, it also voiced concern about the potential dangers of robot-saturated battlefields.
“The unchecked global use of such systems potentially risks unintended conflict escalation and crisis instability,” the report noted. This could occur for a number of reasons, including “because of challenging and untested complexities of interaction between AI-enabled and autonomous weapon systems [that is, emergent behaviors] on the battlefield.” Given that danger, it concluded, “countries must take actions which focus on reducing risks associated with AI-enabled and autonomous weapon systems.”
When the leading advocates of autonomous weaponry tell us to be concerned about the unintended dangers posed by their use in battle, the rest of us should be worried indeed. Even if we lack the mathematical skills to understand emergent behavior in AI, it should be obvious that humanity could face a significant risk to its existence, should killing machines acquire the ability to think on their own. Perhaps they would surprise everyone and decide to take on the role of international peacekeepers, but given that they’re being designed to fight and kill, it’s far more probable that they might simply choose to carry out those instructions in an independent and extreme fashion.
If so, there could be no one around to put an R.I.P. on humanity’s gravestone.
Keep ReadingShow Less
Most Popular