Archive · War & Autonomous Systems

Autonomous Systems in War.

Removing the human decision from the act of lethal targeting does not remove the deaths. It removes the accountability for them, and lowers the threshold at which they occur.

Filed under the public record7 min readBylined: Honne and Tatemae

A civilization that hands the act of killing to machines has already surrendered the moral authority to ask what kind of civilization it is.

The specific line being asked for

What is being requested is not a ban on military artificial intelligence. It is not a ban on defensive systems, or on logistics systems, or on intelligence analysis systems, or on the many uses of automation that already shape modern armed forces.

It is a categorical prohibition on weapon systems that select and engage human targets without meaningful human control. The line is drawn at the act of killing, not at the use of computation. This distinction is well understood by the militaries that have studied it, by the lawyers who have drafted around it, and by the diplomats who have negotiated similar lines for other categories of weapon.

It does not remove the deaths. It removes the responsibility for them.

Why accountability is the core issue

Every legal and ethical framework governing the use of lethal force, in every functioning society, rests on the assumption that a human being is accountable for the act. Accountability presumes a decision maker who can be questioned, charged, judged, and held responsible. The entire architecture of military law, of the laws of armed conflict, of war crimes prosecution, of civilian protection, presupposes such a person.

A weapon that selects its own targets removes that person. It does not remove the deaths. It distributes the responsibility for them across a chain of designers, vendors, operators, and procurement officers, none of whom, individually, made the decision to kill. In practice, accountability for the resulting civilian harm becomes legally diffuse and politically convenient to no one to pursue.

Why the threshold matters

Wars become more likely, not less, when the human and political cost of initiating them falls. The development of fully autonomous lethal systems lowers that cost in a specific and dangerous way. It removes the need to put a soldier in a position where they might refuse an order. It removes the need to ask a public for the casualty count it will accept. It removes, in short, the political friction that has historically constrained the use of force.

Once such weapons are normalized in any military, the pressure to match them becomes uniform across all militaries. Once they are exported, they become available to actors with no obligation to international humanitarian law. Once they are cheap, they become available to anyone. There is no version of this story in which their diffusion is contained by the goodwill of the firms that built them.

Some categories of weapon, once normalized, cannot be put back. The window for refusing this one is still open. Briefly.

Continue the record