Aspiration vs reality: the use of AI in autonomous weapon systems 

1

Lead on international engagement of autonomous weapons 

The UK, like all states, is bound by obligations under international law with respect to the development and use of new weapons. The use of autonomous weapons in armed conflict is primarily governed by International Humanitarian Law.

The international community has been debating the regulation of autonomous weapons for several years. Outcomes from this debate could be a legally binding treaty or non-binding measures clarifying the application of international humanitarian law - each approach has its advocates. Despite differences on form, the key goal is accelerating efforts to achieving an effective international instrument.  

We call for a swift agreement of an effective international instrument on lethal autonomous weapons. It is crucial to develop an international consensus on what criteria should be met in order for a system to be compliant with international humanitarian law. Central to this is the retention of human moral agency. Non-compliant systems should be prohibited. Consistent with its ambitions to promote the safe and responsible development of AI around the world, the Government should be a leader in this effort.

2

Lead international engagement to prohibit the use of AI in nuclear command, control and communications  

While advances in AI have the potential to bring greater effectiveness to nuclear command, control and communications, use of AI also has the potential to spur arms races or increase the likelihood of states escalating to nuclear use – either intentionally or accidentally – during a crisis. The compressed time for decision-making when using AI may lead to increased tensions, miscommunication, and misunderstanding. Moreover, an AI tool could be hacked, its training data compromised, or its outputs interpreted as fact when they are statistical correlations, all leading to potentially catastrophic outcomes. 

The risks inherent in current AI systems, combined with their enhanced escalatory risk, are of particular concern in the context of nuclear command, control and communications. The Government should lead international efforts to achieve a prohibition on the use of AI in nuclear command, control and communications.

3

Adopt an operational definition of autonomous weapons

There is no single accepted definition of what constitutes an autonomous weapon; and clarifying the term has been a focus of international policymaking for many years. Many states and international organisations have adopted working definitions of what constitutes an autonomous weapon or an autonomous system. 

Surprisingly, the UK does not have an operational definition of autonomous weapons. The Ministry of Defence has stated it is cautious about adopting a definition.  

However, we believe it is possible to create a future-proofed definition. Doing so would aid the UK’s ability to make meaningful policy on autonomous weapons and engage fully in discussions in international fora. Other states and organisations have adopted flexible, technology agnostic definitions and we see no good reason why the UK cannot do the same. 

In acknowledgement that autonomy exists on a spectrum and can be present in certain critical functions and not others, the Government should adopt operational definitions of 'fully' and 'partially' autonomous weapon systems.

4

Ensure meaningful human control at all stages 

Much of the concern about autonomous weapons is focused on systems in which the autonomy is enabled by AI technologies, with an AI system undertaking analysis on information obtained from sensors. However, it is essential to have human control over the deployment of the system both to ensure human moral agency and legal compliance. 

Autonomy exists on a spectrum (see figure below). The required level of human involvement, whether in oversight, verification, or control of actions made by the system, will vary depending upon:  

• the design of the system,  
• the mission objectives and  
• the operational context of where and how the AI system is being used. 

Autonomy spectrum framework 

The Ministry of Defence specifies that AI weapon systems which identify, select and attack targets must have “context-appropriate human involvement”.  

Determining whether there is a sufficient degree of human control and setting a minimum level of human involvement in a system involves considering many nuanced factors such as the complexity and transparency of the system, the training of the operators, and physical factors such as when, where and for how long a system is deployed. Nonetheless, it is important that the Ministry of Defence undertakes this task from the design of a system to its deployment. 

We note the Ministry of Defence's definition of "context appropriate" and "human involvement". The Government must ensure that human control is consistently embedded at all stages of a systems lifecycle, from design to deployment. This is particularly important for the selection and attacking of targets.