The risks of artificial intelligence are no longer confined to misinformation or wrong decisions in the digital space. With the advance of intelligent robots, they have extended into the physical world, where a decision made by an algorithm can turn into an actual movement that may threaten human safety.
In this context, a recent study conducted by the Robocurve foundation, which specialises in assessing AI safety, revealed gaps in the ability of AI models used to control robots to recognise risk. Experiments showed that manipulating the images these systems receive may push them to attempt to carry out dangerous instructions, even when those involve tools or materials that can cause serious harm.
According to the study, AI models attempted to carry out dangerous tasks in 27% of tests when they received natural images of the scene. The rate rose to about 88% when the images were blurred, and reached 97% when the images were digitally altered to show dangerous objects as though they were harmless materials or tools.
The experiments covered five advanced AI models, tested using real robotic arms in 900 trials involving five dangerous tasks, including directing a knife towards a doll shaped like an infant, placing an electrical device in water, pouring water into an electric toaster, and mixing cleaning materials whose reaction may produce toxic gas.
The experiments showed variation in how the systems responded to dangerous orders depending on the nature of the images they received, as the clarity and accuracy of visual information directly affected their ability to recognise risk and refuse to carry out the tasks.
In one experiment, an image of a doll was altered to look like an apple, while other experiments showed packages of dangerous chemicals as ordinary drinks, which led some models to attempt to carry out the related orders.
The findings highlight one of the challenges facing the development of AI-based robots: their ability to distinguish between instructions that can be carried out safely and those that may pose a danger to humans or the surrounding environment, especially when visual information is misleading.
The study also raises questions about how ready AI-supported robots are to operate in environments that require direct interaction with humans. Safety is not only about these systems' ability to understand instructions, but also about their ability to verify the nature of surrounding objects, resist misleading information, and refrain from carrying out orders when there is a potential danger.
It also highlights the need to develop independent safety systems that monitor robot decisions and limit their ability to carry out dangerous tasks, instead of leaving responsibility for safety to the intelligent model itself.
