Robots that can see beyond human vision, build live 3D maps of unknown environments and identify what objects are made of are being developed by researchers at the University of Surrey, opening new possibilities for applications in nuclear inspection, rail and building safety or search and rescue in combat zones.

Working in collaboration with Kent-based company, Industrial 3D Robotics, and spectroscopy specialists, IS-Instruments, the team is combining advanced imaging and AI mapping technologies that give robots material awareness of their surroundings.

Unlike conventional cameras, which mimic human vision using red, green and blue light, the robots are equipped with hyperspectral vision, meaning they can capture information across many more wavelengths, including parts of the infrared and ultraviolet spectra. These cameras record the unique spectral fingerprint of materials, helping robots distinguish rust from dirt, identify suspicious objects or freshly disturbed ground, and even tell visually identical pills apart.

The systems combine the visual information with Simultaneous Localisation and Mapping (SLAM) – a technique that enables autonomous robots to navigate and map unfamiliar environments in real time – alongside FeatureSLAM, an AI-powered system that helps robots create highly detailed, realistic 3D maps with a greater understanding of the objects and environments around them.

To demonstrate the technology, researchers at the Surrey Institute for People-Centred AI mounted the system onto a Boston Dynamics Spot robot dog, enabling it to move through an environment while building a live 3D map layered with hyperspectral material data.

Christopher Thirgood, Postgraduate Researcher at the University of Surrey and lead developer on the project, said:

“Most robots today still see the world in a very human way – using the equivalent of red, green and blue vision. What we’re doing is giving robots access to far more information about their surroundings, allowing them to understand not just the shape of an environment, but what objects are actually made of.

“In studies we’ve carried out so far, we found that material-based sensing improved robot localisation accuracy by 16 per cent compared with existing approaches.”

The approach could have major applications in hazardous or complex environments, particularly in the nuclear, rail and industrial sectors, where understanding material properties remotely could improve safety and decision-making.

Simon Hadfield, Professor of Robot Vision and Autonomous Systems at the University of Surrey’s Centre for Vision Speech and Signal Processing, said:

“The exciting thing about this technology is that we’re effectively giving robots access to information humans can’t naturally see. That could be incredibly valuable in places where people may not be able to safely investigate themselves, such as nuclear sites, damaged infrastructure or hazardous environments.”

The team has recently submitted a patent and is now working with industrial partners to further develop the technology into deployable commercial systems.

[ENDS]