Shanghai researchers develop breakthrough AI-powered vision system

A research team from the University of Shanghai for Science and Technology published a paper in Science Advances on Friday, presenting a groundbreaking AI-assisted biomimetic vision system.

The system features a biomimetic compound eye camera with an effective resolution of 4.3 megapixels. It can capture full-color panoramic images with a viewing angle of 165 degrees and resolving power of 40 micrometers.

The system was inspired by the structure and information processing methods of arthropod compound eyes. It consists of many small units called ommatidia that work together to quickly respond to complex environments. These units can detect light from different directions, offering a wide field of view and allowing the eyes to perceive depth.

Previous efforts to mimic these compound eyes have produced imaging devices using microlens arrays and sensors. However, traditional designs often generated low-quality, mosaic-like images due to their limited ability to handle complex three-dimensional structures and spatial details.

By combining the optical structure of arthropod eyes with deep learning algorithms, the USST team's system not only achieves megapixel-level imaging, but also enables accurate recognition, precise 3D localization, and real-time tracking of specific targets in confined spaces.

The research was guided of Zhuang Songlin, a member of the Chinese Academy of Engineering, and carried out by Professor Zhang Dawei, Professor Dai Bo, and PhD student Long Yan, in collaboration with Professor Tony Jun Huang's team from Duke University.

According to the team, their goal was not only to enable the biomimetic vision system to "see" the world, but also to let it "see clearly" and "understand" the vast and complex world around us.

Their findings are detailed in the paper titled "Seeing through arthropod eyes: An AI-assisted, biomimetic approach for high-resolution, multi-task imaging."

The team plans to further optimize the compound eye design and explore its applications in equipment such as micro unmanned platforms and endoscopic inspection devices. They hope this will lead to smarter, AI-powered scientific instruments in the future.

The achievement is yet another example of Shanghai's scientific strength. The city accounts for 29.2 percent of China's publications in top academic journals such as Science, Nature, and Cell in the past five years, according to the city government's science and technology commission.