In VR school, fish teach robots

(Press-News.org) Fish are masters of coordinated motion. Schools of fish have no leader, yet individuals manage to stay in formation, avoid collisions, and respond with liquid flexibility to changes in their environment. Reproducing this combination of robustness and flexibility has been a long-standing challenge for human engineered systems like robots. Now, using virtual reality for freely-moving fish, a research team based in Konstanz has taken an important step towards that goal.

“Our work illustrates that solutions evolved by nature over millennia can inspire robust and efficient control laws in engineered systems,” said first author Liang Li from the University of Konstanz. Co-author Máté Nagy from Eötvös University underscores this: “The discovery opens up exciting possibilities for future applications in robotics and autonomous vehicle design.”

Deciphering nature’s hidden algorithm
Using a virtual reality (VR) setup that mimics natural schooling, researchers placed individual juvenile zebrafish into networked arenas where each fish could freely interact with “holographic” virtual conspecifics. Each virtual fish was a projection of a real fish from another arena, meaning that fish could swim and interact together in the same virtual world. The fully immersive 3D environment lets researchers precisely manipulate visual stimuli and record how the fish respond. This high level of control allowed the scientists to isolate exactly which cues the fish were using to guide their interactions with other fish. In other words, they could reverse engineer the behaviour of schooling in zebrafish to understand how fish solve the complex problem of coordinating their motion.

The solution they discovered was a simple and robust law based only on the perceived position, not the speed, of their neighbors to regulate their following behaviour.

“We were surprised by how little information the fish need to effectively coordinate movements within a school,” says Iain Couzin, senior author on the study and Director of MPI-AB and Speaker at the Cluster of Excellence Collective Behaviour. “They use local rules that are cognitively minimal, but functionally excellent.”

To see just how realistic the control law was, the team tested it with real fish. They conducted a VR “Turing test”, based on the concept of testing whether people can tell if they are interacting with a real human or with artificial intelligence. In the aquatic Turing test, a real fish would swim with a virtual fish that switched between being real and being controlled by the algorithm they discovered. The real fish could not tell the difference. They behaved the same whether interacting with a real conspecific or the virtual follower governed by the algorithm.

From fish to machines
To test the broader utility of their discovery, the team embedded it in swarms of robotic cars, drones, and boats. The robots were tasked with following a moving target using either parameters from the zebrafish algorithm or from a state-of-the-art method used in autonomous vehicles called Model Predictive Controller (MPC). Across all tests, the natural control law that fish have evolved delivered performance that was nearly indistinguishable from MPC in terms of accuracy and energy consumption – but at a fraction of the complexity.

Oliver Deussen, a co-author on the study and Professor in computer science at the University of Konstanz and Speaker at the Cluster of Excellence Collective Behaviour: “This work highlights the reciprocal relationship between robotics and biology – using robotics to explore biological mechanisms, which in turn can inspire new and effective robotic control strategies.”

 

Key facts:

Embargoed: Not for release until Wednesday, 30 April 2025, 14:00 U.S. Eastern Time Original publication: Liang Li, Máté Nagy, Guy Amichay, Ruiheng Wu, Wei Wang, Oliver Deussen, Daniela Rus, and Iain D. Couzin, Reverse engineering the control law for schooling in zebrafish using virtual reality, published in Science Robotics, 30. April 2025 The research was led by scientists at the Cluster of Excellence Collective Behaviour, University of Konstanz, and the Max Planck Institute of Animal Behavior (MPI-AB), Germany, in collaboration with researchers at MIT in the United States and Eötvös University in Hungary. The research was funded, among others, by the German Research Foundation (DFG), the European Union’s Horizon 2020 Research and Innovation Programme, the Hungarian Academy of Sciences, and the Messmer Foundation Research Award from the Werner and Erika Messmer Foundation. The Centre for the Advanced Study of Collective Behaviour at the University of Konstanz is a global hotspot for the study of collective behaviour across a wide range of species and across scales of organization. It is a Cluster of Excellence within the framework of the German Excellence Strategy of the federal and state governments.
Note to editors:
Watch our video here:
https://youtu.be/CgZuYNvBHkY


You can download photos here:

1) https://www.uni-konstanz.de/fileadmin/pi/fileserver/2025_extra/virtual_reality_for_fish.jpg
Caption: “The Matrix” for fish: Researchers placed individual zebrafish into networked virtual reality arenas where each fish could freely interact with “holographic” virtual conspecifics.
Copyright: Christian Ziegler, Mate Nagy, and Liang Li

2) https://www.uni-konstanz.de/fileadmin/pi/fileserver/2025_extra/virtual_reality_for_fish_2.jpg
Caption: From fish to machines: The natural “control law” of fish was embedded in swarms of robotic cars, drones, and boats.
Copyright: Christian Ziegler, Liang Li

END