Animal energy usage made visible through video

(Press-News.org) Energy scarcity is a central driver of animal behavior and evolution. The amazing diversity of life on this planet is a testament to the plethora of novel biological solutions to the problem of securing and maintaining energy. However, despite being so central to biology, it remains difficult to quantify, and thereby empirically analyze, energy consumption.

While organisms use energy for a very wide variety of processes – from growth to cognition – one activity is a major drain for many animals: movement. For highly mobile animals, movement is as such a powerful lens through which to estimate energy usage.

Strong methods do exist for measuring animal movement in the context of energy expenditure, but these are limited by the physical size of the equipment used. Now, in a paper published in the Journal of Experimental Biology, researchers from the Marine Biophysics Unit at the Okinawa Institute of Science and Technology (OIST), in collaboration with Professor Amatzia Genin from the Hebrew University of Jerusalem, describe an innovative method for measuring energy usage during movement with video and 3D-tracking via deep learning. “The best method without space limitations has until now been ruled out in the study of about half of the world’s species, due to the reliance on wearable equipment,” says Dr. Kota Ishikawa, first author of the study. “With video, we now have a more inclusive method for studying energy usage in the context of animal behavior and ecology.”

Dynamic Body Acceleration (DBA) has long been the state-of-the-art method for estimating energy usage during movement. In essence, DBA involves measuring the oxygen consumption of a given species performing a given behavior in the lab, by simply measuring the amount of oxygen consumed through the activity. Oxygen is a good indicator of energy, as it is consumed as part of aerobic respiration to produce ATP – the ‘fuel’ providing energy for most bodily processes, including muscle contraction. The acceleration of the animal is measured simultaneously with an accelerometer, and in most cases, because the correlation between acceleration and oxygen consumption during the behavior is very strong, DBA provides a reliable estimate of energy consumption.

With the standard set in the lab, DBA is then measured in the wild, where reliably measuring oxygen consumption is impossible, through a wearable accelerometer. However, relying on physical equipment presents a major barrier. “To ensure accurate measurements without influencing the behavior during observation, researchers have used equipment that weighs at least ten times less than the animal. Given that the accelerometer and battery pack weigh 10-20 grams, this rules out the study of any animals below 100 grams – about half the world’s vertebrate species. It can also affect movement, especially when it depends on drag efficiency, such as swimming or flight,” says Dr. Ishikawa.

Their solution to this problem is elegantly simple. Instead of using a physical accelerometer to measure the movements, two cameras capture video footage of the behavior – in this case, a damselfish swimming in a fishtank – from multiple angles to reconstruct the behavior in 3D space.  A few frames of the videos are then used to train a deep learning neural network to track the position of body features such as eyes, which allows researchers to subsequently measure the movement-related acceleration.

Both in the field and in the lab, once cameras are set up to capture animal movements, energy consumption can be estimated. For example, video-based DBA can be used in the context of collective behavior: “Energy expenditure during schooling of small fish has long remained mysterious,” explains Dr. Ishikawa. “For example, do the leading fish use more energy, and is schooling an energy-efficient form of movement? And what can that tell us about the ecology and evolution of fish schooling?”

With video-based DBA, the accurate measurement of energy usage during free-ranging animal behavior has been opened to the smaller half of the world’s vertebrate species, potentially enabling many new research avenues into the breadth of life on our planet.

END