Early warning system for environmental changes: New animal tags with cameras and AI complete maiden flight

How can a vulture in a Berlin Zoo help its conspecifics and their habitats in Namibia? Perhaps it can serve as a model and patron for a new generation of animal tags.

The prototype of an innovative animal tag system developed by the Leibniz Institute for Zoo and Wildlife Research (Leibniz-IZW) and the Fraunhofer Institute for Integrated Circuits (Fraunhofer IIS) completed its maiden flight on a vulture at Tierpark Berlin today. The tags will be equipped with sensor-based artificial intelligence (AI), a camera, energy-efficient electronics and satellite-based communication technology. This will allow for entirely new insights into the world of animals and their habitats. The tags detect and transmit animal behavior in real time and are thus an early warning system for ecological changes.

Climate change, species extinction, pandemics—owing to human influence, our planet is changing faster than we can understand and protect it. To keep up with the accelerating pace of environmental change, environmental research and conservation must break new ground and embrace the potential of the latest technological developments. Leibniz-IZW and Fraunhofer IIS have therefore been working together since the beginning of 2022 in two major research and development projects. In the GAIA-Sat-IoT (Guardian of the wild using Artifical Intelligence Applications and Satellite-based IoT Networks) and SyNaKI (Synergy of natural and artificial intelligence in the swarm) projects, they have developed a new generation of animal tag that allows significantly faster and more accurate insights into ecosystems.

To this end, the tags will sport significant innovations: Firstly, they are divided into two parts, one of which is attached to the vultures' backs and records position and acceleration data, among others. The second module features a camera and therefore sits on the birds' chests at the front. Both parts are connected and work hand in hand. This is made possible by a second innovation: on-board AI directly on the tag recognizes certain behavioral patterns of the animals in the data, classifies their behavior and thereby generates valuable information directly on the tag. The camera then takes photos at crucial moments, which in turn are evaluated by another AI system. Last but not least, a new satellite-based communication link will ensure that information can be transmitted directly from the tag to a satellite and to the scientists even in the most remote ecosystems.

"We are following this route because it allows us to see much more precisely and quickly what is going on in an ecosystem," says Dr. Jörg Melzheimer, GAIA project head at Leibniz-IZW. "We are developing our tags exemplarily for use on vultures in southern Africa. These scavengers have amazing sensory abilities and intelligence, each bird on its own and together in a wake. They find carrion with tremendous precision and speed, making them ideal allies for us."

On the one hand, the occurrence of carrion is a completely natural circumstance, yet critical changes in the ecosystem can also be detected from irregularities in the carrion occurrence, for example the outbreak of wildlife diseases. The tags make it possible to detect such local ecological hotspots almost in real time. "We designed the GAIA projects as a network of animal, human and artificial intelligence. We use high-tech and the evolutionary intelligence of animals to identify and solve pressing environmental problems," says Melzheimer.

The design and development of the GAIA animal tag system is being carried out in close cooperation between specialists in wildlife biology and ecology, artificial intelligence, communication technology and sensor technology at Leibniz-IZW and Fraunhofer IIS. "The processing of the data by an AI directly on the tag is a key feature, as it can be used to decide in real time which of the collected data are relevant enough to send to the satellite," says Dipl.-Inf. Nina Holzer, Group Leader Multimodal Human Sensing and project head of GAIA-Sat-IoT at Fraunhofer IIS. "Sending all the collected data in real time is virtually impossible, a satellite connection doesn't allow for that and would consume too much power."

In addition, the split tag design is a novelty with its very own technical requirements, reports Dipl.-Ing. Jürgen Ernst, Chief Engineer at Fraunhofer IIS and lead engineer of GAIA-Sat-IoT. "The camera module sits on the front of the vulture's chest. All other sensors as well as the AI processors and the solar power supply are housed in the tag on the back. Both electronic components have to work together smoothly and the whole systems also needs be perfectly adapted to the anatomy of the animals so that it can be brought into use in accordance with animal welfare aspects."

The Tierpark Berlin—one of two zoos in the German capital—is an indispensable cooperation partner of the consortium. In an early phase of the project, GAIA scientists and zoo keepers tagged two white-backed vultures in the Tierpark's aviary with commercially available tags, which record acceleration data (ACC) and GPS positions. ACC data provide a very precise insight into the movements of the tags and the animals. At the same time, the scientists recorded footage of typical vulture behavior with a video camera. The combination of ACC data and video recordings made it possible to train the artificial intelligence.

"In the videos we can identify exactly to the second when the bird shows which typical behavior and can thus identify corresponding patterns in the ACC data," explains Wanja Rast, AI specialist at Leibniz-IZW. "This training data is the basis of the AI, which in the future will perform automated behavior classification directly on the tag in a very energy-efficient and accurate way." The AI is not yet implemented on the current prototype and is currently still being developed desktop-based with downloaded data, but it will become a unique selling point of the new animal tags over the course of the projects.

"In the SyNaKI project, we will also map natural swarm intelligence virtually in a swarm of microprocessors," adds SyNaKI project head Felix Kreyß from Fraunhofer IIS. "By doing so, we enable distributed and AI-based data analysis directly in the field, in a wake of vultures. To achieve this, we combine natural with artificial intelligence on the animal tags as well as with approaches from extreme edge computing."

Another major challenge in the development of the tags is the efficient and reliable transmission of data from the animal to the user, for example the scientists or conservationists.

"Since the tag is to be used early in the project to collect training data, the transmission function is currently (still) limited. In the next step, we will develop the transmitter further and integrate the AI signal processing electronics," says Florian Leschka, Group Leader System Design at Fraunhofer IIS. "Another essential work step is the integration of a powerful satellite IoT radio module into the transmitter to guarantee the subsequent transmission of the extracted information."

Especially in areas beyond the reach of terrestrial communication infrastructures, the support of satellite-based networks is required. To enable direct transmission from the transmitter node to the satellite, the Fraunhofer experts are developing a communication system based on a terrestrial mioty technology. As a result of the developments by Leibniz-IZW and Fraunhofer IIS, a small animal tag is being created which intelligently pre-processes and combines data and sends only relevant environmental information via a satellite-based communication link.