Computers can count seals from aerial photographs with lightning speed and reliability. Based on their spatial patterns, the tiny dots on the aerial images can even be assigned to one of the two major species of seals in the Wadden Sea. That is shown in the thesis that marine biologist Jeroen Hoekendijk will defend on January 26 in Wageningen. "To better understand if and how marine mammals like seals are affected by climate change and the disappearance of sea ice, this help from artificial intelligence (AI) in observations is crucial," Hoekendijk said. Hoekendijk carried out his research at the Royal Netherlands Institute for Sea Research and Wageningen University & Research.

Training AI with Aerial Photo’s

Harbor and grey seals in the Wadden Sea have long been counted using aerial photographs taken annually from a small aircraft. As a result, a large amount of aerial imagery is available with verified numbers of seals of both species. "We showed stacks of those old photos to a computer program and asked the computer, for example: 'find the fifty grey seals we saw in this photo'. This allowed us to train the computer program in recognizing and counting seals," Hoekendijk said.

Previously, computers were trained to count any objects by first manually marking each individual object on the images, which is a time-consuming task.

Spatial Distribution

Grey and harbor seals also behave differently when lying on the sandbanks in the Wadden Sea. Harbor seals are clearly more dispersed than grey seals. "Based on that specific distribution pattern, we can now recognize and count species on images in which the resolution is too low to distinguish the species based on their external characteristics," Hoekendijk said.

Some species of pinnipeds haul out solitarily, such as ringed seals (a), while harbour seals (b) and grey seals (c)—haul out in colonies while preserving some distance from conspecifics and walruses (d)—may cluster. (Image credit: (a)(d) by Eelke Folmer (Aeria))

Wadden Sea field Lab for Big Arctic

The Wadden Sea has proven to be a perfect ‘field lab’ to develop these new methodologies, according to Hoekendijk. “We can now search for and count seals on a much larger scale in, for example, the Arctic. If you don't know exactly where to look, then even looking for 7 million harp seals in the entire Arctic is like looking for needles in a haystack."

Those counts of marine mammals like harp seals in the Arctic are essential, however, Hoekendijk argues. "With the disappearance of sea ice, it is expected that seals that rest, molt and give birth to their young on that ice, will have an increasingly hard time. To know the exact consequences, we will have to find and count the animals year by year," Hoekendijk knows. The Arctic Ocean is expected to be completely ice-free possibly by the summer of 2035.

Walruses in the Arctic. (Image credit: Jeroen Hoekendijk)

Satellite Imagery

Hoekendijk expects that satellite imagery will play an increasingly important role in this work. "The resolution of satellite images is getting higher and higher. One pixel on a satellite photo measures only 31 by 31 cm nowadays. That means you can already see an individual seal from space. Added to that, 'deep learning' makes computers better and better at recognizing details. I expect that they are becoming indispensable in researching the consequences of climate change for marine mammals."