Center of Expertise in Marine Mammalogy

Scientific Research Report
2015-2017

Table of Contents

• Complete Text
• Introduction
• Using aerial infrared images to count ringed seals on ice
• The use of unmanned aerial vehicles (UAV)
• 2017: A Marine Mammal Odyssey, Eh!
• Mark-recapture analysis from long-term study on Sable Island identifies changes in demographic rates in northwest Atlantic Grey Seals
• OTN – Using grey seals (Halichoerus grypus) as bioprobes to estimate phytoplankton biomass
• Northwest Atlantic International Sightings Survey (NAISS) of Marine Megafauna on the Continental Shelf From Northern Labrador to the Bay of Fundy
• Monitoring Movements of Whelping Seals on Drifting Pack Ice
• Marine Mammal Genomics Research in the Central and Arctic Region
• OTN and predator-prey interactions
• Listening in on the Deep: Passive Acoustic Monitoring of Whales off Nova Scotia
• Sharing Meals Keeps Killer Whale Families Together: Provisioning relatives maintains long-term social bonds and helps pass on shared genes
• New developments in the use of fatty acids to determine marine mammal diets
• More than a mouthful – unlocking bowhead whale foraging and reproductive histories from baleen
• Observing walrus behaviour at haulout sites in quasi real-time
• Moving towards automated counting
• References

Moving towards automated counting

Garry Stenson, Mike Hammill

Obtaining an estimate of animal abundance is needed to identify conservation objectives, for setting harvest limits and evaluating the impact of development. To do this, we often carry out visual or photographic surveys, to determine the number of animals in an area. During visual surveys, trained observers detect and tabulate animals as they pass by along a predetermined transect in a boat or aircraft. The advantages of such surveys are that observations can be made quickly, the data entered and analyzed to produce an estimate within a period of days to only a few weeks. However, visual observations are transient; the observer quickly sees the animal, identifies species, number and other data which are recorded, but there is no way to return later to verify if the data are correct.

The preferred approach is to use a sensor, such as a camera, to photograph the area. However, each photograph must be examined which can be difficult and time consuming. Detection of animals in a photo depends on the quality of the image and the ability of the reader to identify the animal on the photograph (Figure 16). This ability to detect an animal will vary between readers, but because there is a physical image that can be re-checked, protocols can be developed and quality of identifications evaluated. Over the last three decades we have seen tremendous improvements in image quality as platforms have incorporated motion-compensation mechanisms, and moved from film to digital systems. With current digital systems the resolution is so great that readers can hone in on a single image, magnify it and determine if it is an animal or some other object. This has reduced the amount of correction that has been applied to counts to compensate for missed animals. Incorporating the imagery into Geographic Information Systems (GIS) has also helped to accelerate counting since each animal is georeferenced on the screen and entered automatically into a database, thereby saving time; any questionable observation can be checked right away, and time-wasting physical entry of data is avoided. Geo-located data can also be used for additional studies such as identifying the types of ice used for pupping. However, many surveys, particularly those of marine mammals, cover large areas, so there can be an enormous number of photographs to read. In our harp seal surveys we generate 25,000 to 35,000 images each year that a survey is flown. Currently, this requires three people, working full-time, a year to read all of the imagery.

An automated detection and counting system would speed up the processing of photographs and reduce both the amount of time and energy needed to count animals on images. Such an approach could be semi- or fully automated. In a fully automated system, the number of animals present on an image is detected and tabulated by the system. This reduces the amount of effort needed and speeds up the final results, but is very difficult to actually develop. A semi-automated system, where potential animals are highlighted and a trained observer decides if it is a marine mammal or not, is more feasible. Over the years, various attempts have been made to create a semi-automated system to count seal pups but it is only in the last decade with significant improvements in computing power, and major advances in facial recognition systems, that some progress has been made. These features, combined with an approach referred to as deep learning have opened up possibilities to developing new automated detection systems. Deep learning is where the computer, exposed to hundreds of images of marine mammals (e.g. seals on the ice), slowly learns to distinguish what a seal on the ice looks like. This differs from earlier, more traditional approaches in which scientists attempted to define the shape of a seal using very complex algorithms.

DFO scientists are working with researchers from the Norwegian Institute of Marine Research and the Norwegian Computing Center to develop a program that automatically identifies potential harp and hooded seal pups in aerial images using a Deep Convolutional Neural Network (CNN). While still in the developmental stage, early results suggest that this is a useful approach. The method was developed using images from older Canadian and Norwegian surveys and is now being tested against images obtained during the harp seal survey DFO carried out in March of 2017. It is hoped that the program will be operational in time for the next survey.

The greatest challenge in identifying marine mammals, and in particular seals that give birth to white-coated pups like harp and grey seals, on images obtained using normal sensors is the lack of contrast between the seals and their background. In the past, ultra-violet cameras have been used to improve the contrast of harp seals but such cameras were expensive and hard to obtain. However, over the last few years, sensors that operate using other wavelengths, have become readily available. Thermal sensors which can measure the difference between the warm temperatures emitted from an animal and its colder surroundings are one example of a sensor that offers potential. Working with researchers from the Duke University Marine Laboratory, we recently deployed a thermal sensor in a small Unmanned Aerial Vehicle (UAV) to collect imagery of grey seal pups born on small islands during January in Atlantic Canada.

A seal detection model was developed to scan thermal imagery, detect seals, and count them. Counts obtained using the automated counter were compared to manual counts obtained from two island breeding colonies in Atlantic Canada. The tool used temperature thresholds and pixel cluster size sorting to detect grey seal adults and pups. The images were first examined manually to provide a baseline count. They were then examined using the automated thermal model. At one island, the automated counter detected 5% fewer seals than were detected manually. At the second island, the automated counter only detected 2% fewer seals than were counted manually (Figure 17).

In our analyses, the automated method performed better than manual counts at the prediction site (site two) where ambient landscape temperatures were lower, allowing for better contrast between seals and the environment. However, the model failed to detect young animals that were not warm enough. These ‘cold’ animals were likely dead animals that did not have a thermal signature but could be seen by the human readers. This automated system is relatively easy to set up and apply using commonly used GIS software. In areas where multiple sensors are used, it offers a clear way forward to improve counting and tabulation of imagery obtained from our surveys.