NEMO combines information from our Awarion™ Autonomous Lookout system with deep learning YOLO neural networks and probabilistic data fusion methods to accurately detect and classify marine mammals in both visual and acoustic feeds.
Charles River Analytics was awarded a $1.5 million follow-on contract from the Defense Advanced Research Projects Agency (DARPA) to develop a multi-sensor fusion module that incorporates deep learning to autonomously and accurately detect whales and other aquatic mammals.
To protect marine mammals from collisions and sonar exposure, sailors must keep a vigilant lookout. Operators of remotely operated vehicles must also stay alert, and uncrewed surface vehicles’ (USVs) currently limited and narrow video feeds create risk, too. To reduce the strain on human lookouts, and give USVs the ability to steer clear, NEMO (Nautical Evaluation of Mammal Observation) will improve visual detection and classification onboard crewed and uncrewed vessels by developing new animal and ship detectors.
Charles River is working with marine mammal experts at the New England Aquarium to understand whale behavior patterns. These experts shared some indicators of the presence of whales, including circling birds, foaming patterns, and blows. This knowledge was incorporated into the multi-sensor module, to improve detection of whales which are near the surface but not visible. Collaborators at Raytheon contributed physics-based acoustic simulations to help model the effects of range, background noise, and sea state on acoustic detection reliability.
“At Charles River Analytics, we work on a lot of interesting tech to support autonomy,” said Ross Eaton, Director of Marine Systems in Charles River’s Sensing, Processing, and Applied Robotics Division and Principal Investigator on the NEMO effort. “NEMO is especially cool because we’re also working to protect some of the most vulnerable animal populations on the planet. Being able to work on interesting science that will have a positive impact on the natural environment is a win-win for our team.”
NEMO will reduce fatigue for human lookouts and improve safety on any vessel on which it is installed, and will serve as a key building block for future marine autonomy development.
Contact us to learn more about NEMO and our other object detection and sensor processing capabilities.
This material is based upon work supported by the Defense Advanced Research Projects Agency (DARPA) and the Army Contracting Command - Aberdeen Proving Grounds (ACC-APG) under Contract Number W911NF-18-C-0038. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the Defense Advanced Research Projects Agency (DARPA) and the Army Contracting Command - Aberdeen Proving Grounds (ACC-APG).
