AI initiative registry

Every initiative listed in the registry can be consulted here, without an account. Each record shows its progress, its milestones and the history of its decisions. Requests still under triage do not appear: they join the registry once reviewed.

Perception and Understanding

AI that interprets the world, whether through text, images, or sound.

What is “Perception and Understanding”?

This category includes AI systems that sense, interpret, or perceive data from the environment, emulating human senses like sight, sound, touch, smell, and language comprehension. These capabilities form the foundation of AI awareness, enabling systems to extract meaning from unstructured inputs like text, images, video, audio, or environmental signals.

Perception and Understanding
Subcategory	Description	Examples
Language understanding — Natural Language Processing (NLP)	Understanding and generating human language	Chatbots, document summarization
Computer vision	Understanding images, video, and spatial data	Object detection, satellite imagery analysis, species ID
Speech recognition	Converting audio to text	Voice-to-text for transcription, voice-based commands for virtual assistants
Multimodal AI	Combining input types (text + images, etc.)	Image captioning, document understanding tools
Olfactory AI (artificial smell recognition)	Detecting and interpreting smells using e-noses	Gas leak detection, spoilage sensing, pollutant ID in fish labs
Haptic AI (artificial touch sensing)	Interpreting touch and tactile feedback	Robotic sampling, pressure detection, texture, temperature
Audio event detection	Recognizing non-speech sounds	Marine mammal calls (whale song), mechanical alerts, engine noise anomaly detection
Sensor fusion AI	Combining multiple sensory inputs	Autonomous underwater vehicle navigation, habitat monitoring

Key indicator: If the AI’s primary task is to interpret raw input (e.g. image, text, sound, sensor data), this is where it belongs.
Common input types: Visual data (images, video)

Audio and speech

Natural language text

Sensor readings (e.g. haptics, chemical, temperature)

Filter 1

Search titles and descriptions

Sector

Program

Stage

Region

Strategic alignment

AI category

AI sub-category

AI capability

Prime Minister's mandate commitment

Scope

Declared to TBS On hold

Reset

71 current initiative(s)

AI initiative registry
ID	AI Initiative	Stage	Description	Sector	Region	AI Bucket	SME
147	Optical Character Recognition Data transcription tool	Pilot	This completed project built a proof of concept tool to facilitate data transcription and validation from hand-written field notes. tested...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
145	Automatic detection of unidentified fish sounds	Pilot	This completed project used Random Forests and Convolutional Neural Networks (CNN) algorithms trained on manually detected fish sounds to de...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
132	Salmon Scale Aging	Pilot	A model using computer vision to look at images of salmon scale and predict the age of salmon and also explain how it arrived at its predict...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
130	Salmon Data Digitization	Pilot	Digitizing non-machine-readable (handwritten) documents and extracts characters into a digital format so that it is accessible, editable, an...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
126	Automated assessment of satellite imagery	Pilot	AI routines are being developed and tested to assess large volume of satellite imagery and data to: i) Automatically Identify marine hazards...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
124	Development of an underwater videography and AI system for studying fish passage effectiveness	Pilot	Designed to train existing DFO tools with information on fish passage effectiveness and develop an SOP for training videography-based AI mod...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
122	Development and testing of artificial intelligence and machine learning to analyze underwater photo and video data to assess marine impacts and resources	Pilot	This completed project (2020/21) aimed to utilize machine learning and imaging technology in fisheries science. The proponent planned to use...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
119	Computer vision to automate fish data extraction from video	Pilot	Our purpose has been to build a tool to broadly apply computer vision methods in marine research. Our work has generated several publication...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
118	Use of artificial intelligence models to facilitate seafloor video annotations in conservation areas.	Pilot	The purpose of this initiative is to utilize AI to facilitate seafloor video annotations collected as part of the MCT and Benthic Ecology pr...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
106	Finwave: an online photo-identification database and AI matching tool	Pilot	This project aims to improve a publicly accessible machine learning platform capable of identifying individual killer whales in real-time...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
105	Combining artificial intelligence and drone imagery for automatic extraction of morphometric attributes of small cetaceans: Application to the St. Lawrence Estuary Beluga population.	Pilot	This project proposes to use a multiyear time series of drone images of the St. Lawrence Estuary Beluga population to develop a series of al...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
104	Zooplankton classification from Video Plankton Recorder	Pilot	Using machine learning algorithms to identify plankton for rapid zooplankton classification from the Video Plankton Recorder, in support of...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
103	Improved understanding of the processes impacting harmful algal blooms	Pilot	Using machine learning algorithms to analyze phytoplankton images from the Imaging Flow Cytobot (IFCB) to recognize and quantify species ass...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
100	Development of AI technology for otolith and scale ageing	Pilot	Ageing of scales and otoliths is time-consuming, relies on skilled personnel, and often faces delays. AI tools like DFOdots offer a solution...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
97	Satellite image analysis for mapping eelgrass habitat in the southern Gulf of St. Lawrence.	Pilot	This project supports Gulf Region needs for spatial data representing eelgrass habitat through a service level agreement with Marine Plannin...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
96	Use of machine-learning (ML) algorithms and high-performance computing (HPC) for the optimization of satellite image analysis for mapping coastal habitats	Pilot	This project aims to improve satellite image analysis of marine habitats leveraging NRC HPC and ML assets to apply cutting-edge ML technique...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
95	Optimization of Machine Learning modelling strategy for coastal zooplankton taxonomic identification from flow microscopy imaging.	Pilot	As part of Aquaculture Monitoring Program we are using Ecotaxa's ML algorithm to identify zooplankton. The project aims at identifying the b...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
94	DFO Bot, deep learning application to analyse otolith images available in DFO Dots	Pilot	Development of a neural network trained using otolith images, expert-derived age estimates, otolith annotations and annuli to obtain age est...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
93	Détection de homards et de crabes communs dans les images sous-marines par intelligence artificielle. Not translated · original: French	Pilot	Développer un processus de capture et d’analyse d’images sous-marine par Intelligence Artificielle pour la détection de homards et de crabes...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
92	Identification automatisée de la condition de carapace de crabe des neiges Not translated · original: French	Pilot	Détermination de la condition de carapace de crabe des neiges à partir d’images. La méthode offre une façon de valider/améliorer des données...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
91	Localization and Tracking of Beluga Whales in Aerial video Using Deep Learning	Pilot	Deep-learning model was developed us YOLOv7 to detect beluga whales from imagery footage. The main contribution of this research is providin...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
90	Deep Learning Enhances Beluga Whale Research in the Arctic	Pilot	Esri Canada built a deep learning model that automatically detects beluga whales in imagery. The model was developed using satellite, drone...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
89	Augmenting Whale Detection in Satellite Images using synthetic data	Pilot	An autodetector model was built to detect whales in satellite imagery using synthetic imagery. The primary author is now testing the algorit...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
88	Development of an AI matching model for bowhead whale photo-identification	Pilot	Using DFO’s existing database of bowhead whale photographs, the specific objectives are: 1) Create a machine learning model “detector” for b...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
87	ANMPA Habitat Mapping Project	Pilot	Drop camera survey to document seafloor habitats and biological communities in the Anguniaqvia niqiqyuam Marine Protected Area (ANMPA) in th...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda

147
Optical Character Recognition Data transcription tool
This completed project built a proof of concept tool to facilitate data transcription and validation from hand-written field notes. tested on ~100 year old salmon observations and present-day salmon stream inspection logs to determine the cost/feasibility/effectiveness of implementing such a tool in Salmon monitoring contexts.

Pilot Ecosystems and Oceans Science Perception and Understanding
145
Automatic detection of unidentified fish sounds
This completed project used Random Forests and Convolutional Neural Networks (CNN) algorithms trained on manually detected fish sounds to detect fish sounds collected on a passive acoustic monitoring project. The result is an easy-to-use, open-source software called FishSoundFinder, implemented with the CNN detector.

Pilot Ecosystems and Oceans Science Perception and Understanding
132
Salmon Scale Aging
A model using computer vision to look at images of salmon scale and predict the age of salmon and also explain how it arrived at its prediction.

Pilot Ecosystems and Oceans Science Perception and Understanding
130
Salmon Data Digitization
Digitizing non-machine-readable (handwritten) documents and extracts characters into a digital format so that it is accessible, editable, and useable for analysis.

Pilot Ecosystems and Oceans Science Perception and Understanding
126
Automated assessment of satellite imagery
AI routines are being developed and tested to assess large volume of satellite imagery and data to: i) Automatically Identify marine hazards and ii) Change detection – i.e. shorelines or riverbank changes.

Pilot Ecosystems and Oceans Science Perception and Understanding
124
Development of an underwater videography and AI system for studying fish passage effectiveness
Designed to train existing DFO tools with information on fish passage effectiveness and develop an SOP for training videography-based AI models in support of fish passage projects.

Pilot Ecosystems and Oceans Science Perception and Understanding
122
Development and testing of artificial intelligence and machine learning to analyze underwater photo and video data to assess marine impacts and resources
This completed project (2020/21) aimed to utilize machine learning and imaging technology in fisheries science. The proponent planned to use Video and Image Analytics for Marine Environments (VIAME), an open-source system developed by NOAA, for two specific purposes: 1. Measure the movement of Atlantic cod across boundaries at the Gilbert Bay Marine Protected Area in Newfoundland and Labrador. 2. Assess the impact of seismic surveying on Atlantic cod as part of a project funded by the Environmental Studies Research Fund (ESRF). The automatic detection and count of fish near MPA boundaries was expected to reduce use of personnel time and effort.

Pilot Ecosystems and Oceans Science Perception and Understanding
119
Computer vision to automate fish data extraction from video
Our purpose has been to build a tool to broadly apply computer vision methods in marine research. Our work has generated several publications illustrating how we built, tested/validated, and applied computer vision methodology to collect / extract / and analyze data pertaining to marine fish in the Newfoundland and Labrador region, from video. For example, we have used it to examine the impact of seismic surveying on commercial fish in the Newfoundland and Labrador offshore. Our continuing purpose is now broadly sharing our system with commercial harvesters (fishermen), non-profit organization (AHOI), government researchers (aquaculture, MPAs, AIS), and academics (computing scientists), to collect and rapidly analyzing video data for a wide range of applications.

Pilot Ecosystems and Oceans Science Perception and Understanding
118
Use of artificial intelligence models to facilitate seafloor video annotations in conservation areas.
The purpose of this initiative is to utilize AI to facilitate seafloor video annotations collected as part of the MCT and Benthic Ecology programs in the NL Region. A large number of vidEcosystems and Oceans Science is collected annually, and manual video annotation (e.g., locating and counting observations) is extremely time-consuming. A published “object detection model” (FathomNet Megalodon Detector, YOLOv8x) trained using marine taxa by MBARI is currently being used to identify objects in our seafloor images. The model does not identify the objects (e.g., to species), but it largely accelerates the process given that locating objects is the most time-consuming part of the work.

Pilot Ecosystems and Oceans Science Perception and Understanding
106
Finwave: an online photo-identification database and AI matching tool
This project aims to improve a publicly accessible machine learning platform capable of identifying individual killer whales in real-time from submitted photographs. The ultimate goal is to improve the web-based interface and validate the algorithm used for photo identification against known individuals.

Pilot Ecosystems and Oceans Science Perception and Understanding
105
Combining artificial intelligence and drone imagery for automatic extraction of morphometric attributes of small cetaceans: Application to the St. Lawrence Estuary Beluga population.
This project proposes to use a multiyear time series of drone images of the St. Lawrence Estuary Beluga population to develop a series of algorithms for automatic extraction of morphometric information of individual beluga whales using AI.

Pilot Ecosystems and Oceans Science Perception and Understanding
104
Zooplankton classification from Video Plankton Recorder
Using machine learning algorithms to identify plankton for rapid zooplankton classification from the Video Plankton Recorder, in support of efforts to assess North Atlantic Right Whale foraging habitat.

Pilot Ecosystems and Oceans Science Perception and Understanding
103
Improved understanding of the processes impacting harmful algal blooms
Using machine learning algorithms to analyze phytoplankton images from the Imaging Flow Cytobot (IFCB) to recognize and quantify species associated with harmful algal blooms.

Pilot Ecosystems and Oceans Science Perception and Understanding
100
Development of AI technology for otolith and scale ageing
Ageing of scales and otoliths is time-consuming, relies on skilled personnel, and often faces delays. AI tools like DFOdots offer a solution by enabling automated ageing using annotated reference collections, helping reduce backlogs and support training. Developing these AI models can enhance resilience, efficiency, and training capacity in ageing programs.

Pilot Ecosystems and Oceans Science Perception and Understanding
97
Satellite image analysis for mapping eelgrass habitat in the southern Gulf of St. Lawrence.
This project supports Gulf Region needs for spatial data representing eelgrass habitat through a service level agreement with Marine Planning and Conservation. Machine learning algorithms have been applied for classification of satellite imagery. MPC and Science have been collaborating for several years, annually purchasing satellite imagery, with classification conducted by both groups.

Pilot Ecosystems and Oceans Science Perception and Understanding
96
Use of machine-learning (ML) algorithms and high-performance computing (HPC) for the optimization of satellite image analysis for mapping coastal habitats
This project aims to improve satellite image analysis of marine habitats leveraging NRC HPC and ML assets to apply cutting-edge ML techniques to the challenges of remote sensing data analysis. Relates to ongoing project (below) through MPC service-level agreement.

Pilot Ecosystems and Oceans Science Perception and Understanding
95
Optimization of Machine Learning modelling strategy for coastal zooplankton taxonomic identification from flow microscopy imaging.
As part of Aquaculture Monitoring Program we are using Ecotaxa's ML algorithm to identify zooplankton. The project aims at identifying the best strategy in building model training sets for each region monitored. This strategy will be the basis for future AMP zooplankton identification work.

Pilot Ecosystems and Oceans Science Perception and Understanding
94
DFO Bot, deep learning application to analyse otolith images available in DFO Dots
Development of a neural network trained using otolith images, expert-derived age estimates, otolith annotations and annuli to obtain age estimates from otoliths.

Pilot Ecosystems and Oceans Science Perception and Understanding
93
Détection de homards et de crabes communs dans les images sous-marines par intelligence artificielle.
Développer un processus de capture et d’analyse d’images sous-marine par Intelligence Artificielle pour la détection de homards et de crabes communs.

Pilot Ecosystems and Oceans Science Perception and Understanding
92
Identification automatisée de la condition de carapace de crabe des neiges
Détermination de la condition de carapace de crabe des neiges à partir d’images. La méthode offre une façon de valider/améliorer des données de condition de carapace prises sur le terrain. Ces déterminations sont importantes pour quantifier le recrutement à la pêche durant le relevé scientifique ainsi qu’améliorer la qualité des données de crabe mou dans programme observateur-en-mer.

Pilot Ecosystems and Oceans Science Perception and Understanding
91
Localization and Tracking of Beluga Whales in Aerial video Using Deep Learning
Deep-learning model was developed us YOLOv7 to detect beluga whales from imagery footage. The main contribution of this research is providing a system that accurately detects and tracks features of beluga whales, both adults and calves, from aerial footage.

Pilot Ecosystems and Oceans Science Perception and Understanding
90
Deep Learning Enhances Beluga Whale Research in the Arctic
Esri Canada built a deep learning model that automatically detects beluga whales in imagery. The model was developed using satellite, drone and aerial imagery and reported accuracy is high, but the model has not been tested by DFO Science yet.

Pilot Ecosystems and Oceans Science Perception and Understanding
89
Augmenting Whale Detection in Satellite Images using synthetic data
An autodetector model was built to detect whales in satellite imagery using synthetic imagery. The primary author is now testing the algorithm on satellite imagery obtained by Arctic Region that has been manually read for whales.

Pilot Ecosystems and Oceans Science Perception and Understanding
88
Development of an AI matching model for bowhead whale photo-identification
Using DFO’s existing database of bowhead whale photographs, the specific objectives are: 1) Create a machine learning model “detector” for bowhead whales. 2) Create a machine learning individual re-ID model for bowhead whales to re-identify bowheads. 3) To make the detector and matching models accessible through an easy to use, freely available, web-based application, Flukebook.org.

Pilot Ecosystems and Oceans Science Perception and Understanding
87
ANMPA Habitat Mapping Project
Drop camera survey to document seafloor habitats and biological communities in the Anguniaqvia niqiqyuam Marine Protected Area (ANMPA) in the Inuvialuit Settlement Region. A project sub-objective is to develop AI capacity to detect and record bottom habitat classifications and benthic animals from video still-images, substantially reducing processing times.

Pilot Ecosystems and Oceans Science Perception and Understanding