AI initiative registry

Every initiative listed in the registry can be consulted here, without an account. Each record shows its progress, its milestones and the history of its decisions. Requests still under triage do not appear: they join the registry once reviewed.

Perception and Understanding

AI that interprets the world, whether through text, images, or sound.

What is “Perception and Understanding”?

This category includes AI systems that sense, interpret, or perceive data from the environment, emulating human senses like sight, sound, touch, smell, and language comprehension. These capabilities form the foundation of AI awareness, enabling systems to extract meaning from unstructured inputs like text, images, video, audio, or environmental signals.

Perception and Understanding
Subcategory	Description	Examples
Language understanding — Natural Language Processing (NLP)	Understanding and generating human language	Chatbots, document summarization
Computer vision	Understanding images, video, and spatial data	Object detection, satellite imagery analysis, species ID
Speech recognition	Converting audio to text	Voice-to-text for transcription, voice-based commands for virtual assistants
Multimodal AI	Combining input types (text + images, etc.)	Image captioning, document understanding tools
Olfactory AI (artificial smell recognition)	Detecting and interpreting smells using e-noses	Gas leak detection, spoilage sensing, pollutant ID in fish labs
Haptic AI (artificial touch sensing)	Interpreting touch and tactile feedback	Robotic sampling, pressure detection, texture, temperature
Audio event detection	Recognizing non-speech sounds	Marine mammal calls (whale song), mechanical alerts, engine noise anomaly detection
Sensor fusion AI	Combining multiple sensory inputs	Autonomous underwater vehicle navigation, habitat monitoring

Key indicator: If the AI’s primary task is to interpret raw input (e.g. image, text, sound, sensor data), this is where it belongs.
Common input types: Visual data (images, video)

Audio and speech

Natural language text

Sensor readings (e.g. haptics, chemical, temperature)

Filter 1

Search titles and descriptions

Sector

Program

Stage

Region

Strategic alignment

AI category

AI sub-category

AI capability

Prime Minister's mandate commitment

Scope

Declared to TBS On hold

Reset

71 current initiative(s)

AI initiative registry
ID	AI Initiative	Stage	Description	Sector	Region	AI Bucket	SME
228	SILScanner	Pilot	Deploy on DFO's PBMM Cloud a proof of concept application that uses optical character recognition (OCR) to digitize and validate Stream Insp...	Chief Digital Officer	National Capital Region	Perception and Understanding	Bird, Tom
187	Atlantic Bluefin Tuna Monitoring	Pilot	Automatically perform video analytics from hundred of harvester tuna video Solution being developed in region to assist C&P in batch process...	Indigenous Affairs, Aquaculture and Governance	Gulf	Perception and Understanding	Dupuis, Marc
188	Coastal Surveillance	Pilot	Automatically process thousands of drone images as part of continuous surveillance Solution being developed in region to assist C&P in batch...	Indigenous Affairs, Aquaculture and Governance	Gulf	Perception and Understanding	Dupuis, Marc
218	L’observation des bélugas de l’estuaire du Saint-Laurent Not translated · original: French	Pilot	Entente de contribution avec le Groupe de recherche et d'éducation sur les mammifères marins (GREMM) pour le développement d'un module de me...	Marine Planning and Conservation	Quebec	Perception and Understanding	Robert, Sonia
217	Suite du projet de recherche de recherche et développement d’outils de vision automatisée pour l’étude des refuges marins des coraux et des éponges du golfe du Saint-Laurent. Not translated · original: French	Pilot	Une entente de contribution pour l’année 2025-2026 (Programme de gestion des océans) visant à l’amélioration de l’outil développé par le Cen...	Marine Planning and Conservation	Quebec	Perception and Understanding	Brossard, Jade
213	Numérisation de documents manuscrits Not translated · original: French	Pilot	En collaboration avec l’équipe nationale de l’Intendant principal des données (PSSI), visant principalement la pêche au saumon. L’objectif e...	Statistics, Policy and Licensing	Quebec	Perception and Understanding	Rémillard, Vincent
212	Surveillance électronique en mer Not translated · original: French	Pilot	Il s’agit de capture d’images vidéo sur les navires de pêche pour ensuite les faire analyser par des ressources humaines et un modèle d’inte...	Statistics, Policy and Licensing	Quebec	Perception and Understanding	Rémillard, Vincent
190	Lobster Stock Assessment	Pilot	Automatically assess lobster count and habitat from hours of video efficiently Solution being developed for Science to review hundreds of ho...	Indigenous Affairs, Aquaculture and Governance	Gulf	Perception and Understanding	Dupuis, Marc
105	Combining artificial intelligence and drone imagery for automatic extraction of morphometric attributes of small cetaceans: Application to the St. Lawrence Estuary Beluga population.	Pilot	This project proposes to use a multiyear time series of drone images of the St. Lawrence Estuary Beluga population to develop a series of al...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
103	Improved understanding of the processes impacting harmful algal blooms	Pilot	Using machine learning algorithms to analyze phytoplankton images from the Imaging Flow Cytobot (IFCB) to recognize and quantify species ass...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
96	Use of machine-learning (ML) algorithms and high-performance computing (HPC) for the optimization of satellite image analysis for mapping coastal habitats	Pilot	This project aims to improve satellite image analysis of marine habitats leveraging NRC HPC and ML assets to apply cutting-edge ML technique...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
92	Identification automatisée de la condition de carapace de crabe des neiges Not translated · original: French	Pilot	Détermination de la condition de carapace de crabe des neiges à partir d’images. La méthode offre une façon de valider/améliorer des données...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
93	Détection de homards et de crabes communs dans les images sous-marines par intelligence artificielle. Not translated · original: French	Pilot	Développer un processus de capture et d’analyse d’images sous-marine par Intelligence Artificielle pour la détection de homards et de crabes...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
94	DFO Bot, deep learning application to analyse otolith images available in DFO Dots	Pilot	Development of a neural network trained using otolith images, expert-derived age estimates, otolith annotations and annuli to obtain age est...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
95	Optimization of Machine Learning modelling strategy for coastal zooplankton taxonomic identification from flow microscopy imaging.	Pilot	As part of Aquaculture Monitoring Program we are using Ecotaxa's ML algorithm to identify zooplankton. The project aims at identifying the b...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
161	Remote sensing of the internal tide in the St. Lawrence	Pilot	Using model outputs (STLE500/STLE200) to 1) help identify internal tide surface signature in SWOT (Surface Water and Ocean Topography) data...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
100	Development of AI technology for otolith and scale ageing	Pilot	Ageing of scales and otoliths is time-consuming, relies on skilled personnel, and often faces delays. AI tools like DFOdots offer a solution...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
106	Finwave: an online photo-identification database and AI matching tool	Pilot	This project aims to improve a publicly accessible machine learning platform capable of identifying individual killer whales in real-time...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
118	Use of artificial intelligence models to facilitate seafloor video annotations in conservation areas.	Pilot	The purpose of this initiative is to utilize AI to facilitate seafloor video annotations collected as part of the MCT and Benthic Ecology pr...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
119	Computer vision to automate fish data extraction from video	Pilot	Our purpose has been to build a tool to broadly apply computer vision methods in marine research. Our work has generated several publication...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
122	Development and testing of artificial intelligence and machine learning to analyze underwater photo and video data to assess marine impacts and resources	Pilot	This completed project (2020/21) aimed to utilize machine learning and imaging technology in fisheries science. The proponent planned to use...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
124	Development of an underwater videography and AI system for studying fish passage effectiveness	Pilot	Designed to train existing DFO tools with information on fish passage effectiveness and develop an SOP for training videography-based AI mod...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
126	Automated assessment of satellite imagery	Pilot	AI routines are being developed and tested to assess large volume of satellite imagery and data to: i) Automatically Identify marine hazards...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
130	Salmon Data Digitization	Pilot	Digitizing non-machine-readable (handwritten) documents and extracts characters into a digital format so that it is accessible, editable, an...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda
132	Salmon Scale Aging	Pilot	A model using computer vision to look at images of salmon scale and predict the age of salmon and also explain how it arrived at its predict...	Ecosystems and Oceans Science	National Capital Region	Perception and Understanding	Winegardner, Amanda

228
SILScanner
Deploy on DFO's PBMM Cloud a proof of concept application that uses optical character recognition (OCR) to digitize and validate Stream Inspection Logs (SILs). The PoC was originally developed on SSC's cloud infrastructure. It is now being deployed in DFO PB cloud infrastructure. BLADES initiative and IT security teams are involved.

Pilot Chief Digital Officer Perception and Understanding
187
Atlantic Bluefin Tuna Monitoring
Automatically perform video analytics from hundred of harvester tuna video Solution being developed in region to assist C&P in batch processing of at-sea video monitoring footage.

Pilot Indigenous Affairs, Aquaculture and Governance Perception and Understanding
188
Coastal Surveillance
Automatically process thousands of drone images as part of continuous surveillance Solution being developed in region to assist C&P in batch processing of drone surveillance images.

Pilot Indigenous Affairs, Aquaculture and Governance Perception and Understanding
218
L’observation des bélugas de l’estuaire du Saint-Laurent
Entente de contribution avec le Groupe de recherche et d'éducation sur les mammifères marins (GREMM) pour le développement d'un module de mesures morphométriques assistées par l’intelligence artificielle pour les bélugas de l'estuaire du Saint-Laurent L’entente s’est terminée le 31 mars 2024 mais le module est en cours d’utilisation. Considérant la contrainte de temps imposée par les mesures manuelles des images prises par drones de bélugas, l’équipe de recherche du GREMM a développé une méthode exploitant l’IA pour mesurer plus rapidement et plus efficacement les bélugas.

Pilot Marine Planning and Conservation Perception and Understanding
217
Suite du projet de recherche de recherche et développement d’outils de vision automatisée pour l’étude des refuges marins des coraux et des éponges du golfe du Saint-Laurent.
Une entente de contribution pour l’année 2025-2026 (Programme de gestion des océans) visant à l’amélioration de l’outil développé par le Centre de développement et de recherche en intelligence numérique (CDRIN) en 2024-2025. Le projet a comme objectif global de concevoir et développer un outil d’intelligence artificielle permettant de détecter et suivre les organismes benthiques filmés dans les refuges marins des coraux et des éponges du Golfe du Saint-Laurent.

Pilot Marine Planning and Conservation Perception and Understanding
213
Numérisation de documents manuscrits
En collaboration avec l’équipe nationale de l’Intendant principal des données (PSSI), visant principalement la pêche au saumon. L’objectif est de numériser des documents avec des données manuscrites et de les faire analyser par un modèle d’intelligence artificielle qui va décoder l’écriture pour la transformer en données numériques et les insérer dans une base de données. Le but est de remplacer la saisie manuelle de ces données.

Pilot Statistics, Policy and Licensing Perception and Understanding
212
Surveillance électronique en mer
Il s’agit de capture d’images vidéo sur les navires de pêche pour ensuite les faire analyser par des ressources humaines et un modèle d’intelligence artificielle doté de capacité d’identification d’événements, d’espèces, etc. Projet fait en collaboration avec l’équipe nationale de l’Intendant principal des données.

Pilot Statistics, Policy and Licensing Perception and Understanding
190
Lobster Stock Assessment
Automatically assess lobster count and habitat from hours of video efficiently Solution being developed for Science to review hundreds of hours of video for lobster count and state of habitat.

Pilot Indigenous Affairs, Aquaculture and Governance Perception and Understanding
105
Combining artificial intelligence and drone imagery for automatic extraction of morphometric attributes of small cetaceans: Application to the St. Lawrence Estuary Beluga population.
This project proposes to use a multiyear time series of drone images of the St. Lawrence Estuary Beluga population to develop a series of algorithms for automatic extraction of morphometric information of individual beluga whales using AI.

Pilot Ecosystems and Oceans Science Perception and Understanding
103
Improved understanding of the processes impacting harmful algal blooms
Using machine learning algorithms to analyze phytoplankton images from the Imaging Flow Cytobot (IFCB) to recognize and quantify species associated with harmful algal blooms.

Pilot Ecosystems and Oceans Science Perception and Understanding
96
Use of machine-learning (ML) algorithms and high-performance computing (HPC) for the optimization of satellite image analysis for mapping coastal habitats
This project aims to improve satellite image analysis of marine habitats leveraging NRC HPC and ML assets to apply cutting-edge ML techniques to the challenges of remote sensing data analysis. Relates to ongoing project (below) through MPC service-level agreement.

Pilot Ecosystems and Oceans Science Perception and Understanding
92
Identification automatisée de la condition de carapace de crabe des neiges
Détermination de la condition de carapace de crabe des neiges à partir d’images. La méthode offre une façon de valider/améliorer des données de condition de carapace prises sur le terrain. Ces déterminations sont importantes pour quantifier le recrutement à la pêche durant le relevé scientifique ainsi qu’améliorer la qualité des données de crabe mou dans programme observateur-en-mer.

Pilot Ecosystems and Oceans Science Perception and Understanding
93
Détection de homards et de crabes communs dans les images sous-marines par intelligence artificielle.
Développer un processus de capture et d’analyse d’images sous-marine par Intelligence Artificielle pour la détection de homards et de crabes communs.

Pilot Ecosystems and Oceans Science Perception and Understanding
94
DFO Bot, deep learning application to analyse otolith images available in DFO Dots
Development of a neural network trained using otolith images, expert-derived age estimates, otolith annotations and annuli to obtain age estimates from otoliths.

Pilot Ecosystems and Oceans Science Perception and Understanding
95
Optimization of Machine Learning modelling strategy for coastal zooplankton taxonomic identification from flow microscopy imaging.
As part of Aquaculture Monitoring Program we are using Ecotaxa's ML algorithm to identify zooplankton. The project aims at identifying the best strategy in building model training sets for each region monitored. This strategy will be the basis for future AMP zooplankton identification work.

Pilot Ecosystems and Oceans Science Perception and Understanding
161
Remote sensing of the internal tide in the St. Lawrence
Using model outputs (STLE500/STLE200) to 1) help identify internal tide surface signature in SWOT (Surface Water and Ocean Topography) data and 2) train an AI to detect internal tide in SWOT data.

Pilot Ecosystems and Oceans Science Perception and Understanding
100
Development of AI technology for otolith and scale ageing
Ageing of scales and otoliths is time-consuming, relies on skilled personnel, and often faces delays. AI tools like DFOdots offer a solution by enabling automated ageing using annotated reference collections, helping reduce backlogs and support training. Developing these AI models can enhance resilience, efficiency, and training capacity in ageing programs.

Pilot Ecosystems and Oceans Science Perception and Understanding
106
Finwave: an online photo-identification database and AI matching tool
This project aims to improve a publicly accessible machine learning platform capable of identifying individual killer whales in real-time from submitted photographs. The ultimate goal is to improve the web-based interface and validate the algorithm used for photo identification against known individuals.

Pilot Ecosystems and Oceans Science Perception and Understanding
118
Use of artificial intelligence models to facilitate seafloor video annotations in conservation areas.
The purpose of this initiative is to utilize AI to facilitate seafloor video annotations collected as part of the MCT and Benthic Ecology programs in the NL Region. A large number of vidEcosystems and Oceans Science is collected annually, and manual video annotation (e.g., locating and counting observations) is extremely time-consuming. A published “object detection model” (FathomNet Megalodon Detector, YOLOv8x) trained using marine taxa by MBARI is currently being used to identify objects in our seafloor images. The model does not identify the objects (e.g., to species), but it largely accelerates the process given that locating objects is the most time-consuming part of the work.

Pilot Ecosystems and Oceans Science Perception and Understanding
119
Computer vision to automate fish data extraction from video
Our purpose has been to build a tool to broadly apply computer vision methods in marine research. Our work has generated several publications illustrating how we built, tested/validated, and applied computer vision methodology to collect / extract / and analyze data pertaining to marine fish in the Newfoundland and Labrador region, from video. For example, we have used it to examine the impact of seismic surveying on commercial fish in the Newfoundland and Labrador offshore. Our continuing purpose is now broadly sharing our system with commercial harvesters (fishermen), non-profit organization (AHOI), government researchers (aquaculture, MPAs, AIS), and academics (computing scientists), to collect and rapidly analyzing video data for a wide range of applications.

Pilot Ecosystems and Oceans Science Perception and Understanding
122
Development and testing of artificial intelligence and machine learning to analyze underwater photo and video data to assess marine impacts and resources
This completed project (2020/21) aimed to utilize machine learning and imaging technology in fisheries science. The proponent planned to use Video and Image Analytics for Marine Environments (VIAME), an open-source system developed by NOAA, for two specific purposes: 1. Measure the movement of Atlantic cod across boundaries at the Gilbert Bay Marine Protected Area in Newfoundland and Labrador. 2. Assess the impact of seismic surveying on Atlantic cod as part of a project funded by the Environmental Studies Research Fund (ESRF). The automatic detection and count of fish near MPA boundaries was expected to reduce use of personnel time and effort.

Pilot Ecosystems and Oceans Science Perception and Understanding
124
Development of an underwater videography and AI system for studying fish passage effectiveness
Designed to train existing DFO tools with information on fish passage effectiveness and develop an SOP for training videography-based AI models in support of fish passage projects.

Pilot Ecosystems and Oceans Science Perception and Understanding
126
Automated assessment of satellite imagery
AI routines are being developed and tested to assess large volume of satellite imagery and data to: i) Automatically Identify marine hazards and ii) Change detection – i.e. shorelines or riverbank changes.

Pilot Ecosystems and Oceans Science Perception and Understanding
130
Salmon Data Digitization
Digitizing non-machine-readable (handwritten) documents and extracts characters into a digital format so that it is accessible, editable, and useable for analysis.

Pilot Ecosystems and Oceans Science Perception and Understanding
132
Salmon Scale Aging
A model using computer vision to look at images of salmon scale and predict the age of salmon and also explain how it arrived at its prediction.

Pilot Ecosystems and Oceans Science Perception and Understanding