AI initiative registry
Every initiative listed in the registry can be consulted here, without an account. Each record shows its progress, its milestones and the history of its decisions. Requests still under triage do not appear: they join the registry once reviewed.
Perception and Understanding
AI that interprets the world, whether through text, images, or sound.
What is “Perception and Understanding”?
This category includes AI systems that sense, interpret, or perceive data from the environment, emulating human senses like sight, sound, touch, smell, and language comprehension. These capabilities form the foundation of AI awareness, enabling systems to extract meaning from unstructured inputs like text, images, video, audio, or environmental signals.
| Subcategory | Description | Examples |
|---|---|---|
| Language understanding — Natural Language Processing (NLP) | Understanding and generating human language | Chatbots, document summarization |
| Computer vision | Understanding images, video, and spatial data | Object detection, satellite imagery analysis, species ID |
| Speech recognition | Converting audio to text | Voice-to-text for transcription, voice-based commands for virtual assistants |
| Multimodal AI | Combining input types (text + images, etc.) | Image captioning, document understanding tools |
| Olfactory AI (artificial smell recognition) | Detecting and interpreting smells using e-noses | Gas leak detection, spoilage sensing, pollutant ID in fish labs |
| Haptic AI (artificial touch sensing) | Interpreting touch and tactile feedback | Robotic sampling, pressure detection, texture, temperature |
| Audio event detection | Recognizing non-speech sounds | Marine mammal calls (whale song), mechanical alerts, engine noise anomaly detection |
| Sensor fusion AI | Combining multiple sensory inputs | Autonomous underwater vehicle navigation, habitat monitoring |
- Key indicator
- If the AI’s primary task is to interpret raw input (e.g. image, text, sound, sensor data), this is where it belongs.
- Common input types
-
- Visual data (images, video)
- Audio and speech
- Natural language text
- Sensor readings (e.g. haptics, chemical, temperature)
Filter 1
71 current initiative(s)
-
166
Détection de baleines sur les images obtenues par les dronesCe projet vise à générer des bases de données d’images annotées et des modèles IA entrainés pour détecter les différentes espèces ou groupes d’espèces visés.
Pilot Ecosystems and Oceans Science Perception and Understanding
-
163
Recherches sur l’Intelligence artificielle pour le traitement automatisé des données d’acoustiques sous-marineNous utilisons les technologies d’apprentissage profond (deep learning) et les détecteurs pour la détection et identification en temps réal des espèces de mammifères marins telles que la baleine noire de l’Atlantique.
Pilot Ecosystems and Oceans Science Perception and Understanding
-
93
Détection de homards et de crabes communs dans les images sous-marines par intelligence artificielle.Développer un processus de capture et d’analyse d’images sous-marine par Intelligence Artificielle pour la détection de homards et de crabes communs.
Pilot Ecosystems and Oceans Science Perception and Understanding
-
218
L’observation des bélugas de l’estuaire du Saint-LaurentEntente de contribution avec le Groupe de recherche et d'éducation sur les mammifères marins (GREMM) pour le développement d'un module de mesures morphométriques assistées par l’intelligence artificielle pour les bélugas de l'estuaire du Saint-Laurent L’entente s’est terminée le 31 mars 2024 mais le module est en cours d’utilisation. Considérant la contrainte de temps imposée par les mesures manuelles des images prises par drones de bélugas, l’équipe de recherche du GREMM a développé une méthode exploitant l’IA pour mesurer plus rapidement et plus efficacement les bélugas.
Pilot Marine Planning and Conservation Perception and Understanding
-
217
Suite du projet de recherche de recherche et développement d’outils de vision automatisée pour l’étude des refuges marins des coraux et des éponges du golfe du Saint-Laurent.Une entente de contribution pour l’année 2025-2026 (Programme de gestion des océans) visant à l’amélioration de l’outil développé par le Centre de développement et de recherche en intelligence numérique (CDRIN) en 2024-2025. Le projet a comme objectif global de concevoir et développer un outil d’intelligence artificielle permettant de détecter et suivre les organismes benthiques filmés dans les refuges marins des coraux et des éponges du Golfe du Saint-Laurent.
Pilot Marine Planning and Conservation Perception and Understanding
-
213
Numérisation de documents manuscritsEn collaboration avec l’équipe nationale de l’Intendant principal des données (PSSI), visant principalement la pêche au saumon. L’objectif est de numériser des documents avec des données manuscrites et de les faire analyser par un modèle d’intelligence artificielle qui va décoder l’écriture pour la transformer en données numériques et les insérer dans une base de données. Le but est de remplacer la saisie manuelle de ces données.
Pilot Statistics, Policy and Licensing Perception and Understanding
-
212
Surveillance électronique en merIl s’agit de capture d’images vidéo sur les navires de pêche pour ensuite les faire analyser par des ressources humaines et un modèle d’intelligence artificielle doté de capacité d’identification d’événements, d’espèces, etc. Projet fait en collaboration avec l’équipe nationale de l’Intendant principal des données.
Pilot Statistics, Policy and Licensing Perception and Understanding
-
162
Recherche et développement d’outils de vision automatisée pour l’étude des refuges marins des coraux et des éponges du golfe du Saint-Laurent.Projet de contribution avec le Centre de développement et de recherche en intelligence numérique (CDRIN). L’objectif est de concevoir et développer un outil d’intelligence artificielle permettant de détecter et suivre les organismes benthiques filmés dans refuges marins des coraux et des éponges du Golfe du Saint-Laurent.
Ideation Ecosystems and Oceans Science Perception and Understanding
-
92
Identification automatisée de la condition de carapace de crabe des neigesDétermination de la condition de carapace de crabe des neiges à partir d’images. La méthode offre une façon de valider/améliorer des données de condition de carapace prises sur le terrain. Ces déterminations sont importantes pour quantifier le recrutement à la pêche durant le relevé scientifique ainsi qu’améliorer la qualité des données de crabe mou dans programme observateur-en-mer.
Pilot Ecosystems and Oceans Science Perception and Understanding
-
52
"Chumputer Vision" (see B32) Salmon Scale AgingA model using computer vision to look at images of salmon scale and predict the age of salmon and also monitor its ring growth. CNN and YOLO 9. Currently trained on 1478 images.
Pilot Programs Sector Perception and Understanding
-
174
AI model for Video reviewAn AI model that reviews video monitoring footage for a fishery or given vessel and machine learns potential events of non compliance
Ideation Programs Sector Perception and Understanding
-
34
AI-assisted Marine Mammal Annotation ToolThe goal of the project is to improve the efficiency of the aerial image annotation process to reduce manual efforts of the marine mammal science team and reduce time-to-insight for marine mammal survey flights. This project is currently in progress. Business requirements have been finalized and development has commenced Client: Ecosystems and Oceans Science - Ecosystems Science Directorate
Pilot Strategic Policy Perception and Understanding
-
35
AI-assisted Solution to Detect Ghost Gear from Side Scan Sonar ImagesThe goal of the project is use AI to automate data analysis in detecting Ghost Gear. Client: Ghost Gear Program
Ideation On hold Strategic Policy Perception and Understanding
-
36
AI-assisted Solution to summarize C&P aerial surveillance videosA pilot project to explore the use of AI/ML for automating the identification of relevant segments in EM video and imagery, such as human activity or vessel registration numbers (VRNs). This initiative aims to reduce manual review time and enhance the efficiency of information management in support of fisheries oversight.
Pilot Strategic Policy Perception and Understanding
-
29
AI-Enabled Detection of Watercraft for Aquatic Invasive Species Risk ManagementThis pilot project, in collabration with Aquatic Invasive Species Program, develops an AI-powered system to automatically detect and track watercraft at key entry points to prevent the spread of Aquatic Invasive Species (AIS). By enabling early identification of vessels that may carry invasive organisms, the system supports faster inspections and intervention to protect aquatic ecosystems. Initial exploration using sample data is underway, with the project pending ADM approval for installing cameras and data collection. Client: Aquatic Invasive Species Program
Pilot Strategic Policy Perception and Understanding
-
28
AI-Enabled Electronic Monitoring (EM) SolutionsThe pilot project, in collabration with Fisheries Resources Management, explores the integration of AI-enabled electronic monitoring (EM) solutions to automate the detection and classification of fishing activities using video and sensor data from vessels. This initiative aims to enhance compliance monitoring, reduce manual review time, and support evidence-based fisheries management. Client: Fisheries Resources Management
Pilot Strategic Policy Perception and Understanding
-
165
AIMMS (Artificial Intelligence for Marine Mammals in Survey imagery)The AIMMS project develops a platform to automate and accelerate the interpretation of aerial imagery for marine mammal population assessments. By streamlining annotation and review, and enabling scientists to develop custom machine-learning models without advanced coding skills, the AIMMS workflow significantly reduce processing time, accelerate delivery of science advice, and lower operational costs.
Development Fisheries Management Perception and Understanding
-
87
ANMPA Habitat Mapping ProjectDrop camera survey to document seafloor habitats and biological communities in the Anguniaqvia niqiqyuam Marine Protected Area (ANMPA) in the Inuvialuit Settlement Region. A project sub-objective is to develop AI capacity to detect and record bottom habitat classifications and benthic animals from video still-images, substantially reducing processing times.
Pilot Ecosystems and Oceans Science Perception and Understanding
-
187
Atlantic Bluefin Tuna MonitoringAutomatically perform video analytics from hundred of harvester tuna video Solution being developed in region to assist C&P in batch processing of at-sea video monitoring footage.
Pilot Indigenous Affairs, Aquaculture and Governance Perception and Understanding
-
89
Augmenting Whale Detection in Satellite Images using synthetic dataAn autodetector model was built to detect whales in satellite imagery using synthetic imagery. The primary author is now testing the algorithm on satellite imagery obtained by Arctic Region that has been manually read for whales.
Pilot Ecosystems and Oceans Science Perception and Understanding
-
126
Automated assessment of satellite imageryAI routines are being developed and tested to assess large volume of satellite imagery and data to: i) Automatically Identify marine hazards and ii) Change detection – i.e. shorelines or riverbank changes.
Pilot Ecosystems and Oceans Science Perception and Understanding
-
223
Automated assessment of satellite imageryAI routines are being developed and tested to assess large volume of satellite imagery and data to: i) Automatically Identify marine hazards and ii) Change detection – i.e. shorelines or riverbank changes.
Pilot Ecosystems and Oceans Science Perception and Understanding
-
86
Automatic detection of Arctic whales in aerial imagesIn collaboration with Dr. David Clausi and his team at the Vision and Image Processing (VIP) Research Lab (University of Waterloo), we have been developing an algorithm to detect narwhals and belugas in photos from aerial surveys. We are planning to expand to ice seals and walrus.
Pilot Ecosystems and Oceans Science Perception and Understanding
-
145
Automatic detection of unidentified fish soundsThis completed project used Random Forests and Convolutional Neural Networks (CNN) algorithms trained on manually detected fish sounds to detect fish sounds collected on a passive acoustic monitoring project. The result is an easy-to-use, open-source software called FishSoundFinder, implemented with the CNN detector.
Pilot Ecosystems and Oceans Science Perception and Understanding
-
64
Automatic detection of unidentified fish soundsWe used Random Forestes and Convolutional Neural Networks (CNN) algorithms trained on manually detected fish sounds to detect fish sounds collected on a passive acoustic monitoring project. The result is an easy-to-use, open-source software called FishSoundFinder, implemented with the CNN detector. Random Forests and CNN
Pilot Ecosystems and Oceans Science Perception and Understanding