Enhancing Wildlife Monitoring Through Infrared Imaging: A Deep Learning Approach for Improved Animal Region Segmentation and Species Identification
doi: 10.11916/j.issn.1005-9113.2025082
V Janani , C Shanthi
Department of Computer Science, School of Computing Sciences, Vels Institute of Science Technology and Advanced Studies, Chennai 600117, India
Abstract
Infrared imaging has proven to be a powerful tool in various fields,with particular significance in wildlife monitoring. This study proposes a multi-stage deep learning framework for robust action recognition in surveillance environments,integrating spatial feature extraction,temporal motion modeling,and attention-based fusion. The system combines Mask R-CNN (region-based convolutional neural network )for region-level feature extraction with a lightweight temporal module that captures motion dynamics,supported by optical-flow-based refinement for improved temporal consistency. Evaluated on a curated test set of 50 videos,the framework achieves a mean accuracy of 0.83,precision of 0.86,recall of 0.82,and an F1-score of 0. 84,demonstrating strong performance across diverse activity categories. The results highlight the effectiveness of integrating spatial-temporal cues and showcase the model's capability to handle complex scenes with partial occlusion,variable illumination,and rapid motion. The proposed framework provides a scalable foundation for real-world surveillance applications,with potential for further enhancement through broader benchmarking and domain-specific optimization.
0 Introduction
Wildlife monitoring plays a pivotal role in biodiversity conservation and ecosystem management.Understanding the presence,behavior,and distribution of wildlife species is crucial for making informed decisions about habitat preservation and mitigating human-wildlife conflicts. In recent years,the integration of advanced technologies has revolutionized the field of wildlife monitoring,with infrared imaging emerging as a powerful tool in this endeavor[1-3].
Infrared imaging offers unique advantages for wildlife monitoring due to its ability to capture thermal radiation emitted by animals and the environment.This technology enables the observation of animals in low-light conditions,during nighttime,or in dense vegetation,providing valuable insights that were previously inaccessible through conventional visible light imaging[4-8]. However,to fully leverage the potential of infrared imaging for wildlife monitoring,it is essential to address the challenge of accurately detecting and segmenting animal regions within these images. Accurate animal region segmentation in infrared images is a fundamental step towards extracting meaningful data for species identification,behavior analysis,and population estimation.Achieving this accurately and efficiently presents a significant computational challenge. In our previous work,we proposed a methodology that utilized deep learning techniques,specifically the R-CNN(region-based convolutional neural network)object detection algorithm,to enhance the segmentation of animal regions in enhanced infrared images[9-12].
While accurately segmenting animal regions is a critical task,knowing the species within those regions is equally valuable for conservation efforts. This extension aims to provide a holistic solution that not only identifies animal regions but also classifies them into specific species categories[6]. To accomplish this,we continue to rely on the R-CNN model as our foundational framework and further harness the power of transfer learning. Transfer learning allows us to leverage knowledge acquired from large,diverse datasets,often from different but related domains,and apply it to our specific task. By fine-tuning the R-CNN model on a comprehensive dataset comprising annotated infrared images and species labels,we enhance its ability to both segment animal regions and classify them into species categories[13-16]. The contribution of this research lies in its capacity to provide a unified solution for wildlife monitoring through infrared imaging. By improving the accuracy of animal region segmentation and introducing species identification capabilities,we aim to empower conservationists,ecologists,and wildlife managers with a more comprehensive and efficient toolset for understanding and safeguarding our natural world.This study presents the methodology,experimental results,and implications of our extended approach,highlighting its potential to advance wildlife conservation and habitat management.
1 Related Work
To enhance the segmentation of animal regions in enhanced infrared images,various methodologies have been explored,each contributing valuable insights while also presenting certain limitations. Chen et al.[9]worked on wildlife surveillanced and introduced deep learning methods for this domain,which potentially face challenges adapting to diverse imaging modalities.The Tensormask approach[10]focuses on dense object segmentation,with potential computational complexities as a drawback. R-FCN[11]brings advancements in object detection,but its computational demands could pose challenges.Flownet[12]concentrates on optical flow learning,but its application to animal region segmentation in static infrared images could be limited. Additionally,VGG image annotator(VIA)annotation software[13]and the PASCAL visual object classes(VOC)challenge[14]provide valuable tools but may introduce complexities in integration. On the other hand,Saxena et al.'s[15]animal detection system leverages deep learning but have limitations in diverse environmental conditions. Acknowledging these drawbacks and understanding their implications,the proposed methodology aims to navigate potential challenges and leverage the strengths of each referenced approach to enhance precision and adaptability.
1)Infrared imaging in wildlife monitoring:the use of infrared imaging in wildlife monitoring has gained significant attention in recent years.Castelluccio et al.[7]explored the advantages of infrared technology in capturing thermal signatures of animals,enabling non-intrusive observation during nocturnal activities and in challenging environmental conditions. Various studies have demonstrated the potential of infrared imaging for tracking animal movements and behavior[3,9].
2)Object detection in infrared images:object detection in infrared images is a crucial component of wildlife monitoring[8]. Traditional computer vision methods have been employed for this task,including techniques based on edge detection and feature extraction. However,recent advancements in deep learning,such as convolutional neural networks(CNNs),have shown superior performance in object detection tasks,including those in infrared imagery.
3)Transfer learning in computer vision:transfer learning has become a cornerstone of modern computer vision applications. It involves pre-training deep neural networks on large,diverse datasets and fine-tuning them for specific tasks. In the context of wildlife monitoring,transfer learning has been used to adapt models trained on general image datasets to the unique challenges posed by infrared images and animal region segmentation.
4)Species identification in wildlife monitoring:while detecting and segmenting animal regions is essential,species identification is equally important for conservation efforts[17-19]. Prior research has addressed species classification using visible light imagery and audio data. However,the integration of species identification with infrared image and deep learning techniques is an emerging area of interest.
5)Segmentation metrics:various segmentation metrics,such as Intersection over union(IoU),Dice coefficient,and pixel accuracy,are commonly used to evaluate the accuracy of object segmentation algorithms. These metrics provide quantitative measures of how well segmented regions align with ground truth annotations and are crucial for assessing the performance of segmentation models[19-21].
Integrated approaches:some recent studies have attempted to combine object detection and species identification in the context of wildlife monitoring.These integrated approaches aim to provide a comprehensive understanding of animal presence and identity simultaneously,enabling more detailed ecological studies and conservation efforts[22-25].
In this paper,we build upon these existing research areas by extending our previous work on animal region segmentation in enhanced infrared images. Our approach leverages the strengths of deep learning,transfer learning,and object detection techniques to enhance both the accuracy of region segmentation and the capability for species identification,offering a holistic solution for wildlife monitoring through infrared imaging. Our contributions lie in the integration of these techniques to address the complex challenges of wildlife conservation and habitat management.
2 Methodology
Our methodology for enhancing wildlife monitoring through infrared imaging encompasses three main components:data collection and preparation,the R-CNN-based deep learning model,and transfer learning. We detail each component below.
2.1 Data Collection and Preparation
To train and evaluate our model,we curated a comprehensive dataset of 1 237 enhanced infrared images collected entirely from publicly available sources. These include:1)wildlife research datasets from open-access repositories,such as iWildCam2022[26],provide infrared wildlife images captured in natural habitats;and 2)open datasets from ecological monitoring projects,such as the annotated infrared images from the AAAI wildlife dataset[27],which were captured under diverse environmental conditions,times of day,and geographical locations.
1)Image diversity. The dataset includes variability in lighting conditions(daylight,twilight,nighttime,and low-light conditions),environmental settings(forests,grasslands,wetlands,and urban areas),weather conditions(clear skies,fog,rain,and snow),and geographical locations(multiple regions to ensure ecological diversity).
2)Annotations and class distribution. Each image was annotated with bounding boxes specifying animal regions and labeled with species information,following collaboration with domain experts and using existing species databases. The dataset encompasses 15species of mammals,birds,reptiles,and amphibians.The class distribution is provided in Table 1.
Table 1 Class distribution
3)Data augmentation. To improve model generalization and simulate real-world conditions,the following augmentation techniques were applied:First,random rotations and flips were applied to handle orientation variations. Then,brightness and contrast adjustments were applied for varying lighting conditions. After that,cutout and random erasing were applied to simulate occlusions. Finally,MixUp was used to generate interpolated images for enhanced robustness.
4)Data split. The dataset was divided as follows:the training set is 70% of images,the validation set is 15%,and the test set is 15%. This split ensures a balanced evaluation while maintaining unseen data for final performance assessment.
2.2 Mask R-CNN-Based Deep Learning Model
To achieve accurate pixel-level segmentation of animals in infrared imagery,we adopted the Mask R-CNN architecture. Mask R-CNN extends Faster R-CNN by adding a parallel segmentation mask branch,making it well-suited for detecting individual animals and producing instance-level segmentation masks. The network comprises a ResNet-50+ FPN backbone for feature extraction,a region proposal network(RPN)for generating candidate regions,an RoI align layer for precise spatial feature alignment,and three output heads:classification head predicts the species category;bounding box regression head refines the location of the animal;mask prediction head generates a binary mask for each detected instance.
2.2.1 Model adaptation
Several adaptations were applied to tailor Mask R-CNN to the wildlife infrared dataset. The final classification layer was modified to match the number of species categories in the dataset. Anchor box sizes and aspect ratios were adjusted to better represent typical animal shapes observed in infrared imagery.Infrared-specific preprocessing steps were integrated to improve feature capture under low-light or high-noise conditions.
2.2.2 Training process
The adapted Mask R-CNN model was trained end-to-end on the annotated infrared image dataset.During training,the model simultaneously optimized:segmentation loss for pixel-level mask prediction;classification loss for species identification;bounding box regression loss for accurate location estimation.Through iterative epochs,the model learned to segment animal regions and classify species with increasing accuracy.
2.2.3 Hyperparameter tuning
To ensure optimal performance,key hyperparameters such as learning rate,batch size,and optimizer configuration were fine-tuned through systematic cross-validation. This process improved convergence stability and enhanced overall segmentation and classification precision. The overall architecture is shown in Fig.1.
Fig. 1 Proposed architecture
The proposed wildlife monitoring system processes infrared video frames through a structured,multi-stage workflow. Initially,the raw video frames are preprocessed by converting them to grayscale,applying a dynamic background model,computing frame differences,and performing adaptive thresholding and median filtering to reduce noise. This motion-based filtering step identifies frames likely to contain animal movement while discarding frames with negligible or excessive motion,thereby reducing false positives and computational load. Selected frames are then fed into the Mask R-CNN model,which comprises a ResNet-50+ FPN backbone for feature extraction,a region proposal network(RPN)for candidate regions,RoI align for precise spatial alignment,and three parallel heads for species classification,bounding box regression,and pixel-level mask segmentation. The outputs—instance masks,bounding boxes,and species labels—are stored in a database and simultaneously utilized for action recognition,visualization,and alert generation.This integrated workflow enables efficient detection,segmentation,species identification,and behavior analysis of wildlife in diverse environmental conditions,providing a comprehensive and practical tool for ecological studies and conservation efforts.
2.2.4 Motion⁃based frame filtering
Video footage was processed directly by the trained CNNs,as film can be regarded as a sequence of image frames. To expedite the detection process,all images were transformed into grayscale. When an image was identified as a potential frame of interest,the color framework was utilized for recognition.Images of interest were those containing objects of interest(any animal). In film footage,the motion of an animal led to variations in pixel values in adjacent frames. Consequently,differences between adjacent frames could be computed intuitively. In this case,instead of applying frame differences directly,a dynamic background(B)was utilized,with the following updating process:
Bg(i,j)=Ir(i,j)
(1)
where 𝑖 and 𝑗 denote the vertical and horizontal pixel positions,𝐼ᵣ represents the input frame,and 𝐵𝗀(𝑖,𝑗)signifies the current pixel value at the location index(𝑖,𝑗). Initially,𝐵𝗀 was set as the first input frame and updated dynamically,resulting in the difference between the current frame and the background as given in Eq.(2).
|Ir(i,j)−Bg(i,j)|
(2)
However,it should be noted that frame differences do not necessarily indicate the presence of an animal,as these differences may also be caused by,for example,moving vegetation. To mitigate the false-positive detection rate,the following assumptions were made:1)when an animal is in motion,the frame difference should be relatively substantial,and 2)the movement of the animal is the primary factor behind the pixel changes,and the camera is not obstructed by the animal's body.
To eliminate minor variations,a dynamic threshold process was applied based on the maximum value among all Di,j(t),expressed as follows:
D i , j ( t ) = { 0 , if D i , j < β D i , j , otherwise
(3)
where 𝐷𝑖,𝑗 refers to the theresholded pixel-wise frame-difference value at coordinates(𝑖,𝑗)within the motion map 𝐷.
This process was followed by the application of a median filter aimed at noise reduction. Animal movements typically occur within a confined area;therefore,if a large area is in motion,it is probable that the camera is either in motion or has been obstructed by an animal's body. Consequently,a frame was omitted when its difference map(𝐷)had nonzero values that were either excessively small or too large. The size of the animal was restricted to 200pixels and half of the total pixels in the image. Frames with significant pixel variations were excluded to reduce the false-positive detection rate caused by factors such as camera movement,windy conditions,and sudden changes in the scene.
For the frames under consideration,an energy term 𝐸 was calculated by summing all the nonzero values in 𝐷t,as illustrated in Eq.(4).
E′=∑i,jDi,ji
(4)
The average pixel variation Yt′ was subsequently calculated using Eq.(5),where 𝑛 represents the number of nonzero pixels in the difference image 𝐷t.
Yt′=Et′n
(5)
For frames with animal motion,the image should exhibit a high total energy(𝐸t). Additionally,the pixel variation caused by animals should be more pronounced than other factors,implying that the variation resulting from animal objects should constitute the primary portion of the total energy,and as a result,Yt′ should be substantial. Therefore,by comparingYt′ with a threshold,the 𝑡th frame would be forwarded to the classification stage if its 𝑌 value exceeded the threshold.
2.3 Transfer Learning
1)Pre-trained model selection. Transfer learning played a crucial role in our methodology. We leveraged a pre-trained CNN model(e. g.,a model trained on a large,diverse dataset like ImageNet)as the starting point for our R-CNN. This pre-trained model captured general image features and enabled our model to understand visual patterns that are relevant to our task.
2)Fine-tuning. We fine-tuned the pre-trained model's weights on our specific dataset. During this process,the model adapted to the unique characteristics of infrared images and learned to identify animal regions and their associated species.This transfer of knowledge from a broader domain to our specific task significantly improved our model's performance,especially when faced with limited labeled infrared data.
3)Transfer learning benefits. Transfer learning allowed our model to inherit knowledge about textures,shapes,and features relevant to animal region segmentation and species identification. This approach mitigated the challenges associated with training deep learning models from scratch when working with a relatively small dataset.
The combination of our enhanced dataset,fine-tuned R-CNN model,and transfer learning techniques resulted in a robust and accurate system for wildlife monitoring. This approach enabled both the segmentation of animal regions in infrared images and the classification of these regions into specific species categories,providing a comprehensive solution for ecological studies and conservation efforts.
2.3.1 Data augmentation
The augmentation of the dataset was a critical aspect to ensure the model's exposure to a wide variety of conditions. Advanced data augmentation techniques were strategically integrated to artificially amplify diversity.
1)Cutout. This technique involved the random masking of rectangular sections within the images during training. By obscuring portions of the images,the model was encouraged to learn features resilient to occlusions and background variations.
2)MixUp. During training,pairs of images and their associated labels were subjected to linear interpolation,generating novel combinations. This not only expanded the dataset but also fostered superior generalization by training the model on diverse image pairs.
3)Random erasing. Random rectangular regions within the images were replaced with random pixel values during training. This simulation of occlusions or missing information bolstered the model's resilience to partial obstructions,a common occurrence in real-world scenarios. The synergistic application of these augmentation techniques ensured that the model was exposed to a comprehensive range of conditions,enhancing its adaptability to diverse real-world scenarios.
4)Model training. The model underwent an extensive training regimen spanning multiple epochs.Throughout this iterative process,the model dynamically adjusted its parameters to minimize specified loss functions. The primary focus lay on two key loss functions. Region segmentation loss guided the model in accurately delineating animal regions within the images. Species classification loss directed the model in classifying the identified regions into the appropriate species. This dual emphasis on segmentation and classification loss functions enabled the model to develop a nuanced understanding of both spatial and categorical aspects of the data.
2.3.2 Hyperparameter tuning
Fine-tuning of hyperparameters was a meticulous process critical for achieving optimal model convergence and performance. Parameters such as learning rates,batch sizes,and optimizer settings underwent systematic adjustments. Cross-validation experiments were conducted to identify the most effective hyperparameter configurations. This approach ensured that the model learned efficiently from the training dataset without succumbing to overfitting.
1)Dataset size. The dataset consists of a substantial number of annotated infrared images,comprising thousands of instances. The dataset size allows for effective training and evaluation of the deep learning model.
2)Data split. We divided the dataset into three main subsets. The first was the training set,which was used for training the R-CNN-based deep learning model. Next was the validation set,which was employed for hyperparameter tuning and model evaluation during training. The last one was the test set,which was reserved for final model evaluation to assess its performance on unseen data.
3)Data integrity and quality control. Rigorous quality control measures were implemented to ensure the accuracy and consistency of annotations;anomalies,such as mislabeled species or inaccurate bounding boxes,were identified and corrected during dataset curation.
3 Experimental Setup and Evaluation
In this section,we describe the experimental setup used to train and evaluate our enhanced wildlife monitoring system. We provide details on how the deep learning model was trained,the evaluation metrics employed,and the results of our experiments.
3.1 Training the Deep Learning Model
Data split:We split our dataset into training,validation,and test sets with a ratio of 70%,15%,and 15%,respectively. This division ensured that the model was trained on a diverse range of images while maintaining a separate,unseen test set for performance evaluation.
Training parameters:The model was trained using an NVIDIA GPU,and training parameters were carefully tuned. We used a batch size of 32,a learning rate of 0.001,and the Adam optimizer.Training continued for 50 epochs with early stopping based on validation performance.
Two loss functions were used during training.Region segmentation loss encouraged the model to accurately segment animal regions within the images.Species classification loss guided the model in classifying segmented regions into specific species categories.
The effectiveness of object detection heavily relies on the quality and diversity of the training dataset. To address this,we meticulously curated a dataset of enhanced infrared images,encompassing various animal species and environmental scenarios.Each image in our dataset is meticulously annotated with bounding boxes specifying the precise location of animal regions,providing invaluable ground truth for training the model. Fig. 2 shows the results of the detection of four animals.
3.2 Computational Cost Analysis
3.2.1 Model training computational cost
During the model training phase,we meticulously examined convergence behavior concerning time frames and resource utilization. The computational cost of training was intricately linked to the size and complexity of the annotated infrared image dataset. Our model demonstrated efficient convergence,averaging 12 h per epoch,with a total of 20 epochs. This indicates a balance between training effectiveness and time investment,ensuring a manageable process.
Fig. 2 Object detection clips
3.2.2 Inference time for real⁃time processing
Although the proposed system achieved an average inference latency of 0.5 s per frame(≈2 FPS(frames per second)),this performance did not meet the conventional threshold for real-time video analysis,which typically requires greater than or equal to 20 FPS to match standard video frame rates.Therefore,rather than claiming real-time operation,we clarify that the current implementation provides near real-time or low-latency offline processing,suitable for batch analysis or semi-interactive applications. Future optimizations,including model pruning,TensorRT acceleration,and GPU-optimized temporal modules,will be explored to achieve true real-time throughput.
3.3 Action Recognition in Wildlife Monitoring
In addition to precise animal region segmentation and species identification,the incorporation of action recognition is a significant advancement in our research methodology. Action recognition extends our capabilities beyond static imagery,enabling us to analyze and understand the dynamic behaviors and interactions of wildlife in their natural habitats. This section provides an overview of our approach to action recognition and its implications in the realm of wildlife monitoring and conservation. Action recognition clips are shown in Fig.3.
The integration of action recognition into our methodology enhances our understanding of wildlife dynamics. It enables us to monitor activities that may not be apparent in static images alone. The insights gained from action recognition have broad applications. First,action recognition facilitates the study of animal behaviors,helping researchers gain deeper insights into feeding patterns,social interactions,and other critical aspects of animal ecology. Second,monitoring wildlife behaviors and detecting unusual actions or distress signals can aid in the early detection of threats or environmental changes,contributing to more effective conservation efforts. Third,action recognition can inform habitat assessments by identifying how wildlife species utilize and adapt to their environments over time.
3.4 Evaluation Metrics
To assess the performance of our system,we employed a set of evaluation metrics. Intersection over union(IoU):this metric measured the overlap between the predicted and ground truth bounding boxes,providing insight into the accuracy of animal region segmentation. Dice coefficient:similar to IoU,the dice coefficient quantified the similarity between predicted and ground truth regions,focusing on the spatial overlap. Pixel accuracy:we calculated pixel accuracy to evaluate the model's ability to correctly classify individual pixels within segmented animal regions. Precision,recall,and F1-score:these standard classification metrics were used to evaluate the species identification aspect of our system.Precision measures the accuracy of species classification,recall quantifies the model's ability to correctly identify species,and the F1-score provides a balanced measure of both.
Fig. 3 Action recognition of six successive images of a deer
4 Results and Discussion
4.1 Performance Illustration
This section describes the performance of the Mask R-CNN-based wildlife detection system on the enlarged test set,which includes the segmentation,classification,and action recognition of animals,compared to the normal models,such as YOLOv5,U-Net,and Faster R-CNN. The performance indicators are shown as mean+/-std over the test set with a 95% confidence interval,as well as the computational costs are listed in Table 2.
Table 2 Summary of computational resources
Accurate segmentation of animal regions in wildlife images is a critical step for effective monitoring and ecological research. Deep learning-based segmentation models like Mask R-CNN achieve excellent pixel-level object segmentation performance due to their capacity to jointly predict the bounding boxes of the objects and segmentation masks[20]. More recently,various models including YOLOv5[21],U-Net[22]and Faster R-CNN[23]have been benchmarked in wildlife and infrared image segmentation,demonstrating different trade-offs between detection speed and segmentation precision. We tested our system on a total of 1 237 infrared images belonging to various animal species and covering different environments. The mean IoU,Dice coefficient,pixel accuracy,standard deviation and 95% confidence interval were calculated.
As shown in Table 3,the Mask R-CNN model outperformed YOLOv5,U-Net,and Faster R-CNN in all segmentation metrics,where the test set n represents 1 237 images. The overall IoU of 0.87 shows that the predicted masks overlap significantly with the ground truth labels,and Dice coefficient of 0.91 proves that pixel level prediction is accurately conducted. But the average IoU of YOLOv5 is only 0.81 because it optimizes for speed,not pixel level prediction. The others show improvement,with a low standard deviation(0.03)and a narrow 95% confidence interval(CI)of 0. 86-0. 88 for Mask R-CNN,suggesting robust performance across diverse species and environmental conditions. The results in Fig.4 confirm that Mask R-CNN is particularly effective for segmenting animal regions in infrared images,making it a reliable choice for automated wildlife monitoring and population studies.
Table 3 Segmentation performance across models
Fig. 4 Performance comparison with segmentation model
Species identification is a fundamental prerequisite for biodiversity monitoring,ecological studies,and management and conservation planning. Deep learning techniques,such as Mask R-CNN[20],have been proven to outperform other object classitisation models to identify multiple classes of objects within natural scenes. Few previous works implement deep learning methods,suck as YOLOv5 for fast object detection[21],the U-Net[22]for segmentation-based identification,or the Faster R-CNN[23]for detection and classification,respectively. Our system was tested on 1 237 recorded infrared images,all with annotations on positions and identification of the objects,and evaluate using the standard metrics precision,recall and F1-score,with mean and standard deviation and 95% confidence intervals.
Table4 indicates that,out of the architectures evaluated,Mask R-CNN yields a higher precision(0.86)and recall (0.87)than any other,where the test set n represents 1 237 images. This indicates that Mask R-CNN provides a fairly accurate prediction for the species classification of an infrared photo of a wild animal. The F1-score of 0. 86 indicates that Mask R-CNN achieve a balance between precision and recall in decreasing both false positives and negatives. It seems from Fig.5 that YOLOv5 is trading off recall and precision for the speed of prediction,hence the reduced scores.
Table 4 Species identification performance
Fig. 5 Detection performance comparison across models
U-Net scores average results but without explicit bounding box prediction for classification,its results will not be as accurate as predicted. Faster R-CNN performs well but cannot match Mask R-CNN's advantage of pixel level segmentation. The low standard deviations along with the narrow 95% CI for all measures suggest that all methods perform consistently for a range of species and image types. In fact,Mask R-CNN appears to be the most robust methodology.
Infrared video sequences are the basis for animal action recognition since infrared imagery does not capture the background,making the identification of specific animal motions more robust and less prone to confusion. Image temporal dependencies can be exploited by combining a spatial detection network with a temporal reasoning module like optical flow or temporal convolution. Results of four hybrid architectures on 50 labelled infrared videos are compared in terms of accuracy,precision,recall and F1-score,using statistical criteria(standard deviation and 95% CI bounds)to evaluate their reliability and reproducibility.
As shown in Table 5,the Mask R-CNN+temporal module configuration outperforms all other architectures,achieving the highest mean accuracy(0.83)and F1-score(0. 84),where the test set n represents 50 videos. This improvement stems from Mask R-CNN's ability,as shown in Fig.6,to provide more precise region-level features,enabling the temporal module to capture motion dynamics more effectively.
Table 5 Action recognition performance
Fig. 6 Comparison of action recognition
Faster R-CNN also performs reasonably well the but is slightly inferior to other methods because of the lack of pixel-level segmentation. YOLOv5 with optical flow shows the poorest results as it was developed to prioritize inference speed rather than precise spatial resolution,which is essential for fine-grained action recognition. U-Net with optical flow provides encouraging performance level but again is bounded by its segmentation-only nature,lacking bounding box information at the object level. The low variance in the performance of the methods as observed by the small standard deviation across the results,indicates that the results are stable. The consistency in performance across different models supports the choice of using a segmentation-enriched spatial model combined with temporal reasoning for reliable action recognition in the wildlife context.
4.2 Ablation Study
In order to evaluate the impact of each component within the proposed architecture,we executed a set of ablation experiments on both image and video datasets.Image dataset contains 1 237 infrared images,while video dataset contains 50 infrared videos. Table 6displays the ablation study. The experiments included the following variations:A1-segmentation only(Mask R-CNN):only animal region segmentation was performed;A2-segmentation+ species identification:classification head activated,without temporal reasoning;A3-segmentation+ temporal module(no species ID):evaluates the effect of temporal modeling on action recognition;A4-full model(segmentation+species ID+ temporal module).
Table 6 Ablation performance summary
Overall,the best results are achieved when the entire integrated system(A4)is used across all tasks.The integration of spatial and temporal features improves action recognition performance,reducing in an action F1-score of 0.84 compared with 0.81 for the baseline configuration(0.84>0.81).
5 Conclusions
This study presents a comprehensive wildlife monitoring system by adopting Mask R-CNN-based segmentation,species identification,and temporal action recognition in infrared images and videos. The system was evaluated using a dataset of 1 237 infrared images and 50 annotated videos collected across camera traps,IR drones,and field monitoring systems,showing consistently high performance on all tested wildlife monitoring metrics. The Mask R-CNN-based method outperformed current state-of-the-art baselines at all three tested tasks:animal region segmentation,species identification,and behavior/action recognition. On the segmented animal region task,the Mask R-CNN method achieved the highest Mean IoU of 0.87,with 6% and 4% elevation,respectively,compared to the latest YOLOv5 and U-Net algorithms,and with a small standard deviation of0.03. The high Dice coefficient(0. 91)and pixel accuracy(0.93)proved the spatial resolution. In the species identification task,Mask R-CNN achieved the highest precision(0.86),recall(0.87),and F1-score(0. 86)with 5% and 3% elevation,respectively,compared to YOLOv5 and U-Net algorithms. Small standard deviations of 0.02 further proved the identification capability. Significantly,the metrics are reported with 95% confidence intervals of 0.85,and0.87,supporting generalization. In the behavior/ action recognition task,using the integrated temporal reasoning module,the system achieved the highest accuracy of 0.83,with 5% and 2% elevation,respectively,above YOLOv5 with optical flow and Faster R-CNN with temporal module. The F1-score of0. 84 is in favor of Mask R-CNN with temporal processing,which shows great ranking capacity over13 quite different behavioral clusters. Overall,the combination of Mask R-CNN with temporal processing had the highest accuracy performance. High success rates for both spatial accuracy,species detection and behavior recognition enable the system for wildlife conservation,ecological study,and habitat management under adverse low-light conditions.Future directions will include scaling the dataset,incorporating transformer video models,and applying uncertainty quantification to improve robustness.
Conflict of Interest
The authors declare no potential conflict of interest.
Fig. 1 Proposed architecture
Fig. 2 Object detection clips
Fig. 3 Action recognition of six successive images of a deer
Fig. 4 Performance comparison with segmentation model
Fig. 5 Detection performance comparison across models
Fig. 6 Comparison of action recognition
Table 1 Class distribution
Table 2 Summary of computational resources
Table 3 Segmentation performance across models
Table 4 Species identification performance
Table 5 Action recognition performance
Table 6 Ablation performance summary
Berger-Tal O, Saltz D. Using the movement patterns of reintroduced animals to improve reintroduction success. Current Zoology, 2014, 60(4): 515-526. DOI: 10.1093/czoolo/60.4.515.
Bouché P, Lejeune P, Vermeulen C. How to count elephants in West African savannahs?Synthesis and comparison of main game count methods. Biotechnology, Agronomy, Society and Environment, 2012, 16(1): 77-91.
Brust C A, Burghardt T, Groenenberg M, et al. Towards automated visual monitoring of individual gorillas in the wild. 2017 IEEE International Conference on Computer Vision (ICCV) Workshops. Piscataway: IEEE, 2017: 2820-2830. DOI: 10. 1109/ICCVW. 2017. 333
Buda M, Maki A, Mazurowski M A. A systematic study of the class imbalance problem in convolutional neural networks. Neural Networks, 2018, 106: 249-259. DOI: 10.1016/j.neunet.2018.07.011.
Burghardt T, Ć alić J. Analysing animal behaviour in wildlife videos using face detection and tracking. IEEE Proceedings-Vision Image Signal Processing. Piscataway: IEEE, 2006, 153(3): 305-312. DOI: 10. 1049/ip-vis: 20050052.
Carreira J, Zisserman A. Quo vadis, Action recognition?A new model and the kinetics dataset. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2017: 4724-4733. DOI: 10. 1109/CVPR. 2017. 502.
Castelluccio M P G, Poggi G, Sansone C, et al. Land use classification in remote sensing images by convolutional neural networks. V1. arXiv preprint arXiv: 1508. 00092. (2015-08-01) [2025-01-25]. 2015: 1-11. DOI: 10. 1109/IGARSS. 2015. 7325959.
Chamoso P, Raveane W, Parra V, et al. UAVs applied to the counting and monitoring of animals. Advances in Intelligent Systems and Computing. Cham: Springer, 2014: 71-80. DOI: 10.1007/978-3-319-07596-9_8.
Chen R, Little R, Mihaylova L, et al. Wildlife surveillance using deep learning methods. Ecology and Evolution, 2019, 9 (17): 9453-9466. DOI: 10.1002/ece3.5410.
Chen X, Girshick R, He K, et al. Tensormask: a foundation for dense object segmentation. 2019 IEEE International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2019: 2061-2069. DOI: 10. 1109/ICCV. 2019. 00215.
Dai J, Li Y, He K, et al. R-FCN: Object detection via region-based fully convolutional networks. Proceedings of the 30th International Conference on Neural Information Processing Systems. New York: ACM, 2016: 379-387. DOI: 10.5555/3157096.3157139.
Dosovitskiy A, Fischer P, Ilg E, et al. FlowNet: Learning optical flow with convolutional networks. 2015 IEEE International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2015: 2758-2766. DOI: 10. 1109/ICCV. 2015. 316.
Dutta A, Zisserman A. The VIA annotation software for images, audio and video. Proceedings of the 27th ACM International Conference on Multimedia. New York: ACM Press, 2019: 2276-2279. DOI: 10.1145/3343031.3350535.
Everingham M, Van Gool L, Williams C K I, et al. The pascal visual object classes (VOC) challenge. International Journal of Computer Vision, 2010, 88: 303-338. DOI: 10.1007/s11263-009-0275-4.
Saxena A, Gupta D K, Singh S. An animal detection and collision avoidance system using deep learning. International Conference on Advanced in Communication and Computational Technology. Singapore: Springer, 2021: 1069-1084. DOI: 10.1007/978-981-15-5341-7_81.
Schneider S, Taylor G W, Kremer S. Deep learning object detection methods for ecological camera trap data. 2018 15th Conference on Computer and Robot Vision (CRV). Piscataway: IEEE, 2018: 321-328. DOI: 10. 1109/CRV. 2018. 00052.
Falzon G, Lawson C, Cheung K W, et al. ClassifyMe: a field-scouting software for the identification of wildlife in camera trap images. Animals, 2020, 10(1): 58. DOI: 10.3390/ani10010058.
Feichtenhofer C, Fan H, Malik J, et al. Slowfast networks for video recognition. The IEEE International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2019: 6201-6210. DOI: 10. 1109/ICCV. 2019. 00630.
Mazur-Milecka M, Ruminski J. Deep learning based thermal image segmentation for laboratory animals tracking. Quantitative InfraRed Thermography Journal, 2021, 18 (3): 159-176. DOI: 10.1080/17686733.2020.1720344.
He K, Gkioxari G, Dollár P, et al. Mask R-CNN. 2017 IEEE International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2017: 2980-2988. DOI: 10. 1109/ICCV. 2017. 322.
Ultralytics. Ultralytics/YOLOv5. [2025-01-25]. https://github.com/ultralytics/yolov5.
Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2015: 234-241. DOI: 10.1007/978-3-319-24574-4_28.
Ren S, He K, Girshick R, et al. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149. DOI: 10. 1109/TPAMI. 2016. 2577031.
Simonyan K, Zisserman A. Two-stream convolutional networks for action recognition in videos. Proceedings of the 27th International Conference on Neural Information Processing Systems. Montreal: ACM, 2014, 1: 568-576. DOI: 10.5555/2968826.2968890.
Feichtenhofer C, Fan H, Malik J, et al. SlowFast networks for video recognition. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2019: 6201-6210. DOI: 10. 1109/ICCV. 2019. 00630.
Beery S, Cole E, Gjoka A. The iWildCam 2020 Competition Dataset. V1. arXiv: 2004. 10340. (2020-04-21)[2025-01-25]. DOI: 10.48550/arXiv. 2004. 10340.
Wildlife. Conservation Society. WCS Camera Traps. Labeled Information Library of Alexandria: Biology and Conservation, LILA BC. https://lila.science/datasets/wcscameratraps/.

LINKS