Successes and limitations of pretrained YOLO detectors applied to unseen time-lapse images for automated pollinator monitoring

Successes and limitations of pretrained YOLO detectors applied to unseen time-lapse images for automated pollinator monitoring | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Successes and limitations of pretrained YOLO detectors applied to unseen time-lapse images for automated pollinator monitoring Valentin Ștefan, Thomas Stark, Michael Wurm, Hannes Taubenböck, and 1 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6335312/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 21 Aug, 2025 Read the published version in Scientific Reports → Version 1 posted 10 You are reading this latest preprint version Abstract Pollinating insects provide essential ecosystem services, and using time-lapse photography to automate their observation could improve monitoring efficiency. Computer vision models, trained on clear citizen science photos, can detect insects in similar images with high accuracy, but their performance in images taken using time-lapse photography is unknown. We evaluated the generalisation of three lightweight YOLO detectors (YOLOv5-nano, YOLOv5-small, YOLOv7-tiny), previously trained on citizen science images, for detecting ~ 1,300 flower-visiting arthropod individuals in nearly 24,000 time-lapse images captured with a fixed smartphone setup. These field images featured unseen backgrounds and smaller arthropods than the training data. YOLOv5-small, the model with the highest number of trainable parameters, performed best, localising 91.21% of Hymenoptera and 80.69% of Diptera individuals. However, classification recall was lower (80.45% and 66.90%, respectively), partly due to Syrphidae mimicking Hymenoptera and the challenge of detecting smaller, blurrier flower visitors. This study reveals both the potential and limitations of such models for real-world automated monitoring, suggesting they work well for larger and sharply visible pollinators but need improvement for smaller, less sharp cases. Earth and environmental sciences/Ecology/Ecological networks Biological sciences/Computational biology and bioinformatics/Machine learning Biological sciences/Ecology Pollinator detection Automated insect monitoring Out-of-distribution generalisation YOLO detectors Smartphone images Time-lapse images Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Introduction Pollinators play a crucial role in sustaining our ecosystems and ensuring food security. Yet they face an alarming decline 1 , 2 , which has the potential to alter the structure of plant-pollinator interactions and the services that these pollinators provide 3 . Hence, there is a growing focus on understanding trends in pollinator abundance and diversity, along with plant-pollinator interaction structures, in order to comprehend the drivers of change and guide management strategies (e.g., the EU Pollinators Initiative 4 ). Detecting trends requires standardised monitoring efforts over time and space. Traditional methods involve capturing pollinators and identifying them using microscopy 5 , 6 or DNA barcoding 7 . However, these methods are resource-intensive and require killing the pollinators. In this context, emerging technologies in machine learning, computer vision and portable microcomputers have the potential to automate the monitoring of pollination 8 and to do so in a non-lethal way 9 . Recent advancements in computer vision, particularly in deep convolutional neural networks (CNNs), have seen a surge in popularity. A notable aspect of this trend is the considerable effort developers have invested in documenting the use of such architectures, exemplified by code bases like Ultralytics 10 , Detectron2 11 or Pytorch-Wildlife 12 . This, coupled with ongoing improvements in sensors, camera traps, smartphones and programmable microcomputers equipped with graphics processing units (GPUs, e.g., Raspberry Pi 5 13 , Luxonis OAK modules 14 , NVIDIA Jetson Nano Developer Kit 15 , Coral Dev Board 16 , Qualcomm Snapdragon 17 ), has expanded the application of CNNs in wildlife monitoring 18 , 19 . These technologies are also increasingly being utilised in pollination monitoring 8 , 20 – 25 . CNN performance scales logarithmically with training dataset size 26 . However, these models typically show optimal generalisation primarily with data from imaging techniques similar to those used in training 27 , 28 . Their performance often drops when training and test data distributions differ 29 – 31 . This is less problematic if CNNs are applied to images closely resembling training data. For monitoring plant-pollinator interactions, cameras must be mounted above diverse flowers, inflorescences, or flower patches in varying field conditions. This presents a unique distribution shift challenge for CNNs trained for pollinator localisation and classification using images captured by citizen scientists 32 . Particularly, images from citizen-science platforms can exhibit bias, typically being well-lit and well-focused, with the subject usually centred and tightly framed 27 , 33 as contributors are encouraged to upload their best images, and to crop around the target organism to aid community identification 34 . While these images can be used for training classifiers, they may pose challenges for developing generalisable object detectors that can be used for autonomous cameras mounted above flowers in field conditions, which will capture relatively small pollinators against complex floral backgrounds and with little to no user intervention. CNN studies typically split an available image dataset into training, validation, and test sets, all sampled from the same distribution of images. In-distribution testing evaluates model performance on a test set drawn from this distribution. In contrast, OOD testing evaluates models on unseen images from the same domain (e.g., pollinator monitoring) but with a shifted distribution 28 , 35 , 36 . While model performance is often assessed using an in-distribution test set, OOD tests better reveal a model’s ability to adapt to a wider range of images, providing a tougher, more realistic measure of its learning and generalisation skills. For pollinating insects, images from citizen science platforms are an abundant source for training CNN models. We have shown, using an in-distribution test, that these models perform well in localising and classifying arthropods into broad groups, such as taxonomic orders 32 . We have also shown that a fixed setup using affordable smartphones, mounted on tripods above flowers and set to take time-lapse photos, can capture images of enough quality for experts to identify pollinators to these same broad groups and sometimes even to finer taxonomic levels 37 (family, genus, and species). However, it remains unknown how well CNNs trained on citizen science images will perform at localising and classifying pollinating insects in field images taken with a fixed smartphone setup. In this study, we evaluated the OOD generalisation capabilities of lightweight YOLO models (YOLOv5-nano, YOLOv5-small, and YOLOv7-tiny), trained and tested on curated citizen science images of flower-visiting arthropods 32 , which are typically well focused, cropped and centred on the target organisms. Generally, we assessed the efficacy of these models in localising and classifying pollinators captured in time-lapse sequences, comprising nearly 24,000 field images captured with a fixed smartphone setup. This OOD test set, where relatively smaller arthropods appear against unseen, complex floral backgrounds, presents a distribution shift from the training set. Specifically, we first evaluated the three models for class-agnostic arthropod localisation across all images captured with the fixed smartphone setup. The best-performing model, selected based on F1 score, was then analysed further. Given the rarity of flower visitors in time-lapse images, we tested the model’s false positive rate on a sample of floral-only background frames. Expecting arthropod bounding box area and image sharpness to affect performance, we compared their distributions between successful and unsuccessful localisation and classification outcomes. We assessed the best model’s ability to localise and classify individual pollinators across time-lapse sequences, a more relevant setting for pollination monitoring than independent frames. Diptera and Hymenoptera pollinators were the most common visitors in the dataset. We therefore assessed the model's ability to distinguish between three groups of flower visitors: Diptera, Hymenoptera, and OtherT (other taxa). As hoverflies (Syrphidae, Diptera) mimic bees and wasps (Hymenoptera), we tested whether misclassifications between these two orders were more common than those with other groups. Such mimicry can cause high-confidence mislabels, where the model confidently but incorrectly assigns the pollinator within the bounding box to the wrong group. In contrast, smaller or blurrier pollinators tend to lower model confidence. To investigate these dynamics, we compared the model’s confidence, bounding box size, and image sharpness between correctly and incorrectly classified cases, focusing on Hymenoptera and Diptera taxa most frequently misclassified as each other. Methods Dataset Time-lapse images of flower-visiting arthropods were automatically captured using smartphones from July to September 2021 in urban green spaces in and around Leipzig and Halle, Germany. The detailed methodology of data collection is provided in Ștefan et al. 37 Smartphones were positioned above selected open flowers of 33 plant species. The smartphones captured time-lapse images at an average rate of approximately one frame every 1.6 ± 0.4 seconds (mean ± s.d) for an average session duration of approximately one hour (3,553 ± 372 seconds, mean ± s.d.) on a targeted flower 37 . Following each time-lapse session, the smartphone was relocated to a different flower. For stable mounting, smartphones were secured on tripods and continuously powered through USB cables connected to power banks (e.g., Fig. 1 a). Image capture was facilitated by the OpenCamera app 38 . To ensure that the phone’s autofocus does not target the background instead of the flower, each recording session started with the focus fixed on the target flower and remained unadjusted until the end of the session. To mitigate wind-induced movements, flowers were anchored to wooden sticks with yarn. Smartphones were set 15–20 cm away from the centre of the target flower. More than 94% of images were taken at the resolution of 1600 x 1200 pixels. Automatic exposure adjustment was employed to adapt to changing lighting conditions. We visually parsed 213 distinct time-lapse sessions, each set against a unique floral background drawn from a selection of 33 different plant species, amassing a total of 460,056 time-lapsed images (see appendices in Ștefan et al. 37 ). Manual inspection of each image determined arthropod presence. When detected, a bounding box was drawn around the arthropod, and its taxonomic order was typed in using the VGG Image Annotator (VIA) software 39 . Because our focus was on monitoring pollinators on target flowers, a bounding box was placed around the target flower in each image containing an annotated arthropod, specifying the region of interest (ROI, Fig. 1 b). This also aligns with future research where we aim to develop custom cameras based on the technology proposed by Sittinger et al. 40 , that focus solely on target flowers, discarding noisy backgrounds that may contain out-of-focus flowers or cluttered patches of vegetation, which could confuse the models. 33,502 (7.28%) images contained at least one arthropod. These images resulted in 35,192 annotated arthropod bounding boxes. 94.8% of the images contained just a single arthropod bounding box, and a maximum of four bounding boxes were found in a single image. Our analysis focused exclusively on arthropod flower visitors interacting with the target flowers (which we refer to as pollinators even though flower visitors might not always perform pollination 41 ). This focus on visitors that might perform pollination is in line with our aim to contribute to advancing pollinator monitoring. We excluded any bounding boxes annotated with the Thysanoptera order (thrips), as well as 11 boxes for which the arthropod order could not be identified. Insects belonging to the Thysanoptera order were typically very small (around 1 mm or less) and slender, making them unlikely to be pollinators of our focal flowers and also unlikely to be detected by a CNN in our field settings. To focus on the ROI (i.e., the target flower), the original full-frame images were cropped (e.g., Fig. 1 b,c). This cropping was guided by the union of the bounding boxes for both the ROI and the visiting arthropod, ensuring that target arthropods at the edges of the ROI were not cut off. Following this cropping and filtering process, the refined OOD dataset comprised 201 time-lapse sessions on top of flowers from 32 plant species, 23,899 images, and 24,656 arthropod bounding boxes (Table 1 ). It should be noted that 182 of these bounding boxes contained co-occurring arthropods that, while within or intersecting the ROI, did not interact with the target flower and were removed from the model evaluation. The final cropped images had an average size of 851 pixels in width and 796 pixels in height. The original average dimensions were 1571 pixels wide and 1252 pixels high. The floral backgrounds in these images exhibited a long-tailed distribution, with 60.10% of arthropod bounding boxes (instances) located on flowers of just four plant species: Centaurea jacea (26.62%), Daucus carota (19.05%), Clematis vitalba (8.29%), and Carduus acanthoides (6.14%). A total of 1,281 unique arthropod individuals (each annotated as a series of bounding boxes across a time-lapse sequence of images) were identified in the OOD dataset, spanning six taxonomic groups: Hymenoptera (bees and wasps), Diptera (true flies), Coleoptera (beetles), Hymenoptera-Formicidae (ants), Araneae (spiders), and Hemiptera (true bugs), as detailed in Table 1 and shown in Fig. 2 . Pollinators from Hymenoptera (except ants) and Diptera orders were identified to the lowest taxonomic level possible during a previous study 37 . Given the time-lapse methodology of our image collection, an individual arthropod might be present in a solitary image or persist across multiple images (e.g., Fig. 5 ). In our OOD dataset, instances ranged from a single bounding box to a case where an individual arthropod remained on a flower long enough to be captured in 1,710 time-lapse images, thus resulting in a series of 1,710 bounding boxes. The median number of bounding boxes per arthropod individual was seven, indicating a typical visit duration of approximately 11.2 seconds captured in our dataset. Each arthropod visible across consecutive time-lapse frames received a unique identifier. Small individuals traversing a target flower’s complex structure and temporarily occluded by flower parts, retained the same identifier upon reappearance. While the training dataset had an average relative bounding box area of 0.337, the average in the OOD test set is 4.5 times smaller, at 0.075. Furthermore, the disparity in medians is more pronounced with the median for the OOD dataset at 0.028, which is over ten times smaller than that of the training dataset at 0.288. Table 1 Summary statistics for 1,281 arthropods in the OOD test set. The table enumerates counts of bounding boxes (N. box), their mean relative bounding box area (Mean rel. box area, proportions), counts of images (N. img.), and individual arthropods (N. ids.), alongside their respective percentages (N. ids. %) and the cumulative sum of these percentages (Cumul. sum %). Note that the sum of N. img. exceeds the total number of images in the OOD dataset due to the presence of multiple individuals from different categories in some images. Pollination Arthropod category N. box Mean rel. box area N. img. N. ids. N. ids. % Cumul. sum % Common pollinators Hymenoptera 13,254 0.107 13,084 1,013 79.08 79.08 Diptera 5,018 0.071 4,998 145 11.32 90.40 Other flower visitors (OtherT); usually not pollinating Coleoptera 2,778 0.010 2,770 20 1.56 91.96 Formicidae 1,967 0.013 1,962 82 6.41 98.36 Araneae 1,036 0.014 994 10 0.78 99.14 Hemiptera 603 0.011 603 11 0.86 100 Total 24,656 23,899 1,281 100 Model evaluation In our previous work 32 , we trained three YOLO object detection models, YOLOv5n (nano), YOLOv5s (small), and YOLOv7t (tiny), on a dataset of arthropod images primarily sourced from citizen science platforms. These models were evaluated using a traditional data split approach, where the test images were in-distribution, meaning they originated from the same source as the training images and shared similar characteristics. In contrast, the current study evaluates these pre-trained models on a novel OOD dataset. Unlike the in-distribution test set, which consists of curated images where photographers prioritise high-quality, detailed shots (sometimes using telephoto lenses), the OOD dataset comprises unscripted time-lapse images, modified only by cropping to define the ROI. Captured automatically using a fixed smartphone setup, these images depict arthropods visiting flowers and moving across their complex structures, sometimes becoming obscured by flower parts (e.g., Fig. 5 ), without the careful framing or optimised photography settings typical of curated citizen science photos. While citizen science images also capture arthropods from various angles, contributors typically select and upload images that ensure clear community identification, often favouring close-ups of smaller subjects. As a result, the OOD dataset provides a more challenging and ecologically realistic test for model evaluation, reflecting the variability inherent in automated pollinator monitoring, where images are captured passively, without real-time human selection or framing. The standard detection approach used by the tested YOLO models relies on a technique called non-maximum suppression (NMS). This method operates independently of the ground-truth bounding boxes and helps the detector to eliminate multiple, similar predictions for the same target object, aiming to produce a single, accurate prediction 50 . Specifically, it allows suppressing predicted boxes that substantially overlap with the predicted box having the highest confidence score (e.g., Supplementary Fig. S1 ). NMS, and therefore the detector’s performance, is sensitive to two parameters that impose a trade-off between precision and recall: NMS confidence score and NMS Intersection over Union (NMS-IoU). To optimise localisation performance, we fine-tuned the NMS parameters to maximise the F1 score (harmonic mean of precision and recall). Specifically, we ran detections on the OOD dataset using a near-zero (0.001) NMS confidence score and varied the NMS-IoU threshold from 0.1 to 0.9 in 0.1 steps. The low confidence threshold allowed a broad range of predictions to be considered, each with an associated confidence score. Post-detection, we classified predicted boxes as true positives (TP), false positives (FP), or false negatives (FN) based on an evaluation Intersection over Union (eval-IoU) threshold of 0.5, a standard value 51 also used in our previous work 32 . If detecting arthropod presence alone (rather than precise box alignment) is the priority, a lower eval-IoU threshold could be used (e.g., 0.1). Further, we computed precision, recall, F1 score across confidence scores (F1-confidence curve), and the area under the precision-recall curve (AUC, also known as average precision). We repeated this evaluation for each NMS-IoU threshold, using the maximum attainable F1 score (from the F1-confidence curves) to identify the best model and its optimal NMS parameters. At this step, the focus was arthropod localisation within single images, regardless of whether they belonged to a time-lapse sequence, and evaluated models in a class-agnostic manner, treating all detections as a single "arthropod" class. Inference on the OOD dataset was conducted at an image size of 640 x 640 pixels, consistent with the training image dimensions from our previous study 32 . Subsequently, we employed the optimised detector with the highest F1 score for inference on the OOD dataset, now evaluating predictions across all classes. At this step, we assessed the model's ability to both localise and classify the 1,281 individual arthropods. In this context, an individual arthropod was defined as a series of bounding boxes marked in successive images throughout the time-lapse sequence, which captured the arthropod's presence across multiple frames (e.g., Fig. 5 ). Consequently, in these cases, we will refer to the process as individual arthropod localisation or classification in subsequent discussions. Conversely, when discussing arthropod box localisation or classification , we are referring specifically to the best model’s ability to localise or classify an arthropod instances within a given image, regardless of the time-lapse sequence (that is, consecutive time-lapse images are considered independent from each other). The possible prediction labels for arthropod classification given by the pre-trained YOLO weights 32 were Araneae (spiders), Coleoptera (beetles), Diptera (true flies), Hemiptera (true bugs), Hymenoptera (bees and wasps), Hymenoptera Formicidae (ants), Lepidoptera (moths and butterflies), and Orthoptera (crickets and grasshoppers). Lepidoptera and Orthoptera do not appear in the OOD dataset. For analysis at the individual arthropod level, we used three groups: Hymenoptera, Diptera, and OtherT, comprising the remaining taxa groups. To evaluate model efficacy for the localisation of boxes with arthropods in independent frames, specific criteria were adopted (see also Supplementary Fig. S2): True Positive (box-TP): defined as a predicted bounding box that is adequately paired with a ground truth box, considering only those pairs for which the eval-IoU exceeds or equals a threshold (e.g., 0.5). When multiple such predicted boxes are present, the box with the highest YOLO confidence score is selected, irrespective of its predicted label. False Positive (box-FP): defined as a predicted bounding box that does not correspond to any ground truth box (model “hallucinates”), or when it does correspond but has an eval-IoU score below the threshold (insufficient overlap). Essentially this is an instance where the model inaccurately localises an arthropod (type I error). False Negative (box-FN): defined as a ground truth box without any corresponding predicted box (type II error). This situation occurs when the model fails to place any predicted bounding boxes in an image containing an arthropod or they are all box-FPs as defined above. Based on these, successful localisation of an individual arthropod across sequences (arthropod-TP) is achieved if at least one box-TP is encountered across the time-lapse sequence, regardless of the predicted labels (e.g., Fig. 5 ). The labels were disregarded for the purpose of the arthropod localisation task, in line with our objective to develop a single-class arthropod detector that will be deployed on an autonomous field camera similar to the work of Sittinger et al 40 . In pollination ecology, an arthropod-TP can be used as an indicator of a successful floral visit and classifying these visitors aids in assessing the diversity of pollinators visiting that specific flower. To evaluate the individual arthropod classification performance of the best detector, we employed a maximum confidence rule for label assignment across an entire time-lapse sequence. Specifically, when multiple predicted box-TPs across the sequence correspond to the same arthropod, the label with the highest YOLO confidence score was selected. Performance metrics including precision, recall, F1-score and accuracy were computed for each arthropod category and overall, weighted by the number of individuals. Additionally, we employed the best detector to assess false positives per image (FPPI) on the OOD images that only contained floral backgrounds. This detection test utilised 212 background images selected from the 213 distinct time-lapse sessions, with one session excluded because all images contained a beetle. FPPI is defined as the total number of FPs divided by the total number of images in the test set. We applied a one-tailed exact binomial test to assess whether cross-order Hymenoptera-Diptera misclassifications occurred at a frequency significantly higher than expected by chance, specifically testing for an excess over chance levels. For the independent frames analysis, where the YOLO model classified arthropod instances into eight groups (Araneae, Coleoptera, Diptera, Hemiptera, Hymenoptera, Hymenoptera-Formicidae, Lepidoptera, Orthoptera), each class had seven possible misclassifications, giving an expected probability of 1/7, 14.29%. For the individual arthropod analysis, where arthropods were observed across frames and grouped into Hymenoptera, Diptera, and OtherT, misclassifications had two possible outcomes (expected probability: 1/2, 50%). The test determined whether these misclassifications occurred significantly more often than expected (p < 0.05). To quantify image sharpness within bounding boxes, we applied the Sobel-Tenengrad operator as a proxy 52 . Higher values indicate more edges, signifying increased sharpness. Due to the large absolute values, we normalised them to a 0–1 range (blur to sharp) by dividing each by the maximum observed value. To assess differences in relative bounding box area and normalised image sharpness for localisation and classification tasks, we implemented a nonparametric permutation test. This test examines whether the means and medians of two distributions differ, assuming under the null hypothesis that the distributions are identical, with expected differences in these metrics being zero. We compared two groups: (1) ground truth arthropod boxes that were either localised or not, and (2) among localised instances, those correctly classified versus misclassified. We selected this test due to the long-tailed distributions, which deviate from normality. For each comparison, we reported the observed difference (Δ, absolute value) and the p-value relative to the 0.05 significance threshold. The p-value was computed as the proportion of permuted differences at least as extreme as the observed difference, with 1,000 permutations. Results In the initial class-agnostic test assessing arthropod box localisation within independent frames, YOLOv5-small outperformed YOLOv7-tiny and YOLOv5-nano (Fig. 3 , Supplementary Table S1 ). Grid search optimisation of YOLOv5-small estimated a maximum F1 score of 0.7019 and an AUC of 0.6497 at optimal NMS hyperparameters IoU = 0.3 (Fig. 3 a,c) and confidence score = 0.2019 (Fig. 3 b). This F1 optimisation also maximised AUC (Fig. 3 c,d). Performance remained stable until NMS-IoU exceeded 0.6, then declined (Fig. 3 a,c). In our prior study 32 with citizen science test images (similar to the training set), optimal NMS-IoU was 0.6 and confidence was 0.3. There, YOLOv5-small achieved a higher F1 score of 0.8886, followed by YOLOv7-tiny (0.8672) and YOLOv5-nano (0.8366), mirroring the current ranking. At eval-IoU 0.5, the model produced 2,265 false positive boxes (box-FPs) across the 23,899 OOD arthropod images, yielding a FPPI of 9.48%. At eval-IoU 0.1, box-FPs decreased to 1,799 (FPPI = 7.53%). In the control test on 212 floral background images (without arthropods), the model generated 16 box-FPs (FPPI = 7.55%), each occurring in a separate image. Smaller bounding boxes tended to contain blurrier arthropods (Spearman’s rank correlation ρ = 0.79, p < 0.05). Distributions of both box area and sharpness were long-tailed, with most arthropods appearing small and less sharp (Fig. 4 a-d). Of 24,656 ground truth boxes, 14,654 (59.4%) were localised (eval-IoU = 0.5), while 10,002 (40.6%) remained undetected. Correctly localised arthropods had significantly larger bounding boxes (Δ means = 0.0781, p < 0.05; Δ medians = 0.0672, p < 0.05) and higher image sharpness (Δ means = 0.0916, p < 0.05; Δ medians = 0.0729, p < 0.05) compared to those not localised (Fig. 4 a, b). Among localised arthropods, correctly classified instances also showed greater size (Δ means = 0.0304, p < 0.05; Δ medians = 0.0219, p < 0.05) and sharpness (Δ means = 0.0230, p < 0.05; Δ medians = 0.0133, p < 0.05) than misclassified ones (Fig. 4 c, d). For individual arthropod localisation within sequences, the optimised YOLOv5-small model achieved rates of 91.21% for Hymenoptera, 80.69% for Diptera, and 56.10% for OtherT flower visitors at eval-IoU 0.5. (Table 2 ). Lowering the eval-IoU to 0.1 slightly improved performance (Supplementary Table S2). A localisation example in sequential time-lapse images is shown in Fig. 5 . Classification recall was highest for Hymenoptera (R = 80.45%), followed by Diptera (R = 66.90%) and OtherT flower visitors (R = 47.97%), but accuracy ranked these groups in the opposite order (Table 2 ). Table 2 Performance metrics of the optimised YOLOv5-small model for individual arthropod localisation and classification at eval-IoU 0.5. An arthropod represents a sequence of bounding boxes in time-lapse images. Columns report total individuals (N. ind.), mean relative bounding box area (Rel. b.box area), mean normalised sharpness (Norm. sharp.), localised arthropods (N), localisation recall or rate (R), classification precision (P), recall (R), F1 score and accuracy (Acc.). Confusion matrix results in “Predictions” show percentages and counts for Hymenoptera (Hym.), Diptera (Dip.), other arthropods (OtherT), and background/false negatives (Bg./FN). Arthropod category N. ind. Rel. b.box area Norm. sharp. Localisation Classification Predictions - % and (counts) N R P R F1 Acc. Hym. Dip. OtherT Bg./FN Hymenoptera 1,013 0.1073 0.1020 924 0.9121 0.9772 0.8045 0.8825 0.8306 80.45% (815) 8.49% (86) 2.27% (23) 8.79% (89) Diptera 145 0.0708 0.1178 117 0.8069 0.5243 0.6690 0.5879 0.8938 7.59% (11) 66.90% (97) 6.21% (9) 19.31% (28) OtherT 123 0.0115 0.0321 69 0.5610 0.6484 0.4797 0.5514 0.9251 6.50% (8) 1.63% (2) 47.97% (59) 43.90% (54) Overall 1,281 0.0751 0.0871 1,110 0.8665 0.8944 0.7580 0.8205 0.8468 - - - - The model correctly classified 815/1,013 (80.45%) Hymenoptera and 97/145 (66.90%) Diptera individual arthropods. Notably 86 (8.49%) Hymenoptera were identified as Diptera and 11 (7.59%) Diptera as Hymenoptera (Table 2 ). This bidirectional Hymenoptera-Diptera misclassification was evident in independent frames, with 76.96% of all misclassified Hymenoptera instances (boxes) labelled as Diptera and 64.88% of misclassified Diptera as Hymenoptera, significantly exceeding chance (p < 0.05, exact binomial test, expected probability 1/7 = 14.29%, Supplementary Table S3). Of the 86 Hymenoptera individuals misclassified as Diptera, 43 were Apis mellifera (50.00%), 23 were red-tailed Bombus (26.74%), 8 were Halictidae (9.30%), and 8 were other Hymenopteran taxa (9.30%). The latter two groups (referred to as “non-mimicked”) are not targeted by Syrphidae mimicry. Apis mellifera , red-tailed Bombus , and Halictidae were common in the OOD dataset, collectively accounting for 56.47% (572/1,013) of Hymenoptera individuals, and thus we expected higher total misclassifications due to their higher total abundance in the dataset. However, from all misclassified Hymenoptera individuals (to Diptera or OtherT), the ones to Diptera exceeded chance (p < 0.05; exact binomial test; expected: 50% Diptera, 50% OtherT) for Apis mellifera (43/46, 93.48%) and red-tailed Bombus (23/26, 88.46%), but not for Halictidae (8/12, 66.67%) or Halictidae plus other non-mimicked taxa (“Non-mimicked”, 16/24, 66.67%; p > 0.05, see also Supplementary Table S4). For Apis mellifera , cases misclassified as Diptera showed no significant differences in means or medians for bounding box area, sharpness, or model confidence from correctly classified ones (p > 0.05). Red-tailed Bombus misclassifications had significantly smaller mean bounding box areas (p 0.05). In contrast, non-mimicked Hymenoptera misclassified as Diptera showed significantly higher confidence in correct classifications (p 0.05; Fig. 6 , Supplementary Table S4). Among the 11 Diptera that were misclassified as Hymenoptera, six were Syrphidae and five were individuals that could not be identified by experts to the family level from the image (referred to hereafter as coarsely identified Diptera). The proportion of misclassifications as Hymenoptera did not differ from chance for either Syrphidae (66.67%) or coarsely identified (45.45%) Diptera (p > 0.05). Syrphidae misclassified as Hymenoptera showed no significant differences in bounding box area or sharpness from correctly classified cases (p > 0.05). Conversely, the misclassifications for coarsely identified Diptera were significantly smaller and blurrier than the correct classifications (p < 0.05). In both Diptera groups, model confidence was significantly higher for correct classifications (p < 0.05; Fig. 6 , Supplementary Table S5). Discussion Our results show that the optimised YOLOv5-small model, trained on citizen science images, correctly localised 91.21% and classified 80.45% of Hymenoptera individuals, as well as localized 80.69% and classified 66.90% of Diptera individuals. Detection performance was weaker for other flower visitors (OtherT), which were typically smaller and blurrier. However, their higher accuracy (92.51%) shows the model mislabels Hymenoptera or Diptera as OtherT less frequently. To meet the demands of real-world pollinator monitoring, we chose lightweight models, as they promise energy-efficient deployment in field settings. Among those tested, YOLOv5-small, with the highest parameter count, outperformed others in F1 score, aligning with prior findings that greater model capacity (i.e., more trainable parameters) enhances performance 26 , 53 . Our previous study 32 noted a similar trend. Future work could explore higher-capacity architectures compatible with in situ hardware constraints. The grid search NMS optimisation, maximising the F1 score of arthropod detectors on the unseen OOD image dataset under complex field conditions, has practical implications for camera system design. For instance, adapting Sittinger et al.’s 40 setup for monitoring flower visitors could enhance on-device detection performance beyond default NMS values. This optimisation reflects dataset-specific tuning, as evidenced by comparing prior and current studies. In our earlier work with citizen science test images 32 , a higher NMS-IoU suited dense, overlapping bounding boxes of ants and bugs (e.g., images near ant colonies). Conversely, the OOD flower-visit dataset, dominated by images containing single arthropods, favoured a lower NMS-IoU, with performance declining at higher values (Fig. 3 a,c). A higher NMS-IoU threshold permits overlapping boxes, aiding detection of closely spaced arthropods, whereas a lower threshold enhances precision by minimising redundant predictions for solitary arthropods. Our pollinator localisation tests have practical implications, demonstrating the potential of object detection models trained on citizen science images to assist in annotating time-lapse field datasets, where most frames lack arthropods (e.g., over 90% 37,54,55 ). A single prediction per sequence enables annotators to target relevant frames, bypassing manual review of arthropod-free images. Manual annotation of a 460,056-image time-lapse dataset previously required approximately 1,000 hours 37 , whereas the YOLOv5-small model processed 23,899 OOD images in 419 seconds on an NVIDIA RTX A6000 GPU, suggesting around 2.24-hours runtime for the larger dataset, assuming fast image access. However, false positive (FP) rates on OOD images, including floral-only backgrounds, surpassed those on citizen science images, which more closely resemble the training set 32 . These additional FPs increase manual correction time, and we showed that lowering the eval-IoU threshold could enhance localisation and reduce FPs by allowing larger predicted boxes with sub-0.5 IoU (e.g., Fig. 5 ). Including floral backgrounds without pollinators in training may further reduce FP rates and improve precision. Another challenge is that smaller, less sharp arthropods are more likely to be missed. While the model effectively localised larger, common Hymenoptera and Diptera pollinators, it struggled with other flower visitors in the OOD dataset, which tended to be smaller and blurrier. After localisation, classifying flower visitors challenged the model more, with significant bidirectional Hymenoptera and Diptera misclassifications outnumbering those to other categories, alongside reduced performance for other arthropods. While it distinguished these categories effectively on in-distribution images 32 , this proficiency declined on the OOD dataset, where arthropods were on average 4.5 times smaller than in-distribution counterparts and sometimes occluded by flower parts (e.g., Fig. 5 ). This aligns with studies reporting reduced generalisation on organisms across new locations, time-frames, and sensors 27 , 31 , 54 , 56 – 58 , alongside pollinator-specific occlusion challenges 59 – 61 . Moreover, the pretrained models were not trained with more images of either Hymenoptera or Diptera than other categories, ruling out dataset bias as a cause of cross-order misclassifications. This is further supported by the fact that, despite Lepidoptera being nearly twice as abundant in the training data 32 , the model rarely mislabeled Hymenoptera or Diptera as Lepidoptera (e.g., Supplementary Table S3). Likewise, the higher accuracy for OtherT flower visitors shows the model less often mislabels Hymenoptera or Diptera as OtherT. Given these, Syrphidae mimicry most likely exacerbates the Hymenoptera-Diptera confusion, with syrphids like Eristalis spp. and Volucella bombylans resembling bees (e.g., Apis mellifera 62 ) and red-tailed Bombus (e.g., B. lapidarius , B. pratorum 63 ), respectively, mimicking their warning signals to deter predators. In the OOD dataset, larger or sharper arthropod instances exhibited significantly distinct distributions from smaller or blurrier counterparts for both localisation and classification. However, Apis mellifera and red-tailed Bombus , misclassified as Diptera, were as large and sharp as correctly classified cases, and the model was equally confident in these misclassifications most likely due to mimicry. In contrast, cross-order misclassified taxa not mimicked by Syrphidae (e.g., Halictidae, Cynipidae in Hymenoptera) and a few small, coarsely identified Diptera, had significantly higher model confidence in correct classifications. Their misclassified cases tended to be smaller and blurrier than correctly classified ones, likely explaining the mislabeling. Syrphidae misclassified as Hymenoptera were as large and sharp as correctly classified cases, but the model was significantly less confident in misclassifications. While these results might suggest that mimicry confuses the model more in one direction, with mimicked Hymenoptera more likely to be misclassified as Diptera than mimicking Syrphidae as Hymenoptera, we cannot say this conclusively due to the smaller sample size of Syrphidae individuals that were misclassified as Hymenoptera. To improve localisation and classification, we consider several steps for future research. First, integrating citizen-science and field images, as in recent studies 64 , 65 , would enhance model generalisation for real-world pollinator monitoring using time-lapse photography. Given that multiple studies have highlighted the scarcity of annotated field datasets for small arthropods, including pollinators 23 , 25 , 66 , 67 , our study addresses this gap by providing the OOD dataset (cropped and full-frame images) for training arthropod detectors for custom field cameras. At the same time, maintaining a clear separation between training and test sets is essential. Time-lapse image sequences can introduce a risk of data leakage 68 , 69 if highly similar frames are split between these sets, potentially inflating model performance. In such cases, the network may rely on shortcut learning 28 , recognising near-identical images based on superficial visual similarities (e.g., background patterns, nearly identical insect poses) rather than developing a truly generalisable understanding of arthropod features. To mitigate this, careful dataset partitioning is needed to prevent the model from exploiting temporal redundancies (e.g., highly similar consecutive frames depicting the same individual arthropod should be kept within a single set, either training, validation, or test, rather than split across them). Second, model performance could improve through a two-step approach, as suggested in other studies 40 , 54 , 58 , 64 , 70 . For example, an initial single-class object detector, such as YOLO 50 , could localise arthropods (e.g., arthropod vs. background), followed by a classifier to identify their cropped images at finer taxonomic levels. This approach allows the community to choose object detectors suited to their field hardware while leveraging diverse classification methods in post-processing, such as region-specific classifiers trained on continuously expanding datasets 71 , taxon-specific classifiers 72 , large multimodal models 73 , or hierarchical classification via custom classifier 64 , 74 , 75 and vision foundation models capable of learning hierarchical representations 76 . Furthermore, integrating object detection with segmentation has been shown to improve bumblebee species identification by removing noisy backgrounds and focusing classifiers on the most relevant features 77 . Additionally, citizen science platforms encourage users to upload cropped images of organisms 34 , providing a rich source of training data for such classifiers. Another advantage is the potential for multi-view classification 78 , leveraging sequential images of the same arthropod. Similar to how taxonomists examine multiple frames (e.g., Fig. 5 ) to improve identification despite occlusions or lower-quality frames, a multi-view CNN could refine predictions. In our study, we simplified this by assigning the label with the highest confidence score across a sequence, but a dedicated multi-view CNN could further enhance performance. Third, preprocessing time-lapse images to highlight arthropod features against the background 59 could enhance localisation if compatible with low-energy field cameras, or, if too energy-intensive, applied later on stored images rather than in real-time. Fourth, our results confirm arthropod size and image sharpness as important factors to localisation and classification, aligning with Nguyen et al.’s 66 , 79 findings on small-object detection challenges. The correlation between size and sharpness indicate also that arthropods further from the camera, or small arthropods in general, are most likely to be out of focus. Optimising image capture thus involves defining a region of interest and focusing on the target flower or inflorescence segment within, to maximise arthropod size in the frame. The region of interest can be defined via flower detection, segmentation, or pre-defined at the start of the recording session. Fixed focus is also crucial, and we adopted it when collecting the OOD dataset to prevent autofocus from shifting to background and blurring arthropods, as observed by Bjerge et al. 59 . Additionally, including blurred images in training datasets could further improve generalisation, as shown in larval fish detection 80 . Lastly, tiling full-frame images for detection could improve small-object localisation 81 , 82 by preserving details without downscaling to detector’s resolution. However, sliced inference like SAHI 81 increases computational demands on low-power field devices. While not our primary focus, our preliminary SAHI test with YOLOv5-small on the OOD dataset showed slight F1 gains, but increased false positives and processing time (Supplementary Table S6). Still, fine-tuning SAHI could aid annotation of high-resolution time-lapse datasets when real-time processing is not required. Implementing these proposed steps could enhance the detection of flower visitors, thereby facilitating the tracking of individual pollinators and enabling estimates of floral visit abundance, a key goal for automated pollinator monitoring. Examples of insect tracking can be found in recent studies 40 , 60 , 83 . Conclusion Our findings highlight the potential and limitations of lightweight YOLO detector models, trained on citizen science images, for localising and classifying flower visitors in out-of-distribution (OOD) time-lapse field images captured with a fixed smartphone setup. Localisation was generally effective for common Hymenoptera and Diptera pollinators, defined as cases where at least one bounding box in a time-lapse sequence was correctly placed. However, classification proved more challenging, impacted by arthropod size, image sharpness, and mimicry between Syrphidae (Diptera) and Hymenoptera. Smaller, blurrier arthropods, including less common flower visitors, were harder to detect, and the increase in false positives compared to in-distribution data revealed limitations when generalising to complex field conditions. These results have practical value for pollinator monitoring, showing potential for automating annotation of common Hymenoptera and Diptera pollinators in large time-lapse datasets, likely easing manual workloads. Future work could enhance performance by combining field and citizen science images in training, using a two-step detection-classification approach, optimising image capture to enhance arthropod size and sharpness, or adjusting NMS-IoU for specific ecological contexts. By providing an OOD dataset and identifying key challenges, this work contributes to the development of more robust machine learning tools for pollinator monitoring in natural environments. Declarations Data availability The image dataset related to this research is available at https://doi.org/10.5281/zenodo.15096609. The open-source code for the experiments is hosted on GitHub at https://github.com/valentinitnelav/smartphone-insect-detect. Acknowledgements The authors express gratitude to Bilyana Stoykova, Ricardo Urrego Alvarez, Emil Cyranka, and Anna Scheiper for their invaluable assistance with field work and meticulous efforts in manually annotating arthropods within the images. Special thanks are extended to Aspen Workman, Jared C. Cobain, and Demetra Rakosy for their expertise and contribution in the taxonomic identification of the pollinators studied. Additionally, many thanks go to Anne-Kathrin Thomas and Nina Becker for their comprehensive logistical support throughout this research. We gratefully acknowledge the use of hardware resources from the Leipzig University Computing Center and the German Center for Integrative Biodiversity Research (iDiv) Halle-Jena-Leipzig. We also extend our thanks to Justin Kay and Sara Beery, from whom we learned valuable object detector evaluation methods, and the entire team for organising the Computer Vision for Ecology Summer Workshop, supported by the Resnick Sustainability Institute. Last but not least, we are grateful to Maximilian Sittinger for thought-provoking discussions. Funding This research was funded by the Helmholtz AI initiative (Information & Data Science) Pollination Artificial Intelligence (ZT-I-PF-5-115), led by Prof. Tiffany M. Knight and Prof. Hannes Taubenboeck, the Helmholtz Recruitment Initiative of the Helmholtz Association to Tiffany M. Knight, and iDiv (German Research Foundation FZT 118). Author contributions V.S., T.S., M.W., H.T. and T.M.K. designed the scope of the study. V.S. and T.M.K. coordinated the creation and curation of the image dataset and its annotations. V.S. took the lead in designing and implementing the methods, conducting the analysis, and writing the manuscript. All authors reviewed the manuscript. Competing interests The authors declare no competing interests. References Potts, S. G. et al. Safeguarding pollinators and their values to human well-being. Nature 540 , 220–229 (2016). Potts, S. G. et al. Global pollinator declines: trends, impacts and drivers. Trends Ecol. Evol. 25 , 345–353 (2010). Artamendi, M., Martin, P. A., Bartomeus, I. & Magrach, A. Loss of pollinator diversity consistently reduces reproductive success for wild and cultivated plants. Nat. Ecol. Evol. 9 , 296–313 (2024). European Commission. EU Pollinators Initiative: Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52018DC0395 (2018). Motivans Švara, E. et al. Effects of different types of low‐intensity management on plant‐pollinator interactions in Estonian grasslands. Ecol. Evol. 11 , 16909–16926 (2021). Rakosy, D. et al. Intensive grazing alters the diversity, composition and structure of plant-pollinator interaction networks in Central European grasslands. PLOS ONE 17 , e0263576 (2022). Creedy, T. J. et al. A validated workflow for rapid taxonomic assignment and monitoring of a national fauna of bees (Apiformes) using high throughput DNA barcoding. Mol. Ecol. Resour. 20 , 40–53 (2020). van Klink, R. et al. Emerging technologies revolutionise insect ecology and monitoring. Trends Ecol. Evol. S0169534722001343 (2022) doi:10.1016/j.tree.2022.06.001. Montero‐Castaño, A. et al. Pursuing best practices for minimizing wild bee captures to support biological research. Conserv. Sci. Pract. 4 , e12734 (2022). Ultralytics. https://github.com/ultralytics/ultralytics. Detectron2. https://github.com/facebookresearch/detectron2. Hernandez, A. et al. Pytorch-Wildlife: A Collaborative Deep Learning Framework for Conservation. Preprint at http://arxiv.org/abs/2405.12930 (2024). Introducing: Raspberry Pi 5! https://www.raspberrypi.com/news/introducing-raspberry-pi-5/. Luxonis OAK modules. https://shop.luxonis.com/collections/oak-modules. Get Started With Jetson Nano Developer Kit. https://developer.nvidia.com/embedded/learn/get-started-jetson-nano-devkit. Coral Dev Board. https://coral.ai/docs/dev-board/get-started. Qualcomm Snapdragon. Wikipedia (2005). Tuia, D. et al. Perspectives in machine learning for wildlife conservation. Nat. Commun. 13 , 792 (2022). Jolles, J. W. Broad‐scale applications of the Raspberry Pi: A review and guide for biologists. Methods Ecol. Evol. 12 , 1562–1579 (2021). Martineau, M. et al. A survey on image-based insect classification. Pattern Recognit. 65 , 273–284 (2017). Barlow, S. E. & O’Neill, M. A. Technological advances in field studies of pollinator ecology and the future of e-ecology. Curr. Opin. Insect Sci. 38 , 15–25 (2020). Pegoraro, L., Hidalgo, O., Leitch, I. J., Pellicer, J. & Barlow, S. E. Automated video monitoring of insect pollinators in the field. Emerg. Top. Life Sci. 4 , 87–97 (2020). Høye, T. T. et al. Deep learning and computer vision will transform entomology. Proc. Natl. Acad. Sci. 118 , e2002545117 (2021). Amarathunga, D. C., Grundy, J., Parry, H. & Dorin, A. Methods of insect image capture and classification: A Systematic literature review. Smart Agric. Technol. 1 , 100023 (2021). Høye, T. T., Montagna, M., Oteman, B. & Roy, D. B. Emerging technologies for pollinator monitoring. Curr. Opin. Insect Sci. 101367 (2025) doi:10.1016/j.cois.2025.101367. Sun, C., Shrivastava, A., Singh, S. & Gupta, A. Revisiting unreasonable effectiveness of data in deep learning era. in Proceedings of the IEEE international conference on computer vision 843–852 (2017). Beery, S., Van Horn, G. & Perona, P. Recognition in Terra Incognita. in Proceedings of the European Conference on Computer Vision (ECCV) 456–473 (2018). Geirhos, R. et al. Shortcut learning in deep neural networks. Nat. Mach. Intell. 2 , 665–673 (2020). Kay, J. et al. Align and Distill: Unifying and Improving Domain Adaptive Object Detection. (2024) doi:10.48550/ARXIV.2403.12029. Oza, P., Sindagi, V. A., Sharmini, V. V. & Patel, V. M. Unsupervised Domain Adaptation of Object Detectors: A Survey. IEEE Trans. Pattern Anal. Mach. Intell. 1–24 (2023) doi:10.1109/TPAMI.2022.3217046. Koh, P. W. et al. WILDS: A Benchmark of in-the-Wild Distribution Shifts. (2021). Stark, T. et al. YOLO object detection models can locate and classify broad groups of flower-visiting arthropods in images. Sci. Rep. 13 , 16364 (2023). Elvekjaer, N. et al. Detecting flowers on imagery with computer vision to improve continental scale grassland biodiversity surveying. Ecol. Solut. Evid. 5 , e12324 (2024). Rankin, D. A plea to crop your photos! iNaturalist Community Forum https://forum.inaturalist.org/t/a-plea-to-crop-your-photos/35251 (2022). Hendrycks, D. et al. The many faces of robustness: A critical analysis of out-of-distribution generalization. in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) 8340–8349 (2021). Liu, J. et al. Towards Out-Of-Distribution Generalization: A Survey. (2021) doi:10.48550/ARXIV.2108.13624. Ștefan, V., Workman, A., Cobain, J. C., Rakosy, D. & Knight, T. M. Utilising affordable smartphones and open-source time-lapse photography for pollinator image collection and annotation. J. Pollinat. Ecol. 37 , 1–21 (2025). Harman, M. Open Camera. https://sourceforge.net/projects/opencamera/ (2023). Dutta, A. & Zisserman, A. The VIA Annotation Software for Images, Audio and Video. in Proceedings of the 27th ACM International Conference on Multimedia 2276–2279 (ACM, Nice France, 2019). doi:10.1145/3343031.3350535. Sittinger, M., Uhler, J., Pink, M. & Herz, A. Insect detect: An open-source DIY camera trap for automated insect monitoring. PLOS ONE 19 , e0295474 (2024). King, C., Ballantyne, G. & Willmer, P. G. Why flower visitation is a poor proxy for pollination: measuring single‐visit pollen deposition, with implications for pollination networks and conservation. Methods Ecol. Evol. 4 , 811–818 (2013). lovehiking. Araneae - Araniella alpica. https://inaturalist-open-data.s3.amazonaws.com/photos/84197357/original.jpeg. Arter. Coleoptera - Chrysolina brunsvicensis. https://arter.dk/media/0c15e627-4849-4346-acb1-ada5012d8d13.jpg. de Graaf, T. Diptera - Epistrophe grossulariae. https://observation.org/photos/759251.jpg. Bryukhov, V. Hemiptera - Handianus flavovarius. https://inaturalist-open-data.s3.amazonaws.com/photos/58813032/original.jpg. Jan, S. Hymenoptera - Andrena fulvago. https://observation.org/photos/6724813.jpg. David, R. Hymenoptera, Formicidae - Aphaenogaster spinosa. https://inaturalist-open-data.s3.amazonaws.com/photos/5966960/original.jpeg. Steve, D. Lepidoptera - Adela paludicolella. https://inaturalist-open-data.s3.amazonaws.com/photos/3882519/original.JPG?1464849720. Drepanostoma. Orthoptera- Nadigella formosanta. https://inaturalist-open-data.s3.amazonaws.com/photos/109734596/original.jpg?1610288615. Terven, J. & Cordova-Esparza, D. A Comprehensive Review of YOLO Architectures in Computer Vision: From YOLOv1 to YOLOv8 and YOLO-NAS. (2023) doi:10.48550/ARXIV.2304.00501. Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J. & Zisserman, A. The Pascal Visual Object Classes (VOC) Challenge. Int. J. Comput. Vis. 88 , 303–338 (2010). Pertuz, S., Puig, D. & Garcia, M. A. Analysis of focus measure operators for shape-from-focus. Pattern Recognit. 46 , 1415–1432 (2013). Wang, Y.-X., Ramanan, D. & Hebert, M. Growing a brain: Fine-tuning by increasing model capacity. in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2471–2480 (2017). Bjerge, K., Karstoft, H., Mann, H. M. R. & Høye, T. T. A deep learning pipeline for time-lapse camera monitoring of insects and their floral environments. Ecol. Inform. 84 , 102861 (2024). Ruczyński, I., Hałat, Z., Zegarek, M., Borowik, T. & Dechmann, D. K. N. Camera transects as a method to monitor high temporal and spatial ephemerality of flying nocturnal insects. Methods Ecol. Evol. 11 , 294–302 (2020). Kay, J. et al. The Caltech Fish Counting Dataset: A Benchmark for Multiple-Object Tracking and Counting. in Computer Vision – ECCV 2022 (eds. Avidan, S., Brostow, G., Cissé, M., Farinella, G. M. & Hassner, T.) vol. 13668 290–311 (Springer Nature Switzerland, Cham, 2022). Tabak, M. A. et al. Machine learning to classify animal species in camera trap images: Applications in ecology. Methods Ecol. Evol. 10 , 585–590 (2019). Norouzzadeh, M. S. et al. A deep active learning system for species identification and counting in camera trap images. Methods Ecol. Evol. 12 , 150–161 (2021). Bjerge, K., Frigaard, C. E. & Karstoft, H. Object Detection of Small Insects in Time-Lapse Camera Recordings. Sensors 23 , 7242 (2023). Ratnayake, M. N., Dyer, A. G. & Dorin, A. Tracking individual honeybees among wildflower clusters with computer vision-facilitated pollinator monitoring. Plos One 16 , e0239504 (2021). Risse, B., Mangan, M., Del Pero, L. & Webb, B. Visual tracking of small animals in cluttered natural environments using a freely moving camera. in Proceedings of the IEEE International Conference on Computer Vision (ICCV) Workshops 2840–2849 (2017). Penney, H. D., Hassall, C., Skevington, J. H., Lamborn, B. & Sherratt, T. N. The Relationship between Morphological and Behavioral Mimicry in Hover Flies (Diptera: Syrphidae). Am. Nat. 183 , 281–289 (2014). Edmunds, M. & Reader, T. Evidence for Batesian mimicry in a polymorphic hoverfly: evidence for Batesian mimicry. Evolution 68 , 827–839 (2014). Bjerge, K. et al. Hierarchical classification of insects with multitask learning and anomaly detection. Ecol. Inform. 77 , 102278 (2023). Shepley, A., Falzon, G., Meek, P. & Kwan, P. Automated location invariant animal detection in camera trap images using publicly available data sources. Ecol. Evol. 11 , 4494–4506 (2021). Nguyen, T. T. T. et al. SAWIT: A small-sized animal wild image dataset with annotations. Multimed. Tools Appl. 83 , 34083–34108 (2023). Jain, A. et al. Insect Identification in the Wild: The AMI Dataset. Preprint at https://doi.org/10.48550/ARXIV.2406.12452 (2024). Kapoor, S. & Narayanan, A. Leakage and the reproducibility crisis in machine-learning-based science. Patterns 4 , 100804 (2023). Bernett, J. et al. Guiding questions to avoid data leakage in biological machine learning applications. Nat. Methods 21 , 1444–1453 (2024). Beery, S., Morris, D. & Yang, S. Efficient Pipeline for Camera Trap Image Review. Preprint at http://arxiv.org/abs/1907.06772 (2019). Chiranjeevi, S. et al. InsectNet: Real-time identification of insects using an end-to-end machine learning pipeline. PNAS Nexus 4 , pgae575 (2024). Spiesman, B. J. et al. Assessing the potential for deep learning and computer vision to identify bumble bee species from images. Sci. Rep. 11 , 1–10 (2021). Sastry, S., Khanal, S., Dhakal, A., Ahmad, A. & Jacobs, N. TaxaBind: A Unified Embedding Space for Ecological Applications. Preprint at https://doi.org/10.48550/ARXIV.2411.00683 (2024). Badirli, S. et al. Classifying the unknown: Insect identification with deep hierarchical Bayesian learning. Methods Ecol. Evol. 14 , 1515–1530 (2023). Tresson, P., Carval, D., Tixier, P. & Puech, W. Hierarchical Classification of Very Small Objects: Application to the Detection of Arthropod Species. IEEE Access 9 , 63925–63932 (2021). Stevens, S. et al. BioCLIP: A Vision Foundation Model for the Tree of Life. Preprint at https://doi.org/10.48550/ARXIV.2311.18803 (2023). Choton, J. C., Margapuri, V., Grijalva, I., Spiesman, B. J. & Hsu, W. H. Self-supervised Component Segmentation To Improve Object Detection and Classification For Bumblebee Identification. Preprint at https://doi.org/10.1101/2025.03.12.642757 (2025). Seeland, M. & Mäder, P. Multi-view classification with convolutional neural networks. PLOS ONE 16 , e0245230 (2021). Nguyen, N.-D., Do, T., Ngo, T. D. & Le, D.-D. An Evaluation of Deep Learning Methods for Small Object Detection. J. Electr. Comput. Eng. 2020 , 1–18 (2020). Bar, S., Levy, L., Avidan, S. & Holzman, R. Assessing the determinants of larval fish strike rates using computer vision. Ecol. Inform. 77 , 102195 (2023). Akyon, F. C., Onur Altinuc, S. & Temizel, A. Slicing Aided Hyper Inference and Fine-Tuning for Small Object Detection. in 2022 IEEE International Conference on Image Processing (ICIP) 966–970 (IEEE, Bordeaux, France, 2022). doi:10.1109/ICIP46576.2022.9897990. Tresson, P., Tixier, P., Puech, W. & Carval, D. Insect interaction analysis based on object detection and CNN. in 2019 IEEE 21st International Workshop on Multimedia Signal Processing (MMSP) 1–6 (IEEE, Kuala Lumpur, Malaysia, 2019). doi:10.1109/MMSP.2019.8901798. Bjerge, K., Mann, H. M. R. & Høye, T. T. Real‐time insect tracking and monitoring with computer vision and deep learning. Remote Sens. Ecol. Conserv. 8 , 315–327 (2022). Additional Declarations No competing interests reported. Supplementary Files Stefanetal.SupplementaryInformation.docx Cite Share Download PDF Status: Published Journal Publication published 21 Aug, 2025 Read the published version in Scientific Reports → Version 1 posted Editorial decision: Revision requested 01 Jul, 2025 Reviews received at journal 09 Jun, 2025 Reviews received at journal 01 Jun, 2025 Reviewers agreed at journal 23 May, 2025 Reviewers agreed at journal 23 May, 2025 Reviewers invited by journal 12 May, 2025 Editor assigned by journal 12 May, 2025 Editor invited by journal 07 Apr, 2025 Submission checks completed at journal 05 Apr, 2025 First submitted to journal 29 Mar, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6335312","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":438815246,"identity":"fff2ff71-4c91-474f-98ea-4c39d1aef704","order_by":0,"name":"Valentin Ștefan","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA/0lEQVRIie3PsWrDMBCA4TMCZTFovWJoXsHFSweTZ5EQxFMKpRA6lGJjULZkzeNICNolpC/QwSbQ2VAogUJbJYGGDnIzZtA/CJ24bxBAKHSWkdIdeLjfA1xSEO6F9pHILfAdIQArgOxUAr/E7e+mHpI+m2pzu72+AfZiGv3wWswHUiFMcz9ZiTpbcrwDlCTVT28TFbeOrMdeclEKlcQcRYmEJh21E4WiTiNl/WTRzj73hNnBVn/Zgh7It5cwFIrsCUgKRlnuSNVESveQtk7isYMoMzRze+X+UjV8Lb2EMmne4/xRLJhpO/1hh8NZ0ehuOvKSo/0z8f9BKBQKhXr6AbmmU9ev6lzhAAAAAElFTkSuQmCC","orcid":"","institution":"Helmholtz Centre for Environmental Research - UFZ","correspondingAuthor":true,"prefix":"","firstName":"Valentin","middleName":"","lastName":"Ștefan","suffix":""},{"id":438815249,"identity":"9c385c1b-6a2c-42dc-af83-acf563dc955c","order_by":1,"name":"Thomas Stark","email":"","orcid":"","institution":"German Remote Sensing Data Center (DFD), German Aerospace Center (DLR)","correspondingAuthor":false,"prefix":"","firstName":"Thomas","middleName":"","lastName":"Stark","suffix":""},{"id":438815252,"identity":"d5d3674f-8868-4246-bab9-86b99a358b5d","order_by":2,"name":"Michael Wurm","email":"","orcid":"","institution":"German Remote Sensing Data Center (DFD), German Aerospace Center (DLR)","correspondingAuthor":false,"prefix":"","firstName":"Michael","middleName":"","lastName":"Wurm","suffix":""},{"id":438815257,"identity":"383f164c-5d14-48b1-b99a-de8643be085c","order_by":3,"name":"Hannes Taubenböck","email":"","orcid":"","institution":"German Remote Sensing Data Center (DFD), German Aerospace Center (DLR)","correspondingAuthor":false,"prefix":"","firstName":"Hannes","middleName":"","lastName":"Taubenböck","suffix":""},{"id":438815260,"identity":"fa8389b3-363e-4565-9a66-35830b792594","order_by":4,"name":"Tiffany M. Knight","email":"","orcid":"","institution":"National Tropical Botanical Garden","correspondingAuthor":false,"prefix":"","firstName":"Tiffany","middleName":"M.","lastName":"Knight","suffix":""}],"badges":[],"createdAt":"2025-03-29 17:08:17","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-6335312/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6335312/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1038/s41598-025-16140-z","type":"published","date":"2025-08-21T16:29:37+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":80048431,"identity":"e7098066-c001-46d3-9393-518e6ec01c9d","added_by":"auto","created_at":"2025-04-07 10:03:52","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":369904,"visible":true,"origin":"","legend":"\u003cp\u003e(a) Setup for time-lapse image capture featuring a smartphone (1) on a tripod (2) above the target flower (3), supported by a stick (4) to reduce wind motion and connected to a power bank (5) for continuous operation. (b) An original, full-frame image from the smartphone showing a pollinator and the target flower. (c) A cropped image highlighting the region of interest (ROI) used for analysis.\u003c/p\u003e","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-6335312/v1/bcfcf5b000134cd0b3c7ce8b.png"},{"id":80048432,"identity":"1f8d1b44-0c92-47dc-b91c-18d9e622ca24","added_by":"auto","created_at":"2025-04-07 10:03:52","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":728103,"visible":true,"origin":"","legend":"\u003cp\u003eExample of cropped smartphone-captured images representing the six groups of flower visitors in our out-of-distribution (OOD) test dataset (a to f) vs. the in-distribution training dataset used in Stark et al.\u003ca href=\"https://www.zotero.org/google-docs/?TTVJ2H\"\u003e\u003csup\u003e32\u003c/sup\u003e\u003c/a\u003e (g to n). Taxonomic orders in OOD test and train datasets: Arenae (a, g\u003ca href=\"https://www.zotero.org/google-docs/?HY81ta\"\u003e\u003csup\u003e42\u003c/sup\u003e\u003c/a\u003e), Coleoptera (b, h\u003ca href=\"https://www.zotero.org/google-docs/?5N4KPQ\"\u003e\u003csup\u003e43\u003c/sup\u003e\u003c/a\u003e), Diptera (c, i\u003ca href=\"https://www.zotero.org/google-docs/?ZdbSkU\"\u003e\u003csup\u003e44\u003c/sup\u003e\u003c/a\u003e), Hemiptera (d, j\u003ca href=\"https://www.zotero.org/google-docs/?j3nSLC\"\u003e\u003csup\u003e45\u003c/sup\u003e\u003c/a\u003e), Hymenoptera (e, k\u003ca href=\"https://www.zotero.org/google-docs/?up79ob\"\u003e\u003csup\u003e46\u003c/sup\u003e\u003c/a\u003e), Hymenoptera-Formicidae (f, l\u003ca href=\"https://www.zotero.org/google-docs/?VBPps6\"\u003e\u003csup\u003e47\u003c/sup\u003e\u003c/a\u003e). Lepidoptera (m\u003ca href=\"https://www.zotero.org/google-docs/?XTbz7t\"\u003e\u003csup\u003e48\u003c/sup\u003e\u003c/a\u003e) and Orthoptera (n\u003ca href=\"https://www.zotero.org/google-docs/?9Qon9y\"\u003e\u003csup\u003e49\u003c/sup\u003e\u003c/a\u003e) are exclusive to the training dataset. The average bounding box area in the OOD test set is approximately 4.5 times less than in the training dataset. The image backgrounds in the training dataset are more diverse, whereas the OOD test dataset features exclusively floral backgrounds.\u003c/p\u003e","description":"","filename":"floatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-6335312/v1/ca7674597412bd20d83edf01.png"},{"id":80047549,"identity":"595d4d29-4905-4eef-994d-1d1ccd34c6b6","added_by":"auto","created_at":"2025-04-07 09:55:52","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":288976,"visible":true,"origin":"","legend":"\u003cp\u003eGrid search results for the optimal NMS confidence and NMS-IoU hyperparameters for YOLO detectors (localisation task, independent frames), with a focus on the maximum F1 score (panels a and b) and area under the precision-recall curve (AUC, panels c and d). The YOLOv5-small model demonstrates superior performance (highest F1 and AUC), achieving optimal detection at an NMS confidence estimate of 0.2019 (panel b) and a NMS-IoU of 0.3 (panels a and c), marked with grey dotted vertical lines. The presented F1-confidence curve (panel b) and the precision-recall curve (panel d) correspond to the optimal NMS-IoU for each model. The evaluation was performed using an eval-IoU of 0.5.\u003c/p\u003e","description":"","filename":"floatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-6335312/v1/0b922bb63669fd4bf3e77bd6.png"},{"id":80048433,"identity":"79751e2d-02d9-440e-9469-03a000005c94","added_by":"auto","created_at":"2025-04-07 10:03:52","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":341871,"visible":true,"origin":"","legend":"\u003cp\u003eBoxplots displaying the distributions and interquartile ranges for relative bounding box area and normalised image sharpness (within the bounding box), categorised by successful localisation (eval-IoU = 0.5) and classification status ('no' vs. 'yes'), across all arthropod categories in independent frames.\u003c/p\u003e","description":"","filename":"floatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-6335312/v1/231f9d3fdd855fea234bd672.png"},{"id":80049454,"identity":"549785ba-f0a4-4995-8d6a-febad6367e1a","added_by":"auto","created_at":"2025-04-07 10:11:52","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":439316,"visible":true,"origin":"","legend":"\u003cp\u003eExample of arthropod presence across sequential time-lapse images, demonstrating overall sequence localisation even when partially obscured by flower parts (e.g., panels c–e). Localisation is considered successful when at least one ground truth box (orange) in the sequence achieves an IoU ≥ eval-IoU (0.5) with a predicted box (cyan), regardless of classification. Panels a, b, and f show correctly labelled Hymenoptera predictions, YOLO confidence scores (Conf.), and IoU values between ground truth and predictions. At eval-IoU 0.5, the predicted box in panel b is a false positive, but at eval-IoU 0.1, it is a true positive. The time stamp (bottom right corner of each panel) is provided in hh:mm:ss format.\u003c/p\u003e","description":"","filename":"floatimage5.png","url":"https://assets-eu.researchsquare.com/files/rs-6335312/v1/5a8dfae2e219b5861248c4a5.png"},{"id":80047560,"identity":"501ea46c-5c69-496e-8045-29f54531dcb6","added_by":"auto","created_at":"2025-04-07 09:55:53","extension":"png","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":522183,"visible":true,"origin":"","legend":"\u003cp\u003eBoxplots showing distributions of relative bounding-box area, normalised image sharpness (within the bounding box), and model confidence score (YOLOv5-small) for pollinator taxa in Hymenoptera and Diptera orders, grouped by classification outcome (‘yes’ = correctly classified, ‘no’ = misclassified as the other order). Means are indicated by large diamond symbols. Syrphidae (Diptera) are known to mimic Hymenoptera such as \u003cem\u003eApis mellifera\u003c/em\u003e and red-tailed \u003cem\u003eBombus\u003c/em\u003e. For a detailed list of taxa in “Non-mimicked” and “Coarsely identified” (no family level ID) groups, see Supplementary Tables S4 and S5.\u003c/p\u003e","description":"","filename":"floatimage6.png","url":"https://assets-eu.researchsquare.com/files/rs-6335312/v1/f39ac176434ace165e93c9b3.png"},{"id":89847257,"identity":"0651fd00-4813-472e-9080-7c11784654f4","added_by":"auto","created_at":"2025-08-25 16:42:38","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":3468136,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6335312/v1/5da3cf4c-7a38-4b56-87a1-6c5db2fd25c1.pdf"},{"id":80047551,"identity":"52f4e479-cc43-4f59-a07d-fb15ff860fd3","added_by":"auto","created_at":"2025-04-07 09:55:52","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":1277733,"visible":true,"origin":"","legend":"","description":"","filename":"Stefanetal.SupplementaryInformation.docx","url":"https://assets-eu.researchsquare.com/files/rs-6335312/v1/f7f3d122e5de600fdc2d7c19.docx"}],"financialInterests":"No competing interests reported.","formattedTitle":"Successes and limitations of pretrained YOLO detectors applied to unseen time-lapse images for automated pollinator monitoring","fulltext":[{"header":"Introduction","content":"\u003cp\u003ePollinators play a crucial role in sustaining our ecosystems and ensuring food security. Yet they face an alarming decline\u003csup\u003e\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e,\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e\u003c/sup\u003e, which has the potential to alter the structure of plant-pollinator interactions and the services that these pollinators provide\u003csup\u003e\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e\u003c/sup\u003e. Hence, there is a growing focus on understanding trends in pollinator abundance and diversity, along with plant-pollinator interaction structures, in order to comprehend the drivers of change and guide management strategies (e.g., the EU Pollinators Initiative\u003csup\u003e\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e\u003c/sup\u003e). Detecting trends requires standardised monitoring efforts over time and space. Traditional methods involve capturing pollinators and identifying them using microscopy\u003csup\u003e\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e,\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e\u003c/sup\u003e or DNA barcoding\u003csup\u003e\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e\u003c/sup\u003e. However, these methods are resource-intensive and require killing the pollinators. In this context, emerging technologies in machine learning, computer vision and portable microcomputers have the potential to automate the monitoring of pollination\u003csup\u003e\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e\u003c/sup\u003e and to do so in a non-lethal way\u003csup\u003e\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e\u003c/sup\u003e.\u003c/p\u003e \u003cp\u003eRecent advancements in computer vision, particularly in deep convolutional neural networks (CNNs), have seen a surge in popularity. A notable aspect of this trend is the considerable effort developers have invested in documenting the use of such architectures, exemplified by code bases like \u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003eUltralytics\u003c/span\u003e\u003csup\u003e\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e\u003c/sup\u003e, \u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003eDetectron2\u003c/span\u003e\u003csup\u003e11\u003c/sup\u003e or \u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003ePytorch-Wildlife\u003c/span\u003e\u003csup\u003e\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e\u003c/sup\u003e. This, coupled with ongoing improvements in sensors, camera traps, smartphones and programmable microcomputers equipped with graphics processing units (GPUs, e.g., \u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003eRaspberry Pi 5\u003c/span\u003e\u003csup\u003e13\u003c/sup\u003e, \u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003eLuxonis OAK modules\u003c/span\u003e\u003csup\u003e\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e\u003c/sup\u003e, \u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003eNVIDIA Jetson Nano Developer Kit\u003c/span\u003e\u003csup\u003e\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e\u003c/sup\u003e, \u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003eCoral Dev Board\u003c/span\u003e\u003csup\u003e\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e\u003c/sup\u003e, \u003cspan type=\"Underline\" class=\"Underline\" name=\"Emphasis\"\u003eQualcomm Snapdragon\u003c/span\u003e\u003csup\u003e\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e\u003c/sup\u003e), has expanded the application of CNNs in wildlife monitoring\u003csup\u003e\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e,\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e\u003c/sup\u003e. These technologies are also increasingly being utilised in pollination monitoring\u003csup\u003e\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e,\u003cspan additionalcitationids=\"CR21 CR22 CR23 CR24\" citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e–\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e\u003c/sup\u003e.\u003c/p\u003e \u003cp\u003eCNN performance scales logarithmically with training dataset size\u003csup\u003e\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e\u003c/sup\u003e. However, these models typically show optimal generalisation primarily with data from imaging techniques similar to those used in training\u003csup\u003e\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e,\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e\u003c/sup\u003e. Their performance often drops when training and test data distributions differ\u003csup\u003e\u003cspan additionalcitationids=\"CR30\" citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e–\u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e\u003c/sup\u003e. This is less problematic if CNNs are applied to images closely resembling training data. For monitoring plant-pollinator interactions, cameras must be mounted above diverse flowers, inflorescences, or flower patches in varying field conditions. This presents a unique distribution shift challenge for CNNs trained for pollinator localisation and classification using images captured by citizen scientists\u003csup\u003e\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e\u003c/sup\u003e. Particularly, images from citizen-science platforms can exhibit bias, typically being well-lit and well-focused, with the subject usually centred and tightly framed\u003csup\u003e\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e,\u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e\u003c/sup\u003e as contributors are encouraged to upload their best images, and to crop around the target organism to aid community identification\u003csup\u003e\u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e\u003c/sup\u003e. While these images can be used for training classifiers, they may pose challenges for developing generalisable object detectors that can be used for autonomous cameras mounted above flowers in field conditions, which will capture relatively small pollinators against complex floral backgrounds and with little to no user intervention.\u003c/p\u003e \u003cp\u003eCNN studies typically split an available image dataset into training, validation, and test sets, all sampled from the same distribution of images. In-distribution testing evaluates model performance on a test set drawn from this distribution. In contrast, OOD testing evaluates models on unseen images from the same domain (e.g., pollinator monitoring) but with a shifted distribution\u003csup\u003e\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e,\u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e35\u003c/span\u003e,\u003cspan citationid=\"CR36\" class=\"CitationRef\"\u003e36\u003c/span\u003e\u003c/sup\u003e. While model performance is often assessed using an in-distribution test set, OOD tests better reveal a model’s ability to adapt to a wider range of images, providing a tougher, more realistic measure of its learning and generalisation skills.\u003c/p\u003e \u003cp\u003eFor pollinating insects, images from citizen science platforms are an abundant source for training CNN models. We have shown, using an in-distribution test, that these models perform well in localising and classifying arthropods into broad groups, such as taxonomic orders\u003csup\u003e\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e\u003c/sup\u003e. We have also shown that a fixed setup using affordable smartphones, mounted on tripods above flowers and set to take time-lapse photos, can capture images of enough quality for experts to identify pollinators to these same broad groups and sometimes even to finer taxonomic levels\u003csup\u003e\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e\u003c/sup\u003e (family, genus, and species). However, it remains unknown how well CNNs trained on citizen science images will perform at localising and classifying pollinating insects in field images taken with a fixed smartphone setup.\u003c/p\u003e \u003cp\u003eIn this study, we evaluated the OOD generalisation capabilities of lightweight YOLO models (YOLOv5-nano, YOLOv5-small, and YOLOv7-tiny), trained and tested on curated citizen science images of flower-visiting arthropods\u003csup\u003e\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e\u003c/sup\u003e, which are typically well focused, cropped and centred on the target organisms. Generally, we assessed the efficacy of these models in localising and classifying pollinators captured in time-lapse sequences, comprising nearly 24,000 field images captured with a fixed smartphone setup. This OOD test set, where relatively smaller arthropods appear against unseen, complex floral backgrounds, presents a distribution shift from the training set.\u003c/p\u003e \u003cp\u003eSpecifically, we first evaluated the three models for class-agnostic arthropod localisation across all images captured with the fixed smartphone setup. The best-performing model, selected based on F1 score, was then analysed further. Given the rarity of flower visitors in time-lapse images, we tested the model’s false positive rate on a sample of floral-only background frames. Expecting arthropod bounding box area and image sharpness to affect performance, we compared their distributions between successful and unsuccessful localisation and classification outcomes. We assessed the best model’s ability to localise and classify individual pollinators across time-lapse sequences, a more relevant setting for pollination monitoring than independent frames. Diptera and Hymenoptera pollinators were the most common visitors in the dataset. We therefore assessed the model's ability to distinguish between three groups of flower visitors: Diptera, Hymenoptera, and OtherT (other taxa). As hoverflies (Syrphidae, Diptera) mimic bees and wasps (Hymenoptera), we tested whether misclassifications between these two orders were more common than those with other groups. Such mimicry can cause high-confidence mislabels, where the model confidently but incorrectly assigns the pollinator within the bounding box to the wrong group. In contrast, smaller or blurrier pollinators tend to lower model confidence. To investigate these dynamics, we compared the model’s confidence, bounding box size, and image sharpness between correctly and incorrectly classified cases, focusing on Hymenoptera and Diptera taxa most frequently misclassified as each other.\u003c/p\u003e "},{"header":"Methods","content":"\u003cp\u003eDataset\u003c/p\u003e\u003cp\u003eTime-lapse images of flower-visiting arthropods were automatically captured using smartphones from July to September 2021 in urban green spaces in and around Leipzig and Halle, Germany. The detailed methodology of data collection is provided in Ștefan et al.\u003csup\u003e\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e\u003c/sup\u003e Smartphones were positioned above selected open flowers of 33 plant species. The smartphones captured time-lapse images at an average rate of approximately one frame every 1.6 ± 0.4 seconds (mean ± s.d) for an average session duration of approximately one hour (3,553 ± 372 seconds, mean ± s.d.) on a targeted flower\u003csup\u003e\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e\u003c/sup\u003e. Following each time-lapse session, the smartphone was relocated to a different flower.\u003c/p\u003e\u003cp\u003eFor stable mounting, smartphones were secured on tripods and continuously powered through USB cables connected to power banks (e.g., Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003ea). Image capture was facilitated by the OpenCamera app\u003csup\u003e\u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e\u003c/sup\u003e. To ensure that the phone’s autofocus does not target the background instead of the flower, each recording session started with the focus fixed on the target flower and remained unadjusted until the end of the session. To mitigate wind-induced movements, flowers were anchored to wooden sticks with yarn. Smartphones were set 15–20 cm away from the centre of the target flower. More than 94% of images were taken at the resolution of 1600 x 1200 pixels. Automatic exposure adjustment was employed to adapt to changing lighting conditions.\u003c/p\u003e\u003cp\u003eWe visually parsed 213 distinct time-lapse sessions, each set against a unique floral background drawn from a selection of 33 different plant species, amassing a total of 460,056 time-lapsed images (see appendices in Ștefan et al.\u003csup\u003e\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e\u003c/sup\u003e). Manual inspection of each image determined arthropod presence. When detected, a bounding box was drawn around the arthropod, and its taxonomic order was typed in using the VGG Image Annotator (VIA) software\u003csup\u003e\u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e39\u003c/span\u003e\u003c/sup\u003e. Because our focus was on monitoring pollinators on target flowers, a bounding box was placed around the target flower in each image containing an annotated arthropod, specifying the region of interest (ROI, Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003eb). This also aligns with future research where we aim to develop custom cameras based on the technology proposed by Sittinger et al.\u003csup\u003e\u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e\u003c/sup\u003e, that focus solely on target flowers, discarding noisy backgrounds that may contain out-of-focus flowers or cluttered patches of vegetation, which could confuse the models. 33,502 (7.28%) images contained at least one arthropod. These images resulted in 35,192 annotated arthropod bounding boxes. 94.8% of the images contained just a single arthropod bounding box, and a maximum of four bounding boxes were found in a single image.\u003c/p\u003e\u003cp\u003eOur analysis focused exclusively on arthropod flower visitors interacting with the target flowers (which we refer to as pollinators even though flower visitors might not always perform pollination\u003csup\u003e\u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e41\u003c/span\u003e\u003c/sup\u003e). This focus on visitors that might perform pollination is in line with our aim to contribute to advancing pollinator monitoring. We excluded any bounding boxes annotated with the Thysanoptera order (thrips), as well as 11 boxes for which the arthropod order could not be identified. Insects belonging to the Thysanoptera order were typically very small (around 1 mm or less) and slender, making them unlikely to be pollinators of our focal flowers and also unlikely to be detected by a CNN in our field settings.\u003c/p\u003e\u003cp\u003eTo focus on the ROI (i.e., the target flower), the original full-frame images were cropped (e.g., Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003eb,c). This cropping was guided by the union of the bounding boxes for both the ROI and the visiting arthropod, ensuring that target arthropods at the edges of the ROI were not cut off. Following this cropping and filtering process, the refined OOD dataset comprised 201 time-lapse sessions on top of flowers from 32 plant species, 23,899 images, and 24,656 arthropod bounding boxes (Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e). It should be noted that 182 of these bounding boxes contained co-occurring arthropods that, while within or intersecting the ROI, did not interact with the target flower and were removed from the model evaluation. The final cropped images had an average size of 851 pixels in width and 796 pixels in height. The original average dimensions were 1571 pixels wide and 1252 pixels high. The floral backgrounds in these images exhibited a long-tailed distribution, with 60.10% of arthropod bounding boxes (instances) located on flowers of just four plant species: \u003cem\u003eCentaurea jacea\u003c/em\u003e (26.62%), \u003cem\u003eDaucus carota\u003c/em\u003e (19.05%), \u003cem\u003eClematis vitalba\u003c/em\u003e (8.29%), and \u003cem\u003eCarduus acanthoides\u003c/em\u003e (6.14%).\u003c/p\u003e\u003cp\u003eA total of 1,281 unique arthropod individuals (each annotated as a series of bounding boxes across a time-lapse sequence of images) were identified in the OOD dataset, spanning six taxonomic groups: Hymenoptera (bees and wasps), Diptera (true flies), Coleoptera (beetles), Hymenoptera-Formicidae (ants), Araneae (spiders), and Hemiptera (true bugs), as detailed in Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e and shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e. Pollinators from Hymenoptera (except ants) and Diptera orders were identified to the lowest taxonomic level possible during a previous study\u003csup\u003e\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e\u003c/sup\u003e. Given the time-lapse methodology of our image collection, an individual arthropod might be present in a solitary image or persist across multiple images (e.g., Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003e). In our OOD dataset, instances ranged from a single bounding box to a case where an individual arthropod remained on a flower long enough to be captured in 1,710 time-lapse images, thus resulting in a series of 1,710 bounding boxes. The median number of bounding boxes per arthropod individual was seven, indicating a typical visit duration of approximately 11.2 seconds captured in our dataset. Each arthropod visible across consecutive time-lapse frames received a unique identifier. Small individuals traversing a target flower’s complex structure and temporarily occluded by flower parts, retained the same identifier upon reappearance.\u003c/p\u003e\u003cp\u003eWhile the training dataset had an average relative bounding box area of 0.337, the average in the OOD test set is 4.5 times smaller, at 0.075. Furthermore, the disparity in medians is more pronounced with the median for the OOD dataset at 0.028, which is over ten times smaller than that of the training dataset at 0.288.\u003c/p\u003e\u003cdiv class=\"gridtable\"\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e\u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e\u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e\u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e\u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c8\" colnum=\"8\"\u003e\u003c/div\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e\u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eSummary statistics for 1,281 arthropods in the OOD test set. The table enumerates counts of bounding boxes (N. box), their mean relative bounding box area (Mean rel. box area, proportions), counts of images (N. img.), and individual arthropods (N. ids.), alongside their respective percentages (N. ids. %) and the cumulative sum of these percentages (Cumul. sum %). Note that the sum of N. img. exceeds the total number of images in the OOD dataset due to the presence of multiple individuals from different categories in some images.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e\u003ccolgroup cols=\"8\"\u003e\u003c/colgroup\u003e\u003cthead\u003e\u003ctr\u003e\u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePollination\u003c/p\u003e \u003c/th\u003e\u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eArthropod category\u003c/p\u003e \u003c/th\u003e\u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eN. box\u003c/p\u003e \u003c/th\u003e\u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eMean rel. box area\u003c/p\u003e \u003c/th\u003e\u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eN. img.\u003c/p\u003e \u003c/th\u003e\u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eN. ids.\u003c/p\u003e \u003c/th\u003e\u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003eN. ids. %\u003c/p\u003e \u003c/th\u003e\u003cth align=\"left\" colname=\"c8\"\u003e \u003cp\u003eCumul. sum %\u003c/p\u003e \u003c/th\u003e\u003c/tr\u003e\u003c/thead\u003e\u003ctbody\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eCommon pollinators\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eHymenoptera\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e13,254\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.107\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e13,084\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e1,013\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e79.08\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e79.08\u003c/p\u003e \u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDiptera\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e5,018\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.071\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e4,998\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e145\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e11.32\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e90.40\u003c/p\u003e \u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\" morerows=\"3\" rowspan=\"4\"\u003e \u003cp\u003eOther flower visitors (OtherT); usually not pollinating\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eColeoptera\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e2,778\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.010\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e2,770\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e20\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e1.56\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e91.96\u003c/p\u003e \u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eFormicidae\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e1,967\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.013\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e1,962\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e82\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e6.41\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e98.36\u003c/p\u003e \u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAraneae\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e1,036\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.014\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e994\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e10\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e0.78\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e99.14\u003c/p\u003e \u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eHemiptera\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e603\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.011\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e603\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e11\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e0.86\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eTotal\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e24,656\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e23,899\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e1,281\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e\u0026nbsp;\u003c/td\u003e\u003c/tr\u003e\u003c/tbody\u003e\u003c/table\u003e\u003c/div\u003e\u003cp\u003eModel evaluation\u003c/p\u003e\u003cp\u003eIn our previous work\u003csup\u003e\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e\u003c/sup\u003e, we trained three YOLO object detection models, YOLOv5n (nano), YOLOv5s (small), and YOLOv7t (tiny), on a dataset of arthropod images primarily sourced from citizen science platforms. These models were evaluated using a traditional data split approach, where the test images were in-distribution, meaning they originated from the same source as the training images and shared similar characteristics. In contrast, the current study evaluates these pre-trained models on a novel OOD dataset. Unlike the in-distribution test set, which consists of curated images where photographers prioritise high-quality, detailed shots (sometimes using telephoto lenses), the OOD dataset comprises unscripted time-lapse images, modified only by cropping to define the ROI. Captured automatically using a fixed smartphone setup, these images depict arthropods visiting flowers and moving across their complex structures, sometimes becoming obscured by flower parts (e.g., Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003e), without the careful framing or optimised photography settings typical of curated citizen science photos. While citizen science images also capture arthropods from various angles, contributors typically select and upload images that ensure clear community identification, often favouring close-ups of smaller subjects. As a result, the OOD dataset provides a more challenging and ecologically realistic test for model evaluation, reflecting the variability inherent in automated pollinator monitoring, where images are captured passively, without real-time human selection or framing.\u003c/p\u003e\u003cp\u003eThe standard detection approach used by the tested YOLO models relies on a technique called non-maximum suppression (NMS). This method operates independently of the ground-truth bounding boxes and helps the detector to eliminate multiple, similar predictions for the same target object, aiming to produce a single, accurate prediction\u003csup\u003e\u003cspan citationid=\"CR50\" class=\"CitationRef\"\u003e50\u003c/span\u003e\u003c/sup\u003e. Specifically, it allows suppressing predicted boxes that substantially overlap with the predicted box having the highest confidence score (e.g., Supplementary Fig. \u003cspan refid=\"MOESM1\" class=\"InternalRef\"\u003eS1\u003c/span\u003e). NMS, and therefore the detector’s performance, is sensitive to two parameters that impose a trade-off between precision and recall: NMS confidence score and NMS Intersection over Union (NMS-IoU).\u003c/p\u003e\u003cp\u003eTo optimise localisation performance, we fine-tuned the NMS parameters to maximise the F1 score (harmonic mean of precision and recall). Specifically, we ran detections on the OOD dataset using a near-zero (0.001) NMS confidence score and varied the NMS-IoU threshold from 0.1 to 0.9 in 0.1 steps. The low confidence threshold allowed a broad range of predictions to be considered, each with an associated confidence score. Post-detection, we classified predicted boxes as true positives (TP), false positives (FP), or false negatives (FN) based on an evaluation Intersection over Union (eval-IoU) threshold of 0.5, a standard value\u003csup\u003e\u003cspan citationid=\"CR51\" class=\"CitationRef\"\u003e51\u003c/span\u003e\u003c/sup\u003e also used in our previous work\u003csup\u003e\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e\u003c/sup\u003e. If detecting arthropod presence alone (rather than precise box alignment) is the priority, a lower eval-IoU threshold could be used (e.g., 0.1). Further, we computed precision, recall, F1 score across confidence scores (F1-confidence curve), and the area under the precision-recall curve (AUC, also known as average precision). We repeated this evaluation for each NMS-IoU threshold, using the maximum attainable F1 score (from the F1-confidence curves) to identify the best model and its optimal NMS parameters. At this step, the focus was arthropod localisation within single images, regardless of whether they belonged to a time-lapse sequence, and evaluated models in a class-agnostic manner, treating all detections as a single \"arthropod\" class. Inference on the OOD dataset was conducted at an image size of 640 x 640 pixels, consistent with the training image dimensions from our previous study\u003csup\u003e\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e\u003c/sup\u003e.\u003c/p\u003e\u003cp\u003eSubsequently, we employed the optimised detector with the highest F1 score for inference on the OOD dataset, now evaluating predictions across all classes. At this step, we assessed the model's ability to both localise and classify the 1,281 individual arthropods. In this context, an individual arthropod was defined as a series of bounding boxes marked in successive images throughout the time-lapse sequence, which captured the arthropod's presence across multiple frames (e.g., Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003e). Consequently, in these cases, we will refer to the process as \u003cem\u003eindividual arthropod localisation\u003c/em\u003e or \u003cem\u003eclassification\u003c/em\u003e in subsequent discussions. Conversely, when discussing \u003cem\u003earthropod box localisation\u003c/em\u003e or \u003cem\u003eclassification\u003c/em\u003e, we are referring specifically to the best model’s ability to localise or classify an arthropod instances within a given image, regardless of the time-lapse sequence (that is, consecutive time-lapse images are considered \u003cem\u003eindependent\u003c/em\u003e from each other).\u003c/p\u003e\u003cp\u003eThe possible prediction labels for arthropod classification given by the pre-trained YOLO weights\u003csup\u003e\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e\u003c/sup\u003e were Araneae (spiders), Coleoptera (beetles), Diptera (true flies), Hemiptera (true bugs), Hymenoptera (bees and wasps), Hymenoptera Formicidae (ants), Lepidoptera (moths and butterflies), and Orthoptera (crickets and grasshoppers). Lepidoptera and Orthoptera do not appear in the OOD dataset. For analysis at the individual arthropod level, we used three groups: Hymenoptera, Diptera, and OtherT, comprising the remaining taxa groups.\u003c/p\u003e\u003cp\u003eTo evaluate model efficacy for the localisation of boxes with arthropods in independent frames, specific criteria were adopted (see also Supplementary Fig. S2):\u003c/p\u003e\u003cul\u003e \u003cli\u003e \u003cp\u003eTrue Positive (box-TP): defined as a predicted bounding box that is adequately paired with a ground truth box, considering only those pairs for which the eval-IoU exceeds or equals a threshold (e.g., 0.5). When multiple such predicted boxes are present, the box with the highest YOLO confidence score is selected, irrespective of its predicted label.\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003eFalse Positive (box-FP): defined as a predicted bounding box that does not correspond to any ground truth box (model “hallucinates”), or when it does correspond but has an eval-IoU score below the threshold (insufficient overlap). Essentially this is an instance where the model inaccurately localises an arthropod (type I error).\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003eFalse Negative (box-FN): defined as a ground truth box without any corresponding predicted box (type II error). This situation occurs when the model fails to place any predicted bounding boxes in an image containing an arthropod or they are all box-FPs as defined above.\u003c/p\u003e \u003c/li\u003e \u003c/ul\u003e\u003cp\u003eBased on these, successful localisation of an individual arthropod across sequences (arthropod-TP) is achieved if at least one box-TP is encountered across the time-lapse sequence, regardless of the predicted labels (e.g., Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003e). The labels were disregarded for the purpose of the arthropod localisation task, in line with our objective to develop a single-class arthropod detector that will be deployed on an autonomous field camera similar to the work of Sittinger et al\u003csup\u003e\u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e\u003c/sup\u003e. In pollination ecology, an arthropod-TP can be used as an indicator of a successful floral visit and classifying these visitors aids in assessing the diversity of pollinators visiting that specific flower.\u003c/p\u003e\u003cp\u003eTo evaluate the individual arthropod classification performance of the best detector, we employed a maximum confidence rule for label assignment across an entire time-lapse sequence. Specifically, when multiple predicted box-TPs across the sequence correspond to the same arthropod, the label with the highest YOLO confidence score was selected. Performance metrics including precision, recall, F1-score and accuracy were computed for each arthropod category and overall, weighted by the number of individuals.\u003c/p\u003e\u003cp\u003eAdditionally, we employed the best detector to assess false positives per image (FPPI) on the OOD images that only contained floral backgrounds. This detection test utilised 212 background images selected from the 213 distinct time-lapse sessions, with one session excluded because all images contained a beetle. FPPI is defined as the total number of FPs divided by the total number of images in the test set.\u003c/p\u003e\u003cp\u003eWe applied a one-tailed exact binomial test to assess whether cross-order Hymenoptera-Diptera misclassifications occurred at a frequency significantly higher than expected by chance, specifically testing for an excess over chance levels. For the independent frames analysis, where the YOLO model classified arthropod instances into eight groups (Araneae, Coleoptera, Diptera, Hemiptera, Hymenoptera, Hymenoptera-Formicidae, Lepidoptera, Orthoptera), each class had seven possible misclassifications, giving an expected probability of 1/7, 14.29%. For the individual arthropod analysis, where arthropods were observed across frames and grouped into Hymenoptera, Diptera, and OtherT, misclassifications had two possible outcomes (expected probability: 1/2, 50%). The test determined whether these misclassifications occurred significantly more often than expected (p \u0026lt; 0.05).\u003c/p\u003e\u003cp\u003eTo quantify image sharpness within bounding boxes, we applied the Sobel-Tenengrad operator as a proxy\u003csup\u003e\u003cspan citationid=\"CR52\" class=\"CitationRef\"\u003e52\u003c/span\u003e\u003c/sup\u003e. Higher values indicate more edges, signifying increased sharpness. Due to the large absolute values, we normalised them to a 0–1 range (blur to sharp) by dividing each by the maximum observed value.\u003c/p\u003e\u003cp\u003eTo assess differences in relative bounding box area and normalised image sharpness for localisation and classification tasks, we implemented a nonparametric permutation test. This test examines whether the means and medians of two distributions differ, assuming under the null hypothesis that the distributions are identical, with expected differences in these metrics being zero. We compared two groups: (1) ground truth arthropod boxes that were either localised or not, and (2) among localised instances, those correctly classified versus misclassified. We selected this test due to the long-tailed distributions, which deviate from normality. For each comparison, we reported the observed difference (Δ, absolute value) and the p-value relative to the 0.05 significance threshold. The p-value was computed as the proportion of permuted differences at least as extreme as the observed difference, with 1,000 permutations.\u003c/p\u003e"},{"header":"Results","content":"\u003cp\u003eIn the initial class-agnostic test assessing arthropod box localisation within independent frames, YOLOv5-small outperformed YOLOv7-tiny and YOLOv5-nano (Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e, Supplementary Table \u003cspan refid=\"MOESM1\" class=\"InternalRef\"\u003eS1\u003c/span\u003e). Grid search optimisation of YOLOv5-small estimated a maximum F1 score of 0.7019 and an AUC of 0.6497 at optimal NMS hyperparameters IoU\u0026thinsp;=\u0026thinsp;0.3 (Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ea,c) and confidence score\u0026thinsp;=\u0026thinsp;0.2019 (Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003eb). This F1 optimisation also maximised AUC (Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ec,d). Performance remained stable until NMS-IoU exceeded 0.6, then declined (Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ea,c). In our prior study\u003csup\u003e\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e\u003c/sup\u003e with citizen science test images (similar to the training set), optimal NMS-IoU was 0.6 and confidence was 0.3. There, YOLOv5-small achieved a higher F1 score of 0.8886, followed by YOLOv7-tiny (0.8672) and YOLOv5-nano (0.8366), mirroring the current ranking.\u003c/p\u003e \u003cp\u003eAt eval-IoU 0.5, the model produced 2,265 false positive boxes (box-FPs) across the 23,899 OOD arthropod images, yielding a FPPI of 9.48%. At eval-IoU 0.1, box-FPs decreased to 1,799 (FPPI\u0026thinsp;=\u0026thinsp;7.53%). In the control test on 212 floral background images (without arthropods), the model generated 16 box-FPs (FPPI\u0026thinsp;=\u0026thinsp;7.55%), each occurring in a separate image.\u003c/p\u003e \u003cp\u003eSmaller bounding boxes tended to contain blurrier arthropods (Spearman\u0026rsquo;s rank correlation ρ\u0026thinsp;=\u0026thinsp;0.79, p\u0026thinsp;\u0026lt;\u0026thinsp;0.05). Distributions of both box area and sharpness were long-tailed, with most arthropods appearing small and less sharp (Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003ea-d). Of 24,656 ground truth boxes, 14,654 (59.4%) were localised (eval-IoU\u0026thinsp;=\u0026thinsp;0.5), while 10,002 (40.6%) remained undetected. Correctly localised arthropods had significantly larger bounding boxes (Δ\u003csub\u003emeans\u003c/sub\u003e\u0026thinsp;=\u0026thinsp;0.0781, p\u0026thinsp;\u0026lt;\u0026thinsp;0.05; Δ\u003csub\u003emedians\u003c/sub\u003e\u0026thinsp;=\u0026thinsp;0.0672, p\u0026thinsp;\u0026lt;\u0026thinsp;0.05) and higher image sharpness (Δ\u003csub\u003emeans\u003c/sub\u003e\u0026thinsp;=\u0026thinsp;0.0916, p\u0026thinsp;\u0026lt;\u0026thinsp;0.05; Δ\u003csub\u003emedians\u003c/sub\u003e\u0026thinsp;=\u0026thinsp;0.0729, p\u0026thinsp;\u0026lt;\u0026thinsp;0.05) compared to those not localised (Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003ea, b). Among localised arthropods, correctly classified instances also showed greater size (Δ\u003csub\u003emeans\u003c/sub\u003e\u0026thinsp;=\u0026thinsp;0.0304, p\u0026thinsp;\u0026lt;\u0026thinsp;0.05; Δ\u003csub\u003emedians\u003c/sub\u003e\u0026thinsp;=\u0026thinsp;0.0219, p\u0026thinsp;\u0026lt;\u0026thinsp;0.05) and sharpness (Δ\u003csub\u003emeans\u003c/sub\u003e\u0026thinsp;=\u0026thinsp;0.0230, p\u0026thinsp;\u0026lt;\u0026thinsp;0.05; Δ\u003csub\u003emedians\u003c/sub\u003e\u0026thinsp;=\u0026thinsp;0.0133, p\u0026thinsp;\u0026lt;\u0026thinsp;0.05) than misclassified ones (Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003ec, d).\u003c/p\u003e \u003cp\u003eFor individual arthropod localisation within sequences, the optimised YOLOv5-small model achieved rates of 91.21% for Hymenoptera, 80.69% for Diptera, and 56.10% for OtherT flower visitors at eval-IoU 0.5. (Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e). Lowering the eval-IoU to 0.1 slightly improved performance (Supplementary Table S2). A localisation example in sequential time-lapse images is shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003e. Classification recall was highest for Hymenoptera (R\u0026thinsp;=\u0026thinsp;80.45%), followed by Diptera (R\u0026thinsp;=\u0026thinsp;66.90%) and OtherT flower visitors (R\u0026thinsp;=\u0026thinsp;47.97%), but accuracy ranked these groups in the opposite order (Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e).\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003ePerformance metrics of the optimised YOLOv5-small model for individual arthropod localisation and classification at eval-IoU 0.5. An arthropod represents a sequence of bounding boxes in time-lapse images. Columns report total individuals (N. ind.), mean relative bounding box area (Rel. b.box area), mean normalised sharpness (Norm. sharp.), localised arthropods (N), localisation recall or rate (R), classification precision (P), recall (R), F1 score and accuracy (Acc.). Confusion matrix results in \u0026ldquo;Predictions\u0026rdquo; show percentages and counts for Hymenoptera (Hym.), Diptera (Dip.), other arthropods (OtherT), and background/false negatives (Bg./FN).\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"14\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c8\" colnum=\"8\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c9\" colnum=\"9\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c10\" colnum=\"10\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c11\" colnum=\"11\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c12\" colnum=\"12\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c13\" colnum=\"13\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c14\" colnum=\"14\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eArthropod category\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eN. ind.\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eRel. b.box area\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eNorm. sharp.\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"2\" nameend=\"c6\" namest=\"c5\"\u003e \u003cp\u003eLocalisation\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"4\" nameend=\"c10\" namest=\"c7\"\u003e \u003cp\u003eClassification\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"4\" nameend=\"c14\" namest=\"c11\"\u003e \u003cp\u003ePredictions - % and (counts)\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eN\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eR\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003eP\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c8\"\u003e \u003cp\u003eR\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c9\"\u003e \u003cp\u003eF1\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c10\"\u003e \u003cp\u003eAcc.\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c11\"\u003e \u003cp\u003eHym.\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c12\"\u003e \u003cp\u003eDip.\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c13\"\u003e \u003cp\u003eOtherT\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c14\"\u003e \u003cp\u003eBg./FN\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eHymenoptera\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e1,013\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.1073\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.1020\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e924\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.9121\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.9772\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e0.8045\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c9\"\u003e \u003cp\u003e0.8825\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c10\"\u003e \u003cp\u003e0.8306\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c11\"\u003e \u003cp\u003e80.45%\u003c/p\u003e \u003cp\u003e(815)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c12\"\u003e \u003cp\u003e8.49%\u003c/p\u003e \u003cp\u003e(86)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c13\"\u003e \u003cp\u003e2.27%\u003c/p\u003e \u003cp\u003e(23)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c14\"\u003e \u003cp\u003e8.79%\u003c/p\u003e \u003cp\u003e(89)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDiptera\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e145\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.0708\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.1178\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e117\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.8069\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.5243\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e0.6690\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c9\"\u003e \u003cp\u003e0.5879\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c10\"\u003e \u003cp\u003e0.8938\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c11\"\u003e \u003cp\u003e7.59%\u003c/p\u003e \u003cp\u003e(11)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c12\"\u003e \u003cp\u003e66.90%\u003c/p\u003e \u003cp\u003e(97)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c13\"\u003e \u003cp\u003e6.21%\u003c/p\u003e \u003cp\u003e(9)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c14\"\u003e \u003cp\u003e19.31%\u003c/p\u003e \u003cp\u003e(28)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eOtherT\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e123\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.0115\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.0321\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e69\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.5610\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.6484\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e0.4797\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c9\"\u003e \u003cp\u003e0.5514\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c10\"\u003e \u003cp\u003e0.9251\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c11\"\u003e \u003cp\u003e6.50%\u003c/p\u003e \u003cp\u003e(8)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c12\"\u003e \u003cp\u003e1.63%\u003c/p\u003e \u003cp\u003e(2)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c13\"\u003e \u003cp\u003e47.97%\u003c/p\u003e \u003cp\u003e(59)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c14\"\u003e \u003cp\u003e43.90%\u003c/p\u003e \u003cp\u003e(54)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eOverall\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e1,281\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.0751\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.0871\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e1,110\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.8665\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.8944\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c8\"\u003e \u003cp\u003e0.7580\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c9\"\u003e \u003cp\u003e0.8205\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c10\"\u003e \u003cp\u003e0.8468\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c11\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c12\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c13\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c14\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eThe model correctly classified 815/1,013 (80.45%) Hymenoptera and 97/145 (66.90%) Diptera individual arthropods. Notably 86 (8.49%) Hymenoptera were identified as Diptera and 11 (7.59%) Diptera as Hymenoptera (Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e). This bidirectional Hymenoptera-Diptera misclassification was evident in independent frames, with 76.96% of all misclassified Hymenoptera instances (boxes) labelled as Diptera and 64.88% of misclassified Diptera as Hymenoptera, significantly exceeding chance (p\u0026thinsp;\u0026lt;\u0026thinsp;0.05, exact binomial test, expected probability 1/7\u0026thinsp;=\u0026thinsp;14.29%, Supplementary Table S3).\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eOf the 86 Hymenoptera individuals misclassified as Diptera, 43 were \u003cem\u003eApis mellifera\u003c/em\u003e (50.00%), 23 were red-tailed \u003cem\u003eBombus\u003c/em\u003e (26.74%), 8 were Halictidae (9.30%), and 8 were other Hymenopteran taxa (9.30%). The latter two groups (referred to as \u0026ldquo;non-mimicked\u0026rdquo;) are not targeted by Syrphidae mimicry. \u003cem\u003eApis mellifera\u003c/em\u003e, red-tailed \u003cem\u003eBombus\u003c/em\u003e, and Halictidae were common in the OOD dataset, collectively accounting for 56.47% (572/1,013) of Hymenoptera individuals, and thus we expected higher total misclassifications due to their higher total abundance in the dataset. However, from all misclassified Hymenoptera individuals (to Diptera or OtherT), the ones to Diptera exceeded chance (p\u0026thinsp;\u0026lt;\u0026thinsp;0.05; exact binomial test; expected: 50% Diptera, 50% OtherT) for \u003cem\u003eApis mellifera\u003c/em\u003e (43/46, 93.48%) and red-tailed \u003cem\u003eBombus\u003c/em\u003e (23/26, 88.46%), but not for Halictidae (8/12, 66.67%) or Halictidae plus other non-mimicked taxa (\u0026ldquo;Non-mimicked\u0026rdquo;, 16/24, 66.67%; p\u0026thinsp;\u0026gt;\u0026thinsp;0.05, see also Supplementary Table S4).\u003c/p\u003e \u003cp\u003eFor \u003cem\u003eApis mellifera\u003c/em\u003e, cases misclassified as Diptera showed no significant differences in means or medians for bounding box area, sharpness, or model confidence from correctly classified ones (p\u0026thinsp;\u0026gt;\u0026thinsp;0.05). Red-tailed \u003cem\u003eBombus\u003c/em\u003e misclassifications had significantly smaller mean bounding box areas (p\u0026thinsp;\u0026lt;\u0026thinsp;0.05), but similar medians, sharpness, and model confidence (p\u0026thinsp;\u0026gt;\u0026thinsp;0.05). In contrast, non-mimicked Hymenoptera misclassified as Diptera showed significantly higher confidence in correct classifications (p\u0026thinsp;\u0026lt;\u0026thinsp;0.05). Misclassifications did not show significant differences in area or sharpness (p\u0026thinsp;\u0026gt;\u0026thinsp;0.05; Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e6\u003c/span\u003e, Supplementary Table S4).\u003c/p\u003e \u003cp\u003eAmong the 11 Diptera that were misclassified as Hymenoptera, six were Syrphidae and five were individuals that could not be identified by experts to the family level from the image (referred to hereafter as coarsely identified Diptera). The proportion of misclassifications as Hymenoptera did not differ from chance for either Syrphidae (66.67%) or coarsely identified (45.45%) Diptera (p\u0026thinsp;\u0026gt;\u0026thinsp;0.05). Syrphidae misclassified as Hymenoptera showed no significant differences in bounding box area or sharpness from correctly classified cases (p\u0026thinsp;\u0026gt;\u0026thinsp;0.05). Conversely, the misclassifications for coarsely identified Diptera were significantly smaller and blurrier than the correct classifications (p\u0026thinsp;\u0026lt;\u0026thinsp;0.05). In both Diptera groups, model confidence was significantly higher for correct classifications (p\u0026thinsp;\u0026lt;\u0026thinsp;0.05; Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e6\u003c/span\u003e, Supplementary Table S5).\u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003eOur results show that the optimised YOLOv5-small model, trained on citizen science images, correctly localised 91.21% and classified 80.45% of Hymenoptera individuals, as well as localized 80.69% and classified 66.90% of Diptera individuals. Detection performance was weaker for other flower visitors (OtherT), which were typically smaller and blurrier. However, their higher accuracy (92.51%) shows the model mislabels Hymenoptera or Diptera as OtherT less frequently.\u003c/p\u003e \u003cp\u003eTo meet the demands of real-world pollinator monitoring, we chose lightweight models, as they promise energy-efficient deployment in field settings. Among those tested, YOLOv5-small, with the highest parameter count, outperformed others in F1 score, aligning with prior findings that greater model capacity (i.e., more trainable parameters) enhances performance\u003csup\u003e\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e,\u003cspan citationid=\"CR53\" class=\"CitationRef\"\u003e53\u003c/span\u003e\u003c/sup\u003e. Our previous study\u003csup\u003e\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e\u003c/sup\u003e noted a similar trend. Future work could explore higher-capacity architectures compatible with in situ hardware constraints.\u003c/p\u003e \u003cp\u003eThe grid search NMS optimisation, maximising the F1 score of arthropod detectors on the unseen OOD image dataset under complex field conditions, has practical implications for camera system design. For instance, adapting Sittinger et al.\u0026rsquo;s\u003csup\u003e\u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e\u003c/sup\u003e setup for monitoring flower visitors could enhance on-device detection performance beyond default NMS values. This optimisation reflects dataset-specific tuning, as evidenced by comparing prior and current studies. In our earlier work with citizen science test images\u003csup\u003e\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e\u003c/sup\u003e, a higher NMS-IoU suited dense, overlapping bounding boxes of ants and bugs (e.g., images near ant colonies). Conversely, the OOD flower-visit dataset, dominated by images containing single arthropods, favoured a lower NMS-IoU, with performance declining at higher values (Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003ea,c). A higher NMS-IoU threshold permits overlapping boxes, aiding detection of closely spaced arthropods, whereas a lower threshold enhances precision by minimising redundant predictions for solitary arthropods.\u003c/p\u003e \u003cp\u003eOur pollinator localisation tests have practical implications, demonstrating the potential of object detection models trained on citizen science images to assist in annotating time-lapse field datasets, where most frames lack arthropods (e.g., over 90%\u003csup\u003e37,54,55\u003c/sup\u003e). A single prediction per sequence enables annotators to target relevant frames, bypassing manual review of arthropod-free images. Manual annotation of a 460,056-image time-lapse dataset previously required approximately 1,000 hours\u003csup\u003e\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e\u003c/sup\u003e, whereas the YOLOv5-small model processed 23,899 OOD images in 419 seconds on an NVIDIA RTX A6000 GPU, suggesting around 2.24-hours runtime for the larger dataset, assuming fast image access. However, false positive (FP) rates on OOD images, including floral-only backgrounds, surpassed those on citizen science images, which more closely resemble the training set\u003csup\u003e\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e\u003c/sup\u003e. These additional FPs increase manual correction time, and we showed that lowering the eval-IoU threshold could enhance localisation and reduce FPs by allowing larger predicted boxes with sub-0.5 IoU (e.g., Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003e). Including floral backgrounds without pollinators in training may further reduce FP rates and improve precision. Another challenge is that smaller, less sharp arthropods are more likely to be missed. While the model effectively localised larger, common Hymenoptera and Diptera pollinators, it struggled with other flower visitors in the OOD dataset, which tended to be smaller and blurrier.\u003c/p\u003e \u003cp\u003eAfter localisation, classifying flower visitors challenged the model more, with significant bidirectional Hymenoptera and Diptera misclassifications outnumbering those to other categories, alongside reduced performance for other arthropods. While it distinguished these categories effectively on in-distribution images\u003csup\u003e\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e\u003c/sup\u003e, this proficiency declined on the OOD dataset, where arthropods were on average 4.5 times smaller than in-distribution counterparts and sometimes occluded by flower parts (e.g., Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003e). This aligns with studies reporting reduced generalisation on organisms across new locations, time-frames, and sensors\u003csup\u003e\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e,\u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e,\u003cspan citationid=\"CR54\" class=\"CitationRef\"\u003e54\u003c/span\u003e,\u003cspan additionalcitationids=\"CR57\" citationid=\"CR56\" class=\"CitationRef\"\u003e56\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR58\" class=\"CitationRef\"\u003e58\u003c/span\u003e\u003c/sup\u003e, alongside pollinator-specific occlusion challenges\u003csup\u003e\u003cspan additionalcitationids=\"CR60\" citationid=\"CR59\" class=\"CitationRef\"\u003e59\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR61\" class=\"CitationRef\"\u003e61\u003c/span\u003e\u003c/sup\u003e. Moreover, the pretrained models were not trained with more images of either Hymenoptera or Diptera than other categories, ruling out dataset bias as a cause of cross-order misclassifications. This is further supported by the fact that, despite Lepidoptera being nearly twice as abundant in the training data\u003csup\u003e\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e\u003c/sup\u003e, the model rarely mislabeled Hymenoptera or Diptera as Lepidoptera (e.g., Supplementary Table S3). Likewise, the higher accuracy for OtherT flower visitors shows the model less often mislabels Hymenoptera or Diptera as OtherT.\u003c/p\u003e \u003cp\u003eGiven these, Syrphidae mimicry most likely exacerbates the Hymenoptera-Diptera confusion, with syrphids like \u003cem\u003eEristalis\u003c/em\u003e spp. and \u003cem\u003eVolucella bombylans\u003c/em\u003e resembling bees (e.g., \u003cem\u003eApis mellifera\u003c/em\u003e\u003csup\u003e\u003cspan citationid=\"CR62\" class=\"CitationRef\"\u003e62\u003c/span\u003e\u003c/sup\u003e) and red-tailed \u003cem\u003eBombus\u003c/em\u003e (e.g., \u003cem\u003eB. lapidarius\u003c/em\u003e, \u003cem\u003eB. pratorum\u003c/em\u003e\u003csup\u003e\u003cspan citationid=\"CR63\" class=\"CitationRef\"\u003e63\u003c/span\u003e\u003c/sup\u003e), respectively, mimicking their warning signals to deter predators. In the OOD dataset, larger or sharper arthropod instances exhibited significantly distinct distributions from smaller or blurrier counterparts for both localisation and classification. However, \u003cem\u003eApis mellifera\u003c/em\u003e and red-tailed \u003cem\u003eBombus\u003c/em\u003e, misclassified as Diptera, were as large and sharp as correctly classified cases, and the model was equally confident in these misclassifications most likely due to mimicry. In contrast, cross-order misclassified taxa not mimicked by Syrphidae (e.g., Halictidae, Cynipidae in Hymenoptera) and a few small, coarsely identified Diptera, had significantly higher model confidence in correct classifications. Their misclassified cases tended to be smaller and blurrier than correctly classified ones, likely explaining the mislabeling. Syrphidae misclassified as Hymenoptera were as large and sharp as correctly classified cases, but the model was significantly less confident in misclassifications. While these results might suggest that mimicry confuses the model more in one direction, with mimicked Hymenoptera more likely to be misclassified as Diptera than mimicking Syrphidae as Hymenoptera, we cannot say this conclusively due to the smaller sample size of Syrphidae individuals that were misclassified as Hymenoptera.\u003c/p\u003e \u003cp\u003eTo improve localisation and classification, we consider several steps for future research. First, integrating citizen-science and field images, as in recent studies\u003csup\u003e\u003cspan citationid=\"CR64\" class=\"CitationRef\"\u003e64\u003c/span\u003e,\u003cspan citationid=\"CR65\" class=\"CitationRef\"\u003e65\u003c/span\u003e\u003c/sup\u003e, would enhance model generalisation for real-world pollinator monitoring using time-lapse photography. Given that multiple studies have highlighted the scarcity of annotated field datasets for small arthropods, including pollinators\u003csup\u003e\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e,\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e,\u003cspan citationid=\"CR66\" class=\"CitationRef\"\u003e66\u003c/span\u003e,\u003cspan citationid=\"CR67\" class=\"CitationRef\"\u003e67\u003c/span\u003e\u003c/sup\u003e, our study addresses this gap by providing the OOD dataset (cropped and full-frame images) for training arthropod detectors for custom field cameras. At the same time, maintaining a clear separation between training and test sets is essential. Time-lapse image sequences can introduce a risk of data leakage\u003csup\u003e\u003cspan citationid=\"CR68\" class=\"CitationRef\"\u003e68\u003c/span\u003e,\u003cspan citationid=\"CR69\" class=\"CitationRef\"\u003e69\u003c/span\u003e\u003c/sup\u003e if highly similar frames are split between these sets, potentially inflating model performance. In such cases, the network may rely on shortcut learning\u003csup\u003e\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e\u003c/sup\u003e, recognising near-identical images based on superficial visual similarities (e.g., background patterns, nearly identical insect poses) rather than developing a truly generalisable understanding of arthropod features. To mitigate this, careful dataset partitioning is needed to prevent the model from exploiting temporal redundancies (e.g., highly similar consecutive frames depicting the same individual arthropod should be kept within a single set, either training, validation, or test, rather than split across them).\u003c/p\u003e \u003cp\u003eSecond, model performance could improve through a two-step approach, as suggested in other studies\u003csup\u003e\u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e,\u003cspan citationid=\"CR54\" class=\"CitationRef\"\u003e54\u003c/span\u003e,\u003cspan citationid=\"CR58\" class=\"CitationRef\"\u003e58\u003c/span\u003e,\u003cspan citationid=\"CR64\" class=\"CitationRef\"\u003e64\u003c/span\u003e,\u003cspan citationid=\"CR70\" class=\"CitationRef\"\u003e70\u003c/span\u003e\u003c/sup\u003e. For example, an initial single-class object detector, such as YOLO\u003csup\u003e\u003cspan citationid=\"CR50\" class=\"CitationRef\"\u003e50\u003c/span\u003e\u003c/sup\u003e, could localise arthropods (e.g., arthropod vs. background), followed by a classifier to identify their cropped images at finer taxonomic levels. This approach allows the community to choose object detectors suited to their field hardware while leveraging diverse classification methods in post-processing, such as region-specific classifiers trained on continuously expanding datasets\u003csup\u003e\u003cspan citationid=\"CR71\" class=\"CitationRef\"\u003e71\u003c/span\u003e\u003c/sup\u003e, taxon-specific classifiers\u003csup\u003e\u003cspan citationid=\"CR72\" class=\"CitationRef\"\u003e72\u003c/span\u003e\u003c/sup\u003e, large multimodal models\u003csup\u003e\u003cspan citationid=\"CR73\" class=\"CitationRef\"\u003e73\u003c/span\u003e\u003c/sup\u003e, or hierarchical classification via custom classifier\u003csup\u003e\u003cspan citationid=\"CR64\" class=\"CitationRef\"\u003e64\u003c/span\u003e,\u003cspan citationid=\"CR74\" class=\"CitationRef\"\u003e74\u003c/span\u003e,\u003cspan citationid=\"CR75\" class=\"CitationRef\"\u003e75\u003c/span\u003e\u003c/sup\u003e and vision foundation models capable of learning hierarchical representations\u003csup\u003e\u003cspan citationid=\"CR76\" class=\"CitationRef\"\u003e76\u003c/span\u003e\u003c/sup\u003e. Furthermore, integrating object detection with segmentation has been shown to improve bumblebee species identification by removing noisy backgrounds and focusing classifiers on the most relevant features\u003csup\u003e\u003cspan citationid=\"CR77\" class=\"CitationRef\"\u003e77\u003c/span\u003e\u003c/sup\u003e. Additionally, citizen science platforms encourage users to upload cropped images of organisms\u003csup\u003e\u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e\u003c/sup\u003e, providing a rich source of training data for such classifiers. Another advantage is the potential for multi-view classification\u003csup\u003e\u003cspan citationid=\"CR78\" class=\"CitationRef\"\u003e78\u003c/span\u003e\u003c/sup\u003e, leveraging sequential images of the same arthropod. Similar to how taxonomists examine multiple frames (e.g., Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003e) to improve identification despite occlusions or lower-quality frames, a multi-view CNN could refine predictions. In our study, we simplified this by assigning the label with the highest confidence score across a sequence, but a dedicated multi-view CNN could further enhance performance.\u003c/p\u003e \u003cp\u003eThird, preprocessing time-lapse images to highlight arthropod features against the background\u003csup\u003e\u003cspan citationid=\"CR59\" class=\"CitationRef\"\u003e59\u003c/span\u003e\u003c/sup\u003e could enhance localisation if compatible with low-energy field cameras, or, if too energy-intensive, applied later on stored images rather than in real-time.\u003c/p\u003e \u003cp\u003eFourth, our results confirm arthropod size and image sharpness as important factors to localisation and classification, aligning with Nguyen et al.\u0026rsquo;s\u003csup\u003e\u003cspan citationid=\"CR66\" class=\"CitationRef\"\u003e66\u003c/span\u003e,\u003cspan citationid=\"CR79\" class=\"CitationRef\"\u003e79\u003c/span\u003e\u003c/sup\u003e findings on small-object detection challenges. The correlation between size and sharpness indicate also that arthropods further from the camera, or small arthropods in general, are most likely to be out of focus. Optimising image capture thus involves defining a region of interest and focusing on the target flower or inflorescence segment within, to maximise arthropod size in the frame. The region of interest can be defined via flower detection, segmentation, or pre-defined at the start of the recording session. Fixed focus is also crucial, and we adopted it when collecting the OOD dataset to prevent autofocus from shifting to background and blurring arthropods, as observed by Bjerge et al.\u003csup\u003e\u003cspan citationid=\"CR59\" class=\"CitationRef\"\u003e59\u003c/span\u003e\u003c/sup\u003e. Additionally, including blurred images in training datasets could further improve generalisation, as shown in larval fish detection\u003csup\u003e\u003cspan citationid=\"CR80\" class=\"CitationRef\"\u003e80\u003c/span\u003e\u003c/sup\u003e.\u003c/p\u003e \u003cp\u003eLastly, tiling full-frame images for detection could improve small-object localisation\u003csup\u003e\u003cspan citationid=\"CR81\" class=\"CitationRef\"\u003e81\u003c/span\u003e,\u003cspan citationid=\"CR82\" class=\"CitationRef\"\u003e82\u003c/span\u003e\u003c/sup\u003e by preserving details without downscaling to detector\u0026rsquo;s resolution. However, sliced inference like SAHI\u003csup\u003e\u003cspan citationid=\"CR81\" class=\"CitationRef\"\u003e81\u003c/span\u003e\u003c/sup\u003e increases computational demands on low-power field devices. While not our primary focus, our preliminary SAHI test with YOLOv5-small on the OOD dataset showed slight F1 gains, but increased false positives and processing time (Supplementary Table S6). Still, fine-tuning SAHI could aid annotation of high-resolution time-lapse datasets when real-time processing is not required.\u003c/p\u003e \u003cp\u003eImplementing these proposed steps could enhance the detection of flower visitors, thereby facilitating the tracking of individual pollinators and enabling estimates of floral visit abundance, a key goal for automated pollinator monitoring. Examples of insect tracking can be found in recent studies\u003csup\u003e\u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e,\u003cspan citationid=\"CR60\" class=\"CitationRef\"\u003e60\u003c/span\u003e,\u003cspan citationid=\"CR83\" class=\"CitationRef\"\u003e83\u003c/span\u003e\u003c/sup\u003e.\u003c/p\u003e"},{"header":"Conclusion","content":"\u003cp\u003eOur findings highlight the potential and limitations of lightweight YOLO detector models, trained on citizen science images, for localising and classifying flower visitors in out-of-distribution (OOD) time-lapse field images captured with a fixed smartphone setup. Localisation was generally effective for common Hymenoptera and Diptera pollinators, defined as cases where at least one bounding box in a time-lapse sequence was correctly placed. However, classification proved more challenging, impacted by arthropod size, image sharpness, and mimicry between Syrphidae (Diptera) and Hymenoptera. Smaller, blurrier arthropods, including less common flower visitors, were harder to detect, and the increase in false positives compared to in-distribution data revealed limitations when generalising to complex field conditions.\u003c/p\u003e \u003cp\u003eThese results have practical value for pollinator monitoring, showing potential for automating annotation of common Hymenoptera and Diptera pollinators in large time-lapse datasets, likely easing manual workloads. Future work could enhance performance by combining field and citizen science images in training, using a two-step detection-classification approach, optimising image capture to enhance arthropod size and sharpness, or adjusting NMS-IoU for specific ecological contexts. By providing an OOD dataset and identifying key challenges, this work contributes to the development of more robust machine learning tools for pollinator monitoring in natural environments.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003eData availability\u003c/p\u003e\n\u003cp\u003eThe image dataset related to this research is available at https://doi.org/10.5281/zenodo.15096609. The open-source code for the experiments is hosted on GitHub at https://github.com/valentinitnelav/smartphone-insect-detect.\u003c/p\u003e\u003cp\u003eAcknowledgements\u003c/p\u003e\n\u003cp\u003eThe authors express gratitude to Bilyana Stoykova, Ricardo Urrego Alvarez, Emil Cyranka, and Anna Scheiper for their invaluable assistance with field work and meticulous efforts in manually annotating arthropods within the images. Special thanks are extended to Aspen Workman, Jared C. Cobain, and Demetra Rakosy for their expertise and contribution in the taxonomic identification of the pollinators studied. Additionally, many thanks go to Anne-Kathrin Thomas and Nina Becker for their comprehensive logistical support throughout this research. We gratefully acknowledge the use of hardware resources from the Leipzig University Computing Center and the German Center for Integrative Biodiversity Research (iDiv) Halle-Jena-Leipzig. We also extend our thanks to Justin Kay and Sara Beery, from whom we learned valuable object detector evaluation methods, and the entire team for organising the Computer Vision for Ecology Summer Workshop, supported by the Resnick Sustainability Institute. Last but not least, we are grateful to Maximilian Sittinger for thought-provoking discussions.\u003c/p\u003e\n\u003cp\u003eFunding\u003c/p\u003e\n\u003cp\u003eThis research was funded by the Helmholtz AI initiative (Information \u0026amp; Data Science) Pollination Artificial Intelligence (ZT-I-PF-5-115), led by Prof. Tiffany M. Knight and Prof. Hannes Taubenboeck, the Helmholtz Recruitment Initiative of the Helmholtz Association to Tiffany M. Knight, and iDiv (German Research Foundation FZT 118).\u003c/p\u003e\n\u003cp\u003eAuthor contributions\u003c/p\u003e\n\u003cp\u003eV.S., T.S., M.W., H.T. and T.M.K. designed the scope of the study. V.S. and T.M.K. coordinated the creation and curation of the image dataset and its annotations. V.S. took the lead in designing and implementing the methods, conducting the analysis, and writing the manuscript. All authors reviewed the manuscript.\u003c/p\u003e\n\u003cp\u003eCompeting interests\u003c/p\u003e\n\u003cp\u003eThe authors declare no competing interests.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003ePotts, S. G. \u003cem\u003eet al.\u003c/em\u003e Safeguarding pollinators and their values to human well-being. \u003cem\u003eNature \u003c/em\u003e\u003cstrong\u003e540\u003c/strong\u003e, 220\u0026ndash;229 (2016).\u003c/li\u003e\n\u003cli\u003ePotts, S. G. \u003cem\u003eet al.\u003c/em\u003e Global pollinator declines: trends, impacts and drivers. \u003cem\u003eTrends Ecol. Evol. \u003c/em\u003e\u003cstrong\u003e25\u003c/strong\u003e, 345\u0026ndash;353 (2010).\u003c/li\u003e\n\u003cli\u003eArtamendi, M., Martin, P. A., Bartomeus, I. \u0026amp; Magrach, A. Loss of pollinator diversity consistently reduces reproductive success for wild and cultivated plants. \u003cem\u003eNat. Ecol. Evol. \u003c/em\u003e\u003cstrong\u003e9\u003c/strong\u003e, 296\u0026ndash;313 (2024).\u003c/li\u003e\n\u003cli\u003eEuropean Commission. EU Pollinators Initiative: Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52018DC0395 (2018).\u003c/li\u003e\n\u003cli\u003eMotivans \u0026Scaron;vara, E. \u003cem\u003eet al.\u003c/em\u003e Effects of different types of low‐intensity management on plant‐pollinator interactions in Estonian grasslands. \u003cem\u003eEcol. Evol. \u003c/em\u003e\u003cstrong\u003e11\u003c/strong\u003e, 16909\u0026ndash;16926 (2021).\u003c/li\u003e\n\u003cli\u003eRakosy, D. \u003cem\u003eet al.\u003c/em\u003e Intensive grazing alters the diversity, composition and structure of plant-pollinator interaction networks in Central European grasslands. \u003cem\u003ePLOS ONE \u003c/em\u003e\u003cstrong\u003e17\u003c/strong\u003e, e0263576 (2022).\u003c/li\u003e\n\u003cli\u003eCreedy, T. J. \u003cem\u003eet al.\u003c/em\u003e A validated workflow for rapid taxonomic assignment and monitoring of a national fauna of bees (Apiformes) using high throughput DNA barcoding. \u003cem\u003eMol. Ecol. Resour. \u003c/em\u003e\u003cstrong\u003e20\u003c/strong\u003e, 40\u0026ndash;53 (2020).\u003c/li\u003e\n\u003cli\u003evan Klink, R. \u003cem\u003eet al.\u003c/em\u003e Emerging technologies revolutionise insect ecology and monitoring. \u003cem\u003eTrends Ecol. Evol.\u003c/em\u003e S0169534722001343 (2022) doi:10.1016/j.tree.2022.06.001.\u003c/li\u003e\n\u003cli\u003eMontero‐Casta\u0026ntilde;o, A. \u003cem\u003eet al.\u003c/em\u003e Pursuing best practices for minimizing wild bee captures to support biological research. \u003cem\u003eConserv. Sci. Pract. \u003c/em\u003e\u003cstrong\u003e4\u003c/strong\u003e, e12734 (2022).\u003c/li\u003e\n\u003cli\u003eUltralytics. https://github.com/ultralytics/ultralytics.\u003c/li\u003e\n\u003cli\u003eDetectron2. https://github.com/facebookresearch/detectron2.\u003c/li\u003e\n\u003cli\u003eHernandez, A. \u003cem\u003eet al.\u003c/em\u003e Pytorch-Wildlife: A Collaborative Deep Learning Framework for Conservation. Preprint at http://arxiv.org/abs/2405.12930 (2024).\u003c/li\u003e\n\u003cli\u003eIntroducing: Raspberry Pi 5! https://www.raspberrypi.com/news/introducing-raspberry-pi-5/.\u003c/li\u003e\n\u003cli\u003eLuxonis OAK modules. https://shop.luxonis.com/collections/oak-modules.\u003c/li\u003e\n\u003cli\u003eGet Started With Jetson Nano Developer Kit. https://developer.nvidia.com/embedded/learn/get-started-jetson-nano-devkit.\u003c/li\u003e\n\u003cli\u003eCoral Dev Board. https://coral.ai/docs/dev-board/get-started.\u003c/li\u003e\n\u003cli\u003eQualcomm Snapdragon. \u003cem\u003eWikipedia\u003c/em\u003e (2005).\u003c/li\u003e\n\u003cli\u003eTuia, D. \u003cem\u003eet al.\u003c/em\u003e Perspectives in machine learning for wildlife conservation. \u003cem\u003eNat. Commun. \u003c/em\u003e\u003cstrong\u003e13\u003c/strong\u003e, 792 (2022).\u003c/li\u003e\n\u003cli\u003eJolles, J. W. Broad‐scale applications of the Raspberry Pi: A review and guide for biologists. \u003cem\u003eMethods Ecol. Evol. \u003c/em\u003e\u003cstrong\u003e12\u003c/strong\u003e, 1562\u0026ndash;1579 (2021).\u003c/li\u003e\n\u003cli\u003eMartineau, M. \u003cem\u003eet al.\u003c/em\u003e A survey on image-based insect classification. \u003cem\u003ePattern Recognit. \u003c/em\u003e\u003cstrong\u003e65\u003c/strong\u003e, 273\u0026ndash;284 (2017).\u003c/li\u003e\n\u003cli\u003eBarlow, S. E. \u0026amp; O\u0026rsquo;Neill, M. A. Technological advances in field studies of pollinator ecology and the future of e-ecology. \u003cem\u003eCurr. Opin. Insect Sci. \u003c/em\u003e\u003cstrong\u003e38\u003c/strong\u003e, 15\u0026ndash;25 (2020).\u003c/li\u003e\n\u003cli\u003ePegoraro, L., Hidalgo, O., Leitch, I. J., Pellicer, J. \u0026amp; Barlow, S. E. Automated video monitoring of insect pollinators in the field. \u003cem\u003eEmerg. Top. Life Sci. \u003c/em\u003e\u003cstrong\u003e4\u003c/strong\u003e, 87\u0026ndash;97 (2020).\u003c/li\u003e\n\u003cli\u003eH\u0026oslash;ye, T. T. \u003cem\u003eet al.\u003c/em\u003e Deep learning and computer vision will transform entomology. \u003cem\u003eProc. Natl. Acad. Sci. \u003c/em\u003e\u003cstrong\u003e118\u003c/strong\u003e, e2002545117 (2021).\u003c/li\u003e\n\u003cli\u003eAmarathunga, D. C., Grundy, J., Parry, H. \u0026amp; Dorin, A. Methods of insect image capture and classification: A Systematic literature review. \u003cem\u003eSmart Agric. Technol. \u003c/em\u003e\u003cstrong\u003e1\u003c/strong\u003e, 100023 (2021).\u003c/li\u003e\n\u003cli\u003eH\u0026oslash;ye, T. T., Montagna, M., Oteman, B. \u0026amp; Roy, D. B. Emerging technologies for pollinator monitoring. \u003cem\u003eCurr. Opin. Insect Sci.\u003c/em\u003e 101367 (2025) doi:10.1016/j.cois.2025.101367.\u003c/li\u003e\n\u003cli\u003eSun, C., Shrivastava, A., Singh, S. \u0026amp; Gupta, A. Revisiting unreasonable effectiveness of data in deep learning era. in \u003cem\u003eProceedings of the IEEE international conference on computer vision\u003c/em\u003e 843\u0026ndash;852 (2017).\u003c/li\u003e\n\u003cli\u003eBeery, S., Van Horn, G. \u0026amp; Perona, P. Recognition in Terra Incognita. in \u003cem\u003eProceedings of the European Conference on Computer Vision (ECCV)\u003c/em\u003e 456\u0026ndash;473 (2018).\u003c/li\u003e\n\u003cli\u003eGeirhos, R. \u003cem\u003eet al.\u003c/em\u003e Shortcut learning in deep neural networks. \u003cem\u003eNat. Mach. Intell. \u003c/em\u003e\u003cstrong\u003e2\u003c/strong\u003e, 665\u0026ndash;673 (2020).\u003c/li\u003e\n\u003cli\u003eKay, J. \u003cem\u003eet al.\u003c/em\u003e Align and Distill: Unifying and Improving Domain Adaptive Object Detection. (2024) doi:10.48550/ARXIV.2403.12029.\u003c/li\u003e\n\u003cli\u003eOza, P., Sindagi, V. A., Sharmini, V. V. \u0026amp; Patel, V. M. Unsupervised Domain Adaptation of Object Detectors: A Survey. \u003cem\u003eIEEE Trans. Pattern Anal. Mach. Intell.\u003c/em\u003e 1\u0026ndash;24 (2023) doi:10.1109/TPAMI.2022.3217046.\u003c/li\u003e\n\u003cli\u003eKoh, P. W. \u003cem\u003eet al.\u003c/em\u003e WILDS: A Benchmark of in-the-Wild Distribution Shifts. (2021).\u003c/li\u003e\n\u003cli\u003eStark, T. \u003cem\u003eet al.\u003c/em\u003e YOLO object detection models can locate and classify broad groups of flower-visiting arthropods in images. \u003cem\u003eSci. Rep. \u003c/em\u003e\u003cstrong\u003e13\u003c/strong\u003e, 16364 (2023).\u003c/li\u003e\n\u003cli\u003eElvekjaer, N. \u003cem\u003eet al.\u003c/em\u003e Detecting flowers on imagery with computer vision to improve continental scale grassland biodiversity surveying. \u003cem\u003eEcol. Solut. Evid. \u003c/em\u003e\u003cstrong\u003e5\u003c/strong\u003e, e12324 (2024).\u003c/li\u003e\n\u003cli\u003eRankin, D. A plea to crop your photos! \u003cem\u003eiNaturalist Community Forum \u003c/em\u003ehttps://forum.inaturalist.org/t/a-plea-to-crop-your-photos/35251 (2022).\u003c/li\u003e\n\u003cli\u003eHendrycks, D. \u003cem\u003eet al.\u003c/em\u003e The many faces of robustness: A critical analysis of out-of-distribution generalization. in \u003cem\u003eProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)\u003c/em\u003e 8340\u0026ndash;8349 (2021).\u003c/li\u003e\n\u003cli\u003eLiu, J. \u003cem\u003eet al.\u003c/em\u003e Towards Out-Of-Distribution Generalization: A Survey. (2021) doi:10.48550/ARXIV.2108.13624.\u003c/li\u003e\n\u003cli\u003eȘtefan, V., Workman, A., Cobain, J. C., Rakosy, D. \u0026amp; Knight, T. M. Utilising affordable smartphones and open-source time-lapse photography for pollinator image collection and annotation. \u003cem\u003eJ. Pollinat. Ecol. \u003c/em\u003e\u003cstrong\u003e37\u003c/strong\u003e, 1\u0026ndash;21 (2025).\u003c/li\u003e\n\u003cli\u003eHarman, M. Open Camera. https://sourceforge.net/projects/opencamera/ (2023).\u003c/li\u003e\n\u003cli\u003eDutta, A. \u0026amp; Zisserman, A. The VIA Annotation Software for Images, Audio and Video. in \u003cem\u003eProceedings of the 27th ACM International Conference on Multimedia\u003c/em\u003e 2276\u0026ndash;2279 (ACM, Nice France, 2019). doi:10.1145/3343031.3350535.\u003c/li\u003e\n\u003cli\u003eSittinger, M., Uhler, J., Pink, M. \u0026amp; Herz, A. Insect detect: An open-source DIY camera trap for automated insect monitoring. \u003cem\u003ePLOS ONE \u003c/em\u003e\u003cstrong\u003e19\u003c/strong\u003e, e0295474 (2024).\u003c/li\u003e\n\u003cli\u003eKing, C., Ballantyne, G. \u0026amp; Willmer, P. G. Why flower visitation is a poor proxy for pollination: measuring single‐visit pollen deposition, with implications for pollination networks and conservation. \u003cem\u003eMethods Ecol. Evol. \u003c/em\u003e\u003cstrong\u003e4\u003c/strong\u003e, 811\u0026ndash;818 (2013).\u003c/li\u003e\n\u003cli\u003elovehiking. Araneae - Araniella alpica. https://inaturalist-open-data.s3.amazonaws.com/photos/84197357/original.jpeg.\u003c/li\u003e\n\u003cli\u003eArter. Coleoptera - Chrysolina brunsvicensis. https://arter.dk/media/0c15e627-4849-4346-acb1-ada5012d8d13.jpg.\u003c/li\u003e\n\u003cli\u003ede Graaf, T. Diptera - Epistrophe grossulariae. https://observation.org/photos/759251.jpg.\u003c/li\u003e\n\u003cli\u003eBryukhov, V. Hemiptera - Handianus flavovarius. https://inaturalist-open-data.s3.amazonaws.com/photos/58813032/original.jpg.\u003c/li\u003e\n\u003cli\u003eJan, S. Hymenoptera - Andrena fulvago. https://observation.org/photos/6724813.jpg.\u003c/li\u003e\n\u003cli\u003eDavid, R. Hymenoptera, Formicidae - Aphaenogaster spinosa. https://inaturalist-open-data.s3.amazonaws.com/photos/5966960/original.jpeg.\u003c/li\u003e\n\u003cli\u003eSteve, D. Lepidoptera - Adela paludicolella. https://inaturalist-open-data.s3.amazonaws.com/photos/3882519/original.JPG?1464849720.\u003c/li\u003e\n\u003cli\u003eDrepanostoma. Orthoptera- Nadigella formosanta. https://inaturalist-open-data.s3.amazonaws.com/photos/109734596/original.jpg?1610288615.\u003c/li\u003e\n\u003cli\u003eTerven, J. \u0026amp; Cordova-Esparza, D. A Comprehensive Review of YOLO Architectures in Computer Vision: From YOLOv1 to YOLOv8 and YOLO-NAS. (2023) doi:10.48550/ARXIV.2304.00501.\u003c/li\u003e\n\u003cli\u003eEveringham, M., Van Gool, L., Williams, C. K. I., Winn, J. \u0026amp; Zisserman, A. The Pascal Visual Object Classes (VOC) Challenge. \u003cem\u003eInt. J. Comput. Vis. \u003c/em\u003e\u003cstrong\u003e88\u003c/strong\u003e, 303\u0026ndash;338 (2010).\u003c/li\u003e\n\u003cli\u003ePertuz, S., Puig, D. \u0026amp; Garcia, M. A. Analysis of focus measure operators for shape-from-focus. \u003cem\u003ePattern Recognit. \u003c/em\u003e\u003cstrong\u003e46\u003c/strong\u003e, 1415\u0026ndash;1432 (2013).\u003c/li\u003e\n\u003cli\u003eWang, Y.-X., Ramanan, D. \u0026amp; Hebert, M. Growing a brain: Fine-tuning by increasing model capacity. in \u003cem\u003eProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)\u003c/em\u003e 2471\u0026ndash;2480 (2017).\u003c/li\u003e\n\u003cli\u003eBjerge, K., Karstoft, H., Mann, H. M. R. \u0026amp; H\u0026oslash;ye, T. T. A deep learning pipeline for time-lapse camera monitoring of insects and their floral environments. \u003cem\u003eEcol. Inform. \u003c/em\u003e\u003cstrong\u003e84\u003c/strong\u003e, 102861 (2024).\u003c/li\u003e\n\u003cli\u003eRuczyński, I., Hałat, Z., Zegarek, M., Borowik, T. \u0026amp; Dechmann, D. K. N. Camera transects as a method to monitor high temporal and spatial ephemerality of flying nocturnal insects. \u003cem\u003eMethods Ecol. Evol. \u003c/em\u003e\u003cstrong\u003e11\u003c/strong\u003e, 294\u0026ndash;302 (2020).\u003c/li\u003e\n\u003cli\u003eKay, J. \u003cem\u003eet al.\u003c/em\u003e The Caltech Fish Counting Dataset: A Benchmark for Multiple-Object Tracking and Counting. in \u003cem\u003eComputer Vision \u0026ndash; ECCV 2022\u003c/em\u003e (eds. Avidan, S., Brostow, G., Ciss\u0026eacute;, M., Farinella, G. M. \u0026amp; Hassner, T.) vol. 13668 290\u0026ndash;311 (Springer Nature Switzerland, Cham, 2022).\u003c/li\u003e\n\u003cli\u003eTabak, M. A. \u003cem\u003eet al.\u003c/em\u003e Machine learning to classify animal species in camera trap images: Applications in ecology. \u003cem\u003eMethods Ecol. Evol. \u003c/em\u003e\u003cstrong\u003e10\u003c/strong\u003e, 585\u0026ndash;590 (2019).\u003c/li\u003e\n\u003cli\u003eNorouzzadeh, M. S. \u003cem\u003eet al.\u003c/em\u003e A deep active learning system for species identification and counting in camera trap images. \u003cem\u003eMethods Ecol. Evol. \u003c/em\u003e\u003cstrong\u003e12\u003c/strong\u003e, 150\u0026ndash;161 (2021).\u003c/li\u003e\n\u003cli\u003eBjerge, K., Frigaard, C. E. \u0026amp; Karstoft, H. Object Detection of Small Insects in Time-Lapse Camera Recordings. \u003cem\u003eSensors \u003c/em\u003e\u003cstrong\u003e23\u003c/strong\u003e, 7242 (2023).\u003c/li\u003e\n\u003cli\u003eRatnayake, M. N., Dyer, A. G. \u0026amp; Dorin, A. Tracking individual honeybees among wildflower clusters with computer vision-facilitated pollinator monitoring. \u003cem\u003ePlos One \u003c/em\u003e\u003cstrong\u003e16\u003c/strong\u003e, e0239504 (2021).\u003c/li\u003e\n\u003cli\u003eRisse, B., Mangan, M., Del Pero, L. \u0026amp; Webb, B. Visual tracking of small animals in cluttered natural environments using a freely moving camera. in \u003cem\u003eProceedings of the IEEE International Conference on Computer Vision (ICCV) Workshops\u003c/em\u003e 2840\u0026ndash;2849 (2017).\u003c/li\u003e\n\u003cli\u003ePenney, H. D., Hassall, C., Skevington, J. H., Lamborn, B. \u0026amp; Sherratt, T. N. The Relationship between Morphological and Behavioral Mimicry in Hover Flies (Diptera: Syrphidae). \u003cem\u003eAm. Nat. \u003c/em\u003e\u003cstrong\u003e183\u003c/strong\u003e, 281\u0026ndash;289 (2014).\u003c/li\u003e\n\u003cli\u003eEdmunds, M. \u0026amp; Reader, T. Evidence for Batesian mimicry in a polymorphic hoverfly: evidence for Batesian mimicry. \u003cem\u003eEvolution \u003c/em\u003e\u003cstrong\u003e68\u003c/strong\u003e, 827\u0026ndash;839 (2014).\u003c/li\u003e\n\u003cli\u003eBjerge, K. \u003cem\u003eet al.\u003c/em\u003e Hierarchical classification of insects with multitask learning and anomaly detection. \u003cem\u003eEcol. Inform. \u003c/em\u003e\u003cstrong\u003e77\u003c/strong\u003e, 102278 (2023).\u003c/li\u003e\n\u003cli\u003eShepley, A., Falzon, G., Meek, P. \u0026amp; Kwan, P. Automated location invariant animal detection in camera trap images using publicly available data sources. \u003cem\u003eEcol. Evol. \u003c/em\u003e\u003cstrong\u003e11\u003c/strong\u003e, 4494\u0026ndash;4506 (2021).\u003c/li\u003e\n\u003cli\u003eNguyen, T. T. T. \u003cem\u003eet al.\u003c/em\u003e SAWIT: A small-sized animal wild image dataset with annotations. \u003cem\u003eMultimed. Tools Appl. \u003c/em\u003e\u003cstrong\u003e83\u003c/strong\u003e, 34083\u0026ndash;34108 (2023).\u003c/li\u003e\n\u003cli\u003eJain, A. \u003cem\u003eet al.\u003c/em\u003e Insect Identification in the Wild: The AMI Dataset. Preprint at https://doi.org/10.48550/ARXIV.2406.12452 (2024).\u003c/li\u003e\n\u003cli\u003eKapoor, S. \u0026amp; Narayanan, A. Leakage and the reproducibility crisis in machine-learning-based science. \u003cem\u003ePatterns \u003c/em\u003e\u003cstrong\u003e4\u003c/strong\u003e, 100804 (2023).\u003c/li\u003e\n\u003cli\u003eBernett, J. \u003cem\u003eet al.\u003c/em\u003e Guiding questions to avoid data leakage in biological machine learning applications. \u003cem\u003eNat. Methods \u003c/em\u003e\u003cstrong\u003e21\u003c/strong\u003e, 1444\u0026ndash;1453 (2024).\u003c/li\u003e\n\u003cli\u003eBeery, S., Morris, D. \u0026amp; Yang, S. Efficient Pipeline for Camera Trap Image Review. Preprint at http://arxiv.org/abs/1907.06772 (2019).\u003c/li\u003e\n\u003cli\u003eChiranjeevi, S. \u003cem\u003eet al.\u003c/em\u003e InsectNet: Real-time identification of insects using an end-to-end machine learning pipeline. \u003cem\u003ePNAS Nexus \u003c/em\u003e\u003cstrong\u003e4\u003c/strong\u003e, pgae575 (2024).\u003c/li\u003e\n\u003cli\u003eSpiesman, B. J. \u003cem\u003eet al.\u003c/em\u003e Assessing the potential for deep learning and computer vision to identify bumble bee species from images. \u003cem\u003eSci. Rep. \u003c/em\u003e\u003cstrong\u003e11\u003c/strong\u003e, 1\u0026ndash;10 (2021).\u003c/li\u003e\n\u003cli\u003eSastry, S., Khanal, S., Dhakal, A., Ahmad, A. \u0026amp; Jacobs, N. TaxaBind: A Unified Embedding Space for Ecological Applications. Preprint at https://doi.org/10.48550/ARXIV.2411.00683 (2024).\u003c/li\u003e\n\u003cli\u003eBadirli, S. \u003cem\u003eet al.\u003c/em\u003e Classifying the unknown: Insect identification with deep hierarchical Bayesian learning. \u003cem\u003eMethods Ecol. Evol. \u003c/em\u003e\u003cstrong\u003e14\u003c/strong\u003e, 1515\u0026ndash;1530 (2023).\u003c/li\u003e\n\u003cli\u003eTresson, P., Carval, D., Tixier, P. \u0026amp; Puech, W. Hierarchical Classification of Very Small Objects: Application to the Detection of Arthropod Species. \u003cem\u003eIEEE Access \u003c/em\u003e\u003cstrong\u003e9\u003c/strong\u003e, 63925\u0026ndash;63932 (2021).\u003c/li\u003e\n\u003cli\u003eStevens, S. \u003cem\u003eet al.\u003c/em\u003e BioCLIP: A Vision Foundation Model for the Tree of Life. Preprint at https://doi.org/10.48550/ARXIV.2311.18803 (2023).\u003c/li\u003e\n\u003cli\u003eChoton, J. C., Margapuri, V., Grijalva, I., Spiesman, B. J. \u0026amp; Hsu, W. H. Self-supervised Component Segmentation To Improve Object Detection and Classification For Bumblebee Identification. Preprint at https://doi.org/10.1101/2025.03.12.642757 (2025).\u003c/li\u003e\n\u003cli\u003eSeeland, M. \u0026amp; M\u0026auml;der, P. Multi-view classification with convolutional neural networks. \u003cem\u003ePLOS ONE \u003c/em\u003e\u003cstrong\u003e16\u003c/strong\u003e, e0245230 (2021).\u003c/li\u003e\n\u003cli\u003eNguyen, N.-D., Do, T., Ngo, T. D. \u0026amp; Le, D.-D. An Evaluation of Deep Learning Methods for Small Object Detection. \u003cem\u003eJ. Electr. Comput. Eng. \u003c/em\u003e\u003cstrong\u003e2020\u003c/strong\u003e, 1\u0026ndash;18 (2020).\u003c/li\u003e\n\u003cli\u003eBar, S., Levy, L., Avidan, S. \u0026amp; Holzman, R. Assessing the determinants of larval fish strike rates using computer vision. \u003cem\u003eEcol. Inform. \u003c/em\u003e\u003cstrong\u003e77\u003c/strong\u003e, 102195 (2023).\u003c/li\u003e\n\u003cli\u003eAkyon, F. C., Onur Altinuc, S. \u0026amp; Temizel, A. Slicing Aided Hyper Inference and Fine-Tuning for Small Object Detection. in \u003cem\u003e2022 IEEE International Conference on Image Processing (ICIP)\u003c/em\u003e 966\u0026ndash;970 (IEEE, Bordeaux, France, 2022). doi:10.1109/ICIP46576.2022.9897990.\u003c/li\u003e\n\u003cli\u003eTresson, P., Tixier, P., Puech, W. \u0026amp; Carval, D. Insect interaction analysis based on object detection and CNN. in \u003cem\u003e2019 IEEE 21st International Workshop on Multimedia Signal Processing (MMSP)\u003c/em\u003e 1\u0026ndash;6 (IEEE, Kuala Lumpur, Malaysia, 2019). doi:10.1109/MMSP.2019.8901798.\u003c/li\u003e\n\u003cli\u003eBjerge, K., Mann, H. M. R. \u0026amp; H\u0026oslash;ye, T. T. Real‐time insect tracking and monitoring with computer vision and deep learning. \u003cem\u003eRemote Sens. Ecol. Conserv. \u003c/em\u003e\u003cstrong\u003e8\u003c/strong\u003e, 315\u0026ndash;327 (2022).\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Pollinator detection, Automated insect monitoring, Out-of-distribution generalisation, YOLO detectors, Smartphone images, Time-lapse images","lastPublishedDoi":"10.21203/rs.3.rs-6335312/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6335312/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003ePollinating insects provide essential ecosystem services, and using time-lapse photography to automate their observation could improve monitoring efficiency. Computer vision models, trained on clear citizen science photos, can detect insects in similar images with high accuracy, but their performance in images taken using time-lapse photography is unknown. We evaluated the generalisation of three lightweight YOLO detectors (YOLOv5-nano, YOLOv5-small, YOLOv7-tiny), previously trained on citizen science images, for detecting\u0026thinsp;~\u0026thinsp;1,300 flower-visiting arthropod individuals in nearly 24,000 time-lapse images captured with a fixed smartphone setup. These field images featured unseen backgrounds and smaller arthropods than the training data. YOLOv5-small, the model with the highest number of trainable parameters, performed best, localising 91.21% of Hymenoptera and 80.69% of Diptera individuals. However, classification recall was lower (80.45% and 66.90%, respectively), partly due to Syrphidae mimicking Hymenoptera and the challenge of detecting smaller, blurrier flower visitors. This study reveals both the potential and limitations of such models for real-world automated monitoring, suggesting they work well for larger and sharply visible pollinators but need improvement for smaller, less sharp cases.\u003c/p\u003e","manuscriptTitle":"Successes and limitations of pretrained YOLO detectors applied to unseen time-lapse images for automated pollinator monitoring","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-04-07 09:55:48","doi":"10.21203/rs.3.rs-6335312/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-07-01T09:36:08+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-06-09T11:08:19+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-06-01T14:40:41+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"221973264474839937685334283370441498344","date":"2025-05-23T17:53:41+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"108511455983731770836592867338787050461","date":"2025-05-23T11:52:24+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-05-12T07:52:36+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-05-12T07:42:41+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2025-04-07T05:29:12+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-04-05T12:46:21+00:00","index":"","fulltext":""},{"type":"submitted","content":"Scientific Reports","date":"2025-03-29T16:55:36+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"d472a7e8-9bb7-4096-8d58-e92879ec06ee","owner":[],"postedDate":"April 7th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[{"id":46725076,"name":"Earth and environmental sciences/Ecology/Ecological networks"},{"id":46725077,"name":"Biological sciences/Computational biology and bioinformatics/Machine learning"},{"id":46725078,"name":"Biological sciences/Ecology"}],"tags":[],"updatedAt":"2025-08-25T16:36:54+00:00","versionOfRecord":{"articleIdentity":"rs-6335312","link":"https://doi.org/10.1038/s41598-025-16140-z","journal":{"identity":"scientific-reports","isVorOnly":false,"title":"Scientific Reports"},"publishedOn":"2025-08-21 16:29:37","publishedOnDateReadable":"August 21st, 2025"},"versionCreatedAt":"2025-04-07 09:55:48","video":"","vorDoi":"10.1038/s41598-025-16140-z","vorDoiUrl":"https://doi.org/10.1038/s41598-025-16140-z","workflowStages":[]},"version":"v1","identity":"rs-6335312","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6335312","identity":"rs-6335312","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}