AppNets: An Efficient Multi-Task Fusion Network for Comprehensive Driving Perception

AppNets: An Efficient Multi-Task Fusion Network for Comprehensive Driving Perception | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article AppNets: An Efficient Multi-Task Fusion Network for Comprehensive Driving Perception Yaohan Jia, Xuemei Chen, Zeyuan Xu, Pengfei Ren, Wenzhe Shan This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5358737/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Panoramic driving perception systems are critical for autonomous driving, as they provide essential traffic-related information. This study introduces AppNets, an efficient and effective multi-task learning framework designed for real-time panoptic driving perception. AppNets comprises an encoder for feature extraction and three decoders that concurrently perform traffic object detection, drivable area segmentation, and lane segmentation. We propose the C2fA module to enhance the model's extraction capability. To enhance our dataset, we expanded the SDExpressway dataset by adding 2,000 frames, particularly incorporating nighttime and adverse weather scenarios. Extensive experiments conducted on both the challenging BDD100K dataset and the augmented SDExpressway dataset demonstrate that AppNets achieves state-of-the-art performance, outperforming baseline models by significant margins. Specifically, on the SDExpressway dataset, AppNets attains a mean average precision (mAP) of 85.1% for traffic object detection, a mean intersection over union (mIoU) of 98.7% for drivable area segmentation, and an intersection over union (IoU) of 75.1% for lane segmentation. These results underscore the effectiveness of AppNets in complex driving scenarios, highlighting its potential for practical deployment in autonomous driving systems. the source codes are released at https://github.com/Huniki/Appnet.git multi-tasking traffic object detection lane line segmentation drivable area segmentation autonomous driving Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 Introduction The rapid development of pure vision algorithms and end-to-end deep learning technologies has made autonomous driving a focal point of research in the field of computer vision. Despite this, there are still numerous challenges in cost-effective autonomous driving systems, particularly in vision-based tasks such as object detection, image segmentation, and lane detection. Given the cost-effectiveness of camera systems, these technologies dominate the tasks of object detection and segmentation in real-world deployments. Therefore, ensuring that vehicles can effectively perceive their environment using only a single front-facing camera input has become a critical issue that needs to be addressed. Traditionally, three key tasks are considered fundamental for guiding intelligent vehicles: traffic object detection, drivable area segmentation, and lane line detection. Existing methods for object detection include Single Shot MultiBox Detector (SSD) [ 1 ] and the YOLO series [ 2 – 12 ], while semantic segmentation is addressed by networks like U-Net [ 13 ], SegNet [ 14 ], and ERNet [ 15 ]. For lane line detection, models such as LaneNet [ 16 ] and Spatial Convolutional Neural Networks (SCNN) [ 17 ] are utilized. However, using three separate networks to process the same image data stream can lead to unnecessary latency. To address this, many researchers have proposed integrating these functionalities into a single encoder-decoder architecture, where the backbone and neck collaboratively generate context for the three different tasks, such as in MultiNet [ 18 ], DLT-Net[ 19 ], YOLOP[ 20 ], and hybridnet[ 21 ]. Multitask learning networks offer a computationally efficient solution by utilizing an encoder-decoder framework where the encoder is effectively shared among different tasks. To enhance the speed and performance of our model, we optimized the encoder using a highly efficient yet lightweight multiscale feature fusion method, SimsppF[ 8 ], and adopted CSPDarknet from YOLOv8[ 10 ] as the backbone. This architecture strikes a balance between accuracy and computational cost. In the decoder, we preserved the anchor-based multiscale detection scheme from YOLOP and combined it with the Path Aggregation Network (PAN) and Feature Pyramid Network (FPN). The integration of FPN and PAN enhances feature fusion by facilitating top-down propagation of semantic features and bottom-up propagation of localization features. Additionally, we incorporated an attention mechanism before the detection head to enhance overall performance. At an input resolution of 640x640, the convergence time of our network is slightly longer than that of YOLOP but results in significant improvements in accuracy and recall. In this paper, we conduct an in-depth study of previous methods and propose an efficient multitask learning network. We expanded the SDExpressway dataset by adding 2,000 images, primarily focusing on driving scenes in night and adverse weather conditions, such as rain and fog. We validated our model on this enriched dataset. Furthermore, experiments were conducted on the challenging BDD100K dataset, demonstrating superior performance across all three tasks. The primary contributions of this work are summarized as follows: AppNets : An end-to-end perception network achieving real-time performance on the BDD100K and SDExpressway datasets. Enhanced Dataset : Augmentation of the SDExpressway dataset. Effective Training Loss Function and Strategy : Balanced and optimized multitask network training. Related Works This section will review some solutions for each individual task and then introduce current work on multi-task learning. We focus only on solutions based on deep learning. Traffic Object Detection In recent years, the rapid development of deep learning has led to the emergence of numerous object detection algorithms. These algorithms can be broadly categorized into two-stage and one-stage methods. Two-stage methods complete the detection task in two steps: the region proposal component and the object classification component. The R-CNN series[ 22 ] is the most typical example of this approach, which offers higher detection accuracy and stability but is generally slower than single-stage methods. Consequently, single-stage methods are more popular in real-time detection applications. In contrast, one-stage methods, exemplified by the SSD series and the YOLO series, excel in detection speed. The original YOLO [ 2 ] algorithm partitions the image into an S×S grid, eliminating the need for region proposals via Region Proposal Networks (RPN), which significantly accelerates the detection process. YOLOv4 [ 5 ] enhanced the network structure, activation functions, and loss functions, employing rich data augmentation techniques. YOLOv5[ 7 ] introduced the Focus structure for dimension reduction and feature map compression, along with adaptive anchor box calculation and adaptive gray padding to improve detection accuracy. YOLOX [ 6 ] introduces a decoupled detection head and the highly efficient SimSPPF, significantly accelerating network convergence. Subsequent versions, including YOLOv6 [ 8 ] and YOLOv7 [ 9 ], continue to refine backbone and neck structures, introducing efficient designs such as ELAN and the C2f module to further enhance overall performance. In addition, researchers have made attempts to improve detection accuracy. To address the issue of low detection accuracy caused by unclear image recognition at night, thermal imaging is utilized to assist in target detection and enhance overall precision [ 23 ]. EAPT [ 24 ] has been proposed to capture more comprehensive global attention information. BaGFN [ 25 ] dynamically learns the importance weights of cross features, thereby improving the model's detection performance. Drivable Area and Lane Line Segmentation Drivable area segmentation is a critical component of semantic segmentation tasks. In recent years, deep learning algorithms based on neural networks have made significant strides in semantic segmentation, often outperforming traditional segmentation methods by directly assigning recognition information to each pixel of the image. Fully Convolutional Networks (FCNs) [ 26 ] pioneered end-to-end convolutional architectures for semantic segmentation; however, their upsampling operations often lead to the loss of some image information. To overcome this, U-Net [ 27 ] employs a symmetric structure with a deeper decoder, while SegNet [ 28 ] enhances segmentation resolution by transferring max-pooling indices to the decoder. EEMask [ 29 ] divides the image uniformly according to the grid and then calculates the relevance between the grids based on the distance and grayscale values. DeepLab introduced dilated convolution, which can expand the receptive field without increasing the number of parameters, thereby improving the segmentation network's effectiveness. PSPNet[ 30 ] implemented a pyramid pooling module to aggregate background information from different scales, enhancing semantic segmentation performance. ENet [ 31 ] compressed redundant visual information to reduce the size of feature maps while substantially improving model detection speed without sacrificing significant accuracy. In the context of lane line detection, methods can be classified into anchor-based and segmentation-based approaches. Anchor-based methods, such as UFLD [ 32 ], determines whether each vertical slice of the image contains lane lines. PolyLaneNet [ 33 ] utilizes deep polynomial regression to output polynomials representing each lane line directly, avoiding a curve fitting process in the post-processing phase. LaneATT [ 34 ] emphasizes the importance of global information in inferring lane line positions, proposing an anchor-based attention mechanism to aggregate global information and enhance lane line detection capabilities. Segmentation-based methods, exemplified by LaneNet, builds a dual-branch network: a binary segmentation network responsible for outputting all lane line pixels, while an instance segmentation network assigns the detected lane line pixels to distinct lane line instances. SCNN proposed a slice convolution method, allowing spatial information to flow between neurons in the same layer, effectively maintaining the smoothness and continuity of lane lines and poles, albeit with higher computational demands. Enet-SAD [ 35 ] leverages self-attention distillation to extract deep features from shallow layers, resulting in lightweight models with enhanced segmentation performance. Multi-task Approaches Multitask learning aims to facilitate the learning of different tasks by sharing feature extraction information, thereby avoiding the computational waste of conducting separate feature extraction for each task. In this field, Mask R-CNN [ 36 ] adds a branch for predicting object masks on top of Faster R-CNN, effectively integrating instance segmentation and object detection tasks. However, these models were not specifically designed for perception tasks in autonomous driving, resulting in less-than-ideal performance in practical applications. LSNet [ 37 ] combines object detection, instance segmentation, and pose estimation into position-sensitive visual recognition tasks but demonstrates limited effectiveness in the context of autonomous driving. MultiNet demonstrated the specific relationship between detection tasks and segmentation tasks, successfully performing scene classification, vehicle detection, and drivable area segmentation. However, it overlooked the significance of lane line detection. DLT-Net further supports vehicle detection, lane line detection, and drivable area segmentation by creating context tensors among subtask decoders to facilitate information sharing. Nevertheless, this model exhibits poor performance when lane lines are disconnected. Building upon YOLOv4, YOLOP emerges as the first real-time advanced multitask perception model analyzed on the BDD100K dataset, featuring a shared encoder linked to three independent decoders for Traffic Object Detection, drivable area and lane line segmentation. Despite its advancements, there remains substantial room for optimization in the model structure regarding lane line segmentation and drivable area detection. HyBridNet leverages EfficientNet-B3 as its backbone and integrates a BiFPN structure, leading to enhanced multidimensional detection accuracy. However, despite these improvements, despite these improvements, there is still potential for optimization from the perspective of model performance. Future research could achieve further breakthroughs in the accuracy and efficiency of multi-task learning through architectural enhancements, improved feature sharing, and optimized training strategies. Methodology Based on the identified challenges, this paper presents a novel multitask learning network architecture named AppNets, as depicted in Fig. 2 . This architecture is specifically designed to efficiently address traffic object detection, drivable area segmentation, and lane detection tasks in a coordinated fashion. The following sections will detail the design of the AppNets architecture along with the optimization strategies implemented to enhance its performance across these critical tasks. Encoder Backbone Feature extraction is a critical component of multitask learning networks, directly influencing the model's performance across various tasks. Considering the superior performance of YOLOv8 in object detection, we selected Darknet53 from its backbone as the foundation for AppNets. To enhance the information extraction capabilities, we optimized the C2f module into C2fA. Compared to C2f, C2fA offers a significant advantage by employing diverse convolution kernels for processing, allowing for the parallel integration of a broader range of feature information. Furthermore, the architecture incorporates additional convolutional layers, which facilitates the extraction of more complex, deeper-level features. This design not only reduces gradient duplication during optimization but also improves both feature propagation and feature reuse, the specific structure is shown in Fig. 3 . Our experimental results demonstrate that leveraging multiple convolution kernels and multi-perception layers significantly enhances the model's ability to extract depth information from images, resulting in improved receptive fields and superior feature representation capabilities. Neck The neck is designed to fuse features generated by the backbone network. We chose to replace the original Spatial Pyramid Pooling (SPP) structure with the faster Sim Spatial Pyramid Pooling-Fast (SimSPPF) module from YOLOv6 to efficiently merge features of different scales. Additionally, we incorporated an attention layer after SimSPPF to further enhance feature extraction effectiveness. During the feature fusion process, we employed a Feature Pyramid Network (FPN) structure, alternately arranging C2f and C2fA modules to effectively combine features of different semantic levels. This design ultimately produces feature information that encompasses multiple scales and semantic hierarchies, which will be passed on to the three decoders. The SimSPPF module not only strikes a balance between speed and accuracy, but we also conducted ablation experiments to validate its performance. In the experimental section, we will provide a detailed discussion of the results obtained from these experiments. Decoder The three heads in our network are specific decoders for the three tasks. Detection Head The detection head is designed to identify various object categories, including vehicles, traffic signs, and road markings. In this study, we employ the anchor-based multi-scale detection framework utilized in YOLOP and integrates the Path Aggregation Network (PAN) and the Feature Pyramid Network (FPN). The FPN facilitates the top-down flow of semantic information, while the PAN enables effective bottom-up propagation of localization details. This synergistic combination enhances feature fusion capabilities. To enhance the detection performance of small targets, we introduced the SimAM attention mechanism before making predictions with the three heads. Unlike existing channel or spatial attention modules, SimAM focuses primarily on spatial dimension attention, assigning different weights to various spatial positions within the feature map, thereby improving the overall performance of the task. Notably, SimAM also demonstrates certain advantages in terms of computational cost. The detection head utilizes anchors with three distinct aspect ratios to predict positional offsets and scale variations of targets across multi-scale fused feature maps at each grid cell. This process yields probabilities and confidence scores for each detected category. Furthermore, we conduct ablation studies to evaluate the contribution of the attention mechanisms to model performance. Drivable Area Segmentation Head & Lane Line Segmentation Head The drivable area occupies a significant proportion of the image and has a regular distribution, making it relatively easier for the model to identify. Therefore, extracting deep features for the segmentation of the drivable area is not necessary. These deeper features may not enhance prediction performance and could complicate the convergence of the model during training. Consequently, in this study, the drivable area segmentation head is connected prior to the SimSPPF module. In contrast, lane lines are slender objects characterized by fragmented pixel distributions. As such, the lane line segmentation head accesses the lower layers of the FPN to leverage deeper features. Although both the lane line segmentation head and the drivable area segmentation head share similar structures—consisting of alternating combinations of C2f blocks and upsampling layers using nearest neighbor interpolation—key differences exist. to compensate for potential feature loss due to the shallow integration, the drivable area segmentation head employs an additional upsampling layer and introduces a C2f module at the first layer to enhance feature extraction depth. After multiple upsampling processes, the final feature maps are resized to dimensions of ( W, H, 2 ), representing the probability of each pixel in the input image corresponding to the drivable area/lane line and background. This structural design effectively improves the model's performance when handling different objects. Loss Function In our design of the multitask loss function for traffic object detection, we define the loss L to as a combination of several components: a penalized classification loss L class , a confidence loss L obj ,and a bounding box regression loss L box . This can be mathematically expressed as: $$\:\begin{array}{c}{L}_{to}={\rho\:}_{1}{L}_{class}+{\rho\:}_{2}{L}_{obj}+{\rho\:}_{3}{L}_{box}\#\left(1\right)\end{array}$$ In contrast to YOLOP's use of focal loss, we adopt the more sophisticated varifocal loss for both L class and Lobj . While focal loss uniformly down-weights negative samples to balance the contribution of foreground and background classes, varifocal loss specifically reduces the weight of negative samples while maintaining a strong emphasis on positive samples. This method is particularly advantageous in mitigating the loss contribution from well-classified examples, thereby encouraging the network to concentrate more on hard negatives. The bounding box regression loss L box employs the Distance-IoU loss, which not only considers the overlap between predicted boxes and ground truth but also incorporates the distance, scale, and aspect ratio similarity between them. For the drivable area and lane line segmentation tasks, we define the losses \(\:{L}_{da}\) 和 \(\:{L}_{ll}\) using cross-entropy loss with logits L ce . The goal here is to minimize the classification error between the network's predicted pixels and the actual targets. Given that IoU loss L IOU ​ is particularly effective for the prediction of sparse categories, such as lane lines, it is incorporated into L ll . These losses can be formulated as follows: $$\:\begin{array}{c}{L}_{da}={L}_{ce}\#\left(2\right)\end{array}$$ $$\:\begin{array}{c}{L}_{ll}={L}_{ce}+{L}_{IoU}\#\left(3\right)\end{array}$$ Summarizing, the overall loss function is expressed as: $$\:\begin{array}{c}{L}_{all}={\mu\:}_{1}{L}_{to}+{\mu\:}_{2}{L}_{da}+{\mu\:}_{3}{L}_{ll}\#\left(4\right)\end{array}$$ where the balancing factors \(\:\rho\:\) 1 , \(\:\rho\:\) 2 , \(\:\rho\:\) 3, µ1, µ2 and µ3 ​ are set to 0.5, 1.0, 0.05, 0.3, 0.2, and 0.2, respectively. This careful calibration ensures a harmonious contribution from each component of the total loss function. Experimentation and Evaluation In this project, we sought to fully demonstrate the capabilities of our model by selecting the SDExpressway dataset and the BDD100K dataset, and we expanded the SDExpressway dataset. The SDExpressway dataset (Shandong Expressway Multitask Dataset) originally comprised 4,483 training images and 1,120 testing images. As one of the primary contributors to this dataset, it primarily captures highway scenes characterized by uniform road and lane markings, unidirectional vehicle flow, and a limited variety of traffic and ground signs. Due to the characteristics of high-speed driving, the scenery is expansive and changes rapidly. In such environments, the proportion of small target objects is relatively high, and the frequency of detection is significant. The high proportion of small target detection increases the difficulty of accurate detection, while the high frequency of detection imposes stricter real-time performance requirements. However, the limited sample size of the dataset hampers the full demonstration of the model's performance. However, the limited sample size of the dataset hampers the full demonstration of the model's performance. To enhance the model's generalization capability on the SDExpressway dataset—particularly in addressing challenges related to nighttime scenes and the potential confusion of rare labels such as speed limit signs in low-light conditions—we undertook an expansion of the dataset for this project. Specifically, we increased the training set to 5,983 images by adding 1,000 nighttime images and 500 additional daytime images. Furthermore, we augmented the test set to 1,620 images by including 500 new images. All images were annotated using the LabelMe tool, capturing details regarding drivable areas, lane lines, and traffic objects. The collected images are in JPEG format, while the initial annotations for drivable areas, lane lines, and traffic objects are stored in JSON format. In our research, we primarily utilized the SDExpressway dataset for training and evaluating our network. To further validate the robustness and adaptability of our model, we also incorporated the widely recognized BDD100K dataset for the training and evaluation of the object detection component. BDD100K is a challenging public dataset comprising 100,000 frames captured from a driver’s perspective, and it is extensively used as an evaluation benchmark in autonomous driving computer vision research. The dataset supports ten distinct visual tasks and, relative to other well-known driving datasets such as Cityscapes and CamVid, BDD100K emphasizes a greater variety of weather conditions, scene locations, and lighting situations. Additionally, it provides more extensive lane division data and encompasses a wider range of driving scenarios. In line with common practices in the field, we partitioned the BDD100K dataset into three distinct subsets: a training set of 70,000 images, a validation set of 10,000 images, and a test set of 20,000 images. This strategic partitioning enables us to leverage the dataset's inherent diversity and complexity, thus allowing for a thorough assessment of the model's performance across a broad range of driving situations. Experiment settings Given the relatively limited size of the SDExpressway dataset, we not only expanded its capacity but also increased the maximum number of epochs, setting it to 300. In contrast, when utilizing the significantly larger BDD100K dataset, the maximum epochs count was reduced to 100. The image dimensions for both the SDExpressway and BDD100K datasets were standardized at 1280×720×3. All state-of-the-art (SOTA) comparison experiments and ablation studies were conducted under consistent experimental settings and evaluation metrics, with all experiments performed on an RTX 3090 GPU in an environment configured with Torch 2.0.0 and cu11.8. To enhance the detector's ability to acquire prior knowledge about objects in traffic scenes, we employed the k-means clustering algorithm to generate prior anchor boxes from all detection frames in the dataset. During the training process, we utilized the Adam optimizer with an initial learning rate of 0.001 and momentum parameters β 1​ and β 2​ set to 0.937 and 0.999, respectively. Furthermore, we adopted a warm-up and cosine annealing strategy for adjusting the learning rate, facilitating faster and more effective model convergence. We benchmarked our model's performance against several recent outstanding multitask networks, including MultiNet, DLT-Net, YOLOP, and HybridNets, on both the SDExpressway and BDD100K datasets. It is noteworthy that MultiNet and DLT-Net are designed solely for vehicle detection, which resulted in these models being evaluated exclusively on the BDD100K dataset. Furthermore, during validation, the target detection category for all models was restricted to "vehicles." Additionally, we conducted various ablation studies investigating attention mechanisms and spatial pyramid pooling (SPPF) to substantiate the superiority of our proposed AppNets. These experiments provided valuable insights into the model's characteristics and clarified the contributions of different modules to overall performance, furnishing substantial data to support the final model optimization. Evaluation metrics In this study, we employed several evaluation metrics for traffic object detection, specifically Recall (R) and mean Average Precision (mAP). For evaluating lane line detection, we utilized Accuracy (Acc) and Intersection over Union (IoU) as metrics. The assessment of drivable area segmentation was conducted using mean Intersection over Union (mIoU). Additionally, model detection speed was quantified in terms of Frames Per Second (FPS). Multi-task performance The experimental results encompass three tasks: traffic object detection, drivable area segmentation, and lane line segmentation. We present the outcomes of vehicle detection and compare our model's performance against four other models on the BDD100K and SDExpressway datasets. On the SDExpressway dataset, our detection targets include ten categories, while the BDD100K dataset focuses solely on vehicles. In terms of Recall, AppNets outperforms YOLOP by 3.4% and HybridNets by 1.6% on the SDExpressway dataset. Similarly, on the BDD100K dataset, AppNets demonstrates superior detection accuracy compared to MultiNet, DLT-Net, YOLOP, and HybridNets. Notably, AppNets exhibits significantly higher detection precision on the SDExpressway dataset than on the BDD100K dataset, primarily due to the greater presence of small objects in the SDExpressway dataset. Table 1 presents detailed information on the object detection results of YOLOP, HybridNets, MultiNet, DLT-Net and AppNets across both the SDExpressway and BDD100K datasets. These findings further substantiate the enhanced detection capabilities of our proposed method in complex traffic scenarios. Our model has shown exceptional performance on both the BDD100K and SDExpressway datasets. Although the real-time inference speed of our model is slightly lower than that of YOLOP due to the incorporation of additional techniques, its performance remains within an acceptable range following lightweight optimization. Table 1 Traffic object detection results on SDExpressway and BDD100k datasets. Bdd100k SDExpressway Network R(%) mAP50(%) mAP50:95(%) FPS R(%) mAP50(%) mAP50:95(%) FPS YOLOP(baseline) 89.9 76.3 43.1 223 92.1 77.7 43.1 231 MultiNet 81.3 60.2 33.1 51 — — — — DLT-Net 89.4 68.4 38.4 56 — — — — HybridNets 91.2 79.0 46.0 220 93.9 78.7 45.6 200 AppNets 91.7 79.5 46.5 200 95.5 85.1 53.9 181 The predictions from AppNets are more reasonable, especially in terms of detecting small objects at a distance, with fewer examples of false negatives and more accurate bounding boxes. Lane line segment results The results for lane line segmentation are presented in Table 4 , illustrating that our model outperforms several other models in this regard. Specifically, on the SDExpressway dataset, AppNets achieves an IoU metric that is 1.1% higher than that of YOLOP and 0.8% higher than HybridNets. Table 2 Performance comparison on lane line segment task. SDExpressway Network Acc(%) IoU(%) YOLOP(baseline) 90.4 74.0 HybridNets 90.8 74.3 AppNets(ours) 90.6 75.1 Drivable area segment results The results for drivable area segmentation are presented in the table. Notably, on the SDExpressway dataset, our model demonstrates a slight advantage over the other models. The mean Intersection over Union (mIoU) metric for our model is 1.7% higher than that of YOLOP and 1.5% higher than that of HybridNets, respectively. Furthermore, the accuracy (acc) of our model surpasses that of YOLOP and HybridNets by 0.3% and 0.5%, respectively Table 3 Performance comparison on drivable area segment task. SDExpressway Network Acc(%) mIoU(%) YOLOP(baseline) 98.9 97.0 HybridNets 98.7 97.2 AppNets(ours) 99.2 98.7 The red lines represent lane markings, the green areas indicate drivable regions, and the orange bounding boxes represent traffic objects. Our AppNets demonstrates a more accurate and aligned performance in detecting bounding boxes in most cases, with lane markings appearing more continuous and smoother. Ablation studies We referenced the benchmark model YOLOP and implemented several optimizations to develop AppNets. To demonstrate the effectiveness of our model, we conducted ablation studies focusing on attention mechanisms and Spatial Pyramid Pooling with a Fixed Grid (SPPF). Table 4 presents the modules used in our attention comparison experiments: DAttention[ 38 ], TripletAttention[ 39 ], and iMRB[ 40 ]. For the subsequent SPPF ablation study detailed in Table 5 , we employed SimAttention[ 41 ] as the attention module, alongside several SPPF variants: RFB[ 42 ], SPPELAN, and SPPFCSPC. All evaluation metrics discussed in this section are consistent with those previously outlined. In the attention ablation study, SimAttention that we implemented performed the best in the model, despite not achieving the highest Frames Per Second (FPS). It excelled in other critical metrics. As demonstrated in Table 5 , our implementation of SimSPPF also outperformed other configurations in the SPPF ablation experiments. Table 4 Comparison Experiments of Different Attention Modules Attention model Traffic Object Drivable Area Lane Line Speed D Triplet iM sim R(%) mAP50(%) mAP 50:0.95(%) mIou(%) Iou(%) fps √ 93.1 83.0 55.5 98.1 73.5 188 √ 94.3 83.0 54.7 98.2 70.7 173 √ 94.2 80.5 51.1 98.1 71.5 136 √ 95.2 84.2 57.1 98.2 72.8 178 ·D:Dattention, sim: Simattention, Triplet༚Tripletattention, iM༚iMRB Table 5 Comparison Experiments of Different SPPF Modules sppf model Traffic Object Drivable Area Lane Line Speed RFB cspc elan sim R(%) mAP50(%) mAP 50:0.95(%) mIou(%) Iou(%) fps √ 94.4 83.1 55.0 98.2 73.1 188 √ 95.0 82.7 55.6 98.3 73.2 181 √ 94.0 80.9 54.9 98.3 70.6 188 √ 95.2 84.0 53.7 98.4 73.7 200 ·sim:SimSPPF, elan: SPPELAN, cspc༚SPPFCSPC Conclusion and Perspective In this paper, we introduce the AppNets model and evaluate its performance on the BDD100K and SDExpressway datasets to demonstrate its effectiveness. AppNets is designed to simultaneously address three driving perception tasks: traffic object detection, drivable area segmentation, and lane line segmentation, with support for end-to-end training. To enhance the model's receptive field and feature representation, we developed a novel aggregation network structure termed C2fA. To comprehensively assess the model’s generalization capabilities and mitigate the relative scarcity of the SDExpressway dataset for highway multitasking, we collected and annotated an additional 2,000 images to enrich this dataset. On the SDExpressway and BDD100K datasets, our model has exhibited outstanding performance, surpassing existing technologies in terms of both accuracy and efficiency. While our multitask model delivers exceptional detection precision across all three tasks, it incurs higher computational costs compared to the YOLOP model. Consequently, our future efforts will concentrate on refining the model through the integration of the network heads for lane line segmentation and drivable area segmentation. This strategy is expected to streamline the model, reducing its complexity and computational requirements while simultaneously improving its accuracy. Declarations Author Contribution Xuemei Chen and Yaohan Jia wrote the main manuscript text ,Zeyuan Xu ,Pengfei Ren prepared figures,Wenze Shan prepared tables,All authors reviewed the manuscript. ACKNOWLEDGEMENTS This project would like to acknowledge supports from Key R&D Program of Shandong Province, China, 2023CXPT032, Development and Demonstration of High energy efficiency Co-processor for High-order Autonomous Driving Applications Supported by the Taishan Industrial Experts Program, Supported by the Taishan Industrial Experts Program; project ZR2023MF0766 supported by Shandong Provincial Natural Science Foundation, Research on Intelligent driving vehicle decision making based on multi-source data fusion and passenger ride experience. References Liu W et al (2016) SSD: Single Shot MultiBox Detector. In: Leibe B et al (eds) ComputerVision – ECCV 2016. Springer International Publishing, Cham, pp 21–37. https://doi.org/10.1007/978-3-319-46448-0_2 . Redmon J, Divvala SK, Girshick RB, Farhadi A (2015) You Only Look Once: Unified, Real-Time Object Detection. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 779–788 Redmon J, Farhadi A (2017) YOLO9000: Better, Faster, Stronger, in Proceedings of theIEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.6517–6525, 10.1109/CVPR.2017.690 Joseph Redmon and Ali Farhadi (2018) Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767, 2s [2, 7, 13–15, 25, 30, 31, 47–49, 61–63, 74, 75], Bochkovskiy A, Wang C-Y, Liao H-YM (2020) YOLOv4: Optimal Speed and Accuracy ofObject Detection, arXiv:2004.10934 Ge Z, Liu S, Wang F, Li Z, Sun J YOLOX: Exceeding YOLO series in 2021. arXiv preprint arXiv:2107.08430, 2021. 3 Jocher Glenn (2022) YOLOv5 release v7.0. https://github.com/ultralytics/yolov5/releases/tag/v7 . 0 Li C, Li L, Jiang H, Weng K, Geng Y, Li L, Ke Z, Li Q, Cheng M, Nie W et al (2022) YOLOv6: A single-stage object detection framework for industrial applications. arXiv preprint arXiv:2209.02976 Wang C-Y (2023) Alexey Bochkovskiy, and HongYuan Mark Liao. YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7464–7475 Jocher Glenn (2023) Yolov8. https://github.com/ultralytics/ultralytics/tree / main Wang C, Yeh I, Liao H (2024) YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information. ArXiv, abs/2402.13616 Wang A, Chen H, Liu L, Chen K, Lin Z, Han J, Ding G (2024) YOLOv10: Real-Time End-to-End Object Detection. ArXiv abs/2405.14458 : n. pag Ronneberger O, Fischer P, Brox T U-Net: Convolutional Networks for Biomedical ImageSegmentation, In: Navab N, Hornegger J, Wells W, Frangi A (eds) Medical ImageComputing and Computer-Assisted Intervention – MICCAI 2015. MICCAI 2015. LectureNotes in Computer Science, vol 9351. Springer, Cham. https://doi.org/10.1007/978-3-319-24574-4_28 Badrinarayanan V, Kendall A, Cipolla R (2017) SegNet: A Deep ConvolutionalEncoder-Decoder Architecture for Image Segmentation. IEEE Trans Pattern Anal Mach Intell 39:12, 2481–2495. https://doi.org/10.1109/TPAMI.2016.2644615 Li B, Zang J, Cao J (2021) Efficient Residual Neural Network for Semantic Segmentation,Pattern Recognit. Image Anal 31:2, 212–220. https://doi.org/10.1134/S1054661821020103 Wang Z, Ren W, Qiu Q (2018) LaneNet: Real-Time Lane Detection Networks for AutonomousDriving, arXiv:1807.01726 Pan X, Shi J, Luo P, Wang X, Tang X (2017) Spatial As Deep: Spatial CNN for Traffic SceneUnderstanding, arXiv:1712.06080 Teichmann M, Weber M, Zöllner M, Cipolla R, Urtasun R (2018) MultiNet: Real-time JointSemantic Reasoning for Autonomous Driving, in Proceedings of the IEEE IntelligentVehicles Symposium (IV), pp. 1013–1020, 10.1109/IVS.2018.8500504 Qian Y, Dolan JM, Yang M (2020) DLT-Net: Joint Detection of Drivable Areas, Lane Lines,and Traffic Objects, IEEE Transactions on Intelligent Transportation Systems, vol. 21, no.11, pp. 4670–4679, Nov. 10.1109/TITS.2019.2943777 Wu D, Liao M, Zhang W, Wang X (2022) YOLOP: You Only Look Once for Panoptic DrivingPerception, arXiv:2108.11250 Dat Vu B, Ngo, Phan H (2022) Hybridnets: End-to-end perception network. arXiv preprint arXiv:2203.09035 Ren S, He K, Girshick R, Sun J Faster r-cnn: Towards real-time object detection with region proposal networks. arXiv preprint arXiv:1506.01497, 2015. Kera SB, Tadepalli A, Ranjani JJ (2023) A paced multi-stage block-wise approach for object detection in thermal images. Vis Comput 39:2347–2363. https://doi.org/10.1007/s00371-022-02445-x Xie Z, Zhang W, Sheng B, Li P, Chen CLP (2023) BaGFN: Broad Attentive Graph Fusion Network for High-Order Feature Interactions, in IEEE Transactions on Neural Networks and Learning Systems , vol. 34, no. 8, pp. 4499–4513, Aug. 10.1109/TNNLS.2021.3116209 Lin X, Sun S, Huang W, Sheng B, Li P, Feng DD (2023) EAPT: Efficient Attention Pyramid Transformer for Image Processing, in IEEE Transactions on Multimedia , vol. 25, pp. 50–61, 10.1109/TMM.2021.3120873 Shelhamer E, Jonathan Long and, Darrell T (2014) Fully convolutional networks for semantic segmentation. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) : 3431–3440 Ronneberger O, Fischer P, Brox T (2015) U-Net: Convolutional Networks for Biomedical Image Segmentation. ArXiv abs/1505.04597 : n. pag Badrinarayanan V, Kendall A, Cipolla R (2015) SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation. IEEE Trans Pattern Anal Mach Intell 39:2481–2495 Gao Y, Qi Z, Zhao D (2023) Edge-enhanced instance segmentation by grid regions of interest. Vis Comput 39:1137–1148. https://doi.org/10.1007/s00371-021-02393-y Zhao H, Shi J, Qi X, Wang X, Jia J (2017) Pyramid Scene Parsing Network, in Proceedingsof the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.6230–6239, 10.1109/CVPR.2017.660 Paszke A, Chaurasia A, Kim S, Culurciello E (2016) ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation. ArXiv abs/1606.02147 : n. pag Qin Z, Wang H, Li X (2020) Ultra Fast Structure-aware Deep Lane Detection. ArXiv abs/2004 11757:pag Tabelini L, Berriel R, Paixão TM, Badue CS, Souza AF, Oliveira-Santos T (2020) PolyLaneNet: Lane Estimation via Deep Polynomial Regression. 2020 25th International Conference on Pattern Recognition (ICPR) , 6150–6156 Tabelini L, Berriel R, Paixão TM, Badue CS, Souza AF, Oliveira-Santos T (2020) Keep your Eyes on the Lane: Real-time Attention-guided Lane Detection. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 294–302 Hou Y, Ma Z, Liu C, Loy CC (2019) Learning Lightweight Lane Detection CNNs by SelfAttention Distillation, in Proceedings of the IEEE/CVF International Conference onComputer Vision (ICCV), pp. 1013–1021, 10.1109/ICCV.2019.00110 He K, Gkioxari G (2017) Piotr Dollár and Ross B. Girshick. Mask R-CNN Kaiwen Duan L, Xie H, Qi S, Bai Q, Huang, Tian Q (2021) Location-sensitive visual recognition with cross-iou loss. arXiv preprint arXiv:2104.04899 Xia Z, Pan X, Song S, Li LE, Huang G (2022) Vision Transformer with Deformable Attention. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) : 4784–4793 Misra 39, Nalamada D, Arasanipalai T, A.U., Hou Q (2020) Rotate to Attend: Convolutional Triplet Attention Module. 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) , 3138–3147 Zhang J, Li X, Li J, Liu L, Xue Z, Zhang B, Jiang Z, Huang T, Wang Y, Wang C (2023) Rethinking Mobile Block for Efficient Attention-based Models. IEEE/CVF International Conference on Computer Vision (ICCV) (2023): 1389–1400 Yang L, Zhang R-Y, Li L, Xie X (2021) SimAM: A Simple, Parameter-Free Attention Module for Convolutional Neural Networks. International Conference on Machine Learning Liu S et al (2017) Receptive Field Block Net for Accurate and Fast Object Detection. European Conference on Computer Vision Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-5358737","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":373243367,"identity":"b43bcb14-d907-4ad9-bf4c-88f202be7ec8","order_by":0,"name":"Yaohan Jia","email":"","orcid":"","institution":"Beijing Institute of Technology","correspondingAuthor":false,"prefix":"","firstName":"Yaohan","middleName":"","lastName":"Jia","suffix":""},{"id":373243368,"identity":"82200f6a-a258-4261-a412-88e089dc522c","order_by":1,"name":"Xuemei Chen","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAz0lEQVRIiWNgGAWjYDACCQhV38/MfODAhwoStDDObGdLPDjjDClaNpznMT7M20KEDv7Zzcce87bZMEs283w4wNvAIM8vdoCAJXeOpRvObEtj42fm3XBAcgeD4czZCfi1GEjkmEl8bDvMI9kM1GJ4hiHB4DZBLfnfJBLb/ksYHOZ5cCCxjSgtOWxAWw4YALUwHDhIjBaJG2lmkjPOJSdINrMZHGw4I0HYL/wzkp9J85TZJfDzH378+U+FjTy/NAEtGLaSpnwUjIJRMApGAXYAAMMrRLxYILPLAAAAAElFTkSuQmCC","orcid":"","institution":"Beijing Institute of Technology","correspondingAuthor":true,"prefix":"","firstName":"Xuemei","middleName":"","lastName":"Chen","suffix":""},{"id":373243369,"identity":"52f93b05-3f7d-41a8-951d-be6f558dc21c","order_by":2,"name":"Zeyuan Xu","email":"","orcid":"","institution":"Beijing Institute of Technology","correspondingAuthor":false,"prefix":"","firstName":"Zeyuan","middleName":"","lastName":"Xu","suffix":""},{"id":373243370,"identity":"b71f5ffd-b7d1-405e-8061-02128b52e1c8","order_by":3,"name":"Pengfei Ren","email":"","orcid":"","institution":"Beijing Institute of Technology","correspondingAuthor":false,"prefix":"","firstName":"Pengfei","middleName":"","lastName":"Ren","suffix":""},{"id":373243371,"identity":"5b56d8c4-3403-4cee-81c5-e53194ca6524","order_by":4,"name":"Wenzhe Shan","email":"","orcid":"","institution":"Beijing Institute of Technology","correspondingAuthor":false,"prefix":"","firstName":"Wenzhe","middleName":"","lastName":"Shan","suffix":""}],"badges":[],"createdAt":"2024-10-30 06:23:24","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-5358737/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-5358737/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":68845727,"identity":"811e6a4a-1930-4770-9112-00817d360858","added_by":"auto","created_at":"2024-11-12 15:55:39","extension":"jpeg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":124461,"visible":true,"origin":"","legend":"\u003cp\u003eResults of AppNets inference. The proposed network performs three tasks: traffic object detection, drivable area segmentation, and lane line segmentation. Green areas indicate the drivable areas, blue lines represent lane lines, and boxes indicate traffic objects.\u003c/p\u003e","description":"","filename":"floatimage1.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-5358737/v1/d49bd973d2d5caca65d7433c.jpeg"},{"id":68845021,"identity":"da6b38cb-23ea-4b89-9ab8-f7e2208e0c0b","added_by":"auto","created_at":"2024-11-12 15:47:39","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":315716,"visible":true,"origin":"","legend":"\u003cp\u003eStructural feature map of AppNets\u003c/p\u003e","description":"","filename":"floatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-5358737/v1/a7dd4322008f393e051af012.png"},{"id":68844608,"identity":"b7b316b8-f30e-4f4e-ae8c-f90318bbf261","added_by":"auto","created_at":"2024-11-12 15:39:39","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":47835,"visible":true,"origin":"","legend":"\u003cp\u003eThe structure of the bottleneck in C2fA.\u003c/p\u003e","description":"","filename":"floatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-5358737/v1/1ebeb93d4bf1f636bb2c5166.png"},{"id":68844612,"identity":"ef93f54b-8628-4584-a59b-286662a52b57","added_by":"auto","created_at":"2024-11-12 15:39:39","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":126877,"visible":true,"origin":"","legend":"\u003cp\u003eDataset comparison between SDExpressway+ and SDExpressway.\u003c/p\u003e","description":"","filename":"floatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-5358737/v1/8782299978b86092913f420e.png"},{"id":68845023,"identity":"f3d1d080-c0d7-4880-abda-3f0c2ff9f0b3","added_by":"auto","created_at":"2024-11-12 15:47:39","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":308415,"visible":true,"origin":"","legend":"\u003cp\u003eVisualization of traffic object detection results on the SDExpressway dataset for YOLOP (left), HybridNets (middle), and AppNets (right). a refers to the day-time result, b refers to the night-time result.\u003c/p\u003e","description":"","filename":"5.png","url":"https://assets-eu.researchsquare.com/files/rs-5358737/v1/f8ea5e4e8b578b530876c7a8.png"},{"id":68844613,"identity":"e25bcd2d-6568-4ee0-bb5f-9b25eb5a3bb0","added_by":"auto","created_at":"2024-11-12 15:39:39","extension":"jpeg","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":287090,"visible":true,"origin":"","legend":"\u003cp\u003eVisualization of results on the BDD100K dataset.\u003c/p\u003e","description":"","filename":"floatimage8.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-5358737/v1/4c71aa99e95ed6d07e12fa91.jpeg"},{"id":68844609,"identity":"bacf65d8-a354-408b-976b-92ddf91ad327","added_by":"auto","created_at":"2024-11-12 15:39:39","extension":"png","order_by":7,"title":"Figure 7","display":"","copyAsset":false,"role":"figure","size":248101,"visible":true,"origin":"","legend":"\u003cp\u003eThe multitask visualization detection results for YOLOP (left), HybridNets (middle), and AppNets(right).\u003c/p\u003e","description":"","filename":"7.png","url":"https://assets-eu.researchsquare.com/files/rs-5358737/v1/79e84a8dfb0135980fa76ab5.png"},{"id":78594948,"identity":"10574ec2-c56e-4806-8e62-4198fc032c8e","added_by":"auto","created_at":"2025-03-16 03:46:21","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":2294556,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5358737/v1/03edd40e-66f6-4657-9e13-9d3a9a58ebc4.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"AppNets: An Efficient Multi-Task Fusion Network for Comprehensive Driving Perception","fulltext":[{"header":"Introduction","content":"\u003cp\u003eThe rapid development of pure vision algorithms and end-to-end deep learning technologies has made autonomous driving a focal point of research in the field of computer vision. Despite this, there are still numerous challenges in cost-effective autonomous driving systems, particularly in vision-based tasks such as object detection, image segmentation, and lane detection. Given the cost-effectiveness of camera systems, these technologies dominate the tasks of object detection and segmentation in real-world deployments. Therefore, ensuring that vehicles can effectively perceive their environment using only a single front-facing camera input has become a critical issue that needs to be addressed.\u003c/p\u003e \u003cp\u003eTraditionally, three key tasks are considered fundamental for guiding intelligent vehicles: traffic object detection, drivable area segmentation, and lane line detection. Existing methods for object detection include Single Shot MultiBox Detector (SSD) [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e] and the YOLO series [\u003cspan additionalcitationids=\"CR3 CR4 CR5 CR6 CR7 CR8 CR9 CR10 CR11\" citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e], while semantic segmentation is addressed by networks like U-Net [\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e], SegNet [\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e], and ERNet [\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e]. For lane line detection, models such as LaneNet [\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e] and Spatial Convolutional Neural Networks (SCNN) [\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e] are utilized. However, using three separate networks to process the same image data stream can lead to unnecessary latency.\u003c/p\u003e \u003cp\u003eTo address this, many researchers have proposed integrating these functionalities into a single encoder-decoder architecture, where the backbone and neck collaboratively generate context for the three different tasks, such as in MultiNet [\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e], DLT-Net[\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e], YOLOP[\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e], and hybridnet[\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e]. Multitask learning networks offer a computationally efficient solution by utilizing an encoder-decoder framework where the encoder is effectively shared among different tasks.\u003c/p\u003e \u003cp\u003eTo enhance the speed and performance of our model, we optimized the encoder using a highly efficient yet lightweight multiscale feature fusion method, SimsppF[\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e], and adopted CSPDarknet from YOLOv8[\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e] as the backbone. This architecture strikes a balance between accuracy and computational cost. In the decoder, we preserved the anchor-based multiscale detection scheme from YOLOP and combined it with the Path Aggregation Network (PAN) and Feature Pyramid Network (FPN). The integration of FPN and PAN enhances feature fusion by facilitating top-down propagation of semantic features and bottom-up propagation of localization features. Additionally, we incorporated an attention mechanism before the detection head to enhance overall performance.\u003c/p\u003e \u003cp\u003eAt an input resolution of 640x640, the convergence time of our network is slightly longer than that of YOLOP but results in significant improvements in accuracy and recall.\u003c/p\u003e \u003cp\u003eIn this paper, we conduct an in-depth study of previous methods and propose an efficient multitask learning network. We expanded the SDExpressway dataset by adding 2,000 images, primarily focusing on driving scenes in night and adverse weather conditions, such as rain and fog. We validated our model on this enriched dataset. Furthermore, experiments were conducted on the challenging BDD100K dataset, demonstrating superior performance across all three tasks.\u003c/p\u003e \u003cp\u003eThe primary contributions of this work are summarized as follows:\u003c/p\u003e \u003cp\u003e \u003cul\u003e \u003cli\u003e \u003cp\u003e \u003cb\u003eAppNets\u003c/b\u003e: An end-to-end perception network achieving real-time performance on the BDD100K and SDExpressway datasets.\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003e \u003cb\u003eEnhanced Dataset\u003c/b\u003e: Augmentation of the SDExpressway dataset.\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003e \u003cb\u003eEffective Training Loss Function and Strategy\u003c/b\u003e: Balanced and optimized multitask network training.\u003c/p\u003e \u003c/li\u003e \u003c/ul\u003e \n\u003ch3\u003eRelated Works\u003c/h3\u003e\n\u003cp\u003eThis section will review some solutions for each individual task and then introduce current work on multi-task learning. We focus only on solutions based on deep learning.\u003c/p\u003e \u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003eTraffic Object Detection\u003c/h2\u003e \u003cp\u003eIn recent years, the rapid development of deep learning has led to the emergence of numerous object detection algorithms. These algorithms can be broadly categorized into two-stage and one-stage methods. Two-stage methods complete the detection task in two steps: the region proposal component and the object classification component. The R-CNN series[\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e] is the most typical example of this approach, which offers higher detection accuracy and stability but is generally slower than single-stage methods. Consequently, single-stage methods are more popular in real-time detection applications.\u003c/p\u003e \u003cp\u003eIn contrast, one-stage methods, exemplified by the SSD series and the YOLO series, excel in detection speed. The original YOLO [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e] algorithm partitions the image into an S\u0026times;S grid, eliminating the need for region proposals via Region Proposal Networks (RPN), which significantly accelerates the detection process. YOLOv4 [\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e] enhanced the network structure, activation functions, and loss functions, employing rich data augmentation techniques. YOLOv5[\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e] introduced the Focus structure for dimension reduction and feature map compression, along with adaptive anchor box calculation and adaptive gray padding to improve detection accuracy. YOLOX [\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e] introduces a decoupled detection head and the highly efficient SimSPPF, significantly accelerating network convergence. Subsequent versions, including YOLOv6 [\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e] and YOLOv7 [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e], continue to refine backbone and neck structures, introducing efficient designs such as ELAN and the C2f module to further enhance overall performance.\u003c/p\u003e \u003cp\u003eIn addition, researchers have made attempts to improve detection accuracy. To address the issue of low detection accuracy caused by unclear image recognition at night, thermal imaging is utilized to assist in target detection and enhance overall precision [\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e]. EAPT [\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e] has been proposed to capture more comprehensive global attention information. BaGFN [\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e] dynamically learns the importance weights of cross features, thereby improving the model's detection performance.\u003c/p\u003e \u003c/div\u003e\n\u003ch3\u003eDrivable Area and Lane Line Segmentation\u003c/h3\u003e\n\u003cp\u003eDrivable area segmentation is a critical component of semantic segmentation tasks. In recent years, deep learning algorithms based on neural networks have made significant strides in semantic segmentation, often outperforming traditional segmentation methods by directly assigning recognition information to each pixel of the image. Fully Convolutional Networks (FCNs) [\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e] pioneered end-to-end convolutional architectures for semantic segmentation; however, their upsampling operations often lead to the loss of some image information. To overcome this, U-Net [\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e] employs a symmetric structure with a deeper decoder, while SegNet [\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e] enhances segmentation resolution by transferring max-pooling indices to the decoder. EEMask [\u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e] divides the image uniformly according to the grid and then calculates the relevance between the grids based on the distance and grayscale values. DeepLab introduced dilated convolution, which can expand the receptive field without increasing the number of parameters, thereby improving the segmentation network's effectiveness. PSPNet[\u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e] implemented a pyramid pooling module to aggregate background information from different scales, enhancing semantic segmentation performance. ENet [\u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e] compressed redundant visual information to reduce the size of feature maps while substantially improving model detection speed without sacrificing significant accuracy.\u003c/p\u003e \u003cp\u003eIn the context of lane line detection, methods can be classified into anchor-based and segmentation-based approaches. Anchor-based methods, such as UFLD [\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e], determines whether each vertical slice of the image contains lane lines. PolyLaneNet [\u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e] utilizes deep polynomial regression to output polynomials representing each lane line directly, avoiding a curve fitting process in the post-processing phase. LaneATT [\u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e] emphasizes the importance of global information in inferring lane line positions, proposing an anchor-based attention mechanism to aggregate global information and enhance lane line detection capabilities.\u003c/p\u003e \u003cp\u003eSegmentation-based methods, exemplified by LaneNet, builds a dual-branch network: a binary segmentation network responsible for outputting all lane line pixels, while an instance segmentation network assigns the detected lane line pixels to distinct lane line instances. SCNN proposed a slice convolution method, allowing spatial information to flow between neurons in the same layer, effectively maintaining the smoothness and continuity of lane lines and poles, albeit with higher computational demands. Enet-SAD [\u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e35\u003c/span\u003e] leverages self-attention distillation to extract deep features from shallow layers, resulting in lightweight models with enhanced segmentation performance.\u003c/p\u003e\n\u003ch3\u003eMulti-task Approaches\u003c/h3\u003e\n\u003cp\u003eMultitask learning aims to facilitate the learning of different tasks by sharing feature extraction information, thereby avoiding the computational waste of conducting separate feature extraction for each task. In this field, Mask R-CNN [\u003cspan citationid=\"CR36\" class=\"CitationRef\"\u003e36\u003c/span\u003e] adds a branch for predicting object masks on top of Faster R-CNN, effectively integrating instance segmentation and object detection tasks. However, these models were not specifically designed for perception tasks in autonomous driving, resulting in less-than-ideal performance in practical applications.\u003c/p\u003e \u003cp\u003eLSNet [\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e] combines object detection, instance segmentation, and pose estimation into position-sensitive visual recognition tasks but demonstrates limited effectiveness in the context of autonomous driving. MultiNet demonstrated the specific relationship between detection tasks and segmentation tasks, successfully performing scene classification, vehicle detection, and drivable area segmentation. However, it overlooked the significance of lane line detection. DLT-Net further supports vehicle detection, lane line detection, and drivable area segmentation by creating context tensors among subtask decoders to facilitate information sharing. Nevertheless, this model exhibits poor performance when lane lines are disconnected.\u003c/p\u003e \u003cp\u003eBuilding upon YOLOv4, YOLOP emerges as the first real-time advanced multitask perception model analyzed on the BDD100K dataset, featuring a shared encoder linked to three independent decoders for Traffic Object Detection, drivable area and lane line segmentation. Despite its advancements, there remains substantial room for optimization in the model structure regarding lane line segmentation and drivable area detection.\u003c/p\u003e \u003cp\u003eHyBridNet leverages EfficientNet-B3 as its backbone and integrates a BiFPN structure, leading to enhanced multidimensional detection accuracy. However, despite these improvements, despite these improvements, there is still potential for optimization from the perspective of model performance. Future research could achieve further breakthroughs in the accuracy and efficiency of multi-task learning through architectural enhancements, improved feature sharing, and optimized training strategies.\u003c/p\u003e"},{"header":"Methodology","content":"\u003cp\u003eBased on the identified challenges, this paper presents a novel multitask learning network architecture named AppNets, as depicted in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e. This architecture is specifically designed to efficiently address traffic object detection, drivable area segmentation, and lane detection tasks in a coordinated fashion. The following sections will detail the design of the AppNets architecture along with the optimization strategies implemented to enhance its performance across these critical tasks.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e\n\u003ch3\u003eEncoder\u003c/h3\u003e\n\u003cdiv id=\"Sec8\" class=\"Section2\"\u003e \u003ch2\u003eBackbone\u003c/h2\u003e \u003cp\u003eFeature extraction is a critical component of multitask learning networks, directly influencing the model's performance across various tasks. Considering the superior performance of YOLOv8 in object detection, we selected Darknet53 from its backbone as the foundation for AppNets. To enhance the information extraction capabilities, we optimized the C2f module into C2fA.\u003c/p\u003e \u003cp\u003eCompared to C2f, C2fA offers a significant advantage by employing diverse convolution kernels for processing, allowing for the parallel integration of a broader range of feature information. Furthermore, the architecture incorporates additional convolutional layers, which facilitates the extraction of more complex, deeper-level features. This design not only reduces gradient duplication during optimization but also improves both feature propagation and feature reuse, the specific structure is shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e. Our experimental results demonstrate that leveraging multiple convolution kernels and multi-perception layers significantly enhances the model's ability to extract depth information from images, resulting in improved receptive fields and superior feature representation capabilities.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e\n\u003ch3\u003eNeck\u003c/h3\u003e\n\u003cp\u003eThe neck is designed to fuse features generated by the backbone network. We chose to replace the original Spatial Pyramid Pooling (SPP) structure with the faster Sim Spatial Pyramid Pooling-Fast (SimSPPF) module from YOLOv6 to efficiently merge features of different scales. Additionally, we incorporated an attention layer after SimSPPF to further enhance feature extraction effectiveness.\u003c/p\u003e \u003cp\u003eDuring the feature fusion process, we employed a Feature Pyramid Network (FPN) structure, alternately arranging C2f and C2fA modules to effectively combine features of different semantic levels. This design ultimately produces feature information that encompasses multiple scales and semantic hierarchies, which will be passed on to the three decoders.\u003c/p\u003e \u003cp\u003eThe SimSPPF module not only strikes a balance between speed and accuracy, but we also conducted ablation experiments to validate its performance. In the experimental section, we will provide a detailed discussion of the results obtained from these experiments.\u003c/p\u003e\n\u003ch3\u003eDecoder\u003c/h3\u003e\n\u003cp\u003eThe three heads in our network are specific decoders for the three tasks.\u003c/p\u003e \u003cdiv id=\"Sec11\" class=\"Section2\"\u003e \u003ch2\u003eDetection Head\u003c/h2\u003e \u003cp\u003eThe detection head is designed to identify various object categories, including vehicles, traffic signs, and road markings. In this study, we employ the anchor-based multi-scale detection framework utilized in YOLOP and integrates the Path Aggregation Network (PAN) and the Feature Pyramid Network (FPN). The FPN facilitates the top-down flow of semantic information, while the PAN enables effective bottom-up propagation of localization details. This synergistic combination enhances feature fusion capabilities.\u003c/p\u003e \u003cp\u003eTo enhance the detection performance of small targets, we introduced the SimAM attention mechanism before making predictions with the three heads. Unlike existing channel or spatial attention modules, SimAM focuses primarily on spatial dimension attention, assigning different weights to various spatial positions within the feature map, thereby improving the overall performance of the task. Notably, SimAM also demonstrates certain advantages in terms of computational cost.\u003c/p\u003e \u003cp\u003eThe detection head utilizes anchors with three distinct aspect ratios to predict positional offsets and scale variations of targets across multi-scale fused feature maps at each grid cell. This process yields probabilities and confidence scores for each detected category. Furthermore, we conduct ablation studies to evaluate the contribution of the attention mechanisms to model performance.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec12\" class=\"Section2\"\u003e \u003ch2\u003eDrivable Area Segmentation Head \u0026amp; Lane Line Segmentation Head\u003c/h2\u003e \u003cp\u003eThe drivable area occupies a significant proportion of the image and has a regular distribution, making it relatively easier for the model to identify. Therefore, extracting deep features for the segmentation of the drivable area is not necessary. These deeper features may not enhance prediction performance and could complicate the convergence of the model during training. Consequently, in this study, the drivable area segmentation head is connected prior to the SimSPPF module.\u003c/p\u003e \u003cp\u003eIn contrast, lane lines are slender objects characterized by fragmented pixel distributions. As such, the lane line segmentation head accesses the lower layers of the FPN to leverage deeper features. Although both the lane line segmentation head and the drivable area segmentation head share similar structures—consisting of alternating combinations of C2f blocks and upsampling layers using nearest neighbor interpolation—key differences exist. to compensate for potential feature loss due to the shallow integration, the drivable area segmentation head employs an additional upsampling layer and introduces a C2f module at the first layer to enhance feature extraction depth.\u003c/p\u003e \u003cp\u003eAfter multiple upsampling processes, the final feature maps are resized to dimensions of (\u003cem\u003eW, H, 2\u003c/em\u003e), representing the probability of each pixel in the input image corresponding to the drivable area/lane line and background. This structural design effectively improves the model's performance when handling different objects.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec13\" class=\"Section2\"\u003e \u003ch2\u003eLoss Function\u003c/h2\u003e \u003cp\u003eIn our design of the multitask loss function for traffic object detection, we define the loss \u003cem\u003eL\u003c/em\u003e\u003csub\u003e\u003cem\u003eto\u003c/em\u003e\u003c/sub\u003e as a combination of several components: a penalized classification loss \u003cem\u003eL\u003c/em\u003e\u003csub\u003e\u003cem\u003eclass\u003c/em\u003e\u003c/sub\u003e, a confidence loss \u003cem\u003eL\u003c/em\u003e\u003csub\u003e\u003cem\u003eobj\u003c/em\u003e\u003c/sub\u003e ,and a bounding box regression loss \u003cem\u003eL\u003c/em\u003e\u003csub\u003e\u003cem\u003ebox\u003c/em\u003e\u003c/sub\u003e. This can be mathematically expressed as:\u003c/p\u003e\u003cdiv id=\"Equa\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equa\" name=\"EquationSource\"\u003e\n$$\\:\\begin{array}{c}{L}_{to}={\\rho\\:}_{1}{L}_{class}+{\\rho\\:}_{2}{L}_{obj}+{\\rho\\:}_{3}{L}_{box}\\#\\left(1\\right)\\end{array}$$\u003c/div\u003e\u003c/div\u003e\u003cp\u003e\u003c/p\u003e \u003cp\u003eIn contrast to YOLOP's use of focal loss, we adopt the more sophisticated varifocal loss for both \u003cem\u003eL\u003c/em\u003e\u003csub\u003e\u003cem\u003eclass\u003c/em\u003e\u003c/sub\u003e and \u003cem\u003eLobj\u003c/em\u003e. While focal loss uniformly down-weights negative samples to balance the contribution of foreground and background classes, varifocal loss specifically reduces the weight of negative samples while maintaining a strong emphasis on positive samples. This method is particularly advantageous in mitigating the loss contribution from well-classified examples, thereby encouraging the network to concentrate more on hard negatives.\u003c/p\u003e \u003cp\u003eThe bounding box regression loss \u003cem\u003eL\u003c/em\u003e\u003csub\u003e\u003cem\u003ebox\u003c/em\u003e\u003c/sub\u003e employs the Distance-IoU loss, which not only considers the overlap between predicted boxes and ground truth but also incorporates the distance, scale, and aspect ratio similarity between them.\u003c/p\u003e \u003cp\u003eFor the drivable area and lane line segmentation tasks, we define the losses \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{L}_{da}\\)\u003c/span\u003e\u003c/span\u003e和\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{L}_{ll}\\)\u003c/span\u003e\u003c/span\u003eusing cross-entropy loss with logits \u003cem\u003eL\u003c/em\u003e\u003csub\u003e\u003cem\u003ece\u003c/em\u003e\u003c/sub\u003e. The goal here is to minimize the classification error between the network's predicted pixels and the actual targets. Given that IoU loss \u003cem\u003eL\u003c/em\u003e\u003csub\u003e\u003cem\u003eIOU\u003c/em\u003e\u003c/sub\u003e ​ is particularly effective for the prediction of sparse categories, such as lane lines, it is incorporated into \u003cem\u003eL\u003c/em\u003e\u003csub\u003e\u003cem\u003ell\u003c/em\u003e\u003c/sub\u003e. These losses can be formulated as follows:\u003c/p\u003e\u003cdiv id=\"Equb\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equb\" name=\"EquationSource\"\u003e\n$$\\:\\begin{array}{c}{L}_{da}={L}_{ce}\\#\\left(2\\right)\\end{array}$$\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Equc\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equc\" name=\"EquationSource\"\u003e\n$$\\:\\begin{array}{c}{L}_{ll}={L}_{ce}+{L}_{IoU}\\#\\left(3\\right)\\end{array}$$\u003c/div\u003e\u003c/div\u003e\u003cp\u003e\u003c/p\u003e \u003cp\u003eSummarizing, the overall loss function is expressed as:\u003c/p\u003e\u003cdiv id=\"Equd\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equd\" name=\"EquationSource\"\u003e\n$$\\:\\begin{array}{c}{L}_{all}={\\mu\\:}_{1}{L}_{to}+{\\mu\\:}_{2}{L}_{da}+{\\mu\\:}_{3}{L}_{ll}\\#\\left(4\\right)\\end{array}$$\u003c/div\u003e\u003c/div\u003e\u003cp\u003e\u003c/p\u003e \u003cp\u003ewhere the balancing factors \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\rho\\:\\)\u003c/span\u003e\u003c/span\u003e\u003cem\u003e1\u003c/em\u003e, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\rho\\:\\)\u003c/span\u003e\u003c/span\u003e\u003cem\u003e2\u003c/em\u003e, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\rho\\:\\)\u003c/span\u003e\u003c/span\u003e\u003cem\u003e3, µ1, µ2\u003c/em\u003e and \u003cem\u003eµ3\u003c/em\u003e​ are set to 0.5, 1.0, 0.05, 0.3, 0.2, and 0.2, respectively. This careful calibration ensures a harmonious contribution from each component of the total loss function.\u003c/p\u003e \u003c/div\u003e"},{"header":"Experimentation and Evaluation","content":"\u003cp\u003eIn this project, we sought to fully demonstrate the capabilities of our model by selecting the SDExpressway dataset and the BDD100K dataset, and we expanded the SDExpressway dataset.\u003c/p\u003e\u003cp\u003eThe SDExpressway dataset (Shandong Expressway Multitask Dataset) originally comprised 4,483 training images and 1,120 testing images. As one of the primary contributors to this dataset, it primarily captures highway scenes characterized by uniform road and lane markings, unidirectional vehicle flow, and a limited variety of traffic and ground signs. Due to the characteristics of high-speed driving, the scenery is expansive and changes rapidly. In such environments, the proportion of small target objects is relatively high, and the frequency of detection is significant. The high proportion of small target detection increases the difficulty of accurate detection, while the high frequency of detection imposes stricter real-time performance requirements. However, the limited sample size of the dataset hampers the full demonstration of the model's performance. However, the limited sample size of the dataset hampers the full demonstration of the model's performance.\u003c/p\u003e\u003cp\u003eTo enhance the model's generalization capability on the SDExpressway dataset—particularly in addressing challenges related to nighttime scenes and the potential confusion of rare labels such as speed limit signs in low-light conditions—we undertook an expansion of the dataset for this project. Specifically, we increased the training set to 5,983 images by adding 1,000 nighttime images and 500 additional daytime images. Furthermore, we augmented the test set to 1,620 images by including 500 new images.\u003c/p\u003e\u003cp\u003eAll images were annotated using the LabelMe tool, capturing details regarding drivable areas, lane lines, and traffic objects. The collected images are in JPEG format, while the initial annotations for drivable areas, lane lines, and traffic objects are stored in JSON format.\u003c/p\u003e\u003cp\u003e \u003c/p\u003e\u003cp\u003eIn our research, we primarily utilized the SDExpressway dataset for training and evaluating our network. To further validate the robustness and adaptability of our model, we also incorporated the widely recognized BDD100K dataset for the training and evaluation of the object detection component.\u003c/p\u003e\u003cp\u003eBDD100K is a challenging public dataset comprising 100,000 frames captured from a driver’s perspective, and it is extensively used as an evaluation benchmark in autonomous driving computer vision research. The dataset supports ten distinct visual tasks and, relative to other well-known driving datasets such as Cityscapes and CamVid, BDD100K emphasizes a greater variety of weather conditions, scene locations, and lighting situations. Additionally, it provides more extensive lane division data and encompasses a wider range of driving scenarios.\u003c/p\u003e\u003cp\u003eIn line with common practices in the field, we partitioned the BDD100K dataset into three distinct subsets: a training set of 70,000 images, a validation set of 10,000 images, and a test set of 20,000 images. This strategic partitioning enables us to leverage the dataset's inherent diversity and complexity, thus allowing for a thorough assessment of the model's performance across a broad range of driving situations.\u003c/p\u003e\u003ch2\u003eExperiment settings\u003c/h2\u003e\u003cp\u003eGiven the relatively limited size of the SDExpressway dataset, we not only expanded its capacity but also increased the maximum number of epochs, setting it to 300. In contrast, when utilizing the significantly larger BDD100K dataset, the maximum epochs count was reduced to 100. The image dimensions for both the SDExpressway and BDD100K datasets were standardized at 1280×720×3. All state-of-the-art (SOTA) comparison experiments and ablation studies were conducted under consistent experimental settings and evaluation metrics, with all experiments performed on an RTX 3090 GPU in an environment configured with Torch 2.0.0 and cu11.8.\u003c/p\u003e\u003cp\u003eTo enhance the detector's ability to acquire prior knowledge about objects in traffic scenes, we employed the k-means clustering algorithm to generate prior anchor boxes from all detection frames in the dataset. During the training process, we utilized the Adam optimizer with an initial learning rate of 0.001 and momentum parameters \u003cem\u003eβ\u003c/em\u003e1​ and \u003cem\u003eβ\u003c/em\u003e2​ set to 0.937 and 0.999, respectively. Furthermore, we adopted a warm-up and cosine annealing strategy for adjusting the learning rate, facilitating faster and more effective model convergence.\u003c/p\u003e\u003cp\u003eWe benchmarked our model's performance against several recent outstanding multitask networks, including MultiNet, DLT-Net, YOLOP, and HybridNets, on both the SDExpressway and BDD100K datasets. It is noteworthy that MultiNet and DLT-Net are designed solely for vehicle detection, which resulted in these models being evaluated exclusively on the BDD100K dataset. Furthermore, during validation, the target detection category for all models was restricted to \"vehicles.\"\u003c/p\u003e\u003cp\u003eAdditionally, we conducted various ablation studies investigating attention mechanisms and spatial pyramid pooling (SPPF) to substantiate the superiority of our proposed AppNets. These experiments provided valuable insights into the model's characteristics and clarified the contributions of different modules to overall performance, furnishing substantial data to support the final model optimization.\u003c/p\u003e\u003ch2\u003eEvaluation metrics\u003c/h2\u003e\u003cp\u003eIn this study, we employed several evaluation metrics for traffic object detection, specifically Recall (R) and mean Average Precision (mAP). For evaluating lane line detection, we utilized Accuracy (Acc) and Intersection over Union (IoU) as metrics. The assessment of drivable area segmentation was conducted using mean Intersection over Union (mIoU). Additionally, model detection speed was quantified in terms of Frames Per Second (FPS).\u003c/p\u003e\u003ch2\u003eMulti-task performance\u003c/h2\u003e\u003cp\u003eThe experimental results encompass three tasks: traffic object detection, drivable area segmentation, and lane line segmentation. We present the outcomes of vehicle detection and compare our model's performance against four other models on the BDD100K and SDExpressway datasets.\u003c/p\u003e\u003cp\u003eOn the SDExpressway dataset, our detection targets include ten categories, while the BDD100K dataset focuses solely on vehicles. In terms of Recall, AppNets outperforms YOLOP by 3.4% and HybridNets by 1.6% on the SDExpressway dataset. Similarly, on the BDD100K dataset, AppNets demonstrates superior detection accuracy compared to MultiNet, DLT-Net, YOLOP, and HybridNets. Notably, AppNets exhibits significantly higher detection precision on the SDExpressway dataset than on the BDD100K dataset, primarily due to the greater presence of small objects in the SDExpressway dataset.\u003c/p\u003e\u003cp\u003eTable\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e presents detailed information on the object detection results of YOLOP, HybridNets, MultiNet, DLT-Net and AppNets across both the SDExpressway and BDD100K datasets. These findings further substantiate the enhanced detection capabilities of our proposed method in complex traffic scenarios. Our model has shown exceptional performance on both the BDD100K and SDExpressway datasets. Although the real-time inference speed of our model is slightly lower than that of YOLOP due to the incorporation of additional techniques, its performance remains within an acceptable range following lightweight optimization.\u003c/p\u003e\u003cp\u003e \u003c/p\u003e\u003cdiv class=\"gridtable\"\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c8\" colnum=\"8\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c9\" colnum=\"9\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c10\" colnum=\"10\"\u003e\u003c/div\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e\u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eTraffic object detection results on SDExpressway and BDD100k datasets.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e\u003ccolgroup cols=\"10\"\u003e\u003c/colgroup\u003e\u003cthead\u003e\u003ctr\u003e\u003cth align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/th\u003e\u003cth align=\"left\" colspan=\"4\" nameend=\"c5\" namest=\"c2\"\u003e \u003cp\u003eBdd100k\u003c/p\u003e \u003c/th\u003e\u003cth align=\"left\" colspan=\"5\" nameend=\"c10\" namest=\"c6\"\u003e \u003cp\u003eSDExpressway\u003c/p\u003e \u003c/th\u003e\u003c/tr\u003e\u003c/thead\u003e\u003ctbody\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eNetwork\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eR(%)\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003emAP50(%)\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003emAP50:95(%)\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eFPS\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003eR(%)\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colspan=\"2\" nameend=\"c8\" namest=\"c7\"\u003e \u003cp\u003emAP50(%)\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003emAP50:95(%)\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003eFPS\u003c/p\u003e \u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eYOLOP(baseline)\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e89.9\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e76.3\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e43.1\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003cb\u003e223\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colspan=\"2\" nameend=\"c7\" namest=\"c6\"\u003e \u003cp\u003e92.1\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e77.7\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e43.1\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003e\u003cb\u003e231\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMultiNet\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e81.3\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e60.2\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e33.1\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e51\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colspan=\"2\" nameend=\"c7\" namest=\"c6\"\u003e \u003cp\u003e—\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e—\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e—\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003e—\u003c/p\u003e \u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDLT-Net\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e89.4\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e68.4\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e38.4\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e56\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colspan=\"2\" nameend=\"c7\" namest=\"c6\"\u003e \u003cp\u003e—\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e—\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e—\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003e—\u003c/p\u003e \u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eHybridNets\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e91.2\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e79.0\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e46.0\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e220\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colspan=\"2\" nameend=\"c7\" namest=\"c6\"\u003e \u003cp\u003e93.9\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e78.7\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e45.6\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003e200\u003c/p\u003e \u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAppNets\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cb\u003e91.7\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u003cb\u003e79.5\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003e46.5\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e200\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colspan=\"2\" nameend=\"c7\" namest=\"c6\"\u003e \u003cp\u003e\u003cb\u003e95.5\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e\u003cb\u003e85.1\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e\u003cb\u003e53.9\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003e181\u003c/p\u003e \u003c/td\u003e\u003c/tr\u003e\u003c/tbody\u003e\u003c/table\u003e\u003c/div\u003e\u003cp\u003eThe predictions from AppNets are more reasonable, especially in terms of detecting small objects at a distance, with fewer examples of false negatives and more accurate bounding boxes.\u003c/p\u003e\u003cp\u003e \u003c/p\u003e\u003ch2\u003eLane line segment results\u003c/h2\u003e\u003cp\u003eThe results for lane line segmentation are presented in Table\u0026nbsp;\u003cspan refid=\"Tab4\" class=\"InternalRef\"\u003e4\u003c/span\u003e, illustrating that our model outperforms several other models in this regard. Specifically, on the SDExpressway dataset, AppNets achieves an IoU metric that is 1.1% higher than that of YOLOP and 0.8% higher than HybridNets.\u003c/p\u003e\u003cp\u003e \u003c/p\u003e\u003cdiv class=\"gridtable\"\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e\u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003ePerformance comparison on lane line segment task.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e\u003ccolgroup cols=\"4\"\u003e\u003c/colgroup\u003e\u003cthead\u003e\u003ctr\u003e\u003cth align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/th\u003e\u003cth align=\"left\" colspan=\"3\" nameend=\"c4\" namest=\"c2\"\u003e \u003cp\u003eSDExpressway\u003c/p\u003e \u003c/th\u003e\u003c/tr\u003e\u003c/thead\u003e\u003ctbody\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eNetwork\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAcc(%)\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eIoU(%)\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eYOLOP(baseline)\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e90.4\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e74.0\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eHybridNets\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cb\u003e90.8\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e74.3\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAppNets(ours)\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e90.6\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u003cb\u003e75.1\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e\u003c/tr\u003e\u003c/tbody\u003e\u003c/table\u003e\u003c/div\u003e\u003cp\u003e\u003c/p\u003e\u003ch2\u003eDrivable area segment results\u003c/h2\u003e\u003cp\u003eThe results for drivable area segmentation are presented in the table. Notably, on the SDExpressway dataset, our model demonstrates a slight advantage over the other models. The mean Intersection over Union (mIoU) metric for our model is 1.7% higher than that of YOLOP and 1.5% higher than that of HybridNets, respectively. Furthermore, the accuracy (acc) of our model surpasses that of YOLOP and HybridNets by 0.3% and 0.5%, respectively\u003c/p\u003e\u003cp\u003e \u003c/p\u003e\u003cdiv class=\"gridtable\"\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e\u003ctable float=\"Yes\" id=\"Tab3\" border=\"1\"\u003e\u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003ePerformance comparison on drivable area segment task.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e\u003ccolgroup cols=\"4\"\u003e\u003c/colgroup\u003e\u003cthead\u003e\u003ctr\u003e\u003cth align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/th\u003e\u003cth align=\"left\" colspan=\"3\" nameend=\"c4\" namest=\"c2\"\u003e \u003cp\u003eSDExpressway\u003c/p\u003e \u003c/th\u003e\u003c/tr\u003e\u003c/thead\u003e\u003ctbody\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eNetwork\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAcc(%)\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003emIoU(%)\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eYOLOP(baseline)\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e98.9\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e97.0\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eHybridNets\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e98.7\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e97.2\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAppNets(ours)\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cb\u003e99.2\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u003cb\u003e98.7\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e\u003c/tr\u003e\u003c/tbody\u003e\u003c/table\u003e\u003c/div\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003e \u003c/p\u003e\u003cp\u003eThe red lines represent lane markings, the green areas indicate drivable regions, and the orange bounding boxes represent traffic objects. Our AppNets demonstrates a more accurate and aligned performance in detecting bounding boxes in most cases, with lane markings appearing more continuous and smoother.\u003c/p\u003e\u003ch2\u003eAblation studies\u003c/h2\u003e\u003cp\u003eWe referenced the benchmark model YOLOP and implemented several optimizations to develop AppNets. To demonstrate the effectiveness of our model, we conducted ablation studies focusing on attention mechanisms and Spatial Pyramid Pooling with a Fixed Grid (SPPF). Table\u0026nbsp;\u003cspan refid=\"Tab4\" class=\"InternalRef\"\u003e4\u003c/span\u003e presents the modules used in our attention comparison experiments: DAttention[\u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e], TripletAttention[\u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e39\u003c/span\u003e], and iMRB[\u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e]. For the subsequent SPPF ablation study detailed in Table\u0026nbsp;\u003cspan refid=\"Tab5\" class=\"InternalRef\"\u003e5\u003c/span\u003e, we employed SimAttention[\u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e41\u003c/span\u003e] as the attention module, alongside several SPPF variants: RFB[\u003cspan citationid=\"CR42\" class=\"CitationRef\"\u003e42\u003c/span\u003e], SPPELAN, and SPPFCSPC. All evaluation metrics discussed in this section are consistent with those previously outlined.\u003c/p\u003e\u003cp\u003eIn the attention ablation study, SimAttention that we implemented performed the best in the model, despite not achieving the highest Frames Per Second (FPS). It excelled in other critical metrics. As demonstrated in Table\u0026nbsp;\u003cspan refid=\"Tab5\" class=\"InternalRef\"\u003e5\u003c/span\u003e, our implementation of SimSPPF also outperformed other configurations in the SPPF ablation experiments.\u003c/p\u003e\u003cp\u003e \u003c/p\u003e\u003cdiv class=\"gridtable\"\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c8\" colnum=\"8\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c9\" colnum=\"9\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c10\" colnum=\"10\"\u003e\u003c/div\u003e\u003ctable float=\"Yes\" id=\"Tab4\" border=\"1\"\u003e\u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 4\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eComparison Experiments of Different Attention Modules\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e\u003ccolgroup cols=\"10\"\u003e\u003c/colgroup\u003e\u003cthead\u003e\u003ctr\u003e\u003cth align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003eAttention model\u003c/p\u003e \u003c/th\u003e\u003cth align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e \u003cp\u003eTraffic Object\u003c/p\u003e \u003c/th\u003e\u003cth align=\"left\" colname=\"c8\"\u003e \u003cp\u003eDrivable Area\u003c/p\u003e \u003c/th\u003e\u003cth align=\"left\" colname=\"c9\"\u003e \u003cp\u003eLane Line\u003c/p\u003e \u003c/th\u003e\u003cth align=\"left\" colname=\"c10\"\u003e \u003cp\u003eSpeed\u003c/p\u003e \u003c/th\u003e\u003c/tr\u003e\u003c/thead\u003e\u003ctbody\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eD\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eTriplet\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eiM\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003esim\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eR(%)\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003emAP50(%)\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003emAP 50:0.95(%)\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003emIou(%)\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003eIou(%)\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003efps\u003c/p\u003e \u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e√\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e93.1\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e83.0\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e55.5\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e98.1\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e\u003cb\u003e73.5\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003e\u003cb\u003e188\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e√\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e94.3\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e83.0\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e54.7\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e98.2\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e70.7\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003e173\u003c/p\u003e \u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e√\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e94.2\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e80.5\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e51.1\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e98.1\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e71.5\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003e136\u003c/p\u003e \u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e√\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003cb\u003e95.2\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e\u003cb\u003e84.2\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e\u003cb\u003e57.1\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e98.2\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e72.8\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003e178\u003c/p\u003e \u003c/td\u003e\u003c/tr\u003e\u003c/tbody\u003e\u003ctfoot\u003e\u003ctr\u003e\u003ctd colspan=\"10\"\u003e·D:Dattention, sim: Simattention, Triplet༚Tripletattention, iM༚iMRB\u003c/td\u003e\u003c/tr\u003e\u003c/tfoot\u003e\u003c/table\u003e\u003c/div\u003e\u003cp\u003e \u003c/p\u003e\u003cdiv class=\"gridtable\"\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c8\" colnum=\"8\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c9\" colnum=\"9\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c10\" colnum=\"10\"\u003e\u003c/div\u003e\u003ctable float=\"Yes\" id=\"Tab5\" border=\"1\"\u003e\u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 5\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eComparison Experiments of Different SPPF Modules\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e\u003ccolgroup cols=\"10\"\u003e\u003c/colgroup\u003e\u003cthead\u003e\u003ctr\u003e\u003cth align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003esppf model\u003c/p\u003e \u003c/th\u003e\u003cth align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e \u003cp\u003eTraffic Object\u003c/p\u003e \u003c/th\u003e\u003cth align=\"left\" colname=\"c8\"\u003e \u003cp\u003eDrivable Area\u003c/p\u003e \u003c/th\u003e\u003cth align=\"left\" colname=\"c9\"\u003e \u003cp\u003eLane Line\u003c/p\u003e \u003c/th\u003e\u003cth align=\"left\" colname=\"c10\"\u003e \u003cp\u003eSpeed\u003c/p\u003e \u003c/th\u003e\u003c/tr\u003e\u003c/thead\u003e\u003ctbody\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eRFB\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003ecspc\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eelan\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003esim\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eR(%)\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003emAP50(%)\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003emAP 50:0.95(%)\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003emIou(%)\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003eIou(%)\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003efps\u003c/p\u003e \u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e√\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e94.4\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e83.1\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e55.0\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e98.2\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e73.1\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003e188\u003c/p\u003e \u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e√\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e95.0\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e82.7\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e\u003cb\u003e55.6\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e98.3\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e73.2\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003e181\u003c/p\u003e \u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e√\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e94.0\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e80.9\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e54.9\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e98.3\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e70.6\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003e188\u003c/p\u003e \u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e√\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003cb\u003e95.2\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e\u003cb\u003e84.0\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e53.7\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e\u003cb\u003e98.4\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c9\"\u003e \u003cp\u003e\u003cb\u003e73.7\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e\u003ctd align=\"left\" colname=\"c10\"\u003e \u003cp\u003e\u003cb\u003e200\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e\u003c/tr\u003e\u003c/tbody\u003e\u003ctfoot\u003e\u003ctr\u003e\u003ctd colspan=\"10\"\u003e·sim:SimSPPF, elan: SPPELAN, cspc༚SPPFCSPC\u003c/td\u003e\u003c/tr\u003e\u003c/tfoot\u003e\u003c/table\u003e\u003c/div\u003e"},{"header":"Conclusion and Perspective","content":"\u003cp\u003eIn this paper, we introduce the AppNets model and evaluate its performance on the BDD100K and SDExpressway datasets to demonstrate its effectiveness. AppNets is designed to simultaneously address three driving perception tasks: traffic object detection, drivable area segmentation, and lane line segmentation, with support for end-to-end training. To enhance the model's receptive field and feature representation, we developed a novel aggregation network structure termed C2fA. To comprehensively assess the model’s generalization capabilities and mitigate the relative scarcity of the SDExpressway dataset for highway multitasking, we collected and annotated an additional 2,000 images to enrich this dataset.\u003c/p\u003e\u003cp\u003eOn the SDExpressway and BDD100K datasets, our model has exhibited outstanding performance, surpassing existing technologies in terms of both accuracy and efficiency. While our multitask model delivers exceptional detection precision across all three tasks, it incurs higher computational costs compared to the YOLOP model. Consequently, our future efforts will concentrate on refining the model through the integration of the network heads for lane line segmentation and drivable area segmentation. This strategy is expected to streamline the model, reducing its complexity and computational requirements while simultaneously improving its accuracy.\u003c/p\u003e"},{"header":"Declarations","content":"\u003ch2\u003eAuthor Contribution\u003c/h2\u003e\u003cp\u003eXuemei Chen and Yaohan Jia wrote the main manuscript text ,Zeyuan Xu ,Pengfei Ren prepared figures,Wenze Shan prepared tables,All authors reviewed the manuscript.\u003c/p\u003e\u003ch2\u003eACKNOWLEDGEMENTS\u003c/h2\u003e \u003cp\u003eThis project would like to acknowledge supports from Key R\u0026amp;D Program of Shandong Province, China, 2023CXPT032, Development and Demonstration of High energy efficiency Co-processor for High-order Autonomous Driving Applications Supported by the Taishan Industrial Experts Program, Supported by the Taishan Industrial Experts Program; project ZR2023MF0766 supported by Shandong Provincial Natural Science Foundation, Research on Intelligent driving vehicle decision making based on multi-source data fusion and passenger ride experience.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eLiu W et al (2016) SSD: Single Shot MultiBox Detector. In: Leibe B et al (eds) ComputerVision \u0026ndash; ECCV 2016. Springer International Publishing, Cham, pp 21\u0026ndash;37. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1007/978-3-319-46448-0_2\u003c/span\u003e\u003cspan address=\"10.1007/978-3-319-46448-0_2\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRedmon J, Divvala SK, Girshick RB, Farhadi A (2015) You Only Look Once: Unified, Real-Time Object Detection. \u003cem\u003e2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)\u003c/em\u003e, 779\u0026ndash;788\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRedmon J, Farhadi A (2017) YOLO9000: Better, Faster, Stronger, in Proceedings of theIEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.6517\u0026ndash;6525, \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/CVPR.2017.690\u003c/span\u003e\u003cspan address=\"10.1109/CVPR.2017.690\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJoseph Redmon and Ali Farhadi (2018) Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767, 2s [2, 7, 13\u0026ndash;15, 25, 30, 31, 47\u0026ndash;49, 61\u0026ndash;63, 74, 75],\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBochkovskiy A, Wang C-Y, Liao H-YM (2020) YOLOv4: Optimal Speed and Accuracy ofObject Detection, arXiv:2004.10934\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGe Z, Liu S, Wang F, Li Z, Sun J YOLOX: Exceeding YOLO series in 2021. arXiv preprint arXiv:2107.08430, 2021. 3\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJocher Glenn (2022) YOLOv5 release v7.0. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://github.com/ultralytics/yolov5/releases/tag/v7\u003c/span\u003e\u003cspan address=\"https://github.com/ultralytics/yolov5/releases/tag/v7\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. 0\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLi C, Li L, Jiang H, Weng K, Geng Y, Li L, Ke Z, Li Q, Cheng M, Nie W et al (2022) YOLOv6: A single-stage object detection framework for industrial applications. arXiv preprint arXiv:2209.02976\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWang C-Y (2023) Alexey Bochkovskiy, and HongYuan Mark Liao. YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7464\u0026ndash;7475\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJocher Glenn (2023) Yolov8. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://github.com/ultralytics/ultralytics/tree\u003c/span\u003e\u003cspan address=\"https://github.com/ultralytics/ultralytics/tree\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e/ main\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWang C, Yeh I, Liao H (2024) YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information. \u003cem\u003eArXiv, abs/2402.13616\u003c/em\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWang A, Chen H, Liu L, Chen K, Lin Z, Han J, Ding G (2024) YOLOv10: Real-Time End-to-End Object Detection. \u003cem\u003eArXiv\u003c/em\u003e abs/2405.14458 : n. pag\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRonneberger O, Fischer P, Brox T U-Net: Convolutional Networks for Biomedical ImageSegmentation, In: Navab N, Hornegger J, Wells W, Frangi A (eds) Medical ImageComputing and Computer-Assisted Intervention \u0026ndash; MICCAI 2015. MICCAI 2015. LectureNotes in Computer Science, vol 9351. Springer, Cham.\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1007/978-3-319-24574-4_28\u003c/span\u003e\u003cspan address=\"10.1007/978-3-319-24574-4_28\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBadrinarayanan V, Kendall A, Cipolla R (2017) SegNet: A Deep ConvolutionalEncoder-Decoder Architecture for Image Segmentation. IEEE Trans Pattern Anal Mach Intell 39:12, 2481\u0026ndash;2495. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1109/TPAMI.2016.2644615\u003c/span\u003e\u003cspan address=\"10.1109/TPAMI.2016.2644615\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLi B, Zang J, Cao J (2021) Efficient Residual Neural Network for Semantic Segmentation,Pattern Recognit. Image Anal 31:2, 212\u0026ndash;220. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1134/S1054661821020103\u003c/span\u003e\u003cspan address=\"10.1134/S1054661821020103\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWang Z, Ren W, Qiu Q (2018) LaneNet: Real-Time Lane Detection Networks for AutonomousDriving, arXiv:1807.01726\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePan X, Shi J, Luo P, Wang X, Tang X (2017) Spatial As Deep: Spatial CNN for Traffic SceneUnderstanding, arXiv:1712.06080\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTeichmann M, Weber M, Z\u0026ouml;llner M, Cipolla R, Urtasun R (2018) MultiNet: Real-time JointSemantic Reasoning for Autonomous Driving, in Proceedings of the IEEE IntelligentVehicles Symposium (IV), pp. 1013\u0026ndash;1020, \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/IVS.2018.8500504\u003c/span\u003e\u003cspan address=\"10.1109/IVS.2018.8500504\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eQian Y, Dolan JM, Yang M (2020) DLT-Net: Joint Detection of Drivable Areas, Lane Lines,and Traffic Objects, IEEE Transactions on Intelligent Transportation Systems, vol. 21, no.11, pp. 4670\u0026ndash;4679, Nov. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/TITS.2019.2943777\u003c/span\u003e\u003cspan address=\"10.1109/TITS.2019.2943777\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWu D, Liao M, Zhang W, Wang X (2022) YOLOP: You Only Look Once for Panoptic DrivingPerception, arXiv:2108.11250\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDat Vu B, Ngo, Phan H (2022) Hybridnets: End-to-end perception network. arXiv preprint arXiv:2203.09035\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRen S, He K, Girshick R, Sun J Faster r-cnn: Towards real-time object detection with region proposal networks. arXiv preprint arXiv:1506.01497, 2015.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKera SB, Tadepalli A, Ranjani JJ (2023) A paced multi-stage block-wise approach for object detection in thermal images. Vis Comput 39:2347\u0026ndash;2363. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1007/s00371-022-02445-x\u003c/span\u003e\u003cspan address=\"10.1007/s00371-022-02445-x\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eXie Z, Zhang W, Sheng B, Li P, Chen CLP (2023) BaGFN: Broad Attentive Graph Fusion Network for High-Order Feature Interactions, in \u003cem\u003eIEEE Transactions on Neural Networks and Learning Systems\u003c/em\u003e, vol. 34, no. 8, pp. 4499\u0026ndash;4513, Aug. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/TNNLS.2021.3116209\u003c/span\u003e\u003cspan address=\"10.1109/TNNLS.2021.3116209\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLin X, Sun S, Huang W, Sheng B, Li P, Feng DD (2023) EAPT: Efficient Attention Pyramid Transformer for Image Processing, in \u003cem\u003eIEEE Transactions on Multimedia\u003c/em\u003e, vol. 25, pp. 50\u0026ndash;61, \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/TMM.2021.3120873\u003c/span\u003e\u003cspan address=\"10.1109/TMM.2021.3120873\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eShelhamer E, Jonathan Long and, Darrell T (2014) Fully convolutional networks for semantic segmentation. \u003cem\u003e2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)\u003c/em\u003e : 3431\u0026ndash;3440\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRonneberger O, Fischer P, Brox T (2015) U-Net: Convolutional Networks for Biomedical Image Segmentation. ArXiv abs/1505.04597 : n. pag\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBadrinarayanan V, Kendall A, Cipolla R (2015) SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation. IEEE Trans Pattern Anal Mach Intell 39:2481\u0026ndash;2495\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGao Y, Qi Z, Zhao D (2023) Edge-enhanced instance segmentation by grid regions of interest. Vis Comput 39:1137\u0026ndash;1148. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1007/s00371-021-02393-y\u003c/span\u003e\u003cspan address=\"10.1007/s00371-021-02393-y\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhao H, Shi J, Qi X, Wang X, Jia J (2017) Pyramid Scene Parsing Network, in Proceedingsof the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.6230\u0026ndash;6239, \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/CVPR.2017.660\u003c/span\u003e\u003cspan address=\"10.1109/CVPR.2017.660\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePaszke A, Chaurasia A, Kim S, Culurciello E (2016) ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation. \u003cem\u003eArXiv\u003c/em\u003e abs/1606.02147 : n. pag\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eQin Z, Wang H, Li X (2020) Ultra Fast Structure-aware Deep Lane Detection. ArXiv abs/2004 11757:pag\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTabelini L, Berriel R, Paix\u0026atilde;o TM, Badue CS, Souza AF, Oliveira-Santos T (2020) PolyLaneNet: Lane Estimation via Deep Polynomial Regression. \u003cem\u003e2020 25th International Conference on Pattern Recognition (ICPR)\u003c/em\u003e, 6150\u0026ndash;6156\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTabelini L, Berriel R, Paix\u0026atilde;o TM, Badue CS, Souza AF, Oliveira-Santos T (2020) Keep your Eyes on the Lane: Real-time Attention-guided Lane Detection. \u003cem\u003e2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)\u003c/em\u003e, 294\u0026ndash;302\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHou Y, Ma Z, Liu C, Loy CC (2019) Learning Lightweight Lane Detection CNNs by SelfAttention Distillation, in Proceedings of the IEEE/CVF International Conference onComputer Vision (ICCV), pp. 1013\u0026ndash;1021, \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/ICCV.2019.00110\u003c/span\u003e\u003cspan address=\"10.1109/ICCV.2019.00110\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHe K, Gkioxari G (2017) Piotr Doll\u0026aacute;r and Ross B. Girshick. Mask R-CNN\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKaiwen Duan L, Xie H, Qi S, Bai Q, Huang, Tian Q (2021) Location-sensitive visual recognition with cross-iou loss. arXiv preprint arXiv:2104.04899\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eXia Z, Pan X, Song S, Li LE, Huang G (2022) Vision Transformer with Deformable Attention. \u003cem\u003e2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)\u003c/em\u003e : 4784\u0026ndash;4793\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMisra 39, Nalamada D, Arasanipalai T, A.U., Hou Q (2020) Rotate to Attend: Convolutional Triplet Attention Module. \u003cem\u003e2021 IEEE Winter Conference on Applications of Computer Vision (WACV)\u003c/em\u003e, 3138\u0026ndash;3147\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhang J, Li X, Li J, Liu L, Xue Z, Zhang B, Jiang Z, Huang T, Wang Y, Wang C (2023) Rethinking Mobile Block for Efficient Attention-based Models. \u003cem\u003eIEEE/CVF International Conference on Computer Vision (ICCV)\u003c/em\u003e (2023): 1389\u0026ndash;1400\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eYang L, Zhang R-Y, Li L, Xie X (2021) SimAM: A Simple, Parameter-Free Attention Module for Convolutional Neural Networks. \u003cem\u003eInternational Conference on Machine Learning\u003c/em\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLiu S et al (2017) Receptive Field Block Net for Accurate and Fast Object Detection. \u003cem\u003eEuropean Conference on Computer Vision\u003c/em\u003e\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"multi-tasking, traffic object detection, lane line segmentation, drivable area segmentation, autonomous driving","lastPublishedDoi":"10.21203/rs.3.rs-5358737/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-5358737/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003ePanoramic driving perception systems are critical for autonomous driving, as they provide essential traffic-related information. This study introduces AppNets, an efficient and effective multi-task learning framework designed for real-time panoptic driving perception. AppNets comprises an encoder for feature extraction and three decoders that concurrently perform traffic object detection, drivable area segmentation, and lane segmentation. We propose the C2fA module to enhance the model's extraction capability. To enhance our dataset, we expanded the SDExpressway dataset by adding 2,000 frames, particularly incorporating nighttime and adverse weather scenarios. Extensive experiments conducted on both the challenging BDD100K dataset and the augmented SDExpressway dataset demonstrate that AppNets achieves state-of-the-art performance, outperforming baseline models by significant margins. Specifically, on the SDExpressway dataset, AppNets attains a mean average precision (mAP) of 85.1% for traffic object detection, a mean intersection over union (mIoU) of 98.7% for drivable area segmentation, and an intersection over union (IoU) of 75.1% for lane segmentation. These results underscore the effectiveness of AppNets in complex driving scenarios, highlighting its potential for practical deployment in autonomous driving systems. the source codes are released at \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://github.com/Huniki/Appnet.git\u003c/span\u003e\u003cspan address=\"https://github.com/Huniki/Appnet.git\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e","manuscriptTitle":"AppNets: An Efficient Multi-Task Fusion Network for Comprehensive Driving Perception","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-11-12 15:39:34","doi":"10.21203/rs.3.rs-5358737/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"e3a19022-1673-4346-a80a-5e7c711a13a2","owner":[],"postedDate":"November 12th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2025-03-16T03:38:12+00:00","versionOfRecord":[],"versionCreatedAt":"2024-11-12 15:39:34","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-5358737","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-5358737","identity":"rs-5358737","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}