Smart Traffic Management of Vehicles using Faster R-CNN based Deep Learning Method

Smart Traffic Management of Vehicles using Faster R-CNN based Deep Learning Method | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Smart Traffic Management of Vehicles using Faster R-CNN based Deep Learning Method Arindam Chaudhuri This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-3854436/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 06 May, 2024 Read the published version in Scientific Reports → Version 1 posted 5 You are reading this latest preprint version Abstract With constant growth of civilization and modernization of cities all across the world since past few centuries smart traffic management of vehicles is one of the most sorted after problem by research community. Smart traffic management basically involves segmentation of vehicles, estimation of traffic density and tracking of vehicles. The vehicle segmentation from videos helps realization of niche applications such as monitoring of speed and estimation of traffic. When occlusions, background with clutters and traffic with density variations, this problem becomes more intractable in nature. Keeping this motivation in this research work, we investigate Faster R-CNN based deep learning method towards segmentation of vehicles. This problem is addressed in four steps viz minimization with adaptive background model, Faster R-CNN based subnet operation, Faster R-CNN initial refinement and result optimization with extended topological active nets. The computational framework uses adaptive background modeling. It also addresses shadow and illumination issues. Higher segmentation accuracy is achieved through topological active net deformable models. The topological and extended topological active nets help to achieve stated deformations. Mesh deformation is achieved with minimization of energy. The segmentation accuracy is improved with modified version of extended topological active net. The experimental results demonstrate superiority of this framework. Physical sciences/Engineering Physical sciences/Mathematics and computing Smart traffic management vehicle segmentation traffic density estimation vehicle tracking Faster R-CNN Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 1 Introduction We are facing the challenge of continuous increase in traffic density across different cities of world. The present world population is heavily dependent on vehicles from smooth commutation purposes. This results from constant urbanization of rural areas. In order to address this problem, smart traffic management is studied by researchers as traffic regulations solution. Human vision system has the capability to perform complex tasks very reliably and accurately. Humans detect wide spectrum of objects very easily. With recent developments in computer vision coupled with huge data sets, better algorithms and faster GPUs, precision and accuracy of object detection and classification algorithms 1, 2 have increased to an appreciable amount. For traffic monitoring vehicle localization efficiency is very important. The vehicles from Indian traffic are presented in Fig. 1 . For public safety autonomous vehicle detection methods 3 are in place which detect traffic objects in order to achieve correct decisions. Smart traffic management allows us to take care of various traffic related issues in an optimum manner. This is achieved using computer vision and image processing. Segmentation of vehicle acts as significant enabler in smart traffic management functionality. With occlusions, congestion as well as other environmental factors this problem becomes more severe. For smooth and effective traffic regulations, smart traffic management is always considered as low cost solution. Congestions, emergency vehicles transport, accidents and traffic related violations are easily managed through latest Artificial Intelligence algorithms. Vehicle segmentation is an important area towards smart traffic management. Some other activities here include traffic estimation, speed control and vehicle tracking. During occlusions, fog, haze, clutters and heavy traffic situations things become more complicated. With the growth of deep learning networks vehicle detection is being studied deeply considering traffic congestion and driving safety 4 . Vehicle localization is a crucial problem 5 in order to develop intelligent and autonomous systems. The detection of abnormalities arising from traffic violations leads to the problem of vehicle localization. This is a significant application catering the needs for variety of traffic related problems. The traffic surveillance has been a major concern in densely populated geographical areas. These days’ surveillance systems are well equipped with traffic flow data where various traffic patterns are recorded. Some notable applications here include many smart cities based applications. Different versions of deep learning network based methods are in place which have provided considerable benefits for these applications 3 . In this research work, we investigate Faster R-CNN based deep learning method towards segmentation of vehicles. This problem is addressed in four steps 5 : (a) minimization with adaptive background model (b) Faster R-CNN based subnet operation (c) Faster R-CNN initial refinement and (d) result optimization with extended topological active nets. Adaptive gain function is used adaptive background modeling. The gain function compensates for shadow and illumination issues. Higher accuracy in segmentation is provided by topological active net deformable model in various situations. Deformable models provide various curvatures with respect to image surfaces. The smoothness from deformation is achieved through several forces considering objects of interest. The topological and extended topological active nets are used here in order to achieve stated deformations. This helps in fitting objects on 2D surface mesh of image. The deformation of mesh is achieved with minimization of energy. For problems with complex shapes energy is changed through extended topological active net with certain thresholds. The segmentation accuracy is increased with improved version of extended topological active net. This solution has combined effect from all models using which better segmentation boundaries are achieved. The performance of stated method is compared with respect to some important metrics such as precision and recall. This method provides better segmentation performance in comparison to other methods. The experimental hypothesis is justified with benchmarked datasets. The model provides appreciable results for different sets of image datasets. This chapter is structured as follows. The literature review is presented in section 2 . In section 3 highlights detailed discussion on computational method. The simulation results are shown in section 4 . Finally, conclusion is provided in section 5 . 2 Literature Review During recent decades’ smart traffic management involving vehicle detection has gained considerable attention. Some of the notable works are discussed here. In 6 a pixel wise classification method based on dynamic bayesian network for vehicle detection is proposed. In 7 an object detection scheme is presented which identifies changes in image series. Foreground vehicle segmentation using gaussian mixture model is highlighted in 8 . An adaptive background model having frames averaging with respect to time is discussed in 9 . For vehicle localization ResNet model is used in 10 . A vehicle classification system involving deep learning is presented in 11 . An integrated vehicle detection and classification method is discussed in 12 . 13 discusses YOLOv3 algorithm for vehicle detection. 14 presents receptive field based neural network. Semantic image segmentation is used in 15 . Mask R-CNN with transfer learning is used in 16 . 17 discusses YOLO based solution. A review on vehicle detection is highlighted in 18 . In 19 deep learning assisted vehicle segmentation is discussed. In 20 vehicles in airborne images are detected. An ensemble based method using image descriptors is discussed in 21 . In 22, 23 methods are developed through application of gaussian mixture model. In 24 support vector machines is used for vehicle detection. SIFT algorithm is integrated with support vector machines in 25 to achieve vehicle detection. For autonomous vehicles an object detection system is highlighted in 26 . Some R-CNN version of vehicle detection algorithms are discussed in 28, 44, 54 . Several significant YOLO based multi object vehicle detection algorithms are highlighted in 27. 32, 33, 34, 35, 37, 38, 39, 42 . Vehicle detection algorithms in different weather conditions are presented in 29, 30, 52 . Deep learning networks for vehicle detection are used in 31, 36, 41, 43, 45, 46, 55, 56 . Several other notable works are available in 40, 47, 48, 49, 50, 51, 53, 57 . 3 Computational Method In this section, smart traffic vehicle management using Faster R-CNN based deep learning based ensemble method is highlighted. The research problem revolves around traffic management which involves vehicle segmentation at different background levels 5 . In order to achieve this, we study Faster R-CNN 5 which analyses vehicles in smart traffic. The segmentation activity on various strategic areas of smart traffic analytics provide information related to decision-making activities. 3.1 Datasets Used The experimental datasets are adopted from 58 . The datasets are prepared keeping in view different traffic conditions. At the very first instance multi-class vehicle objects are considered. Several challenge factors such as traffic jams and overlapping vehicles are incorporated in dataset. Broadly speaking datasets are developed with respect to two different situations viz high density and low density traffic scenes. The former comes with several objects in single image while later has single class per image having no overlapping. In order to achieve better training, images from both above mentioned situations are placed in different datasets. The high density traffic scenes are considered from several places having traffic which are less crowded in nature. A total of 1000 images from six classes of vehicles viz two wheelers, three wheelers, four wheelers, six wheelers, eight wheelers and ten wheelers are developed. Figure 2 (a) and 2(b) highlight some sample images. High density traffic scenes are considered from daily traffic which are congested in nature. 5000 images from above stated classes are created. Certain critical factors such as illumination, occlusions etc are incorporated irrespective of appearance, shape, scale and size in this dataset. Table 1 highlights certain high and low density dataset statistics alongwith annotations of class. These datasets are preprocessed and augmented as discussed in subsection 3.2 . Each of these datasets are benchmarked with respect to standard datasets 58 . 3.2 Data Annotation and Augmentation In order to achieve reliable vehicle detection dataset classes are labelled. Based on motivation from 5 an image tool 57 has been used here towards labeling and annotation of this dataset. As shown in Fig. 3 (a) and 3(b) for each object in image, a bounding box is assigned manually. In high density datasets (HDD) from high density traffic many bounding boxes are present. In low density datasets (LDD) from low density traffic few bounding boxes are present in single image. The label of respective class is specified by these bounding boxes. As mentioned in subsection 3.1 , the whole dataset is defined through six classes. In order to increase features of datasets such that better results are obtained data preprocessing and augmentation is used. These addresses various issues which can be present in images such as noise, inconsistency and unbalanced classes. The data augmentation process used here are in lines with discussed in 5, 57 . Since augmented dataset has an unknown distribution it is benchmarked adhering to certain standards 58 . 3.3 Faster R-CNN based Method Now we present a detailed description of proposed method. The method is highlighted considering four steps 5 viz (a) minimization with adaptive background model (b) Faster R-CNN based subnet operation (c) Faster R-CNN initial refinement and (d) result optimization with extended topological active nets. Adaptive background modeling is used to construct background with traffic video as input. The video frames are analyzed in order to develop background model. The objective here is to find best background estimate. With this on foreground model shadow and illumination change impacts are minimized. This process is initiated by first initializing few frames and then continuous updates are done each time. This leads to extraction of foreground from next set of frames. The initialization of background model is done considering first frame’s pixel values. A subsequent update of model is performed through calculation of value of pixel considering background model: $${pixel}_{j}\left(y\right)={\stackrel{-}{pixel}}_{j-1}\left(y\right)+\frac{1}{{GP}_{j}\sqrt{2\pi {\sigma }^{2}}}\text{exp}\left(\frac{-1}{2}{\left(\frac{y-{y}_{j}}{\sigma }\right)}^{2}\right)$$ 1 Here \({y}_{j}\) represents value of pixel with respect to \({j}^{th}\) frame. The value of \({\stackrel{-}{pixel}}_{j}\) is computed as follows: $${\stackrel{-}{pixel}}_{j-1}\left(y\right)=\frac{{pixel}_{j}\left(y\right)}{\sum _{j}{pixel}_{j}\left(y\right)}$$ 2 Here number of frames considered is \(N\) . The gain parameter is denoted by \({GP}_{j}\) . It takes care of background modeling’s learning rate. The new information is learned by increasing \({GP}_{j}\) as presented in Eq. ( 3 ) with prior information disappearing slowly. $${GP}_{j}=Gain\left(\frac{2}{1+\text{exp}\left(-\frac{cont-\alpha }{\beta }\right)}\right)$$ 3 Here \(Gain\) and \(\alpha\) parameters take care of sinusoid function’s inflection point. The parameter \(\beta\) controls gradient. The parameter \(cont\) depends on number of frames. Background model is updated for every frame. Considering every fame with background as adaptive, after its subtraction we reach to foreground objects. After adaptive background model based minimization is performed, we present basic Faster R-CNN architecture used in this research. The architecture of Faster R-CNN is considered from 5 having variation based baseline considered from 70 . The architecture is highlighted in Fig. 4 . In order to validate proposed method, datasets mentioned in subsection 3.1 are used. The convolutional feature map is developed when entire image is processed convolutional and max pooling layers. For each object’s RoI pooling layer, feature vector of fixed length is extracted. The input of sequence of fully connected layers are feature vectors. From a set of fully connected layers, output passes to sibling layers where softmax probability estimates are produced. The object classes estimate softmax probability. Here background class with set of other layers are considered. The encoding is performed on set of values with respect to refined positions in bounding box for each object class. For RoI pooling layer we use max pooling. This allows conversion of features into small feature map. Here convolutional feature map based RoI is defined through 4-tuple. RoI max pooling divides rectangular window into sub window grids. The channel independent pooling is applied for each feature map. The network weights are trained through backpropagation in Faster R-CNN. A hierarchical sampling of RoIs is performed for each image. Stochastic gradient descent (SGD) training for Faster R-CNN is done in mini-batches. For training with larger datasets, execution of SGD is done for more iterations. Faster R-CNN makes use of sibling output layers. For each RoI we have discrete probability distribution as initial output. The probabilities for fully connected layer have outputs as softmax. The labeling for RoI is performed with respect to ground truth class. For each training bounding box regression having ground truth is considered. For each labeled RoI considering multitask loss, joint classification is present with respect to training and bounding-box regression. The optimization of multitask loss is performed as highlighted in 5 . For whole image classification, convolutional layers’ calculation time is greater than fully connected layers. RoIs processing time is appreciably large for detection. Now subnet operation with Faster R-CNN is presented. The foreground image considers a mesh. This is moved towards subnet operation for Faster R-CNN. The training image dataset is adopted from 19 . This is marked with respect to active net grids. The subnet is represented is binary matrix with 1 as indicator for link presence of mesh and 0 as indicator for link absence. Here Faster R-CNN is trained where features are learned in order to produce ground truth specific results. The U-Net based CNN 59 helps us to perform initial subnet operation. The extended topological active net refines initial subnet. Faster R-CNN produces hole based mesh considering all background. The objects have mesh nodes. Those mesh nodes which where boundary is not crossed are removed. It is reached though extended topological active net energy minimization. The energy function used here is highlighted in Eq. ( 4 ). By using Eq. ( 1 ) adaptive background model is developed. $$Energy\left(w\left(a,b\right)\right)=\underset{0}{\overset{1}{\int }}\underset{0}{\overset{1}{\int }}\left[{Energy}_{internal}\left(w\left(a,b\right)\right)+{Energy}_{external}\left(w\left(a,b\right)\right)\right]dadb$$ 4 The mesh deformation is done using greedy approach. In situations with complex deformation of shapes, extended topological active net produces different energy term. This it achieves through various thresholds. When clutters and occlusion are present in vehicle image segmentation extended topological active net always reaches local minima. This results in low segmentation accuracy. The improved version of extended topological active net provides resolution to these problems. The combined effect of all models used here leads us to better results. 3.4 Training Process The training process is now briefly discussed. The annotated and augmented data is trained using Faster R-CNN algorithm. In order to perform training, several parameters like size of batch, epochs needed and resolution of image are used. Since network is trained from scratch, random weight initialization is performed. Here initially trained COCO weights 5 are used towards model training with appreciable time and computation benefits. The best weight values are obtained using initially trained Faster R-CNN having transfer learning. 5 datasets are used as benchmark in order to train stated custom datasets. The batch sizes of 5, 10, 20, 30 and 40 are considered. The epochs are also changed to 100, 200, 300, 400 and 500. The confidence values are considered between 0.4 to 0.6. The best weights are used to detect objects in datasets. The predicted labels and assessment images incorporating bounding boxes with confidence values are also obtained. 3.5 Evaluation Criteria In this research precision and recall criteria 5 are used for measuring robustness. Here, mean average precision (mAP) is also calculated for recall values lying between 0 to 1. 4 Experiments and Results Here a detailed discussion on simulation results is presented. We conducted detailed experiments in Google Colab having T4 GPU with Intel Xeon CPU and 64 GB of RAM. Python version 3.11.5 has been used as simulation tool in this research. In order to assess vehicle detection method performance, several state-of-the-art detectors are evaluated. Also various comparisons are performed with stated method considering accuracy and execution times. All methods are trained on data adapted from COCO [60] and DAWN 61 datasets. The detection and segmentation of objects is performed by COCO dataset considering natural contexts 5 . In Table 2 COCO dataset highlights several objects collected from complex scenes. The dataset has images of 100 different object types with 3 million instance labels. The results are highlighted in Fig. 5 . Certain YOLOv5 semantics are adopted from 57 . All vehicles are accurately detected by stated method considering variation in illumination. In order to further validate superiority of stated method we perform a comparative analysis with 14 methods as presented in Table 3 . Some significant observations are briefly highlighted here. For images with different resolutions, stated computational structure provides best performance in terms of mAP values. The methods presented in 49, 50, 55, 56 also provide promising results with respect to COCO datasets. 54 makes use of ResNet and produces results on lower side for object detection in COCO datasets. Similar results are obtained from methods presented in 45, 47 . Here miscellaneous objects are detected on COCO datasets with promising results. These include vehicles of varying shapes and sizes. All results are highlighted in Fig. 5 . The DAWN dataset is used in order to study and investigate performance of stated method. DAWN dataset has 2000 image with different variations 5 . It shows varying traffic situations in different weather conditions. Figure 6 shows fairly detailed results considering significant observations. The stated method addresses all prevailing weather situations such as rainy days, normal dry days and snowy days. A comparison is performed with results presented in Table 4 . Some observations are highlighted here. In fog situation 54 has highest rank. Our method exceeds results of 54 . Here 43, 52 have lowest ranks. In rain situation our method produces best results followed by 57, 45, 48, 50, 51 . Here 52 has lowest rank. In snow situation our method has highest rank followed by 51, 57 . Here 45, 47, 50, 51 produce similar results. In dry day situation stated method achieves highest rank. Here 42, 48, 49 yield similar results. 52 has lowest rank. Now we present some additional insights in this research with respect to diverse range of environments. All state-of-the-art object detection methods have been studied in this research. These results are briefly discussed here. In 42 BIT-Vehicle and UADETRAC datasets are investigated. In 57 3 different datasets are studied which several variations with respect to road conditions, weather as well as complex background. In 43, 48 primary concentrations are on KITTI and DAWN datasets with certain deviations which produces appreciable results. In 44 investigation of object detection capability is performed on COCO and DAWN datasets. In 45 CARLA dataset is explored. The results presented here extends method's detection capability to 3 other datasets. PASCAL VOC 2007 dataset is studied in 47, 51 . In 37 PASCAL dataset is studied which contains annotated images of various objects. Here detection capability of method is expanded to 3 different vehicle datasets. In 49, 50, 53, 54 COCO and DAWN datasets are used in order to validate methods for object detection. In 54 various object detection methods are presented for detection of different objects. PETS2009 and changedetection.net 2012 datasets are studied in 52 with DLR Munich vehicle and VEDAI datasets in 55 and KITTI and LSVH datasets in 56 . The vehicle detection methods on these datasets produce appreciable results. In DAWN dataset has more challenging images with fog, rain, dry day and snow. However, as shown in Fig. 7 very low vehicle detection results are achieved. But our method performs well in this situation. Considering several state-of-the-art approaches, a detailed comparison of is performed alongwith our method. It is seen that in different situations our method performs well. In COCO dataset our method produces appreciable results in comparison to other methods. 5 Conclusion In this research work smart traffic management of vehicles is studied using Faster R-CNN based deep learning method. It is an intractable problem in computer vision and artificial intelligence domain. When occlusions, background with clutters and traffic with density variations are present, this problem becomes more challenging. The computational paradigm involves four steps viz minimization with adaptive background model, Faster R-CNN based subnet operation, Faster R-CNN initial refinement and result optimization with extended topological active nets. The concept of adaptive background modeling is incorporated in this framework. The shadow and illumination related issues are also addressed. The topological active net deformable models help to achieve higher segmentation accuracy. The deformations are reached with topological and extended topological active nets. Mesh deformation helps in minimization of energy. The segmentation accuracy is improved with modified version of extended topological active net. The superiority of this method is highlighted with experimental results. Declarations Data Availability: All data used in this research has been developed at Samsung R & D Institute New Delhi India as mentioned in Reference 58. The Institution does not allow to share and provide access to research data to public domains. In view of this, data used in this research cannot be shared. However, two more datasets COCO and DAWN used in this research are highlighted in References 60 and 61 can be shared and accessed. Competing Interests: Author has no competing interests to declare which are relevant to content of this article. Author Contribution Statement: Author contributed to the study conception and design. Material preparation, data collection and analysis were performed by Arindam Chaudhuri. The first draft of manuscript was written by Arindam Chaudhuri and author commented on previous versions of manuscript. Author read and approved the final manuscript. Ethical and Informed Consent for Data used: Author has ethical and informed consent for data used in this research. References Mahmood et al. Towards a Fully Automated Car Parking System. IET Int. Trans. Sys. , 13 , 293–302 (2018). Xiaohong et al. Real time object detection based on YOLOv2 for tiny vehicle objects. In Proc. Comp. Sci., 183 , 61–72, Elsevier Press (2021). Jamiya et al. Little YOLO-SPP: A delicate real time vehicle detection algorithm. Optik , 225 , 165818. (2021). Tajar et al. A lightweight Tiny-YOLOv3 vehicle detection approach. J. of Real-Time Image Proc. , 18 , 2389–2401 (2021). Chaudhuri, A. Smart traffic management in varying weather conditions. Tech. Rep. , Samsung R & D Inst. New Delhi, India (2018). Cheng et al. Vehicle detection in aerial surveillance using dynamic bayesian networks. IEEE Tran. on Image Proc. , 21 (4), 2152–2159 (2012). McHugh et al. Foreground adaptive background subtraction. IEEE Sig. Proc. Lett. , 16 , 390–393 (2009). Chen et al. Road vehicle classification using support vector machines. In Proc. IEEE Int. Conf. Intel. Comp. Intel. Sys., IEEE Press (2009). Kul et al. Distributed and collaborative real time vehicle detection and classification over video streams. Int. J. of Adv. Robo. Sys. , 14 (4), 1729881417720782 (2017). Jung et al. ResNet based vehicle classification and localization in traffic surveillance systems. In Proc. IEEE Conf. Comp. Vis. Pat. Rec. Workshops, Honolulu, 934–940, IEEE Press (2017). Maungmai et al. Vehicle classification with deep learning. In Proc. IEEE Int. Conf. Comp. Comm. Sys., Singapore, 294–298, IEEE Press (2019). Bai et al. Classify vehicles in traffic scene images with deformable part based models. Mac. Vis. App. , 29 , 393–403 (2017). Song et al. Vision based vehicle detection and counting system using deep learning in highway scenes. Euro. Trans. Res. Rev. , 11 , 51 (2019). Lee et al. Segmentation of vehicles and roads by low channel lidar. IEEE Tran. Int. Tran. Sys. , 20 (11), 4251–4256 (2019). Kaymak et al. Semantic image segmentation for autonomous driving using fully convolutional networks. In Proc. Int. Art. Int. Data Proc. Symp. (2019). Ojha et al. Vehicle detection through instance segmentation using mask R-CNN for intelligent vehicle system. In Proc. 5 th Int. Conf. Int. Comp. Con. Sys., 954–959 (2021). Shan et al. A real time vehicle detection algorithm based on instance segmentation. In Proc. IEEE 2 nd Int. Conf. Big Data, Art. Int. IoT Eng, 239–243, IEEE Press (2021). Shobha et al. A Review on video based vehicle detection, recognition and tracking, In Proc. 3 rd IEEE Int. Conf. Comp. Sys. Info. Tech. Sus. Soln., IEEE Press (2018). Shobha et al. Deep learning assisted active net segmentation of vehicles for smart traffic management. In Proc. Glob. Trans. 2 , 282–286 (2021). Hamsa et al. Automatic vehicle detection from aerial images using cascaded support vector machine and gaussian mixture model. In Proc. Int. Conf. Sig. Proc. Info. Security, Dubai pp 1–4 (2018). Mikaty et al. Detection of cars in high resolution aerial images of complex urban environments. IEEE Tran. Geo. Remo. Sens. , 55 , 5913–5924 (2017). Arı et al. Detection of compound structures using gaussian mixture model with spectral and spatial constraints. IEEE Tran. Geo. Remo. Sens. , 52 , 6627–6638 (2014). Hbaieb et al. Pedestrian detection for autonomous driving within cooperative communication system. IEEE Wire. Comm. Net. Conf., Morocco, 1–6, IEEE Press (2019). Xiong et al. High speed front vehicle detection based on video multi-feature fusion. In Proceedings IEEE 10 th Int. Conf. Elec., Info. Emer. Comm., Beijing, 348–351, IEEE Press (2020). Yawen et al. Research on vehicle detection technology based on SIFT features. In Proc. 8 th Int. Conf. Elec. Info. Emer. Comm., Beijing, 274–278 (2018). Li et al. A deep learning based hybrid framework for object detection and recognition in autonomous driving. IEEE Access , 8 , 194228–194239 (2020). Li et al. YOLO-ACN: Focusing on small target and occluded object detection. IEEE Access , 8 , 227288–227303 (2020). Cai et al. Cascade R-CNN: Delving into high quality object detection. In Proc. IEEE Conf. Comp. Vis. and Pat. Rec., USA, 6154–6162, IEEE Press (2018). Wang et al. A high precision fast smoky vehicle detection method based on improved YOLOv5 Network. In Proc. IEEE Int. Conf. Art. Int. Ind. Des., Guangzhou, 255–259, IEEE Press (2021). Miao et al. A nighttime vehicle detection method based on YOLOv3. In Proc. Chinese Auto. Cong., Shanghai, 6617–6621 (2020). Sarda et al. Object detection for autonomous driving using YOLO algorithm. In Proc. 3 rd Int. Conf. Int. Comm. Tech. Vir. Mob. Net., Tirunelveli, 1370–1374 (2021). Zhao et al. Vehicle detection based on improved YOLOv3 algorithm. In Proc. Int. Conf. Int. Trans., Big Data Smart City, Vientiane, 76–79 (2020). Ćorović et al. The real time detection of traffic participants using YOLO algorithm. In Proc. 26 th Telecomm. Forum, Belgrade, Serbia, 1–4 (2018). Lou et al. Vehicle detection of traffic flow video using deep learning. In Proc. IEEE 8 th Data Driv. Cont. Learn. Sys. Conf., Dali, China, 1012–1017, IEEE Press (2019). Machiraju et al. Object detection and tracking for community surveillance using transfer learning. In Proc. 6 th Int. Conf. Inv. Comp. Tech., Coimbatore, India, 1035–1042 (2021). Snegireva et al. Vehicle classification application on video using YOLOv5 architecture. In Proc. Int. Russian Auto. Conf., Sochi, Russia, 1008–1013 (2021). Jana et al. YOLO based detection and classification of objects in video records. In Proc. 3 rd IEEE Int. Conf. Rec. Trends Elec., Info. Comm. Tech., India, 2448–2452, IEEE Press (2018). Hu et al. A video streaming vehicle detection algorithm based on YOLOv4. In Proc. IEEE 5 th Adv. Info. Tech., Elec. Auto. Con. Conf., Chongqing, China, 2081–2086, IEEE Press (2021). Kasper et al. Detecting heavy goods vehicles in rest areas in winter conditions using YOLOv5. Algorithms , 14 , 114 (2021). De Carvalho et al. Bounding box free instance segmentation using semi-supervised iterative learning for vehicle detection. IEEE J. Selec. Top. App. Earth Obsv. Remo. Sens. , 15 , 3403–3420 (2022). Tayara et al. Vehicle detection and counting in high-resolution aerial images using convolutional regression neural network. IEEE Access , 6 , 2220–2230 (2018). Zhao et al. Improved vision based vehicle detection and classification by optimized YOLOv4. IEEE Access , 10 , 8590–8603 (2022). Hassaballah et al. Vehicle detection and tracking in adverse weather using deep learning framework. IEEE Tran. Int. Trans. Sys. , 22 , 4230–4242 (2022). Mahmood et al. Towards automatic license plate detection. Sensors , 22 , 1245 (2022). Wu et al. Real time vehicle and distance detection based on improved YOLOv5 network. In Proc. 3 rd World Symp. Art. Int., Guangzhou, China. 24–28 (2021). Lin et al. Microsoft COCO: Common objects in context. In Proc. 13 th Euro. Conf. Comp. Vis., Zurich, Switzerland, 740–755 (2014). He et al. Bounding box regression with uncertainty for accurate object detection. In Proc. IEEE Conf. Comp. Vis. Pat. Reco., Long Beach, CA, USA. 2888–2897, IEEE Press (2019). Liu et al. Dynamic vehicle detection with sparse point clouds based on PE-CPD. IEEE Tran. Int. Trans. Sys. , 20 , 1964–1977 (2019). Zhao et al. M2Det: A single shot object detector based on multi-level feature pyramid network. AAAI Conf. Art. Int., Montréal, Canada, 33 , 9259–9266 (2019). Li et al. Scale aware trident networks for object detection. arXiv :1901.01892 (2019). Zhang et al. Mask SSD: An effective single-stage approach to object instance segmentation. IEEE Tran. Image Proc. , 29 , 2078–2093 (2020). Min et al. A new approach to track multiple vehicles with the combination of robust detection and two classifiers. IEEE Tran. Int. Trans. Sys. , 19 , 174–186 (2018). Law et al. CornerNet: Detecting objects as paired key points. Euro. Conf. Comp. Vis., Glasgow, UK, 734–750 (2018). He et al. Mask R-CNN. In Proc. IEEE Int. Conf. Comp. Vis., Venice, Italy, 2961–2969, IEEE Press (2017). Shi et al. Orientation aware vehicle detection in aerial images via an anchor free object detection approach. IEEE Tran. Geo. Remo. Sens. , 59 , 5221–5233 (2021). Hu et al. SINet: A scale insensitive convolutional neural network for fast vehicle detection. IEEE Tran. Int. Trans. Sys. , 20 , 1010–1019 (2019). Farid et al. A fast and accurate real time vehicle detection method using deep learning for unconstrained environments. App. Sci. , 13 , 3059 (2023). Chaudhuri, A. Benchmarked datasets for vehicle segmentation. Tech. Rep. , Samsung R & D Inst. New Delhi, India (2018). Ronneberger et al. U-Net: convolutional net- works for biomedical image segmentation. Lect. Notes in Comp. Sci., 9351 , 234–241, Springer Verlag (2015). COCO Datasets: https://cocodataset.org/#home DAWN Datasets: https://paperswithcode.com/dataset/dawn Tables Tables 1 to 4 are available in the Supplementary Files section. Additional Declarations No competing interests reported. Supplementary Files Tables.docx Cite Share Download PDF Status: Published Journal Publication published 06 May, 2024 Read the published version in Scientific Reports → Version 1 posted Editorial decision: Revision requested 04 Mar, 2024 Editor assigned by journal 19 Feb, 2024 Editor invited by journal 06 Feb, 2024 Submission checks completed at journal 06 Feb, 2024 First submitted to journal 11 Jan, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-3854436","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":267021645,"identity":"66ba855b-49fd-4984-a962-edb560eb2238","order_by":0,"name":"Arindam Chaudhuri","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABEUlEQVRIiWNgGAWjYFACxgYGHiBlwJDABqQkGPhBggkFRGlJhmiRbABpMSBgEZIWIOMAmMStmn/24bYHb2q25Zmz5x978OGPhb3x+dWJHx4YMMjzix3AqkXiXGK74Zxjt4stex6zG85sk0jcduPtZgmgwwxnzk7Abs0ZxjZpHrbbiRtuJLNJ8zZIJJjdOLsBpCXB4DZ2LfJgLf+gWv78kbA3nnF28w98WgxAWnjboFoY2CQYN/D3bsNriyFQi+TcvtvFBmcem0n2Av0y4wbvNosEAwmcfpE7w/5M4s2323kGxxOfSfz4U2fP3392880fFTby/NI4vA8FSLISYLYEXuVoWvgPEFQ9CkbBKBgFIwsAAHDxZAjOfSgZAAAAAElFTkSuQmCC","orcid":"","institution":"Great Lakes Institute of Management Chennai","correspondingAuthor":true,"prefix":"","firstName":"Arindam","middleName":"","lastName":"Chaudhuri","suffix":""}],"badges":[],"createdAt":"2024-01-11 18:44:12","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-3854436/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-3854436/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1038/s41598-024-60596-4","type":"published","date":"2024-05-06T16:54:17+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":49601331,"identity":"613500e4-be64-4dba-a1b8-501e99440c71","added_by":"auto","created_at":"2024-01-15 05:07:18","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":192797,"visible":true,"origin":"","legend":"\u003cp\u003eA real life crowded place from Indian traffic\u003c/p\u003e","description":"","filename":"image1.png","url":"https://assets-eu.researchsquare.com/files/rs-3854436/v1/1a7ff97ef017f5d5dcc95d0e.png"},{"id":49601758,"identity":"58ca730a-1501-437e-b0af-faa185c8ffb0","added_by":"auto","created_at":"2024-01-15 05:15:18","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":876632,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003e2(a)\u003c/strong\u003e Sample images from dataset - low density traffic\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e2(b)\u003c/strong\u003e Sample images from dataset - high density traffic\u003c/p\u003e","description":"","filename":"image2.png","url":"https://assets-eu.researchsquare.com/files/rs-3854436/v1/e8af2875e4068d05232ff2a1.png"},{"id":49601757,"identity":"630a5d8f-349b-4857-bfb7-44ada9374364","added_by":"auto","created_at":"2024-01-15 05:15:18","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":1009249,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003e3(a)\u003c/strong\u003e Sample annotated images from dataset - video dataset\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e3(b) \u003c/strong\u003eSample annotated images from dataset - annotated images\u003c/p\u003e","description":"","filename":"image3.png","url":"https://assets-eu.researchsquare.com/files/rs-3854436/v1/065c6d9bb725399659f2e4bf.png"},{"id":49601756,"identity":"1bd1ccd7-51bc-405f-b74a-44bc50684b37","added_by":"auto","created_at":"2024-01-15 05:15:18","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":67414,"visible":true,"origin":"","legend":"\u003cp\u003eArchitecture of vehicle segmentation method\u003c/p\u003e","description":"","filename":"image6.png","url":"https://assets-eu.researchsquare.com/files/rs-3854436/v1/35889ac0fccc910ee4b141b1.png"},{"id":49601332,"identity":"3de37d6a-c990-49fb-8130-9cc7cc79b126","added_by":"auto","created_at":"2024-01-15 05:07:18","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":735267,"visible":true,"origin":"","legend":"\u003cp\u003eCOCO dataset vehicle detection results\u003c/p\u003e","description":"","filename":"image7.png","url":"https://assets-eu.researchsquare.com/files/rs-3854436/v1/c307eabbfb02a0472224b733.png"},{"id":49601334,"identity":"39308800-85da-40f3-85da-ad1d305f4bb8","added_by":"auto","created_at":"2024-01-15 05:07:18","extension":"png","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":620875,"visible":true,"origin":"","legend":"\u003cp\u003eVehicle detection results on DAWN datasets\u003c/p\u003e","description":"","filename":"image8.png","url":"https://assets-eu.researchsquare.com/files/rs-3854436/v1/8930e4badfcd74af61f34f3d.png"},{"id":49601336,"identity":"d9a89719-32cb-4f95-aa5d-a9c00281f9e2","added_by":"auto","created_at":"2024-01-15 05:07:18","extension":"png","order_by":7,"title":"Figure 7","display":"","copyAsset":false,"role":"figure","size":626161,"visible":true,"origin":"","legend":"\u003cp\u003eVehicle detection method on sample images\u003c/p\u003e","description":"","filename":"image9.png","url":"https://assets-eu.researchsquare.com/files/rs-3854436/v1/2a6ccc6ac76f5316e2c1185f.png"},{"id":56194377,"identity":"83a1a1aa-991e-40b5-81d6-b9e8e5ca13b8","added_by":"auto","created_at":"2024-05-09 17:52:47","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":4899285,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-3854436/v1/d6eb0942-3538-4540-970c-89d948a4102f.pdf"},{"id":49601338,"identity":"b87eee8f-c8c5-4b29-b795-c49062437758","added_by":"auto","created_at":"2024-01-15 05:07:19","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":6371631,"visible":true,"origin":"","legend":"","description":"","filename":"Tables.docx","url":"https://assets-eu.researchsquare.com/files/rs-3854436/v1/3bdb0b286dccfba9145a5095.docx"}],"financialInterests":"No competing interests reported.","formattedTitle":"Smart Traffic Management of Vehicles using Faster R-CNN based Deep Learning Method","fulltext":[{"header":"1 Introduction","content":"\u003cp\u003eWe are facing the challenge of continuous increase in traffic density across different cities of world. The present world population is heavily dependent on vehicles from smooth commutation purposes. This results from constant urbanization of rural areas. In order to address this problem, smart traffic management is studied by researchers as traffic regulations solution.\u003c/p\u003e \u003cp\u003eHuman vision system has the capability to perform complex tasks very reliably and accurately. Humans detect wide spectrum of objects very easily. With recent developments in computer vision coupled with huge data sets, better algorithms and faster GPUs, precision and accuracy of object detection and classification algorithms\u003csup\u003e1, 2\u003c/sup\u003e have increased to an appreciable amount. For traffic monitoring vehicle localization efficiency is very important. The vehicles from Indian traffic are presented in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e. For public safety autonomous vehicle detection methods\u003csup\u003e3\u003c/sup\u003e are in place which detect traffic objects in order to achieve correct decisions.\u003c/p\u003e \u003cp\u003eSmart traffic management allows us to take care of various traffic related issues in an optimum manner. This is achieved using computer vision and image processing. Segmentation of vehicle acts as significant enabler in smart traffic management functionality. With occlusions, congestion as well as other environmental factors this problem becomes more severe. For smooth and effective traffic regulations, smart traffic management is always considered as low cost solution. Congestions, emergency vehicles transport, accidents and traffic related violations are easily managed through latest Artificial Intelligence algorithms. Vehicle segmentation is an important area towards smart traffic management. Some other activities here include traffic estimation, speed control and vehicle tracking. During occlusions, fog, haze, clutters and heavy traffic situations things become more complicated.\u003c/p\u003e \u003cp\u003eWith the growth of deep learning networks vehicle detection is being studied deeply considering traffic congestion and driving safety\u003csup\u003e4\u003c/sup\u003e. Vehicle localization is a crucial problem\u003csup\u003e5\u003c/sup\u003e in order to develop intelligent and autonomous systems. The detection of abnormalities arising from traffic violations leads to the problem of vehicle localization. This is a significant application catering the needs for variety of traffic related problems. The traffic surveillance has been a major concern in densely populated geographical areas. These days\u0026rsquo; surveillance systems are well equipped with traffic flow data where various traffic patterns are recorded. Some notable applications here include many smart cities based applications. Different versions of deep learning network based methods are in place which have provided considerable benefits for these applications\u003csup\u003e3\u003c/sup\u003e.\u003c/p\u003e \u003cp\u003eIn this research work, we investigate Faster R-CNN based deep learning method towards segmentation of vehicles. This problem is addressed in four steps\u003csup\u003e5\u003c/sup\u003e: (a) minimization with adaptive background model (b) Faster R-CNN based subnet operation (c) Faster R-CNN initial refinement and (d) result optimization with extended topological active nets. Adaptive gain function is used adaptive background modeling. The gain function compensates for shadow and illumination issues. Higher accuracy in segmentation is provided by topological active net deformable model in various situations. Deformable models provide various curvatures with respect to image surfaces. The smoothness from deformation is achieved through several forces considering objects of interest. The topological and extended topological active nets are used here in order to achieve stated deformations. This helps in fitting objects on 2D surface mesh of image. The deformation of mesh is achieved with minimization of energy. For problems with complex shapes energy is changed through extended topological active net with certain thresholds. The segmentation accuracy is increased with improved version of extended topological active net. This solution has combined effect from all models using which better segmentation boundaries are achieved.\u003c/p\u003e \u003cp\u003eThe performance of stated method is compared with respect to some important metrics such as precision and recall. This method provides better segmentation performance in comparison to other methods. The experimental hypothesis is justified with benchmarked datasets. The model provides appreciable results for different sets of image datasets. This chapter is structured as follows. The literature review is presented in section \u003cspan refid=\"Sec2\" class=\"InternalRef\"\u003e2\u003c/span\u003e. In section \u003cspan refid=\"Sec3\" class=\"InternalRef\"\u003e3\u003c/span\u003e highlights detailed discussion on computational method. The simulation results are shown in section \u003cspan refid=\"Sec9\" class=\"InternalRef\"\u003e4\u003c/span\u003e. Finally, conclusion is provided in section \u003cspan refid=\"Sec10\" class=\"InternalRef\"\u003e5\u003c/span\u003e.\u003c/p\u003e"},{"header":"2 Literature Review","content":"\u003cp\u003eDuring recent decades\u0026rsquo; smart traffic management involving vehicle detection has gained considerable attention. Some of the notable works are discussed here. In\u003csup\u003e6\u003c/sup\u003e a pixel wise classification method based on dynamic bayesian network for vehicle detection is proposed. In\u003csup\u003e7\u003c/sup\u003e an object detection scheme is presented which identifies changes in image series. Foreground vehicle segmentation using gaussian mixture model is highlighted in\u003csup\u003e8\u003c/sup\u003e. An adaptive background model having frames averaging with respect to time is discussed in\u003csup\u003e9\u003c/sup\u003e. For vehicle localization ResNet model is used in\u003csup\u003e10\u003c/sup\u003e. A vehicle classification system involving deep learning is presented in\u003csup\u003e11\u003c/sup\u003e. An integrated vehicle detection and classification method is discussed in\u003csup\u003e12\u003c/sup\u003e. \u003csup\u003e13\u003c/sup\u003ediscusses YOLOv3 algorithm for vehicle detection. \u003csup\u003e14\u003c/sup\u003epresents receptive field based neural network. Semantic image segmentation is used in\u003csup\u003e15\u003c/sup\u003e. Mask R-CNN with transfer learning is used in\u003csup\u003e16\u003c/sup\u003e. \u003csup\u003e17\u003c/sup\u003ediscusses YOLO based solution. A review on vehicle detection is highlighted in\u003csup\u003e18\u003c/sup\u003e. In\u003csup\u003e19\u003c/sup\u003e deep learning assisted vehicle segmentation is discussed.\u003c/p\u003e \u003cp\u003eIn\u003csup\u003e20\u003c/sup\u003e vehicles in airborne images are detected. An ensemble based method using image descriptors is discussed in\u003csup\u003e21\u003c/sup\u003e. In\u003csup\u003e22, 23\u003c/sup\u003e methods are developed through application of gaussian mixture model. In\u003csup\u003e24\u003c/sup\u003e support vector machines is used for vehicle detection. SIFT algorithm is integrated with support vector machines in\u003csup\u003e25\u003c/sup\u003e to achieve vehicle detection. For autonomous vehicles an object detection system is highlighted in\u003csup\u003e26\u003c/sup\u003e. Some R-CNN version of vehicle detection algorithms are discussed in\u003csup\u003e28, 44, 54\u003c/sup\u003e. Several significant YOLO based multi object vehicle detection algorithms are highlighted in\u003csup\u003e27. 32, 33, 34, 35, 37, 38, 39, 42\u003c/sup\u003e. Vehicle detection algorithms in different weather conditions are presented in\u003csup\u003e29, 30, 52\u003c/sup\u003e. Deep learning networks for vehicle detection are used in\u003csup\u003e31, 36, 41, 43, 45, 46, 55, 56\u003c/sup\u003e. Several other notable works are available in\u003csup\u003e40, 47, 48, 49, 50, 51, 53, 57\u003c/sup\u003e.\u003c/p\u003e"},{"header":"3 Computational Method","content":"\u003cp\u003eIn this section, smart traffic vehicle management using Faster R-CNN based deep learning based ensemble method is highlighted. The research problem revolves around traffic management which involves vehicle segmentation at different background levels\u003csup\u003e5\u003c/sup\u003e. In order to achieve this, we study Faster R-CNN\u003csup\u003e5\u003c/sup\u003e which analyses vehicles in smart traffic. The segmentation activity on various strategic areas of smart traffic analytics provide information related to decision-making activities.\u003c/p\u003e\n\u003cdiv id=\"Sec4\" class=\"Section2\"\u003e\n\u003ch2\u003e3.1 Datasets Used\u003c/h2\u003e\n\u003cp\u003eThe experimental datasets are adopted from\u003csup\u003e58\u003c/sup\u003e. The datasets are prepared keeping in view different traffic conditions. At the very first instance multi-class vehicle objects are considered. Several challenge factors such as traffic jams and overlapping vehicles are incorporated in dataset. Broadly speaking datasets are developed with respect to two different situations viz high density and low density traffic scenes. The former comes with several objects in single image while later has single class per image having no overlapping. In order to achieve better training, images from both above mentioned situations are placed in different datasets. The high density traffic scenes are considered from several places having traffic which are less crowded in nature. A total of 1000 images from six classes of vehicles viz two wheelers, three wheelers, four wheelers, six wheelers, eight wheelers and ten wheelers are developed. Figure\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e(a) and 2(b) highlight some sample images. High density traffic scenes are considered from daily traffic which are congested in nature. 5000 images from above stated classes are created. Certain critical factors such as illumination, occlusions etc are incorporated irrespective of appearance, shape, scale and size in this dataset. Table\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e highlights certain high and low density dataset statistics alongwith annotations of class. These datasets are preprocessed and augmented as discussed in subsection \u003cspan class=\"InternalRef\"\u003e3.2\u003c/span\u003e. Each of these datasets are benchmarked with respect to standard datasets\u003csup\u003e58\u003c/sup\u003e.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec5\" class=\"Section2\"\u003e\n\u003ch2\u003e3.2 Data Annotation and Augmentation\u003c/h2\u003e\n\u003cp\u003eIn order to achieve reliable vehicle detection dataset classes are labelled. Based on motivation from\u003csup\u003e5\u003c/sup\u003e an image tool\u003csup\u003e57\u003c/sup\u003e has been used here towards labeling and annotation of this dataset. As shown in Fig.\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e(a) and 3(b) for each object in image, a bounding box is assigned manually. In high density datasets (HDD) from high density traffic many bounding boxes are present. In low density datasets (LDD) from low density traffic few bounding boxes are present in single image. The label of respective class is specified by these bounding boxes. As mentioned in subsection \u003cspan class=\"InternalRef\"\u003e3.1\u003c/span\u003e, the whole dataset is defined through six classes.\u003c/p\u003e\n\u003cp\u003eIn order to increase features of datasets such that better results are obtained data preprocessing and augmentation is used. These addresses various issues which can be present in images such as noise, inconsistency and unbalanced classes. The data augmentation process used here are in lines with discussed in\u003csup\u003e5, 57\u003c/sup\u003e. Since augmented dataset has an unknown distribution it is benchmarked adhering to certain standards\u003csup\u003e58\u003c/sup\u003e.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec6\" class=\"Section2\"\u003e\n\u003ch2\u003e3.3 Faster R-CNN based Method\u003c/h2\u003e\n\u003cp\u003eNow we present a detailed description of proposed method. The method is highlighted considering four steps\u003csup\u003e5\u003c/sup\u003e viz (a) minimization with adaptive background model (b) Faster R-CNN based subnet operation (c) Faster R-CNN initial refinement and (d) result optimization with extended topological active nets.\u003c/p\u003e\n\u003cp\u003eAdaptive background modeling is used to construct background with traffic video as input. The video frames are analyzed in order to develop background model. The objective here is to find best background estimate. With this on foreground model shadow and illumination change impacts are minimized. This process is initiated by first initializing few frames and then continuous updates are done each time. This leads to extraction of foreground from next set of frames. The initialization of background model is done considering first frame\u0026rsquo;s pixel values. A subsequent update of model is performed through calculation of value of pixel considering background model:\u003c/p\u003e\n\u003cdiv id=\"Equ1\" class=\"Equation\"\u003e\n\u003cdiv id=\"FileID_Equ1\" class=\"mathdisplay\"\u003e$${pixel}_{j}\\left(y\\right)={\\stackrel{-}{pixel}}_{j-1}\\left(y\\right)+\\frac{1}{{GP}_{j}\\sqrt{2\\pi {\\sigma }^{2}}}\\text{exp}\\left(\\frac{-1}{2}{\\left(\\frac{y-{y}_{j}}{\\sigma }\\right)}^{2}\\right)$$\u003c/div\u003e\n\u003cdiv class=\"EquationNumber\"\u003e1\u003c/div\u003e\n\u003c/div\u003e\n\u003cp\u003eHere \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({y}_{j}\\)\u003c/span\u003e\u003c/span\u003e represents value of pixel with respect to \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({j}^{th}\\)\u003c/span\u003e\u003c/span\u003e frame. The value of \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\stackrel{-}{pixel}}_{j}\\)\u003c/span\u003e\u003c/span\u003e is computed as follows:\u003c/p\u003e\n\u003cdiv id=\"Equ2\" class=\"Equation\"\u003e\n\u003cdiv id=\"FileID_Equ2\" class=\"mathdisplay\"\u003e$${\\stackrel{-}{pixel}}_{j-1}\\left(y\\right)=\\frac{{pixel}_{j}\\left(y\\right)}{\\sum _{j}{pixel}_{j}\\left(y\\right)}$$\u003c/div\u003e\n\u003cdiv class=\"EquationNumber\"\u003e2\u003c/div\u003e\n\u003c/div\u003e\n\u003cp\u003eHere number of frames considered is \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(N\\)\u003c/span\u003e\u003c/span\u003e. The gain parameter is denoted by \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({GP}_{j}\\)\u003c/span\u003e\u003c/span\u003e. It takes care of background modeling\u0026rsquo;s learning rate. The new information is learned by increasing \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({GP}_{j}\\)\u003c/span\u003e\u003c/span\u003e as presented in Eq.\u0026nbsp;(\u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e) with prior information disappearing slowly.\u003c/p\u003e\n\u003cdiv id=\"Equ3\" class=\"Equation\"\u003e\n\u003cdiv id=\"FileID_Equ3\" class=\"mathdisplay\"\u003e$${GP}_{j}=Gain\\left(\\frac{2}{1+\\text{exp}\\left(-\\frac{cont-\\alpha }{\\beta }\\right)}\\right)$$\u003c/div\u003e\n\u003cdiv class=\"EquationNumber\"\u003e3\u003c/div\u003e\n\u003c/div\u003e\n\u003cp\u003eHere \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(Gain\\)\u003c/span\u003e\u003c/span\u003e and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\alpha\\)\u003c/span\u003e\u003c/span\u003e parameters take care of sinusoid function\u0026rsquo;s inflection point. The parameter \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\beta\\)\u003c/span\u003e\u003c/span\u003e controls gradient. The parameter \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(cont\\)\u003c/span\u003e\u003c/span\u003e depends on number of frames. Background model is updated for every frame. Considering every fame with background as adaptive, after its subtraction we reach to foreground objects. After adaptive background model based minimization is performed, we present basic Faster R-CNN architecture used in this research. The architecture of Faster R-CNN is considered from\u003csup\u003e5\u003c/sup\u003e having variation based baseline considered from\u003csup\u003e70\u003c/sup\u003e. The architecture is highlighted in Fig.\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e4\u003c/span\u003e. In order to validate proposed method, datasets mentioned in subsection \u003cspan class=\"InternalRef\"\u003e3.1\u003c/span\u003e are used.\u003c/p\u003e\n\u003cp\u003eThe convolutional feature map is developed when entire image is processed convolutional and max pooling layers. For each object\u0026rsquo;s RoI pooling layer, feature vector of fixed length is extracted. The input of sequence of fully connected layers are feature vectors. From a set of fully connected layers, output passes to sibling layers where softmax probability estimates are produced. The object classes estimate softmax probability. Here background class with set of other layers are considered. The encoding is performed on set of values with respect to refined positions in bounding box for each object class. For RoI pooling layer we use max pooling. This allows conversion of features into small feature map. Here convolutional feature map based RoI is defined through 4-tuple. RoI max pooling divides rectangular window into sub window grids. The channel independent pooling is applied for each feature map. The network weights are trained through backpropagation in Faster R-CNN. A hierarchical sampling of RoIs is performed for each image. Stochastic gradient descent (SGD) training for Faster R-CNN is done in mini-batches. For training with larger datasets, execution of SGD is done for more iterations. Faster R-CNN makes use of sibling output layers. For each RoI we have discrete probability distribution as initial output. The probabilities for fully connected layer have outputs as softmax. The labeling for RoI is performed with respect to ground truth class. For each training bounding box regression having ground truth is considered. For each labeled RoI considering multitask loss, joint classification is present with respect to training and bounding-box regression. The optimization of multitask loss is performed as highlighted in\u003csup\u003e5\u003c/sup\u003e. For whole image classification, convolutional layers\u0026rsquo; calculation time is greater than fully connected layers. RoIs processing time is appreciably large for detection.\u003c/p\u003e\n\u003cp\u003eNow subnet operation with Faster R-CNN is presented. The foreground image considers a mesh. This is moved towards subnet operation for Faster R-CNN. The training image dataset is adopted from\u003csup\u003e19\u003c/sup\u003e. This is marked with respect to active net grids. The subnet is represented is binary matrix with 1 as indicator for link presence of mesh and 0 as indicator for link absence. Here Faster R-CNN is trained where features are learned in order to produce ground truth specific results. The U-Net based CNN\u003csup\u003e59\u003c/sup\u003e helps us to perform initial subnet operation.\u003c/p\u003e\n\u003cp\u003eThe extended topological active net refines initial subnet. Faster R-CNN produces hole based mesh considering all background. The objects have mesh nodes. Those mesh nodes which where boundary is not crossed are removed. It is reached though extended topological active net energy minimization. The energy function used here is highlighted in Eq.\u0026nbsp;(\u003cspan class=\"InternalRef\"\u003e4\u003c/span\u003e). By using Eq.\u0026nbsp;(\u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e) adaptive background model is developed.\u003c/p\u003e\n\u003cdiv id=\"Equ4\" class=\"Equation\"\u003e\n\u003cdiv id=\"FileID_Equ4\" class=\"mathdisplay\"\u003e$$Energy\\left(w\\left(a,b\\right)\\right)=\\underset{0}{\\overset{1}{\\int }}\\underset{0}{\\overset{1}{\\int }}\\left[{Energy}_{internal}\\left(w\\left(a,b\\right)\\right)+{Energy}_{external}\\left(w\\left(a,b\\right)\\right)\\right]dadb$$\u003c/div\u003e\n\u003cdiv class=\"EquationNumber\"\u003e4\u003c/div\u003e\n\u003c/div\u003e\n\u003cp\u003eThe mesh deformation is done using greedy approach. In situations with complex deformation of shapes, extended topological active net produces different energy term. This it achieves through various thresholds. When clutters and occlusion are present in vehicle image segmentation extended topological active net always reaches local minima. This results in low segmentation accuracy. The improved version of extended topological active net provides resolution to these problems. The combined effect of all models used here leads us to better results.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec7\" class=\"Section2\"\u003e\n\u003ch2\u003e3.4 Training Process\u003c/h2\u003e\n\u003cp\u003eThe training process is now briefly discussed. The annotated and augmented data is trained using Faster R-CNN algorithm. In order to perform training, several parameters like size of batch, epochs needed and resolution of image are used. Since network is trained from scratch, random weight initialization is performed. Here initially trained COCO weights\u003csup\u003e5\u003c/sup\u003e are used towards model training with appreciable time and computation benefits. The best weight values are obtained using initially trained Faster R-CNN having transfer learning. \u003csup\u003e5\u003c/sup\u003edatasets are used as benchmark in order to train stated custom datasets. The batch sizes of 5, 10, 20, 30 and 40 are considered. The epochs are also changed to 100, 200, 300, 400 and 500. The confidence values are considered between 0.4 to 0.6. The best weights are used to detect objects in datasets. The predicted labels and assessment images incorporating bounding boxes with confidence values are also obtained.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec8\" class=\"Section2\"\u003e\n\u003ch2\u003e3.5 Evaluation Criteria\u003c/h2\u003e\n\u003cp\u003eIn this research precision and recall criteria\u003csup\u003e5\u003c/sup\u003e are used for measuring robustness. Here, mean average precision (mAP) is also calculated for recall values lying between 0 to 1.\u003c/p\u003e\n\u003c/div\u003e"},{"header":"4 Experiments and Results","content":"\u003cp\u003eHere a detailed discussion on simulation results is presented. We conducted detailed experiments in Google Colab having T4 GPU with Intel Xeon CPU and 64 GB of RAM. Python version 3.11.5 has been used as simulation tool in this research. In order to assess vehicle detection method performance, several state-of-the-art detectors are evaluated. Also various comparisons are performed with stated method considering accuracy and execution times. All methods are trained on data adapted from COCO [60] and DAWN\u003csup\u003e61\u003c/sup\u003e datasets.\u003c/p\u003e\n\u003cp\u003eThe detection and segmentation of objects is performed by COCO dataset considering natural contexts\u003csup\u003e5\u003c/sup\u003e. In Table\u0026nbsp;2 COCO dataset highlights several objects collected from complex scenes. The dataset has images of 100 different object types with 3\u0026nbsp;million instance labels. The results are highlighted in Fig.\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e5\u003c/span\u003e. Certain YOLOv5 semantics are adopted from\u003csup\u003e57\u003c/sup\u003e. All vehicles are accurately detected by stated method considering variation in illumination. In order to further validate superiority of stated method we perform a comparative analysis with 14 methods as presented in Table\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e. Some significant observations are briefly highlighted here.\u003c/p\u003e\n\u003cp\u003eFor images with different resolutions, stated computational structure provides best performance in terms of mAP values. The methods presented in\u003csup\u003e49, 50, 55, 56\u003c/sup\u003e also provide promising results with respect to COCO datasets. \u003csup\u003e54\u003c/sup\u003emakes use of ResNet and produces results on lower side for object detection in COCO datasets. Similar results are obtained from methods presented in\u003csup\u003e45, 47\u003c/sup\u003e. Here miscellaneous objects are detected on COCO datasets with promising results. These include vehicles of varying shapes and sizes. All results are highlighted in Fig.\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e5\u003c/span\u003e.\u003c/p\u003e\n\u003cp\u003eThe DAWN dataset is used in order to study and investigate performance of stated method. DAWN dataset has 2000 image with different variations\u003csup\u003e5\u003c/sup\u003e. It shows varying traffic situations in different weather conditions. Figure\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e6\u003c/span\u003e shows fairly detailed results considering significant observations. The stated method addresses all prevailing weather situations such as rainy days, normal dry days and snowy days.\u003c/p\u003e\n\u003cp\u003eA comparison is performed with results presented in Table\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e4\u003c/span\u003e. Some observations are highlighted here. In fog situation\u003csup\u003e54\u003c/sup\u003e has highest rank. Our method exceeds results of\u003csup\u003e54\u003c/sup\u003e. Here\u003csup\u003e43, 52\u003c/sup\u003e have lowest ranks. In rain situation our method produces best results followed by\u003csup\u003e57, 45, 48, 50, 51\u003c/sup\u003e. Here\u003csup\u003e52\u003c/sup\u003e has lowest rank. In snow situation our method has highest rank followed by\u003csup\u003e51, 57\u003c/sup\u003e. Here\u003csup\u003e45, 47, 50, 51\u003c/sup\u003e produce similar results. In dry day situation stated method achieves highest rank. Here\u003csup\u003e42, 48, 49\u003c/sup\u003e yield similar results. \u003csup\u003e52\u003c/sup\u003ehas lowest rank.\u003c/p\u003e\n\u003cp\u003eNow we present some additional insights in this research with respect to diverse range of environments. All state-of-the-art object detection methods have been studied in this research. These results are briefly discussed here.\u003c/p\u003e\n\u003cp\u003eIn\u003csup\u003e42\u003c/sup\u003e BIT-Vehicle and UADETRAC datasets are investigated. In\u003csup\u003e57\u003c/sup\u003e 3 different datasets are studied which several variations with respect to road conditions, weather as well as complex background. In\u003csup\u003e43, 48\u003c/sup\u003e primary concentrations are on KITTI and DAWN datasets with certain deviations which produces appreciable results. In\u003csup\u003e44\u003c/sup\u003e investigation of object detection capability is performed on COCO and DAWN datasets. In\u003csup\u003e45\u003c/sup\u003e CARLA dataset is explored. The results presented here extends method's detection capability to 3 other datasets. PASCAL VOC 2007 dataset is studied in\u003csup\u003e47, 51\u003c/sup\u003e. In\u003csup\u003e37\u003c/sup\u003e PASCAL dataset is studied which contains annotated images of various objects. Here detection capability of method is expanded to 3 different vehicle datasets. In\u003csup\u003e49, 50, 53, 54\u003c/sup\u003e COCO and DAWN datasets are used in order to validate methods for object detection. In\u003csup\u003e54\u003c/sup\u003e various object detection methods are presented for detection of different objects. PETS2009 and changedetection.net 2012 datasets are studied in\u003csup\u003e52\u003c/sup\u003e with DLR Munich vehicle and VEDAI datasets in\u003csup\u003e55\u003c/sup\u003e and KITTI and LSVH datasets in\u003csup\u003e56\u003c/sup\u003e. The vehicle detection methods on these datasets produce appreciable results.\u003c/p\u003e\n\u003cp\u003eIn DAWN dataset has more challenging images with fog, rain, dry day and snow. However, as shown in Fig.\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e7\u003c/span\u003e very low vehicle detection results are achieved. But our method performs well in this situation. Considering several state-of-the-art approaches, a detailed comparison of is performed alongwith our method. It is seen that in different situations our method performs well. In COCO dataset our method produces appreciable results in comparison to other methods.\u003c/p\u003e"},{"header":"5 Conclusion","content":"\u003cp\u003eIn this research work smart traffic management of vehicles is studied using Faster R-CNN based deep learning method. It is an intractable problem in computer vision and artificial intelligence domain. When occlusions, background with clutters and traffic with density variations are present, this problem becomes more challenging. The computational paradigm involves four steps viz minimization with adaptive background model, Faster R-CNN based subnet operation, Faster R-CNN initial refinement and result optimization with extended topological active nets. The concept of adaptive background modeling is incorporated in this framework. The shadow and illumination related issues are also addressed. The topological active net deformable models help to achieve higher segmentation accuracy. The deformations are reached with topological and extended topological active nets. Mesh deformation helps in minimization of energy. The segmentation accuracy is improved with modified version of extended topological active net. The superiority of this method is highlighted with experimental results.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eData Availability:\u0026nbsp;\u003c/strong\u003eAll data used in this research has been developed at Samsung R \u0026amp; D Institute New Delhi India as mentioned in Reference 58. The Institution does not allow to share and provide access to research data to public domains. In view of this, data used in this research cannot be shared. However, two more datasets COCO and DAWN used in this research are highlighted in References 60 and 61 can be shared and accessed.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting Interests:\u0026nbsp;\u003c/strong\u003eAuthor has no competing interests to declare which are relevant to content of this article.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthor Contribution Statement:\u0026nbsp;\u003c/strong\u003eAuthor contributed to the study conception and design. Material preparation, data collection and analysis were performed by Arindam Chaudhuri. The first draft of manuscript was written by Arindam Chaudhuri and author commented on previous versions of manuscript. Author read and approved the final manuscript.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eEthical and Informed Consent for Data used:\u003c/strong\u003e Author has ethical and informed consent for data used in this research.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eMahmood et al. Towards a Fully Automated Car Parking System. \u003cem\u003eIET Int. Trans. Sys.\u003c/em\u003e, \u003cstrong\u003e13\u003c/strong\u003e, 293\u0026ndash;302 (2018). \u003c/li\u003e\n\u003cli\u003eXiaohong et al. Real time object detection based on YOLOv2 for tiny vehicle objects. In Proc. Comp. Sci., \u003cstrong\u003e183\u003c/strong\u003e, 61\u0026ndash;72, Elsevier Press (2021).\u003c/li\u003e\n\u003cli\u003eJamiya et al. Little YOLO-SPP: A delicate real time vehicle detection algorithm. \u003cem\u003eOptik\u003c/em\u003e, \u003cstrong\u003e225\u003c/strong\u003e, 165818. (2021).\u003c/li\u003e\n\u003cli\u003eTajar et al. A lightweight Tiny-YOLOv3 vehicle detection approach. \u003cem\u003eJ. of Real-Time Image Proc.\u003c/em\u003e, \u003cstrong\u003e18\u003c/strong\u003e, 2389\u0026ndash;2401 (2021). \u003c/li\u003e\n\u003cli\u003eChaudhuri, A. Smart traffic management in varying weather conditions. \u003cem\u003eTech. Rep.\u003c/em\u003e, Samsung R \u0026amp; D Inst. New Delhi, India (2018).\u003c/li\u003e\n\u003cli\u003eCheng et al. Vehicle detection in aerial surveillance using dynamic bayesian networks. \u003cem\u003eIEEE Tran. on Image Proc.\u003c/em\u003e, \u003cstrong\u003e21\u003c/strong\u003e(4), 2152\u0026ndash;2159 (2012).\u003c/li\u003e\n\u003cli\u003eMcHugh et al. Foreground adaptive background subtraction. \u003cem\u003eIEEE Sig. Proc. Lett.\u003c/em\u003e, \u003cstrong\u003e16\u003c/strong\u003e, 390\u0026ndash;393 (2009). \u003c/li\u003e\n\u003cli\u003eChen et al. Road vehicle classification using support vector machines. In Proc. IEEE Int. Conf. Intel. Comp. Intel. Sys., IEEE Press (2009). \u003c/li\u003e\n\u003cli\u003eKul et al. Distributed and collaborative real time vehicle detection and classification over video streams. \u003cem\u003eInt. J. of Adv. Robo. Sys.\u003c/em\u003e, \u003cstrong\u003e14\u003c/strong\u003e(4), 1729881417720782 (2017).\u003c/li\u003e\n\u003cli\u003eJung et al. ResNet based vehicle classification and localization in traffic surveillance systems. In Proc. IEEE Conf. Comp. Vis. Pat. Rec. Workshops, Honolulu, 934\u0026ndash;940, IEEE Press (2017).\u003c/li\u003e\n\u003cli\u003eMaungmai et al. Vehicle classification with deep learning. In Proc. IEEE Int. Conf. Comp. Comm. Sys., Singapore, 294\u0026ndash;298, IEEE Press (2019). \u003c/li\u003e\n\u003cli\u003eBai et al. Classify vehicles in traffic scene images with deformable part based models. \u003cem\u003eMac. Vis. App.\u003c/em\u003e, \u003cstrong\u003e29\u003c/strong\u003e, 393\u0026ndash;403 (2017). \u003c/li\u003e\n\u003cli\u003eSong et al. Vision based vehicle detection and counting system using deep learning in highway scenes. \u003cem\u003eEuro. Trans. Res. Rev.\u003c/em\u003e, \u003cstrong\u003e11\u003c/strong\u003e, 51 (2019). \u003c/li\u003e\n\u003cli\u003eLee et al. Segmentation of vehicles and roads by low channel lidar. \u003cem\u003eIEEE Tran. Int. Tran. Sys.\u003c/em\u003e, \u003cstrong\u003e20\u003c/strong\u003e(11), 4251\u0026ndash;4256 (2019). \u003c/li\u003e\n\u003cli\u003eKaymak et al. Semantic image segmentation for autonomous driving using fully convolutional networks. In Proc. Int. Art. Int. Data Proc. Symp. (2019). \u003c/li\u003e\n\u003cli\u003eOjha et al. Vehicle detection through instance segmentation using mask R-CNN for intelligent vehicle system. In Proc. 5\u003csup\u003eth\u003c/sup\u003e Int. Conf. Int. Comp. Con. Sys., 954\u0026ndash;959 (2021). \u003c/li\u003e\n\u003cli\u003eShan et al. A real time vehicle detection algorithm based on instance segmentation. In Proc. IEEE 2\u003csup\u003end\u003c/sup\u003e Int. Conf. Big Data, Art. Int. IoT Eng, 239\u0026ndash;243, IEEE Press (2021). \u003c/li\u003e\n\u003cli\u003eShobha et al. A Review on video based vehicle detection, recognition and tracking, In Proc. 3\u003csup\u003erd\u003c/sup\u003e IEEE Int. Conf. Comp. Sys. Info. Tech. Sus. Soln., IEEE Press (2018).\u003c/li\u003e\n\u003cli\u003eShobha et al. Deep learning assisted active net segmentation of vehicles for smart traffic management. In Proc. Glob. Trans. \u003cstrong\u003e2\u003c/strong\u003e, 282\u0026ndash;286 (2021).\u003c/li\u003e\n\u003cli\u003eHamsa et al. Automatic vehicle detection from aerial images using cascaded support vector machine and gaussian mixture model. In Proc. Int. Conf. Sig. Proc. Info. Security, Dubai pp 1\u0026ndash;4 (2018).\u003c/li\u003e\n\u003cli\u003eMikaty et al. Detection of cars in high resolution aerial images of complex urban environments. \u003cem\u003eIEEE Tran. Geo. Remo. Sens.\u003c/em\u003e, \u003cstrong\u003e55\u003c/strong\u003e, 5913\u0026ndash;5924 (2017).\u003c/li\u003e\n\u003cli\u003eArı et al. Detection of compound structures using gaussian mixture model with spectral and spatial constraints. \u003cem\u003eIEEE Tran. Geo. Remo. Sens.\u003c/em\u003e, \u003cstrong\u003e52\u003c/strong\u003e, 6627\u0026ndash;6638 (2014). \u003c/li\u003e\n\u003cli\u003eHbaieb et al. Pedestrian detection for autonomous driving within cooperative communication system. IEEE Wire. Comm. Net. Conf., Morocco, 1\u0026ndash;6, IEEE Press (2019).\u003c/li\u003e\n\u003cli\u003eXiong et al. High speed front vehicle detection based on video multi-feature fusion. In Proceedings IEEE 10\u003csup\u003eth\u003c/sup\u003e Int. Conf. Elec., Info. Emer. Comm., Beijing, 348\u0026ndash;351, IEEE Press (2020).\u003c/li\u003e\n\u003cli\u003eYawen et al. Research on vehicle detection technology based on SIFT features. In Proc. 8\u003csup\u003eth\u003c/sup\u003e Int. Conf. Elec. Info. Emer. Comm., Beijing, 274\u0026ndash;278 (2018).\u003c/li\u003e\n\u003cli\u003eLi et al. A deep learning based hybrid framework for object detection and recognition in autonomous driving. \u003cem\u003eIEEE Access\u003c/em\u003e, \u003cstrong\u003e8\u003c/strong\u003e, 194228\u0026ndash;194239 (2020). \u003c/li\u003e\n\u003cli\u003eLi et al. YOLO-ACN: Focusing on small target and occluded object detection. \u003cem\u003eIEEE Access\u003c/em\u003e, \u003cstrong\u003e8\u003c/strong\u003e, 227288\u0026ndash;227303 (2020). \u003c/li\u003e\n\u003cli\u003eCai et al. Cascade R-CNN: Delving into high quality object detection. In Proc. IEEE Conf. Comp. Vis. and Pat. Rec., USA, 6154\u0026ndash;6162, IEEE Press (2018).\u003c/li\u003e\n\u003cli\u003eWang et al. A high precision fast smoky vehicle detection method based on improved YOLOv5 Network. In Proc. IEEE Int. Conf. Art. Int. Ind. Des., Guangzhou, 255\u0026ndash;259, IEEE Press (2021). \u003c/li\u003e\n\u003cli\u003eMiao et al. A nighttime vehicle detection method based on YOLOv3. In Proc. Chinese Auto. Cong., Shanghai, 6617\u0026ndash;6621 (2020). \u003c/li\u003e\n\u003cli\u003eSarda et al. Object detection for autonomous driving using YOLO algorithm. In Proc. 3\u003csup\u003erd\u003c/sup\u003e Int. Conf. Int. Comm. Tech. Vir. Mob. Net., Tirunelveli, 1370\u0026ndash;1374 (2021).\u003c/li\u003e\n\u003cli\u003eZhao et al. Vehicle detection based on improved YOLOv3 algorithm. In Proc. Int. Conf. Int. Trans., Big Data Smart City, Vientiane, 76\u0026ndash;79 (2020). \u003c/li\u003e\n\u003cli\u003eĆorović et al. The real time detection of traffic participants using YOLO algorithm. In Proc. 26\u003csup\u003eth\u003c/sup\u003e Telecomm. Forum, Belgrade, Serbia, 1\u0026ndash;4 (2018). \u003c/li\u003e\n\u003cli\u003eLou et al. Vehicle detection of traffic flow video using deep learning. In Proc. IEEE 8\u003csup\u003eth\u003c/sup\u003e Data Driv. Cont. Learn. Sys. Conf., Dali, China, 1012\u0026ndash;1017, IEEE Press (2019). \u003c/li\u003e\n\u003cli\u003eMachiraju et al. Object detection and tracking for community surveillance using transfer learning. In Proc. 6\u003csup\u003eth\u003c/sup\u003e Int. Conf. Inv. Comp. Tech., Coimbatore, India, 1035\u0026ndash;1042 (2021). \u003c/li\u003e\n\u003cli\u003eSnegireva et al. Vehicle classification application on video using YOLOv5 architecture. In Proc. Int. Russian Auto. Conf., Sochi, Russia, 1008\u0026ndash;1013 (2021). \u003c/li\u003e\n\u003cli\u003eJana et al. YOLO based detection and classification of objects in video records. In Proc. 3\u003csup\u003erd\u003c/sup\u003e IEEE Int. Conf. Rec. Trends Elec., Info. Comm. Tech., India, 2448\u0026ndash;2452, IEEE Press (2018). \u003c/li\u003e\n\u003cli\u003eHu et al. A video streaming vehicle detection algorithm based on YOLOv4. In Proc. IEEE 5\u003csup\u003eth\u003c/sup\u003e Adv. Info. Tech., Elec. Auto. Con. Conf., Chongqing, China, 2081\u0026ndash;2086, IEEE Press (2021). \u003c/li\u003e\n\u003cli\u003eKasper et al. Detecting heavy goods vehicles in rest areas in winter conditions using YOLOv5. \u003cem\u003eAlgorithms\u003c/em\u003e, \u003cstrong\u003e14\u003c/strong\u003e, 114 (2021). \u003c/li\u003e\n\u003cli\u003eDe Carvalho et al. Bounding box free instance segmentation using semi-supervised iterative learning for vehicle detection. \u003cem\u003eIEEE J. Selec. Top. App. Earth Obsv. Remo. Sens.\u003c/em\u003e, \u003cstrong\u003e15\u003c/strong\u003e, 3403\u0026ndash;3420 (2022). \u003c/li\u003e\n\u003cli\u003eTayara et al. Vehicle detection and counting in high-resolution aerial images using convolutional regression neural network. \u003cem\u003eIEEE Access\u003c/em\u003e, \u003cstrong\u003e6\u003c/strong\u003e, 2220\u0026ndash;2230 (2018). \u003c/li\u003e\n\u003cli\u003eZhao et al. Improved vision based vehicle detection and classification by optimized YOLOv4. \u003cem\u003eIEEE Access\u003c/em\u003e, \u003cstrong\u003e10\u003c/strong\u003e, 8590\u0026ndash;8603 (2022). \u003c/li\u003e\n\u003cli\u003eHassaballah et al. Vehicle detection and tracking in adverse weather using deep learning framework. \u003cem\u003eIEEE Tran. Int. Trans. Sys.\u003c/em\u003e, \u003cstrong\u003e22\u003c/strong\u003e, 4230\u0026ndash;4242 (2022). \u003c/li\u003e\n\u003cli\u003eMahmood et al. Towards automatic license plate detection. \u003cem\u003eSensors\u003c/em\u003e, \u003cstrong\u003e22\u003c/strong\u003e, 1245 (2022). \u003c/li\u003e\n\u003cli\u003eWu et al. Real time vehicle and distance detection based on improved YOLOv5 network. In Proc. 3\u003csup\u003erd \u003c/sup\u003eWorld Symp. Art. Int., Guangzhou, China. 24\u0026ndash;28 (2021).\u003c/li\u003e\n\u003cli\u003eLin et al. Microsoft COCO: Common objects in context. In Proc. 13\u003csup\u003eth\u003c/sup\u003e Euro. Conf. Comp. Vis., Zurich, Switzerland, 740\u0026ndash;755 (2014).\u003c/li\u003e\n\u003cli\u003eHe et al. Bounding box regression with uncertainty for accurate object detection. In Proc. IEEE Conf. Comp. Vis. Pat. Reco., Long Beach, CA, USA. 2888\u0026ndash;2897, IEEE Press (2019).\u003c/li\u003e\n\u003cli\u003eLiu et al. Dynamic vehicle detection with sparse point clouds based on PE-CPD. \u003cem\u003eIEEE Tran. Int. Trans. Sys.\u003c/em\u003e, \u003cstrong\u003e20\u003c/strong\u003e, 1964\u0026ndash;1977 (2019). \u003c/li\u003e\n\u003cli\u003eZhao et al. M2Det: A single shot object detector based on multi-level feature pyramid network. AAAI Conf. Art. Int., Montr\u0026eacute;al, Canada, \u003cstrong\u003e33\u003c/strong\u003e, 9259\u0026ndash;9266 (2019).\u003c/li\u003e\n\u003cli\u003eLi et al. Scale aware trident networks for object detection. \u003cem\u003earXiv\u003c/em\u003e:1901.01892 (2019).\u003c/li\u003e\n\u003cli\u003eZhang et al. Mask SSD: An effective single-stage approach to object instance segmentation. \u003cem\u003eIEEE Tran. Image Proc.\u003c/em\u003e, \u003cstrong\u003e29\u003c/strong\u003e, 2078\u0026ndash;2093 (2020). \u003c/li\u003e\n\u003cli\u003eMin et al. A new approach to track multiple vehicles with the combination of robust detection and two classifiers. \u003cem\u003eIEEE Tran. Int. Trans. Sys.\u003c/em\u003e, \u003cstrong\u003e19\u003c/strong\u003e, 174\u0026ndash;186 (2018). \u003c/li\u003e\n\u003cli\u003eLaw et al. CornerNet: Detecting objects as paired key points. Euro. Conf. Comp. Vis., Glasgow, UK, 734\u0026ndash;750 (2018). \u003c/li\u003e\n\u003cli\u003eHe et al. Mask R-CNN. In Proc. IEEE Int. Conf. Comp. Vis., Venice, Italy, 2961\u0026ndash;2969, IEEE Press (2017).\u003c/li\u003e\n\u003cli\u003eShi et al. Orientation aware vehicle detection in aerial images via an anchor free object detection approach. \u003cem\u003eIEEE Tran. Geo. Remo. Sens.\u003c/em\u003e, \u003cstrong\u003e59\u003c/strong\u003e, 5221\u0026ndash;5233 (2021). \u003c/li\u003e\n\u003cli\u003eHu et al. SINet: A scale insensitive convolutional neural network for fast vehicle detection. \u003cem\u003eIEEE Tran. Int. Trans. Sys.\u003c/em\u003e, \u003cstrong\u003e20\u003c/strong\u003e, 1010\u0026ndash;1019 (2019). \u003c/li\u003e\n\u003cli\u003eFarid et al. A fast and accurate real time vehicle detection method using deep learning for unconstrained environments. \u003cem\u003eApp. Sci.\u003c/em\u003e, \u003cstrong\u003e13\u003c/strong\u003e, 3059 (2023).\u003c/li\u003e\n\u003cli\u003eChaudhuri, A. Benchmarked datasets for vehicle segmentation. \u003cem\u003eTech. Rep.\u003c/em\u003e, Samsung R \u0026amp; D Inst. New Delhi, India (2018).\u003c/li\u003e\n\u003cli\u003eRonneberger et al. U-Net: convolutional net- works for biomedical image segmentation. Lect. Notes in Comp. Sci., \u003cstrong\u003e9351\u003c/strong\u003e, 234\u0026ndash;241, Springer Verlag (2015).\u003c/li\u003e\n\u003cli\u003eCOCO Datasets: https://cocodataset.org/#home\u003c/li\u003e\n\u003cli\u003eDAWN Datasets: https://paperswithcode.com/dataset/dawn\u003c/li\u003e\n\u003c/ol\u003e"},{"header":"Tables","content":"\u003cp\u003eTables 1 to 4 are available in the Supplementary Files section.\u003c/p\u003e "}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Smart traffic management, vehicle segmentation, traffic density estimation, vehicle tracking, Faster R-CNN","lastPublishedDoi":"10.21203/rs.3.rs-3854436/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-3854436/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eWith constant growth of civilization and modernization of cities all across the world since past few centuries smart traffic management of vehicles is one of the most sorted after problem by research community. Smart traffic management basically involves segmentation of vehicles, estimation of traffic density and tracking of vehicles. The vehicle segmentation from videos helps realization of niche applications such as monitoring of speed and estimation of traffic. When occlusions, background with clutters and traffic with density variations, this problem becomes more intractable in nature. Keeping this motivation in this research work, we investigate Faster R-CNN based deep learning method towards segmentation of vehicles. This problem is addressed in four steps viz minimization with adaptive background model, Faster R-CNN based subnet operation, Faster R-CNN initial refinement and result optimization with extended topological active nets. The computational framework uses adaptive background modeling. It also addresses shadow and illumination issues. Higher segmentation accuracy is achieved through topological active net deformable models. The topological and extended topological active nets help to achieve stated deformations. Mesh deformation is achieved with minimization of energy. The segmentation accuracy is improved with modified version of extended topological active net. The experimental results demonstrate superiority of this framework.\u003c/p\u003e","manuscriptTitle":"Smart Traffic Management of Vehicles using Faster R-CNN based Deep Learning Method","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-01-15 05:07:13","doi":"10.21203/rs.3.rs-3854436/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2024-03-05T04:02:27+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-02-19T17:40:41+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2024-02-06T14:19:49+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-02-06T10:31:12+00:00","index":"","fulltext":""},{"type":"submitted","content":"Scientific Reports","date":"2024-01-11T18:36:34+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"4b8d6793-9695-4a20-be5a-e9b9aceecad9","owner":[],"postedDate":"January 15th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[{"id":28129960,"name":"Physical sciences/Engineering"},{"id":28129961,"name":"Physical sciences/Mathematics and computing"}],"tags":[],"updatedAt":"2024-05-09T16:54:18+00:00","versionOfRecord":{"articleIdentity":"rs-3854436","link":"https://doi.org/10.1038/s41598-024-60596-4","journal":{"identity":"scientific-reports","isVorOnly":false,"title":"Scientific Reports"},"publishedOn":"2024-05-06 16:54:17","publishedOnDateReadable":"May 6th, 2024"},"versionCreatedAt":"2024-01-15 05:07:13","video":"","vorDoi":"10.1038/s41598-024-60596-4","vorDoiUrl":"https://doi.org/10.1038/s41598-024-60596-4","workflowStages":[]},"version":"v1","identity":"rs-3854436","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-3854436","identity":"rs-3854436","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}