Real time big-data processing and monitoring tool for object detection using Gaussian Mixture model with improved noise reduction

Real time big-data processing and monitoring tool for object detection using Gaussian Mixture model with improved noise reduction | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Real time big-data processing and monitoring tool for object detection using Gaussian Mixture model with improved noise reduction Pranjal Pandey, Rakesh K. Prasad, Dilip K. Singh This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4104551/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Increasing number of road accidents draws major concern over on-road safety. Road accidents are leading causes of deaths across the world. Numerous approaches have been made to improve the monitoring and control system to avoid road accidents like use of sensors, radar based onboard electronic devices etc, Machine learning based monitoring is one approach towards object detection to avoid accidents due to human error and system failure. In the present work we have reported about development of an improved monitoring tool to detect moving objects in a video frame in real time and with improved noise reduction using Gaussian Mixture Model. We have used machine learning model with the help of MATLAB’s computer vision tool which do not require an external digital data storage space. This results in development of a tool which is capable of detecting the moving object in a live video footage which is capable of differentiating between foreground and background in continuously varying daylight conditions and effective for noise reduction. The developed technique can be applied to selectively record the eventful frames from live video reducing the requirement of human intervention for monitoring and large data storage for record. In addition to surveillance for road transport, the tool can be used for the purpose of monitoring at hospitals, airports, military and defence purpose. Big Data processing Computer vision Gaussian Mixture Model Machine learning Noise reduction Real Time object detection Road transport Surveillance Figures Figure 1 Figure 2 Figure 3 Figure 4 Introduction In recent years, on road safety has been a matter of great concern world-wide with increasing number of accidents annually. Approximately 1.3 million people die each year as a result of road traffic crashes. More than half of all road traffic deaths are among vulnerable road users: pedestrians, cyclists, and motorcyclists. In fact, road traffic injuries are the leading cause of death for children and young adults aged between 5–29 years. 93% of the world's fatalities on the roads occur in low- and middle-income countries in-spite of the fact that these countries have only ~ 60% of the world's vehicles [ 1 ]. This is due to lack of proper monitoring system for persons having tendency of rash driving, lane jumping and alcohol addiction. The problem can be addressed to greater extent by analysing and monitoring few parameters for each individual from network of data from different domains. The prime reasons for road accidents are (a) driving under effect of alcohol, (b) tendency to overpass the speed limit (c) frequent tendency of lane jumping (d) driving in the wrong direction and (e) sleep disorders of the drivers. In addition to it, there have been other concerns in terms of vehicle theft for the purpose of crime, driving by untrained / unskilled persons, accidents due to failure of control systems, sudden health failure of drivers and poor monitoring during nights. The safety concerns have been scaling up at much higher pace due to everyday increase of number of vehicles on the road, and change in the societal patterns of crowded streets and markets with narrow passages for vehicles [ 1 ]. The existing electronic monitoring devices used for this purpose requires to deal with number of challenges in terms of accessing vast data base (related to vehicle, driver, local traffic pattern and recognition of individuals), limited network communication speed for transmission and control, monitoring at road sides, analysing with ideal expected patterns and method of generating and transmitting information to concern people and authorities. A vast volume of dynamic data being generated, analysed and controlled to achieve safer road transportation system requires techniques to compress using algorithms transmission over networks, electronic devices and techniques which can on-board process data and generate data of lower volume in real time. This can be achieved by selective intelligent sensing and pickup of events to be reported from long stretch of vast data generated by each individual/ tool. Gaurav Goel et al. in 2016 carried out analysis of data of 04 years (2007 to 2010) for a stretch of 50 Km of distance. Through the data analysis they arrived to the conclusion that the maximum accidents fall in the category of non-injury type (49%). Serious injury type accidents are found to be more than fatal accidents. Findings show that head on/rear end collisions, caused mainly due to over-speeding /driver's fault account for 46% of the accidents. It is seen that trucks/canter/buses are found involved in maximum number of accidents (42%). One of the most interesting findings was that the day time accidents are found to be more than night time accidents. There have been number of geographically localized studies of traffic patterns and accidents, like study of traffic accident characteristics of Kolkata, showing majority of accident involves buses [ 2 ] and of Nashik, Maharashtra by Baviskar et al. [ 3 ]. In the international scenario, there have been ongoing strong research activities related to data collection and analysis through big data, intelligent systems and artificial intelligent tools. In 2015, Eyad Abdullah and Ahmed Emam developed traffic accidents analyser using Big Data. They used massive real traffic datasets from New York's traffic collisions dataset and used it as a source of data to developed application. The developed application consists of several functions and web services to analyse and visualize the major traffic accident information. The developed application stores the massive traffic data on Hadoop with a parallel computing framework for diverse and mine based Map-Reduce technique, which then uses Web services interface to support developed mining application. Seong-hun Park and group through their work in 2016 showed possibility of highway traffic accident prediction using VDS big data analysis [ 4 ]. In another work published in 2017, Hamzah Al Najada and Imad Mahgoub from USA designed a real-time Big Data system that receives online streamed data from vehicles on the road in addition to real-time average speed data from vehicles detectors on the road side to (1) Provide accurate Estimated Time of Arrival (ETA) using a Linear Regression (LR) model (2) Predict accidents and congestions before they happen using Naive Bayes (NB) and Distributed Random Forest (DRF) classifiers (3) Update ETA if an accident or a congestion takes place by predicting accurate clearance time. They optimized the efficiency, the speed, and the accuracy of the designed model by securely selecting the most relevant and significant set of features required for the analysis [ 5 ]. In 2018 M. Mazhar Rathore from Korea has worked to integrate all smart systems to exploit IOT and Big data analytics to achieve smart digital city [ 6 ]. Recently in 2019 Li Zhu et al. from China have shown several case studies of big data analytics applications in intelligent transportation systems, including road traffic accidents analysis, road traffic flow prediction, public transportation service plan, personal travel route plan, rail transportation management and control, and assets maintenance are introduced. Finally, this paper discusses some open challenges of using big data analytics in ITS (intelligent traffic systems) [ 7 ]. In another work, Yi Qu et al. (2019) studied the causes of road accidents using big real-time accidents data obtained from Florida Department of Transportation (FDOT) - District 4. The ultimate goal was to prevent or decrease traffic accidents and congestions. Their approach was based on dividing the roadway into segments, based on the infrastructure availability and the secondary accident factors. They designed a real-time Big data system which receives online streamed data from vehicles on the road in addition to real-time average speed data from vehicles detectors on the road side to (a) Provide accurate Estimated Time of Arrival (ETA) using a Linear Regression (LR) model (b) Predict accidents and congestions before they happen using Naive Bayes (NB) and Distributed Random Forest (DRF) classifiers (c) Update ETA if an accident or a congestion takes place by predicting accurate clearance time. To make the system fast, accurate, and reliable, they implemented Lambda Architecture (LA) in their framework because of its speed, scalability, and fault tolerance. Furthermore, they optimized the efficiency, the speed, and the accuracy of the designed model by securely selecting the most relevant and significant set of features required for the analysis [ 8 ] Camilo Gutierrez-Osorio recently (2020) reviewed the work for the prediction of road accidents through machine learning algorithms and advanced techniques for analysing information like conventional neural networks, long-term memory networks and other deep learning architectures. They proposed a classification depending upon origin and characteristics such as open data, measurement technologies, onboard equipment and social media data [ 9 ]. For object detection, numerous methods have been developed and various object detection techniques are being developed, it started back in 1991 by Panos G. [ 10 ] where he described about the auto scope system in his approach he did image segmentation using the parameters like every point in a scene is represented by the cartesian coordinate including the time that is S(X,Y,Z,t) and extracts various features like intensity, energy and reflectivity of the scene. It uses the statistical temporal analysis for object detection and form artificial database. The problem was it requires lots of initial parameters and it is costly in storage point of view. In 1998, Chris Stauffer et al. used the gaussian mixture model for the object detection. They used the intuition that the object remains at any pixel for limited number of frames. That means the pixels are background most of the time. Gaussian are classified by the ratio of number of evidences (ω) and standard deviation (σ) and it was proposed that if the ratio (ω /σ) is small, then it will be considered as moving object. The problem was the computational speed at that time and with various applications the system requires different parameters related to cameras and lighting [ 11 ]. Recently, Isha Jain et al. used fuzzy logic for image processing. They used the frame without the object as a reference and the vehicle classification are done using binary operation. Based on the objects size they remove the undesired object in the frame. They used the edge detection techniques and used blob analysis for each detected object to determine the actual dimension of the object by reshaping into near polygon shape. The applicability of this model is limited in terms of requirement of aerial camera footage and reference frame with no object in it. Also, if any changes occurred into the background, then it will be considered as a detected object as the reference frame remains the same [ 12 ]. Zhengxia Zou et al. proposed a model where road side cameras are used to create a 3-D model for camera-based perception where multiple cameras are used to create a 3-d model and using the Kalman filter to detect the object by defining the path of the moving object. The drawback of this model is in terms of requirement of multiple cameras making model cost inefficient [ 13 ]. We have improved the monitoring system using MATLAB’s computer vision tool and successfully detected the moving object in the video frame in real time with approximately zero time lag. We have been able to distinguish the moving object in the frame from the background. The developed technique does not require any external data storage along with reduction in the time for object detection. The Gaussian Mixture model (GMM) based machine learning tool was used to train our system to detect moving objects in the live video footage. The developed tool enables reduction of digital memory space required for storage, processing and continuous monitoring of vehicles. Methodology For real time object detection, we used MATLAB’s computer vision tool and to begin with the real time object detection we need to have a live video footage, in this particular case a webcam is used. A frame is extracted from the live footage. Figure 1 shows flow-chart of the execution of work for detection of object in the live video feed. We used MATLAB’s computer vision tool and to begin with the real time object detection we need to have a live video footage, in this particular case a webcam is used. A frame is extracted from the live footage. For every 150 frames 40 frames are being used to train the system to detect moving object in a loop. In the first loop, after every 150 frames system gets trained with new conditions. Then the extracted frame is passed through the trained system which forms a noise ridden gaussian surface. Noise is then filtered out and blob analysis is done on it and then the object gets detected on the extracted frame by creating a red box around the detected moving objects. The whole process is then repeated in a loop for continuous frames to detect the object in real time with approximately no lag. The 'Gaussian Mixture Model (GMM)' based machine learning was used to train the system to recognise the moving object. GMM creates a gaussian surface of the extracted frames. Gaussian Mixture model is a Deep Learning model where it produces multiple gaussian peaks of the change in pixel values in continuous frames to differentiate the moving object in the frame [ 14 ]. In this particular case, gaussian peaks denotes foreground and backgrounds. Gaussian functions are often used to represent the probability density function of a normally distributed random variable. We take the expected value as µ and variance as \({\sigma }^{2}\) . Hence, the normalised gaussian distribution takes the form, $$g\left(x\right)= \frac{1}{\sigma \sqrt{2\pi }}{e}^{\left(\frac{-{(x-\mu )}^{2}}{2{\sigma }^{2}}\right)}$$ If we consider only the intensity (only brightness values now) distribution at each pixel over time we get a histogram which appears as combination of Gaussian peaks. Figure 2 (a) shows the extracted frame from the live video and Fig. 2 (b) shows the image histogram of the image extracted. The histogram depicts pixel value change throughout the frame and leads to the fluctuations in gaussian peaks. Here in this case, fluctuations are because of static scene which is change in illumination of road and the buildings, second due to wind which is creating the noise and also due to occasional moving objects which passes through the pixel rarely. Here we proceeded with the intuition that objects are background most of the time i.e., number of evidences where static scene is present as dominant component [ 11 ]. 1-Dimensional Gaussian is expressed as $$\omega \eta \left(x,\mu ,\sigma \right)=\omega \frac{1}{\sigma \sqrt{2\pi }}{e}^{-\left(\frac{{\left(x-\mu \right)}^{2}}{2{\sigma }^{2}}\right)}$$ Where η denotes the probability density function, \(\omega\) is the number of evidences, \(\mu\) is the Gaussian’s mean values and \(\sigma\) is the standard deviation. If P(x) denotes the Probability Distribution, where x denotes the pixel intensity. Assuming P(x) is made of k different gaussians denoted by \({\omega }_{k}{\eta }_{k}\left(\eta ,{\mu }_{k},{\sigma }_{k}\right)\) , $$P\left(x\right)=\sum _{k=1}^{k}{\omega }_{k}{\eta }_{k}\left(x,{\mu }_{k},{\sigma }_{k}\right)$$ The sum of all evidences must be unity i.e, \(\sum _{k=1}^{k}{\omega }_{k}=1\) , i.e., the GMM Distribution is the weighted sum of k Gaussians. If P(x) be a probability distribution of a D-Dimensional random variable \(x\in {R}^{D}\) . And also considering all the red, green and blue components of the pixel, $$X={\left[r,g,b\right]}^{T}$$ GMM of P(X): Sum of k-Dimensional Gaussians. To classify individual Gaussian as foreground or background we proceeded with an intuition that Pixels are background most of the time. That means gaussian with large supporting evidence ( \(\omega\) ) and small \(\sigma\) . Then we can classify that if the ratio \(\omega /\sigma\) is large then it will be considered as background and if the ratio \(\omega /\sigma\) is small then it will be considered as foreground. [ 11 ]. In our experiment we have taken minimum background ratio \(\omega /\sigma\) equals to 0.7. For every 150 frames in the live footage last 40 live video frames are being used to train our system to produce gaussian surface. It makes the learning process faster and detects the moving objects precisely in real time. Hence, chances of false detection due to continuously changing conditions like change in illumination of roads and buildings, environmental conditions like wind, rain and fog where background have some movements alongside the object which is in the frame that has been reduced. Once we got the gaussian surface we found that it had lot of noises hence those noises need to be filtered out. For noise filtration, we used MATLAB’s ‘strel’ function to declare the morphological structure. We took a \(3\times 3\) square pixel structure to filter out the noise as it will be declared as outlier as it does not form a square matrix of pixel of order 3, then applied the morphological structure on the gaussian surface of the frame. Hence, we got the noise filtered gaussian image of the extracted frame. In order to display the detected moving object in the frame we made a box around the object. The problem was every moving object does not have any peculiar shape, also there was lot of blobs which can’t be considered as boxes hence we did blob analysis and through that we can remove the blobs in frame. We choose the minimum blob area of 300 pixels as any blob having area less than 300 pixels will be ignored. This is how we have created a bounding box around the detected moving object in the extracted frame after noise reduction. After blob analysis, we inserted a rectangular shaped box around the bounding box on the initially extracted video frame and inserted the count of the bounding box in the frame at the top left corner of the frame. Finally, we have a moving object detected in the extracted frame from a live video footage with the number of the moving object count on the top left corner of the frame. Finally, we needed a video player before we initiate a loop in order to display the live video footage. The whole process is then repeated in a loop where the continuous frames are being extracted and the process to train the system, object detection and noise filtering is repeated up till the maximum frame count is reached in this case maximum frame count chosen was 5000. Results and Discussion Figure 3 (a) & (b) shows two different frames extracted from a live video where the system has detected moving object from that video, with the number of detected objects count in the frames at the top left corner. One can easily differentiate between the two figures where a car entered the video which is shown in the Fig. 3 (b) that is the second frame and that has been detected by the system. Not only the detected object is moving (i.e., the car) but also due to windy conditions where branches and leaves of the trees in the frame is also moving which goes undetected by the system which shows the main feature of this system, i.e system is capable of differentiating between the foreground and the background in real time. Figure 3 (a) shows no moving object is present on that particular frame with the number of objects count on that particular frame shows 0 on the top left corner. The branches and the leaves of the trees are still moving on that frame due to wind but get undetected by the system and shows the count as zero for the Fig. 3 (a). Similarly, in Fig. 3 (b) the external conditions remain the same and wind is still causing the movement of the branches and leaves which has been considered as noise by system apart from the detected moving object. Figure 4 (a) & (b) shows the detection of the moving object from the frames, by creating a gaussian surface of the extracted frames and processing the changes on the pixel values on those frames. Each frame in the video is first analysed by the system and with every passing frame the system trains itself and detects the changes in pixel values for each frame. The machine learning model to train the system is “The Gaussian Mixture Model” and hence with the help of that model the system creates the gaussian surface. With the intuition that pixels are background most of the time the system interprets the changes in the pixel values, if any moving object passes by any small portion of the frame the pixel values changes and the system detects the moving object there. Figure 4 (a) shows a frame where the moving objects in the video sequence are humans and rest other objects in the video sequence are stationary. Figure 4 (b) shows the gaussian surface of the above extracted frame shown in Fig. 4 (a) with detected objects. All the moving objects in the frame are declared as the foreground and rest of the frame as background. A Gaussian surface is formed by keeping the foreground and by scrapping the background from the frame as one can see in Fig. 4 (b). A closer look to the Fig. 4 (b) indicates presence of strong noise which depicts a small and frequent movements in the background and hence we need additional step for noise filtration. Figure 4 (c) shows the Gaussian Surface after filtering out the noise of the frame shown in Fig. 4 (a), by using the ‘Sterl’ function of the MATLAB. ‘Sterl’ function creates a matrix of structuring element, where arbitrary point or pixels were omitted and cluster of pixels were retained as noise filtered detected object [ 15 – 17 ]. One can clearly figure out from Fig. 4 (b) that the detected objects have associated shadows with them, which moves with the objects. Hence it’s a limitation of the system that it cannot filter out the shadows as detected object. Object along with its shadow appears as one complete moving object. Figure 4 (d) shows the identified moving object in the extracted frame from live video footage. A rectangular red coloured box is placed to mark identified moving object around the extracted frame shown in Fig. 3 (a). The number of moving object is displayed on the top left corner of the frame. The Real time object detection takes almost no time for this whole process hence it can be said that detention works with nearly zero time lag. Conclusion In this system, real time object detection is achieved by the help of machine learning and computer vision tools using MATLAB. Gaussian mixture model creates a Gaussian Surfaces (a black and white image) of the frames by keeping the moving object in the frame and eliminating the background. Real time object detection helps to discard irrelevant footage from the video file, which reduces the need of external storages where requirement of data storage is huge like traffic video footage, surveillance and security systems. Real time object detection takes less processing time which reduces the lag in detection which can be beneficial in the area of traffic monitoring and security surveillance system. In the first loop, first 40 frames are used to train the system for every 150 frames that means after every 150 frames system gets trained with new conditions which improves the accuracy and reduces the processing time. Declarations Author Contribution Dilip K. Singh had the original idea. Pranjal Pandey wrote codes and implanted idea. Pranjal Pandey, Rakesh K. Prasad and Dilip K. Singh discussed the results and wrote the article. ACKNOWLEDGMENT Dilip K. Singh thanks UGC-DAE CSR Indore (CRS/2021-22/01/358), NM-ICPS ISI Kolkata (ISI/TIH/2022/48) and DST, Government of India (CRG/2021/002179; CRG/2021/003705) for financial support. References World Health Organization. "Global Status Report on Road Safety 2018 (Report No. ISBN 978-92-4-156568-4)." Geneva: World Health Organization (2018). Chakraborty, Sandip, and Sudip K. Roy. "Traffic accident characteristics of Kolkata." Transport and Communications Bulletin for Asia and the Pacific 74 (2005): 75-86. Baviskar, S. B. "Road accidents in Nashik municipal corporation area: a case study." Indian Journal of Transport Management 23, (9) (1999): 543-555. Park, Seong-hun, Sung-min Kim, and Young-guk Ha.“Highway traffic accident prediction using VDS big data analysis,” The Journal of Supercomputing, 72, (2016):2815-2831. Al Najada, Hamzah, and Imad Mahgoub."Anticipation and alert system of congestion and accidents in VANET using Big Data analysis for Intelligent Transportation Systems." In 2016 IEEE Symposium Series on Computational Intelligence (SSCI) ,IEEE,(2016):1-8. Rathore, M. Mazhar, Anand Paul, Won-Hwa Hong, HyunCheol Seo, Imtiaz Awan, and Sharjil Saeed . “Exploiting IoT and big data analytics: Defining smart digital city using real-time urban data,” Sustainable cities and society . 40, (2018): 600-610. Zhu, Li, Fei Richard Yu, Yige Wang, Bin Ning, and Tao Tang.“Big data analytics in intelligent transportation systems: A survey,” IEEE Transactions on Intelligent Transportation Systems, 20, (1) (2018): 383-398. Qu, Yi, Zhengkui Lin, Honglei Li, and Xiaonan Zhang.“Feature recognition of urban road traffic accidents based on GA-XGBoost in the context of big data,” IEEE Access, 7 (2019): 170106-170115. Gutierrez-Osorio, Camilo, and César Pedraza. “Modern data sources and techniques for analysis and forecast of road accidents: A review,” Journal of traffic and transportation engineering (English edition), 7, (4) (2020): 432-446. Michalopoulos, Panos G.“Vehicle detection video through image processing: the autoscope system,” IEEE Transactions on vehicular technology, 40, (1) (1991): 21-29. Stauffer, Chris, and W. Eric L. Grimson."Adaptive background mixture models for real-time tracking." In Proceedings. 1999 IEEE computer society conference on computer vision and pattern recognition (Cat. No PR00149) , IEEE, (2) (1999):246-252. Jain, Isha, and Babita Rani.“Vehicle detection using image processing and fuzzy logic,” International Journal of Computer Science & Communication, 1, (2) (2010): 255-257. Zou, Zhengxia, Rusheng Zhang, Shengyin Shen, Gaurav Pandey, Punarjay Chakravarty, Armin Parchami, and Henry X. Liu."Real-time full-stack traffic scene perception for autonomous driving with roadside cameras." In 2022 International Conference on Robotics and Automation (ICRA) , IEEE, (2022): 890-896. Wei, Zhiqiang, Xiaopeng Ji, and Peng Wang.“Real-time moving object detection for video monitoring systems,” Journal of Systems Engineering and Electronics, 17, (4) (2006): 731-736. Van Den Boomgaard, Rein, and Richard Van Balen “Methods for fast morphological image transforms using bitmapped binary images,” CVGIP: Graphical Models and Image Processing, 54, (3), (1992): 252-258. Adams, Rolf. “Radial decomposition of disks and spheres,” CVGIP: Graphical models and image processing, 55, (5), (1993): 325-332. Jones, Ronald, and Pierre Soille.“Periodic lines: Definition, cascades, and application to granulometries,” Pattern Recognition Letters, 17, (10) (1996): 1057-1063. Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4104551","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":279994336,"identity":"05e0f886-1a75-4209-a88c-d9e69b63bd86","order_by":0,"name":"Pranjal Pandey","email":"","orcid":"","institution":"Birla Institute of Technology Mesra","correspondingAuthor":false,"prefix":"","firstName":"Pranjal","middleName":"","lastName":"Pandey","suffix":""},{"id":279994338,"identity":"b412c9d8-5e3b-4948-b246-46c867c048d0","order_by":1,"name":"Rakesh K. Prasad","email":"","orcid":"","institution":"Birla Institute of Technology Mesra","correspondingAuthor":false,"prefix":"","firstName":"Rakesh","middleName":"K.","lastName":"Prasad","suffix":""},{"id":279994340,"identity":"ee3bec63-9e4d-4a04-9c53-5926b334e503","order_by":2,"name":"Dilip K. Singh","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABAElEQVRIiWNgGAWjYDCCw8wNBxgYJMAICCTkQOSBB3i1MKJosTAGa0nAp+UAYwOEAdFSkQjm4tPCd5yx8XDBH4s8+ejeg58LKiTS54cdfgi0xU5OtwG7Fkmgww7PbJMoNrxzLll6xhmJ3I230wyAWpKNzQ5g12IA0sLbIJG4cUaOgTRvG1DL7ASQlgOJ2/Bp4fkD1mL8G6gl3XB2+gcitLBJJM6XyDED2ZIgL52D3xaYXxI3ALVY85yRMNwgnVNwIMEAt1/4zh8+/LngT13ifKDDbvNU1MnLz07f/OFDhZ0cLi0gwAx2IUwBhGGAWzlci3wDlAdnjIJRMApGwSiAAgCYCGVLu3w6AgAAAABJRU5ErkJggg==","orcid":"","institution":"Birla Institute of Technology Mesra","correspondingAuthor":true,"prefix":"","firstName":"Dilip","middleName":"K.","lastName":"Singh","suffix":""}],"badges":[],"createdAt":"2024-03-15 03:44:14","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4104551/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4104551/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":53022849,"identity":"02c5df44-3fb1-4e51-b5d6-b24c31087405","added_by":"auto","created_at":"2024-03-19 17:05:05","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":83252,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cem\u003eShows the process flow for live video object detection in real time.\u003c/em\u003e\u003c/p\u003e","description":"","filename":"ProcessflowBW1.png","url":"https://assets-eu.researchsquare.com/files/rs-4104551/v1/d58a778c8d0177d97dcd224a.png"},{"id":53022850,"identity":"5366daf7-6839-462d-9a3d-4e1cf5a06e5e","added_by":"auto","created_at":"2024-03-19 17:05:05","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":3049582,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cem\u003e(a) Shows the extracted frame from the live video and (b) shows the histogram of the extracted frame from the video file depicting the Gaussian peaks.\u003c/em\u003e\u003c/p\u003e","description":"","filename":"figure2.png","url":"https://assets-eu.researchsquare.com/files/rs-4104551/v1/74461aca522f552e2b279acf.png"},{"id":53023377,"identity":"21addc3f-e52c-4204-9edd-48536f6da6dd","added_by":"auto","created_at":"2024-03-19 17:13:05","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":4050804,"visible":true,"origin":"","legend":"\u003cp\u003e(a)\u003cstrong\u003e \u003c/strong\u003e\u003cem\u003eshows frame with no moving object detected and Figure (b) shows frame with car as detected object.\u003c/em\u003e\u003c/p\u003e","description":"","filename":"figure3.png","url":"https://assets-eu.researchsquare.com/files/rs-4104551/v1/a7d2ca873f5c4911ca81db12.png"},{"id":53022852,"identity":"f763f6ec-a3ff-49e6-8784-c8ba7411c543","added_by":"auto","created_at":"2024-03-19 17:05:05","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":4559460,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cem\u003e(a) shows a frame from a traffic camera, (b) shows the gaussian surface of figure (a), (c) Shows detected objects in the frame after removing the noise and (d) Shows the detected objects in the first frame with the count of number of objects present on the top left corner of the frame.\u003c/em\u003e\u003c/p\u003e","description":"","filename":"figure4.png","url":"https://assets-eu.researchsquare.com/files/rs-4104551/v1/d7c068b0383a588ba538372a.png"},{"id":54861926,"identity":"1ecfe924-86b3-422f-a569-9ff7e43468d6","added_by":"auto","created_at":"2024-04-17 19:59:44","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":5127823,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4104551/v1/28cf24a2-b076-4f4b-9ea8-59c423f23df8.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Real time big-data processing and monitoring tool for object detection using Gaussian Mixture model with improved noise reduction","fulltext":[{"header":"Introduction","content":"\u003cp\u003eIn recent years, on road safety has been a matter of great concern world-wide with increasing number of accidents annually. Approximately 1.3\u0026nbsp;million people die each year as a result of road traffic crashes. More than half of all road traffic deaths are among vulnerable road users: pedestrians, cyclists, and motorcyclists. In fact, road traffic injuries are the leading cause of death for children and young adults aged between 5\u0026ndash;29 years. 93% of the world's fatalities on the roads occur in low- and middle-income countries in-spite of the fact that these countries have only\u0026thinsp;~\u0026thinsp;60% of the world's vehicles [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e]. This is due to lack of proper monitoring system for persons having tendency of rash driving, lane jumping and alcohol addiction. The problem can be addressed to greater extent by analysing and monitoring few parameters for each individual from network of data from different domains.\u003c/p\u003e \u003cp\u003eThe prime reasons for road accidents are (a) driving under effect of alcohol, (b) tendency to overpass the speed limit (c) frequent tendency of lane jumping (d) driving in the wrong direction and (e) sleep disorders of the drivers. In addition to it, there have been other concerns in terms of vehicle theft for the purpose of crime, driving by untrained / unskilled persons, accidents due to failure of control systems, sudden health failure of drivers and poor monitoring during nights. The safety concerns have been scaling up at much higher pace due to everyday increase of number of vehicles on the road, and change in the societal patterns of crowded streets and markets with narrow passages for vehicles [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eThe existing electronic monitoring devices used for this purpose requires to deal with number of challenges in terms of accessing vast data base (related to vehicle, driver, local traffic pattern and recognition of individuals), limited network communication speed for transmission and control, monitoring at road sides, analysing with ideal expected patterns and method of generating and transmitting information to concern people and authorities. A vast volume of dynamic data being generated, analysed and controlled to achieve safer road transportation system requires techniques to compress using algorithms transmission over networks, electronic devices and techniques which can on-board process data and generate data of lower volume in real time. This can be achieved by selective intelligent sensing and pickup of events to be reported from long stretch of vast data generated by each individual/ tool.\u003c/p\u003e \u003cp\u003eGaurav Goel et al. in 2016 carried out analysis of data of 04 years (2007 to 2010) for a stretch of 50 Km of distance. Through the data analysis they arrived to the conclusion that the maximum accidents fall in the category of non-injury type (49%). Serious injury type accidents are found to be more than fatal accidents. Findings show that head on/rear end collisions, caused mainly due to over-speeding /driver's fault account for 46% of the accidents. It is seen that trucks/canter/buses are found involved in maximum number of accidents (42%). One of the most interesting findings was that the day time accidents are found to be more than night time accidents. There have been number of geographically localized studies of traffic patterns and accidents, like study of traffic accident characteristics of Kolkata, showing majority of accident involves buses [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e] and of Nashik, Maharashtra by Baviskar et al. [\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e]. In the international scenario, there have been ongoing strong research activities related to data collection and analysis through big data, intelligent systems and artificial intelligent tools. In 2015, Eyad Abdullah and Ahmed Emam developed traffic accidents analyser using Big Data. They used massive real traffic datasets from New York's traffic collisions dataset and used it as a source of data to developed application. The developed application consists of several functions and web services to analyse and visualize the major traffic accident information. The developed application stores the massive traffic data on Hadoop with a parallel computing framework for diverse and mine based Map-Reduce technique, which then uses Web services interface to support developed mining application. Seong-hun Park and group through their work in 2016 showed possibility of highway traffic accident prediction using VDS big data analysis [\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e]. In another work published in 2017, Hamzah Al Najada and Imad Mahgoub from USA designed a real-time Big Data system that receives online streamed data from vehicles on the road in addition to real-time average speed data from vehicles detectors on the road side to (1) Provide accurate Estimated Time of Arrival (ETA) using a Linear Regression (LR) model (2) Predict accidents and congestions before they happen using Naive Bayes (NB) and Distributed Random Forest (DRF) classifiers (3) Update ETA if an accident or a congestion takes place by predicting accurate clearance time. They optimized the efficiency, the speed, and the accuracy of the designed model by securely selecting the most relevant and significant set of features required for the analysis [\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e]. In 2018 M. Mazhar Rathore from Korea has worked to integrate all smart systems to exploit IOT and Big data analytics to achieve smart digital city [\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e]. Recently in 2019 Li Zhu et al. from China have shown several case studies of big data analytics applications in intelligent transportation systems, including road traffic accidents analysis, road traffic flow prediction, public transportation service plan, personal travel route plan, rail transportation management and control, and assets maintenance are introduced. Finally, this paper discusses some open challenges of using big data analytics in ITS (intelligent traffic systems) [\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e]. In another work, Yi Qu et al. (2019) studied the causes of road accidents using big real-time accidents data obtained from Florida Department of Transportation (FDOT) - District 4. The ultimate goal was to prevent or decrease traffic accidents and congestions. Their approach was based on dividing the roadway into segments, based on the infrastructure availability and the secondary accident factors. They designed a real-time Big data system which receives online streamed data from vehicles on the road in addition to real-time average speed data from vehicles detectors on the road side to (a) Provide accurate Estimated Time of Arrival (ETA) using a Linear Regression (LR) model (b) Predict accidents and congestions before they happen using Naive Bayes (NB) and Distributed Random Forest (DRF) classifiers (c) Update ETA if an accident or a congestion takes place by predicting accurate clearance time. To make the system fast, accurate, and reliable, they implemented Lambda Architecture (LA) in their framework because of its speed, scalability, and fault tolerance. Furthermore, they optimized the efficiency, the speed, and the accuracy of the designed model by securely selecting the most relevant and significant set of features required for the analysis [\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e] Camilo Gutierrez-Osorio recently (2020) reviewed the work for the prediction of road accidents through machine learning algorithms and advanced techniques for analysing information like conventional neural networks, long-term memory networks and other deep learning architectures. They proposed a classification depending upon origin and characteristics such as open data, measurement technologies, onboard equipment and social media data [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eFor object detection, numerous methods have been developed and various object detection techniques are being developed, it started back in 1991 by Panos G. [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e] where he described about the auto scope system in his approach he did image segmentation using the parameters like every point in a scene is represented by the cartesian coordinate including the time that is S(X,Y,Z,t) and extracts various features like intensity, energy and reflectivity of the scene. It uses the statistical temporal analysis for object detection and form artificial database. The problem was it requires lots of initial parameters and it is costly in storage point of view. In 1998, Chris Stauffer et al. used the gaussian mixture model for the object detection. They used the intuition that the object remains at any pixel for limited number of frames. That means the pixels are background most of the time. Gaussian are classified by the ratio of number of evidences (ω) and standard deviation (σ) and it was proposed that if the ratio (ω /σ) is small, then it will be considered as moving object. The problem was the computational speed at that time and with various applications the system requires different parameters related to cameras and lighting [\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eRecently, Isha Jain et al. used fuzzy logic for image processing. They used the frame without the object as a reference and the vehicle classification are done using binary operation. Based on the objects size they remove the undesired object in the frame. They used the edge detection techniques and used blob analysis for each detected object to determine the actual dimension of the object by reshaping into near polygon shape. The applicability of this model is limited in terms of requirement of aerial camera footage and reference frame with no object in it. Also, if any changes occurred into the background, then it will be considered as a detected object as the reference frame remains the same [\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e]. Zhengxia Zou et al. proposed a model where road side cameras are used to create a 3-D model for camera-based perception where multiple cameras are used to create a 3-d model and using the Kalman filter to detect the object by defining the path of the moving object. The drawback of this model is in terms of requirement of multiple cameras making model cost inefficient [\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eWe have improved the monitoring system using MATLAB\u0026rsquo;s computer vision tool and successfully detected the moving object in the video frame in real time with approximately zero time lag. We have been able to distinguish the moving object in the frame from the background. The developed technique does not require any external data storage along with reduction in the time for object detection. The Gaussian Mixture model (GMM) based machine learning tool was used to train our system to detect moving objects in the live video footage. The developed tool enables reduction of digital memory space required for storage, processing and continuous monitoring of vehicles.\u003c/p\u003e"},{"header":"Methodology","content":"\u003cp\u003eFor real time object detection, we used MATLAB\u0026rsquo;s computer vision tool and to begin with the real time object detection we need to have a live video footage, in this particular case a webcam is used. A frame is extracted from the live footage. Figure \u003cspan\u003e1\u003c/span\u003e shows flow-chart of the execution of work for detection of object in the live video feed. We used MATLAB\u0026rsquo;s computer vision tool and to begin with the real time object detection we need to have a live video footage, in this particular case a webcam is used. A frame is extracted from the live footage. For every 150 frames 40 frames are being used to train the system to detect moving object in a loop. In the first loop, after every 150 frames system gets trained with new conditions. Then the extracted frame is passed through the trained system which forms a noise ridden gaussian surface. Noise is then filtered out and blob analysis is done on it and then the object gets detected on the extracted frame by creating a red box around the detected moving objects. The whole process is then repeated in a loop for continuous frames to detect the object in real time with approximately no lag.\u003c/p\u003e\n\u003cp\u003eThe \u0026apos;Gaussian Mixture Model (GMM)\u0026apos; based machine learning was used to train the system to recognise the moving object. GMM creates a gaussian surface of the extracted frames. Gaussian Mixture model is a Deep Learning model where it produces multiple gaussian peaks of the change in pixel values in continuous frames to differentiate the moving object in the frame [\u003cspan\u003e14\u003c/span\u003e]. In this particular case, gaussian peaks denotes foreground and backgrounds. Gaussian functions are often used to represent the probability density function of a normally distributed random variable. We take the expected value as \u0026micro; and variance as \u003cspan\u003e\u003cspan\u003e\\({\\sigma }^{2}\\)\u003c/span\u003e\u003c/span\u003e. Hence, the normalised gaussian distribution takes the form,\u003c/p\u003e\n\u003cdiv id=\"Equa\"\u003e\n \u003cdiv id=\"FileID_Equa\" name=\"EquationSource\"\u003e$$g\\left(x\\right)= \\frac{1}{\\sigma \\sqrt{2\\pi }}{e}^{\\left(\\frac{-{(x-\\mu )}^{2}}{2{\\sigma }^{2}}\\right)}$$\u003c/div\u003e\n\u003c/div\u003e\n\u003cp\u003eIf we consider only the intensity (only brightness values now) distribution at each pixel over time we get a histogram which appears as combination of Gaussian peaks.\u003c/p\u003e\n\u003cp\u003eFigure \u003cspan\u003e2\u003c/span\u003e(a) shows the extracted frame from the live video and Fig. \u003cspan\u003e2\u003c/span\u003e(b) shows the image histogram of the image extracted. The histogram depicts pixel value change throughout the frame and leads to the fluctuations in gaussian peaks. Here in this case, fluctuations are because of static scene which is change in illumination of road and the buildings, second due to wind which is creating the noise and also due to occasional moving objects which passes through the pixel rarely. Here we proceeded with the intuition that objects are background most of the time i.e., number of evidences where static scene is present as dominant component [\u003cspan\u003e11\u003c/span\u003e].\u003c/p\u003e\n\u003cp\u003e1-Dimensional Gaussian is expressed as\u003c/p\u003e\n\u003cdiv id=\"Equb\"\u003e\n \u003cdiv id=\"FileID_Equb\" name=\"EquationSource\"\u003e$$\\omega \\eta \\left(x,\\mu ,\\sigma \\right)=\\omega \\frac{1}{\\sigma \\sqrt{2\\pi }}{e}^{-\\left(\\frac{{\\left(x-\\mu \\right)}^{2}}{2{\\sigma }^{2}}\\right)}$$\u003c/div\u003e\n\u003c/div\u003e\n\u003cp\u003eWhere \u0026eta; denotes the probability density function, \u003cspan\u003e\u003cspan\u003e\\(\\omega\\)\u003c/span\u003e\u003c/span\u003e is the number of evidences, \u003cspan\u003e\u003cspan\u003e\\(\\mu\\)\u003c/span\u003e\u003c/span\u003e is the Gaussian\u0026rsquo;s mean values and \u003cspan\u003e\u003cspan\u003e\\(\\sigma\\)\u003c/span\u003e\u003c/span\u003e is the standard deviation. If P(x) denotes the Probability Distribution, where x denotes the pixel intensity. Assuming P(x) is made of k different gaussians denoted by \u003cspan\u003e\u003cspan\u003e\\({\\omega }_{k}{\\eta }_{k}\\left(\\eta ,{\\mu }_{k},{\\sigma }_{k}\\right)\\)\u003c/span\u003e\u003c/span\u003e,\u003c/p\u003e\n\u003cdiv id=\"Equc\"\u003e\n \u003cdiv id=\"FileID_Equc\" name=\"EquationSource\"\u003e$$P\\left(x\\right)=\\sum _{k=1}^{k}{\\omega }_{k}{\\eta }_{k}\\left(x,{\\mu }_{k},{\\sigma }_{k}\\right)$$\u003c/div\u003e\n\u003c/div\u003e\n\u003cp\u003eThe sum of all evidences must be unity i.e, \u003cspan\u003e\u003cspan\u003e\\(\\sum _{k=1}^{k}{\\omega }_{k}=1\\)\u003c/span\u003e\u003c/span\u003e, i.e., the GMM Distribution is the weighted sum of k Gaussians.\u003c/p\u003e\n\u003cp\u003eIf P(x) be a probability distribution of a D-Dimensional random variable \u003cspan\u003e\u003cspan\u003e\\(x\\in {R}^{D}\\)\u003c/span\u003e\u003c/span\u003e. And also considering all the red, green and blue components of the pixel,\u003c/p\u003e\n\u003cdiv id=\"Equd\"\u003e\n \u003cdiv id=\"FileID_Equd\" name=\"EquationSource\"\u003e$$X={\\left[r,g,b\\right]}^{T}$$\u003c/div\u003e\n\u003c/div\u003e\n\u003cp\u003eGMM of P(X): Sum of k-Dimensional Gaussians.\u003c/p\u003e\n\u003cdiv id=\"Eque\"\u003e\n \u003cdiv id=\"FileID_Eque\" name=\"EquationSource\"\u003e\u003cimg src=\"https://myfiles.space/user_files/122228_c8a1650c59388082/122228_custom_files/img1710867372.png\"\u003e\u003cbr\u003e\u003c/div\u003e\n\u003c/div\u003e\n\u003cp\u003eTo classify individual Gaussian as foreground or background we proceeded with an intuition that Pixels are background most of the time. That means gaussian with large supporting evidence (\u003cspan\u003e\u003cspan\u003e\\(\\omega\\)\u003c/span\u003e\u003c/span\u003e) and small \u003cspan\u003e\u003cspan\u003e\\(\\sigma\\)\u003c/span\u003e\u003c/span\u003e.\u003c/p\u003e\n\u003cp\u003eThen we can classify that if the ratio \u003cspan\u003e\u003cspan\u003e\\(\\omega /\\sigma\\)\u003c/span\u003e\u003c/span\u003e is large then it will be considered as background and if the ratio \u003cspan\u003e\u003cspan\u003e\\(\\omega /\\sigma\\)\u003c/span\u003e\u003c/span\u003e is small then it will be considered as foreground. [\u003cspan\u003e11\u003c/span\u003e]. In our experiment we have taken minimum background ratio \u003cspan\u003e\u003cspan\u003e\\(\\omega /\\sigma\\)\u003c/span\u003e\u003c/span\u003e equals to 0.7. For every 150 frames in the live footage last 40 live video frames are being used to train our system to produce gaussian surface. It makes the learning process faster and detects the moving objects precisely in real time. Hence, chances of false detection due to continuously changing conditions like change in illumination of roads and buildings, environmental conditions like wind, rain and fog where background have some movements alongside the object which is in the frame that has been reduced.\u003c/p\u003e\n\u003cp\u003eOnce we got the gaussian surface we found that it had lot of noises hence those noises need to be filtered out. For noise filtration, we used MATLAB\u0026rsquo;s \u0026lsquo;strel\u0026rsquo; function to declare the morphological structure. We took a \u003cspan\u003e\u003cspan\u003e\\(3\\times 3\\)\u003c/span\u003e\u003c/span\u003e square pixel structure to filter out the noise as it will be declared as outlier as it does not form a square matrix of pixel of order 3, then applied the morphological structure on the gaussian surface of the frame. Hence, we got the noise filtered gaussian image of the extracted frame.\u003c/p\u003e\n\u003cp\u003eIn order to display the detected moving object in the frame we made a box around the object. The problem was every moving object does not have any peculiar shape, also there was lot of blobs which can\u0026rsquo;t be considered as boxes hence we did blob analysis and through that we can remove the blobs in frame. We choose the minimum blob area of 300 pixels as any blob having area less than 300 pixels will be ignored. This is how we have created a bounding box around the detected moving object in the extracted frame after noise reduction. After blob analysis, we inserted a rectangular shaped box around the bounding box on the initially extracted video frame and inserted the count of the bounding box in the frame at the top left corner of the frame. Finally, we have a moving object detected in the extracted frame from a live video footage with the number of the moving object count on the top left corner of the frame.\u003c/p\u003e\n\u003cp\u003eFinally, we needed a video player before we initiate a loop in order to display the live video footage. The whole process is then repeated in a loop where the continuous frames are being extracted and the process to train the system, object detection and noise filtering is repeated up till the maximum frame count is reached in this case maximum frame count chosen was 5000.\u003c/p\u003e"},{"header":"Results and Discussion","content":"\u003cp\u003eFigure \u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e(a) \u0026amp; (b) shows two different frames extracted from a live video where the system has detected moving object from that video, with the number of detected objects count in the frames at the top left corner. One can easily differentiate between the two figures where a car entered the video which is shown in the Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e(b) that is the second frame and that has been detected by the system. Not only the detected object is moving (i.e., the car) but also due to windy conditions where branches and leaves of the trees in the frame is also moving which goes undetected by the system which shows the main feature of this system, i.e system is capable of differentiating between the foreground and the background in real time.\u003c/p\u003e \u003cp\u003eFigure \u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e(a) shows no moving object is present on that particular frame with the number of objects count on that particular frame shows 0 on the top left corner. The branches and the leaves of the trees are still moving on that frame due to wind but get undetected by the system and shows the count as zero for the Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e(a). Similarly, in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e(b) the external conditions remain the same and wind is still causing the movement of the branches and leaves which has been considered as noise by system apart from the detected moving object.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eFigure \u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e(a) \u0026amp; (b) shows the detection of the moving object from the frames, by creating a gaussian surface of the extracted frames and processing the changes on the pixel values on those frames. Each frame in the video is first analysed by the system and with every passing frame the system trains itself and detects the changes in pixel values for each frame. The machine learning model to train the system is \u0026ldquo;The Gaussian Mixture Model\u0026rdquo; and hence with the help of that model the system creates the gaussian surface. With the intuition that pixels are background most of the time the system interprets the changes in the pixel values, if any moving object passes by any small portion of the frame the pixel values changes and the system detects the moving object there. Figure\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e(a) shows a frame where the moving objects in the video sequence are humans and rest other objects in the video sequence are stationary. Figure\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e(b) shows the gaussian surface of the above extracted frame shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e(a) with detected objects.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eAll the moving objects in the frame are declared as the foreground and rest of the frame as background. A Gaussian surface is formed by keeping the foreground and by scrapping the background from the frame as one can see in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e(b). A closer look to the Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e(b) indicates presence of strong noise which depicts a small and frequent movements in the background and hence we need additional step for noise filtration.\u003c/p\u003e \u003cp\u003eFigure \u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e(c) shows the Gaussian Surface after filtering out the noise of the frame shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e(a), by using the \u0026lsquo;Sterl\u0026rsquo; function of the MATLAB. \u0026lsquo;Sterl\u0026rsquo; function creates a matrix of structuring element, where arbitrary point or pixels were omitted and cluster of pixels were retained as noise filtered detected object [\u003cspan additionalcitationids=\"CR16\" citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e]. One can clearly figure out from Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e(b) that the detected objects have associated shadows with them, which moves with the objects. Hence it\u0026rsquo;s a limitation of the system that it cannot filter out the shadows as detected object. Object along with its shadow appears as one complete moving object.\u003c/p\u003e \u003cp\u003eFigure \u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e(d) shows the identified moving object in the extracted frame from live video footage. A rectangular red coloured box is placed to mark identified moving object around the extracted frame shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e(a). The number of moving object is displayed on the top left corner of the frame. The Real time object detection takes almost no time for this whole process hence it can be said that detention works with nearly zero time lag.\u003c/p\u003e"},{"header":"Conclusion","content":"\u003cp\u003eIn this system, real time object detection is achieved by the help of machine learning and computer vision tools using MATLAB. Gaussian mixture model creates a Gaussian Surfaces (a black and white image) of the frames by keeping the moving object in the frame and eliminating the background. Real time object detection helps to discard irrelevant footage from the video file, which reduces the need of external storages where requirement of data storage is huge like traffic video footage, surveillance and security systems. Real time object detection takes less processing time which reduces the lag in detection which can be beneficial in the area of traffic monitoring and security surveillance system. In the first loop, first 40 frames are used to train the system for every 150 frames that means after every 150 frames system gets trained with new conditions which improves the accuracy and reduces the processing time.\u003c/p\u003e"},{"header":"Declarations","content":"\u003ch2\u003eAuthor Contribution\u003c/h2\u003e\u003cp\u003eDilip K. Singh had the original idea. Pranjal Pandey wrote codes and implanted idea. Pranjal Pandey, Rakesh K. Prasad and Dilip K. Singh discussed the results and wrote the article.\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eACKNOWLEDGMENT\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eDilip K. Singh thanks UGC-DAE CSR Indore (CRS/2021-22/01/358), NM-ICPS ISI Kolkata (ISI/TIH/2022/48) and DST, Government of India (CRG/2021/002179; CRG/2021/003705) for financial support.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eWorld Health Organization. \u0026quot;Global Status Report on Road Safety 2018 (Report No. ISBN 978-92-4-156568-4).\u0026quot; \u003cem\u003eGeneva: World Health Organization\u003c/em\u003e (2018).\u003c/li\u003e\n\u003cli\u003eChakraborty, Sandip, and Sudip K. Roy. \u0026quot;Traffic accident characteristics of Kolkata.\u0026quot; \u003cem\u003eTransport and Communications Bulletin for Asia and the Pacific\u003c/em\u003e 74 (2005): 75-86.\u003c/li\u003e\n\u003cli\u003eBaviskar, S. B. \u0026quot;Road accidents in Nashik municipal corporation area: a case study.\u0026quot; \u003cem\u003eIndian Journal of Transport Management\u003c/em\u003e 23, (9) (1999): 543-555.\u003c/li\u003e\n\u003cli\u003ePark, Seong-hun, Sung-min Kim, and Young-guk Ha.\u0026ldquo;Highway traffic accident prediction using VDS big data analysis,\u0026rdquo; \u003cem\u003eThe Journal of Supercomputing,\u003c/em\u003e72, (2016):2815-2831.\u003c/li\u003e\n\u003cli\u003eAl Najada, Hamzah, and Imad Mahgoub.\u0026quot;Anticipation and alert system of congestion and accidents in VANET using Big Data analysis for Intelligent Transportation Systems.\u0026quot; In \u003cem\u003e2016 IEEE Symposium Series on Computational Intelligence (SSCI)\u003c/em\u003e,IEEE,(2016):1-8.\u003c/li\u003e\n\u003cli\u003eRathore, M. Mazhar, Anand Paul, Won-Hwa Hong, HyunCheol Seo, Imtiaz Awan, and Sharjil Saeed\u003cem\u003e.\u003c/em\u003e\u0026ldquo;Exploiting IoT and big data analytics: Defining smart digital city using real-time urban data,\u0026rdquo; \u003cem\u003eSustainable cities and society\u003c/em\u003e. 40, (2018): 600-610.\u003c/li\u003e\n\u003cli\u003eZhu, Li, Fei Richard Yu, Yige Wang, Bin Ning, and Tao Tang.\u0026ldquo;Big data analytics in intelligent transportation systems: A survey,\u0026rdquo; \u003cem\u003eIEEE Transactions on Intelligent Transportation Systems,\u003c/em\u003e 20, (1) (2018): 383-398.\u003c/li\u003e\n\u003cli\u003eQu, Yi, Zhengkui Lin, Honglei Li, and Xiaonan Zhang.\u0026ldquo;Feature recognition of urban road traffic accidents based on GA-XGBoost in the context of big data,\u0026rdquo; \u003cem\u003eIEEE Access,\u003c/em\u003e 7 (2019): 170106-170115.\u003c/li\u003e\n\u003cli\u003eGutierrez-Osorio, Camilo, and C\u0026eacute;sar Pedraza. \u0026ldquo;Modern data sources and techniques for analysis and forecast of road accidents: A review,\u0026rdquo; \u003cem\u003eJournal of traffic and transportation engineering (English edition),\u003c/em\u003e 7, (4) (2020): 432-446.\u003c/li\u003e\n\u003cli\u003eMichalopoulos, Panos G.\u0026ldquo;Vehicle detection video through image processing: the autoscope system,\u0026rdquo; \u003cem\u003eIEEE Transactions on vehicular technology,\u003c/em\u003e 40, (1) (1991): 21-29.\u003c/li\u003e\n\u003cli\u003eStauffer, Chris, and W. Eric L. Grimson.\u0026quot;Adaptive background mixture models for real-time tracking.\u0026quot; In \u003cem\u003eProceedings. 1999 IEEE computer society conference on computer vision and pattern recognition (Cat. No PR00149)\u003c/em\u003e, IEEE, (2) (1999):246-252.\u003c/li\u003e\n\u003cli\u003eJain, Isha, and Babita Rani.\u0026ldquo;Vehicle detection using image processing and fuzzy logic,\u0026rdquo; \u003cem\u003eInternational Journal of Computer Science \u0026amp; Communication,\u003c/em\u003e 1, (2) (2010): 255-257.\u003c/li\u003e\n\u003cli\u003eZou, Zhengxia, Rusheng Zhang, Shengyin Shen, Gaurav Pandey, Punarjay Chakravarty, Armin Parchami, and Henry X. Liu.\u0026quot;Real-time full-stack traffic scene perception for autonomous driving with roadside cameras.\u0026quot; In \u003cem\u003e2022 International Conference on Robotics and Automation (ICRA)\u003c/em\u003e, IEEE, (2022): 890-896.\u003c/li\u003e\n\u003cli\u003eWei, Zhiqiang, Xiaopeng Ji, and Peng Wang.\u0026ldquo;Real-time moving object detection for video monitoring systems,\u0026rdquo; \u003cem\u003eJournal of Systems Engineering and Electronics,\u003c/em\u003e 17, (4) (2006): 731-736.\u003c/li\u003e\n\u003cli\u003eVan Den Boomgaard, Rein, and Richard Van Balen \u0026ldquo;Methods for fast morphological image transforms using bitmapped binary images,\u0026rdquo; \u003cem\u003eCVGIP: Graphical Models and Image Processing,\u003c/em\u003e 54, (3), (1992): 252-258.\u003c/li\u003e\n\u003cli\u003eAdams, Rolf. \u0026ldquo;Radial decomposition of disks and spheres,\u0026rdquo; \u003cem\u003eCVGIP: Graphical models and image processing,\u003c/em\u003e 55, (5), (1993): 325-332.\u003c/li\u003e\n\u003cli\u003eJones, Ronald, and Pierre Soille.\u0026ldquo;Periodic lines: Definition, cascades, and application to granulometries,\u0026rdquo; \u003cem\u003ePattern Recognition Letters,\u003c/em\u003e 17, (10) (1996): 1057-1063.\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Big Data processing, Computer vision, Gaussian Mixture Model, Machine learning, Noise reduction, Real Time, object detection, Road transport, Surveillance","lastPublishedDoi":"10.21203/rs.3.rs-4104551/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4104551/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eIncreasing number of road accidents draws major concern over on-road safety. Road accidents are leading causes of deaths across the world. Numerous approaches have been made to improve the monitoring and control system to avoid road accidents like use of sensors, radar based onboard electronic devices etc, Machine learning based monitoring is one approach towards object detection to avoid accidents due to human error and system failure. In the present work we have reported about development of an improved monitoring tool to detect moving objects in a video frame in real time and with improved noise reduction using Gaussian Mixture Model. We have used machine learning model with the help of MATLAB\u0026rsquo;s computer vision tool which do not require an external digital data storage space. This results in development of a tool which is capable of detecting the moving object in a live video footage which is capable of differentiating between foreground and background in continuously varying daylight conditions and effective for noise reduction. The developed technique can be applied to selectively record the eventful frames from live video reducing the requirement of human intervention for monitoring and large data storage for record. In addition to surveillance for road transport, the tool can be used for the purpose of monitoring at hospitals, airports, military and defence purpose.\u003c/p\u003e","manuscriptTitle":"Real time big-data processing and monitoring tool for object detection using Gaussian Mixture model with improved noise reduction","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-03-19 17:05:00","doi":"10.21203/rs.3.rs-4104551/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"c262e707-1c3c-41fb-a6d9-bd2bc4c7b13d","owner":[],"postedDate":"March 19th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2024-04-17T19:59:23+00:00","versionOfRecord":[],"versionCreatedAt":"2024-03-19 17:05:00","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4104551","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4104551","identity":"rs-4104551","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}