A Parallel FP-Growth Mining Algorithm with Load Balancing Constraints for Traffic Crash Data

A Parallel FP-Growth Mining Algorithm with Load Balancing Constraints for Traffic Crash Data | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article A Parallel FP-Growth Mining Algorithm with Load Balancing Constraints for Traffic Crash Data Yang Yang, Na Tian, Yunpeng Wang, Zhenzhou Yuan This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-1311180/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Traffic safety is an important part of the roadway in sustainable development. Freeway traffic crashes typically cause serious casualties and property losses, being a serious threat to public safety. Figuring out the potential correlation between various risk factors and revealing their coupling mechanism are of effective ways to explore and identity freeway crash causes. However, the existing association rule mining algorithms still have some limitations in both efficiency and accuracy. Based on this consideration, using the freeway traffic crash data obtained from WDOT (Washington Department of Transportation), this research constructed a multi-dimensional multi-level system for traffic crash analysis. Considering the load balancing, the FP-Growth algorithm was optimized parallelly based on Hadoop platform, to achieve an efficient and accurate association rules mining calculation for massive amounts of traffic crash data; Then, according to the results of the coupling mechanism among the crash precursors, the causes of freeway traffic crashes were identified and revealed. The results show that the parallel FP-growth algorithm with load balancing constraints has a better operating speed than both conventional FP-growth algorithm and parallel FP-growth algorithm towards processing big data. This improved algorithm makes full use of Hadoop cluster resources and is more suitable for traffic crash large data sets mining while retaining the original advantages of conventional association rule mining algorithm. In addition, the mining association rules model with the improvement of multi-dimensional interaction proposed in this research can catch the occurrence mechanism of freeway traffic crash with serious consequences (lower support degree probably) accurately and efficiently. Traffic crash Data mining FP-Growth algorithm Load balancing Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 1. Introduction Traffic crash is a serious threat to public health safety and have become the eighth leading cause of human death. The number of deaths caused by traffic crashes continues to increase, with about 1.35 million people dying every year, and the main cause of death for people aged 5 to 29 is traffic crash [ 1 ]. Especially on freeway, the vehicles usually run at a fast speed, once the traffic crash occurs, it is easier to cause serious casualties. The coupling mechanism analysis of the factors influencing traffic crash on the freeway, has a certain significance to improve the traffic safety level for freeway. With the advent of the era of traffic big data, the technology of traffic crash data collection is becoming mature, which provides a strong data support for the analysis of traffic crash coupling mechanism. It was common to use historical traffic crash data to figure out the coupling mechanism of crashes. The quantity and quality of historical data have a crucial influence on the analysis results. Hamed et al. [ 2 ] proposed a random parameter Logit model with heterogeneity to study the interaction between traffic crashes and vehicle holding quantity. By studying the causes of freeway traffic crashes, Yu [ 3 ] focused on analyzing the causes of secondary rear-end collisions of freeway traffic crashes. Kwayu [ 4 ] et al., tested the universality and interaction among the influencing factors of fatal traffic crashes via structured modeling and network topology analysis, and explored the causes of fatal traffic crashes with the data of fatal traffic crashes in Michigan from 2009 to 2018. Sun et al. [ 5 ] established a Bayesian spatial model to reveal the potential spatial correlation between segments. Association rule mining is a fuzzy recognition and decision method based on data mining technology. It is an efficient data mining method, which can be used to identify the combination of key factors related to crash consequences. Mining association rules for freeway traffic crash data can facilitate figuring out the potential association relationship among the influencing factors. Yang [ 6 – 8 ] proposed a multi-dimensional interaction improved Apriori association rules mining algorithm considering directional constraints and index weights: WOMID-Apriori, based on subjective and objective combined weighting model the interval analytic hierarchy Process (AHP) and grey relational degree, the weight of data fields was calculated and optimized, then, the algorithm was applied to address the cause analysis of freeway crashes. Singh [ 9 ] et al. improved the performance of Apriori algorithm and proposed two improved Apriori algorithms based on MapReduce: VFPC (Elapsed Time-based Dynamic pass-counting) and ETDPC (Elapsed Time-based Dynamic pass-counting). Wang [ 10 ] et al. discussed the algorithm for sorting variables in the data set and proposed two new technologies, MG and SMGR. These two technologies were developed for public safety using classified traffic crash data, which were more intuitive than existing technologies. Yu et al. [ 11 ] proposed a new DSTGCN mixed spatio-temporal graph convolutional network to predict traffic crash. Jiang [ 12 ] et al. proposed a framework based on association rule mining to identify the factors related to the severity of motorcycle injuries. In order to objectively determine the threshold value of parameters, they developed a parameter optimization method. Huang et al. [ 13 ] proposed a parallel FP-growth algorithm based on cloud computing. Samerei [ 14 ] et al. worked on the bus crash data set of Victoria State, Australia from 2006 to 2019, divided the traffic crash data into clustering, extracted the factors affecting bus traffic crash fatality based on association rule mining algorithm. Montella [ 15 ] et al. used classification trees and association rule mining algorithm to analyze 78,611 crash data involving two-wheeled vehicles in Spain, aiming to extract knowledge from a large amount of data and identify understandable association rules. Zheng et al. [ 16 ] applied the gradient lifting algorithm to analyze the relationship between collision severity and influencing factors. Bechini et al. [ 17 ] proposed a distributed association rule mining algorithm based on MapReduce programming model, confirmed the scalability and performance improvement of this method on small computer clusters. Global scholars have made some achievements in the research of traffic crash analysis methods and technologies so far. However, there are also some deficiencies in these studies: (1) Most of the algorithms currently applied in the field of traffic crash data mining are ordinary serial algorithms, a huge number of candidate item sets may be generated in the mining process. The support of the algorithm during the calculation requires a large amount of memory, and it is unable to efficiently mine the massive traffic crash data. In addition, the ordinary parallel association rule mining algorithm generally has the problem of load imbalance, which may lead a low computing efficiency. Most of the existing research improved the algorithm by adding constraint conditions to achieve the efficient extraction of mining results, but lack of improvement on the performance of mining algorithm itself. (2) Existing association rules mining basically focused on the high degree level of support and confidence of association rules, while only few studies focused on the combination rules with low support and high confidence those were prone to lead to serious consequences such as casualties, which may lead that the potential important traffic crash precursors missed. However, the characteristics and mechanism of serious casualty crashes and ordinary crashes may be different, so the conventional safety improvement measurements for ordinary crashes may not work for the serious crashes. In view of the above defects, based on Hadoop platform, this research proposed a parallel FP-growth algorithm considering load balancing for freeway traffic crash data, which is used to mine multi-dimensional and multi-level massive crash data. Then the association rules with high support and confidence from mining results were extracted and interpreted. Finally, based on the mining results, the suggestions to improve freeway traffic safety were put forward. 2. Data Process 2.1 Data source description The data source of this research is the UW DriveNet Intelligent Transportation Big Data platform of the University of Washington in the United States. Through the connection between the platform and WDOT, a total of 345,545 pieces of freeway traffic crash data in Washington state can be obtained, including 384 attributes, specifically includes: Driver's gender, age, driver's degree of soberness, crash location, vehicle type, type of airbag, state of airbag ejection, weather, road type, traffic control, crash time and season, etc. 2.2 The establishment of multi-dimensional and multi-layer data system The multi-dimensional and multi-layer traffic crash coupling mechanism analysis system is the basic framework of association rule mining. In this research, crash influencing factors are taken as the first layer of the system, including six dimensions "driver, vehicle, road, environment, time, crash". In the six dimensions of the first layer of source data, specific influencing factors with analytical value are screened out respectively. As the second layer, there are 29 dimensions in total. The attributes of the influencing factors in the second layer are further refined, and the specific indexes and values of the attributes are clarified, forming the third layer of the system. The specific indexes and values of the crash dimension are shown in Table 1 . Table 1 Specific indexes and values of influencing factors of collision dimension Factors Values Codes of Valus Injury Type No Injury AH1 Possible Injury AH2 Serious Injury AH3 Death AH4 Ejection Status Not Ejected W1 Unknown if Ejected W2 Partially Ejected W3 Totally Ejected W4 Collision Severity PDO AE1 Injury AE2 Fatality AE3 Towed Indicator No D1 Yes D2 Collision Report Type Suburban A1 Urban A2 Interstate A3 Ramp A4 Based on the dimensional and hierarchical analysis results of the above crash attributes, the construction of multi-dimensional OLAP database and data mining in THE SQL Server Analysis Service (SSAS) of the database management system should be addressed. The specific steps are as follows: (1) Import data preprocessed with Excel into Access. (2) The fact table (freeway traffic crash) and 6-dimension tables (driver, vehicle, road, environment, time, crash) are constructed, the primary keys of each table are demarcated, and the relationship between the out keys of fact table and the primary keys of each dimension table are set up. Finally, the Access database is promoted to SQL Server database. (3) Create the research project "Freeway Traffic Crash" in SSAS, import the data source. (4) Set the ID attribute of the fact table "freeway traffic crash" as the metric value and "Count" as the aggregation mode, built the multi-dimensional database and dimension considering the data analysis requirements, and deploy the project to the server. 3. Design And Implementation Of The Algorithm The FP-Growth algorithm constructs the data set as FP-tree and stores it in memory, then mines frequent item sets by recursively calls condition of FP-tree [ 18 ]. However, when mining massive data with low support degree, the FP-tree generated by FP-growth will occupy a large amount of memory, causing problems such as memory overflow and long operation time [ 19 – 20 ]. This research tries to solve the above problems through the optimization design of the parallel FP-growth algorithm, proposes a load balancing optimization scheme based on the parallel FP-growth algorithm, and builds an algorithm experimental environment under condition of the Hadoop framework. Through experiments, the efficiency of traditional FP-growth algorithm, parallel FP-growth algorithm and parallel FP-growth algorithm considering load balancing with different data volume and different minimum support threshold are compared and analyzed respectively. 3.1 Principle of parallel FP-Growth algorithm The performance bottleneck of FP-growth algorithm when processing massive data is solved using horizontal partitioning [ 19 ]. The horizontal partitioning method is to divide the database into various sub-database nodes, make frequency statistics for each sub-node, build local FP-tree, obtain the conditional pattern base of each data frequent item set, gather various conditional pattern bases to the corresponding nodes, and then build conditional FP-tree for each node, operate the local FP-growth mining. The local FP-tree diagram considering the influencing factors of traffic crashes is shown in Fig. 1 . 3.2 Process design of parallel FP-growth algorithm The operation process of the parallel FP-growth algorithm can be summarized into the following five steps, and the algorithm process is introduced in combination with this research case. The specific process design is shown in Fig. 2 : Step 1, Shard: The freeway historical crash database of Washington state is horizontally divided into six consecutive sub-databases from the dimensions of driver, vehicle, road, environment, time and crash, which are stored on several different computers respectively. Each sub-database obtained after segmentation is defined as slice. Step 2, Parallel counting: Read each slice obtained in Step 1 and use "Mapreduce. job" to count frequent "1- itemset FList". Where the "Mapper" function starts to input with the key pairs of , after processing, transaction T i is output as key pairs , where AJ is the J th data item of T i ; All slices are processed by Mapper function, and the key pairs with the same key value are summed by Reduce function. The key values output by the Reduce function are sorted in order of frequency from high to low. Finally, FList is obtained according to the minimum support filtering. Step 3, Grouping for Frequent 1- item sets: All frequent items in FList are evenly divided into GROUP G, in which each group has a unique group ID- Gid. Frequent items and their corresponding group numbers constitute frequent item table group GList and are assigned into all slices. Step 4, design parallel FP-growth: This step is the core step of the parallel algorithm. Partial FP-growth mining is operated through a MapReduce. Job, where the Mapper function is defined by key pairs of to read each slice. Scan all frequent items A i in transaction T i , find the corresponding group ID identifier Gid in the corresponding frequent item table group GList, and output it with the key value. The output of Mapper function is read through the Reduce function corresponding to different Key values, local FP-tree is built, and local FP-growth mining is carried out for frequent items corresponding to each Gid in a recursive way. The frequent item sets obtained from mining are saved in a heap in order from large to small, and the top K frequent item sets with the highest support are finally returned. Step 5. Aggregation: Read the output of Step 4, combine the same key-value pairs via the Mapper function, and then use the Reduce function to calculate and output the frequent item sets of the top K items with the highest global support and confidence. 3.3 Implementation of optimized parallel FP-growth algorithm with load balancing constraints 3.3.1 The load balancing definition and grouping strategy Load balancing refers to the balancing of loads (work tasks) and spreading them across multiple units of operation so that work tasks can be completed cooperatively. The load balancing defined in this research refers to the balanced grouping of frequent 1-itemsets to achieve the relative balance of the running load for each slice, so as to improve the efficiency of data mining. The core strategy of load balancing mainly includes two steps: The first step is to build a load model and recursively mine the total workload of FP-tree on the projection database for each frequent data item. The second step is to divide all frequent items into G groups by balanced grouping strategy and divide transaction T into G groups to ensure the load balancing of each node. Tasks are completed by corresponding nodes of various groups, and FP-tree is recursively mined for frequent items, so as to improve the processing capability of the system. (1) Establishment of load model For the FP-growth algorithm, it is not realistic to accurately calculate the workload of each frequent item, we can only reasonably estimate the workload. The workload of recursively mining the condition FP-tree of each frequent item is equal to the times that the condition FP-tree of the frequent item recursively calls FP-growth. The length of the longest frequent path of each frequent item in the conditional FP-tree is equivalent to the location of the frequent item in the FList. In the conditional FP-tree, the number of times that the same frequent item recurrently calls FP-growth has an exponential relationship with the length of its longest frequent path [ 18 – 20 ], and the load model formula (1) can be obtained, where F i is the load estimated value of each node, and P ( I , FList) represents the position of the frequent item in FList of frequent 1-itemset: F i = log P( i, FList ) (1) (2) Balanced grouping According to formula (1), in frequent 1-itemset FList, the higher the support count of frequent items is, the higher its position is, and the larger its load value is. Therefore, the load value of frequent items in FList forms an increasing trend. Since it is difficult to achieve a globally optimal grouping, this research uses a greedy strategy to group frequent items. Assuming that the quantity of item groups is G , FList is grouped in order from the back to the front. First, the first G frequent items are put into G groups, and then the frequent items to be grouped are put into the group with the smallest total load in sequence. At the same time, the load value of the frequent item is accumulated to the total load of the group. Repeat the steps until all the frequent items in FList are allocated. 3.3.2 Implementation of load balancing algorithm The process of the improved load-balancing parallel FP-growth algorithm is the same as that of the above parallel FP-growth algorithm. Only the frequent 1-itemset grouping in the third step is improved. The pseudo-code and flow chart of the algorithm are shown in Table 2 and Fig. 3 respectively. Table 2 Code implementation of load balancing algorithm Load balancing algorithm pseudocode: generateGList Function: 1. void generateGList(List FList, int G){ 2. HashTable GList; 3. Heap minHP; 4. groupsNum ← G; 5. Gid ← 0; 6. if FList.length() < groupsNum then 7. foreach item a j in FList do 8. GList.put(a j , Gid); 9. Gid ← Gid + 1; 10. end 11. else 12. for j = 0 to groupsNum do 13. F(item[j]); // Load estimation 14. minHP.add(item[j]); 15. minHP.adjust(); 16. end 17. for j = groupsNum to FList.length() do 18. F(item[j]); // Load estimation 19. minHP[0].weight ← minHP[0].weight + item[j].weight; 20. GList.put(item[j], minHP[0].group); 21. minHP.adjust(); 22. end 23. end 24. } 3.4 Construction of experimental environment In this research, there is a large amount of historical crash data on freeways, so the complete distribution mode is selected in the experiment to build a Hadoop distributed processing system. A cluster is built with three computers installed with Linux system. The cluster has three nodes in total, including one master node and two slave nodes. The hardware and software configuration of each computer is shown in Table 3 . Table 3 Virtual machine configuration The hardware configuration The software configuration CPU: Intel(R) Xeon(R) CPU E5-2660 v2 @ 2.2GHz Linux: Ubantu 18.04.5 Memory: 32GB Java version: JDK-8U161-linux-x64 Hard Disk: 360GB Hadoop version: Hadoop-2.7.4 GPU: AMD_Radeon R7 350 Integrated development environment: Eclipse The process of building a Hadoop framework cluster is as follows: Step 1: Install Linux Ubantu 18.04.5 on the three computers respectively, change the host name and network configuration to "master", "slave1", and "slave2". Step 2: Modify the hosts file on the computer, set the public and private keys to implement SSH login without password between computers. Step 3: Upload and install JDK and configure environment variables for these computers. Step 4: Upload and install Hadoop, perform operations on environment variables and configuration files in the conf file of the computer, and format HDFS. Step 5: Start the Hadoop cluster and run the JPS command to view all processes. The Hadoop cluster process starts as shown in Fig. 4 . 3.5 Comparison of improved algorithm efficiency In order to verify the rationality of the algorithm designed in this research, the serial FP-growth algorithm, parallel FP-growth algorithm, the parallel FP-growth algorithm with load balancing constraints are respectively run under the Hadoop cluster. In this experiment, the computational efficiency of the algorithm is compared and analyzed from the perspectives of data volume and minimum support. (1) Sensitivity analysis for data volume on algorithm efficiency Various records are randomly selected from the data set to constitute the data set, and the four virtual processed data sets used in the experiment include 100,000, 500,000, 1,000,000 and 1,500,000 records respectively. The results of running the three algorithms with a minimum support of 0.5 are shown in Fig. 5 . Figure 5 indicates that, when the experimental data set is small, the running time difference of the three algorithms is very small. The possible explanation is that the two parallel FP-growth algorithms need extra time to start the cluster, which fails to take advantage of the parallel algorithm. With the increase of the data set, the operation time of the two parallel FP-growth algorithms is greatly reduced compared with the serial algorithm, and the performance advantage of the algorithms is more obvious with a larger number of records. Compared with the parallel FP-growth algorithm without load balancing, the operation time of the parallel FP-growth algorithm with load balancing based on balanced grouping method can be further reduced, which is more suitable for mining large data sets. The experimental results in Fig. 5 can prove that the optimized load-balancing parallel FP-growth algorithm can not only solve the problems such as large memory consumption and long running time when the traditional serial algorithm performs big data mining, but also further shorten the data mining time and improve the efficiency of the algorithm. (2) Sensitivity analysis for the minimum on algorithm efficiency According to the previous analysis of association rules and algorithms in this research, the setting of minimum support degree has a great impact on the operation efficiency of the algorithm. The smaller the minimum support, the greater the number of frequent items, and the longer the operation time of the algorithm. In this experiment, the data set with 500,000 records in experiment (1) is selected, and three algorithms are adopted for data mining under the condition that the minimum support is set at 0.1–0.7. The calculation time comparison is shown in Fig. 6 . Figure 6 indicates that, when the minimum support is high, such as 0.6 and 0.7, the number of frequent items in the data set is small, and the operation time of the three algorithms is short. With the decrease of support, the number of frequent items in the data set increases, and the operation time of the algorithms begins to rise. When the minimum support is lower than a certain degree, the serial FP-growth algorithm has the problem of memory heap overflow. At this time, the algorithm will automatically report an error and stop running. However, the parallel FP-growth algorithm overcomes this shortcoming, and can still run under a lower support threshold, its running time is less than the serial algorithm. According to the overall running time of the algorithm, the running efficiency of the optimized parallel FP-growth algorithm is higher than that of the original one, its advantages become more prominent as the support threshold decreases. The above experimental results show that the optimized parallel FP-growth algorithm based on Hadoop proposed in this research can perform efficient data mining in the case of low support. The operating efficiency of the improved algorithm is obviously better than that of the serial algorithm and the original parallel FP-growth algorithm, and it can be well applied to the subsequent data mining for freeway traffic crashes in this research. 4. Cause Analysis Of Freeway Traffic Crashes Based On Improved Fp-growth Algorithm During the process of FP- Growth association rule mining algorithm, parameter threshold Settings is closely related to the efficiency and quality of mining, considering the coupling mechanism between precursors related to serious casualty as high practical significance with the traffic safety, consequence with the values of "death or serious injury" in the original data set actually has a lower percentage actually. If the minimum support threshold is set too high, death or serious injury both attribute variables may be cut off in the pruning process. Therefore, in this study, the minimum support threshold will be set based on the occurrence frequency of the casualty attribute and its related attributes, as far as possible to efficiently and accurately mining association rules effectively. 4.1 Mining Results and analysis Multi-dimensional interaction association rule mining refers to that the directed association mining in the form of "LHS = > RHS" is the association rule of constraint condition. After a lot of tests, set the minimum support threshold as 0.05 (in case ignoring the traffic crashes with severe consequences but lower frequency), minimum confidence threshold is set as 0.4, the lift degree threshold to set as 1. The variables in the dimension of "Driver, vehicle, road, environment, time" are constrained in LHS, the crash dimension variables are constrained in the RHS. Then the association rules with high support and high confidence obtained from mining are extracted respectively, as shown in Tables 4 and 5 . Tab.4 Mining results of association rules (In descending order of support) Rules (LHS=>RHS) Sup Conf Lift {Speeding=N1,Drowsy=Q1,Vehicle Condition=AG1, Unlicensed=P1,Distracted=O1}{Number of Motor Vehicles Involved=AD2,Ejection Status=W1} 0.529 0.901 1.013 {Speeding=N1,Age=J3,Drowsy=Q1,Vehicle Condition=AG1, Unlicensed=P1,Distracted=O1}{Number of Motor Vehicles Involved=AD2} 0.522 0.919 1.074 {Speeding=N1,Vehicle Condition=AG1,Roadway Surface Condition=I1,Weather=Z1,LightingCondition=AA3}{Ejection Status=W1,Injury Type=AH1} 0.507 0.782 1.016 {Gender=K2,Speeding=N1,Drowsy=Q1,Roadway Surface Condition=I1}{Ejection Status=W1,Injury Type=AH1} 0.489 0.891 1.006 {Speeding=N1,Restraining System Type=X1,Lighting Condition =AA3}{Ejection Status=W1,Number of Motor Vehicles Involved=AD2} 0.449 0.827 1.127 {Speeding=N1,Drowsy=Q1,VehicleCondition=AG1,Roadway Surface Condition=I1}{Ejection Status=W1,Injury Type=AH1} 0.377 0.907 1.023 {Age=J2,Speeding=N1,Drowsy=Q1,Vehicle Condition=AG1, Sobriety Level=L1,Weather=Z1}{Injury Type=AH1} 0.368 0.877 1.077 {Gender=K1,Speeding=N1,Drowsy=Q1,Vehicle Condition =AG1,Unlicensed Driver=P1,Roadway Surface Condition} {Ejection Status=W1,Injury Type=AH1} 0.329 0.659 1.109 {Speeding=N1,Drowsy=Q1,Vehicle Condition=AG1,Sobriety Level=L1}{Ejection Status=W1,Injury Type=AH1} 0.322 0.792 1.074 {Quarter Number=G1,Speeding=N1,Vehicle Condition=AG1, Lighting Condition=AA3}{Towed Indicator=D1} 0.317 0.843 1.008 {Quarter Number=G3,Vehicle Condition=AG1,Unlicensed Driver=P1,Lighting Condition=AA3}{Injury Type=AH1} 0.310 0.891 1.015 {Speeding=N1,Drowsy=Q1,Roadway Characteristic=B1, Vehicle Action=AF2}{Collision Severity=AE1} 0.303 0.922 1.018 {Speeding=N1,Drowsy=Q1,Vehicle Condition=AG1,Unlicensed Driver=P1}{Number of Motor Vehicles Involved=AD2} 0.293 0.864 1.152 Through the analysis of the results in Table 4 , the following conclusions can be drawn: Traffic crashes in the study area are mostly caused by passenger cars, most of which involves male drivers. Minor crashes those do not cause casualties account for the majority, and most of them occur on urban or interstate freeways. Most of the crashes involve only 1–2 vehicles. Traffic crashes those do not result in injuries are probably not caused by drivers' driving violations. A possible explanation for the higher incidence of minor crashes those do not result in injuries is that bad weather is less common in summer and roads' conditions are basically dry. Non-fatal crashes usually do not cause airbags ejection. Most of the traffic crashes those occur when the vehicle is going straight may result in minor traffic crashes with no casualties. Tab.5 Mining results of association rules under (In descending order of confidence) Rules (LHS=>RHS) Conf Sup Lift {Age=J2,Weather=Z2,Roadway Surface Condition=I2} {Collision Severity=AE2} 0.968 0.158 27.970 {Day Of Week=F1,Quarter Number=G1,Hour=H1,Roadway Surface Condition=I3}{Number of Motor Vehicles Involved=AD3} 0.961 0.057 37.801 {Age=J1,Speeding =N2,Unlicensed=P2,Lighting Condition=AA2} {Collision Severity=AE3} 0.954 0.071 86.598 {Gender=K2,Sobriety Level=L3,Roadway Surface Condition=I7} {Collision Severity=AE3} 0.939 0.069 78.723 {Age=J2,Day Of Week=F1,Hour=H1,Roadway Surface Condition =I3}{Collision Severity=AE2} 0.937 0.125 18.652 {Vehicle Type=T2,Vehicle Condition=AG2,Distracted=O1,Speeding =N2}{Towed Indicator=D2,Collision Severity=AE3} 0.930 0.271 3.672 {Day Of Week=F2,Quarter Number=G4,Hour=H2}{Injury Type =AH1} 0.927 0.193 1.162 {Speeding=N1,Drowsy=Q1,Roadway Characteristic=B1,Vehicle Action=AF2}{Number of Pedestrians Involved=AC1,Collision Severity=AE1} 0.922 0.303 1.018 {Speeding=N1,Age=J3,Drowsy=Q1,Vehicle Condition=AG1, Unlicensed=P1,Distracted=O1}{Number of Motor Vehicles Involved=AD2} 0.919 0.522 1.074 {Age=J4,Drowsy=Q2,Vehicle Type=T1}{Number of Motor Vehicles Involved=AD2} 0.903 0.081 63.190 The following conclusions can be drawn from the results in Table 5 : When the driver has dangerous driving behaviors such as overspeed driving, fatigue driving, distracted driving, unlicensed driving or drunk driving, and the road surface is wet or snowy, the traffic crashes easily tend to cause casualties. When the driver's dangerous driving behavior and slope road appear at the same time, it is also very easy to cause serious consequences of casualties. The combination of unsafe driving and bad weather often leads to serious injury or death. Vehicle failure and bad weather at the same time tend to cause casualties. Among the traffic crash, drivers under the age of 25 are often associated with speeding, and drivers over the age of 65 are often associated with distracted driving. Most of the association rules extracted above include driver factors, so it indicates that human is the main operator in traffic and the key factor that determines whether a crash occurs or not. 4.2 Suggestions for improving freeway traffic safety based on association rule mining The occurrence frequency of association rules with high confidence is typically less than that of association rules with high support. The LHS occurrence of this kind of association rules can easily cause the occurrence of the RHS, but if any field variables in LHS changes, the association rule may no longer hold. Therefore, the control principle for these kind of association rules is to control from LHS and RHS simultaneously. The following suggestions are made for low-support high-confidence association rules that cause serious injuries: (1) Rain or snow weather + large van + curve with slopes Rain and snow may interfere with the driver's vision and visibility, and on the one hand, it may be accompanied by ice or wet on the road, both of which will lead to an increase in the braking distance of the vehicle. Large trucks are relatively heavy in their own inertia, and in the rain and snow, they usually have the risk of rushing out of the road or crashing into nearby vehicles when they are in a crash-prone situation, both of which have serious consequences. In this case, we can avoid the occurrence of RHS by controlling the occurrence of LHS: the first is to optimize the design of the mark line in the curve with slope, set up the speeding capture equipment, and control the speed on the dangerous road. Secondly, we should increase the inspection for large truck overload, and set up the emergency safe lane on the dangerous road such as the continuous slope, and the surface of the road should be made of small stones, so that the vehicles in the emergency can slow or stop immediately, thereby avoiding the occurrence of the crash or reducing the severity. In addition, the safety guard should be installed to prevent the vehicle from sliding out of the road when it collided in a detour, the monitoring for road environment should be improved, severe weather that threatens road traffic safety should be forecasted timely, road cleaning and maintenance during rain and snow should be should be taken into account, such as by sprinkling industrial salt to melt and clear the snow on the road. (2) Tire failure + passenger car + peak hours Traffic crashes of passenger cars with tire faults during peak hours usually lead to injuries and most of them involve multiple motor vehicles. On the one hand, the traffic volume during peak hours is large and the driving distance between vehicles is small. On the other hand, the passenger cars with tire faults may be uncontrolled and easily collide with surrounding vehicles.After the collision, the surrounding vehicles are prone to serial rear-end collision because of a small distance between vehicles, consequently resulting in injuries. For this kind of situation, related department should guide the vehicle owner regularly check, increase the frequency of the motor vehicle compulsory inspection or strictly control the test process, minimize the vehicle potential safety hazard as much as possible. In addition, the traffic volume during peak hour is large, which is prone to congestion, policies such as encouraging off-peak commuting can be adopted to realize the time separation of traffic flow. (3) Main line + straight road with slope + dust or sand blowing weather The majority of crashes on the Main line result in injuries due to dust or sand-blowing weather, which can affect drivers' vision and sight. Druing the periods of design, construction and management traffic safety should be the first consideration. For the continuous steep slope, steep curve, long down ramp and long straight road and other dangerous roads, traffic safety risks should be eliminated timely. Early warning should be given to bad weather such as dust or sand blowing, and monitoring of road environment should be strengthened. (4) Drivers over 65 + distracted driving + flat road Most of the 65 + aged drivers have some impairment of vision and hearing, most crashes involving distracted driving on straight roads involve death or injury. For this situation, elderly drivers should be subjected to an annual physical examination, and their driving licenses should be revoked if they do not meet the conditions. For drivers with distracted driving, some stimuli can be used to make them keep their attention in high concentration, color road signs may work to stimulate their vision, speed bumps can be set to stimulate their senses, and warning lights or slogans can be installed to attract their attention. In road design, long straight lines should be avoided. If unavoidable, raised warning signs must be set up on the road surface every certain distance to prevent the driver from appearing shallow sleep due to the monotonous driving process. The mining results with high support are mostly minor traffic crashes without casualties, indicating simultaneous occurrences of a certain attribute combination account for a high proportion in the total number of items. The safety control principle for such crashes is to avoid RHS by controlling LHS. 5. Conclusions This research proposes a parallel FP-growth association rule mining algorithm considering load balancing, compares the operational efficiency with serial FP-growth algorithm and conventional parallel FP-growth algorithm using various data set volumes and support threshold degree respectively. Based on the proposed association rule mining algorithm considering load balancing, the freeway traffic crash data applied in this research is analyzed with the consideration of directional constraints under multi-dimension and multi-level perspective, the results of association rule mining with high support and high confidence (including low support) are obtained. The specific conclusions of this research are as follows: (1) Compared with serial FP-growth algorithm or traditional parallel FP-growth algorithm, the proposed parallel FP-growth association rule mining algorithm considering load balancing has more advantages in computational efficiency, and it can reduce the possibility of leading to memory heap overflow when processing massive data or low support threshold setting work. It is more suitable for mining large traffic crash data sets. (2) Serious consequences data pieces in crash data sets may be assigned a low degree of support (which may have a high confidence degree) when the association rule mining algorithm is applied, these crashes are easy to be ignored. The algorithm proposed in this research has great advantages in mining association rules with low support threshold. It can overcome the shortcomings of the mining work via traditional association rule mining algorithm towards the data set including serious traffic crash consequence, as well as better reveal the coupling mechanism among the factors related serious traffic crash. (3) Association rules with high confidence indicate that the association rules have high conditional probability: for the situation combinations with high confidence appear in the LHS, attention should be paid, so as to prevent the serious consequences of traffic crashes. An association rule with high support indicates the high occurrence frequency of the association rule. For an association rule with high occurrence frequency but minor crash consequence, the LHS should be controlled to prevent the consequences in RHS. (4) Traffic safety is an important element of sustainable transportation, and association rule mining algorithm is an effective method in the field of fuzzy decision. The approach proposed in this research can provide theoretical guidance for fuzzy identification of sustainable transportation. Due to the limitation of experimental conditions, this research adopts three computers to build a fully distributed experimental mode of Hadoop. This method can realize the database association rule mining, but it is not enough to analyze the impact of cluster node number on operation efficiency. In the future research, the experimental environment should be improved to get more accurate experimental data. Declarations Compliance with Ethical standards: Humans and animals are not involved in this research work. We used our own data. Funding This work was supported by China Postdoctoral Science Foundation [Grant NO. 2021M700333]. Competing Interests The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. References World Health Organization (2018) Global status report on road safety 2018: Summary[R]. World Health Organization Hamed MM, Al-Eideh BM (2020) An exploratory analysis of traffic accidents and vehicle ownership decisions using a random parameters logit model with heterogeneity in means[J]. Analytic methods in accident research 25:100116 Yu Q (2013) Causes and prevention measures of secondary rear-end accidents in the rescue of highway traffic accidents[J]. Procedia Eng 52:571–577 Kwayu KM, Kwigizile V, Lee K et al (2020) Discovering latent themes in traffic fatal crash narratives using text mining analytics and network topology[J], vol 150. Accident Analysis & Prevention, p 105899 Sun J, Li T, Li F et al (2016) Analysis of safety factors for urban expressways considering the effect of congestion in Shanghai, China[J], vol 95. Accident Analysis & Prevention, pp 503–511 Yang Y (2020) Research on the Method of Freeway Crash Risk Identification and Comprehensive Traffic Safety Evaluation Considering the Regional Type Difference[D]. Beijing Jiaotong University Yang Y, Yuan ZZ, Sun DY et al (2019) Analysis of the factors influencing highway crash risk in different regional types based on improved Apriori algorithm. J Adv Transp Stud 49:165–178 Yang Y, Yuan Z, Chen J et al (2017) Assessment of osculating value method based on entropy weight to transportation energy conservation and emission reduction. J Environ Eng Manage J 16:2413–2424 Singh S, Garg R, Mishra PK (2018) Performance Optimization of MapReduce-based Apriori Algorithm on Hadoop Cluster[J].Computers & Electrical Engineering, :348–364 Thangaraj DVSJJ, Khanna MR "An Improved Early Detection Method of Autism Spectrum Anarchy using Euclidean Method," 2020 Fourth International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC), 2020, pp. 1173–1178, doi: 10.1109/I-SMAC49090.2020.9243361 Yu L, Du B, Hu X et al (2021) Deep spatio-temporal graph convolutional network for traffic accident prediction[J]. Neurocomputing 423:135–147 Jiang F, Yuen KKR, Lee EWM (2020) Analysis of motorcycle accidents using association rule mining-based framework with parameter optimization and GIS technology[J]. J Saf Res 75:292–309 Huang Y, Huang J, Liu C et al (2020) PFPMine: A parallel approach for discovering interacting data entities in data-intensive cloud workflows[J]. Future Generation Computer Systems 113:474–487 Samerei SA, Aghabayk K, Mohammadi A et al (2021) Data mining approach to model bus crash severity in Australia[J]. J Saf Res 76:73–82 Montella A, de Oña R, Mauriello F et al (2020) A data mining approach to investigate patterns of powered two-wheeler crashes in Spain[J], vol 134. Accident Analysis & Prevention, p 105251 Zheng Z, Lu P, Lantz B (2018) Commercial truck crash injury severity analysis using gradient boosting data mining model[J]. J Saf Res 65:115–124 Bechini A (2016) A Map Reduce solution for associative classification of big data[J]. Inform Sciences: Int J 332:33–55 Iko Pramudiono and Masaru Kitsuregawa.Parallel FP-Growth on PC cluster.In PAKDD,2003 Le Zhou,Zhiyong Zhong,Jin Chang,Junjie Li,Joshua Zhexue Huang,Shengzhong Feng.Balanced parallel FP-Growth with MapReduce.Information Computing and Telecommunications (YC-ICT),2010 IEEE Youth Conference on 28–30 : 243–246 He K, Yuan Z, Yang Y (2022) “A Roadway Safety Sustainable Approach: Modeling for Real-Time Traffic Crash with Limited Data and Its Reliability Verification,” Advances in Transportation Studies, Journal of Advanced Transportation, vol. Article ID 1570521, 2022 Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-1311180","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":86256535,"identity":"751e259b-eadd-441c-86ba-2a1037607ae6","order_by":0,"name":"Yang Yang","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA10lEQVRIiWNgGAWjYLACxgZmOQMIk5l4Lcaka0ncQLQWg+NnD78u3GGdvl0i+eEHhgrrxAb2swfwazmTl2Y980x67s4ZacYSDGfSExt48hLwazmQY2bM23Y4d8ONBAMJxrbDiQ0SPAb4tZx/A9aSbnAj/fMPxn/EaLmRY/wYqCUByDCTYGwgQovkjTdmzDPb0g139rwps0g4lm7cxpODXwvf+Rzjz4Vt1vLm7Ombb3yosZbtZz+DX4vCAQY2aTgvAYjZ8KoHAvkGBubPhBSNglEwCkbBCAcAjklI5Mft2HoAAAAASUVORK5CYII=","orcid":"","institution":"Beihang University","correspondingAuthor":true,"prefix":"","firstName":"Yang","middleName":"","lastName":"Yang","suffix":""},{"id":86256536,"identity":"3ab14064-ad24-4597-ada3-39e700154fa8","order_by":1,"name":"Na Tian","email":"","orcid":"","institution":"Beijing Jiaotong University School of Traffic and Transportation","correspondingAuthor":false,"prefix":"","firstName":"Na","middleName":"","lastName":"Tian","suffix":""},{"id":86256537,"identity":"4d87abd4-665d-480d-a266-32370f0d580c","order_by":2,"name":"Yunpeng Wang","email":"","orcid":"","institution":"Beihang University","correspondingAuthor":false,"prefix":"","firstName":"Yunpeng","middleName":"","lastName":"Wang","suffix":""},{"id":86256538,"identity":"d601a8a6-3e45-49df-83f8-7cac90eaa6de","order_by":3,"name":"Zhenzhou Yuan","email":"","orcid":"","institution":"Beijing Jiaotong University School of Traffic and Transportation","correspondingAuthor":false,"prefix":"","firstName":"Zhenzhou","middleName":"","lastName":"Yuan","suffix":""}],"badges":[],"createdAt":"2022-01-30 07:51:42","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-1311180/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-1311180/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":18747808,"identity":"10d30663-f19f-4a4e-9011-b2961443b191","added_by":"auto","created_at":"2022-03-01 20:01:29","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":122606,"visible":true,"origin":"","legend":"\u003cp\u003eSchematic diagram of local FP Tree in horizontal division mode\u003c/p\u003e","description":"","filename":"fig1.png","url":"https://assets-eu.researchsquare.com/files/rs-1311180/v1/5adeab99517daa25a20b943e.png"},{"id":18747888,"identity":"256b6bbc-beef-4326-a822-98b3d878eb2b","added_by":"auto","created_at":"2022-03-01 20:04:29","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":34565,"visible":true,"origin":"","legend":"\u003cp\u003eParallel FP-Growth algorithm flow chart\u003c/p\u003e\u003cp\u003e\u003cbr\u003e\u003c/p\u003e","description":"","filename":"fig2.png","url":"https://assets-eu.researchsquare.com/files/rs-1311180/v1/23e22f15dd12420bd7198f16.png"},{"id":18747810,"identity":"82acb2f9-5f45-4654-b7d6-676c974f4ec8","added_by":"auto","created_at":"2022-03-01 20:01:29","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":19542,"visible":true,"origin":"","legend":"\u003cp\u003eLoad balancing algorithm flow chart\u003c/p\u003e","description":"","filename":"fig3.png","url":"https://assets-eu.researchsquare.com/files/rs-1311180/v1/e0d94349dee31b67de236ec4.png"},{"id":18747812,"identity":"000d9e63-7e94-4c1e-9df4-5568a0a26d05","added_by":"auto","created_at":"2022-03-01 20:01:29","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":215082,"visible":true,"origin":"","legend":"\u003cp\u003eHadoop cluster startup results\u003c/p\u003e","description":"","filename":"fig4.png","url":"https://assets-eu.researchsquare.com/files/rs-1311180/v1/79984781dfe1b9446cca3b66.png"},{"id":18747809,"identity":"9122044c-ff3e-4877-a7fa-750035f8ed24","added_by":"auto","created_at":"2022-03-01 20:01:29","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":45745,"visible":true,"origin":"","legend":"\u003cp\u003eComparison results of operation time of three algorithms with various data volume\u003c/p\u003e\u003cp\u003e\u003cbr\u003e\u003c/p\u003e","description":"","filename":"fig5.png","url":"https://assets-eu.researchsquare.com/files/rs-1311180/v1/b47f4f3438455e9cbc4b7e0e.png"},{"id":18747887,"identity":"450cdff2-238d-41f2-ab82-16bf2b1d096c","added_by":"auto","created_at":"2022-03-01 20:04:29","extension":"png","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":45901,"visible":true,"origin":"","legend":"\u003cp\u003eComparison results of operation time with various minimum support thresholds\u003c/p\u003e","description":"","filename":"fig6.png","url":"https://assets-eu.researchsquare.com/files/rs-1311180/v1/6d993ccf456d68ddf3e9c93e.png"},{"id":19981982,"identity":"91bb8109-a7dc-44da-8720-4c81ed15e5d9","added_by":"auto","created_at":"2022-04-05 16:00:03","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":788174,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-1311180/v1/7b2084d6-aa14-4e94-a71f-cd35df91189c.pdf"}],"financialInterests":"","formattedTitle":"\u003cp\u003eA Parallel FP-Growth Mining Algorithm with Load Balancing Constraints for Traffic Crash Data\u003c/p\u003e","fulltext":[{"header":"1. Introduction","content":"\u003cp\u003eTraffic crash is a serious threat to public health safety and have become the eighth leading cause of human death. The number of deaths caused by traffic crashes continues to increase, with about 1.35\u0026nbsp;million people dying every year, and the main cause of death for people aged 5 to 29 is traffic crash [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e]. Especially on freeway, the vehicles usually run at a fast speed, once the traffic crash occurs, it is easier to cause serious casualties. The coupling mechanism analysis of the factors influencing traffic crash on the freeway, has a certain significance to improve the traffic safety level for freeway.\u003c/p\u003e \u003cp\u003eWith the advent of the era of traffic big data, the technology of traffic crash data collection is becoming mature, which provides a strong data support for the analysis of traffic crash coupling mechanism. It was common to use historical traffic crash data to figure out the coupling mechanism of crashes. The quantity and quality of historical data have a crucial influence on the analysis results. Hamed et al. [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e] proposed a random parameter Logit model with heterogeneity to study the interaction between traffic crashes and vehicle holding quantity. By studying the causes of freeway traffic crashes, Yu [\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e] focused on analyzing the causes of secondary rear-end collisions of freeway traffic crashes. Kwayu [\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e] et al., tested the universality and interaction among the influencing factors of fatal traffic crashes via structured modeling and network topology analysis, and explored the causes of fatal traffic crashes with the data of fatal traffic crashes in Michigan from 2009 to 2018. Sun et al. [\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e] established a Bayesian spatial model to reveal the potential spatial correlation between segments.\u003c/p\u003e \u003cp\u003eAssociation rule mining is a fuzzy recognition and decision method based on data mining technology. It is an efficient data mining method, which can be used to identify the combination of key factors related to crash consequences. Mining association rules for freeway traffic crash data can facilitate figuring out the potential association relationship among the influencing factors. Yang [\u003cspan additionalcitationids=\"CR7\" citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e] proposed a multi-dimensional interaction improved Apriori association rules mining algorithm considering directional constraints and index weights: WOMID-Apriori, based on subjective and objective combined weighting model the interval analytic hierarchy Process (AHP) and grey relational degree, the weight of data fields was calculated and optimized, then, the algorithm was applied to address the cause analysis of freeway crashes. Singh [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e] et al. improved the performance of Apriori algorithm and proposed two improved Apriori algorithms based on MapReduce: VFPC (Elapsed Time-based Dynamic pass-counting) and ETDPC (Elapsed Time-based Dynamic pass-counting). Wang [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e] et al. discussed the algorithm for sorting variables in the data set and proposed two new technologies, MG and SMGR. These two technologies were developed for public safety using classified traffic crash data, which were more intuitive than existing technologies. Yu et al. [\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e] proposed a new DSTGCN mixed spatio-temporal graph convolutional network to predict traffic crash. Jiang [\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e] et al. proposed a framework based on association rule mining to identify the factors related to the severity of motorcycle injuries. In order to objectively determine the threshold value of parameters, they developed a parameter optimization method. Huang et al. [\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e] proposed a parallel FP-growth algorithm based on cloud computing. Samerei [\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e] et al. worked on the bus crash data set of Victoria State, Australia from 2006 to 2019, divided the traffic crash data into clustering, extracted the factors affecting bus traffic crash fatality based on association rule mining algorithm. Montella [\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e] et al. used classification trees and association rule mining algorithm to analyze 78,611 crash data involving two-wheeled vehicles in Spain, aiming to extract knowledge from a large amount of data and identify understandable association rules. Zheng et al. [\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e] applied the gradient lifting algorithm to analyze the relationship between collision severity and influencing factors. Bechini et al. [\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e] proposed a distributed association rule mining algorithm based on MapReduce programming model, confirmed the scalability and performance improvement of this method on small computer clusters.\u003c/p\u003e \u003cp\u003eGlobal scholars have made some achievements in the research of traffic crash analysis methods and technologies so far. However, there are also some deficiencies in these studies: (1) Most of the algorithms currently applied in the field of traffic crash data mining are ordinary serial algorithms, a huge number of candidate item sets may be generated in the mining process. The support of the algorithm during the calculation requires a large amount of memory, and it is unable to efficiently mine the massive traffic crash data. In addition, the ordinary parallel association rule mining algorithm generally has the problem of load imbalance, which may lead a low computing efficiency. Most of the existing research improved the algorithm by adding constraint conditions to achieve the efficient extraction of mining results, but lack of improvement on the performance of mining algorithm itself. (2) Existing association rules mining basically focused on the high degree level of support and confidence of association rules, while only few studies focused on the combination rules with low support and high confidence those were prone to lead to serious consequences such as casualties, which may lead that the potential important traffic crash precursors missed. However, the characteristics and mechanism of serious casualty crashes and ordinary crashes may be different, so the conventional safety improvement measurements for ordinary crashes may not work for the serious crashes.\u003c/p\u003e \u003cp\u003eIn view of the above defects, based on Hadoop platform, this research proposed a parallel FP-growth algorithm considering load balancing for freeway traffic crash data, which is used to mine multi-dimensional and multi-level massive crash data. Then the association rules with high support and confidence from mining results were extracted and interpreted. Finally, based on the mining results, the suggestions to improve freeway traffic safety were put forward.\u003c/p\u003e"},{"header":"2. Data Process","content":"\u003cdiv class=\"Section2\" id=\"Sec3\"\u003e\n \u003ch2\u003e2.1 Data source description\u003c/h2\u003e\n \u003cp\u003eThe data source of this research is the UW DriveNet Intelligent Transportation Big Data platform of the University of Washington in the United States. Through the connection between the platform and WDOT, a total of 345,545 pieces of freeway traffic crash data in Washington state can be obtained, including 384 attributes, specifically includes: Driver\u0026apos;s gender, age, driver\u0026apos;s degree of soberness, crash location, vehicle type, type of airbag, state of airbag ejection, weather, road type, traffic control, crash time and season, etc.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv class=\"Section2\" id=\"Sec4\"\u003e\n \u003ch2\u003e2.2 The establishment of multi-dimensional and multi-layer data system\u003c/h2\u003e\n \u003cp\u003eThe multi-dimensional and multi-layer traffic crash coupling mechanism analysis system is the basic framework of association rule mining. In this research, crash influencing factors are taken as the first layer of the system, including six dimensions \u0026quot;driver, vehicle, road, environment, time, crash\u0026quot;. In the six dimensions of the first layer of source data, specific influencing factors with analytical value are screened out respectively. As the second layer, there are 29 dimensions in total. The attributes of the influencing factors in the second layer are further refined, and the specific indexes and values of the attributes are clarified, forming the third layer of the system. The specific indexes and values of the crash dimension are shown in Table \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e.\u003c/p\u003e\n \u003cdiv class=\"gridtable\"\u003e\u003ctable border=\"1\" id=\"Tab1\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eSpecific indexes and values of influencing factors of collision dimension\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\" style=\"width: 37.7952%;\"\u003e\n \u003cp\u003eFactors\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" style=\"width: 34.6457%;\"\u003e\n \u003cp\u003eValues\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eCodes of Valus\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" rowspan=\"4\" style=\"width: 37.7952%;\"\u003e\n \u003cp\u003eInjury Type\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 34.6457%;\"\u003e\n \u003cp\u003eNo Injury\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAH1\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 34.6457%;\"\u003e\n \u003cp\u003ePossible Injury\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAH2\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 34.6457%;\"\u003e\n \u003cp\u003eSerious Injury\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAH3\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 34.6457%;\"\u003e\n \u003cp\u003eDeath\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAH4\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" rowspan=\"4\" style=\"width: 37.7952%;\"\u003e\n \u003cp\u003eEjection Status\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 34.6457%;\"\u003e\n \u003cp\u003eNot Ejected\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eW1\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 34.6457%;\"\u003e\n \u003cp\u003eUnknown if Ejected\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eW2\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 34.6457%;\"\u003e\n \u003cp\u003ePartially Ejected\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eW3\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 34.6457%;\"\u003e\n \u003cp\u003eTotally Ejected\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eW4\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" rowspan=\"3\" style=\"width: 37.7952%;\"\u003e\n \u003cp\u003eCollision Severity\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 34.6457%;\"\u003e\n \u003cp\u003ePDO\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAE1\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 34.6457%;\"\u003e\n \u003cp\u003eInjury\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAE2\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 34.6457%;\"\u003e\n \u003cp\u003eFatality\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAE3\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" rowspan=\"2\" style=\"width: 37.7952%;\"\u003e\n \u003cp\u003eTowed Indicator\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 34.6457%;\"\u003e\n \u003cp\u003eNo\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eD1\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 34.6457%;\"\u003e\n \u003cp\u003eYes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eD2\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" rowspan=\"4\" style=\"width: 37.7952%;\"\u003e\n \u003cp\u003eCollision Report Type\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" style=\"width: 34.6457%;\"\u003e\n \u003cp\u003eSuburban\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eA1\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 34.6457%;\"\u003e\n \u003cp\u003eUrban\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eA2\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 34.6457%;\"\u003e\n \u003cp\u003eInterstate\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eA3\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" style=\"width: 34.6457%;\"\u003e\n \u003cp\u003eRamp\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eA4\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n \u003cp\u003eBased on the dimensional and hierarchical analysis results of the above crash attributes, the construction of multi-dimensional OLAP database and data mining in THE SQL Server Analysis Service (SSAS) of the database management system should be addressed. The specific steps are as follows:\u003c/p\u003e\n \u003cp\u003e(1) Import data preprocessed with Excel into Access.\u003c/p\u003e\n \u003cp\u003e(2) The fact table (freeway traffic crash) and 6-dimension tables (driver, vehicle, road, environment, time, crash) are constructed, the primary keys of each table are demarcated, and the relationship between the out keys of fact table and the primary keys of each dimension table are set up. Finally, the Access database is promoted to SQL Server database.\u003c/p\u003e\n \u003cp\u003e(3) Create the research project \u0026quot;Freeway Traffic Crash\u0026quot; in SSAS, import the data source.\u003c/p\u003e\n \u003cp\u003e(4) Set the ID attribute of the fact table \u0026quot;freeway traffic crash\u0026quot; as the metric value and \u0026quot;Count\u0026quot; as the aggregation mode, built the multi-dimensional database and dimension considering the data analysis requirements, and deploy the project to the server.\u003c/p\u003e\n\u003c/div\u003e"},{"header":"3. Design And Implementation Of The Algorithm","content":"\u003cp\u003eThe FP-Growth algorithm constructs the data set as FP-tree and stores it in memory, then mines frequent item sets by recursively calls condition of FP-tree [\u003cspan class=\"CitationRef\"\u003e18\u003c/span\u003e]. However, when mining massive data with low support degree, the FP-tree generated by FP-growth will occupy a large amount of memory, causing problems such as memory overflow and long operation time [\u003cspan class=\"CitationRef\"\u003e19\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e20\u003c/span\u003e]. This research tries to solve the above problems through the optimization design of the parallel FP-growth algorithm, proposes a load balancing optimization scheme based on the parallel FP-growth algorithm, and builds an algorithm experimental environment under condition of the Hadoop framework. Through experiments, the efficiency of traditional FP-growth algorithm, parallel FP-growth algorithm and parallel FP-growth algorithm considering load balancing with different data volume and different minimum support threshold are compared and analyzed respectively.\u003c/p\u003e\n\u003cdiv class=\"Section2\" id=\"Sec6\"\u003e\n \u003ch2\u003e3.1 Principle of parallel FP-Growth algorithm\u003c/h2\u003e\n \u003cp\u003eThe performance bottleneck of FP-growth algorithm when processing massive data is solved using horizontal partitioning [\u003cspan class=\"CitationRef\"\u003e19\u003c/span\u003e]. The horizontal partitioning method is to divide the database into various sub-database nodes, make frequency statistics for each sub-node, build local FP-tree, obtain the conditional pattern base of each data frequent item set, gather various conditional pattern bases to the corresponding nodes, and then build conditional FP-tree for each node, operate the local FP-growth mining. The local FP-tree diagram considering the influencing factors of traffic crashes is shown in Fig. \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv class=\"Section2\" id=\"Sec7\"\u003e\n \u003ch2\u003e3.2 Process design of parallel FP-growth algorithm\u003c/h2\u003e\n \u003cp\u003eThe operation process of the parallel FP-growth algorithm can be summarized into the following five steps, and the algorithm process is introduced in combination with this research case. The specific process design is shown in Fig. \u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e:\u003c/p\u003e\n \u003cp\u003eStep 1, Shard: The freeway historical crash database of Washington state is horizontally divided into six consecutive sub-databases from the dimensions of driver, vehicle, road, environment, time and crash, which are stored on several different computers respectively. Each sub-database obtained after segmentation is defined as slice.\u003c/p\u003e\n \u003cp\u003eStep 2, Parallel counting: Read each slice obtained in Step 1 and use \u0026quot;Mapreduce. job\u0026quot; to count frequent \u0026quot;1- itemset FList\u0026quot;. Where the \u0026quot;Mapper\u0026quot; function starts to input with the key pairs of \u0026lt;\u0026thinsp;The key, value\u0026thinsp;=\u0026thinsp;\u003cem\u003eT\u003c/em\u003e\u003csub\u003e\u003cem\u003ei\u003c/em\u003e\u003c/sub\u003e\u0026gt;, after processing, transaction \u003cem\u003eT\u003c/em\u003e\u003csub\u003e\u003cem\u003ei\u003c/em\u003e\u003c/sub\u003e is output as key pairs\u0026thinsp;\u0026lt;\u0026thinsp;Key\u0026thinsp;=\u0026thinsp;\u003cem\u003ea\u003c/em\u003e\u003csub\u003e\u003cem\u003ej\u003c/em\u003e\u003c/sub\u003e, value\u0026thinsp;=\u0026thinsp;1\u0026gt;, where AJ is the \u003cem\u003eJ\u003c/em\u003e\u003csub\u003eth\u003c/sub\u003e data item of \u003cem\u003eT\u003c/em\u003e\u003csub\u003e\u003cem\u003ei\u003c/em\u003e\u003c/sub\u003e; All slices are processed by Mapper function, and the key pairs with the same key value are summed by Reduce function. The key values output by the Reduce function are sorted in order of frequency from high to low. Finally, FList is obtained according to the minimum support filtering.\u003c/p\u003e\n \u003cp\u003eStep 3, Grouping for Frequent 1- item sets: All frequent items in FList are evenly divided into GROUP G, in which each group has a unique group ID- Gid. Frequent items and their corresponding group numbers constitute frequent item table group GList and are assigned into all slices.\u003c/p\u003e\n \u003cp\u003eStep 4, design parallel FP-growth: This step is the core step of the parallel algorithm. Partial FP-growth mining is operated through a MapReduce. Job, where the Mapper function is defined by key pairs of \u0026lt;\u0026thinsp;The key, value\u0026thinsp;=\u0026thinsp;\u003cem\u003eT\u003c/em\u003e\u003csub\u003e\u003cem\u003ei\u003c/em\u003e\u003c/sub\u003e\u0026gt; to read each slice. Scan all frequent items \u003cem\u003eA\u003c/em\u003e\u003csub\u003e\u003cem\u003ei\u003c/em\u003e\u003c/sub\u003e in transaction \u003cem\u003eT\u003c/em\u003e\u003csub\u003e\u003cem\u003ei\u003c/em\u003e\u003c/sub\u003e, find the corresponding group ID identifier Gid in the corresponding frequent item table group GList, and output it with the key value. The output of Mapper function is read through the Reduce function corresponding to different Key values, local FP-tree is built, and local FP-growth mining is carried out for frequent items corresponding to each Gid in a recursive way. The frequent item sets obtained from mining are saved in a heap in order from large to small, and the top \u003cem\u003eK\u003c/em\u003e frequent item sets with the highest support are finally returned.\u003c/p\u003e\n \u003cp\u003eStep 5. Aggregation: Read the output of Step 4, combine the same key-value pairs via the Mapper function, and then use the Reduce function to calculate and output the frequent item sets of the top \u003cem\u003eK\u003c/em\u003e items with the highest global support and confidence.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv class=\"Section2\" id=\"Sec8\"\u003e\n \u003ch2\u003e3.3 Implementation of optimized parallel FP-growth algorithm with load balancing constraints\u003c/h2\u003e\n \u003cdiv class=\"Section3\" id=\"Sec9\"\u003e\n \u003ch2\u003e3.3.1 The load balancing definition and grouping strategy\u003c/h2\u003e\n \u003cp\u003eLoad balancing refers to the balancing of loads (work tasks) and spreading them across multiple units of operation so that work tasks can be completed cooperatively. The load balancing defined in this research refers to the balanced grouping of frequent 1-itemsets to achieve the relative balance of the running load for each slice, so as to improve the efficiency of data mining. The core strategy of load balancing mainly includes two steps: The first step is to build a load model and recursively mine the total workload of FP-tree on the projection database for each frequent data item. The second step is to divide all frequent items into \u003cem\u003eG\u003c/em\u003e groups by balanced grouping strategy and divide transaction \u003cem\u003eT\u003c/em\u003e into \u003cem\u003eG\u003c/em\u003e groups to ensure the load balancing of each node. Tasks are completed by corresponding nodes of various groups, and FP-tree is recursively mined for frequent items, so as to improve the processing capability of the system.\u003c/p\u003e\n \u003cp\u003e(1) Establishment of load model\u003c/p\u003e\n \u003cp\u003eFor the FP-growth algorithm, it is not realistic to accurately calculate the workload of each frequent item, we can only reasonably estimate the workload. The workload of recursively mining the condition FP-tree of each frequent item is equal to the times that the condition FP-tree of the frequent item recursively calls FP-growth. The length of the longest frequent path of each frequent item in the conditional FP-tree is equivalent to the location of the frequent item in the FList.\u003c/p\u003e\n \u003cp\u003eIn the conditional FP-tree, the number of times that the same frequent item recurrently calls FP-growth has an exponential relationship with the length of its longest frequent path [\u003cspan class=\"CitationRef\"\u003e18\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e20\u003c/span\u003e], and the load model formula (1) can be obtained, where \u003cem\u003eF\u003c/em\u003e\u003csub\u003e\u003cem\u003ei\u003c/em\u003e\u003c/sub\u003e is the load estimated value of each node, and P (\u003cem\u003eI\u003c/em\u003e, FList) represents the position of the frequent item in FList of frequent 1-itemset:\u003c/p\u003e\n \u003cdiv class=\"Equation\" id=\"Equ1\"\u003e\n \u003cdiv class=\"mathdisplay\" id=\"FileID_Equ1\" name=\"EquationSource\"\u003eF\u003csub\u003ei\u003c/sub\u003e = log P(\u003csub\u003ei, FList\u003c/sub\u003e)\u0026nbsp; (1)\u003c/div\u003e\n \u003c/div\u003e\n \u003cp\u003e(2) Balanced grouping\u003c/p\u003e\n \u003cp\u003eAccording to formula (1), in frequent 1-itemset FList, the higher the support count of frequent items is, the higher its position is, and the larger its load value is. Therefore, the load value of frequent items in FList forms an increasing trend. Since it is difficult to achieve a globally optimal grouping, this research uses a greedy strategy to group frequent items. Assuming that the quantity of item groups is \u003cem\u003eG\u003c/em\u003e, FList is grouped in order from the back to the front. First, the first \u003cem\u003eG\u003c/em\u003e frequent items are put into \u003cem\u003eG\u003c/em\u003e groups, and then the frequent items to be grouped are put into the group with the smallest total load in sequence. At the same time, the load value of the frequent item is accumulated to the total load of the group. Repeat the steps until all the frequent items in FList are allocated.\u003c/p\u003e\n \u003c/div\u003e\n \u003cdiv class=\"Section3\" id=\"Sec10\"\u003e\n \u003ch2\u003e3.3.2 Implementation of load balancing algorithm\u003c/h2\u003e\n \u003cp\u003eThe process of the improved load-balancing parallel FP-growth algorithm is the same as that of the above parallel FP-growth algorithm. Only the frequent 1-itemset grouping in the third step is improved. The pseudo-code and flow chart of the algorithm are shown in Table \u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e and Fig. \u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e respectively.\u0026nbsp;\u003c/p\u003e\n \u003ctable border=\"1\" id=\"Tab2\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eCode implementation of load balancing algorithm\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eLoad balancing algorithm pseudocode:\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003egenerateGList Function:\u003c/p\u003e\n \u003cp\u003e1. void generateGList(List FList, int G){\u003c/p\u003e\n \u003cp\u003e2. HashTable GList;\u003c/p\u003e\n \u003cp\u003e3. Heap minHP;\u003c/p\u003e\n \u003cp\u003e4. groupsNum \u0026larr; G;\u003c/p\u003e\n \u003cp\u003e5. Gid \u0026larr; 0;\u003c/p\u003e\n \u003cp\u003e6. if FList.length()\u0026thinsp;\u0026lt;\u0026thinsp;groupsNum then\u003c/p\u003e\n \u003cp\u003e7. foreach item a\u003csub\u003ej\u003c/sub\u003e in FList do\u003c/p\u003e\n \u003cp\u003e8. GList.put(a\u003csub\u003ej\u003c/sub\u003e, Gid);\u003c/p\u003e\n \u003cp\u003e9. Gid \u0026larr; Gid\u0026thinsp;+\u0026thinsp;1;\u003c/p\u003e\n \u003cp\u003e10. end\u003c/p\u003e\n \u003cp\u003e11. else\u003c/p\u003e\n \u003cp\u003e12. for j\u0026thinsp;=\u0026thinsp;0 to groupsNum do\u003c/p\u003e\n \u003cp\u003e13. F(item[j]); // Load estimation\u003c/p\u003e\n \u003cp\u003e14. minHP.add(item[j]);\u003c/p\u003e\n \u003cp\u003e15. minHP.adjust();\u003c/p\u003e\n \u003cp\u003e16. end\u003c/p\u003e\n \u003cp\u003e17. for j\u0026thinsp;=\u0026thinsp;groupsNum to FList.length() do\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e18. F(item[j]); // Load estimation\u003c/p\u003e\n \u003cp\u003e19. minHP[0].weight \u0026larr; minHP[0].weight\u0026thinsp;+\u0026thinsp;item[j].weight;\u003c/p\u003e\n \u003cp\u003e20. GList.put(item[j], minHP[0].group);\u003c/p\u003e\n \u003cp\u003e21. minHP.adjust();\u003c/p\u003e\n \u003cp\u003e22. end\u003c/p\u003e\n \u003cp\u003e23. end\u003c/p\u003e\n \u003cp\u003e24. }\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003cp\u003e\u003c/p\u003e\n \u003c/div\u003e\n\u003c/div\u003e\n\u003cdiv class=\"Section2\" id=\"Sec11\"\u003e\n \u003ch2\u003e3.4 Construction of experimental environment\u003c/h2\u003e\n \u003cp\u003eIn this research, there is a large amount of historical crash data on freeways, so the complete distribution mode is selected in the experiment to build a Hadoop distributed processing system. A cluster is built with three computers installed with Linux system. The cluster has three nodes in total, including one master node and two slave nodes. The hardware and software configuration of each computer is shown in Table \u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e.\u003c/p\u003e\n \u003cdiv class=\"gridtable\"\u003e\n \u003ctable border=\"1\" id=\"Tab3\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eVirtual machine configuration\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eThe hardware configuration\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eThe software configuration\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eCPU: Intel(R) Xeon(R) CPU E5-2660 v2 @ 2.2GHz\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eLinux: Ubantu 18.04.5\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eMemory: 32GB\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eJava version: JDK-8U161-linux-x64\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eHard Disk: 360GB\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eHadoop version: Hadoop-2.7.4\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eGPU: AMD_Radeon R7 350\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eIntegrated development environment: Eclipse\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n \u003cp\u003eThe process of building a Hadoop framework cluster is as follows:\u003c/p\u003e\n \u003cp\u003eStep 1: Install Linux Ubantu 18.04.5 on the three computers respectively, change the host name and network configuration to \u0026quot;master\u0026quot;, \u0026quot;slave1\u0026quot;, and \u0026quot;slave2\u0026quot;.\u003c/p\u003e\n \u003cp\u003eStep 2: Modify the hosts file on the computer, set the public and private keys to implement SSH login without password between computers.\u003c/p\u003e\n \u003cp\u003eStep 3: Upload and install JDK and configure environment variables for these computers.\u003c/p\u003e\n \u003cp\u003eStep 4: Upload and install Hadoop, perform operations on environment variables and configuration files in the conf file of the computer, and format HDFS.\u003c/p\u003e\n \u003cp\u003eStep 5: Start the Hadoop cluster and run the JPS command to view all processes. The Hadoop cluster process starts as shown in Fig. \u003cspan class=\"InternalRef\"\u003e4\u003c/span\u003e.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv class=\"Section2\" id=\"Sec12\"\u003e\n \u003ch2\u003e3.5 Comparison of improved algorithm efficiency\u003c/h2\u003e\n \u003cp\u003eIn order to verify the rationality of the algorithm designed in this research, the serial FP-growth algorithm, parallel FP-growth algorithm, the parallel FP-growth algorithm with load balancing constraints are respectively run under the Hadoop cluster. In this experiment, the computational efficiency of the algorithm is compared and analyzed from the perspectives of data volume and minimum support.\u003c/p\u003e\n \u003cp\u003e(1) Sensitivity analysis for data volume on algorithm efficiency\u003c/p\u003e\n \u003cp\u003eVarious records are randomly selected from the data set to constitute the data set, and the four virtual processed data sets used in the experiment include 100,000, 500,000, 1,000,000 and 1,500,000 records respectively. The results of running the three algorithms with a minimum support of 0.5 are shown in Fig. \u003cspan class=\"InternalRef\"\u003e5\u003c/span\u003e.\u003c/p\u003e\n \u003cp\u003eFigure \u003cspan class=\"InternalRef\"\u003e5\u003c/span\u003e indicates that, when the experimental data set is small, the running time difference of the three algorithms is very small. The possible explanation is that the two parallel FP-growth algorithms need extra time to start the cluster, which fails to take advantage of the parallel algorithm. With the increase of the data set, the operation time of the two parallel FP-growth algorithms is greatly reduced compared with the serial algorithm, and the performance advantage of the algorithms is more obvious with a larger number of records. Compared with the parallel FP-growth algorithm without load balancing, the operation time of the parallel FP-growth algorithm with load balancing based on balanced grouping method can be further reduced, which is more suitable for mining large data sets. The experimental results in Fig. \u003cspan class=\"InternalRef\"\u003e5\u003c/span\u003e can prove that the optimized load-balancing parallel FP-growth algorithm can not only solve the problems such as large memory consumption and long running time when the traditional serial algorithm performs big data mining, but also further shorten the data mining time and improve the efficiency of the algorithm.\u003c/p\u003e\n \u003cp\u003e(2) Sensitivity analysis for the minimum on algorithm efficiency\u003c/p\u003e\n \u003cp\u003eAccording to the previous analysis of association rules and algorithms in this research, the setting of minimum support degree has a great impact on the operation efficiency of the algorithm. The smaller the minimum support, the greater the number of frequent items, and the longer the operation time of the algorithm. In this experiment, the data set with 500,000 records in experiment (1) is selected, and three algorithms are adopted for data mining under the condition that the minimum support is set at 0.1\u0026ndash;0.7. The calculation time comparison is shown in Fig. \u003cspan class=\"InternalRef\"\u003e6\u003c/span\u003e.\u003c/p\u003e\n \u003cp\u003eFigure \u003cspan class=\"InternalRef\"\u003e6\u003c/span\u003e indicates that, when the minimum support is high, such as 0.6 and 0.7, the number of frequent items in the data set is small, and the operation time of the three algorithms is short. With the decrease of support, the number of frequent items in the data set increases, and the operation time of the algorithms begins to rise. When the minimum support is lower than a certain degree, the serial FP-growth algorithm has the problem of memory heap overflow. At this time, the algorithm will automatically report an error and stop running. However, the parallel FP-growth algorithm overcomes this shortcoming, and can still run under a lower support threshold, its running time is less than the serial algorithm. According to the overall running time of the algorithm, the running efficiency of the optimized parallel FP-growth algorithm is higher than that of the original one, its advantages become more prominent as the support threshold decreases.\u003c/p\u003e\n \u003cp\u003eThe above experimental results show that the optimized parallel FP-growth algorithm based on Hadoop proposed in this research can perform efficient data mining in the case of low support. The operating efficiency of the improved algorithm is obviously better than that of the serial algorithm and the original parallel FP-growth algorithm, and it can be well applied to the subsequent data mining for freeway traffic crashes in this research.\u003c/p\u003e\n\u003c/div\u003e"},{"header":"4. Cause Analysis Of Freeway Traffic Crashes Based On Improved Fp-growth Algorithm","content":"\u003cp\u003eDuring the process of FP- Growth association rule mining algorithm, parameter threshold Settings is closely related to the efficiency and quality of mining, considering the coupling mechanism between precursors related to serious casualty as high practical significance with the traffic safety, consequence with the values of \u0026quot;death or serious injury\u0026quot; in the original data set actually has a lower percentage actually. If the minimum support threshold is set too high, death or serious injury both attribute variables may be cut off in the pruning process. Therefore, in this study, the minimum support threshold will be set based on the occurrence frequency of the casualty attribute and its related attributes, as far as possible to efficiently and accurately mining association rules effectively.\u003c/p\u003e\n\u003cdiv class=\"Section2\" id=\"Sec13\"\u003e\n \u003ch2\u003e4.1 Mining Results and analysis\u003c/h2\u003e\n \u003cp\u003eMulti-dimensional interaction association rule mining refers to that the directed association mining in the form of \u0026quot;LHS\u0026thinsp;=\u0026thinsp;\u0026gt;\u0026thinsp;RHS\u0026quot; is the association rule of constraint condition. After a lot of tests, set the minimum support threshold as 0.05 (in case ignoring the traffic crashes with severe consequences but lower frequency), minimum confidence threshold is set as 0.4, the lift degree threshold to set as 1. The variables in the dimension of \u0026quot;Driver, vehicle, road, environment, time\u0026quot; are constrained in LHS, the crash dimension variables are constrained in the RHS. Then the association rules with high support and high confidence obtained from mining are extracted respectively, as shown in Tables \u003cspan class=\"InternalRef\"\u003e4\u003c/span\u003e and \u003cspan class=\"InternalRef\"\u003e5\u003c/span\u003e.\u003c/p\u003e\n \u003cdiv class=\"gridtable\"\u003eTab.4 Mining results of association rules (In descending order of support)\u003c/div\u003e\n \u003cdiv align=\"center\"\u003e\n \u003ctable border=\"1\" cellpadding=\"0\" cellspacing=\"0\" width=\"0\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" width=\"69.3631669535284%\"\u003e\n \u003cp\u003eRules (LHS=\u0026gt;RHS)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"10.671256454388985%\"\u003e\n \u003cp\u003eSup\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"10.154905335628227%\"\u003e\n \u003cp\u003eConf\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"9.81067125645439%\"\u003e\n \u003cp\u003eLift\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"69.3631669535284%\"\u003e\n \u003cp\u003e{Speeding=N1,Drowsy=Q1,Vehicle Condition=AG1,\u003c/p\u003e\n \u003cp\u003eUnlicensed=P1,Distracted=O1}{Number of Motor Vehicles Involved=AD2,Ejection Status=W1}\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.671256454388985%\"\u003e\n \u003cp\u003e0.529\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.154905335628227%\"\u003e\n \u003cp\u003e0.901\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"9.81067125645439%\"\u003e\n \u003cp\u003e1.013\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"69.3631669535284%\"\u003e\n \u003cp\u003e{Speeding=N1,Age=J3,Drowsy=Q1,Vehicle Condition=AG1,\u003c/p\u003e\n \u003cp\u003eUnlicensed=P1,Distracted=O1}{Number of Motor Vehicles Involved=AD2}\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.671256454388985%\"\u003e\n \u003cp\u003e0.522\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.154905335628227%\"\u003e\n \u003cp\u003e0.919\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"9.81067125645439%\"\u003e\n \u003cp\u003e1.074\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"69.3631669535284%\"\u003e\n \u003cp\u003e{Speeding=N1,Vehicle Condition=AG1,Roadway Surface\u0026nbsp;\u003c/p\u003e\n \u003cp\u003eCondition=I1,Weather=Z1,LightingCondition=AA3}{Ejection Status=W1,Injury Type=AH1}\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.671256454388985%\"\u003e\n \u003cp\u003e0.507\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.154905335628227%\"\u003e\n \u003cp\u003e0.782\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"9.81067125645439%\"\u003e\n \u003cp\u003e1.016\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"69.3631669535284%\"\u003e\n \u003cp\u003e{Gender=K2,Speeding=N1,Drowsy=Q1,Roadway Surface Condition=I1}{Ejection Status=W1,Injury Type=AH1}\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.671256454388985%\"\u003e\n \u003cp\u003e0.489\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.154905335628227%\"\u003e\n \u003cp\u003e0.891\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"9.81067125645439%\"\u003e\n \u003cp\u003e1.006\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"69.3631669535284%\"\u003e\n \u003cp\u003e{Speeding=N1,Restraining System Type=X1,Lighting Condition\u003c/p\u003e\n \u003cp\u003e=AA3}{Ejection Status=W1,Number of Motor Vehicles Involved=AD2}\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.671256454388985%\"\u003e\n \u003cp\u003e0.449\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.154905335628227%\"\u003e\n \u003cp\u003e0.827\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"9.81067125645439%\"\u003e\n \u003cp\u003e1.127\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"69.3631669535284%\"\u003e\n \u003cp\u003e{Speeding=N1,Drowsy=Q1,VehicleCondition=AG1,Roadway Surface Condition=I1}{Ejection Status=W1,Injury Type=AH1}\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.671256454388985%\"\u003e\n \u003cp\u003e0.377\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.154905335628227%\"\u003e\n \u003cp\u003e0.907\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"9.81067125645439%\"\u003e\n \u003cp\u003e1.023\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"69.3631669535284%\"\u003e\n \u003cp\u003e{Age=J2,Speeding=N1,Drowsy=Q1,Vehicle Condition=AG1,\u003c/p\u003e\n \u003cp\u003eSobriety Level=L1,Weather=Z1}{Injury Type=AH1}\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.671256454388985%\"\u003e\n \u003cp\u003e0.368\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.154905335628227%\"\u003e\n \u003cp\u003e0.877\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"9.81067125645439%\"\u003e\n \u003cp\u003e1.077\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"69.3631669535284%\"\u003e\n \u003cp\u003e{Gender=K1,Speeding=N1,Drowsy=Q1,Vehicle Condition\u003c/p\u003e\n \u003cp\u003e=AG1,Unlicensed Driver=P1,Roadway Surface Condition}\u003c/p\u003e\n \u003cp\u003e{Ejection Status=W1,Injury Type=AH1}\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.671256454388985%\"\u003e\n \u003cp\u003e0.329\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.154905335628227%\"\u003e\n \u003cp\u003e0.659\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"9.81067125645439%\"\u003e\n \u003cp\u003e1.109\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"69.3631669535284%\"\u003e\n \u003cp\u003e{Speeding=N1,Drowsy=Q1,Vehicle Condition=AG1,Sobriety Level=L1}{Ejection Status=W1,Injury Type=AH1}\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.671256454388985%\"\u003e\n \u003cp\u003e0.322\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.154905335628227%\"\u003e\n \u003cp\u003e0.792\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"9.81067125645439%\"\u003e\n \u003cp\u003e1.074\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"69.3631669535284%\"\u003e\n \u003cp\u003e{Quarter Number=G1,Speeding=N1,Vehicle Condition=AG1,\u003c/p\u003e\n \u003cp\u003eLighting Condition=AA3}{Towed Indicator=D1}\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.671256454388985%\"\u003e\n \u003cp\u003e0.317\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.154905335628227%\"\u003e\n \u003cp\u003e0.843\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"9.81067125645439%\"\u003e\n \u003cp\u003e1.008\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"69.3631669535284%\"\u003e\n \u003cp\u003e{Quarter Number=G3,Vehicle Condition=AG1,Unlicensed Driver=P1,Lighting Condition=AA3}{Injury Type=AH1}\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.671256454388985%\"\u003e\n \u003cp\u003e0.310\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.154905335628227%\"\u003e\n \u003cp\u003e0.891\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"9.81067125645439%\"\u003e\n \u003cp\u003e1.015\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"69.3631669535284%\"\u003e\n \u003cp\u003e{Speeding=N1,Drowsy=Q1,Roadway Characteristic=B1,\u003c/p\u003e\n \u003cp\u003eVehicle Action=AF2}{Collision Severity=AE1}\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.671256454388985%\"\u003e\n \u003cp\u003e0.303\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.154905335628227%\"\u003e\n \u003cp\u003e0.922\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"9.81067125645439%\"\u003e\n \u003cp\u003e1.018\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"69.3631669535284%\"\u003e\n \u003cp\u003e{Speeding=N1,Drowsy=Q1,Vehicle Condition=AG1,Unlicensed Driver=P1}{Number of Motor Vehicles Involved=AD2}\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.671256454388985%\"\u003e\n \u003cp\u003e0.293\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.154905335628227%\"\u003e\n \u003cp\u003e0.864\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"9.81067125645439%\"\u003e\n \u003cp\u003e1.152\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n \u003cp\u003eThrough the analysis of the results in Table \u003cspan class=\"InternalRef\"\u003e4\u003c/span\u003e, the following conclusions can be drawn: Traffic crashes in the study area are mostly caused by passenger cars, most of which involves male drivers. Minor crashes those do not cause casualties account for the majority, and most of them occur on urban or interstate freeways. Most of the crashes involve only 1\u0026ndash;2 vehicles. Traffic crashes those do not result in injuries are probably not caused by drivers\u0026apos; driving violations. A possible explanation for the higher incidence of minor crashes those do not result in injuries is that bad weather is less common in summer and roads\u0026apos; conditions are basically dry. Non-fatal crashes usually do not cause airbags ejection. Most of the traffic crashes those occur when the vehicle is going straight may result in minor traffic crashes with no casualties.\u0026nbsp;\u003c/p\u003e\n \u003cp\u003eTab.5 Mining results of association rules under (In descending order of confidence)\u003c/p\u003e\n \u003cdiv align=\"center\"\u003e\n \u003ctable border=\"1\" cellpadding=\"0\" cellspacing=\"0\" width=\"0\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" width=\"69.19104991394148%\"\u003e\n \u003cp\u003eRules (LHS=\u0026gt;RHS)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"10.154905335628227%\"\u003e\n \u003cp\u003eConf\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"10.154905335628227%\"\u003e\n \u003cp\u003eSup\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" width=\"10.499139414802066%\"\u003e\n \u003cp\u003eLift\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"69.19104991394148%\"\u003e\n \u003cp\u003e{Age=J2,Weather=Z2,Roadway Surface Condition=I2}\u003c/p\u003e\n \u003cp\u003e{Collision Severity=AE2}\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.154905335628227%\"\u003e\n \u003cp\u003e0.968\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.154905335628227%\"\u003e\n \u003cp\u003e0.158\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.499139414802066%\"\u003e\n \u003cp\u003e27.970\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"69.19104991394148%\"\u003e\n \u003cp\u003e{Day Of Week=F1,Quarter Number=G1,Hour=H1,Roadway Surface Condition=I3}{Number of Motor Vehicles Involved=AD3}\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.154905335628227%\"\u003e\n \u003cp\u003e0.961\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.154905335628227%\"\u003e\n \u003cp\u003e0.057\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.499139414802066%\"\u003e\n \u003cp\u003e37.801\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"69.19104991394148%\"\u003e\n \u003cp\u003e{Age=J1,Speeding =N2,Unlicensed=P2,Lighting Condition=AA2}\u003c/p\u003e\n \u003cp\u003e{Collision Severity=AE3}\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.154905335628227%\"\u003e\n \u003cp\u003e0.954\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.154905335628227%\"\u003e\n \u003cp\u003e0.071\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.499139414802066%\"\u003e\n \u003cp\u003e86.598\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"69.19104991394148%\"\u003e\n \u003cp\u003e{Gender=K2,Sobriety Level=L3,Roadway Surface Condition=I7}\u003c/p\u003e\n \u003cp\u003e{Collision Severity=AE3}\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.154905335628227%\"\u003e\n \u003cp\u003e0.939\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.154905335628227%\"\u003e\n \u003cp\u003e0.069\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.499139414802066%\"\u003e\n \u003cp\u003e78.723\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"69.19104991394148%\"\u003e\n \u003cp\u003e{Age=J2,Day Of Week=F1,Hour=H1,Roadway Surface Condition\u003c/p\u003e\n \u003cp\u003e=I3}{Collision Severity=AE2}\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.154905335628227%\"\u003e\n \u003cp\u003e0.937\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.154905335628227%\"\u003e\n \u003cp\u003e0.125\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.499139414802066%\"\u003e\n \u003cp\u003e18.652\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"69.19104991394148%\"\u003e\n \u003cp\u003e{Vehicle Type=T2,Vehicle Condition=AG2,Distracted=O1,Speeding =N2}{Towed Indicator=D2,Collision Severity=AE3}\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.154905335628227%\"\u003e\n \u003cp\u003e0.930\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.154905335628227%\"\u003e\n \u003cp\u003e0.271\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.499139414802066%\"\u003e\n \u003cp\u003e3.672\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"69.19104991394148%\"\u003e\n \u003cp\u003e{Day Of Week=F2,Quarter Number=G4,Hour=H2}{Injury Type\u003c/p\u003e\n \u003cp\u003e=AH1}\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.154905335628227%\"\u003e\n \u003cp\u003e0.927\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.154905335628227%\"\u003e\n \u003cp\u003e0.193\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.499139414802066%\"\u003e\n \u003cp\u003e1.162\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"69.19104991394148%\"\u003e\n \u003cp\u003e{Speeding=N1,Drowsy=Q1,Roadway Characteristic=B1,Vehicle Action=AF2}{Number of Pedestrians Involved=AC1,Collision Severity=AE1}\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.154905335628227%\"\u003e\n \u003cp\u003e0.922\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.154905335628227%\"\u003e\n \u003cp\u003e0.303\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.499139414802066%\"\u003e\n \u003cp\u003e1.018\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"69.19104991394148%\"\u003e\n \u003cp\u003e{Speeding=N1,Age=J3,Drowsy=Q1,Vehicle Condition=AG1,\u003c/p\u003e\n \u003cp\u003eUnlicensed=P1,Distracted=O1}{Number of Motor Vehicles Involved=AD2}\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.154905335628227%\"\u003e\n \u003cp\u003e0.919\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.154905335628227%\"\u003e\n \u003cp\u003e0.522\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.499139414802066%\"\u003e\n \u003cp\u003e1.074\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"69.19104991394148%\"\u003e\n \u003cp\u003e{Age=J4,Drowsy=Q2,Vehicle Type=T1}{Number of Motor Vehicles Involved=AD2}\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.154905335628227%\"\u003e\n \u003cp\u003e0.903\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.154905335628227%\"\u003e\n \u003cp\u003e0.081\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"10.499139414802066%\"\u003e\n \u003cp\u003e63.190\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n \u003cp\u003eThe following conclusions can be drawn from the results in Table \u003cspan class=\"InternalRef\"\u003e5\u003c/span\u003e: When the driver has dangerous driving behaviors such as overspeed driving, fatigue driving, distracted driving, unlicensed driving or drunk driving, and the road surface is wet or snowy, the traffic crashes easily tend to cause casualties. When the driver\u0026apos;s dangerous driving behavior and slope road appear at the same time, it is also very easy to cause serious consequences of casualties. The combination of unsafe driving and bad weather often leads to serious injury or death. Vehicle failure and bad weather at the same time tend to cause casualties. Among the traffic crash, drivers under the age of 25 are often associated with speeding, and drivers over the age of 65 are often associated with distracted driving. Most of the association rules extracted above include driver factors, so it indicates that human is the main operator in traffic and the key factor that determines whether a crash occurs or not.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv class=\"Section2\" id=\"Sec14\"\u003e\n \u003ch2\u003e4.2 Suggestions for improving freeway traffic safety based on association rule mining\u003c/h2\u003e\n \u003cp\u003eThe occurrence frequency of association rules with high confidence is typically less than that of association rules with high support. The LHS occurrence of this kind of association rules can easily cause the occurrence of the RHS, but if any field variables in LHS changes, the association rule may no longer hold. Therefore, the control principle for these kind of association rules is to control from LHS and RHS simultaneously. The following suggestions are made for low-support high-confidence association rules that cause serious injuries:\u003c/p\u003e\n \u003cp\u003e(1) Rain or snow weather\u0026thinsp;+\u0026thinsp;large van\u0026thinsp;+\u0026thinsp;curve with slopes\u003c/p\u003e\n \u003cp\u003eRain and snow may interfere with the driver\u0026apos;s vision and visibility, and on the one hand, it may be accompanied by ice or wet on the road, both of which will lead to an increase in the braking distance of the vehicle. Large trucks are relatively heavy in their own inertia, and in the rain and snow, they usually have the risk of rushing out of the road or crashing into nearby vehicles when they are in a crash-prone situation, both of which have serious consequences. In this case, we can avoid the occurrence of RHS by controlling the occurrence of LHS: the first is to optimize the design of the mark line in the curve with slope, set up the speeding capture equipment, and control the speed on the dangerous road. Secondly, we should increase the inspection for large truck overload, and set up the emergency safe lane on the dangerous road such as the continuous slope, and the surface of the road should be made of small stones, so that the vehicles in the emergency can slow or stop immediately, thereby avoiding the occurrence of the crash or reducing the severity. In addition, the safety guard should be installed to prevent the vehicle from sliding out of the road when it collided in a detour, the monitoring for road environment should be improved, severe weather that threatens road traffic safety should be forecasted timely, road cleaning and maintenance during rain and snow should be should be taken into account, such as by sprinkling industrial salt to melt and clear the snow on the road.\u003c/p\u003e\n \u003cp\u003e(2) Tire failure\u0026thinsp;+\u0026thinsp;passenger car\u0026thinsp;+\u0026thinsp;peak hours\u003c/p\u003e\n \u003cp\u003eTraffic crashes of passenger cars with tire faults during peak hours usually lead to injuries and most of them involve multiple motor vehicles. On the one hand, the traffic volume during peak hours is large and the driving distance between vehicles is small. On the other hand, the passenger cars with tire faults may be uncontrolled and easily collide with surrounding vehicles.After the collision, the surrounding vehicles are prone to serial rear-end collision because of a small distance between vehicles, consequently resulting in injuries. For this kind of situation, related department should guide the vehicle owner regularly check, increase the frequency of the motor vehicle compulsory inspection or strictly control the test process, minimize the vehicle potential safety hazard as much as possible. In addition, the traffic volume during peak hour is large, which is prone to congestion, policies such as encouraging off-peak commuting can be adopted to realize the time separation of traffic flow.\u003c/p\u003e\n \u003cp\u003e(3) Main line\u0026thinsp;+\u0026thinsp;straight road with slope\u0026thinsp;+\u0026thinsp;dust or sand blowing weather\u003c/p\u003e\n \u003cp\u003eThe majority of crashes on the Main line result in injuries due to dust or sand-blowing weather, which can affect drivers\u0026apos; vision and sight. Druing the periods of design, construction and management traffic safety should be the first consideration. For the continuous steep slope, steep curve, long down ramp and long straight road and other dangerous roads, traffic safety risks should be eliminated timely. Early warning should be given to bad weather such as dust or sand blowing, and monitoring of road environment should be strengthened.\u003c/p\u003e\n \u003cp\u003e(4) Drivers over 65\u0026thinsp;+\u0026thinsp;distracted driving\u0026thinsp;+\u0026thinsp;flat road\u003c/p\u003e\n \u003cp\u003eMost of the 65\u0026thinsp;+\u0026thinsp;aged drivers have some impairment of vision and hearing, most crashes involving distracted driving on straight roads involve death or injury. For this situation, elderly drivers should be subjected to an annual physical examination, and their driving licenses should be revoked if they do not meet the conditions. For drivers with distracted driving, some stimuli can be used to make them keep their attention in high concentration, color road signs may work to stimulate their vision, speed bumps can be set to stimulate their senses, and warning lights or slogans can be installed to attract their attention. In road design, long straight lines should be avoided. If unavoidable, raised warning signs must be set up on the road surface every certain distance to prevent the driver from appearing shallow sleep due to the monotonous driving process.\u003c/p\u003e\n \u003cp\u003eThe mining results with high support are mostly minor traffic crashes without casualties, indicating simultaneous occurrences of a certain attribute combination account for a high proportion in the total number of items. The safety control principle for such crashes is to avoid RHS by controlling LHS.\u003c/p\u003e\n\u003c/div\u003e"},{"header":"5. Conclusions","content":"\u003cp\u003eThis research proposes a parallel FP-growth association rule mining algorithm considering load balancing, compares the operational efficiency with serial FP-growth algorithm and conventional parallel FP-growth algorithm using various data set volumes and support threshold degree respectively. Based on the proposed association rule mining algorithm considering load balancing, the freeway traffic crash data applied in this research is analyzed with the consideration of directional constraints under multi-dimension and multi-level perspective, the results of association rule mining with high support and high confidence (including low support) are obtained. The specific conclusions of this research are as follows:\u003c/p\u003e \u003cp\u003e(1) Compared with serial FP-growth algorithm or traditional parallel FP-growth algorithm, the proposed parallel FP-growth association rule mining algorithm considering load balancing has more advantages in computational efficiency, and it can reduce the possibility of leading to memory heap overflow when processing massive data or low support threshold setting work. It is more suitable for mining large traffic crash data sets.\u003c/p\u003e \u003cp\u003e(2) Serious consequences data pieces in crash data sets may be assigned a low degree of support (which may have a high confidence degree) when the association rule mining algorithm is applied, these crashes are easy to be ignored. The algorithm proposed in this research has great advantages in mining association rules with low support threshold. It can overcome the shortcomings of the mining work via traditional association rule mining algorithm towards the data set including serious traffic crash consequence, as well as better reveal the coupling mechanism among the factors related serious traffic crash.\u003c/p\u003e \u003cp\u003e(3) Association rules with high confidence indicate that the association rules have high conditional probability: for the situation combinations with high confidence appear in the LHS, attention should be paid, so as to prevent the serious consequences of traffic crashes. An association rule with high support indicates the high occurrence frequency of the association rule. For an association rule with high occurrence frequency but minor crash consequence, the LHS should be controlled to prevent the consequences in RHS.\u003c/p\u003e \u003cp\u003e(4) Traffic safety is an important element of sustainable transportation, and association rule mining algorithm is an effective method in the field of fuzzy decision. The approach proposed in this research can provide theoretical guidance for fuzzy identification of sustainable transportation.\u003c/p\u003e \u003cp\u003eDue to the limitation of experimental conditions, this research adopts three computers to build a fully distributed experimental mode of Hadoop. This method can realize the database association rule mining, but it is not enough to analyze the impact of cluster node number on operation efficiency. In the future research, the experimental environment should be improved to get more accurate experimental data.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eCompliance with Ethical standards:\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eHumans and animals are not involved in this research work.\u003c/p\u003e\n\u003cp\u003eWe used our own data.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis work was supported by China Postdoctoral Science Foundation [Grant NO. 2021M700333].\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting Interests\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eWorld Health Organization (2018) Global status report on road safety 2018: Summary[R]. World Health Organization\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHamed MM, Al-Eideh BM (2020) An exploratory analysis of traffic accidents and vehicle ownership decisions using a random parameters logit model with heterogeneity in means[J]. Analytic methods in accident research 25:100116\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eYu Q (2013) Causes and prevention measures of secondary rear-end accidents in the rescue of highway traffic accidents[J]. Procedia Eng 52:571\u0026ndash;577\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKwayu KM, Kwigizile V, Lee K et al (2020) Discovering latent themes in traffic fatal crash narratives using text mining analytics and network topology[J], vol 150. Accident Analysis \u0026amp; Prevention, p 105899\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSun J, Li T, Li F et al (2016) Analysis of safety factors for urban expressways considering the effect of congestion in Shanghai, China[J], vol 95. Accident Analysis \u0026amp; Prevention, pp 503\u0026ndash;511\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eYang Y (2020) Research on the Method of Freeway Crash Risk Identification and Comprehensive Traffic Safety Evaluation Considering the Regional Type Difference[D]. Beijing Jiaotong University\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eYang Y, Yuan ZZ, Sun DY et al (2019) Analysis of the factors influencing highway crash risk in different regional types based on improved Apriori algorithm. J Adv Transp Stud 49:165\u0026ndash;178\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eYang Y, Yuan Z, Chen J et al (2017) Assessment of osculating value method based on entropy weight to transportation energy conservation and emission reduction. J Environ Eng Manage J 16:2413\u0026ndash;2424\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSingh S, Garg R, Mishra PK (2018) Performance Optimization of MapReduce-based Apriori Algorithm on Hadoop Cluster[J].Computers \u0026amp; Electrical Engineering, :348\u0026ndash;364\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eThangaraj DVSJJ, Khanna MR \"An Improved Early Detection Method of Autism Spectrum Anarchy using Euclidean Method,\" 2020 Fourth International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC), 2020, pp.\u0026nbsp;1173\u0026ndash;1178, doi: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/I-SMAC49090.2020.9243361\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eYu L, Du B, Hu X et al (2021) Deep spatio-temporal graph convolutional network for traffic accident prediction[J]. Neurocomputing 423:135\u0026ndash;147\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJiang F, Yuen KKR, Lee EWM (2020) Analysis of motorcycle accidents using association rule mining-based framework with parameter optimization and GIS technology[J]. J Saf Res 75:292\u0026ndash;309\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHuang Y, Huang J, Liu C et al (2020) PFPMine: A parallel approach for discovering interacting data entities in data-intensive cloud workflows[J]. Future Generation Computer Systems 113:474\u0026ndash;487\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSamerei SA, Aghabayk K, Mohammadi A et al (2021) Data mining approach to model bus crash severity in Australia[J]. J Saf Res 76:73\u0026ndash;82\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMontella A, de O\u0026ntilde;a R, Mauriello F et al (2020) A data mining approach to investigate patterns of powered two-wheeler crashes in Spain[J], vol 134. Accident Analysis \u0026amp; Prevention, p 105251\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZheng Z, Lu P, Lantz B (2018) Commercial truck crash injury severity analysis using gradient boosting data mining model[J]. J Saf Res 65:115\u0026ndash;124\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBechini A (2016) A Map Reduce solution for associative classification of big data[J]. Inform Sciences: Int J 332:33\u0026ndash;55\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eIko Pramudiono and Masaru Kitsuregawa.Parallel FP-Growth on PC cluster.In PAKDD,2003\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLe Zhou,Zhiyong Zhong,Jin Chang,Junjie Li,Joshua Zhexue Huang,Shengzhong Feng.Balanced parallel FP-Growth with MapReduce.Information Computing and Telecommunications (YC-ICT),2010 IEEE Youth Conference on 28\u0026ndash;30 : 243\u0026ndash;246\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHe K, Yuan Z, Yang Y (2022) \u0026ldquo;A Roadway Safety Sustainable Approach: Modeling for Real-Time Traffic Crash with Limited Data and Its Reliability Verification,\u0026rdquo; Advances in Transportation Studies, Journal of Advanced Transportation, vol. Article ID 1570521, 2022\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Traffic crash, Data mining, FP-Growth algorithm, Load balancing","lastPublishedDoi":"10.21203/rs.3.rs-1311180/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-1311180/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eTraffic safety is an important part of the roadway in sustainable development. Freeway traffic crashes typically cause serious casualties and property losses, being a serious threat to public safety. Figuring out the potential correlation between various risk factors and revealing their coupling mechanism are of effective ways to explore and identity freeway crash causes. However, the existing association rule mining algorithms still have some limitations in both efficiency and accuracy. Based on this consideration, using the freeway traffic crash data obtained from WDOT (Washington Department of Transportation), this research constructed a multi-dimensional multi-level system for traffic crash analysis. Considering the load balancing, the FP-Growth algorithm was optimized parallelly based on Hadoop platform, to achieve an efficient and accurate association rules mining calculation for massive amounts of traffic crash data; Then, according to the results of the coupling mechanism among the crash precursors, the causes of freeway traffic crashes were identified and revealed. The results show that the parallel FP-growth algorithm with load balancing constraints has a better operating speed than both conventional FP-growth algorithm and parallel FP-growth algorithm towards processing big data. This improved algorithm makes full use of Hadoop cluster resources and is more suitable for traffic crash large data sets mining while retaining the original advantages of conventional association rule mining algorithm. In addition, the mining association rules model with the improvement of multi-dimensional interaction proposed in this research can catch the occurrence mechanism of freeway traffic crash with serious consequences (lower support degree probably) accurately and efficiently.\u003c/p\u003e","manuscriptTitle":"A Parallel FP-Growth Mining Algorithm with Load Balancing Constraints for Traffic Crash Data","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2022-03-01 20:01:26","doi":"10.21203/rs.3.rs-1311180/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"e91909fd-ae8b-47c4-9ed3-b46db8ed740c","owner":[],"postedDate":"March 1st, 2022","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2022-04-05T15:59:54+00:00","versionOfRecord":[],"versionCreatedAt":"2022-03-01 20:01:26","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-1311180","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-1311180","identity":"rs-1311180","version":["v1"]},"buildId":"J0_U0BvcaRcwD8yVFaRlm","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}