Data cleaning method based on multiple interpolation

Abstract To ensure the availability and data quality of integrated and fused data, develop data cleaning methods, and achieve real-time processing of missing data values, this research project studies various methods for filling missing data values. By understanding the principles of each method, a multi-method data missing value filling module is developed, and a multiple interpolation missing value method based on random forest method is proposed. Using 500 sample data from a 250MW gas turbine in 2013 for simulation and comparison tests, in order to test the calculation errors of the four filling methods under different missing rates, five rows of sample data were randomly emptied at missing rates of 25%, 50%, 75%, and 90%. The experimental results compared the mean absolute error (MAE), mean squared error (MSE), root mean squared error (RMSE), running time, and maximum deviation. Compared with traditional regression filling methods, the multiple interpolation method has an accuracy improvement of more than 90% in terms of MAE, MSE, RMSE, running time, and maximum deviation. Due to the complexity of the multiple interpolation algorithm, for 500 sample data, the running processing time is 20s longer. Subsequently, appropriate data cleaning methods can be selected based on actual background conditions.