Severity-oriented Multi-objective Crowdsourced Test Reports Prioritization

Crowdsourced testing has been widely used to improve software quality since it can quickly obtain a vast number of test reports revealing various bugs. One of the most critical tasks in crowdsourced testing is determining how to inspect these valuable reports efficiently and quickly. In recent years, many automated techniques, like clustering and prioritizing, have arisen to assist in examining crowdsourced test reports. Due to the high repetition rate, most existing crowdsourced test report prioritization techniques concentrate on finding different bugs earlier while ignoring bug severity. The bug's severity reflects how detrimental the bug is to the quality of the software. In practice, developers should fix highly severe bugs as early as possible. In this paper, we propose a multi-objective prioritization technique for crowdsourced test reports that considers the diversity and severity of bugs. First, test reports are sorted in order of severity, and then the clustering results of test reports are used to adjust the first sorting sequence to ensure diversity. To acquire the clustering results, we first obtain the text feature vector and image feature vector of the test report separately, then fuse these two feature vectors. Finally, we use the density clustering algorithm to cluster the fused feature vectors. We conducted experiments on an industrial crowdsourced test report dataset from six mobile application projects to validate our method. The results show that our method can detect serious bugs earlier and different bugs faster than existing methods.