Advancing Multilingual Abstract Classification: A Comparative Analysis of Feature Extraction Models in Systematic Reviews

Screening prioritization via active learning plays a vital role in streamlining the systematic review process. However, most existing software implementations are primarily designed for English texts, limiting their effectiveness in handling multilingual abstracts. This study aims to evaluate eight feature extraction models explicitly engineered for multilingual tasks, comparing their performance with two commonly used models, TFiDF and SBERT. Using a simulation-based approach, the models are assessed across four datasets, considering two classification algorithms and two prior knowledge conditions. The results highlight the superiority of models specifically designed for multilingual tasks, such as mLongT5, LaBSE, and MUSE, in classifying multilingual abstracts. These models exhibit faster identification of relevant papers, outperforming traditional models like TFiDF, which struggles to handle non-English papers. The choice of classification algorithm and prior knowledge conditions showed minimal impact on performance. This finding challenges the common reliance on TFiDF in active learning-based tools and emphasizes the need for more effective multilingual models. The study also reveals variations in model performance based on the language composition of datasets. Thus, the generalizability of findings may be limited to the specific dataset used. This emphasizes the importance of consciously addressing language diversity in systematic reviews. By adopting more effective multilingual models, researchers can enhance the efficiency and accuracy of automated abstract screening, overcoming language barriers in scientific research. Future research should explore diverse datasets and evaluate the trade-off between model performance and computational resources. Overall, this study contributes valuable insights for researchers and developers in selecting suitable feature extraction models for multilingual abstract screening tasks.