Automated Research Review Support using Machine Learning, Large Language Models, and Natural Language Processing

Research expands the boundaries of a subject, economy, and civilization. Peer review is at the heart of research and is understandably an expensive process. This work, with human-in-the-loop, aims to support the research community in multiple ways. It predicts quality, and acceptance, and recommends reviewers. It helps the authors and editors to evaluate research work using machine learning models developed based on a dataset comprising of 18,000+ research papers, some of which are from highly acclaimed, top conferences in Artificial Intelligence such as NeurIPS and ICLR, their reviews, aspect scores, and accept/reject decisions. Using Machine Learning algorithms such as Support Vector Machines, Deep Learning Recurrent Neural Network architectures such as LSTM, a wide variety of pre-trained word vectors using Word2Vec, GloVe, FastText, transformer architecture based BERT, DistilBERT, Google’s Large Language Model (LLM), PaLM 2, and TF-IDF vectorizer, a comprehensive system is built. For the system to be readily usable and to facilitate future enhancements, a frontend, a flask server in the cloud, and a NOSQL database at the backend were implemented making it a complete system. The work is novel in using a unique blend of tools and techniques to address most aspects of building a system to support the peer-review process. The experiments resulted in a 86% test accuracy on acceptance prediction using DistilBERT. Results from other models were comparable, with PaLM based LLM embeddings achieving 84% accuracy.