SPECTRUM SRA: An R Shiny Application to Automate Systematic Reviews Using Artificial Intelligence and Large Language Models

SPECTRUM SRA: An R Shiny Application to Automate Systematic Reviews Using Artificial Intelligence and Large Language Models | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article SPECTRUM SRA: An R Shiny Application to Automate Systematic Reviews Using Artificial Intelligence and Large Language Models Hossein Motahari Nezhad, Mehrdad CheshmehSohrabi This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7265470/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Purpose : This study aims to develop an R Shiny application named SPECTRUM SRA (Systematic Review Automation) and to assess its feasibility. The application encompasses eight modules: Search, PRISMA flow diagram generation, dEduplication, sCreening, daTa extraction, Reporting, qUery, and suMmarization. It provides an end‑to‑end workflow to automate and semi‑automate multiple steps of systematic reviews, addressing growing time and labor constraints and accelerating evidence‑to‑practice translation. Design/methodology/approach : SPECTRUM SRA comprises eight interconnected modules and offers an interface for automated article search and retrieval from PubMed, Scopus, and Web of Science. Search functionality is implemented via API integration for PubMed and Scopus, with manual search and import for Web of Science results. Deduplication employs two strategies: DOI‑based matching and fuzzy string deduplication. The application also supports automated title/abstract and full‑text screening using large language model prompting. Automated reporting for both search and screening is incorporated through AI‑driven scripting. Additionally, SPECTRUM SRA automates PRISMA flow diagram generation. Summarization and data extraction of the final set of eligible studies are performed via LLM prompting. The feasibility of the application was assessed through a systematic review of randomized controlled trials comparing virtual reality interventions to non‑virtual‑reality approaches in dentistry. Findings : Keyword identification was completed in 12 seconds; search query generation took less than three seconds; and retrieval of 703 records from the databases required 4 minutes and 30 seconds. Deduplication removed 238 records in 3 seconds, leaving 465 records for screening. ChatGPT o4‑mini‑high and DeepSeek V3 R1 were selected as the first and second reviewers, respectively. Title/abstract screening took 84.5 minutes for Reviewer 1 and 131 minutes for Reviewer 2 (κ = 0.852). Full‑text screening of 43 articles required 43 minutes for Reviewer 1 and 90 minutes for Reviewer 2 (κ = 0.903). Automated summarization and data extraction of the 25 final eligible studies achieved 91% accuracy using ChatGPT in 25 minutes. Originality : To the best of our knowledge, SPECTRUM SRA is the first end‑to‑end Shiny application that supports multiple tasks in systematic reviews. It can reduce the time, labor, and energy required to conduct systematic reviews, thereby accelerating health policy and decision‑making. Artificial Intelligence and Machine Learning Automation Natural language processing NLP Reproducible research Evidence synthesis Evidence based-medicine Meta-analysis Figures Figure 1 Introduction Evidence-based medicine (EBM) underpins contemporary health policy and patient-care decision making, drawing on the most recent and relevant literature (Djulbegovic & Guyatt, 2017 ; Radenkovic et al., 2019 ). However, the volume of publications across disciplines is rising so quickly that keeping pace has become increasingly time-consuming—often exceeding the capacity of individual researchers (Topol, 2019 ). Because new papers appear daily in every domain, maintaining up-to-date systematic reviews is particularly challenging. Among the various forms of literature synthesis, systematic reviews follow explicit, predefined steps to locate and summarize evidence for a specific field and research question; their rigorous, current findings therefore remain essential for sound health policy and clinical decisions (Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, 2022). Systematic reviews, owing to their explicit, predefined methodologies, sit at the pinnacle of the evidence pyramid and are considered the reference standard for evidence synthesis (Wallace et al., 2022 ). Clinicians and other health professionals rely on them to stay abreast of the latest findings (Isfeedvajani, 2016 ). To produce a review, authors must progress through a series of prescribed steps—formulating a research question; defining inclusion and exclusion criteria; systematically searching electronic databases; screening records; extracting data; and synthesizing the evidence—each of which contains further sub-tasks that add to the overall complexity. The breadth and quality of a review hinge, first and foremost, on the thoroughness of its database searches. According to the Cochrane Handbook, PubMed is among the most comprehensive and frequently consulted sources for healthcare reviews (Ossom Williamson & Minter, 2019 ), but authors are advised to search at least two databases. In practice, PubMed, Scopus, and Web of Science are the three most commonly used resources for biomedical systematic reviews. Systematic reviews are highly time-consuming and resource-intensive: completing one manually can take several months—or even years (Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, 2022). The first challenge of the traditional approach is its substantial demand for time, labor, and funding. Estimates indicate that producing and publishing a single review may require more than five years, a team of five reviewers, and a budget of about US $ 140 000 (Michelson & Reuter, 2019 ; Borah et al., 2017 ). Although recently developed automation and semi-automation tools can cut the timeline dramatically—sometimes to as little as two weeks (Clark et al., 2020 )—no single tool yet automates every review step within one unified platform. These resourcing pressures are intensifying as the sheer number of systematic reviews—and the evidence they must cover—continues to surge. The number of published systematic reviews is also climbing rapidly; daily output rose from 80 in 2019 (Hoffmann et al., 2021 ) to 135 in 2021 (Rethlefsen et al., 2024 ), a 1.7-fold increase. A simple PubMed search shows that more than 37 000 systematic reviews appeared in 2024 alone. According to several studies, the typical lifetime of a review is about 5.5 years, after which an update is required to capture the latest literature, highlighting the need for better management and partial automation of review steps (Shojania et al., 2007 ). Yet updating remains difficult: a primary study often takes 2.5–6.5 years to be incorporated into a review, creating a sizeable evidence-to-practice lag. The Cochrane Handbook therefore recommends updating every two years (Cumpston et al., 2019 ). Nevertheless, roughly one quarter of reviews become obsolete within their first two years because newly published evidence is not incorporated, a lapse that can materially alter conclusions (Shojania et al., 2007 ). Systematic reviews comprise numerous steps, each with its own workload, and if reviewers attempt to work by hand at every stage, completing a review can become practically impossible in the face of today’s knowledge explosion. These complexities and time-consuming tasks can be managed or minimized by employing artificial-intelligence (AI) techniques. Numerous fully and semi-automated tools have therefore been established to expedite the completion of systematic reviews. As noted above, the traditional process remains time-consuming, labor-intensive, and financially demanding. In situations such as health emergencies, for example, Covid-19 or whenever decision makers need rapid evidence for treatment, the conventional approach to systematic reviews does not suffice. Consequently, researchers have begun to evaluate which parts of the workflow can be delegated to AI-driven systems and which still require human oversight. Employing artificial-intelligence technologies and programming has led to the creation of a range of platforms, software, tools, and models that accelerate numerous steps of systematic reviews, expand reviewer capacity, minimize errors, and ultimately improve reproducibility. Because the philosophy and specifications of each review step differ, the automation or semi-automation of every stage must follow its own strategy (O’Mara-Eves et al., 2015 ). The adoption of AI automation now has formal backing: the Cochrane Handbook, for example, recognizes the use of such tools—sometimes as a second reviewer (Cumpston et al., 2019 ). Even in fully manual reviews, accuracy varies with reviewers’ expertise and diligence. With two reviewers, about 3% of eligible studies are still missed; the figure rises to 13% when a single reviewer works alone (Blasingame et al., 2024 ). These limitations have prompted the use of AI for automation and semi-automation of different review steps, although most automated workflows still depend on human judgment. Persistent human error, even with dual reviewers, necessitates for end-to-end platforms that can automate, semi-automate, and manage the entire systematic-review process (Zaki et al., 2022 ; Wang et al., 2024 ; Marshall & Wallace, 2019 ). Therefore, the main aim of this study is to design an R Shiny end-to-end application to automate or semi-automate the main steps of systematic reviews including search, article retrieval, deduplication, screening, reporting, Prisma graph generation, summarization and data extraction. Literature Review Efforts to automate systematic reviews date back to at least the early 2000s. By that time, the explosion of published research had made manual evidence synthesis increasingly unsustainable. Through the mid-2010s, research in systematic review automation accelerated. A variety of tasks in the systematic review workflow were targeted for automation. In 2014, Tsafnat and colleagues published a landmark survey paper which mapped out the state of the art at that time (Tsafnat et al., 2014 ). The researchers argued that the creative aspects of review design would always require human expertise, but many technical tasks could eventually be executed by machines. Around 2015–2016, Marshall et al. developed RobotReviewer, an NLP/ML system to automatically evaluate risk of bias in randomized trial reports (Marshall et al., 2016 ). An evaluation in 2016 showed RobotReviewer’s assessments agreed with human reviewers’ judgments much of the time (Marshall et al., 2016 ). However, machine learning-based automation systems of systematic reviews can have limitations in the data-extraction phase and may underperform on complex tasks (Schmidt et al., 2023 ). In the context of systematic reviews, NLP methods are applied to a variety of tasks such as information retrieval, study characterization, data extraction, and summarization. Fundamentally, much of the automation pipeline deals with unstructured text (journal articles), so NLP underlies many tools in this domain. A core NLP task in systematic reviews is classifying documents into categories, often to decide relevance. Early approaches used simple keyword-based or rule-based systems, but modern methods leverage machine learning to learn classification from examples. Beyond identifying which references to include, systematic reviews require pulling out specific data from full-text articles – e.g. sample sizes, patient characteristics, interventions, outcome results, etc. NLP methods have been developed to automate parts of the data extraction step. A common focus is on identifying PICO elements (Population, Intervention, Comparator, Outcome) within an article. For example, one approach by Wallace et al. used supervised NLP to highlight sentences in trial reports corresponding to each PICO element (Wallace et al., 2016 ). In evaluations, automated extraction often has only moderate agreement with human extractors and may miss context needed for correct interpretation (Jardim et al., 2022 ). Another domain where NLP plays a role is in the search stage of a systematic review is for literature search and retrieval. Traditional search strategies rely on Boolean combinations of keywords and indexing terms. NLP and text mining can enhance this in several ways. One is automatic term suggestion. Tools can analyze a set of known relevant articles and suggest additional keywords or MeSH terms that might retrieve similar studies (Wang et al., 2022 ). Moreover, NLP can aid in deduplication of search results by comparing titles and abstracts to identify the same study reported in multiple sources (Bramer et al., 2016 ). A forward-looking application of NLP in systematic reviews is the automatic summarization of evidence. Some prototype systems have been able to auto-generate simple summary paragraphs from meta-analytic data. In recent times, large language models (LLMs) have demonstrated the ability to produce fluent, coherent text, raising the question of whether they could draft systematic review summaries. In early explorations, ChatGPT was able to “identify and summarize relevant information” from a small set of abstracts (Qureshi et al., 2023 ). In systematic reviews, ML approaches have been applied across nearly all stages: from screening and study selection to data extraction and meta-analysis. Early attempts used algorithms like support vector machines or logistic regression on hand-engineered text features. One notable supervised ML system was Abstrackr, introduced around 2012 (Abstrackr, n.d.). The limitations and considerations include the need for an initial training set (often reviewers label a few hundred citations first), and the risk of missing studies if the model’s predictions are trusted too much (Tsou et al., 2020 ). Active learning is implemented in tools like ASReview (Active Supervision for systematic Reviews), which is a more recent open-source tool (van de Schoot et al., 2021 ). It provides a modern interface for active learning in screening. Deciding appropriate stopping criteria (stopping too early could omit relevant studies), and the fact that active learning algorithms may need some tuning depending on the prevalence of relevant studies and the inclusion criteria complexity have resulted in limitations and challenges in these strategies (Kilicoglu et al., 2009 ). RobotAnalyst is another tool (developed by the EPPI-Centre, UCL) that offers a suite of text mining functions for reviews. It can automatically classify studies (with various pre-trained models) and cluster references into topics, helping reviewers target relevant clusters (RobotAnalyst, n.d.). Covidence is a cloud-based platform endorsed by Cochrane and used by many institutions to streamline study selection and data extraction (Covidence, n.d.). Covidence ensured this automation is optional and transparent – users can review the auto-excluded citations and restore any if needed, however, Covidence remains dependent on human decisions and are not fully automated, even if they do save time (Scherbakov et al., 2025 ). DistillerSR is another popular commercial platform (DistillerSR, n.d.). DistillerSR offers a system for managing large reviews with complex forms, and in recent years it has added a module called DistillerAI . DistillerAI includes an ML model that learns from a subset of screening decisions and then suggests includes/excludes for the remaining citations, along with a confidence score. Rayyan deserves special mention as a platform developed by Qatar Computing Research Institute (Rayyan, n.d.). Rayyan’s primary function is to facilitate title/abstract screening with a sleek interface and the ability to blind multiple reviewers to each other’s decisions. A limitation is that its ML suggestions are relatively simplistic and not as transparent or controllable as some academic tools. Tools such as Rayyan, Abstractr, and Research Screener (Chai et al., 2021 ) assist with screening, deduplication, and resource management, yet they rely heavily on human feedback to train their models, which requires a lengthy training phase for reviewers (Khalil et al., 2022 ; Galli, Anna Viktorovna Gavrilova, et al., 2025). EPPI-Reviewer is a software mainly used by information specialists (Park & Thomas, 2018 ). It has incorporated text mining modules and classifiers. For example, EPPIReviewer can automatically classify studies by type (like RCT or observational). It also has features like term extraction, auto-clustering. The advantage of EPPI-Reviewer is that it’s very powerful and flexible – it can handle very large datasets and unusual review designs. The drawback is that it can be less user-friendly to non-experts and historically had a steeper learning curve (Tsou et al., 2020 ). In addition to full-featured platforms, there are specialized tools targeting specific systematic review tasks. RobotReviewer is one focusing on automating risk-of-bias assessment or data extraction (Marshall et al., 2016 ). Quick Clinical is a federated search engine, but is a limited source databases and is not optimized for systematic reviews, or Sherlock is search engine specially for trial registries working only for clinicaltrials.gov database. ExaCT is also for extracting PICO elements of articles (Tsafnat et al., 2014 ). The recent shift from traditional machine learning to large language models, driven by extensive developments in natural-language processing (B et al., 2024 ), has positively revolutionized the automation of systematic reviews (Cao, Sang, et al., 2025 ). With the rapid advancements in large language models (LLMs) in recent years, the hope for revolutionizing the conduction of systematic reviews has increased—to improve speed and accuracy, streamline processes, and ultimately save time and energy—although their use in automating systematic reviews must still be interpreted with caution (Alshami et al., 2023 ). The application of LLMs in various stages of systematic review automation has been increasingly explored. For example, a recent study evaluated the performance of ChatGPT-4 in the data extraction phase of systematic reviews and found that it could serve as a secondary reviewer with an accuracy exceeding 94% (Motzfeldt Jensen et al., 2025 ). Another recent study investigated the use of ChatGPT for title and abstract screening, demonstrating high accuracy in this task as well (Nykvist et al., 2025 ). otto-SR is also a LLM-based workflow to automate systematic reviews supporting screening and data extraction (Cao, Arora, et al., 2025 ). Studies further indicate that the use of AI-automation tools in systematic-review workflows is not yet pervasive, mainly because limited usability and lack of user-friendliness hinder adoption (Hossain, 2024 ). Another barrier is the steep learning curve associated with many automation tools (O’Connor et al., 2018 ). Finally, because there are separate tools for each review step, reviewers must navigate among multiple, non-interoperable systems (Khalil et al., 2022 ). These obstacles help explain why enthusiasm for automation co-exists with persistent practical frustrations among reviewers. As discussed in this literature review, a variety of tools, platforms, and models have been developed to automate or semi-automate different stages of the systematic review process. Many of these tools focus on a single task, such as automating or semi-automating the search process, screening, or data extraction. Others offer techniques that assist reviewers but still rely heavily on human involvement. Consequently, there is a clear need for a comprehensive, end-to-end tool or platform that can support the automation or semi-automation of key tasks in systematic reviews, including literature search, articles retrieval, deduplication, screening, PRISMA flow diagram generation, data extraction, and summarization. Therefore, the primary aim of this study is to develop a Shiny application that automates the main tasks involved in conducting systematic reviews. This tool is designed to assist systematic reviewers in managing their review process, while also automating or semi-automating key stages. By providing a user-friendly interface, the application aims to streamline the systematic review workflow, reduce the time and labor required, and ultimately accelerate evidence generation and heath policy and decision making in medicine. Methods SPECTRUM SRA (Systematic Review Automation) is an R shiny application comprises eight modules including Search, Prisma diagram, dEduplication, sCreening, daTa extraction, Reporting, qUery, and summarization to automate numerous steps of systematic reviews. The application can be accessed using the following webpage link: https://motaharinezhad.shinyapps.io/SPECTRUM_SRA/ . It integrates data from multiple academic databases, including PubMed, Scopus, and Web of Science, allowing researchers to efficiently collect and manage large volumes of relevant literature. The system also supports screening and decision-making processes, with built-in mechanisms for managing disagreements between reviewers. Furthermore, the system generates comprehensive reports, including PRISMA flow diagrams, which visually represent the study selection process, and provides outputs for further analysis. In this section, we will provide a description of the methods developed for each part of the system. Inclusion and exclusion criteria The "Inclusion & Exclusion Criteria" menu is designed as the first and most important part of the system, ensuring that only relevant studies are included in the systematic review process. This section enables authors and researchers to clearly define the criteria for including or excluding studies based on predetermined guidelines. The section is divided into two primary parts: Inclusion Criteria: In this part, the authors must specify the conditions that a study must meet in order to be included in the systematic review. This could include specific study characteristics, such as the population studied, the type of intervention, comparator, outcome(s), study design, or any other relevant factors. Exclusion Criteria: In this part, the authors define the criteria for excluding studies from the review. This could include studies that do not meet the inclusion criteria, studies with methodological issues, or studies that focus on irrelevant populations or interventions. The defined inclusion and exclusion criteria are stored in the system and used to guide the review process in screening process. Keywords Identification The "Keywords Identification" menu in the system is a helpful tool for researchers to extract meaningful keywords from the documents they are working with. This part of the system is designed to automate the extraction of unigrams, bigrams, and trigrams from gold standards uploaded PDF documents, which are then displayed for the users to review. This is particularly useful in systematic reviews where keywords are critical for refining search terms and understanding the content of the research articles. At the core of this menu is the file upload section, which allows users to upload one or more PDF files of gold standard articles from which keywords will be extracted. Once the files are uploaded, the system enters the processing and extraction phase. During this phase, the uploaded PDFs are processed to extract the text contained within them. The system first cleans this text by removing unnecessary elements like stop words and punctuation. It then performs keyword extraction, identifying the most frequent unigrams (single words), bigrams (two-word combinations), and trigrams (three-word combinations) within the text. The frequency of each of these keywords is counted, allowing the system to identify the most common terms within the documents. This structure helps users to quickly identify the most relevant terms from the gold standards to search related literature in databases. The "Mesh Terms" section within the "Keywords Identification" menu is another component designed to help users identify standardized Medical Subject Headings (MeSH) terms associated with their research. MeSH terms are essential for indexing and categorizing scientific literature, ensuring that articles are accurately categorized and easy to search for in databases. This part of the system automates the process of retrieving MeSH terms for the keywords identified from the user's research. These terms are separated into four categories to reflect the common structure of a systematic review, often referred to as PICO (Population, Intervention, Comparator, and Outcome). Once the search terms are entered, the system moves to the next stage, which is preprocessing and phrase extraction. The system processes these keywords by applying text preprocessing techniques. Following preprocessing, the system automatically performs a query to an external MeSH term database. This is a crucial part of the system, as it ensures the retrieval of the most accurate and relevant MeSH terms for each of the user's search terms. Once the MeSH terms are fetched, the system aggregates the results for each category (population, intervention, comparator, and outcome). Search The search terms section is a critical part of the system, where users define the terms, they want to use in the search. This section allows users to add up to five groups of terms. Each group can consist of multiple terms related to the topic of interest. In the search form, each search group has two fields: keywords and their Mesh terms. Each search group is connected by logical operators, either "AND" or "OR. The operator determines how the terms in that group will be combined in the final query. In the Exclusion Terms part, users can specify terms that should be excluded from the search results Publication Date Range The Publication Date Range filter enables the user to define the time frame for the publications they wish to include in their search. The user can select a start and end date using a date picker, which then defines the acceptable publication range for the search results. Language The Language filter lets the user specify the language(s) of the studies they want to include in their search. A drop-down list of languages is provided, and the user can select one or more languages for the search. Document Type The Document Type section allows the user to select the type of publication they are interested in, such as "journal article", "review", "clinical trial", etc. The system provides a list of document types, and the user can choose one or more types that are relevant to their research. Sex, Species, and Age Group Filters The Sex, Species, and Age Group filters allow the user to further refine the search results based on demographic and study-specific factors. Sex filter allows the user to specify the sex of the participants in the studies they are interested in. The user can select from options "Male", "Female", or "All". This is useful for narrowing down studies that involve a specific sex or gender. Species filter allows the user to filter studies based on the species involved, such as "Human" or "Animal". This helps researchers focus their search on studies involving specific species, in medical and biological research. Age Group filter allows the user to select specific age groups, such as "Child", "Adult", or "Elderly". This is useful in studies that focus on particular age demographics, helping to target research that is more relevant to specific populations. Search Queries The "Search Queries" menu generates the specialized search syntax for three databases: PubMed, Scopus, and Web of Science. This menu is accessed after the user has entered the relevant search criteria in the "Search" menu. Once the user presses the "Create Search Queries" button in the "Search" menu, the system automatically constructs the search queries tailored to the specific requirements of each database. The PubMed search query construction process begins after the user presses "Create Search Queries". PubMed uses a specific query syntax that includes search terms (keywords and MeSH terms) and other specific filters including study design, sex, specie, and age group. After collecting all the inputs (search terms, MeSH terms, exclusion terms, etc.), the system generates the final PubMed query. It combines the search terms and filters to ensure they are properly formatted for PubMed’s API. The Scopus search query follows a different syntax. Scopus queries use specific formatting rules for logical operators and filters. The system uses the search terms from the Search section and generates a Scopus-specific query. The user’s search terms are included in the Scopus query, and they are formatted using Scopus’s syntax. Scopus uses AND/OR to combine terms, similar to PubMed. Scopus does not support MeSH terms directly like PubMed, so the system converts MeSH terms into the appropriate keywords or synonyms that will match the articles in Scopus. Scopus also allows the user to filter results based on document type, such as articles, reviews, etc. These filters are included in the query using Scopus’s document type syntax. The system checks the user's input and incorporates these filters into the query. The Web of Science (WoS) query construction is slightly different from the other two. Web of Science has its own format and special operators, so the system needs to adjust the syntax accordingly. Similar to PubMed and Scopus, the search terms are included in the Web of Science query. These terms are joined with AND/OR operators, depending on how the user has configured them. The query syntax for search terms is structured to match the Web of Science requirements. Web of Science does not natively support MeSH terms, so the system similarly converts the MeSH terms into keywords or synonyms that fit the Web of Science search format. These terms are included as part of the query. After generating the queries for each of the databases, the user can execute these queries. The system sends the queries to the respective database’s API (for PubMed, Scopus) to retrieve the relevant records. This is done via the rentrez package for PubMed, and httr package for Scopus. When it comes to the WoS database, the document extraction process differs from that of PubMed and Scopus. Users need to copy the generated search syntax for WoS and run it manually in their institution’s subscribed advanced WoS search portal. After retrieving the relevant studies, they should download the results in Excel format. Finally, in the “Retrieved Studies” menu of the application, they can upload the downloaded Excel file. Retrieved Studies In the "Retrieved Studies" main menu all the studies retrieved from databases are stored, processed, and merged. This section allows users to handle studies obtained from PubMed, Scopus, Web of Science, and other databases. The system provides the capability to automatically import, process, and merge study results. The system has built-in functionality to fetch study data directly from PubMed and Scopus. Once the user generates the search query for PubMed, the system allows the user to run search. This action triggers the system to send the query to PubMed via the Entrez API from the rentrez package. The system fetches the search results from PubMed, specifically retrieving metadata including study titles, authors, journal, publication years, and DOIs. The results are then processed and displayed in the "Retrieved Studies" menu, under the PubMed section. Similarly, for Scopus the system allows the user run search which sends the query to the Scopus database. The query is processed through the Scopus API. Retrieved studies are then automatically displayed in the "Retrieved Studies" section, specifically for Scopus. For Web of Science, the process is a bit different because the system does not have an API integration to automatically fetch results. After the user generates the Web of Science search query, the system prompts the user to manually copy the generated search query and run it directly in the Web of Science database. The user can then upload this Excel file to the "Web of Science" section of the "Retrieved Studies" menu. The file is processed by the system and displayed in the same manner as the PubMed and Scopus studies. The system also provides functionality to handle results from other databases that may not be covered by PubMed, Scopus, or Web of Science. If a user conducts a search in another database, they can download the required Excel file format from the system. Once the user has gathered data from another database and formatted it according to the system’s required structure, they can upload the excel file. The system displays the results similarly to the other database sections. After importing and uploading all the studies from PubMed, Scopus, Web of Science, and other sources, the system offers the "Merge Studies" functionality. This feature is a critical step, as it combines all the studies from the different databases into one consolidated list. Deduplication The deduplication module of the system is implemented as a three-stage pipeline that is surfaced to the user through the “Deduplication” menu. When a search has been completed in the retrieval tab, the application constructs a transient master table by binding the PubMed, Scopus, Web of Science and any user-supplied “Other” datasets. The first stage of deduplication, DOI-based deduplication, is an exact-match procedure that assumes the Digital Object Identifier (DOI) to be the most reliable global key. Using this section, the observer retrieves the current merged table, filters away rows whose DOI field is empty or missing, and then retains only the first occurrence of every unique DOI. The second stage deals with citations that lack a DOI.This section copies those rows and displays them for inspection. A subsequent click on “Run Fuzzy Deduplication” launches a two-part algorithm. First, each DOI-less record is normalized into four comparison keys. The title is lower-cased, punctuation stripped and whitespace squashed; the first author is reduced to “surname plus initial”; the journal name is condensed to an acronym of initial letters after stop-word removal; and the publication year is kept as a four-digit string. Consequently, a deterministic grouping on the quartet of keys removes intra-list duplicates, preserving the first entry in every group. The algorithm then safeguards against residual cross-source duplicates by anti-joining the DOI-less survivors to the DOI-bearing set — if any DOI-less record’s four keys are identical to those of an already accepted DOI record it is discarded. Finally, this data frame is rendered in an interactive table and becomes the sole source for every downstream workflow, including title-abstract screening, full-text appraisal, automatic reporting and PRISMA-flow construction. Title/Abstract Screening The title and abstract screening module translate routine dual-review appraisal practice into a reproducible, state-aware workflow. The application provides the places for one or two reviewers for conducting and completing this part separately. For each reviewer it creates an independent tab listing every record in the deduplicated master list. There are two writable columns entitled Decision (Include or Exclude) and Comments and adding a read-only hyperlink that resolves either to the article’s DOI landing page or, if no DOI is present, to a Google Scholar query. On completion of individual appraisal, the system provides three analytic actions. The Identify disagreements button is pressed, the server identifies and lists the documents which the reviewers’ decisions are not the same. The Identify excluded studies and included studies buttons transform local reviewer decisions into canonical include and exclude lists that will later feed the PRISMA flow diagram. The inclusion of a contextual large language model (LLM)-consult hyperlink for every record in reviewers’ panel provides a documented decision aid: clicking the link opens a modal that presents a pre-formatted prompt combining the study’s PDF with the user’s previously saved inclusion and exclusion criteria and requests an “Include” or “Exclude” recommendation plus justification, due to the high accuracy of LLMs in screening literature. Full-text Screening The full-text screening module operationalizes the second eligibility appraisal phase by building on the record set that emerged from title-and-abstract selection. The parts and mechanism of actions are the same as title/abstract screening. Search and Screening Reports The Search and Screening Reports menu converts the raw reactives produced throughout the workflow into narrative prose blocks that can be pasted directly into the results section of a systematic review manuscript. It is organized as a tabset whose first sub-tab summarizes database retrieval and deduplication, while the second synthesizes the quantitative outcome of title/abstract and full-text appraisal. The server gathers the four data frames that holds retrieval output—PubMed results, Scopus results, WoS results and other results. The total number of retrieved citations is derived from merge studies. DOI duplicates are quantified as the difference between the number of DOI-populated rows in the merged table and the number of rows in deduped studies, which holds the output of the exact DOI pass. The number of studies remaining after deduplication is also presented. A small HTML fragment is then constructed on the fly: it embeds the current date in long format, lists the record counts for each individual database, presents the total retrieved figure, reports the duplicate breakdown and finishes with the final post-deduplication count. The Screening sub-tab is activated by the button “Report Screening Results”. When reporting screening results, the system automatically provides the following metrics separately for the title/abstract and full-text screening phases: the number of screened studies, the number of disagreements, the number of removed studies, and the Kappa statistic between the two reviewers (if applicable). PRISMA flow diagram The PRISMA-flow module converts every numeric outcome accumulated during searching, deduplication and screening into a publication-ready schematic that follows the 2020 PRISMA template. The system first enforces three prerequisites: a deduplicated master list must exist, title-and-abstract screening must have been finalized, and full-text screening must likewise be completed; failure of any guard raises a notification and aborts generation. Once validated, the algorithm interrogates reactives to retrieve every count required by the flow. Download totals come from query-specific data frames; duplicates are partitioned into DOI and non-DOI classes and summed; post-deduplication count is taken from the final studies table. Screening numbers are extracted from helper reactives that mirror the logic used to construct included- and excluded-study lists. If reviewers declared any “other methods” the routine sums those n values and adds them to the count of studies included after full-text so the synthesis box reflects the complete set entering qualitative analysis. The server composes a Graphviz DOT script. The script then constructs a subgraph for database identification containing a header and two child boxes: one enumerating every contributing database with its record count and one listing the duplicate-removal figure. If the user supplied other databases a sister subgraph is generated in a second column, linked to the main header. Exclusion-reason lines are concatenated into a multiline label. Finally, orthogonal arrows connect identification to duplicates removed, then to screening, then to full-text assessment, and finally to inclusion and synthesis, with optional right-hand edges linking supplementary identification boxes back into the main spine. The completed DOT string is cached and passed to DiagrammeR::grViz, which renders an SVG. Summarization and Data Extraction The Summary and Data Extraction of Included Studies module turns the set of articles that survived full-text appraisal into an interactive register and provides an optional, on-demand service that relies on automatic summarization and data extraction. The interface begins with an import button. It copies the records into the reactive store included summary, drops the screening-specific Decision and Comments columns so that only bibliographic fields remain. Clicking a summarization/data extraction button opens modal dialog containing a richly formatted, multi‑line prompt. The established prompt includes two main parts, first to generate a four‑part summary (purpose, methods, results, conclusion), then to extract an extended list of data elements (from setting and funding source, PICO elements, through effect estimates and measures of variability). The modal also embeds two helper controls. The “Edit” button to make the prompt text area editable, allowing on‑the‑fly refinement; the “Copy” button runs a small JavaScript snippet that reads the entire contents of the text area and writes it to the clipboard, ensuring a single‑click transfer into ChatGPT or any other LLM interface. The application is structured into four interconnected phases. During preparation, PICO and exclusion criteria are specified and source documents (PDFs or URLs) are ingested, with automated text extraction, tokenization, n‑gram counting and phrase detection—optionally supplemented by a TextRank summary. In the Query & Retrieval phase, optimized search strings for PubMed, Scopus and Web of Science are generated and executed, and all retrieved records are merged and harmonized into a unified dataset. The Deduplication & Screening phase applies exact‑match DOI filtering and fuzzy string deduplication before guiding reviewers through title/abstract and full‑text screening via interactive tables enriched with LLM prompts. Finally, Reporting produces a publication‑ready PRISMA flow diagram, an HTML screening report, automated study summaries generated by LLM prompts, and an automatic data extraction table for downstream synthesis. See Fig. 1 for detail phases. To evaluate the application’s feasibility, we applied it to our recently published systematic review and meta-analysis examining the effect of virtual reality in dentistry (Nezhad et al., 2024 ). Results Firstly, the inclusion and exclusion criteria of the systematic review were defined in the user interface menu titled “Inclusion and Exclusion Criteria” and saved in the system for subsequent steps. In the first preparatory phase, five articles from the final set of included studies were imported into the SPECTRUM SRA system as gold standards to identify 10 unigrams, 10 bigrams, and 10 trigrams. Consequently, the text-extraction pipeline—utilizing the pdftools and tokenizers packages—was initiated. This automated process took only five seconds. Next, to identify MeSH terms, the PICO elements of the systematic review were entered into the software’s “MeSH Terms” menu. The system completed the MeSH term retrieval using rentrez and rvest in 10 seconds and returned structured, relevant MeSH terms across the PICO categories. Overall, the first phase of the systematic review was completed in just 12 seconds using the software. After introducing the inclusion and exclusion criteria and identifying keywords and their corresponding MeSH terms via the user interface, the Search interface was activated. The identified search keyword “virtual reality” and its corresponding MeSH term “Virtual Reality” were combined with additional keywords, including teeth, tooth, and dental. The search was limited to studies published between January 1, 2020, and April 1, 2024 (the date the hand search was conducted). The document type was restricted to randomized controlled trials. After clicking “Generate Search Queries,” the system automatically generated optimized search strings for PubMed, Scopus, and Web of Science in less than three seconds. For instance, the PubMed search string was: ((("virtual reality"[tiab] OR "virtual reality"[mh])) AND ("tooth"[tiab] OR "teeth"[tiab] OR "dental"[tiab])) AND ("2020/01/01" : "2024/04/01"[dp]) AND ("Randomized Controlled Trial"[pt])). Then, by pressing the “Run Search” button, the system automatically executed the PubMed search using rentrez and retrieved 33 records. The searches in Scopus (via httr) and Web of Science returned 383 and 287 records, respectively. Retrieving and displaying results took approximately 30 seconds for PubMed and 4 minutes for Scopus. For Web of Science, the automatically generated search syntax was manually copied and executed in the WoS Advanced Search interface. The results were downloaded as an Excel file and uploaded into the Web of Science panel in the system. The entire WoS process was completed in one minute. Finally, merging and normalizing the combined 703 records from all sources took only one second, resulting in a single master table ready for deduplication. The deduplication pipeline consisted of two steps: one for studies with a DOI and another for studies without a DOI.In the DOI-based deduplication step, 679 records were processed, and 235 duplicates were identified and removed. The DOI-less deduplication step returned 24 records, from which 3 duplicates were removed. In total, 238 duplicates were removed, leaving 465 records after deduplication, totally in 3 seconds. For title and abstract screening, two reviewers were selected to screen the 465 articles. ChatGPT-o4-mini-high and DeepSeek v3 R1, due to their advanced reasoning and deep-thinking capabilities, were chosen as the first and second reviewers, respectively. The automatically generated prompt from the system was copied and pasted into the chatboxes of the LLMs, and the PDFs of articles were uploaded. For every batch of 10 articles, screening took 60 seconds using the ChatGPT model and 120 seconds using DeepSeek. In total, the screening process took 46.5 minutes for ChatGPT and 93 minutes for DeepSeek. For both reviewers, entering the screening decisions (Include/Exclude) into the system took an additional 5 seconds per record—approximately 38 minutes each. Therefore, the total time required for title and abstract screening was 84.5 minutes for reviewer 1 (ChatGPT) and 131 minutes for reviewer 2 (DeepSeek). The disagreements (12 cases) were identified in one second. These disagreements were then reviewed, and final decisions made. During title/abstract screening, 422 records were excluded, and 43 articles were included and transferred to the full-text screening phase. The full-text screening of 43 studies that passed the title and abstract screening took 43 minutes for Reviewer 1 (ChatGPT) and 90 minutes for Reviewer 2 (DeepSeek). The disagreement panel automatically identified two discrepancies; after resolving them, 25 studies met the inclusion criteria. Upon completing the title/abstract and full‑text screening steps, the system automatically generated the search and screening reports in under one second, as follows: The search was conducted on 23 July 2025. The number of retrieved studies in PubMed: 33. The number of retrieved studies in Scopus: 383. The number of retrieved studies in Web of Science: 287. The number of retrieved studies in other databases: 0. Total number of retrieved studies: 703 The number of duplicates: 238. The number of studies after removing duplicates: 465. Title/Abstract Screening: Number of studies screened: 465 Number of disagreements: 12 Number of studies excluded: 422 Kappa statistic between reviewers: 0.852 Full‑Text Screening: Number of studies screened: 43 Number of disagreements: 2 Number of studies excluded: 18 Kappa statistic between reviewers: 0.903 A PRISMA flow diagram was also automatically generated and displayed after the reasons for exclusion were entered. All 25 included studies were then processed for summarization and data extraction using an automatically generated prompt. ChatGPT o4‑mini‑high was selected for these tasks. First, the system‑generated prompt was copied into ChatGPT’s chatbox, and the PDF of each included study was uploaded sequentially. This phase took 60 seconds per study, completing in a total of 25 minutes. The extracted data were verified for accuracy, achieving a 91% accuracy rate. The most challenging elements to extract were funding sources and PICO details. Discussion In this study, we developed SPECTRUM SRA, an R Shiny application that provides an end-to-end workflow for the automation and semi‑automation of systematic reviews, from keyword selection through data extraction. We also evaluated the application’s feasibility. Our findings demonstrate that the application not only offers systematic reviewers a unified platform encompassing all essential stages of a systematic review but, owing to recent advancements in large language models (LLMs) (O’Connor et al., 2024 ) and their high accuracy in screening (Khraisha et al., 2024 ) and data extraction (Konet et al., 2024 ), can be relied upon to generate accurate outputs. SPECTRUM SRA includes eight core models: search, retrieval, deduplication, screening, PRISMA flow diagram generation, reporting, summarization, and data extraction withing a singe Shiny application. By eliminating the need for navigating between different platforms/models/applications, the application reduces context‑switching overhead and reduces the cognitive load on reviewers. The system supported article retrieval and deduplication, executed in some few seconds. Numerous deduplication algorithms and tools exist—for example, “The Duplicator” (Forbes et al., 2024 ). Although this tool can accelerate the deduplication process, its accuracy requires further evaluation (Forbes et al., 2024 ). Accuracy varies across platforms, and selection often depends on interface features (Guimarães et al., 2022 ). In our platform, we designed a user‑friendly interface that enables automatic deduplication in just three clicks, completing the task in a matter of seconds. The system provides LLM‑based automated prompts for the screening, summarization, and data‑extraction phases. We selected two models—ChatGPT o4‑mini‑high and DeepSeek V3 R1—for these tasks. Thanks to recent advances in evidence‑synthesis automation and the high accuracy of LLMs in both screening and data extraction, our application can substantially reduce the labor intensity and time required to complete systematic reviews. We employed these two LLMs as the first and second reviewers; although we achieved a kappa statistic of 0.852 for title/abstract screening and 0.903 for full‑text screening, disagreements still occur and should be resolved by human reviewers to ensure the validity of final decisions. Moreover, LLM‑generated outputs must be interpreted with caution, as recent studies have noted potential errors (Galli, Anna V. Gavrilova, et al., 2025; Rokhshad et al., 2025 ). Several other LLM‑based platforms for systematic‑review automation have also been developed—for example, Otto SR, which supports both screening and data extraction, reaching accuracy more than 90% for both screening and data extraction (Cao, Arora, et al., 2025 ). Perhaps the most important benefit of this application is the reduction in review time—what once took several months can now be completed in a matter of days. By automating numerous steps in the systematic‑review process, the application frees domain experts to concentrate on high‑value activities such as clinical interpretation, critical appraisal, and evidence synthesis. For instance, some researchers presented a case study demonstrating that a systematic review could be completed in two weeks by employing automation tools (Clark et al., 2020 ). This study has several limitations that warrant cautious interpretation. First, the application currently supports only three electronic databases—PubMed, Scopus, and Web of Science. Although users can upload search results from other sources, this step must be performed manually. Additionally, Web of Science queries are not automated via API; users must run searches in an external portal and import the results. Because screening, summarization, and data extraction rely on LLMs, outputs remain susceptible to limitations such as hallucinations. During testing, we also encountered occasional server errors from PubMed and Scopus. Finally, our feasibility assessment was preliminary; broad generalizability will require evaluation across diverse review topics, disciplines, and database configurations. To enhance the application’s quality, we plan several future extensions. First, we will integrate automatic searching in Web of Science via API keys, mirroring Scopus implementation. Next, we will enable automatic search and article retrieval from additional electronic databases—such as the Cochrane Library, Embase, trial registries (e.g., ClinicalTrials.gov), and preprint servers. Finally, we will incorporate LLM‑based automation for risk‑of‑bias assessment. Conclusion In conclusion, the SPECTRUM SRA application—which supports multiple systematic-review tasks, including keyword identification, search, article retrieval, deduplication, title/abstract screening, full‐text screening, PRISMA flow‐diagram generation, summarization, and data extraction—is an AI‑augmented tool that streamlines the review process. Its use can substantially reduce both the time and workload required for conducting systematic reviews. Because large language models have inherent limitations, we recommend employing at least two models with advanced reasoning and deep‐thinking capabilities (for example, ChatGPT o4‑mini‑high and DeepSeek V3 R1) as the first and second reviewers, respectively, along with a human reviewer as a third check to ensure reliability. References Abstrackr (2025) (n.d.) Abstrackr . [Online] [online]. Available from: http://abstrackr.cebm.brown.edu/account/login (Accessed 12 June Alshami A, Elsayed M, Ali E, Eltoukhy AEE, Zayed T (2023) Harnessing the Power of ChatGPT for Automating Systematic Review Process: Methodology, Case Study, Limitations, and Future Directions. Systems 11(7):351 B S, B PPR, S.M., S K (2024) ‘The Evolution of Large Language Model: Models, Applications and Challenges’, in 2024 International Conference on Current Trends in Advanced Computing (ICCTAC) . [Online]. 8 May 2024 IEEE. pp. 1–8 Blasingame MN, Koonce TY, Williams AM, Giuse DA, Su J, Krump PA, Giuse NB (2024) Evaluating a Large Language Model’s Ability to Answer Clinicians’ Requests for Evidence Summaries Borah R, Brown AW, Capers PL, Kaiser KA (2017) Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ Open 7(2):e012545 Bramer WM, Giustini D, de Jonge GB, Holland L, Bekhuis T (2016) De-duplication of database search results for systematic reviews in EndNote. J Med Libr Association: JMLA 104(3):240–243 Cao C, Arora R, Cento P, Manta K, Farahani E, Cecere M, Selemon A, Sang J, Gong LX, Kloosterman R, Jiang S, Saleh R, Margalik D, Lin J, Jomy J, Xie J, Chen D, Gorla J, Lee S et al (2025) Automation of Systematic Reviews with Large Language Models Cao C, Sang J, Arora R, Chen D, Kloosterman R, Cecere M, Gorla J, Saleh R, Drennan I, Teja B, Fehlings M, Ronksley P, Leung AA, Weisz DE, Ware H, Whelan M, Emerson DB, Arora RK, Bobrovitz N (2025) Development of Prompt Templates for Large Language Model–Driven Screening in Systematic Reviews. Ann Intern Med 178(3):389–401 Chai KEK, Lines RLJ, Gucciardi DF, Ng L (2021) Research Screener: a machine learning tool to semi-automate abstract screening for systematic reviews. Syst Reviews 10(1):93 Clark J, Glasziou P, Del Mar C, Bannach-Brown A, Stehlik P, Scott AM (2020) A full systematic review was completed in 2 weeks using automation tools: a case study. J Clin Epidemiol 121:81–90 Covidence (2025) (n.d.) Covidence . [Online] [online]. Available from: https://www.covidence.org/ (Accessed 10 June Cumpston M, Li T, Page MJ, Chandler J, Welch VA, Higgins JP, Thomas J (2019) Updated guidance for trusted systematic reviews: a new edition of the Cochrane Handbook for Systematic Reviews of Interventions. Cochrane Database of Systematic Reviews DistillerSR (n.d.) DistillerSR . [Online] [online]. Available from: https://www.distillersr.com/ (Accessed 9 June 2025) Djulbegovic B, Guyatt GH (2017) Progress in evidence-based medicine: a quarter century on. Lancet 390(10092):415–423 Forbes C, Greenwood H, Carter M, Clark J (2024) Automation of duplicate record detection for systematic reviews: Deduplicator. Syst Reviews 13(1):206 Galli C, Gavrilova AV, Calciolari E (2025) Large Language Models in Systematic Review Screening: Opportunities, Challenges, and Methodological Considerations. Information 16(5):378 Galli C, Gavrilova A Viktorovna, Calciolari E (2025) Integrating Large Language Models into Systematic Review Screening Guimarães NS, Ferreira AJF, Ribeiro Silva R, de Paula C, Lisboa AA, Magno CS, Ichiara L, M.Y., Barreto ML (2022) Deduplicating records in systematic reviews: there are free, accurate automated ways to do so. J Clin Epidemiol 152:110–115 Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ (2022) W.V. Cochrane Handbook for Systematic Reviews of Interventions version 6.3 (updated February 2022) Hoffmann F, Allers K, Rombey T, Helbach J, Hoffmann A, Mathes T, Pieper D (2021) Nearly 80 systematic reviews were published each day: Observational study on trends in epidemiology and reporting over the years 2000–2019. J Clin Epidemiol 138:1–11 Hossain MM (2024) Using ChatGPT and other forms of generative AI in systematic reviews: Challenges and opportunities. J Med Imaging Radiation Sci 55(1):11–12 Isfeedvajani MS (2016) Systematic Review and its Role in Clinical Decision Making. Int J Med Rev 3(4):495–496 Jardim PSJ, Rose CJ, Ames HM, Echavez JFM, Van de Velde S, Muller AE (2022) Automating risk of bias assessment in systematic reviews: a real-time mixed methods comparison of human researchers to a machine learning system. BMC Med Res Methodol 22(1):167 Khalil H, Ameen D, Zarnegar A (2022) Tools to support the automation of systematic reviews: a scoping review. J Clin Epidemiol 144:22–42 Khraisha Q, Put S, Kappenberg J, Warraitch A, Hadfield K (2024) Can large language models replace humans in systematic reviews? Res Synthesis Methods 15(4):616–626 Kilicoglu H, Demner-Fushman D, Rindflesch TC, Wilczynski NL, Haynes RB (2009) Towards Automatic Recognition of Scientifically Rigorous Clinical Research Evidence. J Am Med Inform Assoc 16(1):25–31 Konet A, Thomas I, Gartlehner G, Kahwati L, Hilscher R, Kugley S, Crotty K, Viswanathan M, Chew R (2024) Performance of two large language models for data extraction in evidence synthesis. Res Synthesis Methods 15(5):818–824 Marshall IJ, Kuiper J, Wallace BC (2016) RobotReviewer: evaluation of a system for automatically assessing bias in clinical trials. J Am Med Inform Assoc 23(1):193–201 Marshall IJ, Wallace BC (2019) Toward systematic review automation: a practical guide to using machine learning tools in research synthesis. Syst Reviews 8(1):163 Michelson M, Reuter K (2019) ‘The significant cost of systematic reviews and meta-analyses: A call for greater involvement of machine learning to assess the promise of clinical trials’. Contemp Clin Trials Commun, 16p. 100443 Motzfeldt Jensen M, Brix Danielsen M, Riis J, Assifuah Kristjansen K, Andersen S, Okubo Y, Jørgensen MG (2025) ChatGPT-4o can serve as the second rater for data extraction in systematic reviews’ Weiqiang (Albert) Jin. PLoS ONE 20(1):e0313401 Nezhad HM, Ashourioun A, Sadeghdaghighi A (2024) The effect of virtual reality for anxiety and pain in dentistry: A systematic review and meta-analysis. Community Dent Health 41(4):248–255 Nykvist B, Macura B, Xylia M, Olsson E (2025) Testing the utility of GPT for title and abstract screening in environmental systematic evidence synthesis. Environ Evid 14(1):7 O’Connor AM, Clark J, Thomas J, Spijker R, Kusa W, Walker VR, Bond M (2024) Large language models, updates, and evaluation of automation tools for systematic reviews: a summary of significant discussions at the eighth meeting of the International Collaboration for the Automation of Systematic Reviews (ICASR). Syst Reviews 13(1):290 O’Connor AM, Tsafnat G, Gilbert SB, Thayer KA, Wolfe MS (2018) Moving toward the automation of the systematic review process: a summary of discussions at the second meeting of International Collaboration for the Automation of Systematic Reviews (ICASR). Syst Reviews 7(1):3 O’Mara-Eves A, Thomas J, McNaught J, Miwa M, Ananiadou S (2015) Using text mining for study identification in systematic reviews: a systematic review of current approaches. Syst Reviews 4(1):5 Ossom Williamson P, Minter CIJ (2019) ‘Exploring PubMed as a reliable resource for scholarly communications services’. J Med Libr Association, 107(1) Park SE, Thomas J (2018) Evidence synthesis software. BMJ Evidence-Based Med 23(4):140–141 Qureshi R, Shaughnessy D, Gill KAR, Robinson KA, Li T, Agai E (2023) Are ChatGPT and large language models the answer to bringing us closer to systematic review automation? Syst Reviews 12(1):72 Radenkovic D, Keogh SB, Maruthappu M (2019) Data science in modern evidence-based medicine. J R Soc Med 112(12):493–494 Rayyan (2025) (n.d.) Rayyan . [Online] [online]. Available from: https://www.rayyan.ai/ (Accessed 3 June Rethlefsen ML, Brigham TJ, Price C, Moher D, Bouter LM, Kirkham JJ, Schroter S, Zeegers MP (2024) ‘Systematic review search strategies are poorly reported and not reproducible: a cross-sectional metaresearch study’. J Clin Epidemiol, 166p. 111229 RobotAnalyst (2025) (n.d.) RobotAnalyst . [Online] [online]. Available from: https://nactem.ac.uk/RA/login (Accessed 11 June Rokhshad R, Bagherianlemraski M, Ehsani SS, Haghighat S, Schwendicke F (2025) ‘Large language models for the screening step in systematic reviews in dentistry’. J Dent, 160p. 105877 Scherbakov D, Hubig N, Jansari V, Bakumenko A, Lenert LA (2025) The emergence of large language models as tools in literature reviews: a large language model-assisted systematic review. J Am Med Inform Assoc 32(6):1071–1086 Schmidt L, Finnerty Mutlu AN, Elmore R, Olorisade BK, Thomas J, Higgins JPT (2023) ‘Data extraction methods for systematic review (semi)automation: Update of a living systematic review’, F1000Research , 10p. 401 van de Schoot R, de Bruin J, Schram R, Zahedi P, de Boer J, Weijdema F, Kramer B, Huijts M, Hoogerwerf M, Ferdinands G, Harkema A, Willemsen J, Ma Y, Fang Q, Hindriks S, Tummers L, Oberski DL (2021) An open source machine learning framework for efficient and transparent systematic reviews. Nat Mach Intell 3(2):125–133 Shojania KG, Sampson M, Ansari MT, Ji J, Doucette S, Moher D (2007) How Quickly Do Systematic Reviews Go Out of Date? A Survival Analysis. Ann Intern Med 147(4):224–233 Topol EJ (2019) High-performance medicine: the convergence of human and artificial intelligence. Nat Med 25(1):44–56 Tsafnat G, Glasziou P, Choong MK, Dunn A, Galgani F, Coiera E (2014) Systematic review automation technologies. Syst Reviews 3(1):74 Tsou AY, Treadwell JR, Erinoff E, Schoelles K (2020) Machine learning for screening prioritization in systematic reviews: comparative performance of Abstrackr and EPPI-Reviewer. Syst Reviews 9(1):73 Wallace BC, Kuiper J, Sharma A, Zhu M, Brian, Marshall IJ (2016) Extracting PICO Sentences from Clinical Trial Reports using Supervised Distant Supervision. J Mach Learn Res 176(25):139–148 Wallace SS, Barak G, Truong G, Parker MW (2022) Hierarchy of Evidence Within the Medical Literature. Hosp Pediatr 12(8):745–750 Wang S, Scells H, Koopman B, Zuccon G (2022) ‘Automated MeSH term suggestion for effective query formulation in systematic reviews literature search’. Intell Syst Appl, 16p. 200141 Wang S, Scells H, Zhuang S, Potthast M, Koopman B, Zuccon G (2024) Zero-Shot Generative Large Language Models for Systematic Review Screening Automation , in [Online]. pp. 403–420 Zaki M, Namireddy SR, Pittie T, Bihani V, Keshri SR, Venugopal V, Gosvami NN, Jayadeva, Krishnan NMA (2022) ‘Natural language processing-guided meta-analysis and structure factor database extraction from glass literature’. J Non-Crystalline Solids: X, 15p. 100103 Additional Declarations The authors declare no competing interests. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7265470","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":493952332,"identity":"1e7f7057-b5e6-412b-8388-e81781f6f492","order_by":0,"name":"Hossein Motahari Nezhad","email":"","orcid":"https://orcid.org/0000-0002-1028-4460","institution":"Department of Knowledge and Information Science, University of Isfahan, Isfahan, Iran","correspondingAuthor":false,"prefix":"","firstName":"Hossein","middleName":"Motahari","lastName":"Nezhad","suffix":""},{"id":493952333,"identity":"9a76ad70-ecde-41e0-a94f-8748befb8aa0","order_by":1,"name":"Mehrdad CheshmehSohrabi","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA6UlEQVRIiWNgGAWjYLACxgYGBgNm5gMMPEAOG1jIgCgtbAmkamHgMQBrIQh02xuYP3zccU/enJ3n44O3bQyJfQzMDz8wFNzDqcXszAE2yZlnig13NvNuNpwL1NLGwGYswWBQjFvLjQQ2Zt62BMYNh3m3SfOCtTCYAd2ZgE8L82egFvsNh3me/4ZoYf9GSAsD0PCERKAWkHUgLTwEbDlzsE1yZltC8s5mNmPJOeckjNuYeYolEvBpOd58+MPHtgTb7fyHH354U2YjO7+9feOHD39wa4FECgJIMDAwAyl8GkbBKBgFo2AUEAYANTtLxFxIuEEAAAAASUVORK5CYII=","orcid":"https://orcid.org/0000-0003-1856-4210","institution":"Department of Knowledge and Information Science, University of Isfahan, Isfahan, Iran","correspondingAuthor":true,"prefix":"","firstName":"Mehrdad","middleName":"","lastName":"CheshmehSohrabi","suffix":""}],"badges":[],"createdAt":"2025-07-31 20:38:04","currentVersionCode":1,"declarations":{"humanSubjects":false,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-7265470/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7265470/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":88212254,"identity":"71b33bf1-897a-41c2-9a3c-5a9b62387e27","added_by":"auto","created_at":"2025-08-04 05:44:59","extension":"jpeg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":209211,"visible":true,"origin":"","legend":"\u003cp\u003eWorkflow of SPECTRUM SRA application\u003c/p\u003e","description":"","filename":"floatimage1.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-7265470/v1/58ddd9ef45abb1bfc216550f.jpeg"},{"id":88213074,"identity":"688a11fa-8c05-49c7-aa97-d7a9e39a45ca","added_by":"auto","created_at":"2025-08-04 06:00:59","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":754215,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7265470/v1/9050db8f-94a1-4efc-8795-6eeb0bc243a3.pdf"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003e\u003cstrong\u003eSPECTRUM SRA: An R Shiny Application to Automate Systematic Reviews Using Artificial Intelligence and Large Language Models\u003c/strong\u003e\u003c/p\u003e","fulltext":[{"header":"Introduction","content":"\u003cp\u003eEvidence-based medicine (EBM) underpins contemporary health policy and patient-care decision making, drawing on the most recent and relevant literature (Djulbegovic \u0026amp; Guyatt, \u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e2017\u003c/span\u003e; Radenkovic et al., \u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e2019\u003c/span\u003e). However, the volume of publications across disciplines is rising so quickly that keeping pace has become increasingly time-consuming\u0026mdash;often exceeding the capacity of individual researchers (Topol, \u003cspan citationid=\"CR49\" class=\"CitationRef\"\u003e2019\u003c/span\u003e). Because new papers appear daily in every domain, maintaining up-to-date systematic reviews is particularly challenging. Among the various forms of literature synthesis, systematic reviews follow explicit, predefined steps to locate and summarize evidence for a specific field and research question; their rigorous, current findings therefore remain essential for sound health policy and clinical decisions (Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, 2022).\u003c/p\u003e\u003cp\u003eSystematic reviews, owing to their explicit, predefined methodologies, sit at the pinnacle of the evidence pyramid and are considered the reference standard for evidence synthesis (Wallace et al., \u003cspan citationid=\"CR53\" class=\"CitationRef\"\u003e2022\u003c/span\u003e). Clinicians and other health professionals rely on them to stay abreast of the latest findings (Isfeedvajani, \u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e2016\u003c/span\u003e). To produce a review, authors must progress through a series of prescribed steps\u0026mdash;formulating a research question; defining inclusion and exclusion criteria; systematically searching electronic databases; screening records; extracting data; and synthesizing the evidence\u0026mdash;each of which contains further sub-tasks that add to the overall complexity. The breadth and quality of a review hinge, first and foremost, on the thoroughness of its database searches. According to the Cochrane Handbook, PubMed is among the most comprehensive and frequently consulted sources for healthcare reviews (Ossom Williamson \u0026amp; Minter, \u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e2019\u003c/span\u003e), but authors are advised to search at least two databases. In practice, PubMed, Scopus, and Web of Science are the three most commonly used resources for biomedical systematic reviews. Systematic reviews are highly time-consuming and resource-intensive: completing one manually can take several months\u0026mdash;or even years (Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, 2022). The first challenge of the traditional approach is its substantial demand for time, labor, and funding. Estimates indicate that producing and publishing a single review may require more than five years, a team of five reviewers, and a budget of about US \u003cspan\u003e$\u003c/span\u003e140 000 (Michelson \u0026amp; Reuter, \u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e2019\u003c/span\u003e; Borah et al., \u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e2017\u003c/span\u003e). Although recently developed automation and semi-automation tools can cut the timeline dramatically\u0026mdash;sometimes to as little as two weeks (Clark et al., \u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e2020\u003c/span\u003e)\u0026mdash;no single tool yet automates every review step within one unified platform. These resourcing pressures are intensifying as the sheer number of systematic reviews\u0026mdash;and the evidence they must cover\u0026mdash;continues to surge.\u003c/p\u003e\u003cp\u003eThe number of published systematic reviews is also climbing rapidly; daily output rose from 80 in 2019 (Hoffmann et al., \u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e2021\u003c/span\u003e) to 135 in 2021 (Rethlefsen et al., \u003cspan citationid=\"CR42\" class=\"CitationRef\"\u003e2024\u003c/span\u003e), a 1.7-fold increase. A simple PubMed search shows that more than 37 000 systematic reviews appeared in 2024 alone. According to several studies, the typical lifetime of a review is about 5.5 years, after which an update is required to capture the latest literature, highlighting the need for better management and partial automation of review steps (Shojania et al., \u003cspan citationid=\"CR48\" class=\"CitationRef\"\u003e2007\u003c/span\u003e). Yet updating remains difficult: a primary study often takes 2.5\u0026ndash;6.5 years to be incorporated into a review, creating a sizeable evidence-to-practice lag. The Cochrane Handbook therefore recommends updating every two years (Cumpston et al., \u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e2019\u003c/span\u003e). Nevertheless, roughly one quarter of reviews become obsolete within their first two years because newly published evidence is not incorporated, a lapse that can materially alter conclusions (Shojania et al., \u003cspan citationid=\"CR48\" class=\"CitationRef\"\u003e2007\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eSystematic reviews comprise numerous steps, each with its own workload, and if reviewers attempt to work by hand at every stage, completing a review can become practically impossible in the face of today\u0026rsquo;s knowledge explosion. These complexities and time-consuming tasks can be managed or minimized by employing artificial-intelligence (AI) techniques. Numerous fully and semi-automated tools have therefore been established to expedite the completion of systematic reviews. As noted above, the traditional process remains time-consuming, labor-intensive, and financially demanding. In situations such as health emergencies, for example, Covid-19 or whenever decision makers need rapid evidence for treatment, the conventional approach to systematic reviews does not suffice. Consequently, researchers have begun to evaluate which parts of the workflow can be delegated to AI-driven systems and which still require human oversight.\u003c/p\u003e\u003cp\u003eEmploying artificial-intelligence technologies and programming has led to the creation of a range of platforms, software, tools, and models that accelerate numerous steps of systematic reviews, expand reviewer capacity, minimize errors, and ultimately improve reproducibility. Because the philosophy and specifications of each review step differ, the automation or semi-automation of every stage must follow its own strategy (O\u0026rsquo;Mara-Eves et al., \u003cspan citationid=\"CR36\" class=\"CitationRef\"\u003e2015\u003c/span\u003e). The adoption of AI automation now has formal backing: the Cochrane Handbook, for example, recognizes the use of such tools\u0026mdash;sometimes as a second reviewer (Cumpston et al., \u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e2019\u003c/span\u003e). Even in fully manual reviews, accuracy varies with reviewers\u0026rsquo; expertise and diligence. With two reviewers, about 3% of eligible studies are still missed; the figure rises to 13% when a single reviewer works alone (Blasingame et al., \u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e2024\u003c/span\u003e). These limitations have prompted the use of AI for automation and semi-automation of different review steps, although most automated workflows still depend on human judgment. Persistent human error, even with dual reviewers, necessitates for end-to-end platforms that can automate, semi-automate, and manage the entire systematic-review process (Zaki et al., \u003cspan citationid=\"CR56\" class=\"CitationRef\"\u003e2022\u003c/span\u003e; Wang et al., \u003cspan citationid=\"CR55\" class=\"CitationRef\"\u003e2024\u003c/span\u003e; Marshall \u0026amp; Wallace, \u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e2019\u003c/span\u003e). Therefore, the main aim of this study is to design an R Shiny end-to-end application to automate or semi-automate the main steps of systematic reviews including search, article retrieval, deduplication, screening, reporting, Prisma graph generation, summarization and data extraction.\u003c/p\u003e"},{"header":"Literature Review","content":"\u003cp\u003eEfforts to automate systematic reviews date back to at least the early 2000s. By that time, the explosion of published research had made manual evidence synthesis increasingly unsustainable. Through the mid-2010s, research in systematic review automation accelerated. A variety of tasks in the systematic review workflow were targeted for automation. In 2014, Tsafnat and colleagues published a landmark survey paper which mapped out the state of the art at that time (Tsafnat et al., \u003cspan citationid=\"CR50\" class=\"CitationRef\"\u003e2014\u003c/span\u003e). The researchers argued that the creative aspects of review design would always require human expertise, but many technical tasks could eventually be executed by machines. Around 2015\u0026ndash;2016, Marshall et al. developed RobotReviewer, an NLP/ML system to automatically evaluate risk of bias in randomized trial reports (Marshall et al., \u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e2016\u003c/span\u003e). An evaluation in 2016 showed RobotReviewer\u0026rsquo;s assessments agreed with human reviewers\u0026rsquo; judgments much of the time (Marshall et al., \u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e2016\u003c/span\u003e). However, machine learning-based automation systems of systematic reviews can have limitations in the data-extraction phase and may underperform on complex tasks (Schmidt et al., \u003cspan citationid=\"CR46\" class=\"CitationRef\"\u003e2023\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eIn the context of systematic reviews, NLP methods are applied to a variety of tasks such as information retrieval, study characterization, data extraction, and summarization. Fundamentally, much of the automation pipeline deals with unstructured text (journal articles), so NLP underlies many tools in this domain. A core NLP task in systematic reviews is classifying documents into categories, often to decide relevance. Early approaches used simple keyword-based or rule-based systems, but modern methods leverage machine learning to learn classification from examples. Beyond identifying which references to include, systematic reviews require pulling out specific data from full-text articles \u0026ndash; e.g. sample sizes, patient characteristics, interventions, outcome results, etc. NLP methods have been developed to automate parts of the data extraction step. A common focus is on identifying PICO elements (Population, Intervention, Comparator, Outcome) within an article. For example, one approach by Wallace et al. used supervised NLP to highlight sentences in trial reports corresponding to each PICO element (Wallace et al., \u003cspan citationid=\"CR52\" class=\"CitationRef\"\u003e2016\u003c/span\u003e). In evaluations, automated extraction often has only moderate agreement with human extractors and may miss context needed for correct interpretation (Jardim et al., \u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e2022\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eAnother domain where NLP plays a role is in the search stage of a systematic review is for literature search and retrieval. Traditional search strategies rely on Boolean combinations of keywords and indexing terms. NLP and text mining can enhance this in several ways. One is automatic term suggestion. Tools can analyze a set of known relevant articles and suggest additional keywords or MeSH terms that might retrieve similar studies (Wang et al., \u003cspan citationid=\"CR54\" class=\"CitationRef\"\u003e2022\u003c/span\u003e). Moreover, NLP can aid in deduplication of search results by comparing titles and abstracts to identify the same study reported in multiple sources (Bramer et al., \u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e2016\u003c/span\u003e). A forward-looking application of NLP in systematic reviews is the automatic summarization of evidence. Some prototype systems have been able to auto-generate simple summary paragraphs from meta-analytic data. In recent times, large language models (LLMs) have demonstrated the ability to produce fluent, coherent text, raising the question of whether they could draft systematic review summaries. In early explorations, ChatGPT was able to \u003cem\u003e\u0026ldquo;identify and summarize relevant information\u0026rdquo;\u003c/em\u003e from a small set of abstracts (Qureshi et al., \u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e2023\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eIn systematic reviews, ML approaches have been applied across nearly all stages: from screening and study selection to data extraction and meta-analysis. Early attempts used algorithms like support vector machines or logistic regression on hand-engineered text features. One notable supervised ML system was Abstrackr, introduced around 2012 (Abstrackr, n.d.). The limitations and considerations include the need for an initial training set (often reviewers label a few hundred citations first), and the risk of missing studies if the model\u0026rsquo;s predictions are trusted too much (Tsou et al., \u003cspan citationid=\"CR51\" class=\"CitationRef\"\u003e2020\u003c/span\u003e). Active learning is implemented in tools like ASReview (Active Supervision for systematic Reviews), which is a more recent open-source tool (van de Schoot et al., \u003cspan citationid=\"CR47\" class=\"CitationRef\"\u003e2021\u003c/span\u003e). It provides a modern interface for active learning in screening. Deciding appropriate stopping criteria (stopping too early could omit relevant studies), and the fact that active learning algorithms may need some tuning depending on the prevalence of relevant studies and the inclusion criteria complexity have resulted in limitations and challenges in these strategies (Kilicoglu et al., \u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e2009\u003c/span\u003e). RobotAnalyst is another tool (developed by the EPPI-Centre, UCL) that offers a suite of text mining functions for reviews. It can automatically classify studies (with various pre-trained models) and cluster references into topics, helping reviewers target relevant clusters (RobotAnalyst, n.d.).\u003c/p\u003e\u003cp\u003eCovidence is a cloud-based platform endorsed by Cochrane and used by many institutions to streamline study selection and data extraction (Covidence, n.d.). Covidence ensured this automation is \u003cem\u003eoptional\u003c/em\u003e and transparent \u0026ndash; users can review the auto-excluded citations and restore any if needed, however, Covidence remains dependent on human decisions and are not fully automated, even if they do save time (Scherbakov et al., \u003cspan citationid=\"CR45\" class=\"CitationRef\"\u003e2025\u003c/span\u003e). DistillerSR is another popular commercial platform (DistillerSR, n.d.). DistillerSR offers a system for managing large reviews with complex forms, and in recent years it has added a module called \u003cem\u003eDistillerAI\u003c/em\u003e. DistillerAI includes an ML model that learns from a subset of screening decisions and then \u003cem\u003esuggests\u003c/em\u003e includes/excludes for the remaining citations, along with a confidence score. Rayyan deserves special mention as a platform developed by Qatar Computing Research Institute (Rayyan, n.d.). Rayyan\u0026rsquo;s primary function is to facilitate title/abstract screening with a sleek interface and the ability to blind multiple reviewers to each other\u0026rsquo;s decisions. A limitation is that its ML suggestions are relatively simplistic and not as transparent or controllable as some academic tools. Tools such as Rayyan, Abstractr, and Research Screener (Chai et al., \u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e2021\u003c/span\u003e) assist with screening, deduplication, and resource management, yet they rely heavily on human feedback to train their models, which requires a lengthy training phase for reviewers (Khalil et al., \u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e2022\u003c/span\u003e; Galli, Anna Viktorovna Gavrilova, et al., 2025).\u003c/p\u003e\u003cp\u003eEPPI-Reviewer is a software mainly used by information specialists (Park \u0026amp; Thomas, \u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e2018\u003c/span\u003e). It has incorporated text mining modules and classifiers. For example, EPPIReviewer can automatically classify studies by type (like RCT or observational). It also has features like term extraction, auto-clustering. The advantage of EPPI-Reviewer is that it\u0026rsquo;s very powerful and flexible \u0026ndash; it can handle very large datasets and unusual review designs. The drawback is that it can be less user-friendly to non-experts and historically had a steeper learning curve (Tsou et al., \u003cspan citationid=\"CR51\" class=\"CitationRef\"\u003e2020\u003c/span\u003e). In addition to full-featured platforms, there are specialized tools targeting specific systematic review tasks. RobotReviewer is one focusing on automating risk-of-bias assessment or data extraction (Marshall et al., \u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e2016\u003c/span\u003e). Quick Clinical is a federated search engine, but is a limited source databases and is not optimized for systematic reviews, or Sherlock is search engine specially for trial registries working only for clinicaltrials.gov database. ExaCT is also for extracting PICO elements of articles (Tsafnat et al., \u003cspan citationid=\"CR50\" class=\"CitationRef\"\u003e2014\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eThe recent shift from traditional machine learning to large language models, driven by extensive developments in natural-language processing (B et al., \u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e2024\u003c/span\u003e), has positively revolutionized the automation of systematic reviews (Cao, Sang, et al., \u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e2025\u003c/span\u003e). With the rapid advancements in large language models (LLMs) in recent years, the hope for revolutionizing the conduction of systematic reviews has increased\u0026mdash;to improve speed and accuracy, streamline processes, and ultimately save time and energy\u0026mdash;although their use in automating systematic reviews must still be interpreted with caution (Alshami et al., \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2023\u003c/span\u003e). The application of LLMs in various stages of systematic review automation has been increasingly explored. For example, a recent study evaluated the performance of ChatGPT-4 in the data extraction phase of systematic reviews and found that it could serve as a secondary reviewer with an accuracy exceeding 94% (Motzfeldt Jensen et al., \u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e2025\u003c/span\u003e). Another recent study investigated the use of ChatGPT for title and abstract screening, demonstrating high accuracy in this task as well (Nykvist et al., \u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e2025\u003c/span\u003e). otto-SR is also a LLM-based workflow to automate systematic reviews supporting screening and data extraction (Cao, Arora, et al., \u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e2025\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eStudies further indicate that the use of AI-automation tools in systematic-review workflows is not yet pervasive, mainly because limited usability and lack of user-friendliness hinder adoption (Hossain, \u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e2024\u003c/span\u003e). Another barrier is the steep learning curve associated with many automation tools (O\u0026rsquo;Connor et al., \u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e2018\u003c/span\u003e). Finally, because there are separate tools for each review step, reviewers must navigate among multiple, non-interoperable systems (Khalil et al., \u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e2022\u003c/span\u003e). These obstacles help explain why enthusiasm for automation co-exists with persistent practical frustrations among reviewers. As discussed in this literature review, a variety of tools, platforms, and models have been developed to automate or semi-automate different stages of the systematic review process. Many of these tools focus on a single task, such as automating or semi-automating the search process, screening, or data extraction. Others offer techniques that assist reviewers but still rely heavily on human involvement. Consequently, there is a clear need for a comprehensive, end-to-end tool or platform that can support the automation or semi-automation of key tasks in systematic reviews, including literature search, articles retrieval, deduplication, screening, PRISMA flow diagram generation, data extraction, and summarization. Therefore, the primary aim of this study is to develop a Shiny application that automates the main tasks involved in conducting systematic reviews. This tool is designed to assist systematic reviewers in managing their review process, while also automating or semi-automating key stages. By providing a user-friendly interface, the application aims to streamline the systematic review workflow, reduce the time and labor required, and ultimately accelerate evidence generation and heath policy and decision making in medicine.\u003c/p\u003e"},{"header":"Methods","content":"\u003cp\u003eSPECTRUM SRA (Systematic Review Automation) is an R shiny application comprises eight modules including Search, Prisma diagram, dEduplication, sCreening, daTa extraction, Reporting, qUery, and summarization to automate numerous steps of systematic reviews. The application can be accessed using the following webpage link:\u003c/p\u003e\u003cp\u003e\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://motaharinezhad.shinyapps.io/SPECTRUM_SRA/\u003c/span\u003e\u003cspan address=\"https://motaharinezhad.shinyapps.io/SPECTRUM_SRA/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. It integrates data from multiple academic databases, including PubMed, Scopus, and Web of Science, allowing researchers to efficiently collect and manage large volumes of relevant literature. The system also supports screening and decision-making processes, with built-in mechanisms for managing disagreements between reviewers. Furthermore, the system generates comprehensive reports, including PRISMA flow diagrams, which visually represent the study selection process, and provides outputs for further analysis. In this section, we will provide a description of the methods developed for each part of the system.\u003c/p\u003e\u003cp\u003e\u003cb\u003eInclusion and exclusion criteria\u003c/b\u003e\u003c/p\u003e\u003cp\u003eThe \"Inclusion \u0026amp; Exclusion Criteria\" menu is designed as the first and most important part of the system, ensuring that only relevant studies are included in the systematic review process. This section enables authors and researchers to clearly define the criteria for including or excluding studies based on predetermined guidelines. The section is divided into two primary parts:\u003c/p\u003e\u003cp\u003eInclusion Criteria: In this part, the authors must specify the conditions that a study must meet in order to be included in the systematic review. This could include specific study characteristics, such as the population studied, the type of intervention, comparator, outcome(s), study design, or any other relevant factors.\u003c/p\u003e\u003cp\u003eExclusion Criteria: In this part, the authors define the criteria for excluding studies from the review. This could include studies that do not meet the inclusion criteria, studies with methodological issues, or studies that focus on irrelevant populations or interventions.\u003c/p\u003e\u003cp\u003eThe defined inclusion and exclusion criteria are stored in the system and used to guide the review process in screening process.\u003c/p\u003e\u003cp\u003e\u003cb\u003eKeywords Identification\u003c/b\u003e\u003c/p\u003e\u003cp\u003eThe \"Keywords Identification\" menu in the system is a helpful tool for researchers to extract meaningful keywords from the documents they are working with. This part of the system is designed to automate the extraction of unigrams, bigrams, and trigrams from gold standards uploaded PDF documents, which are then displayed for the users to review. This is particularly useful in systematic reviews where keywords are critical for refining search terms and understanding the content of the research articles. At the core of this menu is the file upload section, which allows users to upload one or more PDF files of gold standard articles from which keywords will be extracted. Once the files are uploaded, the system enters the processing and extraction phase. During this phase, the uploaded PDFs are processed to extract the text contained within them. The system first cleans this text by removing unnecessary elements like stop words and punctuation. It then performs keyword extraction, identifying the most frequent unigrams (single words), bigrams (two-word combinations), and trigrams (three-word combinations) within the text. The frequency of each of these keywords is counted, allowing the system to identify the most common terms within the documents. This structure helps users to quickly identify the most relevant terms from the gold standards to search related literature in databases.\u003c/p\u003e\u003cp\u003eThe \"Mesh Terms\" section within the \"Keywords Identification\" menu is another component designed to help users identify standardized Medical Subject Headings (MeSH) terms associated with their research. MeSH terms are essential for indexing and categorizing scientific literature, ensuring that articles are accurately categorized and easy to search for in databases. This part of the system automates the process of retrieving MeSH terms for the keywords identified from the user's research. These terms are separated into four categories to reflect the common structure of a systematic review, often referred to as PICO (Population, Intervention, Comparator, and Outcome). Once the search terms are entered, the system moves to the next stage, which is preprocessing and phrase extraction. The system processes these keywords by applying text preprocessing techniques. Following preprocessing, the system automatically performs a query to an external MeSH term database. This is a crucial part of the system, as it ensures the retrieval of the most accurate and relevant MeSH terms for each of the user's search terms. Once the MeSH terms are fetched, the system aggregates the results for each category (population, intervention, comparator, and outcome).\u003c/p\u003e\u003cp\u003e\u003cb\u003eSearch\u003c/b\u003e\u003c/p\u003e\u003cp\u003eThe search terms section is a critical part of the system, where users define the terms, they want to use in the search. This section allows users to add up to five groups of terms. Each group can consist of multiple terms related to the topic of interest. In the search form, each search group has two fields: keywords and their Mesh terms. Each search group is connected by logical operators, either \"AND\" or \"OR. The operator determines how the terms in that group will be combined in the final query. In the Exclusion Terms part, users can specify terms that should be excluded from the search results\u003c/p\u003e\u003cp\u003ePublication Date Range\u003c/p\u003e\u003cp\u003eThe Publication Date Range filter enables the user to define the time frame for the publications they wish to include in their search. The user can select a start and end date using a date picker, which then defines the acceptable publication range for the search results.\u003c/p\u003e\u003cp\u003eLanguage\u003c/p\u003e\u003cp\u003eThe Language filter lets the user specify the language(s) of the studies they want to include in their search. A drop-down list of languages is provided, and the user can select one or more languages for the search.\u003c/p\u003e\u003cp\u003eDocument Type\u003c/p\u003e\u003cp\u003eThe Document Type section allows the user to select the type of publication they are interested in, such as \"journal article\", \"review\", \"clinical trial\", etc. The system provides a list of document types, and the user can choose one or more types that are relevant to their research.\u003c/p\u003e\u003cp\u003eSex, Species, and Age Group Filters\u003c/p\u003e\u003cp\u003eThe Sex, Species, and Age Group filters allow the user to further refine the search results based on demographic and study-specific factors. Sex filter allows the user to specify the sex of the participants in the studies they are interested in. The user can select from options \"Male\", \"Female\", or \"All\". This is useful for narrowing down studies that involve a specific sex or gender.\u003c/p\u003e\u003cp\u003eSpecies filter allows the user to filter studies based on the species involved, such as \"Human\" or \"Animal\". This helps researchers focus their search on studies involving specific species, in medical and biological research. Age Group filter allows the user to select specific age groups, such as \"Child\", \"Adult\", or \"Elderly\". This is useful in studies that focus on particular age demographics, helping to target research that is more relevant to specific populations.\u003c/p\u003e\u003cp\u003e\u003cb\u003eSearch Queries\u003c/b\u003e\u003c/p\u003e\u003cp\u003eThe \"Search Queries\" menu generates the specialized search syntax for three databases: PubMed, Scopus, and Web of Science. This menu is accessed after the user has entered the relevant search criteria in the \"Search\" menu. Once the user presses the \"Create Search Queries\" button in the \"Search\" menu, the system automatically constructs the search queries tailored to the specific requirements of each database.\u003c/p\u003e\u003cp\u003eThe PubMed search query construction process begins after the user presses \"Create Search Queries\". PubMed uses a specific query syntax that includes search terms (keywords and MeSH terms) and other specific filters including study design, sex, specie, and age group. After collecting all the inputs (search terms, MeSH terms, exclusion terms, etc.), the system generates the final PubMed query. It combines the search terms and filters to ensure they are properly formatted for PubMed\u0026rsquo;s API.\u003c/p\u003e\u003cp\u003eThe Scopus search query follows a different syntax. Scopus queries use specific formatting rules for logical operators and filters. The system uses the search terms from the Search section and generates a Scopus-specific query. The user\u0026rsquo;s search terms are included in the Scopus query, and they are formatted using Scopus\u0026rsquo;s syntax. Scopus uses AND/OR to combine terms, similar to PubMed. Scopus does not support MeSH terms directly like PubMed, so the system converts MeSH terms into the appropriate keywords or synonyms that will match the articles in Scopus. Scopus also allows the user to filter results based on document type, such as articles, reviews, etc. These filters are included in the query using Scopus\u0026rsquo;s document type syntax. The system checks the user's input and incorporates these filters into the query.\u003c/p\u003e\u003cp\u003eThe Web of Science (WoS) query construction is slightly different from the other two. Web of Science has its own format and special operators, so the system needs to adjust the syntax accordingly. Similar to PubMed and Scopus, the search terms are included in the Web of Science query. These terms are joined with AND/OR operators, depending on how the user has configured them. The query syntax for search terms is structured to match the Web of Science requirements. Web of Science does not natively support MeSH terms, so the system similarly converts the MeSH terms into keywords or synonyms that fit the Web of Science search format. These terms are included as part of the query.\u003c/p\u003e\u003cp\u003eAfter generating the queries for each of the databases, the user can execute these queries. The system sends the queries to the respective database\u0026rsquo;s API (for PubMed, Scopus) to retrieve the relevant records. This is done via the rentrez package for PubMed, and httr package for Scopus. When it comes to the WoS database, the document extraction process differs from that of PubMed and Scopus. Users need to copy the generated search syntax for WoS and run it manually in their institution\u0026rsquo;s subscribed advanced WoS search portal. After retrieving the relevant studies, they should download the results in Excel format. Finally, in the \u0026ldquo;Retrieved Studies\u0026rdquo; menu of the application, they can upload the downloaded Excel file.\u003c/p\u003e\u003cp\u003e\u003cb\u003eRetrieved Studies\u003c/b\u003e\u003c/p\u003e\u003cp\u003eIn the \"Retrieved Studies\" main menu all the studies retrieved from databases are stored, processed, and merged. This section allows users to handle studies obtained from PubMed, Scopus, Web of Science, and other databases. The system provides the capability to automatically import, process, and merge study results.\u003c/p\u003e\u003cp\u003eThe system has built-in functionality to fetch study data directly from PubMed and Scopus. Once the user generates the search query for PubMed, the system allows the user to run search. This action triggers the system to send the query to PubMed via the Entrez API from the rentrez package. The system fetches the search results from PubMed, specifically retrieving metadata including study titles, authors, journal, publication years, and DOIs. The results are then processed and displayed in the \"Retrieved Studies\" menu, under the PubMed section. Similarly, for Scopus the system allows the user run search which sends the query to the Scopus database. The query is processed through the Scopus API. Retrieved studies are then automatically displayed in the \"Retrieved Studies\" section, specifically for Scopus.\u003c/p\u003e\u003cp\u003eFor Web of Science, the process is a bit different because the system does not have an API integration to automatically fetch results. After the user generates the Web of Science search query, the system prompts the user to manually copy the generated search query and run it directly in the Web of Science database. The user can then upload this Excel file to the \"Web of Science\" section of the \"Retrieved Studies\" menu. The file is processed by the system and displayed in the same manner as the PubMed and Scopus studies.\u003c/p\u003e\u003cp\u003eThe system also provides functionality to handle results from other databases that may not be covered by PubMed, Scopus, or Web of Science. If a user conducts a search in another database, they can download the required Excel file format from the system. Once the user has gathered data from another database and formatted it according to the system\u0026rsquo;s required structure, they can upload the excel file. The system displays the results similarly to the other database sections.\u003c/p\u003e\u003cp\u003eAfter importing and uploading all the studies from PubMed, Scopus, Web of Science, and other sources, the system offers the \"Merge Studies\" functionality. This feature is a critical step, as it combines all the studies from the different databases into one consolidated list.\u003c/p\u003e\u003cp\u003e\u003cb\u003eDeduplication\u003c/b\u003e\u003c/p\u003e\u003cp\u003eThe deduplication module of the system is implemented as a three-stage pipeline that is surfaced to the user through the \u0026ldquo;Deduplication\u0026rdquo; menu. When a search has been completed in the retrieval tab, the application constructs a transient master table by binding the PubMed, Scopus, Web of Science and any user-supplied \u0026ldquo;Other\u0026rdquo; datasets.\u003c/p\u003e\u003cp\u003eThe first stage of deduplication, DOI-based deduplication, is an exact-match procedure that assumes the Digital Object Identifier (DOI) to be the most reliable global key. Using this section, the observer retrieves the current merged table, filters away rows whose DOI field is empty or missing, and then retains only the first occurrence of every unique DOI.\u003c/p\u003e\u003cp\u003eThe second stage deals with citations that lack a DOI.This section copies those rows and displays them for inspection. A subsequent click on \u0026ldquo;Run Fuzzy Deduplication\u0026rdquo; launches a two-part algorithm. First, each DOI-less record is normalized into four comparison keys. The title is lower-cased, punctuation stripped and whitespace squashed; the first author is reduced to \u0026ldquo;surname plus initial\u0026rdquo;; the journal name is condensed to an acronym of initial letters after stop-word removal; and the publication year is kept as a four-digit string. Consequently, a deterministic grouping on the quartet of keys removes intra-list duplicates, preserving the first entry in every group. The algorithm then safeguards against residual cross-source duplicates by anti-joining the DOI-less survivors to the DOI-bearing set \u0026mdash; if any DOI-less record\u0026rsquo;s four keys are identical to those of an already accepted DOI record it is discarded. Finally, this data frame is rendered in an interactive table and becomes the sole source for every downstream workflow, including title-abstract screening, full-text appraisal, automatic reporting and PRISMA-flow construction.\u003c/p\u003e\u003cp\u003e\u003cb\u003eTitle/Abstract Screening\u003c/b\u003e\u003c/p\u003e\u003cp\u003eThe title and abstract screening module translate routine dual-review appraisal practice into a reproducible, state-aware workflow. The application provides the places for one or two reviewers for conducting and completing this part separately. For each reviewer it creates an independent tab listing every record in the deduplicated master list. There are two writable columns entitled Decision (Include or Exclude) and Comments and adding a read-only hyperlink that resolves either to the article\u0026rsquo;s DOI landing page or, if no DOI is present, to a Google Scholar query.\u003c/p\u003e\u003cp\u003eOn completion of individual appraisal, the system provides three analytic actions. The Identify disagreements button is pressed, the server identifies and lists the documents which the reviewers\u0026rsquo; decisions are not the same. The Identify excluded studies and included studies buttons transform local reviewer decisions into canonical include and exclude lists that will later feed the PRISMA flow diagram. The inclusion of a contextual large language model (LLM)-consult hyperlink for every record in reviewers\u0026rsquo; panel provides a documented decision aid: clicking the link opens a modal that presents a pre-formatted prompt combining the study\u0026rsquo;s PDF with the user\u0026rsquo;s previously saved inclusion and exclusion criteria and requests an \u0026ldquo;Include\u0026rdquo; or \u0026ldquo;Exclude\u0026rdquo; recommendation plus justification, due to the high accuracy of LLMs in screening literature.\u003c/p\u003e\u003cp\u003e\u003cb\u003eFull-text Screening\u003c/b\u003e\u003c/p\u003e\u003cp\u003eThe full-text screening module operationalizes the second eligibility appraisal phase by building on the record set that emerged from title-and-abstract selection. The parts and mechanism of actions are the same as title/abstract screening.\u003c/p\u003e\u003cp\u003e\u003cb\u003eSearch and Screening Reports\u003c/b\u003e\u003c/p\u003e\u003cp\u003eThe Search and Screening Reports menu converts the raw reactives produced throughout the workflow into narrative prose blocks that can be pasted directly into the results section of a systematic review manuscript. It is organized as a tabset whose first sub-tab summarizes database retrieval and deduplication, while the second synthesizes the quantitative outcome of title/abstract and full-text appraisal.\u003c/p\u003e\u003cp\u003eThe server gathers the four data frames that holds retrieval output\u0026mdash;PubMed results, Scopus results, WoS results and other results. The total number of retrieved citations is derived from merge studies. DOI duplicates are quantified as the difference between the number of DOI-populated rows in the merged table and the number of rows in deduped studies, which holds the output of the exact DOI pass. The number of studies remaining after deduplication is also presented. A small HTML fragment is then constructed on the fly: it embeds the current date in long format, lists the record counts for each individual database, presents the total retrieved figure, reports the duplicate breakdown and finishes with the final post-deduplication count.\u003c/p\u003e\u003cp\u003eThe Screening sub-tab is activated by the button \u0026ldquo;Report Screening Results\u0026rdquo;. When reporting screening results, the system automatically provides the following metrics separately for the title/abstract and full-text screening phases: the number of screened studies, the number of disagreements, the number of removed studies, and the Kappa statistic between the two reviewers (if applicable).\u003c/p\u003e\u003cp\u003e\u003cb\u003ePRISMA flow diagram\u003c/b\u003e\u003c/p\u003e\u003cp\u003eThe PRISMA-flow module converts every numeric outcome accumulated during searching, deduplication and screening into a publication-ready schematic that follows the 2020 PRISMA template. The system first enforces three prerequisites: a deduplicated master list must exist, title-and-abstract screening must have been finalized, and full-text screening must likewise be completed; failure of any guard raises a notification and aborts generation. Once validated, the algorithm interrogates reactives to retrieve every count required by the flow. Download totals come from query-specific data frames; duplicates are partitioned into DOI and non-DOI classes and summed; post-deduplication count is taken from the final studies table. Screening numbers are extracted from helper reactives that mirror the logic used to construct included- and excluded-study lists. If reviewers declared any \u0026ldquo;other methods\u0026rdquo; the routine sums those n values and adds them to the count of studies included after full-text so the synthesis box reflects the complete set entering qualitative analysis.\u003c/p\u003e\u003cp\u003eThe server composes a Graphviz DOT script. The script then constructs a subgraph for database identification containing a header and two child boxes: one enumerating every contributing database with its record count and one listing the duplicate-removal figure. If the user supplied other databases a sister subgraph is generated in a second column, linked to the main header. Exclusion-reason lines are concatenated into a multiline label. Finally, orthogonal arrows connect identification to duplicates removed, then to screening, then to full-text assessment, and finally to inclusion and synthesis, with optional right-hand edges linking supplementary identification boxes back into the main spine. The completed DOT string is cached and passed to DiagrammeR::grViz, which renders an SVG.\u003c/p\u003e\u003cp\u003e\u003cb\u003eSummarization and Data Extraction\u003c/b\u003e\u003c/p\u003e\u003cp\u003eThe Summary and Data Extraction of Included Studies module turns the set of articles that survived full-text appraisal into an interactive register and provides an optional, on-demand service that relies on automatic summarization and data extraction. The interface begins with an import button. It copies the records into the reactive store included summary, drops the screening-specific Decision and Comments columns so that only bibliographic fields remain.\u003c/p\u003e\u003cp\u003eClicking a summarization/data extraction button opens modal dialog containing a richly formatted, multi‑line prompt. The established prompt includes two main parts, first to generate a four‑part summary (purpose, methods, results, conclusion), then to extract an extended list of data elements (from setting and funding source, PICO elements, through effect estimates and measures of variability). The modal also embeds two helper controls. The \u0026ldquo;Edit\u0026rdquo; button to make the prompt text area editable, allowing on‑the‑fly refinement; the \u0026ldquo;Copy\u0026rdquo; button runs a small JavaScript snippet that reads the entire contents of the text area and writes it to the clipboard, ensuring a single‑click transfer into ChatGPT or any other LLM interface.\u003c/p\u003e\u003cp\u003eThe application is structured into four interconnected phases. During preparation, PICO and exclusion criteria are specified and source documents (PDFs or URLs) are ingested, with automated text extraction, tokenization, n‑gram counting and phrase detection\u0026mdash;optionally supplemented by a TextRank summary. In the Query \u0026amp; Retrieval phase, optimized search strings for PubMed, Scopus and Web of Science are generated and executed, and all retrieved records are merged and harmonized into a unified dataset. The Deduplication \u0026amp; Screening phase applies exact‑match DOI filtering and fuzzy string deduplication before guiding reviewers through title/abstract and full‑text screening via interactive tables enriched with LLM prompts. Finally, Reporting produces a publication‑ready PRISMA flow diagram, an HTML screening report, automated study summaries generated by LLM prompts, and an automatic data extraction table for downstream synthesis. See Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e for detail phases.\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003eTo evaluate the application\u0026rsquo;s feasibility, we applied it to our recently published systematic review and meta-analysis examining the effect of virtual reality in dentistry (Nezhad et al., \u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e2024\u003c/span\u003e).\u003c/p\u003e"},{"header":"Results","content":"\u003cp\u003eFirstly, the inclusion and exclusion criteria of the systematic review were defined in the user interface menu titled \u0026ldquo;Inclusion and Exclusion Criteria\u0026rdquo; and saved in the system for subsequent steps. In the first preparatory phase, five articles from the final set of included studies were imported into the SPECTRUM SRA system as gold standards to identify 10 unigrams, 10 bigrams, and 10 trigrams. Consequently, the text-extraction pipeline\u0026mdash;utilizing the pdftools and tokenizers packages\u0026mdash;was initiated. This automated process took only five seconds. Next, to identify MeSH terms, the PICO elements of the systematic review were entered into the software\u0026rsquo;s \u0026ldquo;MeSH Terms\u0026rdquo; menu. The system completed the MeSH term retrieval using rentrez and rvest in 10 seconds and returned structured, relevant MeSH terms across the PICO categories. Overall, the first phase of the systematic review was completed in just 12 seconds using the software.\u003c/p\u003e\u003cp\u003eAfter introducing the inclusion and exclusion criteria and identifying keywords and their corresponding MeSH terms via the user interface, the Search interface was activated. The identified search keyword \u0026ldquo;virtual reality\u0026rdquo; and its corresponding MeSH term \u0026ldquo;Virtual Reality\u0026rdquo; were combined with additional keywords, including teeth, tooth, and dental. The search was limited to studies published between January 1, 2020, and April 1, 2024 (the date the hand search was conducted). The document type was restricted to randomized controlled trials. After clicking \u0026ldquo;Generate Search Queries,\u0026rdquo; the system automatically generated optimized search strings for PubMed, Scopus, and Web of Science in less than three seconds. For instance, the PubMed search string was:\u003c/p\u003e\u003cp\u003e(((\"virtual reality\"[tiab] OR \"virtual reality\"[mh])) AND (\"tooth\"[tiab] OR \"teeth\"[tiab] OR \"dental\"[tiab])) AND (\"2020/01/01\" : \"2024/04/01\"[dp]) AND (\"Randomized Controlled Trial\"[pt])).\u003c/p\u003e\u003cp\u003eThen, by pressing the \u0026ldquo;Run Search\u0026rdquo; button, the system automatically executed the PubMed search using rentrez and retrieved 33 records. The searches in Scopus (via httr) and Web of Science returned 383 and 287 records, respectively. Retrieving and displaying results took approximately 30 seconds for PubMed and 4 minutes for Scopus. For Web of Science, the automatically generated search syntax was manually copied and executed in the WoS Advanced Search interface. The results were downloaded as an Excel file and uploaded into the Web of Science panel in the system. The entire WoS process was completed in one minute. Finally, merging and normalizing the combined 703 records from all sources took only one second, resulting in a single master table ready for deduplication.\u003c/p\u003e\u003cp\u003eThe deduplication pipeline consisted of two steps: one for studies with a DOI and another for studies without a DOI.In the DOI-based deduplication step, 679 records were processed, and 235 duplicates were identified and removed. The DOI-less deduplication step returned 24 records, from which 3 duplicates were removed. In total, 238 duplicates were removed, leaving 465 records after deduplication, totally in 3 seconds.\u003c/p\u003e\u003cp\u003eFor title and abstract screening, two reviewers were selected to screen the 465 articles. ChatGPT-o4-mini-high and DeepSeek v3 R1, due to their advanced reasoning and deep-thinking capabilities, were chosen as the first and second reviewers, respectively. The automatically generated prompt from the system was copied and pasted into the chatboxes of the LLMs, and the PDFs of articles were uploaded. For every batch of 10 articles, screening took 60 seconds using the ChatGPT model and 120 seconds using DeepSeek. In total, the screening process took 46.5 minutes for ChatGPT and 93 minutes for DeepSeek. For both reviewers, entering the screening decisions (Include/Exclude) into the system took an additional 5 seconds per record\u0026mdash;approximately 38 minutes each. Therefore, the total time required for title and abstract screening was 84.5 minutes for reviewer 1 (ChatGPT) and 131 minutes for reviewer 2 (DeepSeek). The disagreements (12 cases) were identified in one second. These disagreements were then reviewed, and final decisions made. During title/abstract screening, 422 records were excluded, and 43 articles were included and transferred to the full-text screening phase.\u003c/p\u003e\u003cp\u003eThe full-text screening of 43 studies that passed the title and abstract screening took 43 minutes for Reviewer 1 (ChatGPT) and 90 minutes for Reviewer 2 (DeepSeek). The disagreement panel automatically identified two discrepancies; after resolving them, 25 studies met the inclusion criteria.\u003c/p\u003e\u003cp\u003eUpon completing the title/abstract and full‑text screening steps, the system automatically generated the search and screening reports in under one second, as follows:\u003c/p\u003e\u003cp\u003eThe search was conducted on 23 July 2025.\u003c/p\u003e\u003cp\u003eThe number of retrieved studies in PubMed: 33.\u003c/p\u003e\u003cp\u003eThe number of retrieved studies in Scopus: 383.\u003c/p\u003e\u003cp\u003eThe number of retrieved studies in Web of Science: 287.\u003c/p\u003e\u003cp\u003eThe number of retrieved studies in other databases: 0.\u003c/p\u003e\u003cp\u003eTotal number of retrieved studies: 703\u003c/p\u003e\u003cp\u003eThe number of duplicates: 238.\u003c/p\u003e\u003cp\u003eThe number of studies after removing duplicates: 465.\u003c/p\u003e\u003cp\u003eTitle/Abstract Screening:\u003c/p\u003e\u003cp\u003eNumber of studies screened: 465\u003c/p\u003e\u003cp\u003eNumber of disagreements: 12\u003c/p\u003e\u003cp\u003eNumber of studies excluded: 422\u003c/p\u003e\u003cp\u003eKappa statistic between reviewers: 0.852\u003c/p\u003e\u003cp\u003eFull‑Text Screening:\u003c/p\u003e\u003cp\u003eNumber of studies screened: 43\u003c/p\u003e\u003cp\u003eNumber of disagreements: 2\u003c/p\u003e\u003cp\u003eNumber of studies excluded: 18\u003c/p\u003e\u003cp\u003eKappa statistic between reviewers: 0.903\u003c/p\u003e\u003cp\u003eA PRISMA flow diagram was also automatically generated and displayed after the reasons for exclusion were entered. All 25 included studies were then processed for summarization and data extraction using an automatically generated prompt. ChatGPT o4‑mini‑high was selected for these tasks. First, the system‑generated prompt was copied into ChatGPT\u0026rsquo;s chatbox, and the PDF of each included study was uploaded sequentially. This phase took 60 seconds per study, completing in a total of 25 minutes. The extracted data were verified for accuracy, achieving a 91% accuracy rate. The most challenging elements to extract were funding sources and PICO details.\u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003eIn this study, we developed SPECTRUM SRA, an R Shiny application that provides an end-to-end workflow for the automation and semi‑automation of systematic reviews, from keyword selection through data extraction. We also evaluated the application\u0026rsquo;s feasibility. Our findings demonstrate that the application not only offers systematic reviewers a unified platform encompassing all essential stages of a systematic review but, owing to recent advancements in large language models (LLMs) (O\u0026rsquo;Connor et al., \u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e2024\u003c/span\u003e) and their high accuracy in screening (Khraisha et al., \u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e2024\u003c/span\u003e) and data extraction (Konet et al., \u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e2024\u003c/span\u003e), can be relied upon to generate accurate outputs.\u003c/p\u003e\u003cp\u003eSPECTRUM SRA includes eight core models: search, retrieval, deduplication, screening, PRISMA flow diagram generation, reporting, summarization, and data extraction withing a singe Shiny application. By eliminating the need for navigating between different platforms/models/applications, the application reduces context‑switching overhead and reduces the cognitive load on reviewers. The system supported article retrieval and deduplication, executed in some few seconds. Numerous deduplication algorithms and tools exist\u0026mdash;for example, \u0026ldquo;The Duplicator\u0026rdquo; (Forbes et al., \u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e2024\u003c/span\u003e). Although this tool can accelerate the deduplication process, its accuracy requires further evaluation (Forbes et al., \u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e2024\u003c/span\u003e). Accuracy varies across platforms, and selection often depends on interface features (Guimar\u0026atilde;es et al., \u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e2022\u003c/span\u003e). In our platform, we designed a user‑friendly interface that enables automatic deduplication in just three clicks, completing the task in a matter of seconds.\u003c/p\u003e\u003cp\u003eThe system provides LLM‑based automated prompts for the screening, summarization, and data‑extraction phases. We selected two models\u0026mdash;ChatGPT o4‑mini‑high and DeepSeek V3 R1\u0026mdash;for these tasks. Thanks to recent advances in evidence‑synthesis automation and the high accuracy of LLMs in both screening and data extraction, our application can substantially reduce the labor intensity and time required to complete systematic reviews. We employed these two LLMs as the first and second reviewers; although we achieved a kappa statistic of 0.852 for title/abstract screening and 0.903 for full‑text screening, disagreements still occur and should be resolved by human reviewers to ensure the validity of final decisions. Moreover, LLM‑generated outputs must be interpreted with caution, as recent studies have noted potential errors (Galli, Anna V. Gavrilova, et al., 2025; Rokhshad et al., \u003cspan citationid=\"CR44\" class=\"CitationRef\"\u003e2025\u003c/span\u003e). Several other LLM‑based platforms for systematic‑review automation have also been developed\u0026mdash;for example, Otto SR, which supports both screening and data extraction, reaching accuracy more than 90% for both screening and data extraction (Cao, Arora, et al., \u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e2025\u003c/span\u003e).\u003c/p\u003e\u003cp\u003ePerhaps the most important benefit of this application is the reduction in review time\u0026mdash;what once took several months can now be completed in a matter of days. By automating numerous steps in the systematic‑review process, the application frees domain experts to concentrate on high‑value activities such as clinical interpretation, critical appraisal, and evidence synthesis. For instance, some researchers presented a case study demonstrating that a systematic review could be completed in two weeks by employing automation tools (Clark et al., \u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e2020\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eThis study has several limitations that warrant cautious interpretation. First, the application currently supports only three electronic databases\u0026mdash;PubMed, Scopus, and Web of Science. Although users can upload search results from other sources, this step must be performed manually. Additionally, Web of Science queries are not automated via API; users must run searches in an external portal and import the results. Because screening, summarization, and data extraction rely on LLMs, outputs remain susceptible to limitations such as hallucinations. During testing, we also encountered occasional server errors from PubMed and Scopus. Finally, our feasibility assessment was preliminary; broad generalizability will require evaluation across diverse review topics, disciplines, and database configurations.\u003c/p\u003e\u003cp\u003eTo enhance the application\u0026rsquo;s quality, we plan several future extensions. First, we will integrate automatic searching in Web of Science via API keys, mirroring Scopus implementation. Next, we will enable automatic search and article retrieval from additional electronic databases\u0026mdash;such as the Cochrane Library, Embase, trial registries (e.g., ClinicalTrials.gov), and preprint servers. Finally, we will incorporate LLM‑based automation for risk‑of‑bias assessment.\u003c/p\u003e"},{"header":"Conclusion","content":"\u003cp\u003eIn conclusion, the SPECTRUM SRA application\u0026mdash;which supports multiple systematic-review tasks, including keyword identification, search, article retrieval, deduplication, title/abstract screening, full‐text screening, PRISMA flow‐diagram generation, summarization, and data extraction\u0026mdash;is an AI‑augmented tool that streamlines the review process. Its use can substantially reduce both the time and workload required for conducting systematic reviews. Because large language models have inherent limitations, we recommend employing at least two models with advanced reasoning and deep‐thinking capabilities (for example, ChatGPT o4‑mini‑high and DeepSeek V3 R1) as the first and second reviewers, respectively, along with a human reviewer as a third check to ensure reliability.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eAbstrackr (2025) (n.d.) \u003cem\u003eAbstrackr\u003c/em\u003e. [Online] [online]. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://abstrackr.cebm.brown.edu/account/login\u003c/span\u003e\u003cspan address=\"http://abstrackr.cebm.brown.edu/account/login\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (Accessed 12 June\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAlshami A, Elsayed M, Ali E, Eltoukhy AEE, Zayed T (2023) Harnessing the Power of ChatGPT for Automating Systematic Review Process: Methodology, Case Study, Limitations, and Future Directions. Systems 11(7):351\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eB S, B PPR, S.M., S K (2024) \u0026lsquo;The Evolution of Large Language Model: Models, Applications and Challenges\u0026rsquo;, in \u003cem\u003e2024 International Conference on Current Trends in Advanced Computing (ICCTAC)\u003c/em\u003e. [Online]. 8 May 2024 IEEE. pp. 1\u0026ndash;8\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eBlasingame MN, Koonce TY, Williams AM, Giuse DA, Su J, Krump PA, Giuse NB (2024) \u003cem\u003eEvaluating a Large Language Model\u0026rsquo;s Ability to Answer Clinicians\u0026rsquo; Requests for Evidence Summaries\u003c/em\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eBorah R, Brown AW, Capers PL, Kaiser KA (2017) Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ Open 7(2):e012545\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eBramer WM, Giustini D, de Jonge GB, Holland L, Bekhuis T (2016) De-duplication of database search results for systematic reviews in EndNote. J Med Libr Association: JMLA 104(3):240\u0026ndash;243\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eCao C, Arora R, Cento P, Manta K, Farahani E, Cecere M, Selemon A, Sang J, Gong LX, Kloosterman R, Jiang S, Saleh R, Margalik D, Lin J, Jomy J, Xie J, Chen D, Gorla J, Lee S et al (2025) \u003cem\u003eAutomation of Systematic Reviews with Large Language Models\u003c/em\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eCao C, Sang J, Arora R, Chen D, Kloosterman R, Cecere M, Gorla J, Saleh R, Drennan I, Teja B, Fehlings M, Ronksley P, Leung AA, Weisz DE, Ware H, Whelan M, Emerson DB, Arora RK, Bobrovitz N (2025) Development of Prompt Templates for Large Language Model\u0026ndash;Driven Screening in Systematic Reviews. Ann Intern Med 178(3):389\u0026ndash;401\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eChai KEK, Lines RLJ, Gucciardi DF, Ng L (2021) Research Screener: a machine learning tool to semi-automate abstract screening for systematic reviews. Syst Reviews 10(1):93\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eClark J, Glasziou P, Del Mar C, Bannach-Brown A, Stehlik P, Scott AM (2020) A full systematic review was completed in 2 weeks using automation tools: a case study. J Clin Epidemiol 121:81\u0026ndash;90\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eCovidence (2025) (n.d.) \u003cem\u003eCovidence\u003c/em\u003e. [Online] [online]. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.covidence.org/\u003c/span\u003e\u003cspan address=\"https://www.covidence.org/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (Accessed 10 June\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eCumpston M, Li T, Page MJ, Chandler J, Welch VA, Higgins JP, Thomas J (2019) Updated guidance for trusted systematic reviews: a new edition of the Cochrane Handbook for Systematic Reviews of Interventions. Cochrane Database of Systematic Reviews\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eDistillerSR (n.d.) \u003cem\u003eDistillerSR\u003c/em\u003e. [Online] [online]. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.distillersr.com/\u003c/span\u003e\u003cspan address=\"https://www.distillersr.com/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (Accessed 9 June 2025)\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eDjulbegovic B, Guyatt GH (2017) Progress in evidence-based medicine: a quarter century on. Lancet 390(10092):415\u0026ndash;423\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eForbes C, Greenwood H, Carter M, Clark J (2024) Automation of duplicate record detection for systematic reviews: Deduplicator. Syst Reviews 13(1):206\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eGalli C, Gavrilova AV, Calciolari E (2025) Large Language Models in Systematic Review Screening: Opportunities, Challenges, and Methodological Considerations. Information 16(5):378\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eGalli C, Gavrilova A Viktorovna, Calciolari E (2025) \u003cem\u003eIntegrating Large Language Models into Systematic Review Screening\u003c/em\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eGuimar\u0026atilde;es NS, Ferreira AJF, Ribeiro Silva R, de Paula C, Lisboa AA, Magno CS, Ichiara L, M.Y., Barreto ML (2022) Deduplicating records in systematic reviews: there are free, accurate automated ways to do so. J Clin Epidemiol 152:110\u0026ndash;115\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eHiggins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ (2022) W.V. \u003cem\u003eCochrane Handbook for Systematic Reviews of Interventions version 6.3 (updated February 2022)\u003c/em\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eHoffmann F, Allers K, Rombey T, Helbach J, Hoffmann A, Mathes T, Pieper D (2021) Nearly 80 systematic reviews were published each day: Observational study on trends in epidemiology and reporting over the years 2000\u0026ndash;2019. J Clin Epidemiol 138:1\u0026ndash;11\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eHossain MM (2024) Using ChatGPT and other forms of generative AI in systematic reviews: Challenges and opportunities. J Med Imaging Radiation Sci 55(1):11\u0026ndash;12\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eIsfeedvajani MS (2016) Systematic Review and its Role in Clinical Decision Making. Int J Med Rev 3(4):495\u0026ndash;496\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eJardim PSJ, Rose CJ, Ames HM, Echavez JFM, Van de Velde S, Muller AE (2022) Automating risk of bias assessment in systematic reviews: a real-time mixed methods comparison of human researchers to a machine learning system. BMC Med Res Methodol 22(1):167\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eKhalil H, Ameen D, Zarnegar A (2022) Tools to support the automation of systematic reviews: a scoping review. J Clin Epidemiol 144:22\u0026ndash;42\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eKhraisha Q, Put S, Kappenberg J, Warraitch A, Hadfield K (2024) Can large language models replace humans in systematic reviews? Res Synthesis Methods 15(4):616\u0026ndash;626\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eKilicoglu H, Demner-Fushman D, Rindflesch TC, Wilczynski NL, Haynes RB (2009) Towards Automatic Recognition of Scientifically Rigorous Clinical Research Evidence. J Am Med Inform Assoc 16(1):25\u0026ndash;31\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eKonet A, Thomas I, Gartlehner G, Kahwati L, Hilscher R, Kugley S, Crotty K, Viswanathan M, Chew R (2024) Performance of two large language models for data extraction in evidence synthesis. Res Synthesis Methods 15(5):818\u0026ndash;824\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMarshall IJ, Kuiper J, Wallace BC (2016) RobotReviewer: evaluation of a system for automatically assessing bias in clinical trials. J Am Med Inform Assoc 23(1):193\u0026ndash;201\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMarshall IJ, Wallace BC (2019) Toward systematic review automation: a practical guide to using machine learning tools in research synthesis. Syst Reviews 8(1):163\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMichelson M, Reuter K (2019) \u0026lsquo;The significant cost of systematic reviews and meta-analyses: A call for greater involvement of machine learning to assess the promise of clinical trials\u0026rsquo;. Contemp Clin Trials Commun, 16p. 100443\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMotzfeldt Jensen M, Brix Danielsen M, Riis J, Assifuah Kristjansen K, Andersen S, Okubo Y, J\u0026oslash;rgensen MG (2025) ChatGPT-4o can serve as the second rater for data extraction in systematic reviews\u0026rsquo; Weiqiang (Albert) Jin. PLoS ONE 20(1):e0313401\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eNezhad HM, Ashourioun A, Sadeghdaghighi A (2024) The effect of virtual reality for anxiety and pain in dentistry: A systematic review and meta-analysis. Community Dent Health 41(4):248\u0026ndash;255\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eNykvist B, Macura B, Xylia M, Olsson E (2025) Testing the utility of GPT for title and abstract screening in environmental systematic evidence synthesis. Environ Evid 14(1):7\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eO\u0026rsquo;Connor AM, Clark J, Thomas J, Spijker R, Kusa W, Walker VR, Bond M (2024) Large language models, updates, and evaluation of automation tools for systematic reviews: a summary of significant discussions at the eighth meeting of the International Collaboration for the Automation of Systematic Reviews (ICASR). Syst Reviews 13(1):290\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eO\u0026rsquo;Connor AM, Tsafnat G, Gilbert SB, Thayer KA, Wolfe MS (2018) Moving toward the automation of the systematic review process: a summary of discussions at the second meeting of International Collaboration for the Automation of Systematic Reviews (ICASR). Syst Reviews 7(1):3\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eO\u0026rsquo;Mara-Eves A, Thomas J, McNaught J, Miwa M, Ananiadou S (2015) Using text mining for study identification in systematic reviews: a systematic review of current approaches. Syst Reviews 4(1):5\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eOssom Williamson P, Minter CIJ (2019) \u0026lsquo;Exploring PubMed as a reliable resource for scholarly communications services\u0026rsquo;. J Med Libr Association, 107(1)\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003ePark SE, Thomas J (2018) Evidence synthesis software. BMJ Evidence-Based Med 23(4):140\u0026ndash;141\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eQureshi R, Shaughnessy D, Gill KAR, Robinson KA, Li T, Agai E (2023) Are ChatGPT and large language models the answer to bringing us closer to systematic review automation? Syst Reviews 12(1):72\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eRadenkovic D, Keogh SB, Maruthappu M (2019) Data science in modern evidence-based medicine. J R Soc Med 112(12):493\u0026ndash;494\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eRayyan (2025) (n.d.) \u003cem\u003eRayyan\u003c/em\u003e. [Online] [online]. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.rayyan.ai/\u003c/span\u003e\u003cspan address=\"https://www.rayyan.ai/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (Accessed 3 June\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eRethlefsen ML, Brigham TJ, Price C, Moher D, Bouter LM, Kirkham JJ, Schroter S, Zeegers MP (2024) \u0026lsquo;Systematic review search strategies are poorly reported and not reproducible: a cross-sectional metaresearch study\u0026rsquo;. J Clin Epidemiol, 166p. 111229\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eRobotAnalyst (2025) (n.d.) \u003cem\u003eRobotAnalyst\u003c/em\u003e. [Online] [online]. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://nactem.ac.uk/RA/login\u003c/span\u003e\u003cspan address=\"https://nactem.ac.uk/RA/login\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (Accessed 11 June\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eRokhshad R, Bagherianlemraski M, Ehsani SS, Haghighat S, Schwendicke F (2025) \u0026lsquo;Large language models for the screening step in systematic reviews in dentistry\u0026rsquo;. J Dent, 160p. 105877\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eScherbakov D, Hubig N, Jansari V, Bakumenko A, Lenert LA (2025) The emergence of large language models as tools in literature reviews: a large language model-assisted systematic review. J Am Med Inform Assoc 32(6):1071\u0026ndash;1086\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eSchmidt L, Finnerty Mutlu AN, Elmore R, Olorisade BK, Thomas J, Higgins JPT (2023) \u0026lsquo;Data extraction methods for systematic review (semi)automation: Update of a living systematic review\u0026rsquo;, \u003cem\u003eF1000Research\u003c/em\u003e, 10p. 401\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003evan de Schoot R, de Bruin J, Schram R, Zahedi P, de Boer J, Weijdema F, Kramer B, Huijts M, Hoogerwerf M, Ferdinands G, Harkema A, Willemsen J, Ma Y, Fang Q, Hindriks S, Tummers L, Oberski DL (2021) An open source machine learning framework for efficient and transparent systematic reviews. Nat Mach Intell 3(2):125\u0026ndash;133\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eShojania KG, Sampson M, Ansari MT, Ji J, Doucette S, Moher D (2007) How Quickly Do Systematic Reviews Go Out of Date? A Survival Analysis. Ann Intern Med 147(4):224\u0026ndash;233\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eTopol EJ (2019) High-performance medicine: the convergence of human and artificial intelligence. Nat Med 25(1):44\u0026ndash;56\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eTsafnat G, Glasziou P, Choong MK, Dunn A, Galgani F, Coiera E (2014) Systematic review automation technologies. Syst Reviews 3(1):74\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eTsou AY, Treadwell JR, Erinoff E, Schoelles K (2020) Machine learning for screening prioritization in systematic reviews: comparative performance of Abstrackr and EPPI-Reviewer. Syst Reviews 9(1):73\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eWallace BC, Kuiper J, Sharma A, Zhu M, Brian, Marshall IJ (2016) Extracting PICO Sentences from Clinical Trial Reports using Supervised Distant Supervision. J Mach Learn Res 176(25):139\u0026ndash;148\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eWallace SS, Barak G, Truong G, Parker MW (2022) Hierarchy of Evidence Within the Medical Literature. Hosp Pediatr 12(8):745\u0026ndash;750\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eWang S, Scells H, Koopman B, Zuccon G (2022) \u0026lsquo;Automated MeSH term suggestion for effective query formulation in systematic reviews literature search\u0026rsquo;. Intell Syst Appl, 16p. 200141\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eWang S, Scells H, Zhuang S, Potthast M, Koopman B, Zuccon G (2024) \u003cem\u003eZero-Shot Generative Large Language Models for Systematic Review Screening Automation\u003c/em\u003e, in [Online]. pp. 403\u0026ndash;420\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eZaki M, Namireddy SR, Pittie T, Bihani V, Keshri SR, Venugopal V, Gosvami NN, Jayadeva, Krishnan NMA (2022) \u0026lsquo;Natural language processing-guided meta-analysis and structure factor database extraction from glass literature\u0026rsquo;. J Non-Crystalline Solids: X, 15p. 100103\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"University of Isfahan","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Automation, Natural language processing, NLP, Reproducible research, Evidence synthesis, Evidence based-medicine, Meta-analysis","lastPublishedDoi":"10.21203/rs.3.rs-7265470/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7265470/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003ePurpose\u003c/strong\u003e:\u003c/p\u003e\n\u003cp\u003eThis study aims to develop an R Shiny application named SPECTRUM SRA (Systematic Review Automation) and to assess its feasibility. The application encompasses eight modules: Search, PRISMA flow diagram generation, dEduplication, sCreening, daTa extraction, Reporting, qUery, and suMmarization. It provides an end‑to‑end workflow to automate and semi‑automate multiple steps of systematic reviews, addressing growing time and labor constraints and accelerating evidence‑to‑practice translation.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eDesign/methodology/approach\u003c/strong\u003e:\u003c/p\u003e\n\u003cp\u003eSPECTRUM SRA comprises eight interconnected modules and offers an interface for automated article search and retrieval from PubMed, Scopus, and Web of Science. Search functionality is implemented via API integration for PubMed and Scopus, with manual search and import for Web of Science results. Deduplication employs two strategies: DOI‑based matching and fuzzy string deduplication. The application also supports automated title/abstract and full‑text screening using large language model prompting. Automated reporting for both search and screening is incorporated through AI‑driven scripting. Additionally, SPECTRUM SRA automates PRISMA flow diagram generation. Summarization and data extraction of the final set of eligible studies are performed via LLM prompting. The feasibility of the application was assessed through a systematic review of randomized controlled trials comparing virtual reality interventions to non‑virtual‑reality approaches in dentistry.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFindings\u003c/strong\u003e:\u003c/p\u003e\n\u003cp\u003eKeyword identification was completed in 12 seconds; search query generation took less than three seconds; and retrieval of 703 records from the databases required 4 minutes and 30 seconds. Deduplication removed 238 records in 3 seconds, leaving 465 records for screening. ChatGPT o4‑mini‑high and DeepSeek V3 R1 were selected as the first and second reviewers, respectively. Title/abstract screening took 84.5 minutes for Reviewer 1 and 131 minutes for Reviewer 2 (κ = 0.852). Full‑text screening of 43 articles required 43 minutes for Reviewer 1 and 90 minutes for Reviewer 2 (κ = 0.903). Automated summarization and data extraction of the 25 final eligible studies achieved 91% accuracy using ChatGPT in 25 minutes.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eOriginality\u003c/strong\u003e:\u003c/p\u003e\n\u003cp\u003eTo the best of our knowledge, SPECTRUM SRA is the first end‑to‑end Shiny application that supports multiple tasks in systematic reviews. It can reduce the time, labor, and energy required to conduct systematic reviews, thereby accelerating health policy and decision‑making.\u003c/p\u003e","manuscriptTitle":"SPECTRUM SRA: An R Shiny Application to Automate Systematic Reviews Using Artificial Intelligence and Large Language Models","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-08-04 05:44:54","doi":"10.21203/rs.3.rs-7265470/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"402828dc-6582-4b55-bf4f-4b1b2e870029","owner":[],"postedDate":"August 4th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":52460685,"name":"Artificial Intelligence and Machine Learning"}],"tags":[],"updatedAt":"2025-08-04T05:44:54+00:00","versionOfRecord":[],"versionCreatedAt":"2025-08-04 05:44:54","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7265470","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7265470","identity":"rs-7265470","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}