{"paper_id":"6f9a39c0-d186-4eb8-84d9-3b71779ad66f","body_text":"Improved precision oncology question-\nanswering using agentic LLM \nRangan Das5,§, K Maheswari 1,§, Shaheen Siddiqui1,*, Nikita Arora1,*, Ankush Paul 1,*, Jeet Nanshi 1,*, Varun \nUdbalkar1, Apoorva Sarvade 1, Harsha Chaturvedi 1, Tammy Shvartsman1, Shet Masih1, R Thippeswamy6, \nShekar Patil6, S S Nirni 7, Brian Garsson 1, S anghamitra Bandyopadhyay8, Ujjwal Maulik1,5, Mohammed \nFarooq1, Debarka Sengupta1,2,3,4+ \n1. GeneSilico, Inc., 3267 Bee Caves Rd, STE 107-332 Austin, TX 78746 \n2. Department of Computational Biology, Indraprastha Institute of Information Technology-Delhi (IIIT-\nDelhi), Okhla, Phase III, New Delhi, 110020, India \n3. Department of Computer Science and Engineering, Indraprastha Institute of Information \nTechnology-Delhi (IIIT-Delhi), Okhla, Phase III, New Delhi, 110020, India \n4. Centre for Artificial Intelligence, Indraprastha Institute of Information Technology -Delhi (IIIT -\nDelhi), Okhla, Phase III, New Delhi, 110020, India \n5. Department of Computer Science and Engineering, Jadavpur University, Kolkata, India \n6. Department of Medical Oncology, HGC Cancer Centre, Bangalore, Karnataka 560027, India \n7. Department of Medical Oncology, Omega Hospitals and Indo- American Cancer Institute and \nResearch Centre, Hyderabad, Andhra Pradesh, India \n8. Machine Intelligence Unit, Indian Statistical Institute, Kolkata 700108, India \n+ To whom correspondence should be addressed: \ndsengupta@genesilico.ai \n§ co-first authors, * equal contribution \n  \nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \nNOTE: This preprint reports new research that has not been certified by peer review and should not be used to guide clinical practice.\n\n   \n \n   \n \nABSTRACT \nThe clinical adoption of Large Language Models (LLMs) in biomedical research has been limited by concerns \nregarding the quality, accuracy, and reliability of their outputs, particularly in precision oncology, where \nclinical decision-making demands high precision. Current models, often based on fine-tuned foundational \nLLMs, are prone to issues such as hallucinations, incoherent reasoning, and loss of context. In this work, \nwe present GeneSilico Copilot, an advanced agent -based architecture that transforms LLMs from simple \nresponse synthesizers to clinical reasoning systems. Our approach is centred around a bespoke ReAct \nagent that orchestrates a suite of specialized tools for asynchronous information retrieval and synthesis. \nThese tools access curated document vector stores containing clinical treatment guidelines, genomic \ninsights, drug information, clinical trials, and breast cancer -specific literature. To leverage large context \nwindows of current LLMs , we implement a hybrid search strategy that prioritizes key information and \ndynamically integrates summarized content, reducing context fragmentation.  Incorporating additional \nmetadata further allows for precise, transparent and evidence-backed reasoning at each step of the thought \nprocess. The system ensures that at every stage, the agent can synthesize meaningful, context -aware \nobservations that contribute to a coherent and comprehensive final response that align s with clinical \nstandards. Evaluations on real -world breast cancer cases show that GeneSilico Copilot significantly \nimproves response accuracy and personalization. This system represents a critical advancement toward \nmaking LLMs clinically deployable in precision oncology and has potential applications in broader medical \ndomains requiring complex, data-driven decision-making. \n \n  \nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n   \n \n   \n \nINTRODUCTION \nCancer's inherent complexity, driven by both inter and intra-tumoral heterogeneity, presents a significant \nhurdle in clinical management. This genetic heterogeneity allows tumors  to evade traditional treatments. \nHowever, the advent of precision oncology has led to the development of genome- targeted and genome-\ninformed therapies, which aim to address this challenge. From 2006 to 2020, the eligibility for genome -\ntargeted therapies in the U.S. increased from 5.13% to 13.60%, while the response rate improved fr om \n2.73% to 7.04%. Similarly, genome-informed therapies saw a rise in eligibility from 10.70% to 27.30% and \nan increase in response from 3.33% to 11.10% during the same period. Interestingly, most of the eligibility \nincrease for genome-targeted therapies occurred after 2018, whereas most of the response increase was \nobserved before 2018. These findings highlight a concerning trend: while eligibility for these therapies is on \nthe rise, the actual response rate remains low\n1. Large Language Models (LLMs) offer a powerful opportunity \nto address these challenges 2. By leveraging their ability to process and synthesize vast amounts of \nhealthcare data, LLMs can assist oncologists in navigating the ever- expanding landscape of targeted \ntherapies. They can analyze a patient's specific genetic profile, identify relevant clinical trials and treatment \nguidelines, and even suggest potential drug combinations tailored to the unique characteristics of the \npatient's cancer considering comorbidities and side effects . While LLMs can correctly identify some key \nstrategies and offer reasonable, albeit incomplete, suggestions even experts missed, they can also generate \nfactual errors (hallucination), irrelevant, harmful, or biased content\n3.   \nWhile multiple industries have adopted LLM -based models, adoption in biology and medicine is still \nlacking4. With the ever-evolving treatment guidelines and drug approval5,6 it is difficult for fine-tuned LLMs \nto stay updated. State of the art models like Med -PaLM 7, BioGPT 8, and BioBERT 9 are standalone systems \nthat are trained and fine -tuned on domain -specific large-scale biomedical corpora. These have achieved \nnotable results on medical question answering datasets. GatorTron 10 is a similar model that has been \ndesigned and trained from scratch, and subsequently fine-tuned for tasks like clinical concept extraction, \nmedical relation extraction, semantic textual similarity, natural language inference, and medical question \nanswering. While there are domain -specific LLMs, they come with the same drawbacks of foundational \nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n   \n \n   \n \nmodels. Foundational models such as ChatGPT 3.5 itself can act as a support tool for breast tumor board, \nbut suffer from the lack of references, or the potential to produce seemingly credible but incorrect \nresponses11. In the domain of radiation oncology, ChatGPT 3.5 showed high accuracy and completeness in \nradiation oncology queries, but higher -than-recommended readability levels suggest the need for \nrefinement for improved patient accessibility and understanding 12. Newer models such as GPT -4 and \nmodels from Anthropic provide better responses, but all LLMs continue to have clinically significant error \nrates, including examples of overconfidence and consistent inaccuracies 13. In the context of treatment \nguidelines, ChatGPT provides concise, accessible supportive care advice including many non -medical \nsupport recommendations, but its recommendations lacked the specificity observed in National \nComprehensive Cancer Network ( NCCN) guidelines including often not suggesting any \nmedications14. These discrepancies with guidelines raise concerns for patient -facing symptom \nmanagement recommendations. This can be partly attributed to the fact that these models are trained on \npublicly available data and do not have sufficient specialized domain information. Models implemented on \nmore focused domains have better performance. CancerBERT 15 is an example of a model trained on a \nnarrower domain, but the model has only been evaluated on named entity recognition (NER) tasks. The \noutput of standalone models can be further optimized through retrieval augmented generation (RAG). In \nRAGs, the LLM retrieves information from pre-defined storage and synthesizes the response. External data \ncan be used to augment the response, thereby providing more context and reducing false information. For \nexample, in clinical trial screening, GPT -4 has shown promisin g performance when augmented with \nexternal data sources\n16. Similarly, GPT-4 has also been used for retrieving cancer guidelines. GPT- 4 with \nRAG provided significantly higher correct responses when compared to the standalone LLM service 17. \nRefAI18 is a similar tool that uses retrieval augmented generation to fetch medical literature in real time and \nsummarize them. These examples show the potential of retrieval augmented generation to mitigate the \nshortcomings of standalone LLM services.  \nWhile LLMs with extended context windows — such as those seen in models like Gemini 1.5 — show \nimprovements in retrieving relevant facts, these models still fall short of maximizing recall. For example, \nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n   \n \n   \n \nGemini 1.5 demonstrated superior accuracy over GPT- 4 at shorter context lengths but experienced a \nnoticeable drop in recall when dealing with extended contexts 19, achieving only around 60 -80% recall at \nhigher token limits. This gap means that even with the ability to handle massive amounts of information, \nthe LLM may fail to retrieve a significant portion  of available relevant data. This loss significantly \nundermines the system’s reliability in high-stakes fields like precision oncology, where critical pieces of \ninformation must not be overlooked.  \nWhile simple Retrieval-Augmented Generation (RAG) systems attempt to address the challenges of context \nand relevance by fetching documents to augment the LLM's output, they often reduce the LLM to the role of \na passive synthesizer. These systems rely heavily on external data sources, which the LLM incorporates \nwithout engaging its deeper reasoning capabilities. As a result, the LLM may act  more like a writing \nassistant, handling content without participating in any meaningful decision -making process. This \napproach can be limiting, particularly in complex fields like precision oncology, where critical clinical \ndecisions depend on integrating insights from multiple verticals — such as genomic profiles, treatment \nguidelines, and clinical trials. Simple retrieval mechanisms cannot adequately navigate these diverse data \nstreams or provide the nuanced reasoning required for personalized patient care. \nTo overcome these limitations, it is essential to move towards agentic systems that not only retrieve \ninformation but also actively reason over it. Unlike traditional RAG systems, which relegate LLMs to \nperforming static retrieval tasks, agent-based frameworks empower LLMs to function as dynamic reasoning \nsystems. In these frameworks, the LLM is not just a passive receiver of information, but an active \nparticipant in the decision -making loop, integrating data from various sources and engaging in logical \nsynthesis. The GeneSilico Copilot (GSCP) exemplifies this agentic approach. Powered by a bespoke ReAct \nagent\n 20, GSCP actively orchestrates multiple tools to retrieve and process information from curated \nsources. This multi -tool architecture allows the system to engage in complex reasoning, synthesizing \ninsights from a hybrid retrieval mechanism while dynamically filtering out irrelevant data. By leveraging \nthese tools, GSCP transcends the limitations of both long-context-window models and basic RAG systems, \nensuring that the LLM generates transparent, evidence- based responses that are contextual ly aware and \nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n   \n \n   \n \nclinically relevant. This agentic reasoning system is vital in oncology, where accurate and context-specific \nresponses can affect treatment choices. \nRESULTS \nGSCP , an agentic framework for precision oncology \nPrecision oncology requires the integration of detailed information from multiple domains, such as drug \ndosages, treatment guidelines, and clinical trials. A single retrieval system is often inadequate for navigating \nthis complexity. To address this, we propose an agent-based approach that enables the LLM to dynamically \ncontrol multiple specialized tools based on conversational context. \nAt the core of the GeneSilico Copilot (GSCP) is a modified ReAct agent, which improves upon the traditional \nreason-observation-action loop. When compared to commercially available language models services, \nwhat sets GSCP apart is its ability to show the thoughts and observations, along with links to the document \nsources while generating a response. The agent operates with two types of memory: conversational \nmemory for user interactions and working memory for managing tool -based processes. This distinction \nensures coherent communication with both users and tools.  The observations from the tools may often \nserve as additional related information for oncologists. These include treatment guidelines, drug dosage \ninformation, adverse reactions and other such information. \nThe GSCP retrieval system integrates a vector store  with hybrid search  (implemented via Qdrant) and a \nlanguage model. The vector store handles semantic retrieval, while the language model generates insights \nbased on the entire user query. After retrieving documents, the system re-ranks them for relevance before \nprocessing them further to create context-aware insights. \nWhile both the agent and tools utilize language models for interaction, only the agent makes decisions, \ndriving the overall reasoning process. This process involves analysing  the query, retrieving and filtering \nrelevant information via tools, and synthesizing a comprehensive response. Tool s play a crucial role in \ngenerating insights by filtering retrieved documents to those most relevant to the query. The agent then uses \nthese insights to formulate a refined, evidence-based response (Figure 1). \nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n   \n \n   \n \nFor example, if tasked with recommending personalized therapies, the agent directs the appropriate tool to \nsearch the vector store. The tool retrieves and filters the results, selecting only the most relevant documents \nand generating insights, which are then passed to the agent for decision-making. Similarly, for drug dosage \nrecommendations, a dedicated tool extracts and processes relevant data, providing refined insights for the \nagent to integrate into its reasoning loop (Figure 2). \nThe GSCP employs a hybrid vector search strategy, combining dense embeddings (generated using Voyage \nAI) for capturing semantic relationships and sparse embeddings (using SPLADE) for extracting key features. \nThe data corpus is categorized by source and use  case, pre-processed, and siloed into collections, with \neach tool performing vector searches and tailored post- processing to ensure high -quality, relevant data \nretrieval. \nSystematic curation of relevant documents for the GSCP vector store \nVector databases rely on document embeddings for indexing and retrieval based on semantic similarity. \nHowever, semantically similar documents, even if thematically unrelated, can have close vector \nrepresentations, leading to improper partitioning of the search space ( Figure 3a). This often results in \nclustering documents from different sources together, particularly in specialized domains like breast \ncancer. Previously, naive chunking of long documents further complicated retrieval by fragmenting context, \noften breaking critical information connections across document segments. \nIn the proposed vector storage setup,  we have eliminated chunking entirely, as modern LLMs can now \naccommodate entire documents in their context windows. This approach preserves the full document \ncontext, preventing the disruption caused by chunking and ensuring that essential relationships within the \ndocument remain intact. While this has addressed issues of context fragmentation, the embedding model \nis still limited by the number of tokens it can process. To resolve this, we now generate a summary of each \ndocument using LLMs and embed this summary. This ensures that the semantics are correctly encoded in \nthe vector. During retrieval, both the full document and its corresponding summary , along with all the \nmetadata are retrieved, allowing the agent to make use of both concise insights and the complete context \nof the document. \nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n   \n \n   \n \nOur system also tackles the challenge of overlapping vector spaces that arise from thematically similar \ndocuments. For example, breast cancer guidelines from different sources (such as NCCN and ASCO) can \nhave closely aligned content, leading to potential o verlaps in the retrieval process. This issue is \ncompounded by the imbalanced data volume across categories. For instance, breast cancer treatment \nguidelines from the National Comprehensive Cancer Network (NCCN) contain 17.1 thousand tokens, while \nclinical trial documents total 3.4 million tokens. The collated information from PubMed has a total of 460 \nthousand tokens.  The volume discrepancies are illustrated in Fig ure 3b . This can bias retrieval, with \nsmaller but crucial datasets being overshadowed by larger document sets. Despite being much smaller in \nvolume, the guidelines and the generic information were used to synthesize a lot of responses  during the \nevaluation process that is further illustrated in Figure 4.   \nTo address these challenges, we propose a siloed abstractive vector storage  system. This partitions the \ncorpus into thematically distinct silos— such as NCCN Guidelines, ASCO Guidelines, PubMed, PharmGKB, \nClinical Trials, and others. The storage of each silo is backed by a collection in the vector database. Each \nsilo is managed by a dedicated retrieval module that applies tailored filtering, summarization, and \nprocessing techniques, ensuring that each source is retrieved and used independently. This prevents the \nintermixing of guidelines or documents from diff erent sources, ensuring that responses are synthesized \nbased on silo -specific information. For instance, responses that involve treatment guidelines are pulled \nexclusively from their relevant silo without contamination from other guideline sources. \nWith the elimination of document chunking, each silo document now includes only a summarized \nabstraction, rather than a chunked representation. These summaries are designed to be context -specific, \nbased on the silo's purpose. For example, a summary focusing on clinical trial eligibility criteria will differ \nfrom one that highlights key findings from a landmark drug trial, even though both originate from the same \nclinical trial document. This abstraction allows the agent to grasp document semantics more effi ciently, \nreducing the number of reason-action loops required for response generation. \nWe utilize document summaries for both embedding and vector searches. Each document is summarized \nusing an LLM, and the summary is encoded into a vector for efficient retrieval. During vector search, these \nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n   \n \n   \n \nencoded summaries are used to identify relevant documents. However, when retrieving, the system fetches \nthe entire raw document, not just the summary, allowing the agent to have access to the full content. \nAfter retrieval, the relevant portion of the full document is selected based on the query and used to generate \ninsights and observations. These insights are crucial for the reasoning -action loop, where the agent \nsynthesizes information to generate a compre hensive response. By leveraging the full document content \nand not limiting retrieval to just the summary, the agent ensures that all contextually significant information \nis considered. This approach maintains the coherence of the original document while also benefiting from \nthe efficiency of summarized content during vector search. This strikes a balance between the \ncomputational efficiency of summary-based vector searches and the depth of information available from \nfull-document retrieval. \nGSCP agent improves response quality over general purpose LLMs and RAGs \nAgents act as a powerful approach for question answering tasks, particularly in the medical domain where \naccess to comprehensive and informative answers is crucial. However, evaluating the effectiveness of these \nsystems, especially in comparison to standalone LLMs or simpler RAG configurations, requires a rich and \ndiverse set of evaluation datasets. Such datasets should encompass a variety of question formats, difficulty \nlevels, and domains to provide a rigorous assessment of both context retrieval and resp onse generation \ncapabilities. \nThe absence of dedicated breast cancer question- answering datasets necessitated the creation of a \ncomprehensive evaluation suite. We combined publicly available medical question -answering datasets \nwith domain-specific samples encompassing both objective (multiple choice) questions from sources like \nMedMCQA21 and MedQA 22, and subjective (open ended) questions from sources like PubMedQA 23 and \ninternally constructed questions, all related to breast cancer ( Figure 5). For objective dataset, a Python \nscript was utilized to identify all potential answer choices within the dataset for each question. \nSubsequently, we searched  for the corresponding responses for the presence of these choices. This \napproach was essential as the correct answer was frequently not explicitly listed among the provided \noptions, thereby simplifying the downstream evaluation task.  \nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n   \n \n   \n \nThe application employs meticulously designed system prompts to enable the Reaction-Action or the ReAct \nloop as well as the thought and response generation process. The ReAct system prompt that is used here is \na standard public-domain prompt that generates “thoughts”, “actions”, and “observations”. To capture the \nfull complexity and subtleties of user query and generate detailed responses with evidence , we have an \nadditional system prompt that is used in the intermediate stages of the re sponse generation process . \nHowever, for evaluation purposes, this system custom prompt from the application is replaced to generate \nresponses that aligns with the format of the evaluation  to facilitate fair and consistent assessment. These \nprompts for evaluation are discussed in detail in the Supplementary Information, Section 1.  \nWe employed state -of-the-art LLM services, GPT -4 and Claude Opus -3, known for their long context \nwindows as well as their comparable performance on benchmark tests (Figure 6). The evaluation explored \nboth simple RAG and agentic configurations. \nFor Objective Question Answering  (QA) containing 223 questions, four quantitative metrics (accuracy, \nprecision, recall, F1-score) were used to assess performance. Agentic systems significantly outperformed \nboth RAG and standalone LLMs across all metrics. The GPT-4 powered agentic setup consistently achieved \nthe highest scores in accuracy, recall, F1-score. Both LLM services demonstrated comparable performance, \nwith GPT-4 exhibiting a slight edge. Agentic (GPT -4) consistently achieved the highest scores in a ccuracy, \nrecall, F1-score, and precision (0.83, 0.83, 0.83, and 0.83 respectively), followed by the Agentic setup with \nClaude Opus 3 with moderate success. Basic RAG  models showed mixed results, with relatively strong \nprecision but lower accuracy, recall, and F1 -score. Standalone LLMs exhibited the lowest performance \nacross all metrics.  \nFor Subjective QA consisting of 113 questions, the DeepEval framework evaluated retrieval and generation \nperformance for subjective questions. Standalone LLMs were excluded due to the absence of a retrieval \ncontext in their responses. Context precision, context relevancy, faithfulness, and answer relevancy \nmetrics were employed. Agentic systems achieved superior performance in both retrieval and generation \ntasks. In terms of context precision, Agentic (Claude Opus 3) led with a score of 0.44, followed by Agentic \n(GPT-4) at 0.37, while basic models struggled. Both Agentic models achieved high context relevancy scores \nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n   \n \n   \n \nof 0.27, significantly surpassing the basic RAG models. For answer relevancy, Agentic (GPT-4) excelled with \na score of 0.96, followed by Agentic (Claude Opus 3) at 0.90, while basic RAG models performed reasonably \nwell. Finally, while faithfulness scores were comparable overall, Agentic (Claude Opus 3) achieved a perfect \nscore of 1. \nA custom in -house dataset focusing on precision oncology and breast cancer genetics, containing 25 \nquestions, was created to simulate real -world healthcare complexities. This dataset, Precision Oncology \nQA, mimicked genetic markers, disease progression, and personalized treatment options. The same \nmetrics used for subjective questions were applied. Agentic systems significantly outperformed basic \nsystems in both context precision and relevancy. Agentic (Claude Opus 3) achieved a precision score of 0.51 \nand a relevancy score of 0.82, while Agentic (GPT -4) scored 0.52 for precision and 0.80 for relevancy. In \ncontrast, basic RAG systems showed lower scores, with Basic (Claude Opus 3) at 0.20 for precision and \n0.55 for relevancy, and Basic (GPT-4) at 0.15 for precision and 0.55 for relevancy. While answer relevancy \nand faithfulness scores were comparable across models, Agentic (Claude Opus 3) demonstrated slightly \nhigher faithfulness with a score of 0.98 compared to Basic (Claude Opus 3) at 0.85. These evaluation s \ndemonstrate that moving from a RAG configuration to the proposed Agentic setup will improve performance \nno matter which LLM service is being used. \nTo construct a faithful representation of the clinical setting, synthetic patient case studies were generated \nthrough a collaborative effort involving practicing oncologists and LLM services. Oncologists contributed \nessential clinical insights, ensuring the case studies accurately reflected real-world medical complexities. \nThese expert-provided details, devoid of specific patient data, served as the foundation for the LLMs to craft \ncomprehensive case studies. To further enhance the authenticity of these synthetic cases, oncologists were \nasked to review the generated case studies to validate their alignment with the real clinical reports . The \nperformance of agentic models, specifically Claude Opus 3 and GPT- 4, was assessed using these four \nsynthesized patient health records. In these experiments, the models were tasked with formulating suitable \ntreatment plans based exclusively on the presented patient data. The detailed model  outputs along with \ncan be found in Supplementary Information Section 2. \nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n   \n \n   \n \nDecoding the coordination among tools that improve GSCP response quality \nA key strength of the GeneSilico Copilot (GSCP) agent lies in its ability to intelligently coordinate a suite of \nspecialized tools for retrieval, processing, and reasoning across distinct topics. Each tool is responsible for \naccessing and processing content from specific silos, such as NCCN guidelines, PubMed, and clinical trial \ndata, based on the nature of the query. Every tool along with their descriptions and number of documents \nthat are retrieved from that silo is present in Supplementary Information, S ection 3 . This segmented \napproach ensures that the tool retrieves the most relevant information for a given query while maintaining \ntransparency in the agent's decision-making process. By exposing its reasoning steps, the agent provides \ninsight into how different sources contribute to the final response, allowing the end -user to better \nunderstand the weight and relevance of each information source (Supplementary Figure 1). \nFigure 7 breaks down the tool utilization over reasoning steps across different evaluation scenarios. For \ninstance, in objective case study questions, the agent heavily relies on NCCN guidelines, which are critical \nfor treatment protocols and recommendations. Meanwhile, PubMed is consistently utilized as a \nsupplemental tool to enrich responses with broader biomedical literature. This balance between targeted \nguidelines and general literature highlights the flexibility of the agent in adapting to various typ es of \nquestions and datasets. The coordination process typically occurs within a sequence of 3 to 5 reasoning \nsteps, as shown in the Figure 7. In each step, the agent identifies the relevant tool while the tool retrieves \ninformation, integrates the insights, and helps the agent in evolving the response. This breakdown not only \ndemonstrates the logical flow of the agent's reasoning but also showcases its ability to adaptively select the \nmost appropriate tool based on the specific needs of the query. The tool orchestration within this siloed \narchitecture significantly enhances the quality and precision of the responses generated, providing a \npowerful mechanism for clinical decision support (Supplementary Figure 2). \nDISCUSSION \nLarge language models  have demonstrated considerable potential across various domains, including \nhealthcare and biomedical research. However, limitations in transparency and robust evaluation \nmethodologies have hindered their full clinical integration.  GeneSilico CoPilot (GSCP) addresses these \nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n   \n \n   \n \nchallenges by proposing an agent -based framework that leverages the inherent reasoning capabilities of \nLLMs to plan and execute tasks within the healthcare domain.  This work focuses on the specific domain of \nbreast cancer, showcasing the advantages of the  GSCP framework over standalone LLMs and Retrieval -\nAugmented Generation (RAG) systems in both generic question answering and precision oncology tasks. \nA key differentiator of GSCP , when compared to commercially available language models is its ability to \nexpose the reasoning process. Once the user provides a query, the agent, as instructed by system prompts, \ncreates clinically relevant thoughts, actions and observations. Every observation is created with in -text \ncitations. Users can directly go to the URLs to verify the content of the observations from the tools. \nFurthermore, while not all parts of the observations may be necessary in forming the final re sponse, the \nobservations themselves act as crucial supplementary information. The final response also provides URL \nreferences, improving the trustworthiness of the system. \nEvaluations conducted across public and private datasets demonstrate the superiority of the proposed \nagent-based framework compared to traditional RAG systems.  In precision oncology question answering, \nGSCP achieved an improvement of up to 15.29% in answer faithfulness compared to RAGs.  Retrieval \nmetrics also showed significant improvement, with GSCP system achieving up to 200.83% and 47.27% \nbetter performance in context precision and context relevancy, respectively.  These results highlight the \nclear advantage of the agent's reasoning and retrieval mechanisms over basic RAG approaches.  Similar \nimprovements were observed in the subjective question answering dataset, where the GSCP agent achieved \nup to 93.65% and 2600% improvement in context precision and context relevancy, respectively.  The agent's \nretrieval mechanism facilitates a more robust reasoning process, and by incorporating these reasoning \nsteps into the response generation, GSC P system enhances the trustworthiness and transparency of its \nanswers. \nThe GSCP framework surpasses the limitations of simple RAG systems by employing a highly specialized \nbespoke ReAct agent that integrates breast cancer- specific clinical rationale into the reasoning loop. The \nagent leverages a dynamic prompt selection mecha nism to enhance its ability to process and synthesize \ninformation across various silos of biomedical data. \nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n   \n \n   \n \nFurthermore, GSCP employs a suite of specialized tools, each designed to interact with specific types of \ndata silos— such as guidelines, clinical trials, and biomedical literature. Each tool call is triggered by the \nReAct prompt but powered by the custom retrieval prompt that changes based on the user query and the \nsilo, which ensures that the tool extracts relevant information from the documents. The tool uses LLMs to \nassess the retrieved documents based on the agent's query, generating observations that contain evidence, \ncitations, and insights. The insights are critical, as they provide statements that logically support the user \nquery, contributing to the reasoning flow of the agent. The agent's final response is an outcome of the logical \nsynthesis of these observations. It uses the custom system prompt to generate a comprehensive and \ncoherent response, incorporating the relevant insights and evidence. This step ensures that GSCP agent \nproduces detailed, evidence-backed responses, offering clinicians greater confidence in the accuracy and \nrelevance of the system's output. This approach represents a significant evolution over traditional RAG \nsystems, improving both the depth and precision of the agent’s responses. \nOur evaluation revealed that Claude Opus 3 produced well -structured responses that resonated with \noncology experts, despite achieving lower overall evaluation scores compared to OpenA ’s GPT-4. While \nOpus 3 exhibited slower response generation times, often requiring up to three minutes to complete a \nresponse using provided tools, its outputs were characterized by superior readability. In terms of medical \naccuracy, Opus 3 offered more detailed explanations, including comprehensive drug and dosage \ninformation, while GPT-4 produced simpler responses. \nAlthough both systems demonstrated comparable levels of medical accuracy, the significant disparity in \nhuman-perceived readability suggests an inability of the DeepEval evaluation framework to fully capture \nthe nuanced aspects of response quality, particularly when considering human factors such as readability. \nThis finding underscores the limitations of relying solely on automated metrics to assess model \nperformance, particularly in complex domains such as medicine. While medical accuracy is undeniably \ncrucial, it is essential to recognize that it is not the sole determinant of response quality. A comprehensive \nevaluation should consider additional factors, such as response clarity, coherence, and overall clinical \nutility, as perceived by human experts. \nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n   \n \n   \n \nThe GSCP system's transparent planning process, which can be visualized through tool usage, provides \nvaluable insights into the relative importance of information sources. For example, our observations \nindicate a clear preference for NCCN guidelines over American Society of Clinical Oncology (ASCO) and  \nEuropean Society for Medical Oncology  ( ESMO) guidelines. It is noteworthy that NCCN guidelines \nunderwent a meticulous manual paraphrasing process to convert them into plain text while preserving the \ninformation conveyed in the original flowcharts. In contrast, ASCO and ESMO guidelines primarily reli ed \non LLM-based summarization. PubMed also emerged as a significant information source. While PubMed \noffers a wealth of open-access articles containing general knowledge, our focused initial retrieval process \neffectively transformed the PubMed collection i nto a more specialized corpus tailored to the domain of \noncology. Analysis of tool usage statistics can be leveraged to inform future optimizations of the data \nsources, potentially leading to the deprecation, consolidation, or replacement of certain sources based on \ntheir effectiveness within the agent's framework. \nFuture endeavours include expanding our testing to encompass real -life patient cases and evaluating the \nGSCP system's capabilities in therapeutic decision support. This necessitates the development and \nimplementation of robust and reproducible evaluation metrics. Current frameworks like DeepEval, which \nrely on LLM services for evaluation, are susceptible to inconsistencies. Therefore, there is a pressing need \nfor more sophisticated evaluation methods with human -in-the-loop specifically designed to assess th e \nplanning and reasoning capabilities of LLMs. \nThe GSCP system currently faces  some limitations in terms of processing speed. The tool usage and \nfrequent communication with the LLM service contribute to a processing delay, with complex cases \nrequiring up to five minutes for final response generation.  \nDeveloping a patient -specific treatment regimen requires meticulous evaluation of various factors, \nincluding the patient's medical history, comorbidities, prior treatments, and potential drug toxicities. This \nnecessitates a comprehensive review of the patient's medical records, encompassing laboratory results, \nimaging studies, and medication history. A thorough understanding of the patient's current health status \nand any coexisting conditions is also essential. Once this data is collected, the physician can  begin \nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n   \n \n   \n \nexploring treatment options aligned with established clinical guidelines from NCCN, ASCO, and ESMO. \nThese guidelines provide evidence- based recommendations informed by the latest research and clinical \nexperience. However, it is equally important to consider the patient's individual needs and preferences, as \nwell as their eligibility for ongoing clinical trials offering access to potentially groundbreaking therapies. \nThis complex decision-making process necessitates the synthesis of information from diverse sources. The \nGSCP system addresses this challenge by leveraging its knowledge base to recommend personalized \ntreatment plans for each patient case. This streamlines the physician's workflow, facilitates informed \ndecision-making, and ultimately contributes to enhanced patient care. \nIn conclusion, this work demonstrates the potential of developing domain -specific agent ic systems. By \nfocusing on a particular domain, such as oncology, the system can be optimized to effectively process and \ngenerate information within the context of a vast and complex data landscape. \nFIGURES \nFigure 1: Illustration of the working of the GeneSilico CoPilot \nSchematic workflow depicting the overall functionality of the agentic framework for precision oncology. The \nfirst step involves the collection and pre -processing of diverse medical data sources including literature, \nclinical trials, drug information and treatment guidelines – these serve as the tools. The sec ond step \ninvolves the retrieval process where to efficiently extract relevant information given a query by employing \nappropriate tool selection, hybrid search , re-ranking, and summarization. Further, the information \nretrieved is fed into a ReAct Agent that implements a cycle of reasoning, action (tool calls), and observation \nto generate insights. The final step involves the synthesis of the response containing the medical insights \nand recommendations that caters to the use cases such as personalized therapy recommendations, clinical \ntrial suggestions, genomic data analysis, and patient summaries. \nFigure 2: Flow of the process from a patient case to treatment plan \nAn oncologist provides a patient case study and prompts GSCP to recommend a treatment plan. Upon \nreceiving this query, GSCP engages in a structured Reason-Action-Observation process and synthesize the \nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n   \n \n   \n \nresponse based on the patient’s specific clinical details . Step 2 through 4 runs in a loop till the agent has \nenough information to form a coherent response using its clinical reasoning abilities.  \nFigure 3: Representation of the data in the vector database \nA) The  U- Map in 3D to visualize the vector space  that highlights the  overlaps and intersections among \nvarious topics ; B) The distribution of information across diverse topics within the vector database, \nillustrated through token counts, offers a comprehensive view of the content richness and topical breadth. \nToken count was generated using the tiktoken library. \nFigure 4: Representation of tool usage for diverse question-answering tasks \nTool usage distribution across three distinct question answering (QA) tasks: Objective QA, Subjective QA, \nand Precision Oncology QA (from left to right). In Objective QA, the NCCN Guidelines represent the largest \nreference source at 43.8 %, followed by PubMed (17.7%) and Wikipedia (14.8%). Subjective QA shows a \ndominant reliance on PubMed (39.9%), with NCCN Guidelines (2 0.1%) and ASCO Guidelines (11.4%) also \ncontributing significantly. In Precision Oncology QA, PubMed is the predominant source (62%), with NCCN \nGuidelines accounting for 14.1%. Each task demonstrates a distinct pattern of reference source utilization, \nwith NCCN Guidelines and PubMed playing major roles across all categories. \nFigure 5: Distribution of the QA dataset \nThe donut chart presents the breakdown of 361 total questions across the different types of question \nanswering tasks . Objective QA comprises 223 questions , sourced  from MedQA (124 questions) and \nMedMCQA (99 questions). Subjective QA includes 113 questions, derived from PubMedQA (61 questions) \nand InternalQA (52 questions) . Additionally, Precision Oncolo gy QA, developed in-house dataset \ncontributes 25 questions. \nFigure 6: Assessment of the performance of GSCP on question answering tasks \nPerformance evaluation of GSCP across three different types of question answering (QA) tasks: Objective \nQA, Subjective QA, and Precision Oncology QA. The metrics used for Objective QA include Accuracy, F1 -\nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n   \n \n   \n \nscore, Precision, and Recall, while for Subjective QA and Precision Oncology QA, the performance is \nevaluated using Faithfulness, Answer Relevancy, Context Precision, and Context Relevancy. The bar plots \nprovide a comparative view of these metrics across different datasets and models, highlighting the variation \nin performance. \nFigure 7: Methodical use of tools in step-by-step response synthesis \nThe Sankey diagrams illustrate the utilization of various tools across multiple steps in the response \ngeneration process for Objective QA, Subjective QA, and Precision Oncology QA tasks. Each chart \nrepresents the flow of tool usage from Step 1 through to Step 5, detailing the number of steps taken to \ngenerate responses and the specific tool used at each stage. \nMETHODS \nData sources \nGSCP leverages a collection of manually curated data sources specific to breast cancer, compiled with the \nsupport of practicing oncologists. These sources include standard breast cancer guidelines from the \nNational Comprehensive Cancer Network (NCCN), Amer ican Society of Clinical Oncology (ASCO), and \nEuropean Society for Medical Oncology (ESMO). \nTargeted Drug and Gene Information: To incorporate relevant drug and gene information, a curated list of \n68 genes (including HRR and pharmacogenomics genes) and their targeted drugs was compiled. A \ncustomized GeneSilico gene panel for breast cancer therapy recommendations was designed, \nencompassing these 68 genes. The selection criteria for these genes included: genes associated with \ntherapies (FDA-approved, Phase 3, and Phase 4 clinical trials); genes with high research significance and \nfrequent alterations in databases like Human Somatic Mutation Database ( HSMD) \n(digitalinsights.qiagen.com/hsmd/) and cBioPortal  ( www.cbioportal.org); genes associated with \nhomologous recombination repair (HRR) mechanism; pharmacogenomic (PGx) genes relevant to breast \ncancer; normalized codon length of genes; and key genes present in other somatic panels such as MSK -\nIMPACT (www.mskcc.org/msk-impact), Foundation Medicine CDx diagnostic panel  \nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n   \n \n   \n \n(https://www.foundationmedicine.in/our-services/cdx.html), and MedGenome panel  \n(diagnostics.medgenome.com). The rankings from these criteria were combined using a rank aggregation \nalgorithm to determine the final list of top genes, which were then manually validated. Pathogenic and likely \npathogenic variants in breast cancer were selected using the HSMD, COSMIC 24, and ClinVar25 databases. \nAdditionally, the GeneSilico gene panel for breast cancer includes  all exonic and a few selected intronic \nvariants for the 68 genes as well as 32 microsatellite instability (MSI) hotspots.  \nSubsequently, this targeted drug- gene list was used to extract drug data from Drugbank Open Data  \n(go.drugbank.com/releases/latest#open-data) 26, FDA drug labels (labels.fda.gov), RxList (www.rxlist.com), \nTherapeutic Target Database27, Drugs.com (www.drugs.com), and Wikipedia. Drug approval details were \nobtained from the FDA and ClinicalTrials.gov. We used the OpenFDA API and the Clinical Trials API to \naccess the information. PubMed information was gathered using the PMC E- Utilities. PharmGKB and \nJNCCN provided further breast cancer-specific literature.  \nData Abstraction and Summarization with Contextual Focus : To enhance context for lengthy documents, \nall documents were summarized  using LLM services. Instead of generic summaries, task -specific \nsummaries were created. This context-aware process facilitated the summarization of pertinent sections \nrather than entire documents.  The summarization was performed using Google Gemini 1.0 Pro. The \nsummarization prompt was provided depending on the requirement. For summarization of clinical trials \nfor eligibility criteria, the prompt was “Make the following clinical trial information \nconcise, highlighting the key eligibility criteria. Simply respond with the \nshortened text in markdown format.” For clinical trials which  contained drug approval \ninformation, the prompt was changed to “Give an abstract of the trial highlighting the \ndrug approval information. Simply respond with the shortened text in markdown \nformat.” In every case, the summary was formatted with markd own tags. In our experiments, using \nmarkdown tags improved the retrieval quality. This allowed the tools to fetch fewer documents and helped \noptimize the context window  use. Although storing documents with multiple summaries creates \nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n   \n \n   \n \nredundancy across silos, this approach enhance d agent performance. This summarization process was \napplied to all data sources containing long-form textual content. \nData Staging for Manageability and Retrieval: Data staging was implemented to improve manageability and \nretrieval efficiency. Before embedding, a copy of the data, along with extracted metadata, summaries, and \nURLs (for public domain documents), was stored in a NoSQL datastore . This was implemented using \nMongoDB. This approach simplifies and automates the embedding process. The stored URLs and metadata \nare utilized by system prompts for response refinement.  \nManual Curation for Complex Documents : Documents such as NCCN guidelines containing complex \ndiagrams and flowcharts, underwent manual paraphrasing and conversion into plain text while preserving \nthe step -by-step narrative. Furthermore, NCCN documents were segmented based on cancer subtype, \ntreatment phase, and treatment nature. Most other documents, including guidelines from ASCO and ESMO, \nwere summarized using LLMs followed by manual inspection. Each summary was further segmented and \nannotated with markdow n tags to enhance the agent's contextual understanding and facilitate the \ngeneration of more relevant responses. \nVector stores \nIn our vector store implementation, we employ a hybrid search strategy that leverages both dense and \nsparse vector embeddings to enhance retrieval performance. Specifically, we generate and store two types \nof embeddings for each document: a dense embedding  provided by a closed -source algorithm from \nVoyageAI, and a sparse embedding generated using SPLADE. During retrieval, we utilize a multi -stage \nprocess involving sparse vector matching, dense vector search, and Reciprocal Rank Fusion (RRF) to \nproduce the final ranked list of documents. \nEmbedding Generation: For the dense embeddings, we use a proprietary model from VoyageAI that maps \neach document summary 𝒅𝒅 and query 𝒒𝒒 into a 1536-dimensional dense vector space. The embeddings are \nrepresented as: \nd\ndense ∈  R𝟙𝟙𝟙𝟙𝟙𝟙𝟙𝟙 ,  qdense ∈  R𝟙𝟙𝟙𝟙𝟙𝟙𝟙𝟙 \nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n   \n \n   \n \nThese embeddings capture the semantic representations of the documents and queries in a continuous \nvector space, facilitating similarity computations based on their geometric properties. Even though the \ndocument summaries are encoded, the summaries themselves contain the semantics of the whole \ndocument and therefore, should contain the semantics of the entire document.  \nSimilarly, the sparse embeddings are generated using SPLADE, which produces high -dimensional sparse \nvectors reflecting the term importance within the vocabulary space. Let 𝑵𝑵 denote the size of the vocabulary. \nThe sparse embeddings for documents and queries are: \n𝑑𝑑𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠∈  𝑅𝑅𝑁𝑁 ,  𝑞𝑞𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠∈  𝑅𝑅𝑁𝑁 \nHere, N = 30522. Each element 𝑑𝑑𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠, 𝑖𝑖 or qsplade,i  corresponds to the importance of the i th term in the \nvocabulary for the document or query, respectively. The sparsity of these vectors ensures computational \nefficiency during matching. \nStorage of embeddings:  Both the dense and sparse embeddings are stored using 16 -bit floating- point \nprecision (float16) to maintain a balance between storage efficiency and computational accuracy. This dual-\nstorage approach allows us to utilize the strengths of both embedding types during retrieval . To facilitate \nefficient similarity searches during retrieval, we index the embeddings using the Hierarchical Navigable \nSmall World (HNSW) algorithm, implemented via Qdrant. The HNSW index is particularly well- suited for \nhigh-dimensional data, making it an ideal choice for our dense embeddings of dimension 1536 and sparse \nembeddings corresponding to the vocabulary size.  The HNSW index is configured with the following \nparameters: \nm=16: This parameter determines the number of bi -directional links created for each element during the \nconstruction of the HNSW graph. A higher m value increases the connectivity of the graph, potentially \nimproving search accuracy but also increasing memory consumption and indexing time. By setting m=16, \nwe achieve a balance between index performance and resource utilization. \nef_construct=100: This parameter controls the size of the dynamic list of nearest neighbors during the index \nconstruction phase. A larger ef_construct value leads to a more accurate and robust index at the cost of \nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n   \n \n   \n \nlonger indexing time. With ef_construct=100, we enhance the quality of the index without incurring \nexcessive computational overhead. \nfull_scan_threshold=10000: This threshold defines the dataset size below which the system will perform a \nfull brute-force scan instead of using the HNSW index. For datasets smaller than 10,000 vectors, a full scan \nis often more efficient due to the overhead associated with indexing. This ensures that we optimize retrieval \nperformance across different dataset sizes. \nR\netrieval: The retrieval process consists of three main stages: sparse vector pre -fetching, dense vector \nsearching, and Reciprocal Rank Fusion.  Initially, we pre -fetch n candidate documents from the corpus \nusing sparse vector matching based on the SPLADE embeddings. The matching score between a query q \nand a document d is computed using an inverse document frequency (IDF)-weighted inner product: \nScoresplade(𝑞𝑞, 𝑑𝑑) = � 𝐼𝐼𝐼𝐼𝐼𝐼(𝑖𝑖)\n𝑁𝑁\n𝑖𝑖=1\n⋅ 𝑞𝑞𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠,𝑖𝑖⋅ 𝑑𝑑splade,i \nHere, IDF(i) is the inverse document frequency of the i-th term, which emphasizes the significance of less \nfrequent terms in the matching process. The top n documents with the highest Scoresplade(𝑞𝑞, 𝑑𝑑) are selected \nfor further processing.  Within the pre -fetched set of n documents, we perform a dense vector search to \nretrieve the top m documents m < n  that are most semantically like the query. The similarity between the \ndense embeddings of the query and a document is calculated using cosine similarity: \nSimdense(𝑞𝑞, 𝑑𝑑) = cos(𝑞𝑞dense, ddense) = 𝑞𝑞𝑠𝑠𝑠𝑠𝑑𝑑𝑠𝑠𝑠𝑠\n⏉ ddense\n‖𝑞𝑞dense‖ ∙ ‖ddense‖ \nThe documents are ranked based on Simdense(𝑞𝑞, 𝑑𝑑) and the top m documents are selected for the final stage. \nTo combine the strengths of both the sparse and dense retrieval methods, we apply Reciprocal Rank Fusion \n(RRF) on the m documents obtained from the previous stage. RRF is an effective method for aggregating \nrankings from different sources by assigning hig her scores to documents that appear near the top of \nmultiple rankings.  \nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n   \n \n   \n \nLet 𝑟𝑟𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠(𝑑𝑑) be the rank of document d in the initial sparse ranking and 𝑟𝑟𝑠𝑠𝑠𝑠𝑑𝑑𝑠𝑠𝑠𝑠(𝑑𝑑) be its rank in the dense \nranking. The RRF score for each document is computed as: \nRRF(𝑑𝑑) = � 1\n𝑐𝑐+ 𝑟𝑟𝑠𝑠(𝑑𝑑)\n𝑠𝑠∈{splade,dense}\n \nwhere c is a constant to control the impact of the rank positions. The documents are then sorted based on \ntheir RRF scores, and the top k  documents (k<m) are selected as the final retrieval results.  The values of \nn=10, m=7, and k=5 is chosen based on empirical performance evaluations, ensuring that n>m>k to \nprogressively refine the candidate set. \nThe ReAct agent \nThe GSCP employs a bespoke ReAct agent, designed to simulate clinical reasoning processes for precision \noncology decision-making. The agent is built ground up using the LlamaIndex’s Workflows, an event-driven \narchitecture. Based on the general ReAct architecture but has been significantly modified to address the \nunique requirements of clinical workflows. \nReAct Architecture Overview: The ReAct architecture is a framework that enables agents to perform \ncomplex reasoning and action sequences by iteratively generating thoughts and actions based on user \ninputs. In this architecture, a system prompt guides the agent to utilize a set of tools to achieve the desired \noutcome. The workflow begins with the agent receiving a user query. The agent then formulates a thought \non how to address the query, which often involves selecting an appropriate tool to use. The action  is the \nexecution of this tool, and the result is an observation that informs the agent's next thought. This thought-\naction-observation loop continues until the agent arrives at a final response, effectively solving the user's \nquery through a combination o f reasoning and tool usage. The initial system prompt, referred to as the \nReAct system prompt, provides general instructions to the agent on how to perform these steps. \nModifications for Clinical Reasoning:  While the standard ReAct prompt is effective for general -purpose \nreasoning, it lacks the specificity required for clinical reasoning in oncology. To address this, we introduced \na second system prompt, the clinical reasoning system prompt. This specialized prompt takes precedence \nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n   \n \n   \n \nover the default ReAct prompt during key stages of the agent's workflow, particularly those involved with \nthought formation and output generation from observations. \nThis override is essential during these stages because clinical decision-making, especially in complex fields \nlike oncology, demands more than basic reasoning. The agent must process detailed patient data, cross -\nreference clinical guidelines, and synthesiz e information from diverse sources such as clinical trials and \ndrug information databases. By using a specialized prompt, the agent can: \n• Simulate the collective clinical reasoning typically exhibited by a virtual tumour board, considering \nfactors like patient history, disease progression, and personalized treatment options. \n• Ensure that the thoughts generated by the agent are aligned with clinically appropriate logic, driving \nthe selection of the most relevant tools and observations. \n• Prioritize and refine observations based on the established clinical standards, making the agent’s \ndecision-making loop more suited to precision oncology tasks. \nAgent Workflow: The ReAct agent's workflow is broken down into four distinct functions: \n1. Get User Query:  The agent receives the user’s input, typically a clinical question or patient case, \nwhich forms the starting point of the reasoning process. This is where the ReAct system prompt is \nalso introduced.  \n2. Prepare the Chat History: The agent structures the previous interactions and context into a coherent \nhistory, ensuring it can build upon prior information effectively. This is where the clinical reasoning \nsystem prompt takes over.  \n3. Generate Tool Calls (Create Thought): Now, using the clinical reasoning system prompt, the agent \nforms a thought and decides which tool to call. This thought is shaped by clinical logic, guiding the \nselection of appropriate tools for the query. \n4. Process Tool Calls (Get Observations): The selected tool returns observations, which are integrated \nback into the reasoning loop. This also collates and generates the citations. \nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n   \n \n   \n \nSteps 2, 3, and 4 are repeated iteratively until the final thought leads to a comprehensive response.  \nIterative Reasoning with Tool Feedback:  Each tool used by the agent operates with its own tool system \nprompt, which follows a shared template but is customized based on the tool’s function. The tools include \nmodules for retrieving clinical trial data, drug information, precision oncology guidelines, and general \nliterature searches. The tool system prompt is tasked with generating a summary relevant to the agent's \ncurrent query, given the documents or data it has accessed. This process involves distilling complex \nmedical information into actionable insights that the agent can use in its reasoning loop. Depending on the \ntool's nature, the prompt may instruct the tool to perform a tree -summarization—b reaking down \ninformation into hierarchical components— or to compact or refine information to highlight the most critical \nelements. As the agent processes observations from the tools, it continually refines its thoughts and \nactions. The clinical reasoning system prompt  guides this iterative process, encouraging the agent to \nconsider alternative hypotheses, weigh evidence, and explore different facets of the patient's case. This \niterative loop continues until the agent synthesizes a comprehensive response tha t reflects a deep \nunderstanding of the clinical scenario. \nIncorporation of Citations and Metadata: An essential aspect of clinical reasoning is the ability to reference \nauthoritative sources. Both the ReAct and clinical reasoning system prompts instruct the agent to generate \ncitations from metadata associated with the retrieved documents. Our vector store is constructed to \ninclude URLs and other reference information, allowing the agent to provide precise citations at each step \nof its reasoning. This feature not only adds credibility to the agent's responses but a lso enables users to \nverify and further explore the referenced material. \nMemory Architecture: To handle complex medical queries while efficiently managing the limited context \nwindow, the ReAct agent uses two distinct types of memory: \n1. Conversational Memory: This memory stores the user’s messages and the final agent responses. It \nhandles the primary conversation between the agent and the user, ensuring that important \ninteractions are preserved for continuity without overloading the context window. \nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n   \n \n   \n \n2. Working Memory: This memory tracks the internal conversations between the agent and the tools. \nIt stores the agent's tool -specific queries and the corresponding observations, keeping these \ninteractions separate from the user conversation. By isolating the tool interactions, the agent can \nmanage detailed queries and observations while freeing up space in the context window for ongoing \nuser interactions. \nThis separation of memory ensures efficient management of complex interactions, but it comes with one \nlimitation: the user cannot directly reference or ask questions about tool -generated observations since \nthese observations are stored in working memory. This drawback can prevent the user from delving into \nspecific details generated by a tool unless the agent explicitly includes them in the final response. This \nhighlights a limitation in flexibility when it comes to user-driven exploration of tool-generated insights. \nContext Overflow Handling: In scenarios where the response generation requires numerous reasoning loops \nand observations begin to quickly fill up the context window, GSCP employs insight extraction and \ncompression to manage context overflow efficiently specifically within the working memory. This process \nensures that the agent can continue its reasoning tasks without losing critical information or running out \nof context window capacity. Here, we compress the middle portion of the conversation while preservi ng \nthe most critical parts — the initial user input and the current reasoning step. When the content of the \ncurrent working memory has exceeded the context window, we then use another instance of the same LLM \nto create a compressed representation of the past observations  by extracting the necessary insights . By \ncompressing the middle sections of working memory and retaining the most important insights and final \nobservations, the agent ensures that it can continue its reasoning process without disruption. The final \nresponse is generated based on these insights, ensuring a comprehensive and logical outcome. \nExperimental setup \nWe evaluated the proposed method using datasets constructed from public sources and real -life cases. \nStandard public datasets for breast cancer are unavailable. Therefore, we extracted breast cancer-related \nquestions from multiple sources and categorized t hem as subjective (requiring long -form answers) or \nobjective (multiple -choice). A simple keyword -based search facilitated extraction, followed by manual \nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n   \n \n   \n \nreview by practicing oncologists to ensure question correctness. The objective dataset comprised 223 \nquestions from MedMCQA and MedQA (USMLE), while the subjective dataset consisted of 113 questions \nextracted from PubMedQA and an in- house dataset (InternalQA). For objective questions lacking a single \nclear answer where oncologists identified multiple correct options, the questions were reworked as \nsubjective ones. To simulate the complexities encountered by medical professionals in real-world oncology \npractice, we constructed a custom in -house dataset, the Precision Oncology dataset , of 25 questions \nfocused on precision oncology and breast cancer genetics. This dataset embodies case- study like \nscenarios, mimicking an oncologist's investigative process. The internal dataset as well as the precision \noncology datasets were created with the support of practicing oncologists. The dataset for evaluation is \nprovided in Supplementary Data.  \nAccuracy, F1 score, precision, and recall were used to assess system performance for simple multiple-\nchoice questions.  For the subjective and precision oncology datasets, t he DeepEval framework \n(https://docs.confident-ai.com/) evaluated our system and compared its performance to a RAG system. This \nframework employs Contextual Precision (ranking relevant information), Contextual Relevancy (overall \nretrieved context relevance), Faithfulness (factual alignment between response and retrieved context), and \nAnswer Relevancy (ratio of relevant statements in the answer) to measure the retrieval and generation \nperformance. \nDATA AVAILABILITY \nThe datasets used in this study are available in the supplementary materials. Supplementary Data file \ncontains evaluation datasets, including the objective questions extracted from MedMCQA and MedQA \n(USMLE), subjective questions from PubMedQA and our in -house dataset (InternalQA), and custom \nPrecision Oncology dataset. Our in-house datasets (InternalQA and Precision Oncology) were created with \nthe support of practicing oncologists and are included in the supplementary materials. The DeepEval \nframework used for performance evaluation is publicly accessible at https://docs.confident- ai.com/. Any \nadditional data that supports the findings of this study are available from the corresponding author upon \nreasonable request. \nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n   \n \n   \n \nSUPPLEMENTARY INFORMATION \nFiles \nSupplementary Information \nSection 1: Contains description of the datasets present in Supplementary Information Data and the \nprompts used to generate the responses for the evaluation.  \nSection 2: Contains selected synthetic case studies for intervention plan. These case studies were \ndesigned by oncologists based on real-life cases. The file contains the entire response provided by \nGSCP when presented with the case studies.  \nSection 3: Contains the configuration file of the different tools used by the agent. The “top  k” and \nthe “sparse k” determine the number of results fetched based on the dense and the sparse vector \nmatch respectively.  \nSupplementary Figure 1: Implementation of the agent. \nSupplementary Figure 2: A visual explainer of the reasoning process for a case study.  \nSupplementary Data \nExcel spreadsheet consisting of the questions and the corresponding ground truth for the datasets \n– Subjective, Objective and Precision Oncology. \nCompeting interests  \nA provisional patent has been filed (Attorney Docket No. 107881-00005) describing the agentic framework. \nAll the authors, except for SB are GeneSilico. Inc. employees/partners.  \n  \nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n   \n \n   \n \n \nREFERENCES \n1. Haslam, A., Kim, M. S. & Prasad, V. Updated estimates of eligibility for and response to genome -\ntargeted oncology drugs among US cancer patients, 2006 -2020. Annals of Oncology  32, 926 –932 \n(2021). \n2. Thirunavukarasu, A. J. et al.  Large language models in medicine. Nature Medicine 2023 29:8  29, \n1930–1940 (2023). \n3. Gilbert, S., Harvey, H., Melvin, T., Vollebregt, E. & Wicks, P . Large language model AI chatbots require \napproval as medical devices. Nature Medicine 2023 29:10 29, 2396–2398 (2023). \n4. Zhao, H. et al. Explainability for Large Language Models: A Survey. ACM Trans Intell Syst Technol 15, \n38 (2024). \n5. Haltaufderheide, J. & Ranisch, R. The ethics of ChatGPT in medicine and healthcare: a systematic \nreview on Large Language Models (LLMs). npj Digital Medicine 2024 7:1 7, 1–11 (2024). \n6. Lee, P ., Bubeck, S. & Petro, J. Benefits, Limits, and Risks of GPT-4 as an AI Chatbot for Medicine. N. \nEngl. J. Med. 388, 1233–1239 (2023). \n7. Singhal, K. et al. Large language models encode clinical knowledge. Nature 2023 620:7972 620, 172–\n180 (2023). \n8. Luo, R. et al. BioGPT: generative pre-trained transformer for biomedical text generation and mining. \nBrief Bioinform 23, 1–11 (2022). \n9. Lee, J. et al. BioBERT: a pre-trained biomedical language representation model for biomedical text \nmining. Bioinformatics 36, 1234–1240 (2020). \n10. Yang, X. et al. A large language model for electronic health records. npj Digital Medicine 2022 5:1 5, \n1–9 (2022). \nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n   \n \n   \n \n11. Sorin, V. et al. Large language model (ChatGPT) as a support tool for breast tumor board. npj Breast \nCancer 2023 9:1 9, 1–4 (2023). \n12. Yalamanchili, A. et al.  Quality of Large Language Model Responses to Radiation Oncology Patient \nCare Questions. JAMA Netw Open 7, e244630–e244630 (2024). \n13. Rydzewski, N. R. et al. Comparative Evaluation of LLMs in Clinical Oncology. NEJM AI 1, (2024). \n14. Lazris, D., Schenker, Y. & Thomas, T. Exploring AI-generated content and professional guidelines in \ncancer symptom management: A comparative analysis between ChatGPT and NCCN guidelines. \nJournal of Clinical Oncology 42, e13610–e13610 (2024). \n15. Zhou, S., Wang, N., Wang, L., Liu, H. & Zhang, R. CancerBERT: a cancer domain -specific language \nmodel for extracting breast cancer phenotypes from electronic health records. Journal of the \nAmerican Medical Informatics Association 29, 1208–1216 (2022). \n16. Tan, R. et al.  Retrieval-augmented large language models for clinical trial screening. \nhttps://doi.org/10.1200/JCO.2024.42.16_suppl.e13611 42, e13611–e13611 (2024). \n17. Ferber, D. et al. GPT-4 for Information Retrieval and Comparison of Medical Oncology Guidelines. \nNEJM AI 1, (2024). \n18. Li, Y. et al.  RefAI: a GPT -powered retrieval-augmented generative tool for biomedical literature \nrecommendation and summarization. Journal of the American Medical Informatics Association  \n(2024) doi:10.1093/JAMIA/OCAE129. \n19. Gemini Team et al.  Gemini 1.5: Unlocking multimodal understanding across millions of tokens of \ncontext. (2024). \n20. Yao, S. et al. ReAct: Synergizing Reasoning and Acting in Language Models. (2022). \n21. Pal, A., Umapathi, L. K. & Sankarasubbu, M. MedMCQA: A Large -scale Multi-Subject Multi-Choice \nDataset for Medical domain Question Answering. Proceedings of Machine Learning Research vol. 174 \n248–260 Preprint at https://proceedings.mlr.press/v174/pal22a.html (2022). \nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n   \n \n   \n \n22. Jin, D. et al. What Disease Does This Patient Have? A Large-Scale Open Domain Question Answering \nDataset from Medical Exams. Applied Sciences 2021, Vol. 11, Page 6421 11, 6421 (2021). \n23. Jin, Q., Dhingra, B., Liu, Z., Cohen, W. W. & Lu, X. PubMedQA: A Dataset for Biomedical Research \nQuestion Answering. EMNLP -IJCNLP 2019 - 2019 Conference on Empirical Methods in Natural \nLanguage Processing and 9th International Joint Conference on Natural Language Processing, \nProceedings of the Conference 2567–2577 (2019) doi:10.18653/v1/d19-1259. \n24. Bamford, S. et al. The COSMIC (Catalogue of Somatic Mutations in Cancer) database and website. \nBritish Journal of Cancer 2004 91:2 91, 355–358 (2004). \n25. Landrum, M. J. et al. ClinVar: public archive of relationships among sequence variation and human \nphenotype. Nucleic Acids Res 42, D980–D985 (2014). \n26. Wishart, D. S. et al. DrugBank 5.0: a major update to the DrugBank database for 2018. Nucleic Acids \nRes 46, D1074–D1082 (2018). \n27. Chen, X., Ji, Z. L. & Chen, Y. Z. TTD: Therapeutic Target Database. Nucleic Acids Res  30, 412–415 \n(2002). \n  \n  \nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n   \n \n   \n \nACKNOWLEDGEMENTS \nThe authors would like to thank Claire Cohen from GeneSilico Inc., and Ruhani Bhatia for their valuable \ninput. \nAUTHOR CONTRIBUTION STATEMENT \nDS conceived the study. RD and KM designed and implemented the agent and performed all experiments \nunder the supervision of DS. SS, JN, and SM defined the scope for gene panel, and data ingestion. JN and \nSS also performed evaluations and internal checkpoints for review. JN generated diagrams. AP also \nimplemented DeepEval.  NA implemented the vector database and MongoDB, contributed to the agent \ndesign, and led the productionization efforts. AP assisted with vector database implementation, conducted \ndata scraping and collation, and developed tools for agent use. VU and AS developed and deployed the \ncopilot web application in production, with support from TS and HC. RT, SP , and SN provided clinical input \nto ensure the relevance of data and outputs. RT further contributed to the design of clinical questions and \nresponses. BG, UM, SB, and MF provided critical feedback on the agent design. All authors reviewed and \napproved the final manuscript. \n \n \nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n/gid00053/gid00064/gid00082/gid00074/gid00082\n/gid00051/gid00068/gid00034/gid00066/gid00083/gid00001/gid00034/gid00070/gid00068/gid00077/gid00083/gid00051/gid00068/gid00083/gid00081/gid00068/gid00072/gid00085/gid00068/gid00081/gid00082\n/gid00051/gid00068/gid00014/gid00081/gid00064/gid00077/gid00074/gid00068/gid00081\n/gid00036/gid00078/gid00077/gid00083/gid00068/gid00087/gid00083/gid00084/gid00064/gid00075\n/gid00052/gid00084/gid00076/gid00076/gid00064/gid00081/gid00072/gid00089/gid00068/gid00081\n/gid00041/gid00088/gid00065/gid00081/gid00072/gid00067/gid00001/gid00052/gid00068/gid00064/gid00081/gid00066/gid00071\n/gid00034/gid00066/gid00083/gid00072/gid00078/gid00077\n/gid00009/gid00053/gid00078/gid00078/gid00075/gid00001/gid00066/gid00064/gid00075/gid00075/gid00082/gid00010\n/gid00051/gid00068/gid00064/gid00082/gid00078/gid00077/gid00072/gid00077/gid00070\n/gid00048/gid00065/gid00082/gid00068/gid00081/gid00085/gid00064/gid00083/gid00072/gid00078/gid00077/gid00001/gid00064/gid00077/gid00067\n/gid00042/gid00077/gid00082/gid00072/gid00070/gid00071/gid00083/gid00082\n/gid00040/gid00068/gid00077/gid00068/gid00081/gid00064/gid00075/gid00001/gid00050/gid00007/gid00034\n/gid00036/gid00075/gid00072/gid00077/gid00072/gid00066/gid00064/gid00075/gid00001/gid00053/gid00081/gid00072/gid00064/gid00075\n/gid00082/gid00084/gid00070/gid00070/gid00068/gid00082/gid00083/gid00072/gid00078/gid00077/gid00082\n/gid00049/gid00068/gid00081/gid00082/gid00078/gid00077/gid00064/gid00075/gid00072/gid00089/gid00068/gid00067/gid00001\n/gid00037/gid00081/gid00084/gid00070/gid00001/gid00037/gid00078/gid00082/gid00064/gid00070/gid00068/gid00001\n/gid00036/gid00064/gid00077/gid00066/gid00068/gid00081/gid00001\n/gid00040/gid00068/gid00077/gid00078/gid00076/gid00072/gid00066/gid00082/gid00001/gid00050/gid00007/gid00034\n/gid00049/gid00064/gid00083/gid00072/gid00068/gid00077/gid00083\n/gid00052/gid00084/gid00076/gid00076/gid00064/gid00081/gid00088\n/gid00049/gid00068/gid00081/gid00082/gid00078/gid00077/gid00075/gid00072/gid00089/gid00068/gid00067/gid00001\n/gid00053/gid00071/gid00068/gid00081/gid00064/gid00079/gid00088/gid00001\n/gid00051/gid00068/gid00066/gid00078/gid00076/gid00076/gid00068/gid00077/gid00067/gid00064/gid00083/gid00072/gid00078/gid00077\n/gid00037/gid00064/gid00083/gid00064/gid00001/gid00052/gid00078/gid00084/gid00081/gid00066/gid00068/gid00082\n/gid00053/gid00078/gid00078/gid00075/gid00001/gid00053/gid00088/gid00079/gid00068/gid00082\n/gid00049/gid00084/gid00065/gid00046/gid00068/gid00067\n/gid00034/gid00065/gid00082/gid00083/gid00081/gid00064/gid00066/gid00083/gid00082\n/gid00043/gid00047/gid00036/gid00036/gid00047/gid00013/gid00001\n/gid00049/gid00071/gid00064/gid00081/gid00076/gid00040/gid00044/gid00035\n/gid00048/gid00034/gid00001/gid00034/gid00081/gid00083/gid00072/gid00066/gid00075/gid00068/gid00082\n/gid00051/gid00087/gid00045/gid00072/gid00082/gid00083/gid00013\n/gid00039/gid00037/gid00034\n/gid00047/gid00036/gid00036/gid00047/gid00013/gid00001\n/gid00038/gid00052/gid00046/gid00048/gid00013/gid00001/gid00034/gid00052/gid00036/gid00048\n/gid00053/gid00071/gid00068/gid00081/gid00064/gid00079/gid00088\n/gid00040/gid00084/gid00072/gid00067/gid00068/gid00075/gid00072/gid00077/gid00068/gid00082\n/gid00036/gid00075/gid00072/gid00077/gid00072/gid00066/gid00064/gid00075\n/gid00083/gid00081/gid00072/gid00064/gid00075/gid00082/gid00015/gid00070/gid00078/gid00085\n/gid00036/gid00075/gid00072/gid00077/gid00072/gid00066/gid00064/gid00075/gid00001/gid00053/gid00081/gid00072/gid00064/gid00075/gid00082\n/gid00037/gid00081/gid00084/gid00070/gid00001/gid00042/gid00077/gid00069/gid00078/gid00081/gid00076/gid00064/gid00083/gid00072/gid00078/gid00077\n/gid00053/gid00078/gid00078/gid00075/gid00001/gid00071/gid00064/gid00077/gid00067/gid00075/gid00072/gid00077/gid00070\n/gid00079/gid00081/gid00078/gid00076/gid00079/gid00083/gid00082\n/gid00036/gid00075/gid00072/gid00077/gid00072/gid00066/gid00064/gid00075/gid00001/gid00081/gid00068/gid00064/gid00082/gid00078/gid00077/gid00072/gid00077/gid00070\n/gid00079/gid00081/gid00078/gid00076/gid00079/gid00083/gid00082\n/gid00046/gid00068/gid00083/gid00064/gid00067/gid00064/gid00083/gid00064\n/gid00071/gid00064/gid00077/gid00067/gid00075/gid00068/gid00081\n/gid00046/gid00068/gid00076/gid00078/gid00081/gid00088\n/gid00076/gid00064/gid00077/gid00064/gid00070/gid00068/gid00076/gid00068/gid00077/gid00083\nFigure 1\nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n/gid00034/gid00001/gid00021/gid00019/gid00014/gid00088/gid00068/gid00064/gid00081/gid00014/gid00078/gid00075/gid00067/gid00001/gid00086/gid00078/gid00076/gid00064/gid00077/gid00001/gid00086/gid00072/gid00083/gid00071/gid00001/gid00076/gid00068/gid00083/gid00064/gid00082/gid00083/gid00064/gid00083/gid00072/gid00066/gid00001/gid00041/gid00038/gid00051/gid00019/gid00012/gid00001/gid00065/gid00081/gid00068/gid00064/gid00082/gid00083/gid00001/gid00066/gid00064/gid00077/gid00066/gid00068/gid00081/gid00001/gid00071/gid00064/gid00082/gid00001/gid00064/gid00001/gid00070/gid00068/gid00077/gid00068/gid00083/gid00072/gid00066/gid00001/gid00079/gid00081/gid00078/gid00195/gid00075/gid00068/gid00001/gid00072/gid00077/gid00067/gid00072/gid00066/gid00064/gid00083/gid00072/gid00077/gid00070/gid00001/gid00041/gid00038/gid00051/gid00019/gid00001/gid00064/gid00076/gid00079/gid00075/gid00072/gid00195/gid00066/gid00064/gid00083/gid00072/gid00078/gid00077/gid00013/gid00001/gid00049/gid00042/gid00044/gid00020/gid00036/gid00034/gid00001\n/gid00041/gid00018/gid00017/gid00021/gid00024/gid00051/gid00001/gid00076/gid00084/gid00083/gid00064/gid00083/gid00072/gid00078/gid00077/gid00013/gid00001/gid00049/gid00053/gid00038/gid00047/gid00001/gid00075/gid00078/gid00082/gid00082/gid00013/gid00001/gid00038/gid00051/gid00035/gid00035/gid00020/gid00001/gid00052/gid00020/gid00018/gid00017/gid00039/gid00001/gid00076/gid00084/gid00083/gid00064/gid00083/gid00072/gid00078/gid00077/gid00013/gid00001/gid00064/gid00077/gid00067/gid00001/gid00036/gid00037/gid00041/gid00018/gid00001/gid00083/gid00081/gid00084/gid00077/gid00066/gid00064/gid00083/gid00072/gid00078/gid00077/gid00015/gid00001/gid00046/gid00051/gid00042/gid00001/gid00078/gid00069/gid00001/gid00083/gid00071/gid00068/gid00001/gid00065/gid00081/gid00064/gid00072/gid00077/gid00001/gid00082/gid00071/gid00078/gid00086/gid00082/gid00001/gid00083/gid00086/gid00078/gid00001/gid00082/gid00076/gid00064/gid00075/gid00075/gid00001/gid00076/gid00068/gid00083/gid00064/gid00082/gid00083/gid00064/gid00083/gid00072/gid00066/gid00001\n/gid00075/gid00068/gid00082/gid00072/gid00078/gid00077/gid00082/gid00001 /gid00072/gid00077/gid00001 /gid00083/gid00071/gid00068/gid00001 /gid00066/gid00068/gid00081/gid00068/gid00065/gid00068/gid00075/gid00075/gid00084/gid00076/gid00013/gid00001 /gid00064/gid00077/gid00067/gid00001 /gid00064/gid00001 /gid00081/gid00068/gid00066/gid00068/gid00077/gid00083/gid00001 /gid00068/gid00066/gid00071/gid00078/gid00066/gid00064/gid00081/gid00067/gid00072/gid00078/gid00070/gid00081/gid00064/gid00076/gid00001 /gid00072/gid00077/gid00067/gid00072/gid00066/gid00064/gid00083/gid00068/gid00082/gid00001 /gid00064/gid00001 /gid00081/gid00068/gid00067/gid00084/gid00066/gid00068/gid00067/gid00001 /gid00068/gid00073/gid00068/gid00066/gid00083/gid00072/gid00078/gid00077/gid00001 /gid00069/gid00081/gid00064/gid00066/gid00083/gid00072/gid00078/gid00077/gid00001 /gid00009/gid00021/gid00017/gid00006/gid00010/gid00001 /gid00066/gid00078/gid00077/gid00082/gid00072/gid00082/gid00083/gid00068/gid00077/gid00083/gid00001 /gid00086/gid00072/gid00083/gid00071/gid00001\n/gid00066/gid00078/gid00077/gid00070/gid00068/gid00082/gid00083/gid00072/gid00085/gid00068/gid00001 /gid00071/gid00068/gid00064/gid00081/gid00083/gid00001 /gid00069/gid00064/gid00072/gid00075/gid00084/gid00081/gid00068/gid00015/gid00001 /gid00052/gid00071/gid00068/gid00001 /gid00071/gid00064/gid00082/gid00001 /gid00071/gid00088/gid00079/gid00068/gid00081/gid00083/gid00068/gid00077/gid00082/gid00072/gid00078/gid00077/gid00013/gid00001 /gid00086/gid00071/gid00072/gid00066/gid00071/gid00001 /gid00072/gid00082/gid00001 /gid00066/gid00084/gid00081/gid00081/gid00068/gid00077/gid00083/gid00075/gid00088/gid00001 /gid00086/gid00068/gid00075/gid00075/gid00014/gid00076/gid00064/gid00077/gid00064/gid00070/gid00068/gid00067/gid00001 /gid00086/gid00072/gid00083/gid00071/gid00001 /gid00076/gid00068/gid00067/gid00072/gid00066/gid00064/gid00083/gid00072/gid00078/gid00077/gid00013/gid00001 /gid00064/gid00077/gid00067/gid00001 /gid00081/gid00068/gid00079/gid00078/gid00081/gid00083/gid00082/gid00001 /gid00076/gid00072/gid00075/gid00067/gid00001\n/gid00067/gid00072/gid00089/gid00089/gid00072/gid00077/gid00068/gid00082/gid00082/gid00001/gid00064/gid00077/gid00067/gid00001/gid00082/gid00071/gid00078/gid00081/gid00083/gid00077/gid00068/gid00082/gid00082/gid00001/gid00078/gid00069/gid00001/gid00065/gid00081/gid00068/gid00064/gid00083/gid00071/gid00001/gid00078/gid00077/gid00001/gid00068/gid00087/gid00068/gid00081/gid00083/gid00072/gid00078/gid00077/gid00015\n/gid00049/gid00064/gid00083/gid00072/gid00068/gid00077/gid00083/gid00082/gid00001 /gid00086/gid00072/gid00083/gid00071/gid00001 /gid00065/gid00081/gid00064/gid00072/gid00077/gid00001 /gid00076/gid00068/gid00083/gid00064/gid00082/gid00083/gid00064/gid00082/gid00068/gid00082/gid00001 /gid00082/gid00071/gid00078/gid00084/gid00075/gid00067/gid00001 /gid00081/gid00068/gid00066/gid00068/gid00072/gid00085/gid00068/gid00001 /gid00064/gid00079/gid00079/gid00081/gid00078/gid00079/gid00081/gid00072/gid00064/gid00083/gid00068/gid00001 /gid00075/gid00078/gid00066/gid00064/gid00075/gid00001 /gid00083/gid00071/gid00068/gid00081/gid00064/gid00079/gid00088/gid00001 /gid00009/gid00082/gid00084/gid00081/gid00070/gid00068/gid00081/gid00088/gid00013/gid00001 /gid00086/gid00071/gid00078/gid00075/gid00068/gid00014/gid00065/gid00081/gid00064/gid00072/gid00077/gid00001 /gid00081/gid00064/gid00067/gid00072/gid00078/gid00083/gid00071/gid00068/gid00081/gid00064/gid00079/gid00088/gid00013/gid00001\n/gid00082/gid00083/gid00068/gid00081/gid00068/gid00078/gid00083/gid00064/gid00066/gid00083/gid00072/gid00066/gid00001/gid00081/gid00064/gid00067/gid00072/gid00078/gid00082/gid00084/gid00081/gid00070/gid00068/gid00081/gid00088/gid00010/gid00001/gid00064/gid00077/gid00067/gid00001/gid00082/gid00088/gid00082/gid00083/gid00068/gid00076/gid00072/gid00066/gid00001/gid00083/gid00071/gid00068/gid00081/gid00064/gid00079/gid00088/gid00001/gid00072/gid00069/gid00001/gid00072/gid00077/gid00067/gid00072/gid00066/gid00064/gid00083/gid00068/gid00067/gid00015/gid00001/gid00053/gid00081/gid00064/gid00082/gid00083/gid00084/gid00089/gid00084/gid00076/gid00064/gid00065/gid00001/gid00067/gid00068/gid00081/gid00084/gid00087/gid00083/gid00068/gid00066/gid00064/gid00077/gid00001/gid00072/gid00082/gid00001/gid00064/gid00001/gid00081/gid00068/gid00066/gid00078/gid00076/gid00076/gid00068/gid00077/gid00067/gid00068/gid00067/gid00001/gid00082/gid00068/gid00066/gid00078/gid00077/gid00067/gid00014/gid00075/gid00072/gid00077/gid00068/gid00001\n/gid00083/gid00071/gid00068/gid00081/gid00064/gid00079/gid00088/gid00001 /gid00069/gid00078/gid00081/gid00001 /gid00041/gid00038/gid00051/gid00019/gid00012/gid00001 /gid00076/gid00068/gid00083/gid00064/gid00082/gid00083/gid00064/gid00083/gid00072/gid00066/gid00001 /gid00065/gid00081/gid00068/gid00064/gid00082/gid00083/gid00001 /gid00066/gid00064/gid00077/gid00066/gid00068/gid00081/gid00013/gid00001 /gid00079/gid00064/gid00081/gid00083/gid00072/gid00066/gid00084/gid00075/gid00064/gid00081/gid00075/gid00088/gid00001 /gid00069/gid00078/gid00081/gid00001 /gid00079/gid00064/gid00083/gid00072/gid00068/gid00077/gid00083/gid00082/gid00001 /gid00086/gid00071/gid00078/gid00001 /gid00071/gid00064/gid00085/gid00068/gid00001 /gid00079/gid00081/gid00078/gid00070/gid00081/gid00068/gid00082/gid00082/gid00068/gid00067/gid00001 /gid00078/gid00077/gid00001 /gid00083/gid00081/gid00064/gid00082/gid00083/gid00084/gid00089/gid00084/gid00076/gid00064/gid00065/gid00001 /gid00064/gid00077/gid00067/gid00001\n/gid00079/gid00068/gid00081/gid00083/gid00084/gid00089/gid00084/gid00076/gid00064/gid00065/gid00015/gid00015/gid00015/gid00001\n/gid00053/gid00071/gid00068/gid00001/gid00079/gid00064/gid00083/gid00072/gid00068/gid00077/gid00083/gid00099/gid00082/gid00001/gid00066/gid00064/gid00081/gid00067/gid00072/gid00078/gid00085/gid00064/gid00082/gid00066/gid00084/gid00075/gid00064/gid00081/gid00001/gid00079/gid00081/gid00078/gid00195/gid00075/gid00068/gid00013/gid00001/gid00072/gid00077/gid00066/gid00075/gid00084/gid00067/gid00072/gid00077/gid00070/gid00001/gid00064/gid00001/gid00081/gid00068/gid00067/gid00084/gid00066/gid00068/gid00067/gid00001/gid00068/gid00073/gid00068/gid00066/gid00083/gid00072/gid00078/gid00077/gid00001/gid00069/gid00081/gid00064/gid00066/gid00083/gid00072/gid00078/gid00077/gid00001/gid00064/gid00077/gid00067/gid00001/gid00082/gid00088/gid00076/gid00079/gid00083/gid00078/gid00076/gid00082/gid00001/gid00078/gid00069/gid00001/gid00066/gid00078/gid00077/gid00070/gid00068/gid00082/gid00083/gid00072/gid00085/gid00068/gid00001/gid00071/gid00068/gid00064/gid00081/gid00083/gid00001/gid00069/gid00064/gid00072/gid00075/gid00084/gid00081/gid00068/gid00013/gid00001\n/gid00077/gid00068/gid00066/gid00068/gid00082/gid00082/gid00072/gid00083/gid00064/gid00083/gid00068/gid00082/gid00001/gid00064/gid00001/gid00066/gid00064/gid00081/gid00068/gid00069/gid00084/gid00075/gid00001/gid00064/gid00079/gid00079/gid00081/gid00078/gid00064/gid00066/gid00071/gid00001/gid00083/gid00078/gid00001/gid00083/gid00081/gid00068/gid00064/gid00083/gid00076/gid00068/gid00077/gid00083/gid00001/gid00082/gid00068/gid00075/gid00068/gid00066/gid00083/gid00072/gid00078/gid00077/gid00015/gid00015/gid00015\nPreferred regime: Paclitaxel + trastuzumab, TCH, TCHP\nDosage: Paclitaxel: 80 mg/m2 IV weekly for 12 weeks, \nTrastuzumab: 4 mg/kg IV loading dose, then 2 mg/kg \nIV weekly for 1 year. Docetaxel: 75 mg/m2 IV day 1, \nCarboplatin: AUC 6 IV day 1, Cycled every 21 days for ...\nTool: nccn_guidelines\nQuery: HER2+ breast cancer treatment\nTools Observations\nFirst-line: Trastuzumab, pertuzumab, and taxane \ncombination...\nSecond-line: Trastuzumab deruxtecan\nThird-line and beyond: Options include regimens with \nneratinib, lapatinib, abemaciclib plus trastuzumab ...\nTool: asco_guidelines\nQuery: HER2+ breast cancer treatment\nHER-2 directed antibody attached to a topoisomerase \ninhibitor, unique cleavable peptide linker, fully \napproved by FDA in May 2022.\nPotential risks: Neutropenia, Interstitial lung disease...\nPharmacokinetics: Half-life of 5.8 days, ...\nTool: drug_information\nQuery: trastuzumab deruxtecan\nInhibits phosphatidylinositol-3-kinase (PI3K), Highest \nspeci/f_icity for PI3Kα, PI3Kα's p110α catalytic subunit is \noften mutated in cancers, leading to hyperactivation\nHalf-life: 8 to 9 hours\nFood effects: High-fat meals increase drug exposure\nTool: drug_information\nQuery: taselisib\nSummarizer\nLLM Service\nTool-speci/f_ic \nprompts\nVector store\nRetriever, \nReranker\n1 Agent receives user query\n2 Agent invokes\ntools for information\n5\nAgent \ngenerates \nresponse\n4 Agent generates thoughts from \nobservations\n3\nAgent receives observations\nAgent\nFigure 2\nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n/gid00034/gid00052/gid00036/gid00048/gid00001/gid00040/gid00084/gid00072/gid00067/gid00068/gid00075/gid00072/gid00077/gid00068/gid00082\n/gid00036/gid00075/gid00072/gid00077/gid00072/gid00066/gid00064/gid00075/gid00001/gid00053/gid00081/gid00072/gid00064/gid00075/gid00082\n/gid00036/gid00075/gid00072/gid00077/gid00072/gid00066/gid00064/gid00075/gid00001/gid00053/gid00081/gid00072/gid00064/gid00075/gid00082/gid00001/gid00051/gid00068/gid00069/gid00068/gid00081/gid00068/gid00077/gid00066/gid00068/gid00082\n/gid00037/gid00081/gid00084/gid00070/gid00082/gid00001/gid00042/gid00077/gid00069/gid00078/gid00081/gid00076/gid00064/gid00083/gid00072/gid00078/gid00077\n/gid00038/gid00052/gid00046/gid00048/gid00001/gid00040/gid00084/gid00072/gid00067/gid00068/gid00075/gid00072/gid00077/gid00068/gid00082\n/gid00043/gid00047/gid00036/gid00036/gid00047\n/gid00045/gid00064/gid00077/gid00067/gid00076/gid00064/gid00081/gid00074/gid00001/gid00036/gid00075/gid00072/gid00077/gid00072/gid00066/gid00064/gid00075/gid00001/gid00053/gid00081/gid00072/gid00064/gid00075/gid00082\n/gid00047/gid00036/gid00036/gid00047/gid00001/gid00040/gid00084/gid00072/gid00067/gid00068/gid00075/gid00072/gid00077/gid00068/gid00082\n/gid00047/gid00036/gid00036/gid00047/gid00001/gid00051/gid00068/gid00069/gid00068/gid00081/gid00068/gid00077/gid00066/gid00068/gid00082\n/gid00049/gid00071/gid00064/gid00081/gid00076/gid00040/gid00044/gid00035\n/gid00049/gid00084/gid00065/gid00046/gid00068/gid00067\n/gid00056/gid00072/gid00074/gid00072/gid00079/gid00068/gid00067/gid00072/gid00064\n/gid00052/gid00078/gid00084/gid00081/gid00066/gid00068/gid00027/gid00001/gid00036/gid00075/gid00072/gid00077/gid00072/gid00066/gid00064/gid00075/gid00001/gid00053/gid00081/gid00072/gid00064/gid00075/gid00082\n/gid00053/gid00072/gid00083/gid00075/gid00068/gid00027/gid00001/gid00047/gid00036/gid00053/gid00017/gid00022/gid00017/gid00023/gid00026/gid00017/gid00020/gid00025\n/gid00036/gid00078/gid00077/gid00083/gid00068/gid00077/gid00083/gid00027/gid00001/gid00036/gid00075/gid00072/gid00077/gid00072/gid00066/gid00064/gid00075/gid00001/gid00053/gid00081/gid00072/gid00064/gid00075/gid00001/gid00034/gid00082/gid00082/gid00068/gid00082/gid00082/gid00072/gid00077/gid00070/gid00001/gid00083/gid00071/gid00068/gid00001/gid00052/gid00064/gid00069/gid00068/gid00083/gid00088/gid00001/gid00078/gid00069/gid00001\n/gid00047/gid00068/gid00078/gid00064/gid00067/gid00073/gid00084/gid00085/gid00064/gid00077/gid00083/gid00001/gid00049/gid00064/gid00075/gid00065/gid00078/gid00066/gid00072/gid00066/gid00075/gid00072/gid00065/gid00001/gid00072/gid00077/gid00001/gid00036/gid00078/gid00076/gid00065/gid00072/gid00077/gid00064/gid00083/gid00072/gid00078/gid00077/gid00001/gid00056/gid00072/gid00083/gid00071/gid00001\n/gid00038/gid00077/gid00067/gid00078/gid00066/gid00081/gid00072/gid00077/gid00068/gid00001/gid00053/gid00071/gid00068/gid00081/gid00064/gid00079/gid00088\n/gid00052/gid00078/gid00084/gid00081/gid00066/gid00068/gid00027/gid00001/gid00049/gid00084/gid00065/gid00046/gid00068/gid00067\n/gid00053/gid00072/gid00083/gid00075/gid00068/gid00027/gid00001/gid00041/gid00038/gid00051/gid00019/gid00014/gid00049/gid00078/gid00082/gid00072/gid00083/gid00072/gid00085/gid00068/gid00001/gid00046/gid00068/gid00083/gid00064/gid00082/gid00083/gid00064/gid00083/gid00072/gid00066/gid00001/gid00036/gid00078/gid00075/gid00078/gid00081/gid00068/gid00066/gid00083/gid00064/gid00075/gid00001/gid00036/gid00064/gid00077/gid00066/gid00068/gid00081\n/gid00036/gid00078/gid00077/gid00083/gid00068/gid00077/gid00083/gid00027/gid00001/gid00053/gid00081/gid00064/gid00082/gid00083/gid00084/gid00089/gid00084/gid00076/gid00064/gid00065/gid00013/gid00001/gid00064/gid00001/gid00041/gid00038/gid00051/gid00019/gid00014/gid00083/gid00064/gid00081/gid00070/gid00068/gid00083/gid00068/gid00067/gid00001/gid00064/gid00070/gid00068/gid00077/gid00083/gid00013/gid00001/gid00072/gid00082/gid00001\n/gid00084/gid00082/gid00068/gid00067/gid00001/gid00072/gid00077/gid00001/gid00066/gid00078/gid00076/gid00065/gid00072/gid00077/gid00064/gid00083/gid00072/gid00078/gid00077/gid00001/gid00086/gid00072/gid00083/gid00071/gid00001/gid00083/gid00084/gid00066/gid00064/gid00083/gid00072/gid00077/gid00072/gid00065/gid00001/gid00015/gid00015/gid00015\n/gid00052/gid00078/gid00084/gid00081/gid00066/gid00068/gid00027/gid00001/gid00049/gid00071/gid00064/gid00081/gid00076/gid00040/gid00044/gid00035\n/gid00053/gid00072/gid00083/gid00075/gid00068/gid00027/gid00001/gid00034/gid00067/gid00073/gid00084/gid00082/gid00083/gid00072/gid00077/gid00070/gid00001/gid00083/gid00071/gid00068/gid00001/gid00067/gid00078/gid00082/gid00068/gid00001/gid00078/gid00069/gid00001/gid00083/gid00064/gid00076/gid00078/gid00087/gid00072/gid00069/gid00068/gid00077/gid00001/gid00072/gid00077/gid00001/gid00015/gid00015/gid00015\n/gid00036/gid00078/gid00077/gid00083/gid00068/gid00077/gid00083/gid00027/gid00001/gid00036/gid00058/gid00049/gid00019/gid00037/gid00023/gid00001/gid00072/gid00082/gid00001/gid00064/gid00001/gid00074/gid00068/gid00088/gid00001/gid00068/gid00077/gid00089/gid00088/gid00076/gid00068/gid00001/gid00072/gid00077/gid00001/gid00083/gid00064/gid00076/gid00078/gid00087/gid00072/gid00069/gid00068/gid00077/gid00001\n/gid00076/gid00068/gid00083/gid00064/gid00065/gid00078/gid00075/gid00072/gid00082/gid00076/gid00013/gid00001/gid00083/gid00081/gid00064/gid00077/gid00082/gid00069/gid00078/gid00081/gid00076/gid00072/gid00077/gid00070/gid00001/gid00072/gid00083/gid00001/gid00072/gid00077/gid00083/gid00078/gid00001/gid00072/gid00083/gid00082/gid00001/gid00076/gid00064/gid00072/gid00077/gid00001/gid00015/gid00015/gid00015\n/gid00052/gid00078/gid00084/gid00081/gid00066/gid00068/gid00027/gid00001/gid00037/gid00081/gid00084/gid00070/gid00001/gid00042/gid00077/gid00069/gid00078/gid00081/gid00076/gid00064/gid00083/gid00072/gid00078/gid00077\n/gid00053/gid00072/gid00083/gid00075/gid00068/gid00027/gid00001/gid00049/gid00064/gid00075/gid00065/gid00078/gid00066/gid00072/gid00066/gid00075/gid00072/gid00065/gid00001/gid00037/gid00068/gid00082/gid00066/gid00081/gid00072/gid00079/gid00083/gid00072/gid00078/gid00077\n/gid00036/gid00078/gid00077/gid00083/gid00068/gid00077/gid00083/gid00027/gid00001/gid00049/gid00064/gid00075/gid00065/gid00078/gid00066/gid00072/gid00066/gid00075/gid00072/gid00065/gid00013/gid00001/gid00064/gid00001/gid00079/gid00072/gid00079/gid00068/gid00081/gid00064/gid00089/gid00072/gid00077/gid00068/gid00001/gid00079/gid00088/gid00081/gid00072/gid00067/gid00078/gid00079/gid00088/gid00014\n/gid00081/gid00072/gid00076/gid00072/gid00067/gid00072/gid00077/gid00068/gid00001/gid00066/gid00078/gid00076/gid00079/gid00078/gid00084/gid00077/gid00067/gid00013/gid00001/gid00072/gid00082/gid00001/gid00064/gid00001/gid00082/gid00068/gid00066/gid00078/gid00077/gid00067/gid00014/gid00070/gid00068/gid00077/gid00068/gid00081/gid00064/gid00083/gid00072/gid00078/gid00077/gid00015/gid00015/gid00015\n/gid00037/gid00064/gid00083/gid00064/gid00001/gid00052/gid00078/gid00084/gid00081/gid00066/gid00068/gid00082\n/gid00053/gid00078/gid00074/gid00068/gid00077/gid00001/gid00036/gid00078/gid00084/gid00077/gid00083\n/gid00018/gid00024/gid00015/gid00025/gid00074\n/gid00023/gid00020/gid00023/gid00074\n/gid00021/gid00019/gid00015/gid00022/gid00074\n/gid00021/gid00015/gid00019/gid00020/gid00074\n/gid00020/gid00026/gid00022/gid00074\n/gid00022/gid00017/gid00021/gid00074\n/gid00018/gid00024/gid00015/gid00018/gid00074\n/gid00022/gid00017/gid00023/gid00074\n/gid00018/gid00026/gid00018/gid00074\n/gid00021/gid00023/gid00017/gid00074\n/gid00018/gid00017/gid00021/gid00074\n/gid00017\n/gid00018/gid00017/gid00017/gid00074\n/gid00019/gid00017/gid00017/gid00074\n/gid00020/gid00017/gid00017/gid00074\n/gid00021/gid00017/gid00017/gid00074\n/gid00022/gid00017/gid00017/gid00074\n/gid00023/gid00017/gid00017/gid00074\n/gid00024/gid00017/gid00017/gid00074\n/gid00020/gid00015/gid00022/gid00046\n/gid00020/gid00015/gid00021/gid00046\n/gid00034/gid00052/gid00036/gid00048/gid00001/gid00040/gid00084/gid00072/gid00067/gid00068/gid00075/gid00072/gid00077/gid00068/gid00082 /gid00036/gid00075/gid00072/gid00077/gid00072/gid00066/gid00064/gid00075/gid00001/gid00053/gid00081/gid00072/gid00064/gid00075/gid00082 /gid00036/gid00075/gid00072/gid00077/gid00072/gid00066/gid00064/gid00075/gid00001/gid00053/gid00081/gid00072/gid00064/gid00075/gid00082/gid00001/gid00051/gid00068/gid00069/gid00068/gid00081/gid00068/gid00077/gid00066/gid00068/gid00082 /gid00037/gid00081/gid00084/gid00070/gid00082/gid00001/gid00042/gid00077/gid00069/gid00078/gid00081/gid00076/gid00064/gid00083/gid00072/gid00078/gid00077 /gid00038/gid00052/gid00046/gid00048/gid00001/gid00040/gid00084/gid00072/gid00067/gid00068/gid00075/gid00072/gid00077/gid00068/gid00082 /gid00043/gid00047/gid00036/gid00036/gid00047 /gid00045/gid00064/gid00077/gid00067/gid00076/gid00064/gid00081/gid00074/gid00001/gid00036/gid00075/gid00072/gid00077/gid00072/gid00066/gid00064/gid00075/gid00001/gid00053/gid00081/gid00072/gid00064/gid00075/gid00082 /gid00047/gid00036/gid00036/gid00047/gid00001/gid00040/gid00084/gid00072/gid00067/gid00068/gid00075/gid00072/gid00077/gid00068/gid00082 /gid00047/gid00036/gid00036/gid00047/gid00001/gid00051/gid00068/gid00069/gid00068/gid00081/gid00068/gid00077/gid00066/gid00068/gid00082 /gid00049/gid00071/gid00064/gid00081/gid00076/gid00040/gid00044/gid00035 /gid00049/gid00084/gid00065/gid00046/gid00068/gid00067 /gid00056/gid00072/gid00074/gid00072/gid00079/gid00068/gid00067/gid00072/gid00064\n/gid00064\n/gid00065\nFigure 3\nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n/gid00034/gid00052/gid00036/gid00048/gid00001/gid00040/gid00084/gid00072/gid00067/gid00068/gid00075/gid00072/gid00077/gid00068/gid00082\n/gid00021/gid00015/gid00019/gid00020/gid00006\n/gid00036/gid00075/gid00072/gid00077/gid00072/gid00066/gid00064/gid00075/gid00001/gid00053/gid00081/gid00072/gid00064/gid00075/gid00082\n/gid00019/gid00015/gid00025/gid00019/gid00006\n/gid00043/gid00047/gid00036/gid00036/gid00047\n/gid00018/gid00015/gid00021/gid00018/gid00006\n/gid00036/gid00075/gid00072/gid00077/gid00072/gid00066/gid00064/gid00075/gid00001/gid00053/gid00081/gid00072/gid00064/gid00075/gid00082\n/gid00021/gid00015/gid00019/gid00019/gid00006\n/gid00037/gid00081/gid00084/gid00070/gid00082\n/gid00019/gid00015/gid00023/gid00006\n/gid00036/gid00075/gid00072/gid00077/gid00072/gid00066/gid00064/gid00075/gid00001/gid00053/gid00081/gid00072/gid00064/gid00075/gid00001/gid00051/gid00068/gid00069/gid00068/gid00081/gid00068/gid00077/gid00066/gid00068/gid00082\n/gid00018/gid00015/gid00023/gid00019/gid00006\n/gid00045/gid00064/gid00077/gid00067/gid00076/gid00064/gid00081/gid00074/gid00001/gid00036/gid00075/gid00072/gid00077/gid00072/gid00066/gid00064/gid00075/gid00001/gid00053/gid00081/gid00072/gid00064/gid00075/gid00082\n/gid00017/gid00015/gid00023/gid00021/gid00026/gid00006\n/gid00038/gid00052/gid00046/gid00048/gid00001/gid00040/gid00084/gid00072/gid00067/gid00068/gid00075/gid00072/gid00077/gid00068/gid00082\n/gid00017/gid00015/gid00020/gid00019/gid00022/gid00006\n/gid00047/gid00036/gid00036/gid00047/gid00001/gid00051/gid00068/gid00069/gid00068/gid00081/gid00068/gid00077/gid00066/gid00068/gid00082\n/gid00017/gid00015/gid00020/gid00019/gid00022/gid00006\n/gid00049/gid00084/gid00065/gid00046/gid00068/gid00067\n/gid00023/gid00019/gid00006\n/gid00047/gid00036/gid00036/gid00047/gid00001/gid00040/gid00084/gid00072/gid00067/gid00068/gid00075/gid00072/gid00077/gid00068/gid00082\n/gid00018/gid00021/gid00015/gid00021/gid00006\n/gid00056/gid00072/gid00074/gid00072/gid00079/gid00068/gid00067/gid00072/gid00064\n/gid00026/gid00015/gid00025/gid00023/gid00006\n/gid00037/gid00081/gid00084/gid00070/gid00082/gid00001/gid00042/gid00077/gid00069/gid00078/gid00015/gid00001\n/gid00022/gid00015/gid00023/gid00020/gid00006\n/gid00049/gid00084/gid00065/gid00046/gid00068/gid00067\n/gid00020/gid00026/gid00015/gid00026/gid00006\n/gid00047/gid00036/gid00036/gid00047/gid00001/gid00040/gid00084/gid00072/gid00067/gid00068/gid00075/gid00072/gid00077/gid00068/gid00082\n/gid00019/gid00017/gid00015/gid00018/gid00006\n/gid00034/gid00052/gid00036/gid00048/gid00001/gid00040/gid00084/gid00072/gid00067/gid00068/gid00075/gid00072/gid00077/gid00068/gid00082\n/gid00018/gid00018/gid00015/gid00021/gid00006\n/gid00056/gid00072/gid00074/gid00072/gid00079/gid00068/gid00067/gid00072/gid00064\n/gid00026/gid00015/gid00024/gid00021/gid00006\n/gid00043/gid00047/gid00036/gid00036/gid00047\n/gid00026/gid00015/gid00017/gid00026/gid00006\n/gid00036/gid00075/gid00072/gid00077/gid00072/gid00066/gid00064/gid00075/gid00001/gid00053/gid00081/gid00072/gid00064/gid00075/gid00082\n/gid00020/gid00015/gid00023/gid00022/gid00006\n/gid00036/gid00075/gid00072/gid00077/gid00072/gid00066/gid00064/gid00075/gid00001/gid00053/gid00081/gid00072/gid00064/gid00075/gid00001/gid00051/gid00068/gid00069/gid00068/gid00081/gid00068/gid00077/gid00066/gid00068/gid00082\n/gid00017/gid00015/gid00024/gid00025/gid00018/gid00006\n/gid00045/gid00064/gid00077/gid00067/gid00076/gid00064/gid00081/gid00074/gid00001/gid00036/gid00075/gid00072/gid00077/gid00072/gid00066/gid00064/gid00075/gid00001/gid00053/gid00081/gid00072/gid00064/gid00075/gid00082\n/gid00017/gid00015/gid00022/gid00019/gid00018/gid00006\n/gid00047/gid00036/gid00036/gid00047/gid00001/gid00051/gid00068/gid00069/gid00068/gid00081/gid00068/gid00077/gid00066/gid00068/gid00082\n/gid00017/gid00015/gid00019/gid00023/gid00006\n/gid00038/gid00052/gid00046/gid00048/gid00001/gid00040/gid00084/gid00072/gid00067/gid00068/gid00075/gid00072/gid00077/gid00068/gid00082\n/gid00017/gid00015/gid00019/gid00023/gid00006\n/gid00047/gid00036/gid00036/gid00047/gid00001/gid00040/gid00084/gid00072/gid00067/gid00068/gid00075/gid00072/gid00077/gid00068/gid00082\n/gid00021/gid00020/gid00015/gid00025/gid00006\n/gid00049/gid00084/gid00065/gid00046/gid00068/gid00067\n/gid00018/gid00024/gid00015/gid00024/gid00006\n/gid00056/gid00072/gid00074/gid00072/gid00079/gid00068/gid00067/gid00072/gid00064/gid00001\n/gid00018/gid00021/gid00015/gid00025/gid00006\n/gid00034/gid00052/gid00036/gid00048/gid00001/gid00040/gid00084/gid00072/gid00067/gid00075/gid00072/gid00077/gid00068/gid00082\n/gid00018/gid00017/gid00015/gid00024/gid00006\n/gid00037/gid00081/gid00084/gid00070/gid00082/gid00001/gid00042/gid00077/gid00069/gid00078/gid00015\n/gid00024/gid00015/gid00022/gid00022/gid00006\n/gid00048/gid00065/gid00073/gid00068/gid00066/gid00083/gid00072/gid00085/gid00068/gid00001/gid00050/gid00034 /gid00052/gid00084/gid00065/gid00073/gid00068/gid00066/gid00083/gid00072/gid00085/gid00068/gid00001/gid00050/gid00034 /gid00049/gid00081/gid00068/gid00066/gid00072/gid00082/gid00072/gid00078/gid00077/gid00001/gid00048/gid00077/gid00066/gid00078/gid00075/gid00078/gid00070/gid00088/gid00001/gid00050/gid00034\nFigure 4\nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n/gid00049/gid00081/gid00068/gid00066/gid00072/gid00082/gid00072/gid00078/gid00077/gid00001/gid00048/gid00077/gid00066/gid00078/gid00075/gid00078/gid00070/gid00088/gid00001/gid00050/gid00034\n/gid00009/gid00019/gid00022/gid00010\n/gid00046/gid00068/gid00067/gid00050/gid00034\n/gid00009/gid00018/gid00019/gid00021/gid00010\n/gid00046/gid00068/gid00067/gid00046/gid00036/gid00050/gid00034\n/gid00009/gid00026/gid00026/gid00010\n/gid00048/gid00065/gid00073/gid00068/gid00066/gid00083/gid00072/gid00085/gid00068/gid00001/gid00050/gid00034\n/gid00009/gid00019/gid00019/gid00020/gid00010\n/gid00052/gid00084/gid00065/gid00073/gid00068/gid00066/gid00083/gid00072/gid00085/gid00068/gid00001/gid00050/gid00034\n/gid00009/gid00018/gid00018/gid00020/gid00010\n/gid00049/gid00084/gid00065/gid00046/gid00068/gid00067/gid00050/gid00034\n/gid00009/gid00023/gid00018/gid00010\n/gid00042/gid00077/gid00083/gid00068/gid00081/gid00077/gid00064/gid00075/gid00050/gid00034/gid00001\n/gid00009/gid00022/gid00019/gid00010\n/gid00053/gid00078/gid00083/gid00064/gid00075/gid00001/gid00068/gid00085/gid00064/gid00075/gid00084/gid00064/gid00083/gid00072/gid00078/gid00077\n/gid00050/gid00034\n/gid00009/gid00020/gid00023/gid00018/gid00010\nFigure 5\nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n/gid00017/gid00015/gid00024/gid00021\n/gid00017/gid00015/gid00025/gid00020\n/gid00017/gid00015/gid00021/gid00020\n/gid00017/gid00015/gid00024/gid00026\n/gid00017/gid00015/gid00017/gid00021\n/gid00017/gid00015/gid00021/gid00024\n/gid00017/gid00015/gid00017/gid00021\n/gid00017/gid00015/gid00019/gid00021\n/gid00017/gid00015/gid00021/gid00020\n/gid00017/gid00015/gid00023/gid00020\n/gid00017/gid00015/gid00025/gid00020\n/gid00034/gid00066/gid00066/gid00084/gid00081/gid00064/gid00066/gid00088 /gid00039/gid00018/gid00014/gid00082/gid00066/gid00078/gid00081/gid00068 /gid00049/gid00081/gid00068/gid00066/gid00072/gid00082/gid00072/gid00078/gid00077 /gid00051/gid00068/gid00066/gid00064/gid00075/gid00075\n/gid00048/gid00065/gid00073/gid00068/gid00066/gid00083/gid00072/gid00085/gid00068/gid00001/gid00050/gid00034\n/gid00017/gid00015/gid00024/gid00021\n/gid00017/gid00015/gid00025/gid00020\n/gid00017/gid00015/gid00022/gid00018\n/gid00017/gid00015/gid00024/gid00025\n/gid00017/gid00015/gid00017/gid00024\n/gid00017/gid00015/gid00022/gid00026\n/gid00017/gid00015/gid00017/gid00024\n/gid00017/gid00015/gid00019/gid00023\n/gid00017/gid00015/gid00021/gid00022\n/gid00017/gid00015/gid00023/gid00021\n/gid00017/gid00015/gid00025/gid00020\n/gid00017/gid00015/gid00024/gid00022\n/gid00017/gid00015/gid00025/gid00020\n/gid00017/gid00015/gid00024/gid00018\n/gid00017/gid00015/gid00024/gid00026\n/gid00017/gid00015/gid00022/gid00025\n/gid00017/gid00015/gid00025/gid00019\n/gid00017/gid00015/gid00022/gid00025\n/gid00017/gid00015/gid00023/gid00021\n/gid00017/gid00015/gid00024\n/gid00017/gid00015/gid00024/gid00024\n/gid00017/gid00015/gid00025/gid00020\n/gid00017/gid00015/gid00024/gid00021\n/gid00017/gid00015/gid00025/gid00020\n/gid00017/gid00015/gid00021/gid00020\n/gid00017/gid00015/gid00024/gid00026\n/gid00017/gid00015/gid00017/gid00021\n/gid00017/gid00015/gid00021/gid00024\n/gid00017/gid00015/gid00017/gid00021\n/gid00017/gid00015/gid00019/gid00021\n/gid00017/gid00015/gid00021/gid00020\n/gid00017/gid00015/gid00023/gid00020\n/gid00017/gid00015/gid00025/gid00020\n/gid00034/gid00070/gid00068/gid00077/gid00083/gid00072/gid00066/gid00001/gid00009/gid00036/gid00075/gid00064/gid00084/gid00067/gid00068/gid00001/gid00048/gid00079/gid00084/gid00082/gid00001/gid00020/gid00010 /gid00034/gid00070/gid00068/gid00077/gid00083/gid00072/gid00066/gid00001/gid00009/gid00040/gid00049/gid00053/gid00014/gid00021/gid00010/gid00035/gid00064/gid00082/gid00072/gid00066/gid00001/gid00009/gid00036/gid00075/gid00064/gid00084/gid00067/gid00068/gid00001/gid00048/gid00079/gid00084/gid00082/gid00001/gid00020/gid00010 /gid00035/gid00064/gid00082/gid00072/gid00066/gid00001/gid00009/gid00040/gid00049/gid00053/gid00014/gid00021/gid00010\n/gid00034/gid00070/gid00068/gid00077/gid00083/gid00072/gid00066/gid00001/gid00009/gid00036/gid00075/gid00064/gid00084/gid00067/gid00068/gid00001/gid00048/gid00079/gid00084/gid00082/gid00001/gid00020/gid00010 /gid00034/gid00070/gid00068/gid00077/gid00083/gid00072/gid00066/gid00001/gid00009/gid00040/gid00049/gid00053/gid00014/gid00021/gid00010/gid00035/gid00064/gid00082/gid00072/gid00066/gid00001/gid00009/gid00036/gid00075/gid00064/gid00084/gid00067/gid00068/gid00001/gid00048/gid00079/gid00084/gid00082/gid00001/gid00020/gid00010 /gid00035/gid00064/gid00082/gid00072/gid00066/gid00001/gid00009/gid00040/gid00049/gid00053/gid00014/gid00021/gid00010\n/gid00052/gid00083/gid00064/gid00077/gid00067/gid00064/gid00075/gid00078/gid00077/gid00068/gid00001/gid00009/gid00036/gid00075/gid00064/gid00084/gid00067/gid00068/gid00001/gid00048/gid00079/gid00084/gid00082/gid00001/gid00020/gid00010 /gid00052/gid00083/gid00064/gid00077/gid00067/gid00064/gid00075/gid00078/gid00077/gid00068/gid00001/gid00009/gid00040/gid00049/gid00053/gid00014/gid00021/gid00010 /gid00034/gid00070/gid00068/gid00077/gid00083/gid00072/gid00066/gid00001/gid00009/gid00036/gid00075/gid00064/gid00084/gid00067/gid00068/gid00001/gid00048/gid00079/gid00084/gid00082/gid00001/gid00020/gid00010 /gid00034/gid00070/gid00068/gid00077/gid00083/gid00072/gid00066/gid00001/gid00009/gid00040/gid00049/gid00053/gid00014/gid00021/gid00010/gid00035/gid00064/gid00082/gid00072/gid00066/gid00001/gid00009/gid00036/gid00075/gid00064/gid00084/gid00067/gid00068/gid00001/gid00048/gid00079/gid00084/gid00082/gid00001/gid00020/gid00010 /gid00035/gid00064/gid00082/gid00072/gid00066/gid00001/gid00009/gid00040/gid00049/gid00053/gid00014/gid00021/gid00010/gid00052/gid00083/gid00064/gid00077/gid00067/gid00064/gid00075/gid00078/gid00077/gid00068/gid00001/gid00009/gid00036/gid00075/gid00064/gid00084/gid00067/gid00068/gid00001/gid00048/gid00079/gid00084/gid00082/gid00001/gid00020/gid00010 /gid00052/gid00083/gid00064/gid00077/gid00067/gid00064/gid00075/gid00078/gid00077/gid00068/gid00001/gid00009/gid00040/gid00049/gid00053/gid00014/gid00021/gid00010 /gid00034/gid00070/gid00068/gid00077/gid00083/gid00072/gid00066/gid00001/gid00009/gid00036/gid00075/gid00064/gid00084/gid00067/gid00068/gid00001/gid00048/gid00079/gid00084/gid00082/gid00001/gid00020/gid00010 /gid00034/gid00070/gid00068/gid00077/gid00083/gid00072/gid00066/gid00001/gid00009/gid00040/gid00049/gid00053/gid00014/gid00021/gid00010/gid00035/gid00064/gid00082/gid00072/gid00066/gid00001/gid00009/gid00036/gid00075/gid00064/gid00084/gid00067/gid00068/gid00001/gid00048/gid00079/gid00084/gid00082/gid00001/gid00020/gid00010 /gid00035/gid00064/gid00082/gid00072/gid00066/gid00001/gid00009/gid00040/gid00049/gid00053/gid00014/gid00021/gid00010/gid00052/gid00083/gid00064/gid00077/gid00067/gid00064/gid00075/gid00078/gid00077/gid00068/gid00001/gid00009/gid00036/gid00075/gid00064/gid00084/gid00067/gid00068/gid00001/gid00048/gid00079/gid00084/gid00082/gid00001/gid00020/gid00010 /gid00052/gid00083/gid00064/gid00077/gid00067/gid00064/gid00075/gid00078/gid00077/gid00068/gid00001/gid00009/gid00040/gid00049/gid00053/gid00014/gid00021/gid00010 /gid00034/gid00070/gid00068/gid00077/gid00083/gid00072/gid00066/gid00001/gid00009/gid00036/gid00075/gid00064/gid00084/gid00067/gid00068/gid00001/gid00048/gid00079/gid00084/gid00082/gid00001/gid00020/gid00010 /gid00034/gid00070/gid00068/gid00077/gid00083/gid00072/gid00066/gid00001/gid00009/gid00040/gid00049/gid00053/gid00014/gid00021/gid00010/gid00035/gid00064/gid00082/gid00072/gid00066/gid00001/gid00009/gid00036/gid00075/gid00064/gid00084/gid00067/gid00068/gid00001/gid00048/gid00079/gid00084/gid00082/gid00001/gid00020/gid00010 /gid00035/gid00064/gid00082/gid00072/gid00066/gid00001/gid00009/gid00040/gid00049/gid00053/gid00014/gid00021/gid00010/gid00052/gid00083/gid00064/gid00077/gid00067/gid00064/gid00075/gid00078/gid00077/gid00068/gid00001/gid00009/gid00036/gid00075/gid00064/gid00084/gid00067/gid00068/gid00001/gid00048/gid00079/gid00084/gid00082/gid00001/gid00020/gid00010 /gid00052/gid00083/gid00064/gid00077/gid00067/gid00064/gid00075/gid00078/gid00077/gid00068/gid00001/gid00009/gid00040/gid00049/gid00053/gid00014/gid00021/gid00010\n/gid00039/gid00064/gid00072/gid00083/gid00071/gid00069/gid00084/gid00075/gid00077/gid00068/gid00082/gid00082 /gid00034/gid00077/gid00082/gid00086/gid00068/gid00081/gid00001/gid00051/gid00068/gid00075/gid00068/gid00085/gid00064/gid00077/gid00066/gid00088 /gid00036/gid00078/gid00077/gid00083/gid00068/gid00087/gid00083/gid00001/gid00049/gid00081/gid00068/gid00066/gid00072/gid00082/gid00072/gid00078/gid00077 /gid00036/gid00078/gid00077/gid00083/gid00068/gid00087/gid00083/gid00001/gid00051/gid00068/gid00075/gid00068/gid00085/gid00064/gid00077/gid00066/gid00088\n/gid00052/gid00084/gid00065/gid00073/gid00068/gid00066/gid00083/gid00072/gid00085/gid00068/gid00001/gid00050/gid00034\n/gid00018/gid00015/gid00017/gid00017 /gid00017/gid00015/gid00026/gid00023 /gid00017/gid00015/gid00026/gid00026 /gid00017/gid00015/gid00026/gid00024\n/gid00017\n/gid00018\n/gid00017/gid00015/gid00022\n/gid00034/gid00070/gid00068/gid00077/gid00083/gid00072/gid00066/gid00001/gid00009/gid00036/gid00075/gid00064/gid00084/gid00067/gid00068/gid00001/gid00048/gid00079/gid00084/gid00082/gid00001/gid00020/gid00010 /gid00034/gid00070/gid00068/gid00077/gid00083/gid00072/gid00066/gid00001/gid00009/gid00040/gid00049/gid00053/gid00014/gid00021/gid00010/gid00035/gid00064/gid00082/gid00072/gid00066/gid00001/gid00009/gid00036/gid00075/gid00064/gid00084/gid00067/gid00068/gid00001/gid00048/gid00079/gid00084/gid00082/gid00001/gid00020/gid00010 /gid00035/gid00064/gid00082/gid00072/gid00066/gid00001/gid00009/gid00040/gid00049/gid00053/gid00014/gid00021/gid00010\n/gid00017/gid00015/gid00026/gid00017\n/gid00017/gid00015/gid00025/gid00023\n/gid00017/gid00015/gid00026/gid00023 /gid00017/gid00015/gid00026/gid00021\n/gid00017\n/gid00018\n/gid00017/gid00015/gid00022\n/gid00034/gid00070/gid00068/gid00077/gid00083/gid00072/gid00066/gid00001/gid00009/gid00036/gid00075/gid00064/gid00084/gid00067/gid00068/gid00001/gid00048/gid00079/gid00084/gid00082/gid00001/gid00020/gid00010 /gid00034/gid00070/gid00068/gid00077/gid00083/gid00072/gid00066/gid00001/gid00009/gid00040/gid00049/gid00053/gid00014/gid00021/gid00010/gid00035/gid00064/gid00082/gid00072/gid00066/gid00001/gid00009/gid00036/gid00075/gid00064/gid00084/gid00067/gid00068/gid00001/gid00048/gid00079/gid00084/gid00082/gid00001/gid00020/gid00010 /gid00035/gid00064/gid00082/gid00072/gid00066/gid00001/gid00009/gid00040/gid00049/gid00053/gid00014/gid00021/gid00010\n/gid00017/gid00015/gid00021/gid00021\n/gid00017/gid00015/gid00019/gid00018\n/gid00017/gid00015/gid00020/gid00019\n/gid00017/gid00015/gid00018/gid00025\n/gid00017\n/gid00017/gid00015/gid00022\n/gid00017/gid00015/gid00019/gid00022\n/gid00034/gid00070/gid00068/gid00077/gid00083/gid00072/gid00066/gid00001/gid00009/gid00036/gid00075/gid00064/gid00084/gid00067/gid00068/gid00001/gid00048/gid00079/gid00084/gid00082/gid00001/gid00020/gid00010 /gid00034/gid00070/gid00068/gid00077/gid00083/gid00072/gid00066/gid00001/gid00009/gid00040/gid00049/gid00053/gid00014/gid00021/gid00010/gid00035/gid00064/gid00082/gid00072/gid00066/gid00001/gid00009/gid00036/gid00075/gid00064/gid00084/gid00067/gid00068/gid00001/gid00048/gid00079/gid00084/gid00082/gid00001/gid00020/gid00010 /gid00035/gid00064/gid00082/gid00072/gid00066/gid00001/gid00009/gid00040/gid00049/gid00053/gid00014/gid00021/gid00010\n/gid00017/gid00015/gid00019/gid00024\n/gid00017/gid00015/gid00017/gid00018\n/gid00017/gid00015/gid00019/gid00024\n/gid00017/gid00015/gid00017/gid00018\n/gid00017\n/gid00017/gid00015/gid00020\n/gid00017/gid00015/gid00018/gid00022\n/gid00034/gid00070/gid00068/gid00077/gid00083/gid00072/gid00066/gid00001/gid00009/gid00036/gid00075/gid00064/gid00084/gid00067/gid00068/gid00001/gid00048/gid00079/gid00084/gid00082/gid00001/gid00020/gid00010 /gid00034/gid00070/gid00068/gid00077/gid00083/gid00072/gid00066/gid00001/gid00009/gid00040/gid00049/gid00053/gid00014/gid00021/gid00010/gid00035/gid00064/gid00082/gid00072/gid00066/gid00001/gid00009/gid00036/gid00075/gid00064/gid00084/gid00067/gid00068/gid00001/gid00048/gid00079/gid00084/gid00082/gid00001/gid00020/gid00010 /gid00035/gid00064/gid00082/gid00072/gid00066/gid00001/gid00009/gid00040/gid00049/gid00053/gid00014/gid00021/gid00010\n/gid00039/gid00064/gid00072/gid00083/gid00071/gid00069/gid00084/gid00075/gid00077/gid00068/gid00082/gid00082 /gid00034/gid00077/gid00082/gid00086/gid00068/gid00081/gid00001/gid00051/gid00068/gid00075/gid00068/gid00085/gid00064/gid00077/gid00066/gid00088 /gid00036/gid00078/gid00077/gid00083/gid00068/gid00087/gid00083/gid00001/gid00049/gid00081/gid00068/gid00066/gid00072/gid00082/gid00072/gid00078/gid00077 /gid00036/gid00078/gid00077/gid00083/gid00068/gid00087/gid00083/gid00001/gid00051/gid00068/gid00075/gid00068/gid00085/gid00064/gid00077/gid00066/gid00088\n/gid00049/gid00081/gid00068/gid00066/gid00072/gid00082/gid00072/gid00078/gid00077/gid00001/gid00048/gid00077/gid00066/gid00078/gid00075/gid00078/gid00070/gid00088/gid00001/gid00050/gid00034\n/gid00017/gid00015/gid00026/gid00025\n/gid00025/gid00015/gid00022/gid00068/gid00014/gid00020\n/gid00017/gid00015/gid00025/gid00022\n/gid00017/gid00015/gid00026/gid00022 /gid00017/gid00015/gid00026/gid00018\n/gid00017\n/gid00018\n/gid00017/gid00015/gid00022\n/gid00034/gid00070/gid00068/gid00077/gid00083/gid00072/gid00066/gid00001/gid00009/gid00036/gid00075/gid00064/gid00084/gid00067/gid00068/gid00001/gid00048/gid00079/gid00084/gid00082/gid00001/gid00020/gid00010 /gid00034/gid00070/gid00068/gid00077/gid00083/gid00072/gid00066/gid00001/gid00009/gid00040/gid00049/gid00053/gid00014/gid00021/gid00010/gid00035/gid00064/gid00082/gid00072/gid00066/gid00001/gid00009/gid00036/gid00075/gid00064/gid00084/gid00067/gid00068/gid00001/gid00048/gid00079/gid00084/gid00082/gid00001/gid00020/gid00010 /gid00035/gid00064/gid00082/gid00072/gid00066/gid00001/gid00009/gid00040/gid00049/gid00053/gid00014/gid00021/gid00010\n/gid00017/gid00015/gid00026/gid00024\n/gid00017/gid00015/gid00026/gid00021 /gid00017/gid00015/gid00026/gid00024 /gid00017/gid00015/gid00026/gid00023\n/gid00017\n/gid00018\n/gid00017/gid00015/gid00022\n/gid00034/gid00070/gid00068/gid00077/gid00083/gid00072/gid00066/gid00001/gid00009/gid00036/gid00075/gid00064/gid00084/gid00067/gid00068/gid00001/gid00048/gid00079/gid00084/gid00082/gid00001/gid00020/gid00010 /gid00034/gid00070/gid00068/gid00077/gid00083/gid00072/gid00066/gid00001/gid00009/gid00040/gid00049/gid00053/gid00014/gid00021/gid00010/gid00035/gid00064/gid00082/gid00072/gid00066/gid00001/gid00009/gid00036/gid00075/gid00064/gid00084/gid00067/gid00068/gid00001/gid00048/gid00079/gid00084/gid00082/gid00001/gid00020/gid00010 /gid00035/gid00064/gid00082/gid00072/gid00066/gid00001/gid00009/gid00040/gid00049/gid00053/gid00014/gid00021/gid00010\n/gid00017/gid00015/gid00022/gid00018\n/gid00017/gid00015/gid00019/gid00017\n/gid00017/gid00015/gid00022/gid00019\n/gid00017/gid00015/gid00018/gid00022\n/gid00017\n/gid00017/gid00015/gid00023\n/gid00017/gid00015/gid00020\n/gid00034/gid00070/gid00068/gid00077/gid00083/gid00072/gid00066/gid00001/gid00009/gid00036/gid00075/gid00064/gid00084/gid00067/gid00068/gid00001/gid00048/gid00079/gid00084/gid00082/gid00001/gid00020/gid00010 /gid00034/gid00070/gid00068/gid00077/gid00083/gid00072/gid00066/gid00001/gid00009/gid00040/gid00049/gid00053/gid00014/gid00021/gid00010/gid00035/gid00064/gid00082/gid00072/gid00066/gid00001/gid00009/gid00036/gid00075/gid00064/gid00084/gid00067/gid00068/gid00001/gid00048/gid00079/gid00084/gid00082/gid00001/gid00020/gid00010 /gid00035/gid00064/gid00082/gid00072/gid00066/gid00001/gid00009/gid00040/gid00049/gid00053/gid00014/gid00021/gid00010\n/gid00017/gid00015/gid00025/gid00019\n/gid00017/gid00015/gid00022/gid00022\n/gid00017/gid00015/gid00025/gid00017\n/gid00017/gid00015/gid00022/gid00022\n/gid00017\n/gid00017/gid00015/gid00026\n/gid00017/gid00015/gid00021/gid00022\n/gid00019/gid00015/gid00021/gid00068/gid00014/gid00021 /gid00019/gid00015/gid00022/gid00068/gid00014/gid00022 /gid00019/gid00015/gid00022/gid00068/gid00014/gid00019 /gid00021/gid00015/gid00018/gid00068/gid00014/gid00019\n/gid00021/gid00015/gid00024/gid00068/gid00014/gid00020 /gid00025/gid00015/gid00026/gid00068/gid00014/gid00022 /gid00018/gid00068/gid00014/gid00019 /gid00024/gid00015/gid00019/gid00068/gid00014/gid00020/gid00017 /gid00019/gid00015/gid00018/gid00068/gid00014/gid00020/gid00024/gid00019/gid00015/gid00019/gid00068/gid00014/gid00020\nFigure 6\nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint \n\n/gid00052/gid00083/gid00068/gid00079/gid00001/gid00018\n/gid00052/gid00083/gid00068/gid00079/gid00001/gid00018 /gid00052/gid00083/gid00068/gid00079/gid00001/gid00019 /gid00052/gid00083/gid00068/gid00079/gid00001/gid00020 /gid00052/gid00083/gid00068/gid00079/gid00001/gid00021\n/gid00052/gid00083/gid00068/gid00079/gid00001/gid00019 /gid00052/gid00083/gid00068/gid00079/gid00001/gid00020 /gid00052/gid00083/gid00068/gid00079/gid00001/gid00021 /gid00052/gid00083/gid00068/gid00079/gid00001/gid00022\n/gid00048/gid00065/gid00073/gid00068/gid00066/gid00083/gid00072/gid00085/gid00068/gid00001/gid00050/gid00034\n/gid00052/gid00084/gid00065/gid00073/gid00068/gid00066/gid00083/gid00072/gid00085/gid00068/gid00001/gid00050/gid00034\n/gid00052/gid00083/gid00068/gid00079/gid00001/gid00018 /gid00052/gid00083/gid00068/gid00079/gid00001/gid00019 /gid00052/gid00083/gid00068/gid00079/gid00001/gid00020 /gid00052/gid00083/gid00068/gid00079/gid00001/gid00021 /gid00052/gid00083/gid00068/gid00079/gid00001/gid00022\n/gid00049/gid00081/gid00068/gid00066/gid00072/gid00082/gid00072/gid00078/gid00077/gid00001/gid00048/gid00077/gid00066/gid00078/gid00075/gid00078/gid00070/gid00088/gid00001/gid00050/gid00034\n/gid00034/gid00052/gid00036/gid00048/gid00001/gid00040/gid00084/gid00072/gid00067/gid00068/gid00075/gid00072/gid00077/gid00068/gid00082 /gid00036/gid00075/gid00072/gid00077/gid00072/gid00066/gid00064/gid00075/gid00001/gid00053/gid00081/gid00072/gid00064/gid00075/gid00082 /gid00036/gid00075/gid00072/gid00077/gid00072/gid00066/gid00064/gid00075/gid00001/gid00053/gid00081/gid00072/gid00064/gid00075/gid00082/gid00001/gid00051/gid00068/gid00069/gid00068/gid00081/gid00068/gid00077/gid00066/gid00068/gid00082 /gid00037/gid00081/gid00084/gid00070/gid00082/gid00001/gid00042/gid00077/gid00069/gid00078/gid00081/gid00076/gid00064/gid00083/gid00072/gid00078/gid00077 /gid00038/gid00052/gid00046/gid00048/gid00001/gid00040/gid00084/gid00072/gid00067/gid00068/gid00075/gid00072/gid00077/gid00068/gid00082 /gid00043/gid00047/gid00036/gid00036/gid00047\n/gid00045/gid00064/gid00077/gid00067/gid00076/gid00064/gid00081/gid00074/gid00001/gid00036/gid00075/gid00072/gid00077/gid00072/gid00066/gid00064/gid00075/gid00001/gid00053/gid00081/gid00072/gid00064/gid00075/gid00082 /gid00047/gid00036/gid00036/gid00047/gid00001/gid00040/gid00084/gid00072/gid00067/gid00068/gid00075/gid00072/gid00077/gid00068/gid00082 /gid00047/gid00036/gid00036/gid00047/gid00001/gid00051/gid00068/gid00069/gid00068/gid00081/gid00068/gid00077/gid00066/gid00068/gid00082 /gid00049/gid00071/gid00064/gid00081/gid00076/gid00040/gid00044/gid00035 /gid00049/gid00084/gid00065/gid00046/gid00068/gid00067 /gid00056/gid00072/gid00074/gid00072/gid00079/gid00068/gid00067/gid00072/gid00064\nFigure 7\nAll rights reserved. No reuse allowed without permission. \nperpetuity. \npreprint (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in \nThe copyright holder for thisthis version posted October 5, 2024. ; https://doi.org/10.1101/2024.09.20.24314076doi: medRxiv preprint","source_license":"Public-Domain","license_restricted":false}