Leveraging Dynamic Prompting for Outcome Prediction of Cancer Patients Using Large Language Models and Electronic Health Record Notes

Leveraging Dynamic Prompting for Outcome Prediction of Cancer Patients Using Large Language Models and Electronic Health Record Notes | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Leveraging Dynamic Prompting for Outcome Prediction of Cancer Patients Using Large Language Models and Electronic Health Record Notes Shreyas Anil, Bhumika Srinivas, Bo Liu, Anyi Li, Yannet Interian, and 6 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7329357/v1 This work is licensed under a CC BY 4.0 License Status: Under Revision Version 1 posted 11 You are reading this latest preprint version Abstract Outcome prediction from unstructured EHR notes remains challenging, especially for rare cancers with limited pre-training data. We present a dynamic prompting framework that retrieves semantically similar patient examples, constructs tailored few-shot prompts, and integrates note summaries to enhance large language model (LLM)-based outcome predictions. We evaluated this approach on a single-institution cohort of 503 breast and 475 glioma patients (EHR notes within 180 days post-diagnosis), with overall survival dichotomized at five years (breast cancer) and fourteen months (glioma). Using Llama-3 models (8B/70B), we compared zero-shot, dynamic prompting, summarization-only, and combined workflows. Dynamic prompting substantially improved glioma prediction performance, boosting accuracy by 12% and F1 by 11%, whereas gains for breast cancer were modest (<3%). The combined summarization-plus-prompting approach achieved the highest performance while maintaining prediction stability compared to GPT-4, addressing a critical deployment barrier. T-SNE visualizations confirmed that embeddings captured established prognostic markers. Dynamic prompting delivers maximal benefit when pre‑training exposure is low but can be safely applied across all tumor types without degrading performance on common cancers. This selective yet universal enhancement establishes dynamic prompting as a practical, scalable solution for deploying LLMs in clinical oncology, particularly for rare cancers where accurate outcome prediction can meaningfully inform treatment planning. Biological sciences/Cancer Biological sciences/Computational biology and bioinformatics Health sciences/Oncology Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 Introduction Recent advances in natural language processing (NLP) offer a promising avenue to leverage unstructured clinical narratives at scale. Large language models (LLMs), exemplified by ChatGPT 1 and the Llama 2 , 3 family, have shown an unprecedented ability to interpret complex free text, which could transform oncology analytics. Oncology care is especially rich in textual data, such as detailed tumor board notes, radiology impressions, clinic visit summaries, and LLMs, which are emerging as powerful tools to extract prognostic insights from these narratives. Several recent oncology-focused studies have successfully used LLMs to automatically distill key clinical variables (e.g., diagnoses, staging, histopathological details) from unstructured EHR notes with high accuracy 4 – 8 . For example, a recent work by Zhu et al. 9 showed that LLMs can accurately extract cancer progression information from unstructured electronic health record (EHR) notes with expert-level performance. In another recent study, Park et al. 10 applied a general LLM to structure the free-text EHR notes of radiation therapy patients, aiming to improve post-treatment survival predictions, and found that incorporating these LLM-derived features yielded a substantial jump in predictive accuracy of patient mortality. The LLM-driven approach improved risk stratification and model interpretability since the extracted text features aligned with known prognostic factors. These examples demonstrate that LLMs can unlock valuable signals from raw clinical text, overcoming the limitations of structured data-driven models. Leveraging LLMs on unstructured oncology notes has proven effective to offer avenues for better understanding of cancer trajectories, enhancing predictive modeling for outcomes, and facilitating personalized treatment pathways 11 – 13 . Two general strategies have emerged for adapting LLMs to the clinical oncology domain: fine-tuning and in-context learning 14 . Fine-tuning involves retraining the model on domain-specific data to endow it with specialized knowledge, while in-context learning uses prompting techniques to guide its behavior without changing its internal parameters. Traditional fine-tuning can achieve strong results but requires substantial curated data and computational resources, as seen with early biomedical models like Gatortron 8 , PubMedBERT 15 and BioGPT 16 . In contrast, prompt-based adaptation offers flexibility and efficiency, allowing rapid specialization of a general LLM to new tasks with minimal overhead. Recent studies suggest powerful generalist models, when steered with effective prompting, can rival or surpass specialist models on many clinical tasks 17 – 19 . For instance, the MedPrompt 20 approach by Nori et al. demonstrated how strategic prompt design could unlock GPT-4’s latent medical knowledge to achieve state-of-the-art accuracy on the USMLE medical question-answering benchmark without additional model fine-tuning. In oncology, Park et al. achieved their results by prompting a general-domain LLM to structure EHR narratives without further training, noting that even without medical fine-tuning, the model performed on par with or better than domain-specific versions 10 . This indicates that general LLMs can be steered to comprehend clinical text if given the proper guidance, avoiding extensive supervised data collection. In-context learning is also easier to scale across institutions and outcomes: unlike fine-tuning, which requires substantial computational resources and curated datasets, in-context learning provides flexibility and efficiency, allowing rapid adaptation to specific downstream clinical tasks without refitting the model's parameters each time. Another promising strategy is to use LLMs for summarization of clinical notes as an intermediate step: by condensing a patient’s unstructured EHR records into a concise summary or a list of key clinical points, the model can effectively surface the most pertinent information for downstream tasks such as the outcome prediction 21 – 23 . Recent work by Choudhuri et al. 24 exemplifies this approach, where they prompted GPT to read ICU progress notes and generate summaries highlighting potential complications and risk factors for each patient. These LLM-generated summaries were then fed into a predictive model for ICU readmission and length-of-stay prediction, boosting both predictive tasks with significant improvements compared to models that used the raw notes and structured data alone. These gains showcase how LLM-driven summarization can distill the signal from lengthy clinical narratives into a form that traditional models can more readily learn from. In this scenario, LLMs act as a feature extractor, condensing a complex medical history into a few critical sentences or coded facts. This approach is far more scalable than manual abstraction of notes, and it sidesteps the need to train a bespoke natural language processing model for each new outcome. Overall, the convergence of the richness of unstructured oncology notes, the capabilities of modern LLMs, and the efficiency of prompt-based utilization points toward a new generation of EHR-based predictive models. In the framework presented here, we steer LLMs to interpret and summarize patient narratives dynamically, integrating their content into overall survival predictions for diverse cancer types. Dynamic prompting combined with LLM summarization offers a flexible framework to do this at scale: prompts can be tailored to each task or patient context, and summaries provide a compact representation of patient state. Building on this foundation, the present work proposes a scalable approach to cancer outcome prediction that leverages dynamic LLM prompting and summarization of real-world EHR notes. By harnessing an LLM’s understanding of free-text clinical narratives without requiring extensive model re-training, we aim to improve predictive accuracy and generalizability in oncology outcome modeling, while addressing the limitations of prior structured-data-centric and zero-shot learning of LLM methods, paving the way for more intelligent and comprehensive use of EHR notes in cancer prognostic applications. Results Performance of Dynamic Prompting Comparative analysis of model performance across breast cancer and glioma cohorts revealed markedly different baseline capabilities and responses to dynamic prompting interventions (Tables 1 and 2 ). Table 1 Breast – Comparative Performance Analysis between Llama-3.0 8B and Llama-3.0 70B. Model Accuracy Precision Recall F1 AUROC Zero Shot Prompting Zero Shot Full Notes Llama 3.0 8B 0.8267 ± 0.01 0.8416 ± 0.01 0.9652 ± 0.01 0.8955 ± 0.01 0.7431 ± 0.04 Llama 3.0 70B 0.7774 ± 0.01 0.7765 ± 0.01 1.0000 ± 0.00 0.9000 ± 0.00 0.7650 ± 0.05 Zero Shot Summary Notes Llama 3.0 8B 0.8283 ± 0.02 0.8267 ± 0.02 0.9845 ± 0.02 0.8980 ± 0.01 0.7455 ± 0.03 Llama 3.0 70B 0.8204 ± 0.04 0.8661 ± 0.03 0.9095 ± 0.05 0.8880 ± 0.03 0.7532 ± 0.07 Dynamic Prompting Dynamic 2-Shot Full Notes Llama 3.0 8B 0.8250 ± 0.02 0.8338 ± 0.01 0.9666 ± 0.01 0.8948 ± 0.01 0.7405 ± 0.03 Llama 3.0 70B 0.8309 ± 0.02 0.8156 ± 0.02 0.9872 ± 0.01 0.9000 ± 0.00 0.7736 ± 0.04 Dynamic 5-Shot Summary Notes Llama 3.0 8B 0.8320 ± 0.02 0.8417 ± 0.03 0.9769 ± 0.01 0.9000 ± 0.00 0.7910 ± 0.05 Llama 3.0 70B 0.7985 ± 0.05 0.8657 ± 0.03 0.8759 ± 0.06 0.8700 ± 0.03 0.7414 ± 0.06 Table 2 Glioma – Comparative Performance Analysis between Llama-3.0 8B and Llama-3.0 70B. Model Accuracy Precision Recall F1 AUROC Zero Shot Prompting Zero Shot Full Notes Llama 3.0 8B 0.6569 ± 0.04 0.6431 ± 0.02 0.9966 ± 0.01 0.8000 ± 0.00 0.7479 ± 0.09 Llama 3.0 70B 0.6168 ± 0.01 0.6159 ± 0.01 1.0000 ± 0.00 0.7608 ± 0.01 0.7422 ± 0.04 Zero Shot Summary Notes Llama 3.0 8B 0.6548 ± 0.02 0.8427 ± 0.04 0.5409 ± 0.04 0.6580 ± 0.02 0.7534 ± 0.02 Llama 3.0 70B 0.6779 ± 0.04 0.7028 ± 0.03 0.8967 ± 0.03 0.7876 ± 0.02 0.5705 ± 0.01 Dynamic Prompting Dynamic 2-Shot Full Notes Llama 3.0 8B 0.6779 ± 0.07 0.7667 ± 0.07 0.6852 ± 0.08 0.7000 ± 0.07 0.7525 ± 0.07 Llama 3.0 70B 0.7071 ± 0.03 0.7369 ± 0.03 0.8219 ± 0.05 0.7763 ± 0.03 0.7649 ± 0.05 Dynamic 5-Shot Summary Notes Llama 3.0 8B 0.7537 ± 0.02 0.8218 ± 0.02 0.7669 ± 0.06 0.7920 ± 0.03 0.7751 ± 0.02 Llama 3.0 70B 0.7537 ± 0.03 0.8311 ± 0.05 0.7568 ± 0.05 0.7906 ± 0.03 0.8132 ± 0.04 For the breast cancer cohort (n = 503), zero-shot inference achieved robust baseline performance across both model architectures. Llama-3.0 8B attained 83% accuracy with an F1 score of 0.90, while the 70B variant showed comparable metrics. Implementing dynamic prompting with 2-shot full notes yielded minimal performance gains (< 1% accuracy improvement for both models). GPT-4-generated summarization alone produced marginal effects, with accuracy remaining within 1% of baseline. The combined summarization-dynamic prompting approach achieved the highest accuracy for the 8B model (83.2 ± 2%), representing a marginal improvement over zero-shot performance. Notably, recall remained consistently high across all configurations (> 0.9 for most experiments), while precision showed greater variability. In contrast, the glioma dataset (n = 475) exhibited markedly different performance characteristics in the zero-shot configuration: the baseline metrics for both Llama-3.0 8B and 70B models were substantially lower than those observed for breast cancer, with accuracy decreasing by 21% and 16% respectively. This performance differential was accompanied by a pronounced prediction bias, with both models demonstrating near-universal positive class prediction despite the glioma dataset containing only a slight positive skew. The high variance in glioma zero-shot performance (± 4% for 8B) compared to breast cancer (± 1%) further indicated model uncertainty when processing these glioma EHR notes. These quantitative differences suggest fundamental disparities in how the models process breast cancer versus glioma documentation, with glioma notes presenting distinct challenges that zero-shot inference fails to address adequately. The integration of summarization with 5-shot dynamic prompting yielded the most substantial improvements for glioma prediction: both Llama models achieved 75.37% accuracy, representing an absolute improvement of 9.68% (8B) and 13.69% (70B) over zero-shot baselines. The F1 scores also reached 0.79 for both 8B and 70B models, while AUROC improved to 0.77 and 0.81, respectively. Ablative Studies: Comparative Performance of Llama-3.0 8B and Llama-3.0 70B We conducted ablative studies using varying model sizes and architectures to explore model robustness, consistency, and interpretability. This systematic evaluation of multiple models, including Llama-3.0 8B, Llama-3.0 70B, and GPT-4, aimed to assess how different configurations influence performance on the dynamic prompting task. These studies allowed us to determine whether conclusions from the foundational Llama-3.0 8B model generalize across other architectures, providing insights into their scalability and adaptability for clinical applications. Figures 1 and 2 illustrate that the analysis revealed significant trends when comparing the Llama-3.0 8B and 70B models across the breast cancer and glioma cohorts. The performance metrics for the breast cancer dataset were broadly comparable between the two models, with the 8B model slightly outperforming the 70B model in terms of accuracy and AUROC. This finding aligns with classical statistical theories, suggesting that larger models may be prone to overfitting due to their increased parameterization 25 – 27 . This overfitting tendency, coupled with the simplicity of the breast cancer dataset relative to glioma, may explain why the smaller 8B model displayed a marginal advantage in accuracy. The glioma cohort presented a more complex scenario, with dynamic prompting applied to full notes and summaries yielding substantial improvements in accuracy and AUROC for both the 8B and 70B models. These results highlight the generalizability of the dynamic prompting methodology across model sizes. Yet, the interplay between precision and recall revealed notable disparities. For the glioma dataset, precision was on par or higher for the 8B model, while recall was on par or higher for the 70B model. These differences stem from the inherent trade-offs between smaller and larger models 28 : the 8B model, with fewer parameters, tends to be more conservative in its predictions, favoring precision over recall. Conversely, the 70B model, with its broader capacity for parameter representation, leans toward inclusivity, prioritizing recall at the expense of precision. Dynamic prompting mitigated these disparities by providing the contextual scaffolding to balance the models’ predictions. The contextual cues inherent in dynamic prompts allowed both the 8B and 70B models to reduce biases associated with their architectural limitations, leading to more equitable performance. This is evident in the narrowing precision-recall gap, which suggests that dynamic prompting helps align the predictions of these models, making them more robust and reliable across complex datasets like glioma. While the glioma dataset's fluctuations in precision and recall indicate challenges associated with class imbalance, the overall gains in accuracy and AUROC across both model sizes reaffirm the efficacy of dynamic prompting. These results demonstrate that dynamic prompting enhances contextual understanding, balances prediction biases, and facilitates more reliable clinical outcome predictions regardless of the model size. This ability to generalize dynamic prompting across architectures validates its scalability and demonstrates its potential as a versatile tool in deploying large language models for nuanced clinical tasks. Evaluation of Majority Voting and Overrides We evaluate both versions of the Llama 3.0 models on glioma and breast dynamic summary 5-shot experiments, focusing on their ability to deviate from majority voting and correctly override flawed majority decisions. The results reveal clear behavioral tendencies across the two model configurations (8B and 70B) and prompt enforcement levels for the summary dynamic prompting with 5-shot. For glioma, the Llama 3.0 8B model broke the majority vote more frequently, averaging 33 ± 4 times per fold compared to 24 ± 4 times for the 70B model. Despite this higher deviation rate, the 8B model achieved more correct overrides against incorrect majority decisions, with 22 ± 2 correct overrides versus 16 ± 3 correct overrides for the 70B model. These results suggest that the 8B model exhibits greater flexibility and independence in decision-making, whereas the 70B model leads to more consistent but less independent behavior. In the breast classification tasks, the models displayed slightly different dynamics. The 8B and 70B models broke the majority vote at comparable rates, with 15 ± 4 and 22 ± 8 instances, respectively. However, the 70B model demonstrated a higher number of correct overrides against the majority, achieving 15 ± 4 correct overrides compared to 11 ± 4 overrides for the 8B model. This pattern diverges from the glioma results, where the 8B model outperformed in identifying errors in the majority consensus. The analysis suggests that the 70B model has a more substantial capacity for acting independently and identifying errors in breast classification tasks. In contrast, the 8B model exhibits a more balanced performance across different datasets, with some advantages in glioma tasks. This nuanced behavior highlights dataset-specific strengths for each model. Across both glioma and breast tasks, the results reveal a consistent trade-off between model size and prompt enforcement. The Llama 3.0 8B model tends to deviate more frequently from the majority consensus and demonstrates greater success in overriding incorrect majority decisions. This suggests that the smaller model relies more on intrinsic reasoning, which can be advantageous in tasks requiring independent judgments. In contrast, the 70B model aligns more consistently with the majority but exhibits limited flexibility in overriding errors. These results in Table 3 highlight that the performance of Llama 3.0 models, particularly in breaking majority voting and correctly overriding inaccurate majority decisions, is not solely reliant on consensus. Instead, the outcomes reflect a significant degree of independent decision-making, facilitated by dynamic prompting strategies. This autonomy underscores the models’ ability to evaluate and interpret inputs beyond majority-driven biases. Table 3 GPT-4 variability: standard deviations of 3 consecutive runs. Disease Method Accuracy F1 Breast Full notes 0.0057 0.0047 2-shot learning with full notes 0.0361 0.0270 Glioma Full notes 0.0182 0.0382 2-shot learning with full notes 0.0161 0.0133 Embedding Visualization with Prognostic Features To evaluate the ability of LLMs to capture prognostic trends from unstructured EHR data, we visualized patient-specific embeddings generated by our pipeline using t‐distributed stochastic neighbor embedding (t‐SNE) for both breast cancer and glioma cohorts (Figs. 3 and 4). These embeddings were colored by overall survival and overlaid with key clinical annotations to reveal the relationship between latent text representations and patient outcomes. In both cohorts, regions of extended survival (indicated by cooler color gradients) are clearly separable from those associated with poorer outcomes. The breast cancer embeddings exhibit tighter, more defined clusters compared with glioma, reflecting the larger sample size and more structured documentation in the breast cohort, aligning with our prior quantitative analyses showing higher predictive accuracy and F1 scores for breast cancer than for glioma. Figure 3 presents the breast cancer t‐SNE: hormone receptor-positive (HR+) cases concentrate within the high‐survival tail of the embedding space ( \(\:p<7.7\times\:{10}^{-8}\) ), and patients treated with lumpectomy similarly map to this favorable region ( \(\:p<9.5\times\:{10}^{-7}\) ). By contrast, embeddings labeled as metastatic disease localize to clusters of shorter survival ( \(\:p<2.4\times\:{10}^{-4}\) ). As for HER2‐positive status, nearly all HER2+ points fall into the compact bottom cluster associated with the longest median survival ( \(\:p<2.7\times\:{10}^{-2}\) ), demonstrating the transformative impact of modern HER2‐targeted therapy in reshaping prognosis for the breast cancer 29 – 31 . Figure 4 illustrates the glioma t‐SNE, where embeddings marked by 1p-19q co-deletion, a canonical marker of oligodendroglioma, aggregate in regions of prolonged survival ( \(\:p<1.3\times\:{10}^{-6}\) ), whereas MGMT-unmethylated cases map predominantly to the lower-survival clusters ( \(\:p<4.8\times\:{10}^{-7}\) ), consistent with their well-known resistance to temozolomide and adverse prognosis 32 , 33 . Multifocal gliomas, a known harbinger of aggressive behavior, are enriched in the lower‐survival clusters, albeit with more modest significance. A direct comparison of embedding patterns between the breast cancer and glioma cohorts reveals marked differences in clustering quality and survival alignment. In the breast cancer cohort, embeddings exhibit well-defined separations, with distinct sub-clusters corresponding to favorable and unfavorable outcomes. This clarity reflects the model’s ability to leverage larger datasets and more consistent EHR documentation, enabling robust capture of survival trends. In contrast, the glioma embeddings are more diffuse, with overlapping clusters and less distinct separation between prognostic groups. This aligns with the challenges inherent in glioma data, including smaller cohort sizes, more heterogeneous documentation, and the dynamic nature of glioma progression. Discussion Varying Performance in Breast and Glioma Notes In our experiments, the model’s zero-shot performance on glioma clinical notes was substantially lower than on breast cancer notes, for both the Llama-3.0 8B and 70B versions. In other words, without any task-specific fine-tuning or prompting, these models struggled to interpret glioma-related clinical notes. In contrast, their baseline results on breast cancer notes were much stronger. This indicates a limited inherent understanding of glioma-specific nuances in the pre-trained LLMs. This observation aligns with the intuition that glioma content was underrepresented in the model’s pre-training data, while breast cancer is a far more common topic in general discourse. One likely explanation is the difference in domain exposure during pre-training. LLMs are trained on vast internet corpora, but these corpora contain much more information on common diseases (like breast cancer) than on rarer, highly specialized conditions (like gliomas). Breast cancer, being one of the most prevalent cancers worldwide, with over 2.3 million new cases in 2020 34 , is also among the most widely discussed cancers on the internet 35 . As a result, generic LLMs have likely encountered and learned patterns about breast cancer in news, social media, and general health discussions. In contrast, gliomas, though the most common primary brain tumors, are comparatively rare, with roughly 6 cases per 100,000 people per year in the U.S. 36 and they receive far less attention in mainstream media or social platforms. Consequently, glioma-related text is sparse in the common pre-training corpus, leaving the model with a knowledge gap. Indeed, prior studies have noted that LLMs tend to perform worse on under-represented medical topics or rare diseases than on common ones, precisely because rare conditions are underrepresented in internet-scale text data 37 – 39 . Our findings reflect this phenomenon: both Llama 8B and 70B’s near-out-of-the-box ceiling performance on breast cancer notes versus their struggle on glioma notes demonstrates how pre-training data biases can impact zero-shot clinical NLP tasks. Another contributing factor is the complexity and specificity of glioma pathology, which demands deeper contextual understanding. Gliomas are biologically complex and highly heterogeneous tumors 40 , with subtypes and molecular markers (IDH mutation status, 1p/19q codeletion, MGMT promoter methylation, etc.) that carry nuanced implications. Interpreting glioma clinical notes often requires recognizing specialized terminology and subtle clinical context, which a general-purpose model may not readily grasp without exposure to domain-specific knowledge. Breast cancer notes, while not trivial, may include more widely-known clinical terms, such as “lymph node involvement,” “HER2 status,” or “metastases”, which generic LLMs may understand from their broad training. The dynamic and complex nature of glioma cases involving interdisciplinary knowledge of neuro-oncology, neuropathology, and evolving classification criteria based on various molecular biomarkers 41 , 42 means that a model must be aligned with specialized corpora or expert knowledge to perform well. Without that alignment, the zero-shot LLM often fails to capture critical details in glioma notes, resulting in the poor baseline performance we observed. To operationalize rare cancers in this context, we adopt the NCI definition of incidence of fewer than 15 out of 100,000 people each year 43 . This cutoff aligns with limited representation in the corpora used to pre-train generic LLMs. Gliomas meet this criterion and exhibit large performance gains from dynamic prompting, whereas breast cancer (about 130.8 per 100,000 women per year 44 ) did not. Notably, glioma patients often generate numerous clinical notes due to the disease’s rapid progression and aggressive treatment protocols, yet the pre‑training corpus remains sparse, explaining the model’s suboptimal zero‑shot results. Dynamic prompting thus offers the most benefit when pre‑training coverage is low but can be safely applied across all tumor types without performance degradation : In domains where the pre-trained LLM lacks sufficient knowledge, as in the case of glioma, our framework can inject relevant domain-specific context with minimal computational and data costs, dramatically improving performance. By prompting the model with glioma-specific corpora, we enable it to overcome the knowledge deficit from pre-training. In comparison, the breast cancer outcome prediction showed relatively minor gains from domain-specific alignment. This disparity underlines a key point: for highly specialized medical topics, LLM pre-training alone is often insufficient, and tailored approaches, like the dynamic prompting framework in this study, are essential to reach high accuracy and nuanced comprehension. Our framework’s impact is most pronounced in these challenging settings, validating its relevance for oncology NLP in specialized disease areas where general LLMs would otherwise cap out at suboptimal performance due to a lack of pre-training data. Instability of GPT-4 and its clinical implications One of the most critical limitations of GPT-4, as evidenced by our experiments in Table 3 , is its inherent instability and nondeterministic behavior, which poses significant challenges for its deployment in clinical applications. This instability was evaluated through consecutive runs using GPT-4 as the predictive model for breast cancer and glioma classification tasks. Despite controlling for all hyperparameters, including setting the temperature to zero and disabling sampling, we observed notable deviations in prediction consistency across multiple runs. The results highlight the variability inherent in GPT-4's outputs. For breast cancer, dynamic 2-shot inference exhibited a 3.6% deviation in accuracy across three consecutive runs. Similarly, for the glioma dataset, the F1 score deviated by 3.8%, underscoring the unpredictability of GPT-4 in a healthcare context. These deviations are particularly concerning, given the binary nature of the classification task, where the query responses are restricted to positive or negative options. When the log-likelihoods for these two tokens are closely matched, the model's nondeterministic decoding process can cause a positive response to flip to a negative one or vice versa, introducing variability that undermines its reliability. Interestingly, the instability was more pronounced in the dynamic prompting workflow, suggesting that the additional context and reasoning processes introduced by dynamic prompts may amplify the model's inherent uncertainty. This aligns with findings from our ablative study, where model size and architecture influenced prediction consistency. However, while dynamic prompting can mitigate some biases and improve performance in specific scenarios, it also appears to interact with GPT-4’s nondeterminism in unpredictable ways, further compounding the reliability issue. The implications of this variability are profound, particularly in the context of highly sensitive clinical data. Even a slight deviation in model output can significantly impact healthcare applications, leading to misclassification of patient conditions and potentially impacting clinical decision-making. For example, a mislabel in a glioma classification task could result in inappropriate treatment recommendations, compounding the already high stakes of managing complex diseases like glioma. Despite extensive parameter control, the inability to guarantee consistent outputs raises serious concerns about the suitability of GPT-4 for tasks where accuracy and reliability are paramount. Our findings suggest that GPT-4, in its current form, is not a viable standalone solution for predictive modeling in healthcare. While the model excels in natural language understanding and general-purpose tasks, its instability limits its application in domains where consistency and precision are non-negotiable. Addressing this issue would require further refinement of the model’s decoding mechanisms or the development of additional post-processing layers to filter out inconsistencies. Until such solutions are realized, models like Llama-3, which exhibit greater stability and reliability, present a more promising pathway for integrating LLMs into clinical workflows. These results emphasize the need for careful evaluation and adaptation of LLMs to ensure their safe and effective use in sensitive, high-stakes environments. Impact of Note Summarization on Outcome Prediction The concise and focused nature of summary notes generated by GPT-4 demonstrates a significant advantage in outcome prediction tasks, effectively distilling key diagnostic information while filtering out extraneous details in full clinical notes. Full clinical notes, while comprehensive, often contain substantial noise, such as repetitive templates, which can dilute the critical signals necessary for accurate survival prediction. This issue is particularly pronounced in small datasets with lengthy notes, as is the case for both the breast cancer and glioma cohorts. In such scenarios, including irrelevant details can increase the likelihood of model overfitting, whereas summary notes enhance generalizability and mitigate this risk by focusing solely on the essential clinical information. Our results in Tables 1 and 2 clearly illustrate this effect. When evaluated with the baseline Llama-3.0 8B model, summary notes consistently outperformed full notes across both breast and glioma cohorts when dynamic prompting was employed. The improvement was especially pronounced in the glioma cohort, where the challenges of smaller datasets and more complex clinical terminology amplify the benefits of streamlined input. These findings highlight the value of note summarization as a preprocessing step, boosting performance metrics like accuracy and F1 score and reducing the cognitive load on the model by removing unnecessary noise. However, implementing this workflow comes with added computational requirements and costs. The summarization process necessitates an extra preprocessing step, where the clinical data is sent to a secure, encrypted version of GPT-4 45 . While this ensures data privacy and compliance with regulatory standards, it incurs additional costs and demands more computational resources during preprocessing. Despite this, the downstream benefits of using summary notes offset these initial overheads. Notably, less computational power is required for inference when using summarized data, as opposed to full notes, making the overall workflow more efficient during the prediction phase. The efficiency gains are particularly relevant in clinical contexts where real-time or near-real-time predictions are necessary. By reducing the computational burden during inference, the summarization workflow ensures that Llama-based models can be effectively deployed even in resource-constrained environments, such as community hospitals or smaller oncology practices. Additionally, the streamlined input allows the model to focus on the most pertinent features, ensuring more robust and interpretable outputs. Overall, summary notes provide a more effective and computationally efficient approach to outcome prediction. While the workflow requires an additional preprocessing step and associated costs, the improved model performance and reduced inference demands illustrate its practicality and potential for integration into clinical workflows. Methods Patient selection criteria and assembly of the dataset This study focused on oncology patients with confirmed diagnoses of glioma or breast cancer, established through histopathological or radiological evidence. The cohorts were derived from a single institution and spanned diagnoses between 2010 and 2019. Unstructured EHR notes from radiology, pathology, medical oncology, and radiation oncology were included. To ensure adequate data for analysis, only patients with at least three clinical notes recorded within the first 180 days post-diagnosis were considered eligible for inclusion. Patients were categorized based on survival status relative to a predefined survival threshold (14 months for glioma and 5 years for breast cancer). Patients alive during data collection but had not yet reached these thresholds were excluded. Demographic details for the included cohorts are summarized in Table 4 . Stratified random sampling was applied to ensure a balanced representation of cancer stage, grade, and survival outcomes. This approach divided the dataset into 80% training and 20% testing subsets, maintaining a proportional representation of survival status and clinical attributes. Importantly, these distributions remained consistent throughout the study, with no addition or removal of patients during subsequent analyses. For robustness, the patient data was split into training and testing sets five times, mitigating any biases associated with a single data split. Patient-specific documents were dynamically assembled at each iteration by concatenating all EHR notes generated during the first 180 days post-diagnosis. Each patient document was labeled with their survival status based on the relevant threshold (e.g., 14 months for glioma or 5 years for breast cancer). The dataset's dynamic assembly method allowed for including all relevant notes created within the specified timeframe while ensuring the survival label corresponded to the patient's status at the predefined threshold. For patients who passed away within the observation window, their notes remained static beyond their date of death, preserving the integrity of the dataset and the associated survival labels. This study was approved by the Institutional Review Board (UCSF: IRB# 20–32527). Table 4 Patient features for breast cancer and glioma patient cohorts. Feature Category Breast Glioma Patients 503 475 General (median, range) in years Age 57, [26–95] 50, [1-103] Follow-up 5.7, [0.1–8.4] 1.4, [0.04–10.2] Time to death 2.4 [0.1–7.7] 1.1, [0.04–6.6] Sex Male - 291 Female 503 184 Race White 395 (78.5%) 421 (88.6%) Asian 71 (14.1%) 29 (6.1%) Black 25 (5.0%) 5 (1.1%) Other 12 (2.4%) 20 (4.2%) cTNM Stage/Grade 0 55 (10.9%) - 1 158 (31.4%) 3 (0.6%) 2 159 (31.6%) 10 (2.1%) 3 53 (10.5%) 91 (19.2%) 4 48 (9.5%) 371 (78.1%) Unknown 30 (6.0%) - Treatment Surgery 454 (90.3%) 404 (85.1%) Radiotherapy 258 (51.3%) 425 (89.5%) Chemotherapy 238 (47.3%) 402 (84.6%) Endocrine therapy 335 (66.6%) 1 (0.2%) Immunotherapy 29 (5.8%) 49 (10.3%) Alcohol use No or low 339 (67.4%) 315 (66.3%) Yes 150 (29.8%) 156 (32.8%) Unknown 14 (2.8%) 4 (0.8%) Smoking use Never smoker 321 (63.8%) 278 (58.5%) Current or former smoker 182 (36.2%) 197 (41.5%) Marital Status Married 300 (59.6%) 323 (68.0%) Single 103 (20.5%) 108 (22.7%) Divorced or separated 96 (19.1%) 43 (9.1%) Unknown 4 (0.8%) 1 (0.2%) Framework for Dynamic Prompting The dynamic prompting framework is designed to optimize the use of EHR notes for predictive modeling by leveraging both complete and summarized data paths, as indicated by Fig. 5 . The preprocessing phase in the full-note workflow begins with splitting the dataset into training and testing sets. The training samples are then passed through the embedding model to generate patient-specific embeddings. Due to the model's context window limitations, each sample is chunked, and chunk-level embeddings are averaged to create a final, patient-level embedding. This approach preserves the semantic integrity of lengthy notes while ensuring compatibility with the model's computational constraints. The generated embeddings are stored and clustered using a nearest-neighbors algorithm. These clusters, visualized through t-SNE (as shown in Figs. 3 and 4), provide insights into the semantic similarities within the dataset. During inference, embeddings are generated for testing samples using the same embedding model. A k-nearest neighbors algorithm identifies the two most semantically similar training samples, which are then appended to the testing sample and its ground truth labels. This composite dataset and a carefully crafted query are passed to the predictive model. Model temperature is set to zero to enforce greedy decoding, while the maximum output length is restricted to two tokens, ensuring interpretable and extractable predictions. The details of the embedding and predictive backbone models are outlined in Table 5 . Table 5 Models Used in the Dynamic Prompt Study. Model Name Task #. of Parameters Context Window jina-embeddings-v2-base Embedding Generation 137 million 8,192 tokens gpt-4-turbo Summary Generation Unreported 128,000 tokens Llama-3-8B-Instruct-262k Predictive Model 8 billion 262,144 tokens Llama-3-70B-Instruct-Gradient-262k Predictive Model (Ablation) 70 billion 262,144 tokens The summarized notes workflow enhances the efficiency and relevance of the input data. After the dataset is split, the training samples are summarized using a dedicated summarization model guided by a structured outline. This step is performed with deterministic settings - temperature set to zero and sampling disabled - to ensure repeatable outputs. The generated summaries are validated through keyword searches or human review to confirm consistency and accuracy. The summarized data is then processed using the same steps as the full notes workflow, with a few key differences. During inference, the testing samples are summarized using the same structured outline. The shorter sequence lengths resulting from summarization allow for the inclusion of up to five semantically similar training samples, increasing the contextual richness of the input. The streamlined nature of summarized inputs reduces computational demands while preserving critical clinical information, making this workflow particularly advantageous for tasks with limited context windows. All inference tasks were executed on an 8-GPU NVIDIA A6000 cluster, utilizing the vLLM framework to implement tensor parallelism, ensuring efficient and scalable model deployment 46 . Prompting Methods for Note Summary Generation Structured templates were meticulously designed for breast cancer and glioma patient notes to ensure comprehensive inclusion of key clinical elements. These templates covered crucial categories such as diagnosis, treatment history, symptoms, molecular biomarkers, and follow-up recommendations, creating a standardized yet adaptable framework for both cohorts. By emphasizing uniformity, the templates facilitated accurate classification while accommodating the specific characteristics of each cancer type. In the breast cancer cohort, the GPT-4-turbo-128k model showcased exceptional performance in filling out the templates with minimal intervention. The templates were completed efficiently and comprehensively, requiring no re-prompting or iterative refinement. This approach ensured that all relevant information was accurately captured by streamlining the process, enabling a reliable foundation for downstream tasks. Conversely, the glioma cohort presented unique challenges, requiring a more refined and iterative prompting approach, as shown in the right panel of Fig. 6 . Glioma patient notes, characterized by their specialized terminology and intricate clinical details, necessitated iterative prompting to clarify instructions and enhance specificity. For instance, rephrasing general directives such as “Use these notes to fill out the template” to detailed, step-by-step instructions significantly improved the model’s understanding. Contextual prompts, where the model was explicitly guided as an AI assistant tasked with completing the template, further enhanced output precision. Negative prompting 47 , another key strategy, was employed to guide the model in avoiding errors or omissions. Instructions like “Ensure no clinical or molecular data is overlooked” directed the model toward critical glioma-specific elements such as 1p/19q co-deletion, IDH mutation status, Karnofsky Performance Status, and radiation therapy prescription dosage. These strategies compensated for the specialized nature of glioma notes, enabling the model to retrieve the intricate details required for accurate template completion. While the glioma cohort demanded multiple iterations, the combined use of iterative and negative prompting ultimately captured all essential clinical and molecular details. The complexity of the notes, coupled with the rare and nuanced nature of glioma cases, highlighted the importance of these advanced prompting techniques in achieving accurate information retrieval. Figure 7 illustrates the improvement in the accuracy and relevance of the progressive response through systematic prompt refinements. By iteratively specifying key clinical terms and optimizing the structure of the prompts, we demonstrate that fine-tuning prompts significantly enhances the fidelity of GPT-generated responses. The final optimized prompt included all relevant molecular biomarkers, accurately recorded radiation therapy dosages, and addressed missing data by marking fields as “Not Specified” where applicable. This refined prompting framework improved precision in identifying tumor location and treatment response and emphasized the potential of dynamic prompting in overcoming the limitations of unstructured EHR notes. Evaluation Metrics The outcome prediction task for both breast and glioma cohorts represents a binary classification challenge, necessitating comprehensive evaluation metrics such as accuracy, precision, recall, F1 score, and AUROC (Area Under the Receiver Operating Curve). These metrics collectively provide a nuanced understanding of the model’s classification capabilities, particularly in scenarios characterized by imbalanced datasets, as seen in the breast cancer cohort. They also address contexts where different errors, such as false positives and false negatives, hold distinct clinical implications and weights. While most metrics were straightforward to compute from the outputs, AUROC required a customized computation due to the absence of directly extractable probabilities from the Llama model’s output structure. To address this, we leveraged the predicted logits, raw scores reflecting the model’s likelihood estimates for each class (POS and NEG), as a foundation. These logits were transformed into probabilities through a softmax function, which normalized the scores to represent the likelihood of each class. The AUROC was then calculated using these probabilities alongside the ground truth labels (true_labels and predicted_probs), capturing the model’s ability to differentiate between positive and negative instances across varying decision thresholds. This approach ensured a robust evaluation of the model’s discriminatory power, particularly critical in clinical prediction tasks where confidence levels can directly influence decision-making. The remaining metrics offered complementary insights into the model’s performance. Accuracy measured the overall proportion of correct predictions, while precision and recall quantified the model's ability to correctly identify positive and negative instances, respectively. The F1 score, balancing precision and recall, provided a holistic view of performance, particularly valuable in imbalanced datasets. These metrics allowed for a detailed assessment of the trade-offs between precision and recall, clarifying whether the model favored conservatism (higher precision) or inclusivity (higher recall) in its predictions. Declarations Author’s Contributions S.A. and B.S. made equal contributions to this work, playing key roles in formulating the ideas, collecting and analyzing the data and developing the pipeline. W.C.C., N.D.P. and S.E.B. conducted clinical evaluations of the results. B.L., N.D.P., Y.I., A.L., J.C.H., and O.M. engaged in discussions on the problem and participated in the result analysis. H.L. and O.M. led the original data collection and processing. H.L. supervised and provided support throughout the research process. All authors contributed to manuscript editing and review. Competing Interests All authors declare no financial or non-financial competing interests. Acknowledgments H.L. and B.L. received funding from the Department of Defense Impact Award (W81XWH-22-1-0695), a seed grant from Uncle Kory Foundation 2023-2024, a UCSF HDFCCC Computational seed grant 2024-2025, and a UCSF SPORE brain tumor career enhancement program 2024-2025. B.S. received fund from UCSF HDFCCC Computational seed grant 2024-2025. The funders played no role in study design, data collection, analysis and interpretation of data, or the writing of this manuscript. Data Availability The datasets facilitating the findings of this study are not openly accessible due to legitimate privacy and security concerns. The original EHR note data cannot be redistributed to researchers outside of those involved in the study, approved by the IRB at the principal institution. Access to anonymized data can be facilitated through a material transfer agreement (MTA) managed by the principal institution. Owing to privacy concerns, datasets generated and/or analyzed during this study are not publicly available but can be provided by the corresponding author upon a reasonable request. Code Availability Dynamic prompting, summarization, and outcome prediction codes used in this study are available at: https://github.com/OncoNLP/DynamicPrompting-ICL under Apache 2.0 license. References OpenAI. Introducing ChatGPT. https://openai.com/blog/chatgpt (2022). Touvron, H., Lavril, T. & Izacard, G. Llama: Open and efficient foundation language models. arXiv preprint arXiv … (2023). Grattafiori, A., Dubey, A., Jauhri, A. & Pandey, A. The llama 3 herd of models. arXiv preprint arXiv … (2024). Azar, W. S. et al. LLM-Mediated Data Extraction from Patient Records after Radical Prostatectomy. NEJM AI 2 , (2025). Huang, J. et al. A critical assessment of using ChatGPT for extracting structured data from clinical notes. npj Digital Med. 7 , 106 (2024). Li, D., Kadav, A., Gao, A. & Li, R. Automated Clinical Data Extraction with Knowledge Conditioned LLMs. Proceedings of the 31st … (2025). Choi, H. S., Song, J. Y., Shin, K. H., Chang, J. H. & Jang, B.-S. Developing prompts from large language model for extracting clinical information from pathology and ultrasound reports in breast cancer. Radiat. Oncol. J. 41 , 209–216 (2023). Yang, X. et al. A large language model for electronic health records. npj Digital Med. 5 , 194 (2022). Zhu, M. et al. Large language model trained on clinical oncology data predicts cancer progression. npj Digital Med. 8 , 397 (2025). Park, S., Wee, C. W., Choi, S. H., Kim, K. H. & Chang, J. S. Improving mortality prediction after radiotherapy with large language model structuring of large-scale unstructured electronic health records. Radiotherapy and … (2025). Yuan, J., Tang, R., Jiang, X. & Hu, X. Llm for patient-trial matching: Privacy-aware data augmentation towards better performance and generalizability. American Medical Informatics … (2023). Lin, H., Ni, L., Phuong, C. & Hong, J. C. Natural language processing for radiation oncology: personalizing treatment pathways. Pharmgenomics Pers Med 17 , 65–76 (2024). Guevara, M. et al. Large language models to identify social determinants of health in electronic health records. npj Digital Med. 7 , 6 (2024). Mosbach, M., Pimentel, T., Ravfogel, S., Klakow, D. & Elazar, Y. Few-shot Fine-tuning vs. In-context Learning: A Fair Comparison and Evaluation. in Findings of the Association for Computational Linguistics: ACL 2023 12284–12314 (Association for Computational Linguistics, 2023). doi:10.18653/v1/2023.findings-acl.779. Gu, Y. et al. Domain-specific language model pretraining for biomedical natural language processing. ACM Trans. Comput. Healthcare 3 , 1–23 (2022). Luo, R. et al. BioGPT: generative pre-trained transformer for biomedical text generation and mining. Brief. Bioinformatics 23 , (2022). Ferber, D. et al. In-context learning enables multimodal large language models to classify cancer pathology images. Nat. Commun. 15 , 10104 (2024). Nguyen, D., Swanson, D., Newbury, A. & Kim, Y. H. Evaluation of chatgpt and google bard using prompt engineering in cancer screening algorithms. Acad. Radiol. 31 , 1799–1804 (2024). Iivanainen, S., Lagus, J., Viertolahti, H., Sippola, L. & Koivunen, J. Investigating large language model (LLM) performance using in-context learning (ICL) for interpretation of ESMO and NCCN guidelines for lung cancer. JCO 42 , e13637–e13637 (2024). Nori, H., Lee, Y. T., Zhang, S., Carignan, D. & Edgar, R. Can generalist foundation models outcompete special-purpose tuning? case study in medicine. arXiv preprint arXiv … (2023). Zhang, H., Yu, P. S. & Zhang, J. A systematic survey of text summarization: from statistical methods to large language models. ACM Comput. Surv. (2025) doi:10.1145/3731445. Chuang, Y.-N., Tang, R., Jiang, X. & Hu, X. SPeC: A Soft Prompt-Based Calibration on Performance Variability of Large Language Model in Clinical Notes Summarization. J. Biomed. Inform. 151 , 104606 (2024). Van Veen, D. et al. Clinical text summarization: adapting large language models can outperform human experts. Res. Sq. (2023) doi:10.21203/rs.3.rs-3483777/v1. Choudhuri, A., Polgreen, P., Segre, A. & Adhikari, B. Summarizing Clinical Notes using LLMs for ICU Bounceback and Length-of-Stay Prediction. in 2024 IEEE International Conference on Data Mining Workshops (ICDMW) 859–866 (IEEE, 2024). doi:10.1109/ICDMW65004.2024.00118. Hastie, T., Tibshirani, R., Friedman, J. H. & Friedman, J. H. The elements of statistical learning: data mining, inference, and prediction . (Springer, 2009). Belkin, M., Hsu, D., Ma, S. & Mandal, S. Reconciling modern machine-learning practice and the classical bias-variance trade-off. Proc Natl Acad Sci USA 116 , 15849–15854 (2019). Zhang, C., Bengio, S., Hardt, M., Recht, B. & Vinyals, O. Understanding deep learning (still) requires rethinking generalization. Commun. ACM 64 , 107–115 (2021). Juba, B. & Le, H. S. Precision-Recall versus Accuracy and the Role of Large Data Sets. AAAI 33 , 4039–4048 (2019). Maximiano, S., Magalhães, P., Guerreiro, M. P. & Morgado, M. Trastuzumab in the treatment of breast cancer. BioDrugs 30 , 75–86 (2016). O’Sullivan, C. C. et al. Efficacy of Adjuvant Trastuzumab for Patients With Human Epidermal Growth Factor Receptor 2-Positive Early Breast Cancer and Tumors ≤ 2 cm: A Meta-Analysis of the Randomized Trastuzumab Trials. J. Clin. Oncol. 33 , 2600–2608 (2015). Early Breast Cancer Trialists’ Collaborative group (EBCTCG). Trastuzumab for early-stage, HER2-positive breast cancer: a meta-analysis of 13 864 women in seven randomised trials. Lancet Oncol. 22 , 1139–1150 (2021). Hegi, M. E. et al. MGMT gene silencing and benefit from temozolomide in glioblastoma. N. Engl. J. Med. 352 , 997–1003 (2005). Yi, G.-Z. et al. Acquired temozolomide resistance in MGMT-deficient glioblastoma cells is associated with regulation of DNA repair by DHC2. Brain 142 , 2352–2366 (2019). Arnold, M. et al. Current and future burden of breast cancer: Global statistics for 2020 and 2040. Breast 66 , 15–23 (2022). Quinn, E. M. et al. Who’s talking about breast cancer? Analysis of daily breast cancer posts on the internet. Breast 22 , 24–27 (2013). Mesfin, F. B., Karsonovich, T. & Al-Dhahir, M. A. Gliomas. in StatPearls (StatPearls Publishing, 2025). Abdullahi, T., Singh, R. & Eickhoff, C. Learning to make rare and complex diagnoses with generative AI assistance: Qualitative study of popular large language models. JMIR Med. Educ. 10 , e51391 (2024). Chen, X. et al. Rarebench: can llms serve as rare diseases specialists? in Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining 4850–4861 (ACM, 2024). doi:10.1145/3637528.3671576. Yang, J., Shu, L., Duan, H. & Li, H. Rdguru: A conversational intelligent agent for rare diseases. IEEE J. Biomed. Health Inform. PP , (2024). Berens, M. E. et al. Multiscale, multimodal analysis of tumor heterogeneity in IDH1 mutant vs wild-type diffuse gliomas. PLoS ONE 14 , e0219724 (2019). Wesseling, P. & Capper, D. WHO 2016 Classification of gliomas. Neuropathol. Appl. Neurobiol. 44 , 139–150 (2018). Louis, D. N. et al. The 2021 WHO Classification of Tumors of the Central Nervous System: a summary. Neuro Oncol. 23 , 1231–1251 (2021). NCI rare cancer definition. https://www.cancer.gov/publications/dictionaries/cancer-terms/def/rare-cancer. Cancer Stat Facts: Female Breast Cancer. https://seer.cancer.gov/statfacts/html/breast.html. UCSF. Versa API. https://ai.ucsf.edu/versa. Kwon, W. et al. Efficient Memory Management for Large Language Model Serving with PagedAttention. in Proceedings of the 29th Symposium on Operating Systems Principles 611–626 (ACM, 2023). doi:10.1145/3600006.3613165. Ban, Y. et al. Understanding the Impact of Negative Prompts: When and How Do They Take Effect? in Computer vision – ECCV 2024: 18th european conference, milan, italy, september 29–october 4, 2024, proceedings, part LXXXIX (eds. Leonardis, A. et al.) vol. 15147 190–206 (Springer Nature Switzerland, 2025). Additional Declarations No competing interests reported. Cite Share Download PDF Status: Under Revision Version 1 posted Editorial decision: Revision requested 28 Sep, 2025 Reviews received at journal 24 Sep, 2025 Reviews received at journal 14 Sep, 2025 Reviews received at journal 12 Sep, 2025 Reviewers agreed at journal 29 Aug, 2025 Reviewers agreed at journal 22 Aug, 2025 Reviewers agreed at journal 20 Aug, 2025 Reviewers invited by journal 20 Aug, 2025 Editor assigned by journal 12 Aug, 2025 Submission checks completed at journal 12 Aug, 2025 First submitted to journal 08 Aug, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7329357","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":506574697,"identity":"d2e9c46e-f583-4d84-b39c-15d4430fd685","order_by":0,"name":"Shreyas Anil","email":"","orcid":"","institution":"University of California, San Francisco","correspondingAuthor":false,"prefix":"","firstName":"Shreyas","middleName":"","lastName":"Anil","suffix":""},{"id":506574698,"identity":"f9913ddd-9316-49c9-990f-b58d6d7f47b4","order_by":1,"name":"Bhumika Srinivas","email":"","orcid":"","institution":"University of California, San Francisco","correspondingAuthor":false,"prefix":"","firstName":"Bhumika","middleName":"","lastName":"Srinivas","suffix":""},{"id":506574699,"identity":"4c9a2544-22f7-4681-ba1d-b7d25f0462c7","order_by":2,"name":"Bo Liu","email":"","orcid":"","institution":"UCSF-UC Berkeley","correspondingAuthor":false,"prefix":"","firstName":"Bo","middleName":"","lastName":"Liu","suffix":""},{"id":506574700,"identity":"9a4007fd-54eb-4ce7-8837-a32661ae2472","order_by":3,"name":"Anyi Li","email":"","orcid":"","institution":"Memorial Sloan Kettering Cancer Center","correspondingAuthor":false,"prefix":"","firstName":"Anyi","middleName":"","lastName":"Li","suffix":""},{"id":506574701,"identity":"614aa1fc-3289-412b-9182-24d4c8266c0d","order_by":4,"name":"Yannet Interian","email":"","orcid":"","institution":"University of San Francisco","correspondingAuthor":false,"prefix":"","firstName":"Yannet","middleName":"","lastName":"Interian","suffix":""},{"id":506574702,"identity":"d373f816-1fbb-41ca-99ed-bdc6a48cbb86","order_by":5,"name":"William C. Chen","email":"","orcid":"","institution":"University of California, San Francisco","correspondingAuthor":false,"prefix":"","firstName":"William","middleName":"C.","lastName":"Chen","suffix":""},{"id":506574703,"identity":"15157617-49f9-4de8-90f8-d1e10664b176","order_by":6,"name":"Nicolas D. Prionas","email":"","orcid":"","institution":"University of California, San Francisco","correspondingAuthor":false,"prefix":"","firstName":"Nicolas","middleName":"D.","lastName":"Prionas","suffix":""},{"id":506574704,"identity":"541ad2d4-beca-439e-9849-aeeecda2f10f","order_by":7,"name":"Julian C. Hong","email":"","orcid":"","institution":"University of California, San Francisco","correspondingAuthor":false,"prefix":"","firstName":"Julian","middleName":"C.","lastName":"Hong","suffix":""},{"id":506574705,"identity":"83d54772-a016-4c02-ac12-673dd7ac55f5","order_by":8,"name":"Steve E. Braunstein","email":"","orcid":"","institution":"University of California, San Francisco","correspondingAuthor":false,"prefix":"","firstName":"Steve","middleName":"E.","lastName":"Braunstein","suffix":""},{"id":506574706,"identity":"f7b0d676-a63d-4f30-9e95-4b9db1788a7b","order_by":9,"name":"Olivier Morin","email":"","orcid":"","institution":"University of California, San Francisco","correspondingAuthor":false,"prefix":"","firstName":"Olivier","middleName":"","lastName":"Morin","suffix":""},{"id":506574707,"identity":"329eaa4a-16bd-40a4-8ac7-8df9d8390501","order_by":10,"name":"Hui Lin","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAtklEQVRIiWNgGAWjYBACAwbGBoYHBgxyIM4HBgZmIrUkGDAYA9mMM4jUAgQJDAyJDURrMZdIBtpScCd9w/HjDxsYKqxBevEDyxmJIIc9y91wJsewgeFMOmEtBjfAWg7nbrjBw/6Ase0w8VrSDW6wP2xg/EeClgSDGwyGDYwNxGg587DhAFCL4UyQXxKOpRsT1nI8/eGDD38Oy/OBQuxDjbUsQS0gcADOSiBG+SgYBaNgFIwCwgAAyPZF2lL55VcAAAAASUVORK5CYII=","orcid":"","institution":"University of California, San Francisco","correspondingAuthor":true,"prefix":"","firstName":"Hui","middleName":"","lastName":"Lin","suffix":""}],"badges":[],"createdAt":"2025-08-08 17:23:15","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7329357/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7329357/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":90308457,"identity":"549b072a-49ce-4def-97a2-46ce5c84d3f6","added_by":"auto","created_at":"2025-09-01 09:37:26","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":120598,"visible":true,"origin":"","legend":"\u003cp\u003eBreast - Difference in Evaluation Metrics between Llama-3.0 8B and Llama-3.0 70B.\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-7329357/v1/d5e07e84b1066fd0c15d2b8f.png"},{"id":90308458,"identity":"0c4440ee-15c9-40b9-b8d5-a10f07c4360e","added_by":"auto","created_at":"2025-09-01 09:37:26","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":100219,"visible":true,"origin":"","legend":"\u003cp\u003eGlioma - Difference in Evaluation Metrics between Llama-3.0 8B and Llama-3.0 70B.\u003c/p\u003e","description":"","filename":"2.png","url":"https://assets-eu.researchsquare.com/files/rs-7329357/v1/4da965ea767a090f4e0bfefa.png"},{"id":90311766,"identity":"343cd8ec-cc4b-4b08-85ab-3d3d79d609b9","added_by":"auto","created_at":"2025-09-01 09:53:26","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":153139,"visible":true,"origin":"","legend":"\u003cp\u003eT-SNE embeddings for breast cancer cohort demonstrate clear clustering by overall survival and alignment with prognostic features.\u003c/p\u003e","description":"","filename":"3.png","url":"https://assets-eu.researchsquare.com/files/rs-7329357/v1/016668102232272d9709c361.png"},{"id":90308461,"identity":"47243449-dc02-43f1-baba-124508472916","added_by":"auto","created_at":"2025-09-01 09:37:26","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":178594,"visible":true,"origin":"","legend":"\u003cp\u003eT-SNE embeddings for glioma cohort reveal diffuse clustering by overall survival with subplots highlighting alignment with key prognostic features.\u003c/p\u003e","description":"","filename":"4.png","url":"https://assets-eu.researchsquare.com/files/rs-7329357/v1/ab309cc6812cafed6101a783.png"},{"id":90308466,"identity":"d9e88f94-ebfa-4809-b4a7-a700369bcb7f","added_by":"auto","created_at":"2025-09-01 09:37:26","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":281757,"visible":true,"origin":"","legend":"\u003cp\u003eWorkflow showcasing the adaptability of the pipeline across zero-shot, few-shot, with dynamic prompting, and summarization-enhanced setups for survival prediction using EHR notes.\u003c/p\u003e","description":"","filename":"5.png","url":"https://assets-eu.researchsquare.com/files/rs-7329357/v1/36d12c7daff8aec733726135.png"},{"id":90311764,"identity":"8040b4e7-2856-4be7-b9ae-d2f3b3a46efb","added_by":"auto","created_at":"2025-09-01 09:53:26","extension":"png","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":395147,"visible":true,"origin":"","legend":"\u003cp\u003eOverview of prompting methods for breast cancer and glioma cohorts, illustrating summary templates (in red) employed during the initial trials for patient notes. Tailored prompts (in blue) were iteratively refined during testing to optimize template completeness and relevance. Outputs generated by \u003cem\u003eGPT-4-turbo-128k\u003c/em\u003ewith a temperature setting of 0 are highlighted in yellow, accompanied by annotations evaluating their clinical relevance and quality.\u003c/p\u003e","description":"","filename":"6.png","url":"https://assets-eu.researchsquare.com/files/rs-7329357/v1/fc794c7e0d5840ea0a1d8be1.png"},{"id":90311767,"identity":"2b3b8f5a-feba-415c-a61a-7f2557faa6a6","added_by":"auto","created_at":"2025-09-01 09:53:26","extension":"png","order_by":7,"title":"Figure 7","display":"","copyAsset":false,"role":"figure","size":415643,"visible":true,"origin":"","legend":"\u003cp\u003eApplication of iterative and negative prompting strategies for glioma note completion, and glioma summary samples showing improvements of GPT responses through prompt refinement.\u003c/p\u003e","description":"","filename":"7.png","url":"https://assets-eu.researchsquare.com/files/rs-7329357/v1/36bd90a7e66fddd31cb368ba.png"},{"id":90314619,"identity":"6460c129-4d76-4bdc-b27e-e48791ac79f6","added_by":"auto","created_at":"2025-09-01 10:09:27","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":2409990,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7329357/v1/6d1510ba-9445-41a4-ac81-4e30970911cb.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Leveraging Dynamic Prompting for Outcome Prediction of Cancer Patients Using Large Language Models and Electronic Health Record Notes","fulltext":[{"header":"Introduction","content":"\u003cp\u003eRecent advances in natural language processing (NLP) offer a promising avenue to leverage unstructured clinical narratives at scale. Large language models (LLMs), exemplified by ChatGPT\u003csup\u003e\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e\u003c/sup\u003e and the Llama\u003csup\u003e\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e,\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e\u003c/sup\u003e family, have shown an unprecedented ability to interpret complex free text, which could transform oncology analytics. Oncology care is especially rich in textual data, such as detailed tumor board notes, radiology impressions, clinic visit summaries, and LLMs, which are emerging as powerful tools to extract prognostic insights from these narratives. Several recent oncology-focused studies have successfully used LLMs to automatically distill key clinical variables (e.g., diagnoses, staging, histopathological details) from unstructured EHR notes with high accuracy\u003csup\u003e\u003cspan additionalcitationids=\"CR5 CR6 CR7\" citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e\u003c/sup\u003e. For example, a recent work by Zhu \u003cem\u003eet al.\u003c/em\u003e\u003csup\u003e\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e\u003c/sup\u003e showed that LLMs can accurately extract cancer progression information from unstructured electronic health record (EHR) notes with expert-level performance. In another recent study, Park \u003cem\u003eet al.\u003c/em\u003e\u003csup\u003e\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e\u003c/sup\u003e applied a general LLM to structure the free-text EHR notes of radiation therapy patients, aiming to improve post-treatment survival predictions, and found that incorporating these LLM-derived features yielded a substantial jump in predictive accuracy of patient mortality. The LLM-driven approach improved risk stratification and model interpretability since the extracted text features aligned with known prognostic factors. These examples demonstrate that LLMs can unlock valuable signals from raw clinical text, overcoming the limitations of structured data-driven models. Leveraging LLMs on unstructured oncology notes has proven effective to offer avenues for better understanding of cancer trajectories, enhancing predictive modeling for outcomes, and facilitating personalized treatment pathways\u003csup\u003e\u003cspan additionalcitationids=\"CR12\" citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e\u003c/sup\u003e.\u003c/p\u003e\u003cp\u003eTwo general strategies have emerged for adapting LLMs to the clinical oncology domain: fine-tuning and in-context learning\u003csup\u003e\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e\u003c/sup\u003e. Fine-tuning involves retraining the model on domain-specific data to endow it with specialized knowledge, while in-context learning uses prompting techniques to guide its behavior without changing its internal parameters. Traditional fine-tuning can achieve strong results but requires substantial curated data and computational resources, as seen with early biomedical models like Gatortron\u003csup\u003e\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e\u003c/sup\u003e, PubMedBERT\u003csup\u003e\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e\u003c/sup\u003e and BioGPT\u003csup\u003e\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e\u003c/sup\u003e. In contrast, prompt-based adaptation offers flexibility and efficiency, allowing rapid specialization of a general LLM to new tasks with minimal overhead. Recent studies suggest powerful generalist models, when steered with effective prompting, can rival or surpass specialist models on many clinical tasks\u003csup\u003e\u003cspan additionalcitationids=\"CR18\" citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e\u003c/sup\u003e. For instance, the MedPrompt\u003csup\u003e\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e\u003c/sup\u003e approach by Nori \u003cem\u003eet al.\u003c/em\u003e demonstrated how strategic prompt design could unlock GPT-4\u0026rsquo;s latent medical knowledge to achieve state-of-the-art accuracy on the USMLE medical question-answering benchmark without additional model fine-tuning. In oncology, Park \u003cem\u003eet al.\u003c/em\u003e achieved their results by prompting a general-domain LLM to structure EHR narratives without further training, noting that even without medical fine-tuning, the model performed on par with or better than domain-specific versions\u003csup\u003e\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e\u003c/sup\u003e. This indicates that general LLMs can be steered to comprehend clinical text if given the proper guidance, avoiding extensive supervised data collection. In-context learning is also easier to scale across institutions and outcomes: unlike fine-tuning, which requires substantial computational resources and curated datasets, in-context learning provides flexibility and efficiency, allowing rapid adaptation to specific downstream clinical tasks without refitting the model's parameters each time.\u003c/p\u003e\u003cp\u003eAnother promising strategy is to use LLMs for summarization of clinical notes as an intermediate step: by condensing a patient\u0026rsquo;s unstructured EHR records into a concise summary or a list of key clinical points, the model can effectively surface the most pertinent information for downstream tasks such as the outcome prediction\u003csup\u003e\u003cspan additionalcitationids=\"CR22\" citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e\u003c/sup\u003e. Recent work by Choudhuri \u003cem\u003eet al.\u003c/em\u003e\u003csup\u003e\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e\u003c/sup\u003e exemplifies this approach, where they prompted GPT to read ICU progress notes and generate summaries highlighting potential complications and risk factors for each patient. These LLM-generated summaries were then fed into a predictive model for ICU readmission and length-of-stay prediction, boosting both predictive tasks with significant improvements compared to models that used the raw notes and structured data alone. These gains showcase how LLM-driven summarization can distill the signal from lengthy clinical narratives into a form that traditional models can more readily learn from. In this scenario, LLMs act as a feature extractor, condensing a complex medical history into a few critical sentences or coded facts. This approach is far more scalable than manual abstraction of notes, and it sidesteps the need to train a bespoke natural language processing model for each new outcome.\u003c/p\u003e\u003cp\u003eOverall, the convergence of the richness of unstructured oncology notes, the capabilities of modern LLMs, and the efficiency of prompt-based utilization points toward a new generation of EHR-based predictive models. In the framework presented here, we steer LLMs to interpret and summarize patient narratives dynamically, integrating their content into overall survival predictions for diverse cancer types. Dynamic prompting combined with LLM summarization offers a flexible framework to do this at scale: prompts can be tailored to each task or patient context, and summaries provide a compact representation of patient state. Building on this foundation, the present work proposes a scalable approach to cancer outcome prediction that leverages dynamic LLM prompting and summarization of real-world EHR notes. By harnessing an LLM\u0026rsquo;s understanding of free-text clinical narratives without requiring extensive model re-training, we aim to improve predictive accuracy and generalizability in oncology outcome modeling, while addressing the limitations of prior structured-data-centric and zero-shot learning of LLM methods, paving the way for more intelligent and comprehensive use of EHR notes in cancer prognostic applications.\u003c/p\u003e"},{"header":"Results","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e\n \u003ch2\u003ePerformance of Dynamic Prompting\u003c/h2\u003e\n \u003cp\u003eComparative analysis of model performance across breast cancer and glioma cohorts revealed markedly different baseline capabilities and responses to dynamic prompting interventions (Tables \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e and\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e).\u003c/p\u003e\n \u003cdiv class=\"gridtable\"\u003e\n \u003ctable id=\"Tab1\" border=\"1\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eBreast \u0026ndash; Comparative Performance Analysis between Llama-3.0 8B and Llama-3.0 70B.\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eModel\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eAccuracy\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003ePrecision\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eRecall\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eF1\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eAUROC\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003cth align=\"left\" colspan=\"6\"\u003e\n \u003cp\u003eZero Shot Prompting\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colspan=\"6\"\u003e\n \u003cp\u003e\u003cem\u003eZero Shot Full Notes\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eLlama 3.0 8B\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.8267\u0026thinsp;\u0026plusmn;\u0026thinsp;0.01\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.8416\u0026thinsp;\u0026plusmn;\u0026thinsp;0.01\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.9652\u0026thinsp;\u0026plusmn;\u0026thinsp;0.01\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.8955\u0026thinsp;\u0026plusmn;\u0026thinsp;0.01\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.7431\u0026thinsp;\u0026plusmn;\u0026thinsp;0.04\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eLlama 3.0 70B\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.7774\u0026thinsp;\u0026plusmn;\u0026thinsp;0.01\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.7765\u0026thinsp;\u0026plusmn;\u0026thinsp;0.01\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003e1.0000\u0026thinsp;\u0026plusmn;\u0026thinsp;0.00\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.9000\u0026thinsp;\u0026plusmn;\u0026thinsp;0.00\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.7650\u0026thinsp;\u0026plusmn;\u0026thinsp;0.05\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colspan=\"6\"\u003e\n \u003cp\u003e\u003cem\u003eZero Shot Summary Notes\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eLlama 3.0 8B\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.8283\u0026thinsp;\u0026plusmn;\u0026thinsp;0.02\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.8267\u0026thinsp;\u0026plusmn;\u0026thinsp;0.02\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.9845\u0026thinsp;\u0026plusmn;\u0026thinsp;0.02\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.8980\u0026thinsp;\u0026plusmn;\u0026thinsp;0.01\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.7455\u0026thinsp;\u0026plusmn;\u0026thinsp;0.03\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eLlama 3.0 70B\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.8204\u0026thinsp;\u0026plusmn;\u0026thinsp;0.04\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.8661\u0026thinsp;\u0026plusmn;\u0026thinsp;0.03\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.9095\u0026thinsp;\u0026plusmn;\u0026thinsp;0.05\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.8880\u0026thinsp;\u0026plusmn;\u0026thinsp;0.03\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.7532\u0026thinsp;\u0026plusmn;\u0026thinsp;0.07\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colspan=\"6\"\u003e\n \u003cp\u003e\u003cstrong\u003eDynamic Prompting\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colspan=\"6\"\u003e\n \u003cp\u003e\u003cem\u003eDynamic 2-Shot Full Notes\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eLlama 3.0 8B\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.8250\u0026thinsp;\u0026plusmn;\u0026thinsp;0.02\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.8338\u0026thinsp;\u0026plusmn;\u0026thinsp;0.01\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.9666\u0026thinsp;\u0026plusmn;\u0026thinsp;0.01\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.8948\u0026thinsp;\u0026plusmn;\u0026thinsp;0.01\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.7405\u0026thinsp;\u0026plusmn;\u0026thinsp;0.03\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eLlama 3.0 70B\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.8309\u0026thinsp;\u0026plusmn;\u0026thinsp;0.02\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.8156\u0026thinsp;\u0026plusmn;\u0026thinsp;0.02\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.9872\u0026thinsp;\u0026plusmn;\u0026thinsp;0.01\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.9000\u0026thinsp;\u0026plusmn;\u0026thinsp;0.00\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.7736\u0026thinsp;\u0026plusmn;\u0026thinsp;0.04\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colspan=\"6\"\u003e\n \u003cp\u003e\u003cem\u003eDynamic 5-Shot Summary Notes\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eLlama 3.0 8B\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.8320\u0026thinsp;\u0026plusmn;\u0026thinsp;0.02\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.8417\u0026thinsp;\u0026plusmn;\u0026thinsp;0.03\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.9769\u0026thinsp;\u0026plusmn;\u0026thinsp;0.01\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.9000\u0026thinsp;\u0026plusmn;\u0026thinsp;0.00\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.7910\u0026thinsp;\u0026plusmn;\u0026thinsp;0.05\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eLlama 3.0 70B\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.7985\u0026thinsp;\u0026plusmn;\u0026thinsp;0.05\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.8657\u0026thinsp;\u0026plusmn;\u0026thinsp;0.03\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.8759\u0026thinsp;\u0026plusmn;\u0026thinsp;0.06\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.8700\u0026thinsp;\u0026plusmn;\u0026thinsp;0.03\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.7414\u0026thinsp;\u0026plusmn;\u0026thinsp;0.06\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n \u003cdiv class=\"gridtable\"\u003e\n \u003cdiv align=\"left\" class=\"colspec\"\u003e\u003cbr\u003e\u003c/div\u003e\n \u003ctable id=\"Tab2\" border=\"1\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eGlioma \u0026ndash; Comparative Performance Analysis between Llama-3.0 8B and Llama-3.0 70B.\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eModel\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eAccuracy\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003ePrecision\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eRecall\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eF1\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eAUROC\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003cth align=\"left\" colspan=\"6\"\u003e\n \u003cp\u003eZero Shot Prompting\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colspan=\"6\"\u003e\n \u003cp\u003e\u003cem\u003eZero Shot Full Notes\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eLlama 3.0 8B\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.6569\u0026thinsp;\u0026plusmn;\u0026thinsp;0.04\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.6431\u0026thinsp;\u0026plusmn;\u0026thinsp;0.02\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.9966\u0026thinsp;\u0026plusmn;\u0026thinsp;0.01\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.8000\u0026thinsp;\u0026plusmn;\u0026thinsp;0.00\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.7479\u0026thinsp;\u0026plusmn;\u0026thinsp;0.09\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eLlama 3.0 70B\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.6168\u0026thinsp;\u0026plusmn;\u0026thinsp;0.01\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.6159\u0026thinsp;\u0026plusmn;\u0026thinsp;0.01\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003e1.0000\u0026thinsp;\u0026plusmn;\u0026thinsp;0.00\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.7608\u0026thinsp;\u0026plusmn;\u0026thinsp;0.01\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.7422\u0026thinsp;\u0026plusmn;\u0026thinsp;0.04\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colspan=\"6\"\u003e\n \u003cp\u003e\u003cem\u003eZero Shot Summary Notes\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eLlama 3.0 8B\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.6548\u0026thinsp;\u0026plusmn;\u0026thinsp;0.02\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.8427\u0026thinsp;\u0026plusmn;\u0026thinsp;0.04\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.5409\u0026thinsp;\u0026plusmn;\u0026thinsp;0.04\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.6580\u0026thinsp;\u0026plusmn;\u0026thinsp;0.02\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.7534\u0026thinsp;\u0026plusmn;\u0026thinsp;0.02\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eLlama 3.0 70B\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.6779\u0026thinsp;\u0026plusmn;\u0026thinsp;0.04\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.7028\u0026thinsp;\u0026plusmn;\u0026thinsp;0.03\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.8967\u0026thinsp;\u0026plusmn;\u0026thinsp;0.03\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.7876\u0026thinsp;\u0026plusmn;\u0026thinsp;0.02\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.5705\u0026thinsp;\u0026plusmn;\u0026thinsp;0.01\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colspan=\"6\"\u003e\n \u003cp\u003e\u003cstrong\u003eDynamic Prompting\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colspan=\"6\"\u003e\n \u003cp\u003e\u003cem\u003eDynamic 2-Shot Full Notes\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eLlama 3.0 8B\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.6779\u0026thinsp;\u0026plusmn;\u0026thinsp;0.07\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.7667\u0026thinsp;\u0026plusmn;\u0026thinsp;0.07\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.6852\u0026thinsp;\u0026plusmn;\u0026thinsp;0.08\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.7000\u0026thinsp;\u0026plusmn;\u0026thinsp;0.07\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.7525\u0026thinsp;\u0026plusmn;\u0026thinsp;0.07\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eLlama 3.0 70B\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.7071\u0026thinsp;\u0026plusmn;\u0026thinsp;0.03\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.7369\u0026thinsp;\u0026plusmn;\u0026thinsp;0.03\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.8219\u0026thinsp;\u0026plusmn;\u0026thinsp;0.05\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.7763\u0026thinsp;\u0026plusmn;\u0026thinsp;0.03\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.7649\u0026thinsp;\u0026plusmn;\u0026thinsp;0.05\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" colspan=\"6\"\u003e\n \u003cp\u003e\u003cem\u003eDynamic 5-Shot Summary Notes\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eLlama 3.0 8B\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.7537\u0026thinsp;\u0026plusmn;\u0026thinsp;0.02\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.8218\u0026thinsp;\u0026plusmn;\u0026thinsp;0.02\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.7669\u0026thinsp;\u0026plusmn;\u0026thinsp;0.06\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.7920\u0026thinsp;\u0026plusmn;\u0026thinsp;0.03\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.7751\u0026thinsp;\u0026plusmn;\u0026thinsp;0.02\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eLlama 3.0 70B\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.7537\u0026thinsp;\u0026plusmn;\u0026thinsp;0.03\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.8311\u0026thinsp;\u0026plusmn;\u0026thinsp;0.05\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.7568\u0026thinsp;\u0026plusmn;\u0026thinsp;0.05\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.7906\u0026thinsp;\u0026plusmn;\u0026thinsp;0.03\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.8132\u0026thinsp;\u0026plusmn;\u0026thinsp;0.04\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n \u003cp\u003eFor the breast cancer cohort (n\u0026thinsp;=\u0026thinsp;503), zero-shot inference achieved robust baseline performance across both model architectures. Llama-3.0 8B attained 83% accuracy with an F1 score of 0.90, while the 70B variant showed comparable metrics. Implementing dynamic prompting with 2-shot full notes yielded minimal performance gains (\u0026lt;\u0026thinsp;1% accuracy improvement for both models). GPT-4-generated summarization alone produced marginal effects, with accuracy remaining within 1% of baseline. The combined summarization-dynamic prompting approach achieved the highest accuracy for the 8B model (83.2\u0026thinsp;\u0026plusmn;\u0026thinsp;2%), representing a marginal improvement over zero-shot performance. Notably, recall remained consistently high across all configurations (\u0026gt;\u0026thinsp;0.9 for most experiments), while precision showed greater variability.\u003c/p\u003e\n \u003cp\u003eIn contrast, the glioma dataset (n\u0026thinsp;=\u0026thinsp;475) exhibited markedly different performance characteristics in the zero-shot configuration: the baseline metrics for both Llama-3.0 8B and 70B models were substantially lower than those observed for breast cancer, with accuracy decreasing by 21% and 16% respectively. This performance differential was accompanied by a pronounced prediction bias, with both models demonstrating near-universal positive class prediction despite the glioma dataset containing only a slight positive skew. The high variance in glioma zero-shot performance (\u0026plusmn;\u0026thinsp;4% for 8B) compared to breast cancer (\u0026plusmn;\u0026thinsp;1%) further indicated model uncertainty when processing these glioma EHR notes. These quantitative differences suggest fundamental disparities in how the models process breast cancer versus glioma documentation, with glioma notes presenting distinct challenges that zero-shot inference fails to address adequately. The integration of summarization with 5-shot dynamic prompting yielded the most substantial improvements for glioma prediction: both Llama models achieved 75.37% accuracy, representing an absolute improvement of 9.68% (8B) and 13.69% (70B) over zero-shot baselines. The F1 scores also reached 0.79 for both 8B and 70B models, while AUROC improved to 0.77 and 0.81, respectively.\u003c/p\u003e\n\u003c/div\u003e\n\u003ch3\u003eAblative Studies: Comparative Performance of Llama-3.0 8B and Llama-3.0 70B\u003c/h3\u003e\n\u003cp\u003eWe conducted ablative studies using varying model sizes and architectures to explore model robustness, consistency, and interpretability. This systematic evaluation of multiple models, including Llama-3.0 8B, Llama-3.0 70B, and GPT-4, aimed to assess how different configurations influence performance on the dynamic prompting task. These studies allowed us to determine whether conclusions from the foundational Llama-3.0 8B model generalize across other architectures, providing insights into their scalability and adaptability for clinical applications.\u003c/p\u003e\n\u003cp\u003eFigures \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e and \u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e illustrate that the analysis revealed significant trends when comparing the Llama-3.0 8B and 70B models across the breast cancer and glioma cohorts. The performance metrics for the breast cancer dataset were broadly comparable between the two models, with the 8B model slightly outperforming the 70B model in terms of accuracy and AUROC. This finding aligns with classical statistical theories, suggesting that larger models may be prone to overfitting due to their increased parameterization\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e25\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e27\u003c/span\u003e\u003c/sup\u003e. This overfitting tendency, coupled with the simplicity of the breast cancer dataset relative to glioma, may explain why the smaller 8B model displayed a marginal advantage in accuracy.\u003c/p\u003e\n\u003cp\u003eThe glioma cohort presented a more complex scenario, with dynamic prompting applied to full notes and summaries yielding substantial improvements in accuracy and AUROC for both the 8B and 70B models. These results highlight the generalizability of the dynamic prompting methodology across model sizes. Yet, the interplay between precision and recall revealed notable disparities. For the glioma dataset, precision was on par or higher for the 8B model, while recall was on par or higher for the 70B model. These differences stem from the inherent trade-offs between smaller and larger models\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e28\u003c/span\u003e\u003c/sup\u003e: the 8B model, with fewer parameters, tends to be more conservative in its predictions, favoring precision over recall. Conversely, the 70B model, with its broader capacity for parameter representation, leans toward inclusivity, prioritizing recall at the expense of precision. Dynamic prompting mitigated these disparities by providing the contextual scaffolding to balance the models\u0026rsquo; predictions. The contextual cues inherent in dynamic prompts allowed both the 8B and 70B models to reduce biases associated with their architectural limitations, leading to more equitable performance. This is evident in the narrowing precision-recall gap, which suggests that dynamic prompting helps align the predictions of these models, making them more robust and reliable across complex datasets like glioma. While the glioma dataset\u0026apos;s fluctuations in precision and recall indicate challenges associated with class imbalance, the overall gains in accuracy and AUROC across both model sizes reaffirm the efficacy of dynamic prompting. These results demonstrate that dynamic prompting enhances contextual understanding, balances prediction biases, and facilitates more reliable clinical outcome predictions regardless of the model size. This ability to generalize dynamic prompting across architectures validates its scalability and demonstrates its potential as a versatile tool in deploying large language models for nuanced clinical tasks.\u003c/p\u003e\n\u003ch3\u003eEvaluation of Majority Voting and Overrides\u003c/h3\u003e\n\u003cp\u003eWe evaluate both versions of the Llama 3.0 models on glioma and breast dynamic summary 5-shot experiments, focusing on their ability to deviate from majority voting and correctly override flawed majority decisions. The results reveal clear behavioral tendencies across the two model configurations (8B and 70B) and prompt enforcement levels for the summary dynamic prompting with 5-shot.\u003c/p\u003e\n\u003cp\u003eFor glioma, the Llama 3.0 8B model broke the majority vote more frequently, averaging 33 \u0026plusmn;\u003c/p\u003e\n\u003cp\u003e4 times per fold compared to 24\u0026thinsp;\u0026plusmn;\u0026thinsp;4 times for the 70B model. Despite this higher deviation rate, the 8B model achieved more correct overrides against incorrect majority decisions, with 22\u0026thinsp;\u0026plusmn;\u0026thinsp;2 correct overrides versus 16\u0026thinsp;\u0026plusmn;\u0026thinsp;3 correct overrides for the 70B model. These results suggest that the 8B model exhibits greater flexibility and independence in decision-making, whereas the 70B model leads to more consistent but less independent behavior.\u003c/p\u003e\n\u003cp\u003eIn the breast classification tasks, the models displayed slightly different dynamics. The 8B and 70B models broke the majority vote at comparable rates, with 15\u0026thinsp;\u0026plusmn;\u0026thinsp;4 and 22\u0026thinsp;\u0026plusmn;\u0026thinsp;8 instances, respectively. However, the 70B model demonstrated a higher number of correct overrides against the majority, achieving 15\u0026thinsp;\u0026plusmn;\u0026thinsp;4 correct overrides compared to 11\u0026thinsp;\u0026plusmn;\u0026thinsp;4 overrides for the 8B model.\u003c/p\u003e\n\u003cp\u003eThis pattern diverges from the glioma results, where the 8B model outperformed in identifying errors in the majority consensus. The analysis suggests that the 70B model has a more substantial capacity for acting independently and identifying errors in breast classification tasks. In contrast, the 8B model exhibits a more balanced performance across different datasets, with some advantages in glioma tasks. This nuanced behavior highlights dataset-specific strengths for each model.\u003c/p\u003e\n\u003cp\u003eAcross both glioma and breast tasks, the results reveal a consistent trade-off between model size and prompt enforcement. The Llama 3.0 8B model tends to deviate more frequently from the majority consensus and demonstrates greater success in overriding incorrect majority decisions. This suggests that the smaller model relies more on intrinsic reasoning, which can be advantageous in tasks requiring independent judgments. In contrast, the 70B model aligns more consistently with the majority but exhibits limited flexibility in overriding errors.\u003c/p\u003e\n\u003cp\u003eThese results in Table\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e highlight that the performance of Llama 3.0 models, particularly in breaking majority voting and correctly overriding inaccurate majority decisions, is not solely reliant on consensus. Instead, the outcomes reflect a significant degree of independent decision-making, facilitated by dynamic prompting strategies. This autonomy underscores the models\u0026rsquo; ability to evaluate and interpret inputs beyond majority-driven biases.\u003c/p\u003e\n\u003cdiv class=\"gridtable\"\u003e\n \u003ctable id=\"Tab3\" border=\"1\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eGPT-4 variability: standard deviations of 3 consecutive runs.\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eDisease\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eMethod\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eAccuracy\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eF1\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" rowspan=\"2\"\u003e\n \u003cp\u003e\u003cem\u003eBreast\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eFull notes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.0057\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.0047\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2-shot learning with full notes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.0361\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.0270\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" rowspan=\"2\"\u003e\n \u003cp\u003e\u003cem\u003eGlioma\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eFull notes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.0182\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.0382\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2-shot learning with full notes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.0161\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.0133\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n\u003c/div\u003e\n\u003ch3\u003eEmbedding Visualization with Prognostic Features\u003c/h3\u003e\n\u003cp\u003eTo evaluate the ability of LLMs to capture prognostic trends from unstructured EHR data, we visualized patient-specific embeddings generated by our pipeline using t‐distributed stochastic neighbor embedding (t‐SNE) for both breast cancer and glioma cohorts (Figs. \u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e and 4). These embeddings were colored by overall survival and overlaid with key clinical annotations to reveal the relationship between latent text representations and patient outcomes. In both cohorts, regions of extended survival (indicated by cooler color gradients) are clearly separable from those associated with poorer outcomes. The breast cancer embeddings exhibit tighter, more defined clusters compared with glioma, reflecting the larger sample size and more structured documentation in the breast cohort, aligning with our prior quantitative analyses showing higher predictive accuracy and F1 scores for breast cancer than for glioma. Figure \u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e presents the breast cancer t‐SNE: hormone receptor-positive (HR+) cases concentrate within the high‐survival tail of the embedding space ( \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:p\u0026lt;7.7\\times\\:{10}^{-8}\\)\u003c/span\u003e\u003c/span\u003e), and patients treated with lumpectomy similarly map to this favorable region (\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:p\u0026lt;9.5\\times\\:{10}^{-7}\\)\u003c/span\u003e\u003c/span\u003e). By contrast, embeddings labeled as metastatic disease localize to clusters of shorter survival (\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:p\u0026lt;2.4\\times\\:{10}^{-4}\\)\u003c/span\u003e\u003c/span\u003e). As for HER2‐positive status, nearly all HER2+ points fall into the compact bottom cluster associated with the longest median survival (\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:p\u0026lt;2.7\\times\\:{10}^{-2}\\)\u003c/span\u003e\u003c/span\u003e), demonstrating the transformative impact of modern HER2‐targeted therapy in reshaping prognosis for the breast cancer\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e29\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e31\u003c/span\u003e\u003c/sup\u003e. Figure\u0026nbsp;4 illustrates the glioma t‐SNE, where embeddings marked by 1p-19q co-deletion, a canonical marker of oligodendroglioma, aggregate in regions of prolonged survival (\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:p\u0026lt;1.3\\times\\:{10}^{-6}\\)\u003c/span\u003e\u003c/span\u003e), whereas MGMT-unmethylated cases map predominantly to the lower-survival clusters (\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:p\u0026lt;4.8\\times\\:{10}^{-7}\\)\u003c/span\u003e\u003c/span\u003e), consistent with their well-known resistance to temozolomide and adverse prognosis\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e32\u003c/span\u003e,\u003cspan class=\"CitationRef\"\u003e33\u003c/span\u003e\u003c/sup\u003e. Multifocal gliomas, a known harbinger of aggressive behavior, are enriched in the lower‐survival clusters, albeit with more modest significance.\u003c/p\u003e\n\u003cp\u003eA direct comparison of embedding patterns between the breast cancer and glioma cohorts reveals marked differences in clustering quality and survival alignment. In the breast cancer cohort, embeddings exhibit well-defined separations, with distinct sub-clusters corresponding to favorable and unfavorable outcomes. This clarity reflects the model\u0026rsquo;s ability to leverage larger datasets and more consistent EHR documentation, enabling robust capture of survival trends. In contrast, the glioma embeddings are more diffuse, with overlapping clusters and less distinct separation between prognostic groups. This aligns with the challenges inherent in glioma data, including smaller cohort sizes, more heterogeneous documentation, and the dynamic nature of glioma progression.\u003c/p\u003e"},{"header":"Discussion","content":"\u003cdiv id=\"Sec8\" class=\"Section2\"\u003e\n \u003ch2\u003eVarying Performance in Breast and Glioma Notes\u003c/h2\u003e\n \u003cp\u003eIn our experiments, the model\u0026rsquo;s zero-shot performance on glioma clinical notes was substantially lower than on breast cancer notes, for both the Llama-3.0 8B and 70B versions. In other words, without any task-specific fine-tuning or prompting, these models struggled to interpret glioma-related clinical notes. In contrast, their baseline results on breast cancer notes were much stronger. This indicates a limited inherent understanding of glioma-specific nuances in the pre-trained LLMs. This observation aligns with the intuition that glioma content was underrepresented in the model\u0026rsquo;s pre-training data, while breast cancer is a far more common topic in general discourse. One likely explanation is the difference in domain exposure during pre-training. LLMs are trained on vast internet corpora, but these corpora contain much more information on common diseases (like breast cancer) than on rarer, highly specialized conditions (like gliomas). Breast cancer, being one of the most prevalent cancers worldwide, with over 2.3\u0026nbsp;million new cases in 2020\u003csup\u003e34\u003c/sup\u003e, is also among the most widely discussed cancers on the internet\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e35\u003c/span\u003e\u003c/sup\u003e. As a result, generic LLMs have likely encountered and learned patterns about breast cancer in news, social media, and general health discussions. In contrast, gliomas, though the most common primary brain tumors, are comparatively rare, with roughly 6 cases per 100,000 people per year in the U.S.\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e36\u003c/span\u003e\u003c/sup\u003e and they receive far less attention in mainstream media or social platforms. Consequently, glioma-related text is sparse in the common pre-training corpus, leaving the model with a knowledge gap. Indeed, prior studies have noted that LLMs tend to perform worse on under-represented medical topics or rare diseases than on common ones, precisely because rare conditions are underrepresented in internet-scale text data\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e37\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e39\u003c/span\u003e\u003c/sup\u003e. Our findings reflect this phenomenon: both Llama 8B and 70B\u0026rsquo;s near-out-of-the-box ceiling performance on breast cancer notes versus their struggle on glioma notes demonstrates how pre-training data biases can impact zero-shot clinical NLP tasks.\u003c/p\u003e\n \u003cp\u003eAnother contributing factor is the complexity and specificity of glioma pathology, which demands deeper contextual understanding. Gliomas are biologically complex and highly heterogeneous tumors\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e40\u003c/span\u003e\u003c/sup\u003e, with subtypes and molecular markers (IDH mutation status, 1p/19q codeletion, MGMT promoter methylation, etc.) that carry nuanced implications. Interpreting glioma clinical notes often requires recognizing specialized terminology and subtle clinical context, which a general-purpose model may not readily grasp without exposure to domain-specific knowledge. Breast cancer notes, while not trivial, may include more widely-known clinical terms, such as \u0026ldquo;lymph node involvement,\u0026rdquo; \u0026ldquo;HER2 status,\u0026rdquo; or \u0026ldquo;metastases\u0026rdquo;, which generic LLMs may understand from their broad training. The dynamic and complex nature of glioma cases involving interdisciplinary knowledge of neuro-oncology, neuropathology, and evolving classification criteria based on various molecular biomarkers\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e41\u003c/span\u003e,\u003cspan class=\"CitationRef\"\u003e42\u003c/span\u003e\u003c/sup\u003e means that a model must be aligned with specialized corpora or expert knowledge to perform well. Without that alignment, the zero-shot LLM often fails to capture critical details in glioma notes, resulting in the poor baseline performance we observed.\u003c/p\u003e\n \u003cp\u003eTo operationalize rare cancers in this context, we adopt the NCI definition of incidence of fewer than 15 out of 100,000 people each year\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e43\u003c/span\u003e\u003c/sup\u003e. This cutoff aligns with limited representation in the corpora used to pre-train generic LLMs. Gliomas meet this criterion and exhibit large performance gains from dynamic prompting, whereas breast cancer (about 130.8 per 100,000 women per year\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e44\u003c/span\u003e\u003c/sup\u003e) did not. Notably, glioma patients often generate numerous clinical notes due to the disease\u0026rsquo;s rapid progression and aggressive treatment protocols, yet the pre‑training corpus remains sparse, explaining the model\u0026rsquo;s suboptimal zero‑shot results. \u003cem\u003eDynamic prompting thus offers the most benefit when pre‑training coverage is low but can be safely applied across all tumor types without performance degradation\u003c/em\u003e: In domains where the pre-trained LLM lacks sufficient knowledge, as in the case of glioma, our framework can inject relevant domain-specific context with minimal computational and data costs, dramatically improving performance. By prompting the model with glioma-specific corpora, we enable it to overcome the knowledge deficit from pre-training. In comparison, the breast cancer outcome prediction showed relatively minor gains from domain-specific alignment. This disparity underlines a key point: for highly specialized medical topics, LLM pre-training alone is often insufficient, and tailored approaches, like the dynamic prompting framework in this study, are essential to reach high accuracy and nuanced comprehension. Our framework\u0026rsquo;s impact is most pronounced in these challenging settings, validating its relevance for oncology NLP in specialized disease areas where general LLMs would otherwise cap out at suboptimal performance due to a lack of pre-training data.\u003c/p\u003e\n\u003c/div\u003e\n\u003ch3\u003eInstability of GPT-4 and its clinical implications\u003c/h3\u003e\n\u003cp\u003eOne of the most critical limitations of GPT-4, as evidenced by our experiments in Table\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e, is its inherent instability and nondeterministic behavior, which poses significant challenges for its deployment in clinical applications. This instability was evaluated through consecutive runs using GPT-4 as the predictive model for breast cancer and glioma classification tasks. Despite controlling for all hyperparameters, including setting the temperature to zero and disabling sampling, we observed notable deviations in prediction consistency across multiple runs.\u003c/p\u003e\n\u003cp\u003eThe results highlight the variability inherent in GPT-4\u0026apos;s outputs. For breast cancer, dynamic 2-shot inference exhibited a 3.6% deviation in accuracy across three consecutive runs. Similarly, for the glioma dataset, the F1 score deviated by 3.8%, underscoring the unpredictability of GPT-4 in a healthcare context. These deviations are particularly concerning, given the binary nature of the classification task, where the query responses are restricted to positive or negative options. When the log-likelihoods for these two tokens are closely matched, the model\u0026apos;s nondeterministic decoding process can cause a positive response to flip to a negative one or vice versa, introducing variability that undermines its reliability.\u003c/p\u003e\n\u003cp\u003eInterestingly, the instability was more pronounced in the dynamic prompting workflow, suggesting that the additional context and reasoning processes introduced by dynamic prompts may amplify the model\u0026apos;s inherent uncertainty. This aligns with findings from our ablative study, where model size and architecture influenced prediction consistency. However, while dynamic prompting can mitigate some biases and improve performance in specific scenarios, it also appears to interact with GPT-4\u0026rsquo;s nondeterminism in unpredictable ways, further compounding the reliability issue.\u003c/p\u003e\n\u003cp\u003eThe implications of this variability are profound, particularly in the context of highly sensitive clinical data. Even a slight deviation in model output can significantly impact healthcare applications, leading to misclassification of patient conditions and potentially impacting clinical decision-making. For example, a mislabel in a glioma classification task could result in inappropriate treatment recommendations, compounding the already high stakes of managing complex diseases like glioma. Despite extensive parameter control, the inability to guarantee consistent outputs raises serious concerns about the suitability of GPT-4 for tasks where accuracy and reliability are paramount. Our findings suggest that GPT-4, in its current form, is not a viable standalone solution for predictive modeling in healthcare. While the model excels in natural language understanding and general-purpose tasks, its instability limits its application in domains where consistency and precision are non-negotiable. Addressing this issue would require further refinement of the model\u0026rsquo;s decoding mechanisms or the development of additional post-processing layers to filter out inconsistencies. Until such solutions are realized, models like Llama-3, which exhibit greater stability and reliability, present a more promising pathway for integrating LLMs into clinical workflows. These results emphasize the need for careful evaluation and adaptation of LLMs to ensure their safe and effective use in sensitive, high-stakes environments.\u003c/p\u003e\n\u003ch3\u003eImpact of Note Summarization on Outcome Prediction\u003c/h3\u003e\n\u003cp\u003eThe concise and focused nature of summary notes generated by GPT-4 demonstrates a significant advantage in outcome prediction tasks, effectively distilling key diagnostic information while filtering out extraneous details in full clinical notes. Full clinical notes, while comprehensive, often contain substantial noise, such as repetitive templates, which can dilute the critical signals necessary for accurate survival prediction. This issue is particularly pronounced in small datasets with lengthy notes, as is the case for both the breast cancer and glioma cohorts. In such scenarios, including irrelevant details can increase the likelihood of model overfitting, whereas summary notes enhance generalizability and mitigate this risk by focusing solely on the essential clinical information.\u003c/p\u003e\n\u003cp\u003eOur results in Tables\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e and \u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e clearly illustrate this effect. When evaluated with the baseline Llama-3.0 8B model, summary notes consistently outperformed full notes across both breast and glioma cohorts when dynamic prompting was employed. The improvement was especially pronounced in the glioma cohort, where the challenges of smaller datasets and more complex clinical terminology amplify the benefits of streamlined input. These findings highlight the value of note summarization as a preprocessing step, boosting performance metrics like accuracy and F1 score and reducing the cognitive load on the model by removing unnecessary noise.\u003c/p\u003e\n\u003cp\u003eHowever, implementing this workflow comes with added computational requirements and costs. The summarization process necessitates an extra preprocessing step, where the clinical data is sent to a secure, encrypted version of GPT-4\u003csup\u003e45\u003c/sup\u003e. While this ensures data privacy and compliance with regulatory standards, it incurs additional costs and demands more computational resources during preprocessing. Despite this, the downstream benefits of using summary notes offset these initial overheads. Notably, less computational power is required for inference when using summarized data, as opposed to full notes, making the overall workflow more efficient during the prediction phase.\u003c/p\u003e\n\u003cp\u003eThe efficiency gains are particularly relevant in clinical contexts where real-time or near-real-time predictions are necessary. By reducing the computational burden during inference, the summarization workflow ensures that Llama-based models can be effectively deployed even in resource-constrained environments, such as community hospitals or smaller oncology practices. Additionally, the streamlined input allows the model to focus on the most pertinent features, ensuring more robust and interpretable outputs. Overall, summary notes provide a more effective and computationally efficient approach to outcome prediction. While the workflow requires an additional preprocessing step and associated costs, the improved model performance and reduced inference demands illustrate its practicality and potential for integration into clinical workflows.\u003c/p\u003e"},{"header":"Methods","content":"\u003ch2\u003ePatient selection criteria and assembly of the dataset\u003c/h2\u003e\n\u003cp\u003eThis study focused on oncology patients with confirmed diagnoses of glioma or breast cancer, established through histopathological or radiological evidence. The cohorts were derived from a single institution and spanned diagnoses between 2010 and 2019. Unstructured EHR notes from radiology, pathology, medical oncology, and radiation oncology were included. To ensure adequate data for analysis, only patients with at least three clinical notes recorded within the first 180 days post-diagnosis were considered eligible for inclusion. Patients were categorized based on survival status relative to a predefined survival threshold (14 months for glioma and 5 years for breast cancer). Patients alive during data collection but had not yet reached these thresholds were excluded. Demographic details for the included cohorts are summarized in Table\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e4\u003c/span\u003e. Stratified random sampling was applied to ensure a balanced representation of cancer stage, grade, and survival outcomes. This approach divided the dataset into 80% training and 20% testing subsets, maintaining a proportional representation of survival status and clinical attributes. Importantly, these distributions remained consistent throughout the study, with no addition or removal of patients during subsequent analyses. For robustness, the patient data was split into training and testing sets five times, mitigating any biases associated with a single data split. Patient-specific documents were dynamically assembled at each iteration by concatenating all EHR notes generated during the first 180 days post-diagnosis. Each patient document was labeled with their survival status based on the relevant threshold (e.g., 14 months for glioma or 5 years for breast cancer). The dataset\u0026apos;s dynamic assembly method allowed for including all relevant notes created within the specified timeframe while ensuring the survival label corresponded to the patient\u0026apos;s status at the predefined threshold. For patients who passed away within the observation window, their notes remained static beyond their date of death, preserving the integrity of the dataset and the associated survival labels. This study was approved by the Institutional Review Board (UCSF: IRB# 20\u0026ndash;32527).\u003c/p\u003e\n\u003cdiv class=\"gridtable\"\u003e\n \u003ctable id=\"Tab4\" border=\"1\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 4\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003ePatient features for breast cancer and glioma patient cohorts.\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eFeature\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eCategory\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eBreast\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eGlioma\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ePatients\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e503\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e475\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" rowspan=\"3\"\u003e\n \u003cp\u003eGeneral\u003c/p\u003e\n \u003cp\u003e(median, range) in years\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAge\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e57, [26\u0026ndash;95]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e50, [1-103]\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eFollow-up\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e5.7, [0.1\u0026ndash;8.4]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1.4, [0.04\u0026ndash;10.2]\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTime to death\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2.4 [0.1\u0026ndash;7.7]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1.1, [0.04\u0026ndash;6.6]\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" rowspan=\"2\"\u003e\n \u003cp\u003eSex\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eMale\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e291\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eFemale\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e503\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e184\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" rowspan=\"4\"\u003e\n \u003cp\u003eRace\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eWhite\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e395 (78.5%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e421 (88.6%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAsian\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e71 (14.1%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e29 (6.1%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eBlack\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e25 (5.0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e5 (1.1%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eOther\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e12 (2.4%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e20 (4.2%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" rowspan=\"6\"\u003e\n \u003cp\u003ecTNM Stage/Grade\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e55 (10.9%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e158 (31.4%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3 (0.6%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e159 (31.6%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e10 (2.1%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e53 (10.5%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e91 (19.2%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e48 (9.5%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e371 (78.1%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eUnknown\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e30 (6.0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e-\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" rowspan=\"5\"\u003e\n \u003cp\u003eTreatment\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eSurgery\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e454 (90.3%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e404 (85.1%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eRadiotherapy\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e258 (51.3%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e425 (89.5%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eChemotherapy\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e238 (47.3%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e402 (84.6%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eEndocrine therapy\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e335 (66.6%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1 (0.2%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eImmunotherapy\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e29 (5.8%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e49 (10.3%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" rowspan=\"3\"\u003e\n \u003cp\u003eAlcohol use\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNo or low\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e339 (67.4%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e315 (66.3%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eYes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e150 (29.8%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e156 (32.8%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eUnknown\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e14 (2.8%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4 (0.8%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" rowspan=\"2\"\u003e\n \u003cp\u003eSmoking use\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNever smoker\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e321 (63.8%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e278 (58.5%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eCurrent or former smoker\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e182 (36.2%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e197 (41.5%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" rowspan=\"4\"\u003e\n \u003cp\u003eMarital Status\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eMarried\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e300 (59.6%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e323 (68.0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eSingle\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e103 (20.5%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e108 (22.7%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eDivorced or separated\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e96 (19.1%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e43 (9.1%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eUnknown\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4 (0.8%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1 (0.2%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n\u003c/div\u003e\n\u003ch2\u003eFramework for Dynamic Prompting\u003c/h2\u003e\n\u003cp\u003eThe dynamic prompting framework is designed to optimize the use of EHR notes for predictive modeling by leveraging both complete and summarized data paths, as indicated by Fig. \u003cspan class=\"InternalRef\"\u003e5\u003c/span\u003e. The preprocessing phase in the full-note workflow begins with splitting the dataset into training and testing sets. The training samples are then passed through the embedding model to generate patient-specific embeddings. Due to the model\u0026apos;s context window limitations, each sample is chunked, and chunk-level embeddings are averaged to create a final, patient-level embedding. This approach preserves the semantic integrity of lengthy notes while ensuring compatibility with the model\u0026apos;s computational constraints. The generated embeddings are stored and clustered using a nearest-neighbors algorithm. These clusters, visualized through t-SNE (as shown in Figs. \u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e and 4), provide insights into the semantic similarities within the dataset. During inference, embeddings are generated for testing samples using the same embedding model. A k-nearest neighbors algorithm identifies the two most semantically similar training samples, which are then appended to the testing sample and its ground truth labels. This composite dataset and a carefully crafted query are passed to the predictive model. Model temperature is set to zero to enforce greedy decoding, while the maximum output length is restricted to two tokens, ensuring interpretable and extractable predictions. The details of the embedding and predictive backbone models are outlined in Table\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e5\u003c/span\u003e.\u003c/p\u003e\n\u003cdiv class=\"gridtable\"\u003e\n \u003ctable id=\"Tab5\" border=\"1\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 5\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eModels Used in the Dynamic Prompt Study.\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eModel Name\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eTask\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003e#. of Parameters\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eContext Window\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cem\u003ejina-embeddings-v2-base\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eEmbedding Generation\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e137 million\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e8,192 tokens\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cem\u003egpt-4-turbo\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eSummary Generation\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eUnreported\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e128,000 tokens\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cem\u003eLlama-3-8B-Instruct-262k\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ePredictive Model\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e8 billion\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e262,144 tokens\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cem\u003eLlama-3-70B-Instruct-Gradient-262k\u003c/em\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ePredictive Model (Ablation)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e70 billion\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e262,144 tokens\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n\u003c/div\u003e\n\u003cp\u003eThe summarized notes workflow enhances the efficiency and relevance of the input data. After the dataset is split, the training samples are summarized using a dedicated summarization model guided by a structured outline. This step is performed with deterministic settings - temperature set to zero and sampling disabled - to ensure repeatable outputs. The generated summaries are validated through keyword searches or human review to confirm consistency and accuracy. The summarized data is then processed using the same steps as the full notes workflow, with a few key differences. During inference, the testing samples are summarized using the same structured outline. The shorter sequence lengths resulting from summarization allow for the inclusion of up to five semantically similar training samples, increasing the contextual richness of the input. The streamlined nature of summarized inputs reduces computational demands while preserving critical clinical information, making this workflow particularly advantageous for tasks with limited context windows. All inference tasks were executed on an 8-GPU NVIDIA A6000 cluster, utilizing the vLLM framework to implement tensor parallelism, ensuring efficient and scalable model deployment\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e46\u003c/span\u003e\u003c/sup\u003e.\u003c/p\u003e\n\u003ch2\u003ePrompting Methods for Note Summary Generation\u003c/h2\u003e\n\u003cp\u003eStructured templates were meticulously designed for breast cancer and glioma patient notes to ensure comprehensive inclusion of key clinical elements. These templates covered crucial categories such as diagnosis, treatment history, symptoms, molecular biomarkers, and follow-up recommendations, creating a standardized yet adaptable framework for both cohorts. By emphasizing uniformity, the templates facilitated accurate classification while accommodating the specific characteristics of each cancer type.\u003c/p\u003e\n\u003cp\u003eIn the breast cancer cohort, the GPT-4-turbo-128k model showcased exceptional performance in filling out the templates with minimal intervention. The templates were completed efficiently and comprehensively, requiring no re-prompting or iterative refinement. This approach ensured that all relevant information was accurately captured by streamlining the process, enabling a reliable foundation for downstream tasks. Conversely, the glioma cohort presented unique challenges, requiring a more refined and iterative prompting approach, as shown in the right panel of Fig. \u003cspan class=\"InternalRef\"\u003e6\u003c/span\u003e. Glioma patient notes, characterized by their specialized terminology and intricate clinical details, necessitated iterative prompting to clarify instructions and enhance specificity. For instance, rephrasing general directives such as \u0026ldquo;Use these notes to fill out the template\u0026rdquo; to detailed, step-by-step instructions significantly improved the model\u0026rsquo;s understanding. Contextual prompts, where the model was explicitly guided as an AI assistant tasked with completing the template, further enhanced output precision.\u003c/p\u003e\n\u003cp\u003eNegative prompting\u003csup\u003e\u003cspan class=\"CitationRef\"\u003e47\u003c/span\u003e\u003c/sup\u003e, another key strategy, was employed to guide the model in avoiding errors or omissions. Instructions like \u0026ldquo;Ensure no clinical or molecular data is overlooked\u0026rdquo; directed the model toward critical glioma-specific elements such as 1p/19q co-deletion, IDH mutation status, Karnofsky Performance Status, and radiation therapy prescription dosage. These strategies compensated for the specialized nature of glioma notes, enabling the model to retrieve the intricate details required for accurate template completion. While the glioma cohort demanded multiple iterations, the combined use of iterative and negative prompting ultimately captured all essential clinical and molecular details. The complexity of the notes, coupled with the rare and nuanced nature of glioma cases, highlighted the importance of these advanced prompting techniques in achieving accurate information retrieval.\u003c/p\u003e\n\u003cp\u003eFigure \u003cspan class=\"InternalRef\"\u003e7\u003c/span\u003e illustrates the improvement in the accuracy and relevance of the progressive response through systematic prompt refinements. By iteratively specifying key clinical terms and optimizing the structure of the prompts, we demonstrate that fine-tuning prompts significantly enhances the fidelity of GPT-generated responses. The final optimized prompt included all relevant molecular biomarkers, accurately recorded radiation therapy dosages, and addressed missing data by marking fields as \u0026ldquo;Not Specified\u0026rdquo; where applicable. This refined prompting framework improved precision in identifying tumor location and treatment response and emphasized the potential of dynamic prompting in overcoming the limitations of unstructured EHR notes.\u003c/p\u003e\n\u003ch2\u003eEvaluation Metrics\u003c/h2\u003e\n\u003cp\u003eThe outcome prediction task for both breast and glioma cohorts represents a binary classification challenge, necessitating comprehensive evaluation metrics such as accuracy, precision, recall, F1 score, and AUROC (Area Under the Receiver Operating Curve). These metrics collectively provide a nuanced understanding of the model\u0026rsquo;s classification capabilities, particularly in scenarios characterized by imbalanced datasets, as seen in the breast cancer cohort. They also address contexts where different errors, such as false positives and false negatives, hold distinct clinical implications and weights. While most metrics were straightforward to compute from the outputs, AUROC required a customized computation due to the absence of directly extractable probabilities from the Llama model\u0026rsquo;s output structure. To address this, we leveraged the predicted logits, raw scores reflecting the model\u0026rsquo;s likelihood estimates for each class (POS and NEG), as a foundation. These logits were transformed into probabilities through a softmax function, which normalized the scores to represent the likelihood of each class. The AUROC was then calculated using these probabilities alongside the ground truth labels (true_labels and predicted_probs), capturing the model\u0026rsquo;s ability to differentiate between positive and negative instances across varying decision thresholds. This approach ensured a robust evaluation of the model\u0026rsquo;s discriminatory power, particularly critical in clinical prediction tasks where confidence levels can directly influence decision-making. The remaining metrics offered complementary insights into the model\u0026rsquo;s performance. Accuracy measured the overall proportion of correct predictions, while precision and recall quantified the model\u0026apos;s ability to correctly identify positive and negative instances, respectively. The F1 score, balancing precision and recall, provided a holistic view of performance, particularly valuable in imbalanced datasets. These metrics allowed for a detailed assessment of the trade-offs between precision and recall, clarifying whether the model favored conservatism (higher precision) or inclusivity (higher recall) in its predictions.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003eAuthor\u0026rsquo;s Contributions\u003c/p\u003e\n\u003cp\u003eS.A. and B.S. made equal contributions to this work, playing key roles in formulating the ideas, collecting and analyzing the data and developing the pipeline. W.C.C., N.D.P. and S.E.B. conducted clinical evaluations of the results. B.L., N.D.P., Y.I., A.L., J.C.H., and O.M. engaged in discussions on the problem and participated in the result analysis. H.L. and O.M. led the original data collection and processing. H.L. supervised and provided support throughout the research process. All authors contributed to manuscript editing and review.\u003c/p\u003e\n\u003cp\u003eCompeting Interests\u003c/p\u003e\n\u003cp\u003eAll authors declare no financial or non-financial competing interests.\u003c/p\u003e\n\u003cp\u003eAcknowledgments\u003c/p\u003e\n\u003cp\u003eH.L. and B.L. received funding from the Department of Defense Impact Award (W81XWH-22-1-0695), a seed grant from Uncle Kory Foundation 2023-2024, a UCSF HDFCCC Computational seed grant 2024-2025, and a UCSF SPORE brain tumor career enhancement program 2024-2025. B.S. received fund from UCSF HDFCCC Computational seed grant 2024-2025. The funders played no role in study design, data collection, analysis and interpretation of data, or the writing of this manuscript.\u003c/p\u003e\n\u003cp\u003eData Availability\u003c/p\u003e\n\u003cp\u003eThe datasets facilitating the findings of this study are not openly accessible due to legitimate privacy and security concerns. The original EHR note data cannot be redistributed to researchers outside of those involved in the study, approved by the IRB at the principal institution. Access to anonymized data can be facilitated through a material transfer agreement (MTA) managed by the principal institution. Owing to privacy concerns, datasets generated and/or analyzed during this study are not publicly available but can be provided by the corresponding author upon a reasonable request.\u003c/p\u003e\n\u003cp\u003eCode Availability\u003c/p\u003e\n\u003cp\u003eDynamic prompting, summarization, and outcome prediction codes used in this study are available at: https://github.com/OncoNLP/DynamicPrompting-ICL under Apache 2.0 license.\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003e\u0026ensp;OpenAI. Introducing ChatGPT. https://openai.com/blog/chatgpt (2022).\u003c/li\u003e\n\u003cli\u003e\u0026ensp;Touvron, H., Lavril, T. \u0026amp; Izacard, G. Llama: Open and efficient foundation language models. \u003cem\u003earXiv preprint arXiv \u0026hellip;\u003c/em\u003e (2023).\u003c/li\u003e\n\u003cli\u003e\u0026ensp;Grattafiori, A., Dubey, A., Jauhri, A. \u0026amp; Pandey, A. The llama 3 herd of models. \u003cem\u003earXiv preprint arXiv \u0026hellip;\u003c/em\u003e (2024).\u003c/li\u003e\n\u003cli\u003e\u0026ensp;Azar, W. S. \u003cem\u003eet al.\u003c/em\u003e LLM-Mediated Data Extraction from Patient Records after Radical Prostatectomy. \u003cem\u003eNEJM AI\u003c/em\u003e\u003cstrong\u003e2\u003c/strong\u003e, (2025).\u003c/li\u003e\n\u003cli\u003e\u0026ensp;Huang, J. \u003cem\u003eet al.\u003c/em\u003e A critical assessment of using ChatGPT for extracting structured data from clinical notes. \u003cem\u003enpj Digital Med.\u003c/em\u003e\u003cstrong\u003e7\u003c/strong\u003e, 106 (2024).\u003c/li\u003e\n\u003cli\u003e\u0026ensp;Li, D., Kadav, A., Gao, A. \u0026amp; Li, R. Automated Clinical Data Extraction with Knowledge Conditioned LLMs. \u003cem\u003eProceedings of the 31st \u0026hellip;\u003c/em\u003e (2025).\u003c/li\u003e\n\u003cli\u003e\u0026ensp;Choi, H. S., Song, J. Y., Shin, K. H., Chang, J. H. \u0026amp; Jang, B.-S. Developing prompts from large language model for extracting clinical information from pathology and ultrasound reports in breast cancer. \u003cem\u003eRadiat. Oncol. J.\u003c/em\u003e\u003cstrong\u003e41\u003c/strong\u003e, 209\u0026ndash;216 (2023).\u003c/li\u003e\n\u003cli\u003e\u0026ensp;Yang, X. \u003cem\u003eet al.\u003c/em\u003e A large language model for electronic health records. \u003cem\u003enpj Digital Med.\u003c/em\u003e\u003cstrong\u003e5\u003c/strong\u003e, 194 (2022).\u003c/li\u003e\n\u003cli\u003e\u0026ensp;Zhu, M. \u003cem\u003eet al.\u003c/em\u003e Large language model trained on clinical oncology data predicts cancer progression. \u003cem\u003enpj Digital Med.\u003c/em\u003e\u003cstrong\u003e8\u003c/strong\u003e, 397 (2025).\u003c/li\u003e\n\u003cli\u003ePark, S., Wee, C. W., Choi, S. H., Kim, K. H. \u0026amp; Chang, J. S. Improving mortality prediction after radiotherapy with large language model structuring of large-scale unstructured electronic health records. \u003cem\u003eRadiotherapy and \u0026hellip;\u003c/em\u003e (2025).\u003c/li\u003e\n\u003cli\u003eYuan, J., Tang, R., Jiang, X. \u0026amp; Hu, X. Llm for patient-trial matching: Privacy-aware data augmentation towards better performance and generalizability. \u003cem\u003eAmerican Medical Informatics \u0026hellip;\u003c/em\u003e (2023).\u003c/li\u003e\n\u003cli\u003eLin, H., Ni, L., Phuong, C. \u0026amp; Hong, J. C. Natural language processing for radiation oncology: personalizing treatment pathways. \u003cem\u003ePharmgenomics Pers Med\u003c/em\u003e\u003cstrong\u003e17\u003c/strong\u003e, 65\u0026ndash;76 (2024).\u003c/li\u003e\n\u003cli\u003eGuevara, M. \u003cem\u003eet al.\u003c/em\u003e Large language models to identify social determinants of health in electronic health records. \u003cem\u003enpj Digital Med.\u003c/em\u003e\u003cstrong\u003e7\u003c/strong\u003e, 6 (2024).\u003c/li\u003e\n\u003cli\u003eMosbach, M., Pimentel, T., Ravfogel, S., Klakow, D. \u0026amp; Elazar, Y. Few-shot Fine-tuning vs. In-context Learning: A Fair Comparison and Evaluation. in \u003cem\u003eFindings of the Association for Computational Linguistics: ACL 2023\u003c/em\u003e 12284\u0026ndash;12314 (Association for Computational Linguistics, 2023). doi:10.18653/v1/2023.findings-acl.779.\u003c/li\u003e\n\u003cli\u003eGu, Y. \u003cem\u003eet al.\u003c/em\u003e Domain-specific language model pretraining for biomedical natural language processing. \u003cem\u003eACM Trans. Comput. Healthcare\u003c/em\u003e\u003cstrong\u003e3\u003c/strong\u003e, 1\u0026ndash;23 (2022).\u003c/li\u003e\n\u003cli\u003eLuo, R. \u003cem\u003eet al.\u003c/em\u003e BioGPT: generative pre-trained transformer for biomedical text generation and mining. \u003cem\u003eBrief. Bioinformatics\u003c/em\u003e\u003cstrong\u003e23\u003c/strong\u003e, (2022).\u003c/li\u003e\n\u003cli\u003eFerber, D. \u003cem\u003eet al.\u003c/em\u003e In-context learning enables multimodal large language models to classify cancer pathology images. \u003cem\u003eNat. Commun.\u003c/em\u003e\u003cstrong\u003e15\u003c/strong\u003e, 10104 (2024).\u003c/li\u003e\n\u003cli\u003eNguyen, D., Swanson, D., Newbury, A. \u0026amp; Kim, Y. H. Evaluation of chatgpt and google bard using prompt engineering in cancer screening algorithms. \u003cem\u003eAcad. Radiol.\u003c/em\u003e\u003cstrong\u003e31\u003c/strong\u003e, 1799\u0026ndash;1804 (2024).\u003c/li\u003e\n\u003cli\u003eIivanainen, S., Lagus, J., Viertolahti, H., Sippola, L. \u0026amp; Koivunen, J. Investigating large language model (LLM) performance using in-context learning (ICL) for interpretation of ESMO and NCCN guidelines for lung cancer. \u003cem\u003eJCO\u003c/em\u003e\u003cstrong\u003e42\u003c/strong\u003e, e13637\u0026ndash;e13637 (2024).\u003c/li\u003e\n\u003cli\u003eNori, H., Lee, Y. T., Zhang, S., Carignan, D. \u0026amp; Edgar, R. Can generalist foundation models outcompete special-purpose tuning? case study in medicine. \u003cem\u003earXiv preprint arXiv \u0026hellip;\u003c/em\u003e (2023).\u003c/li\u003e\n\u003cli\u003eZhang, H., Yu, P. S. \u0026amp; Zhang, J. A systematic survey of text summarization: from statistical methods to large language models. \u003cem\u003eACM Comput. Surv.\u003c/em\u003e (2025) doi:10.1145/3731445.\u003c/li\u003e\n\u003cli\u003eChuang, Y.-N., Tang, R., Jiang, X. \u0026amp; Hu, X. SPeC: A Soft Prompt-Based Calibration on Performance Variability of Large Language Model in Clinical Notes Summarization. \u003cem\u003eJ. Biomed. Inform.\u003c/em\u003e\u003cstrong\u003e151\u003c/strong\u003e, 104606 (2024).\u003c/li\u003e\n\u003cli\u003eVan Veen, D. \u003cem\u003eet al.\u003c/em\u003e Clinical text summarization: adapting large language models can outperform human experts. \u003cem\u003eRes. Sq.\u003c/em\u003e (2023) doi:10.21203/rs.3.rs-3483777/v1.\u003c/li\u003e\n\u003cli\u003eChoudhuri, A., Polgreen, P., Segre, A. \u0026amp; Adhikari, B. Summarizing Clinical Notes using LLMs for ICU Bounceback and Length-of-Stay Prediction. in \u003cem\u003e2024 IEEE International Conference on Data Mining Workshops (ICDMW)\u003c/em\u003e 859\u0026ndash;866 (IEEE, 2024). doi:10.1109/ICDMW65004.2024.00118.\u003c/li\u003e\n\u003cli\u003eHastie, T., Tibshirani, R., Friedman, J. H. \u0026amp; Friedman, J. H. \u003cem\u003eThe elements of statistical learning: data mining, inference, and prediction\u003c/em\u003e. (Springer, 2009).\u003c/li\u003e\n\u003cli\u003eBelkin, M., Hsu, D., Ma, S. \u0026amp; Mandal, S. Reconciling modern machine-learning practice and the classical bias-variance trade-off. \u003cem\u003eProc Natl Acad Sci USA\u003c/em\u003e\u003cstrong\u003e116\u003c/strong\u003e, 15849\u0026ndash;15854 (2019).\u003c/li\u003e\n\u003cli\u003eZhang, C., Bengio, S., Hardt, M., Recht, B. \u0026amp; Vinyals, O. Understanding deep learning (still) requires rethinking generalization. \u003cem\u003eCommun. ACM\u003c/em\u003e\u003cstrong\u003e64\u003c/strong\u003e, 107\u0026ndash;115 (2021).\u003c/li\u003e\n\u003cli\u003eJuba, B. \u0026amp; Le, H. S. Precision-Recall versus Accuracy and the Role of Large Data Sets. \u003cem\u003eAAAI\u003c/em\u003e\u003cstrong\u003e33\u003c/strong\u003e, 4039\u0026ndash;4048 (2019).\u003c/li\u003e\n\u003cli\u003eMaximiano, S., Magalh\u0026atilde;es, P., Guerreiro, M. P. \u0026amp; Morgado, M. Trastuzumab in the treatment of breast cancer. \u003cem\u003eBioDrugs\u003c/em\u003e\u003cstrong\u003e30\u003c/strong\u003e, 75\u0026ndash;86 (2016).\u003c/li\u003e\n\u003cli\u003eO\u0026rsquo;Sullivan, C. C. \u003cem\u003eet al.\u003c/em\u003e Efficacy of Adjuvant Trastuzumab for Patients With Human Epidermal Growth Factor Receptor 2-Positive Early Breast Cancer and Tumors \u0026le; 2 cm: A Meta-Analysis of the Randomized Trastuzumab Trials. \u003cem\u003eJ. Clin. Oncol.\u003c/em\u003e\u003cstrong\u003e33\u003c/strong\u003e, 2600\u0026ndash;2608 (2015).\u003c/li\u003e\n\u003cli\u003eEarly Breast Cancer Trialists\u0026rsquo; Collaborative group (EBCTCG). Trastuzumab for early-stage, HER2-positive breast cancer: a meta-analysis of 13 864 women in seven randomised trials. \u003cem\u003eLancet Oncol.\u003c/em\u003e\u003cstrong\u003e22\u003c/strong\u003e, 1139\u0026ndash;1150 (2021).\u003c/li\u003e\n\u003cli\u003eHegi, M. E. \u003cem\u003eet al.\u003c/em\u003e MGMT gene silencing and benefit from temozolomide in glioblastoma. \u003cem\u003eN. Engl. J. Med.\u003c/em\u003e\u003cstrong\u003e352\u003c/strong\u003e, 997\u0026ndash;1003 (2005).\u003c/li\u003e\n\u003cli\u003eYi, G.-Z. \u003cem\u003eet al.\u003c/em\u003e Acquired temozolomide resistance in MGMT-deficient glioblastoma cells is associated with regulation of DNA repair by DHC2. \u003cem\u003eBrain\u003c/em\u003e\u003cstrong\u003e142\u003c/strong\u003e, 2352\u0026ndash;2366 (2019).\u003c/li\u003e\n\u003cli\u003eArnold, M. \u003cem\u003eet al.\u003c/em\u003e Current and future burden of breast cancer: Global statistics for 2020 and 2040. \u003cem\u003eBreast\u003c/em\u003e\u003cstrong\u003e66\u003c/strong\u003e, 15\u0026ndash;23 (2022).\u003c/li\u003e\n\u003cli\u003eQuinn, E. M. \u003cem\u003eet al.\u003c/em\u003e Who\u0026rsquo;s talking about breast cancer? Analysis of daily breast cancer posts on the internet. \u003cem\u003eBreast\u003c/em\u003e\u003cstrong\u003e22\u003c/strong\u003e, 24\u0026ndash;27 (2013).\u003c/li\u003e\n\u003cli\u003eMesfin, F. B., Karsonovich, T. \u0026amp; Al-Dhahir, M. A. Gliomas. in \u003cem\u003eStatPearls\u003c/em\u003e (StatPearls Publishing, 2025).\u003c/li\u003e\n\u003cli\u003eAbdullahi, T., Singh, R. \u0026amp; Eickhoff, C. Learning to make rare and complex diagnoses with generative AI assistance: Qualitative study of popular large language models. \u003cem\u003eJMIR Med. Educ.\u003c/em\u003e\u003cstrong\u003e10\u003c/strong\u003e, e51391 (2024).\u003c/li\u003e\n\u003cli\u003eChen, X. \u003cem\u003eet al.\u003c/em\u003e Rarebench: can llms serve as rare diseases specialists? in \u003cem\u003eProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining\u003c/em\u003e 4850\u0026ndash;4861 (ACM, 2024). doi:10.1145/3637528.3671576.\u003c/li\u003e\n\u003cli\u003eYang, J., Shu, L., Duan, H. \u0026amp; Li, H. Rdguru: A conversational intelligent agent for rare diseases. \u003cem\u003eIEEE J. Biomed. Health Inform.\u003c/em\u003e\u003cstrong\u003ePP\u003c/strong\u003e, (2024).\u003c/li\u003e\n\u003cli\u003eBerens, M. E. \u003cem\u003eet al.\u003c/em\u003e Multiscale, multimodal analysis of tumor heterogeneity in IDH1 mutant vs wild-type diffuse gliomas. \u003cem\u003ePLoS ONE\u003c/em\u003e\u003cstrong\u003e14\u003c/strong\u003e, e0219724 (2019).\u003c/li\u003e\n\u003cli\u003eWesseling, P. \u0026amp; Capper, D. WHO 2016 Classification of gliomas. \u003cem\u003eNeuropathol. Appl. Neurobiol.\u003c/em\u003e\u003cstrong\u003e44\u003c/strong\u003e, 139\u0026ndash;150 (2018).\u003c/li\u003e\n\u003cli\u003eLouis, D. N. \u003cem\u003eet al.\u003c/em\u003e The 2021 WHO Classification of Tumors of the Central Nervous System: a summary. \u003cem\u003eNeuro Oncol.\u003c/em\u003e\u003cstrong\u003e23\u003c/strong\u003e, 1231\u0026ndash;1251 (2021).\u003c/li\u003e\n\u003cli\u003eNCI rare cancer definition. https://www.cancer.gov/publications/dictionaries/cancer-terms/def/rare-cancer.\u003c/li\u003e\n\u003cli\u003eCancer Stat Facts: Female Breast Cancer. https://seer.cancer.gov/statfacts/html/breast.html.\u003c/li\u003e\n\u003cli\u003eUCSF. Versa API. https://ai.ucsf.edu/versa.\u003c/li\u003e\n\u003cli\u003eKwon, W. \u003cem\u003eet al.\u003c/em\u003e Efficient Memory Management for Large Language Model Serving with PagedAttention. in \u003cem\u003eProceedings of the 29th Symposium on Operating Systems Principles\u003c/em\u003e 611\u0026ndash;626 (ACM, 2023). doi:10.1145/3600006.3613165.\u003c/li\u003e\n\u003cli\u003eBan, Y. \u003cem\u003eet al.\u003c/em\u003e Understanding the Impact of Negative Prompts: When and How Do They Take Effect? in \u003cem\u003eComputer vision \u0026ndash; ECCV 2024: 18th european conference, milan, italy, september 29\u0026ndash;october 4, 2024, proceedings, part LXXXIX\u003c/em\u003e (eds. Leonardis, A. et al.) vol. 15147 190\u0026ndash;206 (Springer Nature Switzerland, 2025).\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"npj-digital-medicine","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"npjdigitalmed","sideBox":"Learn more about [npj Digital Medicine](http://www.nature.com/npjdigitalmed/)","snPcode":"41746","submissionUrl":"https://submission.springernature.com/new-submission/41746/3","title":"npj Digital Medicine","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"NPJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-7329357/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7329357/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eOutcome prediction from unstructured EHR notes remains challenging, especially for rare cancers with limited pre-training data. We present a dynamic prompting framework that retrieves semantically similar patient examples, constructs tailored few-shot prompts, and integrates note summaries to enhance large language model (LLM)-based outcome predictions. We evaluated this approach on a single-institution cohort of 503 breast and 475 glioma patients (EHR notes within 180 days post-diagnosis), with overall survival dichotomized at five years (breast cancer) and fourteen months (glioma). Using Llama-3 models (8B/70B), we compared zero-shot, dynamic prompting, summarization-only, and combined workflows. Dynamic prompting substantially improved glioma prediction performance, boosting accuracy by 12% and F1 by 11%, whereas gains for breast cancer were modest (\u0026lt;3%). The combined summarization-plus-prompting approach achieved the highest performance while maintaining prediction stability compared to GPT-4, addressing a critical deployment barrier. T-SNE visualizations confirmed that embeddings captured established prognostic markers. Dynamic prompting delivers maximal benefit when pre‑training exposure is low but can be safely applied across all tumor types without degrading performance on common cancers. This selective yet universal enhancement establishes dynamic prompting as a practical, scalable solution for deploying LLMs in clinical oncology, particularly for rare cancers where accurate outcome prediction can meaningfully inform treatment planning.\u003c/p\u003e","manuscriptTitle":"Leveraging Dynamic Prompting for Outcome Prediction of Cancer Patients Using Large Language Models and Electronic Health Record Notes","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-09-01 09:37:21","doi":"10.21203/rs.3.rs-7329357/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-09-28T15:20:07+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-09-24T23:10:26+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-09-14T15:40:24+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-09-12T04:58:53+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"337895706370649246890852170467588635800","date":"2025-08-29T15:59:39+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"300495151200834491892269257186204440251","date":"2025-08-22T15:07:01+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"311296281425479420881056086602396013736","date":"2025-08-21T03:44:58+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-08-20T15:01:24+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-08-12T13:25:31+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-08-12T04:47:13+00:00","index":"","fulltext":""},{"type":"submitted","content":"npj Digital Medicine","date":"2025-08-08T17:18:29+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"npj-digital-medicine","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"npjdigitalmed","sideBox":"Learn more about [npj Digital Medicine](http://www.nature.com/npjdigitalmed/)","snPcode":"41746","submissionUrl":"https://submission.springernature.com/new-submission/41746/3","title":"npj Digital Medicine","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"NPJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"639d9866-9baa-42a5-8356-f2266c545b31","owner":[],"postedDate":"September 1st, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"in-revision","subjectAreas":[{"id":53808668,"name":"Biological sciences/Cancer"},{"id":53808669,"name":"Biological sciences/Computational biology and bioinformatics"},{"id":53808670,"name":"Health sciences/Oncology"}],"tags":[],"updatedAt":"2026-05-15T07:28:32+00:00","versionOfRecord":[],"versionCreatedAt":"2025-09-01 09:37:21","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7329357","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7329357","identity":"rs-7329357","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}