Decoding Clinician Authorial Style: A Style-Informed Pipeline for Clinical Document Summary Generation with Large Language Models

Decoding Clinician Authorial Style: A Style-Informed Pipeline for Clinical Document Summary Generation with Large Language Models | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Decoding Clinician Authorial Style: A Style-Informed Pipeline for Clinical Document Summary Generation with Large Language Models Scott Zhao, Abbas Alili, Usman Afzaal, Muhammet F. Demir, Hao Lu, and 2 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-9054955/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 9 You are reading this latest preprint version Abstract Large language models (LLMs) can automate clinical document summary generation. However, even clinically accurate outputs often fail to reflect individual clinicians’ writing styles, leading to substantial post-editing. We examine this stylistic gap using a multi-author corpus of de-identified clinical summaries. We propose a style-informed generation framework that extracts clinician-specific stylistic features through LLM feedback and applies a Train→Generate paradigm to produce personalized clinical summaries. Conventional metrics (ROUGE, BERTScore, cosine similarity) largely failed to distinguish intra-author from inter-author writing patterns, while Jaro-Winkler and BLEU demonstrated limited sensitivity. Targeted LLM-guided feature extraction—emphasizing rhythm, narration, and sentence or list structure—improved authorship classification up to 73% of accuracy. In blinded clinician A/B testing, GPT-4-generated drafts were preferred less often than original notes, whereas the Gemini 2.5 Pro pipeline produced drafts preferred at rates comparable to, and in some cases exceeding, clinician-authored summaries. While inherent hallucination risks were noted, they were mitigated via high-fidelity prompt engineering and explicit adherence to source-only data constraints. These results suggest that style-informed generation can reduce the style gap and produce clinically acceptable clinical summaries that better align with the clinician’s voice. Biological sciences/Computational biology and bioinformatics Health sciences/Diseases Health sciences/Health care Health sciences/Medical research Figures Figure 1 Figure 2 Figure 3 Figure 4 1. Introduction The hospital clinical summary serves as the primary clinical bridge for patient handoffs, acting as the definitive communication tool to ensure continuity of care between treatment teams and reduce the risk of avoidable readmissions 1 , 2 . During the precarious transition period covering the first 72 hours following a hospital stay, patients are uniquely vulnerable to adverse drug events and fragmented follow-up care due to a lack of or poor communication. This vulnerability comes due to the complexity of medical interventions and frequent transitions between care settings 2 . Ensuring the precision of this documentation is vital for patient safety. However, this necessity creates a significant administrative load on healthcare providers. In practice, generating a single clinical summary is a demanding process, requiring an average of 8.1 minutes for dictation and a median of 29.2 minutes for transcription and editing, often resulting in documents several pages in length 3 . This substantial time investment increases clinicians' cognitive load, fueling documentation-related burnout and diverting essential time from direct patient care 4 . Large language models (LLMs) offer a transformative solution to this administrative burden by automating the synthesis of fragmented patient data. Advanced architectures have demonstrated remarkable capabilities in semantic processing and clinical reasoning. Recent benchmarks even suggest that LLM-generated summaries can rival physician-authored texts in coherence and fluency 5 . Research by Ganzinger et al. 6 and Oliveira et al. 7 confirms that open-source models, when appropriately fine-tuned, can achieve high factual fidelity. However, the current body of literature predominantly operationalizes 'quality' through the lens of correctness. Studies focus primarily on mitigating hallucinations and ensuring data completeness 8 . While these results validate the feasibility of LLMs as drafting tools, they largely treat the clinical summary as a standardized informational output, emphasizing factual inclusion over clinical intent. This perspective overlooks the intrinsic variability in clinical documentation. Consequently, current models yield more generic narratives that, while factually accurate, fail to align with the attending physician’s distinct 'voice' and communicative preferences. The inability of current LLMs to replicate individual writing styles represents a critical yet overlooked barrier to clinical adoption. This challenge shifts the research focus from simple data extraction to the more complex domain of provider-specific adaptation. While modern models are adept at synthesizing facts, they frequently suffer from a 'homogenization effect' which is hypothesized to be a byproduct of standard Reinforcement Learning from Human Feedback (RLHF) strategies that prioritize safety and uniformity over diversity 9 , 10 . This alignment process often homogenizes idiosyncratic vocabulary and professional shorthand, effectively 'bleaching' the nuanced clinical gestalt from the documentation 11 , 12 . This lack of stylistic alignment imposes a secondary editing burden on clinicians 13 . Providers are forced not merely to verify facts, but to actively rewrite outputs to restore the subtle linguistic cues of diagnostic uncertainty and professional judgment—a process that can negate the time-saving benefits of automation 6 . These stylistic markers are not cosmetic; they are functional components of safe patient handovers, often signaling case complexity or urgency to receiving teams. Consequently, closing this gap requires moving beyond generic summarization prompts. It demands bespoke architectures (e.g., provider-specific in-context learning or few-shot style injection 13 , 14 ) that enable the AI to align with the clinician's established professional identity. To address this gap, we propose a novel framework for style-informed clinical summary generation that goes beyond generic fidelity metrics. In this study, we first systematically evaluate the limitations of traditional quantitative similarity metrics (e.g., ROUGE, BLEU, BERTScore) in capturing authorial voice, demonstrating their insufficiency for differentiating between clinician-authored texts. Second, we introduce an LLM-driven feedback loop (Fig. 1) designed to extract high-dimensional qualitative features (e.g. narrative density, syntactic preferences, and hedging strategies) that correlate with authorship identity. Third, we utilize these identified stylistic markers to inform a custom generation pipeline using GPT-4 and Gemini 2.5 Pro, comparing zero-shot and few-shot approaches. Finally, we validate this framework through a blinded A/B preference study with attending clinicians, comparing their historical notes generated a year ago against our style-adapted generations. This work contributes to a foundational pathway for personalized clinical NLP, offering a reproducible methodology to reduce the editing efforts and enhance the clinical utility of automated documentation. 2. Methods 2.1. Study Design and Data Collection We utilized a cross-sectional study design featuring a parallel corpus of clinical summaries to isolate authorial style from clinical content. Ten attending clinicians were recruited to review the charts of 30 unique, de-identified patients. Each clinician independently reviewed the same set of patient charts and authored a clinical summary. This design controlled clinical content while allowing stylistic variation across authors. This yielded a matrix of N = 300 ground-truth summaries, enabling direct "head-to-head" comparison of inter-annotator variability. Our study design comprised four phases (see Fig. 1). The goal of phase 1 of our methods was to extract quantitative features to identify similarities and differences in authorial style. This was done by evaluating a variety of computational similarity metrics on pairs of summaries written by the same or different clinicians. In phase 2 , we used LLMs to explore more nuanced qualitative features, focusing specifically on the linguistic and stylistic elements of clinician writing. To assess Sthe usability of these features, we created an LLM classification task to determine whether pairs of written summaries could be successfully classified as written by the same author or different authors. This ideal set of features was then used in phase 3 to drive stylistically informed LLM-generated hospital discharge summaries. Finally, in phase 4 , we compared our LLM-generated summaries against clinician-written summaries. The available attending clinicians were each presented with 10 clinical reports and asked to indicate their preference for either their self-authored previously written summary or the stylistically informed LLM-generated summary, or both. 2.2. Phase 1: Quantitative Feature Extraction To test the hypothesis that traditional NLP metrics are insufficient for capturing authorial style, we conducted a systematic similarity analysis. We defined two distinct similarity conditions: Intra-Author Similarity ( \({\varvec{S}}_{\varvec{i}\varvec{n}\varvec{t}\varvec{r}\varvec{a}}\) ) : Comparison of summaries written by the same clinician for different patients. High scores here indicate a consistent, template-driven style. Inter-Author Similarity ( \({\varvec{S}}_{\varvec{i}\varvec{n}\varvec{t}\varvec{e}\varvec{r}}\) ) : Comparison of summaries written by different clinicians for the same patient. High scores here indicate that the metric is capturing factual content rather than stylistic nuance. We computed similarity scores, ranging from 0 for no similarity to 1 for complete similarity, using a suite of structural and semantic metrics: Lexical Metrics : Cosine Similarity 15 and Jaccard Index 16 to measure word-level overlap. Character-Level Metrics : Levenshtein Distance 17 , Jaro 18 , and Jaro-Winkler 19 scores to capture formatting and abbreviation variances. Semantic Metrics : ROUGE (1, 2, L) 20 and BLEU 21 to assess n-gram recall/precision, and BERTScore to measure embedding-space semantic equivalence. We refer to lexical and character-level metrics as “simple similarity metrics,” while embedding-based metrics are categorized as “advanced semantic metrics.” We hypothesized that an ideal style-sensitive metric would yield significant divergence between \({\varvec{S}}_{\varvec{i}\varvec{n}\varvec{t}\varvec{r}\varvec{a}}\) and \({\varvec{S}}_{\varvec{i}\varvec{n}\varvec{t}\varvec{e}\varvec{r}}\) , where \({\varvec{S}}_{\varvec{i}\varvec{n}\varvec{t}\varvec{r}\varvec{a}}\) would be high and \({\varvec{S}}_{\varvec{i}\varvec{n}\varvec{t}\varvec{e}\varvec{r}}\) would be low. However, in practical clinical settings, patient conditions and care trajectories differ substantially. As a result, summaries written by the same clinician for different patients are expected to share limited surface similarity, yielding lower \({\varvec{S}}_{\text{intra}}\) . Conversely, summaries written by different clinicians for the same patient necessarily describe the same diagnoses, procedures, and hospital course, leading to higher \({\varvec{S}}_{\text{inter}}\) . Therefore, we tested the effect of applying a targeted lexical masking to the written summaries on the computation of similarity scores. To do this, we tokenized the summaries into individual words and removed predefined sets of words and phrases to reduce the influence of patient-specific context on similarity comparisons. For inter-author similarity, \({\varvec{S}}_{\varvec{i}\varvec{n}\varvec{t}\varvec{e}\varvec{r}}\) , we removed gendered terms (e.g., “male”, “female”), medications, numerical values (e.g., ages, dates), and common basic phrases (e.g., “year old”, “hospitalist”). For intra-author similarity, \({\varvec{S}}_{\varvec{i}\varvec{n}\varvec{t}\varvec{r}\varvec{a}}\) , we removed gendered terms, medications, numerical values, and additionally, patient medical history and symptoms. Because when evaluating \({\varvec{S}}_{\varvec{i}\varvec{n}\varvec{t}\varvec{e}\varvec{r}}\) , doctors’ descriptions of medical history and symptoms may reflect stylistic preference. In contrast, when evaluating \({\varvec{S}}_{\varvec{i}\varvec{n}\varvec{t}\varvec{r}\varvec{a}}\) , differences in medical history and symptoms primarily reflects differences in patient contexts rather than contextual authorial style. To evaluate the effectiveness of each similarity metric and the masking, two hypothesis tests were performed on each metric: Comparing \({\varvec{S}}_{\varvec{i}\varvec{n}\varvec{t}\varvec{e}\varvec{r}}\) and \({\varvec{S}}_{\varvec{i}\varvec{n}\varvec{t}\varvec{r}\varvec{a}}\) before applying any masking, and Comparing \({\varvec{S}}_{\varvec{i}\varvec{n}\varvec{t}\varvec{e}\varvec{r}}\) and \({\varvec{S}}_{\varvec{i}\varvec{n}\varvec{t}\varvec{r}\varvec{a}}\) after applying appropriate masking. Each hypothesis test was performed using a two-sample t-test, and significance was determined if the associated p-value was less than 0.05. Because we aimed to find similarity metrics where \({\varvec{S}}_{\varvec{i}\varvec{n}\varvec{t}\varvec{r}\varvec{a}}\) was greater than \({\varvec{S}}_{\varvec{i}\varvec{n}\varvec{t}\varvec{e}\varvec{r}}\) , we used a one-sided alternative hypothesis (H A : \({\varvec{S}}_{\varvec{i}\varvec{n}\varvec{t}\varvec{e}\varvec{r}}\) < \({\varvec{S}}_{\varvec{i}\varvec{n}\varvec{t}\varvec{r}\varvec{a}}\) ) for each hypothesis test. 2.3. Phase 2: LLM-Driven Feature Extraction Given the hypothesized limitations of quantitative metrics, we developed a qualitative feedback loop using Large Language Models (LLMs) to identify high-dimensional stylistic features (Fig. 1). To isolate the specific linguistic features that define a clinician’s style, we employed a two-stage classification strategy using Large Language Models (LLMs). We considered three possible labels for pairs of summary data: 1) SD : two summaries written by the same author about different patients, 2) DS : two summaries written by different authors about the same patient, and 3) DD : two summaries written by different authors about different patients. Before generating new content, the model was evaluated on its ability to distinguish summary pairs, where “SD” is classified as “same author” and “DS” and “DD” are classified as “different authors” . This was done by computing the overall accuracy and accuracy within each of the three possible summary pairs. Zero-Shot Classification We established a baseline using a Zero-Shot classification approach 22 , 23 , utilizing the entire dataset without prior training examples or task-specific training. The model was presented with paired clinical summaries and given the following instructions "For each of the entry pairs, verify if the two input texts were written by the same author using qualitative, detailed linguistic breakdown. Analyze the writing styles of the input texts, disregarding differences in topic and content. Reasoning based on linguistic features including but not limited to phrasal verbs, modal verbs, rare words, affixes, quantities, tone, and abbreviation usage. Note that some pairs are written by the same author, and some pairs are written by different authors." Few-Shot Classification (Train-Test Split) : To improve classification accuracy, we implemented a few-shot classification approach 24 , 25 using a train-test split. In this setting, the model was first provided with a labelled training set consisting of summary pairs annotated as either authored by the same clinician or by different clinicians. This in-context learning allowed the model to learn corpus-specific stylistic cues before generating predictions on a held-out test set. We evaluated multiple train-test split ratios (100:100, 120:80, 140:60, 160:40). ). The model was given the following instructions: "Take the attached data as the training data set and develop a predictor using qualitative detailed linguistic features to predict whether two input texts are written by the same author. Analyze the writing styles of the input texts, disregarding differences in topic, content, and grammatical errors." For the test set, we evaluated three distinct prompt 26 , 27 endings to determine the optimal level of instruction specificity: Prompt Ending A (Baseline) : An uninformative instruction. "For each input text pair, verify if they were written by the same or different author using the previously developed predictor." Prompt Ending B (Broad Qualitative) : Focused on general linguistic features. "Using the previously developed predictor, verify if the two input texts were written by the same author using qualitative detailed linguistic authorship breakdown. Analyze the writing styles of the input texts, disregarding differences in topic and content and grammatical errors. Reasoning based on linguistic features including sentence and list structure, phrasal verbs, modal verbs, affixes, tone, and abbreviation usage." Prompt Ending C (Targeted Feature) : Narrowed focus based on features identified during preliminary qualitative extraction (e.g., rhythm, narration style). "Using the previously developed predictor, verify if the two input texts were written by the same author using qualitative detailed linguistic authorship breakdown. Analyze the writing styles of the input texts, disregarding differences in topic and content and grammatical errors. Reasoning focused on writing style elements of sentence and list structure, phrasing, tone, rhythm, and narration style." 2.4. Phase 3: Style-Informed Generation Based on the superior performance (see Results 3.2) of the targeted feature extraction (Prompt Ending C) in our testing classification accuracy, we advanced to the generation phase. This workflow utilized a "Train-Generate" paradigm to create new summaries that mimicked a specific provider's style. Early attempts at summary generation led to hallucinations, as clinicians identified fabricated and false clinical information in LLM summaries that contradicted the original text of the clinical summary. To address this issue, we employed a strong and explicit prompting strategy to create an authoritative style guide. This style guide outlined how to capture provider-specific stylistic features and what information the LLM could and couldn’t use in its summary generation. Training Phase (Style Profiling) For each target clinician, we allocated 20 previously authored summaries to a training set for in-context learning. In this phase, GPT-4 and Gemini 2.5 Pro were instructed to analyze these examples to create an authoritative style guide for that specific user. "Use the attached data as a training data set to analyze the writing style of the author writing the summaries. Focus on writing style elements of sentence and list structure, phrasing, tone, rhythm, and narration style. Use this analyzed writing style as the authoritative style guide for generating all future summaries." Generation Phase In the final stage, the model generated new summaries for held-out reports (N = 10 per provider). The prompt included strict safety constraints to prevent hallucinations, explicitly requiring the model to adhere to the previously created authoritative style guide while using only the facts presented in the new report. "Use the identified writing style of this author developed from the training data set to generate a one paragraph stylistically consistent summary for the report below labeled 'Report:'. When generating the summary for the new report, your output must strictly follow the stylistic conventions of the training summaries, including sentence and list structure, phrasing, tone, rhythm, and narration style. Be sure that generated summaries are concise like the original summaries in the training data set and concentrate on the specific content mentioned in the report to make them informative and clinically focused. Do not fabricate or infer any information. Only summarize details explicitly stated in the report. If information is missing, leave it out — do not guess or create new clinical facts." 2.5. Phase 4: Clinician Preference Evaluation (A/B Testing) To validate clinical utility, we conducted a blinded A/B preference study 28 with practicing clinicians. Clinicians were presented with pairs of summaries for cases they had previously documented (one Human-authored, which they had previously written, one GPT-4 or Gemini 2.5 Pro generated). For each pair, clinicians were asked to select a preference: "Prefer Human," "Prefer LLM," or "Looks the Same". Statistical significance was assessed using a one-sample proportion hypothesis test ( \({H}_{0}:p=0.5\) ), treating "Looks the Same" as a neutral outcome weighted equally between the two categories. $$\widehat{p}=\frac{PreferHuman+0.5*LookstheSame}{TotalResponses}$$ This four-phase workflow illustrates the transition from traditional quantitative analysis to personalized clinical documentation. The process begins with Phase 1, evaluating the limitations of standard metrics through lexical masking and similarity testing. Phase 2 uses a feedback loop to identify high-dimensional linguistic markers that traditional metrics miss. Phase 3 implements these markers into a clinician-specific style guide for summary generation, while Phase 4 validates the output through blinded preference testing. 3. Results 3.1. Limitations of Traditional Quantitative Metrics An ideal similarity metric should have high intra-author scores ( \({S}_{intra}\) ) and low inter-author scores \({(S}_{inter})\) . The simple similarity metrics (lexical and character-level) demonstrated a limited ability to isolate authorial style from patient-specific content. As shown in Fig. 2, for the Cosine, Jaccard, and Levenshtein metrics, the inter-author similarity was consistently higher than the intra-author similarity. The high \({(S}_{inter})\) scores indicate that these metrics primarily track shared medical terminology and factual overlap, rather than unique phrasing. Table 1 highlights that Jaro-Winkler (JW) was the only simple metric to achieve statistical significance in both masked ( \(p=4.43e-28\) ) and unmasked ( \(p=5.76e-122\) ) conditions. The relative success of Jaro-Winkler suggests it is more sensitive to "character-level" stylistic habits, such as specific ways of abbreviating or consistent sentence patterns, which remain consistent for an author even when the medical content changes. Figure 2: Effectiveness of Simple Similarity Metrics on Masked and Unmasked Data For each metric, we computed similarity scores between all possible pairs of written summaries in the masked and unmasked case. We used this data to perform hypotheses tests on the difference between \({S}_{inter}\) and \({S}_{intra}\) . Jaro-Winkler abbreviated to JW, Levenshtein abbreviated to LV. Table 1 P-values of Hypothesis Tests on difference of \({\varvec{S}}_{\varvec{i}\varvec{n}\varvec{t}\varvec{e}\varvec{r}}\) and \({\varvec{S}}_{\varvec{i}\varvec{n}\varvec{t}\varvec{r}\varvec{a}}\) for Simple Metrics For each metric, we performed two one-sided t-tests: 1) masked vs. masked and 2) unmasked vs. unmasked to evaluate if there is a significant difference between and. The p-values for these hypothesis tests are reported in the table. Cosine Masked Unmasked 1 1 Jaccard 1 1 Levenshtein 1 1 Jaro 0.846 0.318 Jaro-Winkler 4.43e-28 5.76e-122 The advanced semantic metrics (ROUGE, BLEU, BERTScore) were expected to better capture "clinical intent," yet they largely mirrored the failures of lexical metrics. As illustrated in Fig. 3, ROUGE-1, ROUGE-L, and BERTScore failed to produce a significant difference between \({S}_{intra}\) and \({S}_{inter}\) . In fact, the \(pvalues\) for ROUGE-1 and ROUGE-L were 1.0, indicating that these metrics computed higher scores for \({S}_{inter}\) than \({S}_{intra}\) , the exact opposite of what we wanted to capture in our similarity metrics. Interestingly, applying targeted lexical masking (removing medications, dates, and gendered terms) did not made substantial change in these metrics. Even after removing patient-specific facts, the BERTScore embeddings still gravitated toward the shared medical narrative rather than the authorial voice. Figure 3: Effect of Masking on Advanced Similarity Metrics For each metric, we computed similarity scores between all possible pairs of written summaries in the masked and unmasked cases. We used this data to perform hypothesis tests on the difference between \({S}_{inter}\) and \({S}_{intra}\) . BERTScore is abbreviated to BERT. Table 2 shows that BLEU achieved statistical significance ( \(p=2.55e-16\) ) only in the unmasked state. This suggests that BLEU’s sensitivity to specific n-gram sequences is heavily tied to the raw clinical data, and once that data is masked, its ability to detect authorial voice vanishes. Table 2 P-values of Hypothesis Tests on the difference of \({\varvec{S}}_{\varvec{i}\varvec{n}\varvec{t}\varvec{e}\varvec{r}}\) and \({\varvec{S}}_{\varvec{i}\varvec{n}\varvec{t}\varvec{r}\varvec{a}}\) for Advanced Metrics For each metric, we performed two one-sided t-tests: 1) masked vs. masked and 2) unmasked vs. unmasked to evaluate if there is a significant difference between and. The p-values for these hypotheses tests are reported in the table. BERTScore Masked Unmasked 0.177 1 BLEU 0.304 2.55e-16 Rouge1 1 1 Rouge2 0.980 1 RougeL 1 1 3.2. Efficacy of LLM-Driven Feature Extraction Transitioning to qualitative feature extraction using LLMs yielded a marked improvement in authorship attribution, validating the hypothesis that style is better defined by high-dimensional linguistic features than by n-gram overlap. Zero-Shot Baseline : As shown in Table 3 , the Zero-Shot baseline struggled significantly, with overall accuracy fluctuating between 44.67% and 62%. Without specific stylistic training, the model often performed only slightly better than a random guess when trying to identify if two summaries were written by the same author. The model showed extreme variance in identifying "Same Author" (SD) versus "Different Author" (DS/DD) pairs across different trials. In some trials, it over-identified same-author pairs, while in others, it did the opposite, indicating a lack of a stable internal stylistic capture. Few-Shot & Prompt Engineering : Results changed dramatically with the introduction of these methods. Specifically, Prompt Ending C which focused on rhythm, narration, and sentence/list structure, achieved the highest performance. Utilizing a 100:100 train-test split, the model reached an overall accuracy of 73%. Most notably, this approach achieved 80% accuracy in correctly identifying "Same Author" (SD) pairs. This suggests that once the LLM was instructed to look for specific high-dimensional markers like narrative rhythm and hedging strategies, it could successfully recognize the professional identity of the clinician. Table 3 Performance of Zero-Shot LLM Author Attribution This table displays the baseline classification results across six independent trials using a zero-shot approach without prior stylistic training. Accuracy scores represent the model's ability to distinguish between same-author pairs (SD) and different-author pairs (DS and DD). The results highlight the high variance and limited reliability of zero-shot models in identifying authorial signatures without specific training examples. Each row represents a trial with 150 total summary pairs, with 50 from each label. Overall (%) SD Label (%) DS Label (%) DD Label (%) 44.67 78 30 26 44.67 62 32 40 55.33 60 38 68 58 48 54 72 61.33 28 78 78 62 18 82 86 Table 4 Efficacy of Few-Shot Learning and Targeted Prompting This table compares the classification accuracy of three distinct prompt specificity levels using a 100:100 train-test split. The data demonstrates the significant performance improvement achieved through few-shot learning, particularly with Prompt 3 (Targeted Feature Extraction). This specific prompt reached a peak overall accuracy of 73% and an 80% success rate in identifying clinician-specific signatures, validating the use of high-dimensional linguistic markers for style replication. Each row represents a trial with 200 total summary pairs with 100 SD, 50 DS, and 50 DD. Prompt Overall (%) SD Label (%) DS Label (%) DD Label (%) 1 46 46 48 35 2 47 44 52 50 3 73 80 60 65 3.3. Clinician Preference and Style Replication (A/B Testing) The ultimate validation of the framework was the blinded A/B preference study. Results indicated a performance divergence between model architectures (GPT-4 vs. Gemini 2.5 Pro), with Gemini demonstrating stronger style replication capabilities. GPT-4 Performance : In the GPT-4 arm, clinicians retained a preference for human-authored summaries. Clinicians preferred the Human draft in 57 responses compared to 24 responses for the LLM. With a calculated \(p\) -value of 1.0 (Table 5 ) and a preference proportion \(\left(p\right)\) of 0.6833, the results failed to achieve statistical significance, indicating that GPT-4 did not sufficiently capture the unique nuances of the provider's style. Gemini 2.5 Pro Performance : The Gemini model demonstrated superior style alignment. Clinicians showed a slight preference for the LLM-generated draft (43 responses) over their own historical notes (35 responses), with 12 responses rated as indistinguishable. The hypothesis test yielded a p-value of 0.4606 for Gemini. This suggests that the Gemini-based pipeline, informed by the authorial voice, generated summaries that were noninferior to the clinician’s original documentation. In some cases, the LLM generated clinical summaries were preferred over the clinician-authored notes. However, these differences did not reach statistical significance. This comparative visualization displays the results of the blinded preference study for both model architectures. The data indicate the number of responses in which clinicians preferred their original human-authored notes, the LLM-generated drafts, or found the two to be indistinguishable. The results show a notable performance divergence, with Gemini 2.5 Pro achieving a higher rate of style alignment and clinician preference than GPT-4. Table 5 Statistical Significance of Clinician Preferences This table reports the outcomes of the one-sample proportion hypothesis test used to evaluate model performance. The p-values indicate whether the observed preference for a draft significantly deviated from the neutral baseline of 0.5. Results show that while GPT-4 results favored human drafts, the Gemini 2.5 Pro results indicated non-inferiority, as the preference difference was not statistically significant. LLM \(\widehat{\varvec{p}}\) p-value GPT-4 0.6833 1 Gemini 2.5 Pro 0.4556 0.4606 4. Discussion Our findings challenge the prevailing assumption in medical NLP that factual correctness is the sole proxy for clinical utility. While prior studies demonstrate that LLMs achieve high semantic fidelity, our quantitative analysis reveals that traditional metrics—such as ROUGE, BLEU, and BERTScore—are fundamentally ill-suited for measuring stylistic concordance. As shown in our "Masking Effect" analysis, these metrics failed to reliably distinguish between clinical summaries written by different clinicians for the same patient. This confirms that high n-gram overlap captures the content of the medical case but fundamentally misses the style of the provider. For the clinical community, this implies that a high-scoring AI summary may still feel alien to the signing clinician, effectively "bleaching" the nuanced clinical gestalt and professional identity from the document. This perpetuates the editing load as providers are forced to rewrite accurate but tonally mismatched drafts. These findings suggest that stylistic personalization may represent a critical next step in clinical documentation automation. To overcome the "homogenization effect" typical of RLHF-aligned models—which prioritize uniform, safe outputs over diverse authorial voices—this study introduces a Style-Informed Generation Framework . By shifting from zero-shot prompting to a "Train-Generate" pipeline, we successfully extracted high-dimensional features (such as narrative rhythm and hedging strategies) that elude traditional statistical detection methods. The efficacy of this approach was evidenced by a jump in authorship attribution accuracy to 73% with targeted feature extraction, compared to the near-random performance of zero-shot baselines. When the LLM is explicitly constrained to a stylistic profile derived from historical data, it begins to convey the authorial voice of the care team, producing drafts that are often indistinguishable from human writing. A critical finding of this study is the divergence in performance between model architectures. While GPT-4 produced fluent text, clinicians largely preferred their own human-written notes. In contrast, Gemini 2.5 Pro appeared more responsive to style-profile constraints in this experimental setting, with 43 clinician preferences compared to 35 for human drafts. This variance likely stems from how each model weights long-context instructions versus its underlying safety alignment. Gemini’s ability to generate "non-inferior" summaries suggests it may be better suited for applications requiring high-fidelity style transfer in-context learning. Limitations While our framework mitigated hallucinations through strict prompt constraints, the safety-critical nature of clinical documentation requires ongoing verification. The small sample size (N = 10) and the use of static ("frozen") Style Profiles limited an immediate generalizability. Future research should expand the cohort size beyond ten clinicians to improve the generalizability of these findings. The current approach should transition to dynamic models that evolve in real-time as a clinician's documentation style changes over their career. Additionally, investigations into architectural differences are needed to determine why specific models, such as Gemini 2.5 Pro, demonstrate superior adaptability to in-context style constraints. Finally, future studies should employ downstream impact metrics such as clinician vs LLM time difference to provide a direct quantitative measure of how much the editing load is reduced in active clinical workflows. Conclusion The integration of Large Language Models into electronic health records offers a path to mitigate clinician burnout, but only if the technology aligns with users' professional identity. We demonstrate that while current models are factually capable, they face a "Style Gap" that traditional metrics fail to detect. By implementing a Style-Informed Feedback Loop, we provide a reproducible proof-of-concept for personalized clinical documentation. This approach moves the field closer to an AI assistant that not only "summarize," but also aligns with the stylistic conventions of the clinical author, ultimately reducing administrative burden and allowing clinicians to return to patient care. Declarations Human Ethics and Consent to Participate This project was approved by the Wake Forest University School of Medicine Research Ethics Board Protocol review board approval (IRB00127840) in accordance with the Declaration of Helsinki. All the records in this study were de-identified. Clinical trial number not applicable. Conflict of Interest: The authors have declared that no competing interests exist. Competing interests: The authors declare no competing interests. Funding: This work was supported by the National Institutes of Health (NIH) / National Center for Advancing Translational Sciences (NCATS) under award number U01 TR003629 , titled 'Analytics & Machine-learning for Maternal-health Interventions (AMMI): A Cross-CTSA Collaboration'. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. Author Contribution - **Scott Zhao** : Software; formal analysis; data curation; writing – original draft.- **Abbas Alili** : Supervision; project administration; methodology; writing – original draft; writing – review and editing.- **Usman Afzaal** : Data curation; software; validation.- **Hao Lu** : Methodology; software; writing review.- **Muhammet F. Demir** : Investigation; validation; data curation.- **Padageshwar Sunkara** : Conceptualization; writing – original draft; investigation; clinical validation.- **Metin N. Gurcan** : Conceptualization; senior supervision; funding acquisition; resources; writing – review and editing. Acknowledgement We are deeply grateful to Raghava Nagaraj, Suneel Kumar Parvathareddy, Sumera Andleeb, William B. Winfrey, Nicolas Haller, Vani Khajuria, Megan Jodrey, Sneha Chebrolu, Katherine Rose Sommers, Marc Holden Perlman, Harsh Barot, and Kinchit K. Shah for their invaluable support and contributions to this work. Their clinical perspectives greatly strengthened the quality and clarity of the research.Informed consent was obtained from all the experts involved in the study. Data Availability The datasets generated and analyzed during the current study are not publicly available due to IRB restrictions at Wake Forest University School of Medicine (IRB00127840) regarding the protection of sensitive clinical data. However, de-identified data can be made available from the corresponding author upon reasonable request and following the execution of a formal Data Use Agreement (DUA) to ensure ethical and legal compliance. References Kind, A. J. H. & SMA. &. Documentation of Mandated Discharge Summary Components in Transitions from Acute to Subacute Care. Advances in Patient Safety: New Directions and Alternative Approaches (Vol 2: Culture and Redesign) . Published online 2008. Van Walraven, C., Seth, R., Austin, P. C. & Laupacis, A. Effect of Discharge Summary Availability During Post-Discharge Visits on Hospital Readmission . CvW. Li, Y. et al. A comparative study of recent large language models on generating hospital discharge summaries for lung cancer patients. J. Biomed. Inf. 168 10.1016/j.jbi.2025.104867 (2025). Wu, Y. et al. Evaluating the Prevalence of Burnout Among Health Care Professionals Related to Electronic Health Record Use: Systematic Review and Meta-Analysis. JMIR Med. Inform JMIR Publications Inc . 12 10.2196/54811 (2024). Williams, C. Y. K. et al. Physician- and Large Language Model-Generated Hospital Discharge Summaries. JAMA Intern. Med Published online 2025. 10.1001/jamainternmed.2025.0821 Ganzinger, M. et al. Automated generation of discharge summaries: leveraging large language models with clinical data. Sci. Rep. 15 (1). 10.1038/s41598-025-01618-7 (2025). Oliveira, J. D. et al. Development and evaluation of a clinical note summarization system using large language models. Commun. Med. 5 (1). 10.1038/s43856-025-01091-3 (2025). Koh, M. C. Y. et al. Using ChatGPT for writing hospital inpatient discharge summaries – perspectives from an inpatient infectious diseases service. BMC Health Serv. Res. 25 (1). 10.1186/s12913-025-12373-w (2025). Sourati, Z., Ziabari, A. S. & Dehghani, M. The Homogenizing Effect of Large Language Models on Human Expression and Thought. Published online August 2, (2025). http://arxiv.org/abs/2508.01491 Kirk, R. et al. Understanding the Effects of RLHF on LLM Generalisation and Diversity . Hains, L. et al. Large language model discharge summary preparation using real-world electronic medical record data shows promise. Intern. Med. J. 55 (7), 1188–1192. 10.1111/imj.70073 (2025). Satheakeerthy, S., Jesudason, D., Pietris, J., Bacchi, S. & Chan, W. O. LLM-assisted medical documentation: efficacy, errors, and ethical considerations in ophthalmology. Eye (Basingstoke) Springer Nature . 39 (8), 1440–1442. 10.1038/s41433-025-03767-5 (2025). Chua, C. E. et al. Integration of customised LLM for discharge summary generation in real-world clinical settings: a pilot study on RUSSELL GPT. Lancet Reg. Health West. Pac Elsevier Ltd . 51 10.1016/j.lanwpc.2024.101211 (2024). Alqahtani, M., Al-Barakati, A., Alotaibi, F., Al Shibli, M. & Almousa, S. Impact of Detailed Versus Generic Instructions on Fine-Tuned Language Models for Patient Discharge Instructions Generation: Comparative Statistical Analysis. JMIR Form. Res. 9 10.2196/80917 (2025). Orkphol, K. & Yang, W. Word Sense Disambiguation Using Cosine Similarity Collaborates with Word2vec and WordNet. Futur Internet 2019 . 11 (Page 114 11), 114 (2019). Chauvin, L., Kumar, K., Desrosiers, C., Wells, W. & Toews, M. Efficient Pairwise Neuroimage Analysis Using the Soft Jaccard Index and 3D Keypoint Sets. IEEE Trans. Med. Imaging . 41 (4), 836–845. 10.1109/TMI.2021.3123252 (2022). Epub 2022 Apr 1. PMID: 34699353; PMCID: PMC9022638. Faes, J. & Gillis, S. Speech production accuracy of children with auditory brainstem implants: A comparison with peers with cochlear implants and typical hearing using Levenshtein Distance. Rozinek, O. & Mareš, J. Fast and Precise Convolutional Jaro and Jaro-Winkler Similarity, 2024 35th Conference of Open Innovations Association (FRUCT), Tampere, Finland, 2024, pp. 604–613. 10.23919/FRUCT61870.2024.10516360 Xia, K. Design and application of efficient English learning system based on Jaro-Winkler. J. Comput. Methods Sci. Eng. 10.1177/14727978251371179 (2025). Lin, C. Y. ROUGE: A Package for Automatic Evaluation of Summaries. Papineni, K., Roukos, S., Ward, T. & Zhu, W. J. BLEU: a Method for Automatic Evaluation of Machine Translation. Austrian, J. et al. Applying A/B Testing to Clinical Decision Support: Rapid Randomized Controlled Trials. J. Med. Internet Res. 23 (4), e16651. 10.2196/16651 (2021). PMID: 33835035; PMCID: PMC8065554. Pre-trained Language Models Can be Fully Zero-Shot Learners]( https://aclanthology.org/2023.acl-long.869/ ) (Zhao et al., ACL 2023). Yan, L., Zheng, Y. & Cao, J. Few-shot learning for short text classification. Multimed Tools Appl. 77 , 29799–29810. https://doi.org/10.1007/s11042-018-5772-4 (2018). Yisheng Song, T., Wang, P., Cai, S. K., Mondal & Jyoti Prakash, S. A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities. ACM Comput. Surv. 55 , 40pages. https://doi.org/10.1145/3582688 (2023). 13s, Article 271 (December 2023. Lee, Y. et al. Prompt engineering in ChatGPT for literature review: practical guide exemplified with studies on white phosphors. Sci. Rep. 15 , 15310. https://doi.org/10.1038/s41598-025-99423-9 (2025). Prompting science report 1: Prompt engineering is complicated and contingent L Meincke, Mollick, E. & Mollick, L. D Shapiro arXiv preprint arXiv:2503.04818, 2025•arxiv.org. Zhang, Z. et al. A zero-shot prompt learning approach on fine-grained text classification. Sci. Rep. 16 , 5260. https://doi.org/10.1038/s41598-025-34825-3 (2026). Additional Declarations No competing interests reported. Cite Share Download PDF Status: Under Review Version 1 posted Reviews received at journal 19 Apr, 2026 Reviewers agreed at journal 29 Mar, 2026 Reviewers agreed at journal 29 Mar, 2026 Reviewers agreed at journal 25 Mar, 2026 Reviewers invited by journal 24 Mar, 2026 Editor assigned by journal 24 Mar, 2026 Editor invited by journal 17 Mar, 2026 Submission checks completed at journal 16 Mar, 2026 First submitted to journal 16 Mar, 2026 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-9054955","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":612232642,"identity":"8c63c279-4710-4309-85f9-2118b4730601","order_by":0,"name":"Scott Zhao","email":"","orcid":"","institution":"Wake Forest University","correspondingAuthor":false,"prefix":"","firstName":"Scott","middleName":"","lastName":"Zhao","suffix":""},{"id":612232645,"identity":"4b4506a2-0a41-4150-b28c-815252ddbfe0","order_by":1,"name":"Abbas Alili","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABMklEQVRIie3QvWqDUBTA8SsXrsuxrrco5hUUoUtD+iqVC5kc2iUIKUQJXJekXQN9iTxCRWiW1K6GZmhwzRK62E/qx2hMOxZ6/4tw8Me5HIREor8YheqjE1kKnhGph7icIHqYAMHS2KwJrgjAT6T4jdBfEfV2Gu1y1IMjjLmXD2JDDR+j7MLrwZnmS5n31lyyTpgGiJUP46tJEtt0ybA9WzIA/Q5bD9fNNalrasVbKpIqvO/MESOawnHxsHNyHEwaopO69muORhW5/CyJmsnvyteolZipe1JcIK4IVnjXmVNGsOLHNfHzBrHWSf8UzEV1ZE1PujZNM1uD+wVA6oytwG8Q42kar3JvaHTCcLPbDqih3jibF7gaGvKMRRv/Y/+lkbl/LPlI4i3kQG1bRCKR6B/1DfaJXWNubD6yAAAAAElFTkSuQmCC","orcid":"","institution":"Wake Forest University","correspondingAuthor":true,"prefix":"","firstName":"Abbas","middleName":"","lastName":"Alili","suffix":""},{"id":612232648,"identity":"14c9c2bf-f54f-4d91-8b8c-ca51d6189a55","order_by":2,"name":"Usman Afzaal","email":"","orcid":"","institution":"The Ohio State University","correspondingAuthor":false,"prefix":"","firstName":"Usman","middleName":"","lastName":"Afzaal","suffix":""},{"id":612232650,"identity":"57f6bae3-43a8-4bc5-a4cc-256783631e24","order_by":3,"name":"Muhammet F. Demir","email":"","orcid":"","institution":"Wake Forest University","correspondingAuthor":false,"prefix":"","firstName":"Muhammet","middleName":"F.","lastName":"Demir","suffix":""},{"id":612232651,"identity":"57c17394-9c1f-4870-870e-5ca941f881f5","order_by":4,"name":"Hao Lu","email":"","orcid":"","institution":"Wake Forest University","correspondingAuthor":false,"prefix":"","firstName":"Hao","middleName":"","lastName":"Lu","suffix":""},{"id":612232660,"identity":"dbc33e53-0205-4f5c-9a9b-647490637b4d","order_by":5,"name":"Padageshwar Sunkara","email":"","orcid":"","institution":"Wake Forest University","correspondingAuthor":false,"prefix":"","firstName":"Padageshwar","middleName":"","lastName":"Sunkara","suffix":""},{"id":612232667,"identity":"911866ac-2c39-459e-8ebc-9b739978a6f2","order_by":6,"name":"Metin N. Gurcan","email":"","orcid":"","institution":"Wake Forest University","correspondingAuthor":false,"prefix":"","firstName":"Metin","middleName":"N.","lastName":"Gurcan","suffix":""}],"badges":[],"createdAt":"2026-03-07 03:09:05","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-9054955/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-9054955/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":105476796,"identity":"cb99f210-5eaa-4495-b255-4474da214dd8","added_by":"auto","created_at":"2026-03-26 13:00:04","extension":"jpg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":822716,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eThe Style-Informed Generation Framework\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis four-phase workflow illustrates the transition from traditional quantitative analysis to personalized clinical documentation. The process begins with Phase 1, evaluating the limitations of standard metrics through lexical masking and similarity testing. Phase 2 uses a feedback loop to identify high-dimensional linguistic markers that traditional metrics miss. Phase 3 implements these markers into a clinician-specific style guide for summary generation, while Phase 4 validates the output through blinded preference testing.\u003c/p\u003e","description":"","filename":"1.jpg","url":"https://assets-eu.researchsquare.com/files/rs-9054955/v1/2c6c466e6058c5dc163d6329.jpg"},{"id":105476795,"identity":"b9de669d-f7ca-4e6c-9f1d-f20a24992947","added_by":"auto","created_at":"2026-03-26 13:00:03","extension":"jpg","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":69225,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eEffectiveness of Simple Similarity Metrics on Masked and Unmasked Data\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFor each metric, we computed similarity scores between all possible pairs of written summaries in the masked and unmasked case. We used this data to perform hypotheses tests on the difference between S\u003c/strong\u003e\u003csub\u003e\u003cstrong\u003einter\u003c/strong\u003e\u003c/sub\u003e\u003cstrong\u003e and S\u003c/strong\u003e\u003csub\u003e\u003cstrong\u003eintra\u003c/strong\u003e\u003c/sub\u003e\u003cstrong\u003e. \u0026nbsp;Jaro-Winkler abbreviated to JW, Levenshtein abbreviated to LV.\u003c/strong\u003e\u003c/p\u003e","description":"","filename":"2.jpg","url":"https://assets-eu.researchsquare.com/files/rs-9054955/v1/0d60e430ecbf19418b181702.jpg"},{"id":105476797,"identity":"884eda5c-5f78-423c-8050-95572511c66d","added_by":"auto","created_at":"2026-03-26 13:00:04","extension":"jpg","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":65240,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eEffect of Masking on Advanced Similarity Metrics\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFor each metric, we computed similarity scores between all possible pairs of written summaries in the masked and unmasked cases. We used this data to perform hypothesis tests on the difference between S\u003c/strong\u003e\u003csub\u003e\u003cstrong\u003einter\u003c/strong\u003e\u003c/sub\u003e\u003cstrong\u003e and S\u003c/strong\u003e\u003csub\u003e\u003cstrong\u003eintra\u003c/strong\u003e\u003c/sub\u003e\u003cstrong\u003e. \u0026nbsp;BERTScore is abbreviated to BERT.\u003c/strong\u003e\u003c/p\u003e","description":"","filename":"3.jpg","url":"https://assets-eu.researchsquare.com/files/rs-9054955/v1/cfca5a71446044988ffe9115.jpg"},{"id":105476798,"identity":"772eeea2-283e-4d32-91cf-705c1aaccc0e","added_by":"auto","created_at":"2026-03-26 13:00:04","extension":"jpg","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":45539,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eDistribution of Clinician Preferences in A/B Testing\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis comparative visualization displays the results of the blinded preference study for both model architectures. The data indicate the number of responses in which clinicians preferred their original human-authored notes, the LLM-generated drafts, or found the two to be indistinguishable. The results show a notable performance divergence, with Gemini 2.5 Pro achieving a higher rate of style alignment and clinician preference than GPT-4.\u003c/p\u003e","description":"","filename":"4.jpg","url":"https://assets-eu.researchsquare.com/files/rs-9054955/v1/6863ccb721c99a83b553f7bc.jpg"},{"id":105476826,"identity":"c3664d90-f4fe-46ea-bf41-4b47d2d1889c","added_by":"auto","created_at":"2026-03-26 13:00:17","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":2620372,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-9054955/v1/1dc66aff-c322-4bff-8410-46638f643ecb.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Decoding Clinician Authorial Style: A Style-Informed Pipeline for Clinical Document Summary Generation with Large Language Models","fulltext":[{"header":"1. Introduction","content":"\u003cp\u003eThe hospital clinical summary serves as the primary clinical bridge for patient handoffs, acting as the definitive communication tool to ensure continuity of care between treatment teams and reduce the risk of avoidable readmissions\u003csup\u003e\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e,\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e\u003c/sup\u003e. During the precarious transition period covering the first 72 hours following a hospital stay, patients are uniquely vulnerable to adverse drug events and fragmented follow-up care due to a lack of or poor communication. This vulnerability comes due to the complexity of medical interventions and frequent transitions between care settings\u003csup\u003e\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e\u003c/sup\u003e. Ensuring the precision of this documentation is vital for patient safety. However, this necessity creates a significant administrative load on healthcare providers. In practice, generating a single clinical summary is a demanding process, requiring an average of 8.1 minutes for dictation and a median of 29.2 minutes for transcription and editing, often resulting in documents several pages in length\u003csup\u003e\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e\u003c/sup\u003e. This substantial time investment increases clinicians' cognitive load, fueling documentation-related burnout and diverting essential time from direct patient care\u003csup\u003e\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e\u003c/sup\u003e.\u003c/p\u003e \u003cp\u003eLarge language models (LLMs) offer a transformative solution to this administrative burden by automating the synthesis of fragmented patient data. Advanced architectures have demonstrated remarkable capabilities in semantic processing and clinical reasoning. Recent benchmarks even suggest that LLM-generated summaries can rival physician-authored texts in coherence and fluency\u003csup\u003e\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e\u003c/sup\u003e. Research by Ganzinger et al.\u003csup\u003e6\u003c/sup\u003e and Oliveira et al.\u003csup\u003e7\u003c/sup\u003e confirms that open-source models, when appropriately fine-tuned, can achieve high factual fidelity. However, the current body of literature predominantly operationalizes 'quality' through the lens of correctness. Studies focus primarily on mitigating hallucinations and ensuring data completeness\u003csup\u003e\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e\u003c/sup\u003e. While these results validate the feasibility of LLMs as drafting tools, they largely treat the clinical summary as a standardized informational output, emphasizing factual inclusion over clinical intent. This perspective overlooks the intrinsic variability in clinical documentation. Consequently, current models yield more generic narratives that, while factually accurate, fail to align with the attending physician\u0026rsquo;s distinct 'voice' and communicative preferences.\u003c/p\u003e \u003cp\u003eThe inability of current LLMs to replicate individual writing styles represents a critical yet overlooked barrier to clinical adoption. This challenge shifts the research focus from simple data extraction to the more complex domain of provider-specific adaptation. While modern models are adept at synthesizing facts, they frequently suffer from a 'homogenization effect' which is hypothesized to be a byproduct of standard Reinforcement Learning from Human Feedback (RLHF) strategies that prioritize safety and uniformity over diversity\u003csup\u003e\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e,\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e\u003c/sup\u003e. This alignment process often homogenizes idiosyncratic vocabulary and professional shorthand, effectively 'bleaching' the nuanced clinical gestalt from the documentation\u003csup\u003e\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e,\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e\u003c/sup\u003e.\u003c/p\u003e \u003cp\u003eThis lack of stylistic alignment imposes a secondary editing burden on clinicians\u003csup\u003e\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e\u003c/sup\u003e. Providers are forced not merely to verify facts, but to actively rewrite outputs to restore the subtle linguistic cues of diagnostic uncertainty and professional judgment\u0026mdash;a process that can negate the time-saving benefits of automation\u003csup\u003e\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e\u003c/sup\u003e. These stylistic markers are not cosmetic; they are functional components of safe patient handovers, often signaling case complexity or urgency to receiving teams. Consequently, closing this gap requires moving beyond generic summarization prompts. It demands bespoke architectures (e.g., provider-specific in-context learning or few-shot style injection\u003csup\u003e\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e,\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e\u003c/sup\u003e) that enable the AI to align with the clinician's established professional identity.\u003c/p\u003e \u003cp\u003eTo address this gap, we propose a novel framework for \u003cb\u003estyle-informed clinical summary generation\u003c/b\u003e that goes beyond generic fidelity metrics. In this study, we first systematically evaluate the limitations of traditional quantitative similarity metrics (e.g., ROUGE, BLEU, BERTScore) in capturing authorial voice, demonstrating their insufficiency for differentiating between clinician-authored texts. Second, we introduce an \u003cb\u003eLLM-driven feedback loop\u003c/b\u003e (Fig.\u0026nbsp;1) designed to extract high-dimensional qualitative features (e.g. narrative density, syntactic preferences, and hedging strategies) that correlate with authorship identity. Third, we utilize these identified stylistic markers to inform a custom generation pipeline using GPT-4 and Gemini 2.5 Pro, comparing zero-shot and few-shot approaches. Finally, we validate this framework through a blinded A/B preference study with attending clinicians, comparing their historical notes generated a year ago against our style-adapted generations. This work contributes to a foundational pathway for personalized clinical NLP, offering a reproducible methodology to reduce the editing efforts and enhance the clinical utility of automated documentation.\u003c/p\u003e"},{"header":"2. Methods","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003e2.1. Study Design and Data Collection\u003c/h2\u003e \u003cp\u003eWe utilized a cross-sectional study design featuring a parallel corpus of clinical summaries to isolate authorial style from clinical content. Ten attending clinicians were recruited to review the charts of 30 unique, de-identified patients. Each clinician independently reviewed the same set of patient charts and authored a clinical summary. This design controlled clinical content while allowing stylistic variation across authors. This yielded a matrix of N\u0026thinsp;=\u0026thinsp;300 ground-truth summaries, enabling direct \"head-to-head\" comparison of inter-annotator variability.\u003c/p\u003e \u003cp\u003eOur study design comprised four phases (see Fig.\u0026nbsp;1). The goal of \u003cem\u003ephase 1\u003c/em\u003e of our methods was to extract quantitative features to identify similarities and differences in authorial style. This was done by evaluating a variety of computational similarity metrics on pairs of summaries written by the same or different clinicians. In \u003cem\u003ephase 2\u003c/em\u003e, we used LLMs to explore more nuanced qualitative features, focusing specifically on the linguistic and stylistic elements of clinician writing. To assess Sthe usability of these features, we created an LLM classification task to determine whether pairs of written summaries could be successfully classified as written by the same author or different authors. This ideal set of features was then used in \u003cem\u003ephase 3\u003c/em\u003e to drive stylistically informed LLM-generated hospital discharge summaries. Finally, in \u003cem\u003ephase 4\u003c/em\u003e, we compared our LLM-generated summaries against clinician-written summaries. The available attending clinicians were each presented with 10 clinical reports and asked to indicate their preference for either their self-authored previously written summary or the stylistically informed LLM-generated summary, or both.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec4\" class=\"Section2\"\u003e \u003ch2\u003e2.2. Phase 1: Quantitative Feature Extraction\u003c/h2\u003e \u003cp\u003eTo test the hypothesis that traditional NLP metrics are insufficient for capturing authorial style, we conducted a systematic similarity analysis. We defined two distinct similarity conditions:\u003c/p\u003e \u003cp\u003e \u003col\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003e \u003cb\u003eIntra-Author Similarity (\u003c/b\u003e \u003cspan class=\"InlineEquation\"\u003e \u003cspan class=\"mathinline\"\u003e\\({\\varvec{S}}_{\\varvec{i}\\varvec{n}\\varvec{t}\\varvec{r}\\varvec{a}}\\)\u003c/span\u003e \u003c/span\u003e \u003cb\u003e)\u003c/b\u003e: Comparison of summaries written by the \u003cem\u003esame\u003c/em\u003e clinician for \u003cem\u003edifferent\u003c/em\u003e patients. High scores here indicate a consistent, template-driven style.\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003e \u003cb\u003eInter-Author Similarity (\u003c/b\u003e \u003cspan class=\"InlineEquation\"\u003e \u003cspan class=\"mathinline\"\u003e\\({\\varvec{S}}_{\\varvec{i}\\varvec{n}\\varvec{t}\\varvec{e}\\varvec{r}}\\)\u003c/span\u003e \u003c/span\u003e \u003cb\u003e)\u003c/b\u003e: Comparison of summaries written by \u003cem\u003edifferent\u003c/em\u003e clinicians for the \u003cem\u003esame\u003c/em\u003e patient. High scores here indicate that the metric is capturing factual content rather than stylistic nuance.\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003c/ol\u003e \u003c/p\u003e \u003cp\u003eWe computed similarity scores, ranging from 0 for no similarity to 1 for complete similarity, using a suite of structural and semantic metrics:\u003c/p\u003e \u003cp\u003e \u003cul\u003e \u003cli\u003e \u003cp\u003e \u003cb\u003eLexical Metrics\u003c/b\u003e: Cosine Similarity\u003csup\u003e\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e\u003c/sup\u003e and Jaccard Index\u003csup\u003e\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e\u003c/sup\u003e to measure word-level overlap.\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003e \u003cb\u003eCharacter-Level Metrics\u003c/b\u003e: Levenshtein Distance\u003csup\u003e\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e\u003c/sup\u003e, Jaro\u003csup\u003e\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e\u003c/sup\u003e, and Jaro-Winkler\u003csup\u003e\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e\u003c/sup\u003e scores to capture formatting and abbreviation variances.\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003e \u003cb\u003eSemantic Metrics\u003c/b\u003e: ROUGE (1, 2, L)\u003csup\u003e20\u003c/sup\u003e and BLEU\u003csup\u003e\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e\u003c/sup\u003e to assess n-gram recall/precision, and BERTScore to measure embedding-space semantic equivalence.\u003c/p\u003e \u003c/li\u003e \u003c/ul\u003e \u003c/p\u003e \u003cp\u003eWe refer to lexical and character-level metrics as \u003cb\u003e\u0026ldquo;simple similarity metrics,\u0026rdquo;\u003c/b\u003e while embedding-based metrics are categorized as \u003cb\u003e\u0026ldquo;advanced semantic metrics.\u0026rdquo;\u003c/b\u003e We hypothesized that an ideal style-sensitive metric would yield significant divergence between \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\varvec{S}}_{\\varvec{i}\\varvec{n}\\varvec{t}\\varvec{r}\\varvec{a}}\\)\u003c/span\u003e\u003c/span\u003e and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\varvec{S}}_{\\varvec{i}\\varvec{n}\\varvec{t}\\varvec{e}\\varvec{r}}\\)\u003c/span\u003e\u003c/span\u003e, where \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\varvec{S}}_{\\varvec{i}\\varvec{n}\\varvec{t}\\varvec{r}\\varvec{a}}\\)\u003c/span\u003e\u003c/span\u003e would be high and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\varvec{S}}_{\\varvec{i}\\varvec{n}\\varvec{t}\\varvec{e}\\varvec{r}}\\)\u003c/span\u003e\u003c/span\u003e would be low.\u003c/p\u003e \u003cp\u003eHowever, in practical clinical settings, patient conditions and care trajectories differ substantially. As a result, summaries written by the same clinician for different patients are expected to share limited surface similarity, yielding lower \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\varvec{S}}_{\\text{intra}}\\)\u003c/span\u003e\u003c/span\u003e. Conversely, summaries written by different clinicians for the same patient necessarily describe the same diagnoses, procedures, and hospital course, leading to higher \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\varvec{S}}_{\\text{inter}}\\)\u003c/span\u003e\u003c/span\u003e. Therefore, we tested the effect of applying a targeted lexical masking to the written summaries on the computation of similarity scores. To do this, we tokenized the summaries into individual words and removed predefined sets of words and phrases to reduce the influence of patient-specific context on similarity comparisons. For inter-author similarity, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\varvec{S}}_{\\varvec{i}\\varvec{n}\\varvec{t}\\varvec{e}\\varvec{r}}\\)\u003c/span\u003e\u003c/span\u003e, we removed gendered terms (e.g., \u0026ldquo;male\u0026rdquo;, \u0026ldquo;female\u0026rdquo;), medications, numerical values (e.g., ages, dates), and common basic phrases (e.g., \u0026ldquo;year old\u0026rdquo;, \u0026ldquo;hospitalist\u0026rdquo;). For intra-author similarity, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\varvec{S}}_{\\varvec{i}\\varvec{n}\\varvec{t}\\varvec{r}\\varvec{a}}\\)\u003c/span\u003e\u003c/span\u003e, we removed gendered terms, medications, numerical values, and additionally, patient medical history and symptoms. Because when evaluating \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\varvec{S}}_{\\varvec{i}\\varvec{n}\\varvec{t}\\varvec{e}\\varvec{r}}\\)\u003c/span\u003e\u003c/span\u003e, doctors\u0026rsquo; descriptions of medical history and symptoms may reflect stylistic preference. In contrast, when evaluating \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\varvec{S}}_{\\varvec{i}\\varvec{n}\\varvec{t}\\varvec{r}\\varvec{a}}\\)\u003c/span\u003e\u003c/span\u003e, differences in medical history and symptoms primarily reflects differences in patient contexts rather than contextual authorial style.\u003c/p\u003e \u003cp\u003eTo evaluate the effectiveness of each similarity metric and the masking, two hypothesis tests were performed on each metric:\u003c/p\u003e \u003cp\u003e \u003col\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003eComparing \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\varvec{S}}_{\\varvec{i}\\varvec{n}\\varvec{t}\\varvec{e}\\varvec{r}}\\)\u003c/span\u003e\u003c/span\u003e and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\varvec{S}}_{\\varvec{i}\\varvec{n}\\varvec{t}\\varvec{r}\\varvec{a}}\\)\u003c/span\u003e\u003c/span\u003e \u003cem\u003ebefore\u003c/em\u003e applying any masking, and\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003eComparing \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\varvec{S}}_{\\varvec{i}\\varvec{n}\\varvec{t}\\varvec{e}\\varvec{r}}\\)\u003c/span\u003e\u003c/span\u003e and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\varvec{S}}_{\\varvec{i}\\varvec{n}\\varvec{t}\\varvec{r}\\varvec{a}}\\)\u003c/span\u003e\u003c/span\u003e \u003cem\u003eafter\u003c/em\u003e applying appropriate masking.\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003c/ol\u003e \u003c/p\u003e \u003cp\u003eEach hypothesis test was performed using a two-sample t-test, and significance was determined if the associated p-value was less than 0.05. Because we aimed to find similarity metrics where \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\varvec{S}}_{\\varvec{i}\\varvec{n}\\varvec{t}\\varvec{r}\\varvec{a}}\\)\u003c/span\u003e\u003c/span\u003e was greater than \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\varvec{S}}_{\\varvec{i}\\varvec{n}\\varvec{t}\\varvec{e}\\varvec{r}}\\)\u003c/span\u003e\u003c/span\u003e, we used a one-sided alternative hypothesis (H\u003csub\u003eA\u003c/sub\u003e : \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\varvec{S}}_{\\varvec{i}\\varvec{n}\\varvec{t}\\varvec{e}\\varvec{r}}\\)\u003c/span\u003e\u003c/span\u003e \u0026lt; \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\varvec{S}}_{\\varvec{i}\\varvec{n}\\varvec{t}\\varvec{r}\\varvec{a}}\\)\u003c/span\u003e\u003c/span\u003e) for each hypothesis test.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec5\" class=\"Section2\"\u003e \u003ch2\u003e2.3. Phase 2: LLM-Driven Feature Extraction\u003c/h2\u003e \u003cp\u003eGiven the hypothesized limitations of quantitative metrics, we developed a qualitative feedback loop using Large Language Models (LLMs) to identify high-dimensional stylistic features (Fig.\u0026nbsp;1).\u003c/p\u003e \u003cp\u003eTo isolate the specific linguistic features that define a clinician\u0026rsquo;s style, we employed a two-stage classification strategy using Large Language Models (LLMs). We considered three possible labels for pairs of summary data:\u003c/p\u003e \u003c/div\u003e\n\u003cdiv class=\"Heading\"\u003e1) \u003cb\u003eSD\u003c/b\u003e: two summaries written by the same author about different patients,\u003c/div\u003e\n\u003cdiv class=\"Heading\"\u003e2) \u003cb\u003eDS\u003c/b\u003e: two summaries written by different authors about the same patient, and\u003c/div\u003e\n\u003cdiv class=\"Heading\"\u003e3) \u003cb\u003eDD\u003c/b\u003e: two summaries written by different authors about different patients.\u003c/div\u003e \u003cp\u003eBefore generating new content, the model was evaluated on its ability to distinguish summary pairs, where \u0026ldquo;SD\u0026rdquo; is classified as \u003cem\u003e\u0026ldquo;same author\u0026rdquo;\u003c/em\u003e and \u0026ldquo;DS\u0026rdquo; and \u0026ldquo;DD\u0026rdquo; are classified as \u003cem\u003e\u0026ldquo;different authors\u0026rdquo;\u003c/em\u003e. This was done by computing the overall accuracy and accuracy within each of the three possible summary pairs.\u003c/p\u003e \u003cp\u003e \u003cstrong\u003eZero-Shot Classification\u003c/strong\u003e \u003cp\u003eWe established a baseline using a Zero-Shot classification approach\u003csup\u003e\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e, \u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e\u003c/sup\u003e, utilizing the entire dataset without prior training examples or task-specific training. The model was presented with paired clinical summaries and given the following instructions\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cem\u003e\"For each of the entry pairs, verify if the two input texts were written by the same author using qualitative, detailed linguistic breakdown. Analyze the writing styles of the input texts, disregarding differences in topic and content. Reasoning based on linguistic features including but not limited to phrasal verbs, modal verbs, rare words, affixes, quantities, tone, and abbreviation usage. Note that some pairs are written by the same author, and some pairs are written by different authors.\"\u003c/em\u003e \u003c/p\u003e \u003cp\u003e \u003cb\u003eFew-Shot Classification (Train-Test Split)\u003c/b\u003e: To improve classification accuracy, we implemented a few-shot classification approach\u003csup\u003e\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e, \u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e\u003c/sup\u003e using a train-test split. In this setting, the model was first provided with a labelled training set consisting of summary pairs annotated as either authored by the \u003cem\u003esame\u003c/em\u003e clinician or by \u003cem\u003edifferent\u003c/em\u003e clinicians. This in-context learning allowed the model to learn corpus-specific stylistic cues before generating predictions on a held-out test set. We evaluated multiple train-test split ratios (100:100, 120:80, 140:60, 160:40). ). The model was given the following instructions:\u003c/p\u003e \u003cp\u003e \u003cem\u003e\"Take the attached data as the training data set and develop a predictor using qualitative detailed linguistic features to predict whether two input texts are written by the same author. Analyze the writing styles of the input texts, disregarding differences in topic, content, and grammatical errors.\"\u003c/em\u003e \u003c/p\u003e \u003cp\u003eFor the test set, we evaluated three distinct \u003cb\u003eprompt\u003c/b\u003e\u003csup\u003e\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e, \u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e\u003c/sup\u003e \u003cb\u003eendings\u003c/b\u003e to determine the optimal level of instruction specificity:\u003c/p\u003e \u003cp\u003e \u003cul\u003e \u003cli\u003e \u003cp\u003e \u003cb\u003ePrompt Ending A (Baseline)\u003c/b\u003e: An uninformative instruction.\u003c/p\u003e \u003c/li\u003e \u003c/ul\u003e \u003cdiv class=\"BlockQuote\"\u003e \u003cp\u003e \u003cem\u003e\"For each input text pair, verify if they were written by the same or different author using the previously developed predictor.\"\u003c/em\u003e \u003c/p\u003e \u003c/div\u003e \u003c/p\u003e \u003cp\u003e \u003cul\u003e \u003cli\u003e \u003cp\u003e \u003cb\u003ePrompt Ending B (Broad Qualitative)\u003c/b\u003e: Focused on general linguistic features.\u003c/p\u003e \u003c/li\u003e \u003c/ul\u003e \u003cdiv class=\"BlockQuote\"\u003e \u003cp\u003e \u003cem\u003e\"Using the previously developed predictor, verify if the two input texts were written by the same author using qualitative detailed linguistic authorship breakdown. Analyze the writing styles of the input texts, disregarding differences in topic and content and grammatical errors. Reasoning based on linguistic features including sentence and list structure, phrasal verbs, modal verbs, affixes, tone, and abbreviation usage.\"\u003c/em\u003e \u003c/p\u003e \u003c/div\u003e \u003c/p\u003e \u003cp\u003e \u003cul\u003e \u003cli\u003e \u003cp\u003e \u003cb\u003ePrompt Ending C (Targeted Feature)\u003c/b\u003e: Narrowed focus based on features identified during preliminary qualitative extraction (e.g., rhythm, narration style).\u003c/p\u003e \u003c/li\u003e \u003c/ul\u003e \u003cdiv class=\"BlockQuote\"\u003e \u003cp\u003e \u003cem\u003e\"Using the previously developed predictor, verify if the two input texts were written by the same author using qualitative detailed linguistic authorship breakdown. Analyze the writing styles of the input texts, disregarding differences in topic and content and grammatical errors. Reasoning focused on writing style elements of sentence and list structure, phrasing, tone, rhythm, and narration style.\"\u003c/em\u003e \u003c/p\u003e \u003c/div\u003e \u003c/p\u003e \u003cdiv id=\"Sec9\" class=\"Section2\"\u003e \u003ch2\u003e2.4. Phase 3: Style-Informed Generation\u003c/h2\u003e \u003cp\u003eBased on the superior performance (see Results 3.2) of the targeted feature extraction (Prompt Ending C) in our testing classification accuracy, we advanced to the generation phase. This workflow utilized a \"Train-Generate\" paradigm to create new summaries that mimicked a specific provider's style. Early attempts at summary generation led to hallucinations, as clinicians identified fabricated and false clinical information in LLM summaries that contradicted the original text of the clinical summary. To address this issue, we employed a strong and explicit prompting strategy to create an authoritative style guide. This style guide outlined how to capture provider-specific stylistic features and what information the LLM could and couldn\u0026rsquo;t use in its summary generation.\u003c/p\u003e \u003cp\u003e \u003cstrong\u003eTraining Phase (Style Profiling)\u003c/strong\u003e \u003cp\u003eFor each target clinician, we allocated 20 previously authored summaries to a training set for in-context learning. In this phase, GPT-4 and Gemini 2.5 Pro were instructed to analyze these examples to create an authoritative style guide for that specific user.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cem\u003e\"Use the attached data as a training data set to analyze the writing style of the author writing the summaries. Focus on writing style elements of sentence and list structure, phrasing, tone, rhythm, and narration style. Use this analyzed writing style as the authoritative style guide for generating all future summaries.\"\u003c/em\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eGeneration Phase\u003c/strong\u003e \u003cp\u003eIn the final stage, the model generated new summaries for held-out reports (N\u0026thinsp;=\u0026thinsp;10 per provider). The prompt included strict safety constraints to prevent hallucinations, explicitly requiring the model to adhere to the previously created authoritative style guide while using only the facts presented in the new report.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cem\u003e\"Use the identified writing style of this author developed from the training data set to generate a one paragraph stylistically consistent summary for the report below labeled 'Report:'. When generating the summary for the new report, your output must strictly follow the stylistic conventions of the training summaries, including sentence and list structure, phrasing, tone, rhythm, and narration style. Be sure that generated summaries are concise like the original summaries in the training data set and concentrate on the specific content mentioned in the report to make them informative and clinically focused. Do not fabricate or infer any information. Only summarize details explicitly stated in the report. If information is missing, leave it out \u0026mdash; do not guess or create new clinical facts.\"\u003c/em\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec10\" class=\"Section2\"\u003e \u003ch2\u003e2.5. Phase 4: Clinician Preference Evaluation (A/B Testing)\u003c/h2\u003e \u003cp\u003eTo validate clinical utility, we conducted a blinded A/B preference study\u003csup\u003e\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e\u003c/sup\u003e with practicing clinicians. Clinicians were presented with pairs of summaries for cases they had previously documented (one Human-authored, which they had previously written, one GPT-4 or Gemini 2.5 Pro generated). For each pair, clinicians were asked to select a preference: \"Prefer Human,\" \"Prefer LLM,\" or \"Looks the Same\". Statistical significance was assessed using a one-sample proportion hypothesis test (\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({H}_{0}:p=0.5\\)\u003c/span\u003e\u003c/span\u003e), treating \"Looks the Same\" as a neutral outcome weighted equally between the two categories.\u003cdiv id=\"Equa\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equa\" name=\"EquationSource\"\u003e\n$$\\widehat{p}=\\frac{PreferHuman+0.5*LookstheSame}{TotalResponses}$$\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eThis four-phase workflow illustrates the transition from traditional quantitative analysis to personalized clinical documentation. The process begins with Phase 1, evaluating the limitations of standard metrics through lexical masking and similarity testing. Phase 2 uses a feedback loop to identify high-dimensional linguistic markers that traditional metrics miss. Phase 3 implements these markers into a clinician-specific style guide for summary generation, while Phase 4 validates the output through blinded preference testing.\u003c/p\u003e \u003c/div\u003e"},{"header":"3. Results","content":"\u003cdiv id=\"Sec12\" class=\"Section2\"\u003e \u003ch2\u003e3.1. Limitations of Traditional Quantitative Metrics\u003c/h2\u003e \u003cp\u003eAn ideal similarity metric should have high intra-author scores (\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({S}_{intra}\\)\u003c/span\u003e\u003c/span\u003e) and low inter-author scores \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({(S}_{inter})\\)\u003c/span\u003e\u003c/span\u003e. The simple similarity metrics (lexical and character-level) demonstrated a limited ability to isolate authorial style from patient-specific content. As shown in Fig.\u0026nbsp;2, for the \u003cb\u003eCosine, Jaccard, and Levenshtein\u003c/b\u003e metrics, the inter-author similarity was consistently higher than the intra-author similarity. The high \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({(S}_{inter})\\)\u003c/span\u003e\u003c/span\u003escores indicate that these metrics primarily track shared medical terminology and factual overlap, rather than unique phrasing. Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e highlights that \u003cb\u003eJaro-Winkler (JW)\u003c/b\u003e was the only simple metric to achieve statistical significance in both masked (\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(p=4.43e-28\\)\u003c/span\u003e\u003c/span\u003e) and unmasked (\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(p=5.76e-122\\)\u003c/span\u003e\u003c/span\u003e) conditions. The relative success of Jaro-Winkler suggests it is more sensitive to \"character-level\" stylistic habits, such as specific ways of abbreviating or consistent sentence patterns, which remain consistent for an author even when the medical content changes.\u003c/p\u003e \u003cp\u003e \u003cb\u003eFigure 2: Effectiveness of Simple Similarity Metrics on Masked and Unmasked Data\u003c/b\u003e \u003c/p\u003e \u003cp\u003eFor each metric, we computed similarity scores between all possible pairs of written summaries in the masked and unmasked case. We used this data to perform hypotheses tests on the difference between \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({S}_{inter}\\)\u003c/span\u003e\u003c/span\u003e and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({S}_{intra}\\)\u003c/span\u003e\u003c/span\u003e. Jaro-Winkler abbreviated to JW, Levenshtein abbreviated to LV.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003e\u003cb\u003eP-values of Hypothesis Tests on difference of\u003c/b\u003e \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\varvec{S}}_{\\varvec{i}\\varvec{n}\\varvec{t}\\varvec{e}\\varvec{r}}\\)\u003c/span\u003e\u003c/span\u003e \u003cb\u003eand\u003c/b\u003e \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\varvec{S}}_{\\varvec{i}\\varvec{n}\\varvec{t}\\varvec{r}\\varvec{a}}\\)\u003c/span\u003e\u003c/span\u003e\u003cb\u003efor Simple Metrics\u003c/b\u003e For each metric, we performed two one-sided t-tests: 1) masked vs. masked and 2) unmasked vs. unmasked to evaluate if there is a significant difference between and. The p-values for these hypothesis tests are reported in the table.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"3\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eCosine\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eMasked\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eUnmasked\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eJaccard\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eLevenshtein\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eJaro\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0.846\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0.318\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eJaro-Winkler\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e4.43e-28\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e5.76e-122\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eThe advanced semantic metrics (ROUGE, BLEU, BERTScore) were expected to better capture \"clinical intent,\" yet they largely mirrored the failures of lexical metrics. As illustrated in Fig.\u0026nbsp;3, \u003cb\u003eROUGE-1, ROUGE-L, and BERTScore\u003c/b\u003e failed to produce a significant difference between \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({S}_{intra}\\)\u003c/span\u003e\u003c/span\u003e and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({S}_{inter}\\)\u003c/span\u003e\u003c/span\u003e. In fact, the \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(pvalues\\)\u003c/span\u003e\u003c/span\u003e for ROUGE-1 and ROUGE-L were 1.0, indicating that these metrics computed higher scores for \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({S}_{inter}\\)\u003c/span\u003e\u003c/span\u003e than \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({S}_{intra}\\)\u003c/span\u003e\u003c/span\u003e, the exact opposite of what we wanted to capture in our similarity metrics. Interestingly, applying targeted lexical masking (removing medications, dates, and gendered terms) did not made substantial change in these metrics. Even after removing patient-specific facts, the BERTScore embeddings still gravitated toward the shared medical narrative rather than the authorial voice.\u003c/p\u003e \u003cp\u003e \u003cb\u003eFigure 3: Effect of Masking on Advanced Similarity Metrics\u003c/b\u003e \u003c/p\u003e \u003cp\u003eFor each metric, we computed similarity scores between all possible pairs of written summaries in the masked and unmasked cases. We used this data to perform hypothesis tests on the difference between \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({S}_{inter}\\)\u003c/span\u003e\u003c/span\u003e and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({S}_{intra}\\)\u003c/span\u003e\u003c/span\u003e. BERTScore is abbreviated to BERT.\u003c/p\u003e \u003cp\u003eTable\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e shows that BLEU achieved statistical significance (\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(p=2.55e-16\\)\u003c/span\u003e\u003c/span\u003e) only in the unmasked state. This suggests that BLEU\u0026rsquo;s sensitivity to specific n-gram sequences is heavily tied to the raw clinical data, and once that data is masked, its ability to detect authorial voice vanishes.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003e\u003cb\u003eP-values of Hypothesis Tests on the difference of\u003c/b\u003e \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\varvec{S}}_{\\varvec{i}\\varvec{n}\\varvec{t}\\varvec{e}\\varvec{r}}\\)\u003c/span\u003e\u003c/span\u003e \u003cb\u003eand\u003c/b\u003e \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({\\varvec{S}}_{\\varvec{i}\\varvec{n}\\varvec{t}\\varvec{r}\\varvec{a}}\\)\u003c/span\u003e\u003c/span\u003e\u003cb\u003efor Advanced Metrics\u003c/b\u003e For each metric, we performed two one-sided t-tests: 1) masked vs. masked and 2) unmasked vs. unmasked to evaluate if there is a significant difference between and. The p-values for these hypotheses tests are reported in the table.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"3\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eBERTScore\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eMasked\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eUnmasked\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0.177\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eBLEU\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0.304\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e2.55e-16\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eRouge1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eRouge2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0.980\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eRougeL\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec13\" class=\"Section2\"\u003e \u003ch2\u003e3.2. Efficacy of LLM-Driven Feature Extraction\u003c/h2\u003e \u003cp\u003eTransitioning to qualitative feature extraction using LLMs yielded a marked improvement in authorship attribution, validating the hypothesis that style is better defined by high-dimensional linguistic features than by n-gram overlap.\u003c/p\u003e \u003cp\u003e \u003cul\u003e \u003cli\u003e \u003cp\u003e \u003cb\u003eZero-Shot Baseline\u003c/b\u003e: As shown in Table\u0026nbsp;\u003cspan refid=\"Tab3\" class=\"InternalRef\"\u003e3\u003c/span\u003e, the Zero-Shot baseline struggled significantly, with overall accuracy fluctuating between 44.67% and 62%. Without specific stylistic training, the model often performed only slightly better than a random guess when trying to identify if two summaries were written by the same author. The model showed extreme variance in identifying \"Same Author\" (SD) versus \"Different Author\" (DS/DD) pairs across different trials. In some trials, it over-identified same-author pairs, while in others, it did the opposite, indicating a lack of a stable internal stylistic capture.\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003e \u003cb\u003eFew-Shot \u0026amp; Prompt Engineering\u003c/b\u003e: Results changed dramatically with the introduction of these methods. Specifically, Prompt Ending C which focused on rhythm, narration, and sentence/list structure, achieved the highest performance. Utilizing a 100:100 train-test split, the model reached an overall accuracy of 73%. Most notably, this approach achieved 80% accuracy in correctly identifying \"Same Author\" (SD) pairs. This suggests that once the LLM was instructed to look for specific high-dimensional markers like narrative rhythm and hedging strategies, it could successfully recognize the professional identity of the clinician.\u003c/p\u003e \u003c/li\u003e \u003c/ul\u003e \u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab3\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003e\u003cb\u003ePerformance of Zero-Shot LLM Author Attribution\u003c/b\u003e This table displays the baseline classification results across six independent trials using a zero-shot approach without prior stylistic training. Accuracy scores represent the model's ability to distinguish between same-author pairs (SD) and different-author pairs (DS and DD). The results highlight the high variance and limited reliability of zero-shot models in identifying authorial signatures without specific training examples. Each row represents a trial with 150 total summary pairs, with 50 from each label.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"4\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eOverall (%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eSD Label (%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eDS Label (%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eDD Label (%)\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e44.67\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e78\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e30\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e26\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e44.67\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e62\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e32\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e40\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e55.33\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e60\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e38\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e68\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e58\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e48\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e54\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e72\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e61.33\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e28\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e78\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e78\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e62\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e18\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e82\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e86\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab4\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 4\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003e\u003cb\u003eEfficacy of Few-Shot Learning and Targeted Prompting\u003c/b\u003e This table compares the classification accuracy of three distinct prompt specificity levels using a 100:100 train-test split. The data demonstrates the significant performance improvement achieved through few-shot learning, particularly with Prompt 3 (Targeted Feature Extraction). This specific prompt reached a peak overall accuracy of 73% and an 80% success rate in identifying clinician-specific signatures, validating the use of high-dimensional linguistic markers for style replication. Each row represents a trial with 200 total summary pairs with 100 SD, 50 DS, and 50 DD.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"5\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePrompt\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eOverall (%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eSD Label (%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eDS Label (%)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eDD Label (%)\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e46\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e46\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e48\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e35\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e47\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e44\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e52\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e50\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e73\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e80\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e60\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e65\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec14\" class=\"Section2\"\u003e \u003ch2\u003e3.3. Clinician Preference and Style Replication (A/B Testing)\u003c/h2\u003e \u003cp\u003eThe ultimate validation of the framework was the blinded A/B preference study. Results indicated a performance divergence between model architectures (GPT-4 vs. Gemini 2.5 Pro), with Gemini demonstrating stronger style replication capabilities.\u003c/p\u003e \u003cp\u003e \u003cul\u003e \u003cli\u003e \u003cp\u003e \u003cb\u003eGPT-4 Performance\u003c/b\u003e: In the GPT-4 arm, clinicians retained a preference for human-authored summaries. Clinicians preferred the Human draft in 57 responses compared to 24 responses for the LLM. With a calculated \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(p\\)\u003c/span\u003e\u003c/span\u003e-value of 1.0 (Table\u0026nbsp;\u003cspan refid=\"Tab5\" class=\"InternalRef\"\u003e5\u003c/span\u003e) and a preference proportion \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\left(p\\right)\\)\u003c/span\u003e\u003c/span\u003eof 0.6833, the results failed to achieve statistical significance, indicating that GPT-4 did not sufficiently capture the unique nuances of the provider's style.\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003e \u003cb\u003eGemini 2.5 Pro Performance\u003c/b\u003e: The Gemini model demonstrated superior style alignment. Clinicians showed a slight preference for the LLM-generated draft (43 responses) over their own historical notes (35 responses), with 12 responses rated as indistinguishable. The hypothesis test yielded a p-value of 0.4606 for Gemini. This suggests that the Gemini-based pipeline, informed by the authorial voice, generated summaries that were noninferior to the clinician\u0026rsquo;s original documentation. In some cases, the LLM generated clinical summaries were preferred over the clinician-authored notes. However, these differences did not reach statistical significance.\u003c/p\u003e \u003c/li\u003e \u003c/ul\u003e \u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eThis comparative visualization displays the results of the blinded preference study for both model architectures. The data indicate the number of responses in which clinicians preferred their original human-authored notes, the LLM-generated drafts, or found the two to be indistinguishable. The results show a notable performance divergence, with Gemini 2.5 Pro achieving a higher rate of style alignment and clinician preference than GPT-4.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab5\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 5\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003e\u003cb\u003eStatistical Significance of Clinician Preferences\u003c/b\u003e This table reports the outcomes of the one-sample proportion hypothesis test used to evaluate model performance. The p-values indicate whether the observed preference for a draft significantly deviated from the neutral baseline of 0.5. Results show that while GPT-4 results favored human drafts, the Gemini 2.5 Pro results indicated non-inferiority, as the preference difference was not statistically significant.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"3\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eLLM\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\widehat{\\varvec{p}}\\)\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003ep-value\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eGPT-4\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.6833\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eGemini 2.5 Pro\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.4556\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0.4606\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003c/div\u003e"},{"header":"4. Discussion","content":"\u003cp\u003eOur findings challenge the prevailing assumption in medical NLP that factual correctness is the sole proxy for clinical utility. While prior studies demonstrate that LLMs achieve high semantic fidelity, our quantitative analysis reveals that traditional metrics\u0026mdash;such as ROUGE, BLEU, and BERTScore\u0026mdash;are fundamentally ill-suited for measuring stylistic concordance. As shown in our \"Masking Effect\" analysis, these metrics failed to reliably distinguish between clinical summaries written by different clinicians for the same patient. This confirms that high n-gram overlap captures the \u003cb\u003econtent\u003c/b\u003e of the medical case but fundamentally misses the \u003cb\u003estyle\u003c/b\u003e of the provider. For the clinical community, this implies that a high-scoring AI summary may still feel alien to the signing clinician, effectively \"bleaching\" the nuanced clinical gestalt and professional identity from the document. This perpetuates the editing load as providers are forced to rewrite accurate but tonally mismatched drafts. These findings suggest that stylistic personalization may represent a critical next step in clinical documentation automation.\u003c/p\u003e \u003cp\u003eTo overcome the \"homogenization effect\" typical of RLHF-aligned models\u0026mdash;which prioritize uniform, safe outputs over diverse authorial voices\u0026mdash;this study introduces a \u003cb\u003eStyle-Informed Generation Framework\u003c/b\u003e. By shifting from zero-shot prompting to a \"Train-Generate\" pipeline, we successfully extracted high-dimensional features (such as narrative rhythm and hedging strategies) that elude traditional statistical detection methods. The efficacy of this approach was evidenced by a jump in authorship attribution accuracy to \u003cb\u003e73%\u003c/b\u003e with targeted feature extraction, compared to the near-random performance of zero-shot baselines. When the LLM is explicitly constrained to a stylistic profile derived from historical data, it begins to convey the authorial voice of the care team, producing drafts that are often indistinguishable from human writing.\u003c/p\u003e \u003cp\u003eA critical finding of this study is the divergence in performance between model architectures. While GPT-4 produced fluent text, clinicians largely preferred their own human-written notes. In contrast, Gemini 2.5 Pro appeared more responsive to style-profile constraints in this experimental setting, with 43 clinician preferences compared to 35 for human drafts. This variance likely stems from how each model weights long-context instructions versus its underlying safety alignment. Gemini\u0026rsquo;s ability to generate \"non-inferior\" summaries suggests it may be better suited for applications requiring high-fidelity style transfer in-context learning.\u003c/p\u003e \u003cp\u003e \u003cb\u003eLimitations\u003c/b\u003e \u003c/p\u003e \u003cp\u003eWhile our framework mitigated hallucinations through strict prompt constraints, the safety-critical nature of clinical documentation requires ongoing verification. The small sample size (N\u0026thinsp;=\u0026thinsp;10) and the use of static (\"frozen\") Style Profiles limited an immediate generalizability. Future research should expand the cohort size beyond ten clinicians to improve the generalizability of these findings. The current approach should transition to dynamic models that evolve in real-time as a clinician's documentation style changes over their career. Additionally, investigations into architectural differences are needed to determine why specific models, such as Gemini 2.5 Pro, demonstrate superior adaptability to in-context style constraints. Finally, future studies should employ downstream impact metrics such as clinician vs LLM time difference to provide a direct quantitative measure of how much the editing load is reduced in active clinical workflows.\u003c/p\u003e"},{"header":"Conclusion","content":"\u003cp\u003eThe integration of Large Language Models into electronic health records offers a path to mitigate clinician burnout, but only if the technology aligns with users' professional identity. We demonstrate that while current models are factually capable, they face a \"Style Gap\" that traditional metrics fail to detect. By implementing a Style-Informed Feedback Loop, we provide a reproducible proof-of-concept for personalized clinical documentation. This approach moves the field closer to an AI assistant that not only \"summarize,\" but also aligns with the stylistic conventions of the clinical author, ultimately reducing administrative burden and allowing clinicians to return to patient care.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e \u003ch2\u003eHuman Ethics and Consent to Participate\u003c/h2\u003e \u003cp\u003eThis project was approved by the Wake Forest University School of Medicine Research Ethics Board Protocol review board approval (IRB00127840) in accordance with the Declaration of Helsinki. All the records in this study were de-identified.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eClinical trial number\u003c/strong\u003e \u003cp\u003enot applicable.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eConflict of Interest:\u003c/strong\u003e \u003cp\u003eThe authors have declared that no competing interests exist.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eCompeting interests:\u003c/strong\u003e \u003cp\u003eThe authors declare no competing interests.\u003c/p\u003e \u003c/p\u003e\u003ch2\u003eFunding:\u003c/h2\u003e \u003cp\u003eThis work was supported by the \u003cb\u003eNational Institutes of Health (NIH) / National Center for Advancing Translational Sciences (NCATS)\u003c/b\u003e under award number \u003cb\u003eU01 TR003629\u003c/b\u003e, titled 'Analytics \u0026amp; Machine-learning for Maternal-health Interventions (AMMI): A Cross-CTSA Collaboration'. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.\u003c/p\u003e\u003ch2\u003eAuthor Contribution\u003c/h2\u003e\u003cp\u003e- **Scott Zhao** : Software; formal analysis; data curation; writing \u0026ndash; original draft.- **Abbas Alili** : Supervision; project administration; methodology; writing \u0026ndash; original draft; writing \u0026ndash; review and editing.- **Usman Afzaal** : Data curation; software; validation.- **Hao Lu** : Methodology; software; writing review.- **Muhammet F. Demir** : Investigation; validation; data curation.- **Padageshwar Sunkara** : Conceptualization; writing \u0026ndash; original draft; investigation; clinical validation.- **Metin N. Gurcan** : Conceptualization; senior supervision; funding acquisition; resources; writing \u0026ndash; review and editing.\u003c/p\u003e\u003ch2\u003eAcknowledgement\u003c/h2\u003e\u003cp\u003eWe are deeply grateful to Raghava Nagaraj, Suneel Kumar Parvathareddy, Sumera Andleeb, William B. Winfrey, Nicolas Haller, Vani Khajuria, Megan Jodrey, Sneha Chebrolu, Katherine Rose Sommers, Marc Holden Perlman, Harsh Barot, and Kinchit K. Shah for their invaluable support and contributions to this work. Their clinical perspectives greatly strengthened the quality and clarity of the research.Informed consent was obtained from all the experts involved in the study.\u003c/p\u003e\u003ch2\u003eData Availability\u003c/h2\u003e\u003cp\u003eThe datasets generated and analyzed during the current study are not publicly available due to IRB restrictions at Wake Forest University School of Medicine (IRB00127840) regarding the protection of sensitive clinical data. However, de-identified data can be made available from the corresponding author upon reasonable request and following the execution of a formal Data Use Agreement (DUA) to ensure ethical and legal compliance.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eKind, A. J. H. \u0026amp; SMA. \u0026amp;. Documentation of Mandated Discharge Summary Components in Transitions from Acute to Subacute Care. \u003cem\u003eAdvances in Patient Safety: New Directions and Alternative Approaches (Vol 2: Culture and Redesign)\u003c/em\u003e. Published online 2008.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eVan Walraven, C., Seth, R., Austin, P. C. \u0026amp; Laupacis, A. \u003cem\u003eEffect of Discharge Summary Availability During Post-Discharge Visits on Hospital Readmission\u003c/em\u003e. CvW.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLi, Y. et al. A comparative study of recent large language models on generating hospital discharge summaries for lung cancer patients. \u003cem\u003eJ. Biomed. Inf.\u003c/em\u003e \u003cb\u003e168\u003c/b\u003e \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1016/j.jbi.2025.104867\u003c/span\u003e\u003cspan address=\"10.1016/j.jbi.2025.104867\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2025).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWu, Y. et al. Evaluating the Prevalence of Burnout Among Health Care Professionals Related to Electronic Health Record Use: Systematic Review and Meta-Analysis. \u003cem\u003eJMIR Med. Inform JMIR Publications Inc\u003c/em\u003e. \u003cb\u003e12\u003c/b\u003e \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.2196/54811\u003c/span\u003e\u003cspan address=\"10.2196/54811\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2024).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWilliams, C. Y. K. et al. Physician- and Large Language Model-Generated Hospital Discharge Summaries. \u003cem\u003eJAMA Intern. Med\u003c/em\u003e Published online 2025. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1001/jamainternmed.2025.0821\u003c/span\u003e\u003cspan address=\"10.1001/jamainternmed.2025.0821\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGanzinger, M. et al. Automated generation of discharge summaries: leveraging large language models with clinical data. \u003cem\u003eSci. Rep.\u003c/em\u003e \u003cb\u003e15\u003c/b\u003e (1). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1038/s41598-025-01618-7\u003c/span\u003e\u003cspan address=\"10.1038/s41598-025-01618-7\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2025).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eOliveira, J. D. et al. Development and evaluation of a clinical note summarization system using large language models. \u003cem\u003eCommun. Med.\u003c/em\u003e \u003cb\u003e5\u003c/b\u003e (1). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1038/s43856-025-01091-3\u003c/span\u003e\u003cspan address=\"10.1038/s43856-025-01091-3\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2025).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKoh, M. C. Y. et al. Using ChatGPT for writing hospital inpatient discharge summaries \u0026ndash; perspectives from an inpatient infectious diseases service. \u003cem\u003eBMC Health Serv. Res.\u003c/em\u003e \u003cb\u003e25\u003c/b\u003e (1). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1186/s12913-025-12373-w\u003c/span\u003e\u003cspan address=\"10.1186/s12913-025-12373-w\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2025).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSourati, Z., Ziabari, A. S. \u0026amp; Dehghani, M. The Homogenizing Effect of Large Language Models on Human Expression and Thought. Published online August 2, (2025). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://arxiv.org/abs/2508.01491\u003c/span\u003e\u003cspan address=\"http://arxiv.org/abs/2508.01491\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKirk, R. et al. \u003cem\u003eUnderstanding the Effects of RLHF on LLM Generalisation and Diversity\u003c/em\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHains, L. et al. Large language model discharge summary preparation using real-world electronic medical record data shows promise. \u003cem\u003eIntern. Med. J.\u003c/em\u003e \u003cb\u003e55\u003c/b\u003e (7), 1188\u0026ndash;1192. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1111/imj.70073\u003c/span\u003e\u003cspan address=\"10.1111/imj.70073\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2025).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSatheakeerthy, S., Jesudason, D., Pietris, J., Bacchi, S. \u0026amp; Chan, W. O. LLM-assisted medical documentation: efficacy, errors, and ethical considerations in ophthalmology. \u003cem\u003eEye (Basingstoke) Springer Nature\u003c/em\u003e. \u003cb\u003e39\u003c/b\u003e (8), 1440\u0026ndash;1442. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1038/s41433-025-03767-5\u003c/span\u003e\u003cspan address=\"10.1038/s41433-025-03767-5\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2025).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChua, C. E. et al. Integration of customised LLM for discharge summary generation in real-world clinical settings: a pilot study on RUSSELL GPT. \u003cem\u003eLancet Reg. Health West. Pac Elsevier Ltd\u003c/em\u003e. \u003cb\u003e51\u003c/b\u003e \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1016/j.lanwpc.2024.101211\u003c/span\u003e\u003cspan address=\"10.1016/j.lanwpc.2024.101211\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2024).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAlqahtani, M., Al-Barakati, A., Alotaibi, F., Al Shibli, M. \u0026amp; Almousa, S. Impact of Detailed Versus Generic Instructions on Fine-Tuned Language Models for Patient Discharge Instructions Generation: Comparative Statistical Analysis. \u003cem\u003eJMIR Form. Res.\u003c/em\u003e \u003cb\u003e9\u003c/b\u003e \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.2196/80917\u003c/span\u003e\u003cspan address=\"10.2196/80917\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2025).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eOrkphol, K. \u0026amp; Yang, W. Word Sense Disambiguation Using Cosine Similarity Collaborates with Word2vec and WordNet. \u003cem\u003eFutur Internet 2019\u003c/em\u003e. \u003cb\u003e11\u003c/b\u003e (Page 114 11), 114 (2019).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChauvin, L., Kumar, K., Desrosiers, C., Wells, W. \u0026amp; Toews, M. Efficient Pairwise Neuroimage Analysis Using the Soft Jaccard Index and 3D Keypoint Sets. \u003cem\u003eIEEE Trans. Med. Imaging\u003c/em\u003e. \u003cb\u003e41\u003c/b\u003e (4), 836\u0026ndash;845. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1109/TMI.2021.3123252\u003c/span\u003e\u003cspan address=\"10.1109/TMI.2021.3123252\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2022). Epub 2022 Apr 1. PMID: 34699353; PMCID: PMC9022638.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFaes, J. \u0026amp; Gillis, S. Speech production accuracy of children with auditory brainstem implants: A comparison with peers with cochlear implants and typical hearing using Levenshtein Distance.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRozinek, O. \u0026amp; Mareš, J. Fast and Precise Convolutional Jaro and Jaro-Winkler Similarity, 2024 35th Conference of Open Innovations Association (FRUCT), Tampere, Finland, 2024, pp. 604\u0026ndash;613. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.23919/FRUCT61870.2024.10516360\u003c/span\u003e\u003cspan address=\"10.23919/FRUCT61870.2024.10516360\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eXia, K. Design and application of efficient English learning system based on Jaro-Winkler. \u003cem\u003eJ. Comput. Methods Sci. Eng.\u003c/em\u003e \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1177/14727978251371179\u003c/span\u003e\u003cspan address=\"10.1177/14727978251371179\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2025).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLin, C. Y. ROUGE: A Package for Automatic Evaluation of Summaries.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePapineni, K., Roukos, S., Ward, T. \u0026amp; Zhu, W. J. BLEU: a Method for Automatic Evaluation of Machine Translation.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAustrian, J. et al. Applying A/B Testing to Clinical Decision Support: Rapid Randomized Controlled Trials. \u003cem\u003eJ. Med. Internet Res.\u003c/em\u003e \u003cb\u003e23\u003c/b\u003e (4), e16651. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.2196/16651\u003c/span\u003e\u003cspan address=\"10.2196/16651\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2021). PMID: 33835035; PMCID: PMC8065554.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePre-trained Language Models Can be Fully Zero-Shot Learners](\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://aclanthology.org/2023.acl-long.869/\u003c/span\u003e\u003cspan address=\"https://aclanthology.org/2023.acl-long.869/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e) (Zhao et al., ACL 2023).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eYan, L., Zheng, Y. \u0026amp; Cao, J. Few-shot learning for short text classification. \u003cem\u003eMultimed Tools Appl.\u003c/em\u003e \u003cb\u003e77\u003c/b\u003e, 29799\u0026ndash;29810. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1007/s11042-018-5772-4\u003c/span\u003e\u003cspan address=\"10.1007/s11042-018-5772-4\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2018).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eYisheng Song, T., Wang, P., Cai, S. K., Mondal \u0026amp; Jyoti Prakash, S. A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities. \u003cem\u003eACM Comput. Surv.\u003c/em\u003e \u003cb\u003e55\u003c/b\u003e, 40pages. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1145/3582688\u003c/span\u003e\u003cspan address=\"10.1145/3582688\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2023). 13s, Article 271 (December 2023.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLee, Y. et al. Prompt engineering in ChatGPT for literature review: practical guide exemplified with studies on white phosphors. \u003cem\u003eSci. Rep.\u003c/em\u003e \u003cb\u003e15\u003c/b\u003e, 15310. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1038/s41598-025-99423-9\u003c/span\u003e\u003cspan address=\"10.1038/s41598-025-99423-9\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2025).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePrompting science report 1: Prompt engineering is complicated and contingent L Meincke, Mollick, E. \u0026amp; Mollick, L. D Shapiro arXiv preprint arXiv:2503.04818, 2025\u0026bull;arxiv.org.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eZhang, Z. et al. A zero-shot prompt learning approach on fine-grained text classification. \u003cem\u003eSci. Rep.\u003c/em\u003e \u003cb\u003e16\u003c/b\u003e, 5260. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1038/s41598-025-34825-3\u003c/span\u003e\u003cspan address=\"10.1038/s41598-025-34825-3\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e (2026).\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-9054955/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-9054955/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eLarge language models (LLMs) can automate clinical document summary generation. However, even clinically accurate outputs often fail to reflect individual clinicians\u0026rsquo; writing styles, leading to substantial post-editing. We examine this stylistic gap using a multi-author corpus of de-identified clinical summaries. We propose a style-informed generation framework that extracts clinician-specific stylistic features through LLM feedback and applies a Train\u0026rarr;Generate paradigm to produce personalized clinical summaries. Conventional metrics (ROUGE, BERTScore, cosine similarity) largely failed to distinguish intra-author from inter-author writing patterns, while Jaro-Winkler and BLEU demonstrated limited sensitivity. Targeted LLM-guided feature extraction\u0026mdash;emphasizing rhythm, narration, and sentence or list structure\u0026mdash;improved authorship classification up to 73% of accuracy. In blinded clinician A/B testing, GPT-4-generated drafts were preferred less often than original notes, whereas the Gemini 2.5 Pro pipeline produced drafts preferred at rates comparable to, and in some cases exceeding, clinician-authored summaries. While inherent hallucination risks were noted, they were mitigated via high-fidelity prompt engineering and explicit adherence to source-only data constraints. These results suggest that style-informed generation can reduce the style gap and produce clinically acceptable clinical summaries that better align with the clinician\u0026rsquo;s voice.\u003c/p\u003e","manuscriptTitle":"Decoding Clinician Authorial Style: A Style-Informed Pipeline for Clinical Document Summary Generation with Large Language Models","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-03-26 12:59:59","doi":"10.21203/rs.3.rs-9054955/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"editorInvitedReview","content":"","date":"2026-04-19T22:59:04+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"17453932526591696473830815151962117965","date":"2026-03-29T12:38:44+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"15233791711705340843138867149884588519","date":"2026-03-29T09:06:02+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"250929513909404215731204607598632955866","date":"2026-03-25T19:35:34+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2026-03-24T08:57:48+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2026-03-24T08:56:12+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2026-03-18T03:21:30+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2026-03-16T16:36:16+00:00","index":"","fulltext":""},{"type":"submitted","content":"Scientific Reports","date":"2026-03-16T13:53:18+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"738f914e-1ad6-452c-bd09-7cc12a684b29","owner":[],"postedDate":"March 26th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[{"id":65146544,"name":"Biological sciences/Computational biology and bioinformatics"},{"id":65146545,"name":"Health sciences/Diseases"},{"id":65146546,"name":"Health sciences/Health care"},{"id":65146547,"name":"Health sciences/Medical research"}],"tags":[],"updatedAt":"2026-03-26T12:59:59+00:00","versionOfRecord":[],"versionCreatedAt":"2026-03-26 12:59:59","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-9054955","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-9054955","identity":"rs-9054955","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}