User evaluations of AI- and human-generated responses in digital health communication: A paired survey study

User evaluations of AI- and human-generated responses in digital health communication: A paired survey study | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article User evaluations of AI- and human-generated responses in digital health communication: A paired survey study Christin Schulz, Patrycja Adämmer, Emma Fechner, Eva Kassberg, and 6 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-9252051/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 11 You are reading this latest preprint version Abstract Background Digital health communication is gaining importance as health care systems face increasing demand and structural constraints. Generative artificial intelligence (AI) tools such as ChatGPT are increasingly used for health information seeking; however, direct comparisons between AI-generated and human responses from the user perspective remain limited. Methods A quantitative exploratory cross-sectional online survey using a paired-response design was conducted in Germany. Participants evaluated two responses to the same real-world health query: one written by a physiotherapist and one generated by ChatGPT-4. Outcomes included perceived empathy, comprehensibility, potential discrimination, and expert-rated clinical quality. Paired comparisons were analyzed using Wilcoxon signed-rank tests, and logistic regression models were applied to explore associations between participant characteristics and preference patterns. Results The analytical sample comprised 224 participants. Across most items, the AI-generated response was rated more favorably than the human response, particularly with regard to comprehensibility and empathy (all p < .001). Preference patterns were highly consistent across both domains, whereas perceptions of discriminatory content were rare and did not differ significantly. Expert evaluations also favored the AI-generated response in terms of clinical accuracy and completeness, although these findings should be interpreted cautiously given the exploratory design. Conclusions The findings suggest that AI-generated communication may align closely with central user expectations in text-based health information contexts, particularly regarding clarity and supportive language. At the same time, perceived communicative strengths should be considered alongside ongoing concerns related to clinical appropriateness, safety, and equity. These results support a differentiated understanding of AI-supported communication as context-dependent rather than universally preferable. AI-generated responses may complement professional consultation by improving access to understandable health information, particularly in low-threshold informational settings. Careful integration into existing care structures will be essential to ensure that technological support enhances the quality of health care delivery without replacing professional expertise. Artificial intelligence Digital health communication Patient perspective Health information Empathy Health services research Background Digital health communication under increasing pressure Access to timely and reliable health advice is becoming more challenging across many health care systems. Workforce shortages, demographic change, and rising demand for care are contributing to structural constraints that affect both the availability and quality of professional consultation. At the same time, health communication is increasingly shifting into digital environments. In the European Union, 93% of individuals aged 16–74 years reported using the internet in 2024 (1), and among internet users, 63% searched online for health-related information such as illness, nutrition, or health promotion (2). While digital access may expand the reach of health information, it also raises important questions about the quality of communication—particularly with respect to comprehensibility, structural clarity, empathy, and the potential for biased or discriminatory content. Expansion of generative AI in health information seeking Beyond traditional digital sources such as search engines, online portals, and forums, generative artificial intelligence (AI) systems are becoming increasingly prominent in the context of health information seeking. Tools such as ChatGPT provide immediate, anonymous, and interactive responses, which may lower barriers to accessing advice. International data indicate that the use of ChatGPT for health-related questions is already widespread in the general population and varies across sociodemographic groups (3). This development aligns with the expanding use of digital counseling formats, including emerging approaches such as avatar-mediated online counseling (4). Context-dependent acceptance of digital counseling Acceptance of digital consultation is not uniform but appears to depend strongly on situational factors. Virtual formats are often perceived as appropriate for clearly defined concerns, whereas direct interpersonal communication tends to be preferred in cases involving initial contact, greater uncertainty, or complex health situations (5, 6). At the same time, qualitative research points to unintended consequences of digital consultations, including communication difficulties and the risk of disadvantaging digitally excluded populations (7). The nature of the health concern itself also appears to be relevant. Chatbots may be viewed more favorably when addressing stigmatized topics, yet skepticism tends to increase when symptoms are perceived as serious (8). Similar patterns have been described in studies focusing on sexual and reproductive health (9, 10). Taken together, these findings suggest that the perceived appropriateness of AI-supported communication is shaped by both contextual and problem-specific considerations. Perceived communication quality of AI-generated responses Emerging evidence suggests that AI-generated responses may be evaluated positively with regard to core communication characteristics. A systematic review with meta-analysis reported that in text-based study settings, AI chatbots sometimes received higher empathy ratings than human health care professionals, although methodological limitations should be taken into account (11). Analyses based on real forum questions similarly found chatbot responses to be more frequently rated as empathetic and higher in overall quality than physician responses (12). In oncological contexts, individuals affected by cancer also perceived chatbot replies as more empathetic, while highlighting differences in perspective between patients and physicians (13). At the same time, domain-specific evaluations indicate that ChatGPT can explain complex information in an accessible manner but requires systematic scrutiny with regard to accuracy (14). These findings point to a potential tension between perceived communicative strengths and the need for careful content validation. Clinical limitations, safety concerns, and bias Despite promising evaluations of communication quality, several studies emphasize limitations in the clinical appropriateness of AI-generated advice, particularly for complex medical questions in which human expertise often remains superior (15). Concerns have also been raised about inconsistent response quality and the possibility of harmful recommendations, for example when risks are insufficiently communicated (16). Importantly, users do not always recognize problematic content when it is presented in a confident and coherent manner (17). Performance analyses of frequently searched conditions, such as back pain, further demonstrate variability in both quality and readability (18). In addition, issues of fairness and equity warrant attention. Tools for systematically identifying bias and potential equity harms in large language model (LLM) outputs have been proposed, underscoring the relevance of discrimination-sensitive evaluation (19). Research gap and study objective Although a growing body of literature examines AI-supported health communication, direct comparisons between AI-generated and human responses remain limited—particularly when both are based on the same real-world user query and evaluated across multiple communication dimensions from the user perspective. Evidence therefore remains fragmented regarding how such responses are comparatively perceived in terms of empathy, comprehensibility, and non-discrimination. It is well established that empathy, comprehensibility, and non-discrimination represent central dimensions of healthcare communication and patient experience. Discrimination—understood as perceived unequal or negative treatment—is associated with reduced patient engagement, lower satisfaction, and impaired communication processes (20, 21). In contrast, empathy is a key relational component of care, strongly linked to patient satisfaction, trust, and adherence (22, 23). At the same time, comprehensibility constitutes a core element of health literacy, determining individuals’ ability to understand and apply health information for decision-making (24, 25). Together, these dimensions reflect essential structural, relational, and cognitive prerequisites for effective and equitable healthcare communication. The objective of this study was to empirically compare a ChatGPT-4 response with the response of a physiotherapist to a real question from an online health forum. By examining user evaluations across key communication dimensions, the study aims to contribute to a more differentiated understanding of the potential, limitations, and risks of AI-supported digital health communication in comparison with human consultation. Methods A quantitative exploratory cross-sectional online survey using a paired-response evaluation design was conducted. Participants evaluated two separate responses to the same health-related case example: one written by a human physiotherapist and one generated by the publicly available LLM ChatGPT-4. The case example was derived from a real post in a publicly accessible online health forum, where individuals can seek advice from health professionals. The original response provided by the physiotherapist served as the human reference response. The LLM response was generated using the publicly accessible ChatGPT-4 (OpenAI). The original forum question was entered verbatim as a single input without additional prompt instructions; no follow-up questions or persona or role prompts were used. The model output was recorded as returned. The study investigated how these responses were evaluated with regard to empathy, comprehensibility, potential discrimination, and professional quality. Data were collected in Germany using a standardized, self-administered online questionnaire. Sample A total of 224 participants completed the online survey. Participants were recruited using a convenience sampling approach through open online dissemination. Eligibility criteria included being at least 18 years old and having sufficient German language proficiency to complete the questionnaire. Individuals without internet access or sufficient German proficiency were not eligible to participate. Participation was voluntary and anonymous. Because of the open recruitment strategy, a response rate could not be calculated. Given the convenience sampling approach, selection bias cannot be excluded. No formal sample size calculation was performed; the study was designed as an exploratory survey. Data collection Data were collected between 26 January and 23 April 2025, using the online survey platform QUAMP®. The survey link was distributed through multiple channels, including websites, social media, professional and personal networks, and printed flyers. Before starting the survey, participants provided electronic informed consent after receiving study information. Both responses were presented in a fixed order, with the physiotherapist's response shown first and the ChatGPT-4-generated response shown second. No randomization or counterbalancing of the presentation order was applied. Questionnaire The survey instrument was developed based on a real case example from an online health forum. The case received responses from both a doctoral-level physiotherapist and a publicly available LLM (ChatGPT-4). Participants were asked to evaluate these responses across three dimensions: empathy (5 items), comprehensibility (7 items), and potential discrimination (1 global screening item). In addition, clinical quality was assessed by experts. The primary outcomes were perceived empathy and comprehensibility; discrimination and expert-rated clinical quality were assessed as secondary outcomes. All items were rated on five-point Likert scales ranging from 1 (“absolutely not”) to 5 (“absolutely”). For example, empathy was assessed using items such as: “Does the response demonstrate empathy?” Comprehensibility was evaluated using items such as: “Is the response easy to understand?” Potential discrimination was assessed with the item: “Does the response contain any discriminatory content?” Additionally, the professional quality of the responses was evaluated by three practicing physiotherapists using a separate online questionnaire focusing on clinical accuracy. These expert assessments were conducted independently. Given the exploratory nature of this assessment, a small expert sample was considered appropriate. The questionnaire items were developed based on the study objectives and refined through pretesting to enhance clarity and relevance. Data a nalysis Data analysis was performed using IBM SPSS Statistics (Version 29). Descriptive statistics, including means, standard deviations, and frequencies, were calculated. Depending on the scale level and distribution of the data, Pearson or Spearman correlation analyses were conducted to examine relationships between variables. Correlation coefficients were interpreted using absolute values (|r| or |rho|) as follows: values of 0.00–0.29 indicated weak correlations, 0.30–0.59 moderate correlations, and ≥0.60 strong correlations. Additionally, binary logistic regression analyses were performed to examine whether participant characteristics were associated with overall LLM dominance in empathy and comprehensibility ratings. For these analyses, dimension-level dominance variables were dichotomized into LLM dominant versus not LLM dominant (human dominant and equal combined). Predictor variables included age (continuous), sex (female/male), native German language status (yes/no), highest educational attainment (ordinal), and having a health-related professional background (yes/no). Because of the small number of participants reporting diverse sex (n = 3), regression analyses were restricted to female and male participants. Odds ratios (ORs) with 95% confidence intervals (CIs) were reported. Given the high prevalence of LLM dominance, regression results were interpreted cautiously. To compare paired ordinal ratings between the human response and the LLM response, Wilcoxon signed-rank tests were performed. A post-hoc sensitivity analysis using G*Power 3.1 (26) indicated that with a sample size of 224 paired observations, a two-tailed significance level of .05, and 80% statistical power, the Wilcoxon signed-rank test was able to detect a minimum effect size of Cohen's d = 0.19, corresponding to a small effect. Statistical significance was defined as p < .05. Missing data were not imputed; analyses were conducted using available data. As a sensitivity analysis, all Wilcoxon signed-rank tests were repeated after excluding participants who reported diverse sex (n = 3) to assess the robustness of the findings. Results remained unchanged in direction, magnitude, and significance across all comparisons. Due to the anonymous design of the online survey and the open dissemination strategy, data on the number of individuals who accessed the survey link or started but did not complete the questionnaire were not available. A participant flow diagram and a non-response analysis could therefore not be provided. Results The final analytical sample comprised 224 participants. Their mean age was 37.0 years (standard deviation, 15.8). Most participants were native German speakers (90.2%), while 9.4% reported a different first language. Regarding educational attainment, nearly half of the sample held a university degree (48.2%). Overall, 37.1% of participants had a degree in a health-related field (Table 1: Sample Characteristics). Table 1: Sample characteristics n = 224 Age ; mean (sd) 37.0 (15.8) Sex ; n (%) female 154 (68.8) male 67 (29.9) divers 3 (1.3) Native German Speaker ; n (%)* yes 202 (90.2) no 21 (9.4) Highest educational attainment ; n (%) Lower secondary school certificate 3 (1.3) Intermediate secondary school certificate 10 (4.5) University entrance qualification 45 (20.1) Vocational training 58 (25.9) University degree 108 (48.2) Degree in a health-related field ; n (%)* yes 83 (37.1) no 140 (62.5) *Up to 100% are missing values Overall, 71.9% of participants did not rate the human response as superior on any item, whereas only 6.3% did not rate the LLM-generated response as superior on at least one item. Ratings indicating equal evaluations were broadly distributed across participants, suggesting differentiated rather than polarized evaluation patterns. Across all evaluated items, 88.8% of participants tended to rate LLM-generated responses as superior to human responses, whereas 11.2% showed no such tendency (Table 2). Table 2: Comparison between Human and AI-Counseling Construct N= 224 Wilcoxon signed rank Z, p-value n (%)* Median Human better Equal LLM better Human LLM Empathy To what extent is the response formulated in a respectful manner? 6 (2.7) 41 (18.3) 164 (73.2) 3 5 -11.152, <0.001 To what extent does the response consider the patient’s needs? 6 (2.7) 28 (12.5) 177 (79.0) 2 5 -11.544, <0.001 To what extent is the response empathetic? 6 (2.7) 28 (12.5) 175 (78.1) 2 4 -11.229, <0.001 To what extent does the response demonstrate an understanding of the patient’s concerns and fears? 13 (5.8) 41 (18.3) 153 (68.3) 3 4 -10.338, <0.001 To what extent does the response demonstrate the ability to put itself in the patient’s situation? 13 (5.8) 38 (17.0) 157 (70.1) 2 4 -10.398, <0.001 Comprehensibility How understandable is the information provided? 16 (7.1) 61 (27.2) 135 (60.3) 4 5 -9.242, <0.001 How understandable is the language used (word choice and sentence structure)? 18 (8.0) 96 (42.9) 99 (44.2) 4 5 -7.430, <0.001 Were technical terms or complex expressions explained sufficiently? 16 (7.1) 49 (21.9) 96 (42.9) 4 4 -7.748, <0.001 How easy is it to follow and comprehend the information? 18 (8.0) 44 (19.6) 148 (66.1) 4 5 -9.920, <0.001 How appropriate is the length of the response? 4 (1.8) 14 (6.3) 194 (86.6) 2 4 -12.268, <0.001 How well is the response structured and organized? 2 (0.9) 31 (13.8) 174 (77.7) 2 5 -11.541, <0.001 Would you know what your next steps should be after reading the response? 16 (7.1) 47 (21.0) 142 (63.4) 3 4 -9.917, <0.001 Potential discrimination Do you perceive any wording or content in the response as discriminatory? 2 (0.9) 188 (83.9) 8 (3.6) 2 2 -1.897, 0.058 *Up to 100% are missing values, bold indicates significance at 0.05 (All reported p-values are two-tailed.) Table 3. Overall and dimension-specific dominance of participant ratings N = 224 , n (%) Dimension Human dominant Equal LLM dominant Overall 25 (11.2) – 199 (88.8) Empathy 12 (5.4) 20 (8.9) 192 (85.7) Comprehensibility 11 (4.9) 19 (8.5) 194 (86.6) Discrimination 2 (1.0) 188 (94.9) 8 (4.0) *Up to 100% are missing values At the dimension level, empathy and comprehensibility showed highly consistent patterns (Table 3). Across both domains, most participants tended to rate LLM-generated responses as superior more often than human responses, whereas only small proportions showed the opposite tendency or no clear dominance. Preference distributions were highly comparable between these two dimensions. For discrimination, assessed using a single global screening item, evaluations were predominantly neutral. Most participants rated both responses as equal, and only small proportions indicated greater discrimination in either response. For the human response, perceived discriminatory elements were, in isolated cases, attributed to age-related assumptions (n = 2) and gender-related wording (n = 2). Free-text comments mainly referred to generalizing or potentially stereotypical statements regarding physical capacity or gender role expectations. For the ChatGPT response, isolated perceptions of potentially discriminatory content were reported regarding age (n = 1), gender (n = 1), and origin or cultural background (n = 1). These perceptions primarily reflected concerns about overgeneralization or overly broad statements rather than explicitly discriminatory intent. A binary logistic regression analysis was conducted to examine whether participant characteristics were associated with overall LLM dominance in empathy ratings. The overall model was statistically significant (χ²(6) = 16.65, p = .011), explaining a small proportion of variance (Nagelkerke R² = .13). Higher educational attainment was significantly associated with higher odds of showing LLM dominance (OR = 1.86, 95% CI [1.28–2.70], p = .001). Age was also significantly associated with LLM dominance, with increasing age being linked to slightly lower odds of LLM dominance (OR = 0.97, 95% CI [0.95–1.00], p = .028). Sex, native German language status, and having a health-related professional background were not significantly associated with LLM dominance. Given the high proportion of participants showing LLM dominance, coefficient estimates should be interpreted with caution. A second binary logistic regression model examined associations between participant characteristics and LLM dominance in comprehensibility ratings. The overall model was statistically significant (χ²(6) = 13.00, p = .043), explaining a small proportion of variance (Nagelkerke R² = .11). Increasing age was significantly associated with lower odds of LLM dominance (OR = 0.97, 95% CI [0.94–0.99], p = .003). Educational attainment, sex, native German language status, and having a health-related professional background were not significantly associated with LLM dominance. As in the previous model, the high proportion of participants showing LLM dominance warrants cautious interpretation of the estimates. Discussion This study examined how AI-generated and human responses to the same real-world health query are perceived across key communication dimensions. The findings indicate a consistent preference for the AI-generated response, particularly with regard to empathy and comprehensibility, while perceptions of discrimination were largely comparable. Taken together, the results suggest that AI-supported communication may meet central user expectations in text-based health information contexts. Interpretation of principal findings One of the most notable findings is the strong preference for the AI-generated response in terms of comprehensibility. Participants frequently rated the language as clearer, the structure as more coherent, and the explanations of technical terms as more accessible. These observations align with domain-specific evaluations showing that ChatGPT can present complex information in an understandable manner ( 14 ). Given that comprehensibility is a core prerequisite for informed health decision-making, this result is particularly relevant in light of the growing reliance on digital information sources within the general population. A descriptive pattern further suggested that participants without German as a first language tended to perceive the AI response as more understandable. Although this observation should be interpreted cautiously because of the small subgroup size, it may point to a potential advantage of AI-generated language in reducing linguistic barriers. At the same time, prior research has highlighted the risk that digital formats may disadvantage already excluded populations ( 7 ). Whether AI-supported communication mitigates or reproduces such disparities remains an open question that requires further investigation. The results related to empathy are similarly noteworthy. Across all subdimensions, the AI-generated response was perceived as more empathetic. This finding is consistent with previous research reporting higher empathy ratings for chatbot responses in text-based settings ( 11 – 13 ). One possible explanation is that LLMs are optimized for supportive and validation-oriented language patterns, which may systematically enhance the perception of empathy even in the absence of genuine emotional understanding. These findings highlight a distinction between perceived and experienced empathy that warrants closer conceptual attention in future research. By contrast, perceptions of discriminatory content were rare for both responses and did not differ significantly. While this may suggest that both human and AI-generated communication can meet baseline expectations of respectful language, existing work has emphasized the importance of systematically examining bias and potential equity harms in LLM outputs ( 19 ). Although reports of potentially discriminatory content were rare and did not differ significantly between response types, these findings should be interpreted cautiously given the reliance on a single global screening item and the very small number of observed events. The present data therefore do not allow conclusions regarding the absence of bias but instead indicate that discriminatory elements were not salient in this specific case scenario. From a safety and equity perspective, this underscores the importance of systematically evaluating both human and AI-generated health communication for subtle bias and unintended stereotyping. Future research should apply multidimensional and validated discrimination or bias assessment frameworks and include multiple case scenarios to enable more robust and generalizable conclusions. Expert evaluations additionally favored the AI-generated response in terms of clinical accuracy and completeness. Although this observation contrasts with studies emphasizing limitations in the clinical appropriateness of AI-generated advice, particularly in complex cases ( 15 ), the present findings should be interpreted within the boundaries of a case-based comparison. Rather than challenging existing concerns, the results point to the need for more differentiated assessments that take task complexity and context into account. Beyond individual dimensions, the broadly distributed equal ratings suggest that participants engaged in differentiated evaluations rather than uniformly favoring one response type. This pattern supports the interpretation that user perceptions of AI-supported communication are nuanced and context-sensitive, echoing prior work describing the situational acceptance of digital consultation formats ( 5 , 6 ). The post-hoc sensitivity analysis indicated that the study was powered to detect effect sizes as small as Cohen's d = 0.17. Given that all paired comparisons for empathy and comprehensibility reached significance at p < .001, the observed effects substantially exceeded this detection threshold, supporting the robustness of the reported preference patterns. Implications for digital health communication The findings indicate that AI-generated responses may serve as a meaningful complement within digital health communication, particularly in low-threshold informational settings such as initial orientation or clarification of non-urgent concerns. Features such as consistent structure, accessible language, and supportive phrasing appear to align with user expectations in text-based environments. In health care systems facing increasing demand and resource constraints, these characteristics may help expand access to understandable health information. At the same time, the results should not be interpreted as supporting a substitution of professional consultation. Prior research has shown that acceptance of digital counseling decreases with increasing complexity and uncertainty ( 5 , 6 ), and concerns regarding inconsistent quality and potentially harmful recommendations remain relevant ( 16 , 17 ). From this perspective, AI-supported tools may be most appropriately positioned within a stepped communication model in which automated information complements, but does not replace, professional expertise. The growing presence of AI in health information seeking also raises questions about professional roles and competencies. Ensuring the safe integration of such tools will likely require not only technical oversight but also strengthened digital competencies among health care professionals. Educational approaches that enable critical appraisal of AI-generated content may therefore become increasingly important as digital counseling formats continue to expand ( 4 ). Limitations Several limitations should be considered when interpreting the findings. First, the comparison was based on a single case example and one human response, which may not fully capture the variability of professional communication styles. The results therefore reflect a case-based evaluation rather than a generalized comparison between human and AI-supported consultation. Because evaluations were based on self-reported perceptions, reporting bias cannot be excluded. Although the study was sufficiently powered to detect small effects (Cohen's d = 0.17), the ability to detect such small differences also implies that statistically significant results may not necessarily reflect practically meaningful distinctions in all cases. The clinical or communicative relevance of the observed differences should therefore be assessed independently of statistical significance. Second, the study relied on a convenience sample recruited through open online dissemination. Although this approach enabled broad participation, selection bias cannot be excluded, and the relatively high educational attainment within the sample may limit generalizability. Third, participants evaluated static written responses without the opportunity for interaction. Health communication is typically dynamic and allows for clarification, follow-up questions, and adaptation to individual needs. The extent to which the observed preferences would persist in interactive settings remains uncertain. Fourth, expert assessments were conducted with a small sample and should be interpreted as exploratory. Larger expert panels and standardized evaluation frameworks would strengthen future comparisons of clinical quality. Fifth, the two responses were presented in a fixed order without randomization or counterbalancing. This design may have introduced order effects, as participants' evaluation of the second response could have been influenced by their prior reading of the first. Although the direction of such effects cannot be determined with certainty, anchoring or contrast effects may have systematically favored or disfavored the response presented second. Future studies should employ randomized or counterbalanced presentation sequences to control for this potential source of bias. Finally, AI-generated responses are inherently sensitive to prompt design, model configuration, and ongoing technological development. The findings should therefore be understood as reflecting the capabilities of the model at the time of data collection rather than representing a fixed performance estimate. Future research Further studies should examine multiple case scenarios across different clinical domains and levels of complexity to better understand where AI-supported communication is perceived as beneficial and where its limitations become more apparent. Research incorporating interactive dialogue formats may provide additional insight into how users engage with AI-generated advice in more realistic communication settings. In addition, qualitative approaches could help clarify which linguistic and structural features shape perceptions of empathy and comprehensibility. Establishing robust evaluation criteria for AI-supported health communication remains an important step toward ensuring both quality and safety as these technologies become more widely integrated into digital care environments. Conclusion The present study suggests that AI-generated responses can meet central user expectations regarding clarity, structure, and perceived empathy in text-based health communication. At the same time, the findings highlight that perceived communicative strength should be considered alongside ongoing concerns related to clinical appropriateness, safety, and equity. Rather than positioning AI as a replacement for professional consultation, the results support a complementary perspective in which AI-assisted communication may enhance access to understandable health information while professional expertise remains essential for complex decision-making. Continued empirical evaluation will be necessary to further define appropriate roles, boundaries, and quality standards for the responsible use of AI in digital health communication. Against the backdrop of increasing demand and limited professional resources, AI-supported communication may contribute to improving access to understandable health information, particularly in low-threshold informational settings. At the same time, careful integration into existing care structures will be essential to ensure that technological support enhances, rather than fragments, the quality of health care delivery. Abbreviations AI: Artificial intelligence CI: Confidence interval ICN: International Council of Nurses ICMJE: International Committee of Medical Journal Editors LLM: Large language model OR: Odds ratio SD: Standard deviation SPSS: Statistical Package for the Social Sciences Declarations Ethics approval and consent to participate The study did not involve patients or vulnerable populations and was restricted to an anonymous online survey in which participants evaluated two pre-existing, publicly available text-based responses to a health forum query. According to the Information Sheet on the Ethics Committee Regulations of Alice Salomon University of Applied Sciences Berlin, such projects do not require formal ethical approval (27). Nevertheless, the study was conducted in accordance with internationally recognized ethical principles, including the ICN Code of Ethics and the Declaration of Helsinki. All participants were fully informed about the study’s purpose and procedures, provided informed and voluntary consent, and were free to withdraw at any time without consequences. Consent for publication Not applicable. Availability of data and materials The datasets used and analyzed during the current study are available from the corresponding author upon reasonable request. Competing interests The authors declare that they have no competing interests. Funding No specific funding was received for this research from public, commercial, or not-for-profit funding agencies. Open access publication was supported by the Open Access Publication Fund of Alice Salomon University of Applied Sciences Berlin. Authors' contribution According to ICMJE guidelines, all authors 1) have made substantial contributions to the conception and design, or acquisition of data, or analysis and interpretation of data; 2) have been involved in drafting the manuscript or revising it critically for important intellectual content; 3) have given final approval of the version to be published; and 4) agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. CS: study design, data collection, data analysis, manuscript preparation; PA: data collection, data analysis, manuscript approval; EF: data collection, data analysis, manuscript approval; EK: data collection, data analysis, manuscript approval; IL: data collection, data analysis, manuscript approval; EP: data collection, data analysis, manuscript approval; KP: data collection, data analysis, manuscript approval; LR: data collection, data analysis, manuscript approval; IT: data collection, data analysis, manuscript approval; JG: study design, data collection, data analysis, manuscript preparation. Acknowledgements Not applicable. References Eurostat. People online in 2024: Eurostat News; 2024 [updated 2024/12/17. Available from: https://ec.europa.eu/eurostat/web/products-eurostat-news/w/ddn-20241217-1. Eurostat. Digitalisation in Europe – 2025 edition: Finding health information online: Eurostat; 2025 [Available from: https://ec.europa.eu/eurostat/web/interactive-publications/digitalisation-2025. Ayre J, Cvejic E, McCaffery KJ. Use of ChatGPT to obtain health information in Australia, 2024: insights from a nationally representative survey. Medical Journal of Australia. 2025;222(4):210-2. Kiuchi K, Umehara H, Irizawa K, Kang X, Nakataki M, Yoshida M. An exploratory study of the potential of online counseling for university students by a human-operated avatar counselor. Healthcare. 2024;12(13):1287. Campbell K, Greenfield G, Li E, O’Brien N, Hayhoe B, Beaney T. The impact of virtual consultations on the quality of primary care: systematic review. Journal of Medical Internet Research. 2023;25:e48920. Moulaei K, Sheikhtaheri A, Fatehi F, Shanbehzadeh M, Bahaadinbeigy K. Patients’ perspectives and preferences toward telemedicine versus in-person visits: A mixed-methods study on 1226 patients. BMC Medical Informatics and Decision Making. 2023;23(1):261. Turner A, Morris R, Rakhra D, Stevenson F, Hamilton F, Atherton H. Unintended consequences of online consultations: a qualitative study in UK primary care. British Journal of General Practice. 2022;72(715):e128-e37. Miles O, West R, Nadarzynski T. Health chatbots acceptability moderated by perceived stigma and severity: a cross-sectional survey. Digital Health. 2021;7:20552076211063012. Nadarzynski T, Bayley J, Llewellyn C, Kidsley S, Graham CA. Acceptability of artificial intelligence-enabled chatbots, video consultations and live webchats as online platforms for sexual health advice. BMJ Sexual & Reproductive Health. 2020;46(3):210-7. Nadarzynski T, Puentes V, Pawlak I, Mendes T, Montgomery I, Bayley J, et al. Barriers and facilitators to engagement with artificial intelligence-based chatbots for sexual and reproductive health advice. Sexual Health. 2021;18(5):385-93. Howcroft A, Bennett-Weston A, Khan A, Griffiths J, Gay S, Howick J. AI chatbots versus human healthcare professionals: A systematic review and meta-analysis of empathy in patient care. British Medical Bulletin. 2025;156(1):ldaf017. Ayers JW, Poliak A, Dredze M, Leas EC, Zhu Z, Kelley JB. Comparing physician and artificial intelligence chatbot responses to patient questions posted to a public social media forum. JAMA Internal Medicine. 2023;183(6):589-96. Chen D, Chauhan K, Parsa R, Liu ZA, Liu FF, Mak E. Patient perceptions of empathy in physician and artificial intelligence chatbot responses to patient questions about cancer. NPJ Digital Medicine. 2025;8(1):275. Mathes S, Seurig S, Bluhme F, Beyer K, Heizmann F, Wagner M. ChatGPT performance on 120 interdisciplinary allergology questions—systematic evaluation with clinical error impact assessment for critical erroneous AI-guided chatbot advice. Journal of Allergy and Clinical Immunology: In Practice. 2025;13(6):1350-7.e4. Jo E, Song S, Kim JH, Lim S, Kim JH, Cha JJ. Assessing GPT-4’s performance in delivering medical advice: Comparative analysis with human experts. JMIR Medical Education. 2024;10:e51282. Wilhelm TI, Roos J, Kaczmarczyk R. Large language models for therapy recommendations across 3 clinical specialties: Comparative study. Journal of Medical Internet Research. 2023;25:e49324. Armbruster J, Bussmann F, Rothhaas C, Titze N, Grützner PA, Freischmidt H. “Doctor ChatGPT, can you help me?” The patient’s perspective: Cross-sectional study. Journal of Medical Internet Research. 2024;26:e58831. Scaff SPS, Reis FJJ, Ferreira GE, Jacob MF, Saragiotto BT. Assessing the performance of AI chatbots in answering patients’ common questions about low back pain. Annals of the Rheumatic Diseases. 2025;84(1):143-9. Pfohl SR, Cole-Lewis H, Sayres R. A toolbox for surfacing health equity harms and biases in large language models. Nature Medicine. 2024;30(12):3590-600. Hausmann LRM, Hannon MJ, Kresevic DM, Hanusa BH, Kwoh CK, Ibrahim SA. Impact of perceived discrimination in health care on patient–provider communication. Medical Care. 2011;49(7):626-33. von dem Knesebeck O, Dingoyan D, Makowski A, Klein J, Ludecke D. Intersectional inequalities in interpersonal discrimination in outpatient care according to sex, history of migration, and income in Germany. European Journal of Public Health. 2025;36(1):20-4. Babaii A, Mohammadi E, Sadooghiasl A. The meaning of the empathetic nurse–patient communication: A qualitative study. Journal of Patient Experience. 2021;8:1-9. Campos CFC, Olivo CR, Martins MdA, Tempski PZ. Physicians’ attention to patients’ communication cues can improve patient satisfaction with care and perception of physicians’ empathy. Clinics. 2024;79:100377. Sørensen K, Van den Broucke S, Fullam J, Doyle G, Pelikan J, Slonska Z, et al. Health literacy and public health: A systematic review and integration of definitions and models. BMC Public Health. 2012;12:80. Szabó P, Bíró É, Kósa K. Readability and comprehensibility of printed patient education materials. Frontiers in Public Health. 2021;9:725840. Faul F, Erdfelder E, Lang A-G, Buchner A. G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods. 2007;39(2):175-91. Borde T. Satzung der Ethikkommission der Alice Salomon Hochschule Berlin [Regulations governing the Ethics Committee of Alice Salomon University of Applied Sciences Berlin]. Alice Salomon Hochschule Berlin; 2019. Additional Declarations No competing interests reported. Supplementary Files AdditionalMaterialCaseResponses.docx Cite Share Download PDF Status: Under Review Version 1 posted Reviews received at journal 18 May, 2026 Reviews received at journal 07 May, 2026 Reviewers agreed at journal 07 May, 2026 Reviewers agreed at journal 04 May, 2026 Reviews received at journal 29 Apr, 2026 Reviewers agreed at journal 27 Apr, 2026 Reviewers invited by journal 27 Apr, 2026 Editor invited by journal 02 Apr, 2026 Editor assigned by journal 30 Mar, 2026 Submission checks completed at journal 30 Mar, 2026 First submitted to journal 28 Mar, 2026 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-9252051","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":634787207,"identity":"af6c80bc-e91a-4100-b6c7-35f6c0a93f03","order_by":0,"name":"Christin Schulz","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABIElEQVRIie3RsUrDQBjA8a8cnMtXs15pSV7hJGAdfJgLQrq0JdClg4QDoZtkDdiX8A0uBHSJuGYQQQpOHXSLFIp3kbiYZHa4/3CQgx+5+w7AZvuPMQBVrwAUIAIXAIHorwntJaIhHPyGYB8BEPBLAtmQLuHc3b5lX9XFEpz8fYc8niXJkyLR+hVPx3Kwa4Hs5ZHnKNgKWDj1kdNFWi4FSYsV0okifgvhLIRc3yWQDOkYOS5kidPDcCOQMqF32klWGeIUhrCZ91xwMjzW5OTQQRQaAnNDuOBqron8+Qtpu0upD4YhCzYsPB9tuTi7LzVJHwwJbkbblomlIfmsLuMg0RNj+3XsueZg0bVwvfQq+9h3jRrqR/nTQPYAm81ms/X0DVhDVBooGwSqAAAAAElFTkSuQmCC","orcid":"","institution":"Alice Salomon University of Applied Sciences Berlin","correspondingAuthor":true,"prefix":"","firstName":"Christin","middleName":"","lastName":"Schulz","suffix":""},{"id":634787208,"identity":"55d26068-c458-48d7-9f01-caa734f2b737","order_by":1,"name":"Patrycja Adämmer","email":"","orcid":"","institution":"Alice Salomon University of Applied Sciences Berlin","correspondingAuthor":false,"prefix":"","firstName":"Patrycja","middleName":"","lastName":"Adämmer","suffix":""},{"id":634787209,"identity":"3139ea1f-f6bc-4ac6-93f2-2054c3a3800a","order_by":2,"name":"Emma Fechner","email":"","orcid":"","institution":"Alice Salomon University of Applied Sciences Berlin","correspondingAuthor":false,"prefix":"","firstName":"Emma","middleName":"","lastName":"Fechner","suffix":""},{"id":634787210,"identity":"636b249d-5eea-4876-8ada-a80d1affedeb","order_by":3,"name":"Eva Kassberg","email":"","orcid":"","institution":"Alice Salomon University of Applied Sciences Berlin","correspondingAuthor":false,"prefix":"","firstName":"Eva","middleName":"","lastName":"Kassberg","suffix":""},{"id":634787211,"identity":"46e3014a-d354-46fc-82f4-55ef08724ed9","order_by":4,"name":"Ida Laux","email":"","orcid":"","institution":"Alice Salomon University of Applied Sciences Berlin","correspondingAuthor":false,"prefix":"","firstName":"Ida","middleName":"","lastName":"Laux","suffix":""},{"id":634787212,"identity":"841e0253-a6fc-4a87-9cce-bb1925d01687","order_by":5,"name":"Ella Pollatscheck","email":"","orcid":"","institution":"Alice Salomon University of Applied Sciences Berlin","correspondingAuthor":false,"prefix":"","firstName":"Ella","middleName":"","lastName":"Pollatscheck","suffix":""},{"id":634787213,"identity":"407f248d-2dd8-449c-96ad-731781aa1a0e","order_by":6,"name":"Katja Porsch","email":"","orcid":"","institution":"Alice Salomon University of Applied Sciences Berlin","correspondingAuthor":false,"prefix":"","firstName":"Katja","middleName":"","lastName":"Porsch","suffix":""},{"id":634787214,"identity":"56397377-5654-46f4-b884-ad68fcc3354a","order_by":7,"name":"Lady Romamti","email":"","orcid":"","institution":"Alice Salomon University of Applied Sciences Berlin","correspondingAuthor":false,"prefix":"","firstName":"Lady","middleName":"","lastName":"Romamti","suffix":""},{"id":634787215,"identity":"7468fcf3-aa34-458b-9d3f-cbddaed72857","order_by":8,"name":"Iryna Tymchuk","email":"","orcid":"","institution":"Alice Salomon University of Applied Sciences Berlin","correspondingAuthor":false,"prefix":"","firstName":"Iryna","middleName":"","lastName":"Tymchuk","suffix":""},{"id":634787216,"identity":"07e8caf7-c677-4dc7-8c7b-0f3ce79e1afb","order_by":9,"name":"Johannes Gräske","email":"","orcid":"","institution":"Alice Salomon University of Applied Sciences Berlin","correspondingAuthor":false,"prefix":"","firstName":"Johannes","middleName":"","lastName":"Gräske","suffix":""}],"badges":[],"createdAt":"2026-03-28 10:53:17","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-9252051/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-9252051/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":108805212,"identity":"a1269ada-066b-418a-aefd-e2292e2062d8","added_by":"auto","created_at":"2026-05-08 15:25:13","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":338331,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-9252051/v1/0a58b05d-6a67-4102-8d01-9d2ce9138607.pdf"},{"id":108637598,"identity":"056f0dd4-f3ed-49f4-8213-8ed4d2eb7893","added_by":"auto","created_at":"2026-05-06 18:31:45","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":19279,"visible":true,"origin":"","legend":"","description":"","filename":"AdditionalMaterialCaseResponses.docx","url":"https://assets-eu.researchsquare.com/files/rs-9252051/v1/d39cdbcf0fa872bec53cddee.docx"}],"financialInterests":"No competing interests reported.","formattedTitle":"User evaluations of AI- and human-generated responses in digital health communication: A paired survey study","fulltext":[{"header":"Background","content":"\u003cp\u003e\u003cstrong\u003eDigital health communication under increasing pressure\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAccess to timely and reliable health advice is becoming more challenging across many health care systems. Workforce shortages, demographic change, and rising demand for care are contributing to structural constraints that affect both the availability and quality of professional consultation. At the same time, health communication is increasingly shifting into digital environments. In the European Union, 93% of individuals aged 16\u0026ndash;74 years reported using the internet in 2024 (1), and among internet users, 63% searched online for health-related information such as illness, nutrition, or health promotion (2).\u003c/p\u003e\n\u003cp\u003eWhile digital access may expand the reach of health information, it also raises important questions about the quality of communication\u0026mdash;particularly with respect to comprehensibility, structural clarity, empathy, and the potential for biased or discriminatory content.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eExpansion of generative AI in health information seeking\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eBeyond traditional digital sources such as search engines, online portals, and forums, generative artificial intelligence (AI) systems are becoming increasingly prominent in the context of health information seeking. Tools such as ChatGPT provide immediate, anonymous, and interactive responses, which may lower barriers to accessing advice. International data indicate that the use of ChatGPT for health-related questions is already widespread in the general population and varies across sociodemographic groups (3). This development aligns with the expanding use of digital counseling formats, including emerging approaches such as avatar-mediated online counseling (4).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eContext-dependent acceptance of digital counseling\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAcceptance of digital consultation is not uniform but appears to depend strongly on situational factors. Virtual formats are often perceived as appropriate for clearly defined concerns, whereas direct interpersonal communication tends to be preferred in cases involving initial contact, greater uncertainty, or complex health situations (5, 6). At the same time, qualitative research points to unintended consequences of digital consultations, including communication difficulties and the risk of disadvantaging digitally excluded populations (7).\u003c/p\u003e\n\u003cp\u003eThe nature of the health concern itself also appears to be relevant. Chatbots may be viewed more favorably when addressing stigmatized topics, yet skepticism tends to increase when symptoms are perceived as serious (8). Similar patterns have been described in studies focusing on sexual and reproductive health (9, 10). Taken together, these findings suggest that the perceived appropriateness of AI-supported communication is shaped by both contextual and problem-specific considerations.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePerceived communication quality of AI-generated responses\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eEmerging evidence suggests that AI-generated responses may be evaluated positively with regard to core communication characteristics. A systematic review with meta-analysis reported that in text-based study settings, AI chatbots sometimes received higher empathy ratings than human health care professionals, although methodological\u0026nbsp;limitations should be taken into account (11). Analyses based on real forum questions similarly found chatbot responses to be more frequently rated as empathetic and higher in overall quality than physician responses (12). In oncological contexts, individuals affected by cancer also perceived chatbot replies as more empathetic, while highlighting differences in perspective between patients and physicians (13).\u003c/p\u003e\n\u003cp\u003eAt the same time, domain-specific evaluations indicate that ChatGPT can explain complex information in an accessible manner but requires systematic scrutiny with regard to accuracy (14). These findings point to a potential tension between perceived communicative strengths and the need for careful content validation.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eClinical limitations, safety concerns, and bias\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eDespite promising evaluations of communication quality, several studies emphasize limitations in the clinical appropriateness of AI-generated advice, particularly for complex medical questions in which human expertise often remains superior (15). Concerns have also been raised about inconsistent response quality and the possibility of harmful recommendations, for example when risks are insufficiently communicated (16). Importantly, users do not always recognize problematic content when it is presented in a confident and coherent manner (17). Performance analyses of frequently searched conditions, such as back pain, further demonstrate variability in both quality and readability (18).\u003c/p\u003e\n\u003cp\u003eIn addition, issues of fairness and equity warrant attention. Tools for systematically identifying bias and potential equity harms in large language model (LLM) outputs have been proposed, underscoring the relevance of discrimination-sensitive evaluation (19).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eResearch gap and study objective\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAlthough a growing body of literature examines AI-supported health communication, direct comparisons between AI-generated and human responses remain limited\u0026mdash;particularly when both are based on the same real-world user query and evaluated across multiple communication dimensions from the user perspective. Evidence therefore remains fragmented regarding how such responses are comparatively perceived in terms of empathy, comprehensibility, and non-discrimination.\u0026nbsp;It is well established that empathy, \u0026nbsp; comprehensibility, and non-discrimination represent central dimensions of healthcare communication and patient experience. Discrimination\u0026mdash;understood as perceived unequal or negative treatment\u0026mdash;is associated with reduced patient engagement, lower satisfaction, and impaired communication processes (20, 21). In contrast, empathy is a key relational component of care, strongly linked to patient satisfaction, trust, and adherence (22, 23). At the same time, comprehensibility constitutes a core element of health literacy, determining individuals\u0026rsquo; ability to understand and apply health information for decision-making (24, 25). Together, these dimensions reflect essential structural, relational, and cognitive prerequisites for effective and equitable healthcare communication.\u003c/p\u003e\n\u003cp\u003eThe objective of this study was to empirically compare a ChatGPT-4 response with the response of a physiotherapist to a real question from an online health forum. By examining user evaluations across key communication dimensions, the study aims to contribute to a more differentiated understanding of the potential, limitations, and risks of AI-supported digital health communication in comparison with human consultation.\u003c/p\u003e"},{"header":"Methods","content":"\u003cp\u003eA quantitative\u0026nbsp;exploratory\u0026nbsp;cross-sectional online survey using a paired-response evaluation design was conducted. Participants evaluated two separate responses to the same health-related case example: one written by a human physiotherapist and one generated by the publicly available LLM ChatGPT-4. The case example was derived from a real post in a publicly accessible online health forum, where individuals can seek advice from health professionals. The original response provided by the physiotherapist served as the human reference response. The LLM response was generated using the publicly accessible ChatGPT-4 (OpenAI). The original forum question was entered verbatim as a single input without additional prompt instructions; no follow-up questions or persona or role prompts were used. The model output was recorded as returned. The study investigated how these responses were evaluated with regard to empathy, comprehensibility, potential discrimination, and professional quality. Data were collected in Germany using a standardized, self-administered online questionnaire.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eSample\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eA total of 224 participants completed the online survey. Participants were recruited using a convenience sampling approach through open online dissemination. Eligibility criteria included being at least 18 years old and having sufficient German language proficiency to complete the questionnaire. Individuals without internet access or sufficient German proficiency were not eligible to participate. Participation was voluntary and anonymous. Because of the open recruitment strategy, a response rate could not be calculated. Given the convenience sampling approach, selection bias cannot be excluded. No formal sample size calculation was performed; the study was designed as an exploratory survey.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eData collection\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eData were collected between 26 January and 23 April 2025, using the online survey platform QUAMP\u0026reg;. The survey link was distributed through multiple channels, including websites, social media, professional and personal networks, and printed flyers. Before starting the survey, participants provided electronic informed consent after receiving study information. Both responses were presented in a fixed order, with the physiotherapist\u0026apos;s response shown first and the ChatGPT-4-generated response shown second. No randomization or counterbalancing of the presentation order was applied.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eQuestionnaire\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe survey instrument was developed based on a real case example from an online health forum. The case received responses from both a doctoral-level physiotherapist and a publicly available LLM (ChatGPT-4). Participants were asked to evaluate these responses across three dimensions: empathy (5 items), comprehensibility (7 items), and potential discrimination (1 global screening item). In addition, clinical quality was assessed by experts. The primary outcomes were perceived empathy and comprehensibility; discrimination and expert-rated clinical quality were assessed as secondary outcomes. All items were rated on five-point Likert scales ranging from 1 (\u0026ldquo;absolutely not\u0026rdquo;) to 5 (\u0026ldquo;absolutely\u0026rdquo;).\u003c/p\u003e\n\u003cp\u003eFor example, empathy was assessed using items such as: \u0026ldquo;Does the response demonstrate empathy?\u0026rdquo; Comprehensibility was evaluated using items such as: \u0026ldquo;Is the response easy to understand?\u0026rdquo; Potential discrimination was assessed with the item: \u0026ldquo;Does the response contain any discriminatory content?\u0026rdquo;\u003c/p\u003e\n\u003cp\u003eAdditionally, the professional quality of the responses was evaluated by three practicing physiotherapists using a separate online questionnaire focusing on clinical accuracy. These expert assessments were conducted independently. Given the exploratory nature of this assessment, a small expert sample was considered appropriate. The questionnaire items were developed based on the study objectives and refined through pretesting to enhance clarity and relevance.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eData\u0026nbsp;\u003c/strong\u003e\u003cstrong\u003ea\u003c/strong\u003e\u003cstrong\u003enalysis\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eData analysis was performed using IBM SPSS Statistics (Version 29). Descriptive statistics, including means, standard deviations, and frequencies, were calculated. Depending on the scale level and distribution of the data, Pearson or Spearman correlation analyses were conducted to examine relationships between variables. Correlation coefficients were interpreted using absolute values (|r| or |rho|) as follows: values of 0.00\u0026ndash;0.29 indicated weak correlations, 0.30\u0026ndash;0.59 moderate correlations, and \u0026ge;0.60 strong correlations. Additionally, binary logistic regression analyses were performed to examine whether participant characteristics were associated with overall LLM dominance in empathy and comprehensibility ratings. For these analyses, dimension-level dominance variables were dichotomized into LLM dominant versus not LLM dominant (human dominant and equal combined). Predictor variables included age (continuous), sex (female/male), native German language status (yes/no), highest educational attainment (ordinal), and having a health-related professional background (yes/no). Because of the small number of participants reporting diverse sex (n = 3), regression analyses were restricted to female and male participants. Odds ratios (ORs) with 95% confidence intervals (CIs) were reported. Given the high prevalence of LLM dominance, regression results were interpreted cautiously. To compare paired ordinal ratings between the human response and the LLM response, Wilcoxon signed-rank tests were performed. A post-hoc sensitivity analysis using G*Power 3.1 (26) indicated that with a sample size of 224 paired observations, a two-tailed significance level of .05, and 80% statistical power, the Wilcoxon signed-rank test was able to detect a minimum effect size of Cohen\u0026apos;s d = 0.19, corresponding to a small effect. Statistical significance was defined as p \u0026lt; .05. Missing data were not imputed; analyses were conducted using available data. As a sensitivity analysis, all Wilcoxon signed-rank tests were repeated after excluding participants who reported diverse sex (n = 3) to assess the robustness of the findings. Results remained unchanged in direction, magnitude, and significance across all comparisons. Due to the anonymous design of the online survey and the open dissemination strategy, data on the number of individuals who accessed the survey link or started but did not complete the questionnaire were not available. A participant flow diagram and a non-response analysis could therefore not be provided.\u003c/p\u003e"},{"header":"Results","content":"\u003cp\u003eThe final analytical sample comprised 224 participants. Their mean age was 37.0 years (standard deviation, 15.8). Most participants were native German speakers (90.2%), while 9.4% reported a different first language. Regarding educational attainment, nearly half of the sample held a university degree (48.2%). Overall, 37.1% of participants had a degree in a health-related field (Table 1: Sample Characteristics).\u003c/p\u003e\n\u003cp\u003eTable 1: Sample characteristics\u003c/p\u003e\n\u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 75%;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 25%;\"\u003e\n \u003cp\u003e\u003cstrong\u003en = 224\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 75%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eAge\u003c/strong\u003e; mean (sd)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 25%;\"\u003e\n \u003cp\u003e37.0 (15.8)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 75%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eSex\u003c/strong\u003e; n (%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 25%;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 75%;\"\u003e\n \u003cp\u003efemale\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 25%;\"\u003e\n \u003cp\u003e154 (68.8)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 75%;\"\u003e\n \u003cp\u003emale\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 25%;\"\u003e\n \u003cp\u003e67 (29.9)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 75%;\"\u003e\n \u003cp\u003edivers\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 25%;\"\u003e\n \u003cp\u003e3 (1.3)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 75%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eNative German Speaker\u003c/strong\u003e; n (%)*\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 25%;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 75%;\"\u003e\n \u003cp\u003eyes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 25%;\"\u003e\n \u003cp\u003e202 (90.2)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 75%;\"\u003e\n \u003cp\u003eno\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 25%;\"\u003e\n \u003cp\u003e21 (9.4)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 75%;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003cstrong\u003eHighest educational attainment\u003c/strong\u003e; n (%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 25%;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 75%;\"\u003e\n \u003cp\u003eLower secondary school certificate\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 25%;\"\u003e\n \u003cp\u003e3 (1.3)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 75%;\"\u003e\n \u003cp\u003eIntermediate secondary school certificate\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 25%;\"\u003e\n \u003cp\u003e10 (4.5)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 75%;\"\u003e\n \u003cp\u003eUniversity entrance qualification\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 25%;\"\u003e\n \u003cp\u003e45 (20.1)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 75%;\"\u003e\n \u003cp\u003eVocational training\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 25%;\"\u003e\n \u003cp\u003e58 (25.9)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 75%;\"\u003e\n \u003cp\u003eUniversity degree\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 25%;\"\u003e\n \u003cp\u003e108 (48.2)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 75%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eDegree in a health-related field\u003c/strong\u003e; n (%)*\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 25%;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 75%;\"\u003e\n \u003cp\u003eyes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 25%;\"\u003e\n \u003cp\u003e83 (37.1)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 75%;\"\u003e\n \u003cp\u003eno\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 25%;\"\u003e\n \u003cp\u003e140 (62.5)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e*Up to 100% are missing values\u003c/p\u003e\n\u003cp\u003eOverall, 71.9% of participants did not rate the human response as superior on any item, whereas only 6.3% did not rate the LLM-generated response as superior on at least one item. Ratings indicating equal evaluations were broadly distributed across participants, suggesting differentiated rather than polarized evaluation patterns. Across all evaluated items, 88.8% of participants tended to rate LLM-generated responses as superior to human responses, whereas 11.2% showed no such tendency (Table 2).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eTable 2: Comparison between Human and AI-Counseling\u003c/p\u003e\n\u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\" width=\"964\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd rowspan=\"3\" valign=\"top\" style=\"width: 425px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cstrong\u003eConstruct\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd colspan=\"6\" style=\"width: 416px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eN= 224\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd rowspan=\"3\" style=\"width: 123px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eWilcoxon signed rank Z, p-value\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd colspan=\"3\" style=\"width: 237px;\"\u003e\n \u003cp\u003e\u003cstrong\u003en (%)*\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd colspan=\"3\" style=\"width: 179px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eMedian\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eHuman better\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eEqual\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd colspan=\"2\" style=\"width: 83px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eLLM better\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eHuman\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eLLM\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 425px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eEmpathy\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd colspan=\"2\" style=\"width: 83px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 83px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 83px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 123px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 425px;\"\u003e\n \u003cp\u003eTo what extent is the response formulated in a respectful manner?\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e6 (2.7)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e41 (18.3)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd colspan=\"2\" style=\"width: 83px;\"\u003e\n \u003cp\u003e164 (73.2)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e5\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 123px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e-11.152, \u0026lt;0.001\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 425px;\"\u003e\n \u003cp\u003eTo what extent does the response consider the patient\u0026rsquo;s needs?\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e6 (2.7)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e28 (12.5)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd colspan=\"2\" style=\"width: 83px;\"\u003e\n \u003cp\u003e177 (79.0)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e5\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 123px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e-11.544, \u0026lt;0.001\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 425px;\"\u003e\n \u003cp\u003eTo what extent is the response empathetic?\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e6 (2.7)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e28 (12.5)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd colspan=\"2\" style=\"width: 83px;\"\u003e\n \u003cp\u003e175 (78.1)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 123px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e-11.229, \u0026lt;0.001\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 425px;\"\u003e\n \u003cp\u003eTo what extent does the response demonstrate an understanding of the patient\u0026rsquo;s concerns and fears?\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e13 (5.8)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e41 (18.3)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd colspan=\"2\" style=\"width: 83px;\"\u003e\n \u003cp\u003e153 (68.3)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 123px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e-10.338, \u0026lt;0.001\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 425px;\"\u003e\n \u003cp\u003eTo what extent does the response demonstrate the ability to put itself in the patient\u0026rsquo;s situation?\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e13 (5.8)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e38 (17.0)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd colspan=\"2\" style=\"width: 83px;\"\u003e\n \u003cp\u003e157 (70.1)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 123px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e-10.398, \u0026lt;0.001\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 425px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eComprehensibility\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd colspan=\"2\" style=\"width: 83px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 123px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 425px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eHow understandable is the information provided?\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e16 (7.1)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e61 (27.2)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd colspan=\"2\" style=\"width: 83px;\"\u003e\n \u003cp\u003e135 (60.3)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e5\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 123px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e-9.242, \u0026lt;0.001\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 425px;\"\u003e\n \u003cp\u003eHow understandable is the language used (word choice and sentence structure)?\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e18 (8.0)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e96 (42.9)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd colspan=\"2\" style=\"width: 83px;\"\u003e\n \u003cp\u003e99 (44.2)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e5\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 123px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e-7.430, \u0026lt;0.001\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 425px;\"\u003e\n \u003cp\u003eWere technical terms or complex expressions explained sufficiently?\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e16 (7.1)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e49 (21.9)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd colspan=\"2\" style=\"width: 83px;\"\u003e\n \u003cp\u003e96 (42.9)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 123px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e-7.748, \u0026lt;0.001\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 425px;\"\u003e\n \u003cp\u003eHow easy is it to follow and comprehend the information?\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e18 (8.0)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e44 (19.6)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd colspan=\"2\" style=\"width: 83px;\"\u003e\n \u003cp\u003e148 (66.1)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e5\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 123px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e-9.920, \u0026lt;0.001\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 425px;\"\u003e\n \u003cp\u003eHow appropriate is the length of the response?\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e4 (1.8)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e14 (6.3)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd colspan=\"2\" style=\"width: 83px;\"\u003e\n \u003cp\u003e194 (86.6)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 123px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e-12.268, \u0026lt;0.001\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 425px;\"\u003e\n \u003cp\u003eHow well is the response structured and organized?\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e2 (0.9)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e31 (13.8)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd colspan=\"2\" style=\"width: 83px;\"\u003e\n \u003cp\u003e174 (77.7)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e5\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 123px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e-11.541, \u0026lt;0.001\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 425px;\"\u003e\n \u003cp\u003eWould you know what your next steps should be after reading the response?\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e16 (7.1)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e47 (21.0)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd colspan=\"2\" style=\"width: 83px;\"\u003e\n \u003cp\u003e142 (63.4)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 123px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e-9.917, \u0026lt;0.001\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 425px;\"\u003e\n \u003cp\u003e\u003cstrong\u003ePotential discrimination\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd colspan=\"2\" style=\"width: 83px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 123px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 425px;\"\u003e\n \u003cp\u003eDo you perceive any wording or content in the response as discriminatory?\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e2 (0.9)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e188 (83.9)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd colspan=\"2\" style=\"width: 83px;\"\u003e\n \u003cp\u003e8 (3.6)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 83px;\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 123px;\"\u003e\n \u003cp\u003e-1.897, 0.058\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e*Up to 100% are missing values, bold indicates significance at 0.05 (All reported p-values are two-tailed.)\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eTable 3. Overall and dimension-specific dominance of participant ratings\u003c/p\u003e\n\u003ctable border=\"0\" cellspacing=\"3\" cellpadding=\"0\"\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 144px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd colspan=\"3\" style=\"width: 255px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eN = 224\u003c/strong\u003e, n (%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 144px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eDimension\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 72px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eHuman dominant\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 90px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eEqual\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 89px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eLLM dominant\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 144px;\"\u003e\n \u003cp\u003eOverall\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 72px;\"\u003e\n \u003cp\u003e25 (11.2)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 90px;\"\u003e\n \u003cp\u003e\u0026ndash;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 89px;\"\u003e\n \u003cp\u003e199 (88.8)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 144px;\"\u003e\n \u003cp\u003eEmpathy\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 72px;\"\u003e\n \u003cp\u003e12 (5.4)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 90px;\"\u003e\n \u003cp\u003e20 (8.9)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 89px;\"\u003e\n \u003cp\u003e192 (85.7)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 144px;\"\u003e\n \u003cp\u003eComprehensibility\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 72px;\"\u003e\n \u003cp\u003e11 (4.9)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 90px;\"\u003e\n \u003cp\u003e19 (8.5)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 89px;\"\u003e\n \u003cp\u003e194 (86.6)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 144px;\"\u003e\n \u003cp\u003eDiscrimination\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 72px;\"\u003e\n \u003cp\u003e2 (1.0)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 90px;\"\u003e\n \u003cp\u003e188 (94.9)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 89px;\"\u003e\n \u003cp\u003e8 (4.0)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e*Up to 100% are missing values\u003c/p\u003e\n\u003cp\u003eAt the dimension level, empathy and comprehensibility showed highly consistent patterns (Table 3). Across both domains, most participants tended to rate LLM-generated responses as superior more often than human responses, whereas only small proportions showed the opposite tendency or no clear dominance. Preference distributions were highly comparable between these two dimensions. For discrimination, assessed using a single global screening item, evaluations were predominantly neutral. Most participants rated both responses as equal, and only small proportions indicated greater discrimination in either response. For the human response, perceived discriminatory elements were, in isolated cases, attributed to age-related assumptions (n = 2) and gender-related wording (n = 2). Free-text comments mainly referred to generalizing or potentially stereotypical statements regarding physical capacity or gender role expectations.\u003c/p\u003e\n\u003cp\u003eFor the ChatGPT response, isolated perceptions of potentially discriminatory content were reported regarding age (n = 1), gender (n = 1), and origin or cultural background (n = 1). These perceptions primarily reflected concerns about overgeneralization or overly broad statements rather than explicitly discriminatory intent.\u003c/p\u003e\n\u003cp\u003eA binary logistic regression analysis was conducted to examine whether participant characteristics were associated with overall LLM dominance in empathy ratings. The overall model was statistically significant (\u0026chi;\u0026sup2;(6) = 16.65, p = .011), explaining a small proportion of variance (Nagelkerke R\u0026sup2; = .13). Higher educational attainment was significantly associated with higher odds of showing LLM dominance (OR = 1.86, 95% CI [1.28\u0026ndash;2.70], p = .001). Age was also significantly associated with LLM dominance, with increasing age being linked to slightly lower odds of LLM dominance (OR = 0.97, 95% CI [0.95\u0026ndash;1.00], p = .028). Sex, native German language status, and having a health-related professional background were not significantly associated with LLM dominance. Given the high proportion of participants showing LLM dominance, coefficient estimates should be interpreted with caution.\u003c/p\u003e\n\u003cp\u003eA second binary logistic regression model examined associations between participant characteristics and LLM dominance in comprehensibility ratings. The overall model was statistically significant (\u0026chi;\u0026sup2;(6) = 13.00, p = .043), explaining a small proportion of variance (Nagelkerke R\u0026sup2; = .11). Increasing age was significantly associated with lower odds of LLM dominance (OR = 0.97, 95% CI [0.94\u0026ndash;0.99], p = .003). Educational attainment, sex, native German language status, and having a health-related professional background were not significantly associated with LLM dominance. As in the previous model, the high proportion of participants showing LLM dominance warrants cautious interpretation of the estimates.\u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003eThis study examined how AI-generated and human responses to the same real-world health query are perceived across key communication dimensions. The findings indicate a consistent preference for the AI-generated response, particularly with regard to empathy and comprehensibility, while perceptions of discrimination were largely comparable. Taken together, the results suggest that AI-supported communication may meet central user expectations in text-based health information contexts.\u003c/p\u003e \u003cdiv id=\"Sec15\" class=\"Section2\"\u003e \u003ch2\u003eInterpretation of principal findings\u003c/h2\u003e \u003cp\u003eOne of the most notable findings is the strong preference for the AI-generated response in terms of comprehensibility. Participants frequently rated the language as clearer, the structure as more coherent, and the explanations of technical terms as more accessible. These observations align with domain-specific evaluations showing that ChatGPT can present complex information in an understandable manner (\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e). Given that comprehensibility is a core prerequisite for informed health decision-making, this result is particularly relevant in light of the growing reliance on digital information sources within the general population.\u003c/p\u003e \u003cp\u003e A descriptive pattern further suggested that participants without German as a first language tended to perceive the AI response as more understandable. Although this observation should be interpreted cautiously because of the small subgroup size, it may point to a potential advantage of AI-generated language in reducing linguistic barriers. At the same time, prior research has highlighted the risk that digital formats may disadvantage already excluded populations (\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e). Whether AI-supported communication mitigates or reproduces such disparities remains an open question that requires further investigation.\u003c/p\u003e \u003cp\u003eThe results related to empathy are similarly noteworthy. Across all subdimensions, the AI-generated response was perceived as more empathetic. This finding is consistent with previous research reporting higher empathy ratings for chatbot responses in text-based settings (\u003cspan additionalcitationids=\"CR12\" citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e). One possible explanation is that LLMs are optimized for supportive and validation-oriented language patterns, which may systematically enhance the perception of empathy even in the absence of genuine emotional understanding. These findings highlight a distinction between perceived and experienced empathy that warrants closer conceptual attention in future research.\u003c/p\u003e \u003cp\u003eBy contrast, perceptions of discriminatory content were rare for both responses and did not differ significantly. While this may suggest that both human and AI-generated communication can meet baseline expectations of respectful language, existing work has emphasized the importance of systematically examining bias and potential equity harms in LLM outputs (\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e). Although reports of potentially discriminatory content were rare and did not differ significantly between response types, these findings should be interpreted cautiously given the reliance on a single global screening item and the very small number of observed events. The present data therefore do not allow conclusions regarding the absence of bias but instead indicate that discriminatory elements were not salient in this specific case scenario. From a safety and equity perspective, this underscores the importance of systematically evaluating both human and AI-generated health communication for subtle bias and unintended stereotyping. Future research should apply multidimensional and validated discrimination or bias assessment frameworks and include multiple case scenarios to enable more robust and generalizable conclusions.\u003c/p\u003e \u003cp\u003eExpert evaluations additionally favored the AI-generated response in terms of clinical accuracy and completeness. Although this observation contrasts with studies emphasizing limitations in the clinical appropriateness of AI-generated advice, particularly in complex cases (\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e), the present findings should be interpreted within the boundaries of a case-based comparison. Rather than challenging existing concerns, the results point to the need for more differentiated assessments that take task complexity and context into account.\u003c/p\u003e \u003cp\u003eBeyond individual dimensions, the broadly distributed equal ratings suggest that participants engaged in differentiated evaluations rather than uniformly favoring one response type. This pattern supports the interpretation that user perceptions of AI-supported communication are nuanced and context-sensitive, echoing prior work describing the situational acceptance of digital consultation formats (\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e, \u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eThe post-hoc sensitivity analysis indicated that the study was powered to detect effect sizes as small as Cohen's d\u0026thinsp;=\u0026thinsp;0.17. Given that all paired comparisons for empathy and comprehensibility reached significance at p \u0026lt; .001, the observed effects substantially exceeded this detection threshold, supporting the robustness of the reported preference patterns.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec16\" class=\"Section2\"\u003e \u003ch2\u003eImplications for digital health communication\u003c/h2\u003e \u003cp\u003eThe findings indicate that AI-generated responses may serve as a meaningful complement within digital health communication, particularly in low-threshold informational settings such as initial orientation or clarification of non-urgent concerns. Features such as consistent structure, accessible language, and supportive phrasing appear to align with user expectations in text-based environments. In health care systems facing increasing demand and resource constraints, these characteristics may help expand access to understandable health information.\u003c/p\u003e \u003cp\u003eAt the same time, the results should not be interpreted as supporting a substitution of professional consultation. Prior research has shown that acceptance of digital counseling decreases with increasing complexity and uncertainty (\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e, \u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e), and concerns regarding inconsistent quality and potentially harmful recommendations remain relevant (\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e, \u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e). From this perspective, AI-supported tools may be most appropriately positioned within a stepped communication model in which automated information complements, but does not replace, professional expertise.\u003c/p\u003e \u003cp\u003eThe growing presence of AI in health information seeking also raises questions about professional roles and competencies. Ensuring the safe integration of such tools will likely require not only technical oversight but also strengthened digital competencies among health care professionals. Educational approaches that enable critical appraisal of AI-generated content may therefore become increasingly important as digital counseling formats continue to expand (\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e).\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec17\" class=\"Section2\"\u003e \u003ch2\u003eLimitations\u003c/h2\u003e \u003cp\u003eSeveral limitations should be considered when interpreting the findings. First, the comparison was based on a single case example and one human response, which may not fully capture the variability of professional communication styles. The results therefore reflect a case-based evaluation rather than a generalized comparison between human and AI-supported consultation. Because evaluations were based on self-reported perceptions, reporting bias cannot be excluded. Although the study was sufficiently powered to detect small effects (Cohen's d\u0026thinsp;=\u0026thinsp;0.17), the ability to detect such small differences also implies that statistically significant results may not necessarily reflect practically meaningful distinctions in all cases. The clinical or communicative relevance of the observed differences should therefore be assessed independently of statistical significance.\u003c/p\u003e \u003cp\u003eSecond, the study relied on a convenience sample recruited through open online dissemination. Although this approach enabled broad participation, selection bias cannot be excluded, and the relatively high educational attainment within the sample may limit generalizability.\u003c/p\u003e \u003cp\u003e Third, participants evaluated static written responses without the opportunity for interaction. Health communication is typically dynamic and allows for clarification, follow-up questions, and adaptation to individual needs. The extent to which the observed preferences would persist in interactive settings remains uncertain.\u003c/p\u003e \u003cp\u003eFourth, expert assessments were conducted with a small sample and should be interpreted as exploratory. Larger expert panels and standardized evaluation frameworks would strengthen future comparisons of clinical quality.\u003c/p\u003e \u003cp\u003eFifth, the two responses were presented in a fixed order without randomization or counterbalancing. This design may have introduced order effects, as participants' evaluation of the second response could have been influenced by their prior reading of the first. Although the direction of such effects cannot be determined with certainty, anchoring or contrast effects may have systematically favored or disfavored the response presented second. Future studies should employ randomized or counterbalanced presentation sequences to control for this potential source of bias.\u003c/p\u003e \u003cp\u003eFinally, AI-generated responses are inherently sensitive to prompt design, model configuration, and ongoing technological development. The findings should therefore be understood as reflecting the capabilities of the model at the time of data collection rather than representing a fixed performance estimate.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec18\" class=\"Section2\"\u003e \u003ch2\u003eFuture research\u003c/h2\u003e \u003cp\u003eFurther studies should examine multiple case scenarios across different clinical domains and levels of complexity to better understand where AI-supported communication is perceived as beneficial and where its limitations become more apparent. Research incorporating interactive dialogue formats may provide additional insight into how users engage with AI-generated advice in more realistic communication settings.\u003c/p\u003e \u003cp\u003eIn addition, qualitative approaches could help clarify which linguistic and structural features shape perceptions of empathy and comprehensibility. Establishing robust evaluation criteria for AI-supported health communication remains an important step toward ensuring both quality and safety as these technologies become more widely integrated into digital care environments.\u003c/p\u003e \u003c/div\u003e"},{"header":"Conclusion","content":"\u003cp\u003eThe present study suggests that AI-generated responses can meet central user expectations regarding clarity, structure, and perceived empathy in text-based health communication. At the same time, the findings highlight that perceived communicative strength should be considered alongside ongoing concerns related to clinical appropriateness, safety, and equity.\u003c/p\u003e \u003cp\u003eRather than positioning AI as a replacement for professional consultation, the results support a complementary perspective in which AI-assisted communication may enhance access to understandable health information while professional expertise remains essential for complex decision-making. Continued empirical evaluation will be necessary to further define appropriate roles, boundaries, and quality standards for the responsible use of AI in digital health communication.\u003c/p\u003e \u003cp\u003eAgainst the backdrop of increasing demand and limited professional resources, AI-supported communication may contribute to improving access to understandable health information, particularly in low-threshold informational settings. At the same time, careful integration into existing care structures will be essential to ensure that technological support enhances, rather than fragments, the quality of health care delivery.\u003c/p\u003e"},{"header":"Abbreviations","content":"\u003cp\u003eAI: Artificial intelligence\u003c/p\u003e\n\u003cp\u003eCI: Confidence interval\u003c/p\u003e\n\u003cp\u003eICN: International Council of Nurses\u003c/p\u003e\n\u003cp\u003eICMJE: International Committee of Medical Journal Editors\u003c/p\u003e\n\u003cp\u003eLLM: Large language model\u003c/p\u003e\n\u003cp\u003eOR: Odds ratio\u003c/p\u003e\n\u003cp\u003eSD: Standard deviation\u003c/p\u003e\n\u003cp\u003eSPSS: Statistical Package for the Social Sciences\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eEthics approval and consent to participate\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe study did not involve patients or vulnerable populations and was restricted to an anonymous online survey in which participants evaluated two pre-existing, publicly available text-based responses to a health forum query.\u0026nbsp;According to the Information Sheet on the Ethics Committee Regulations of Alice Salomon University of Applied Sciences Berlin, such projects do not require formal ethical approval (27). \u0026nbsp;Nevertheless, the study was conducted in accordance with internationally recognized ethical principles, including the ICN Code of Ethics and the Declaration of Helsinki. All participants were fully informed about the study\u0026rsquo;s purpose and procedures, provided informed and voluntary consent, and were free to withdraw at any time without consequences.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConsent for publication\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAvailability of data and materials\u003c/strong\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eThe datasets used and analyzed during the current study are available from the corresponding author upon reasonable request.\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003e\u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003e\u003cem\u003e\u003cstrong\u003eCompeting interests\u003c/strong\u003e\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eThe authors declare that they have no competing interests.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNo specific funding was received for this research from public, commercial, or not-for-profit funding agencies. Open access publication was supported by the Open Access Publication Fund of Alice Salomon University of Applied Sciences Berlin.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthors\u0026apos;\u003c/strong\u003e\u003cstrong\u003e\u0026nbsp;contribution\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAccording to ICMJE guidelines, all authors 1) have made substantial contributions to the conception and design, or acquisition of data, or analysis and interpretation of data; 2) have been involved in drafting the manuscript or revising it critically for important intellectual content; 3) have given final approval of the version to be published; and 4) agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eCS: study design, data collection, data analysis, manuscript preparation; PA: data collection, data analysis, manuscript approval; EF: data collection, data analysis, manuscript approval; EK: data collection, data analysis, manuscript approval; IL: data collection, data analysis, manuscript approval; EP: data collection, data analysis, manuscript approval; KP: data collection, data analysis, manuscript approval; LR: data collection, data analysis, manuscript approval; IT: data collection, data analysis, manuscript approval; JG: study design, data collection, data analysis, manuscript preparation.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAcknowledgements\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eEurostat. People online in 2024: Eurostat News; 2024 [updated 2024/12/17. Available from: https://ec.europa.eu/eurostat/web/products-eurostat-news/w/ddn-20241217-1.\u003c/li\u003e\n\u003cli\u003eEurostat. Digitalisation in Europe \u0026ndash; 2025 edition: Finding health information online: Eurostat; 2025 [Available from: https://ec.europa.eu/eurostat/web/interactive-publications/digitalisation-2025.\u003c/li\u003e\n\u003cli\u003eAyre J, Cvejic E, McCaffery KJ. Use of ChatGPT to obtain health information in Australia, 2024: insights from a nationally representative survey. Medical Journal of Australia. 2025;222(4):210-2.\u003c/li\u003e\n\u003cli\u003eKiuchi K, Umehara H, Irizawa K, Kang X, Nakataki M, Yoshida M. An exploratory study of the potential of online counseling for university students by a human-operated avatar counselor. Healthcare. 2024;12(13):1287.\u003c/li\u003e\n\u003cli\u003eCampbell K, Greenfield G, Li E, O\u0026rsquo;Brien N, Hayhoe B, Beaney T. The impact of virtual consultations on the quality of primary care: systematic review. Journal of Medical Internet Research. 2023;25:e48920.\u003c/li\u003e\n\u003cli\u003eMoulaei K, Sheikhtaheri A, Fatehi F, Shanbehzadeh M, Bahaadinbeigy K. Patients\u0026rsquo; perspectives and preferences toward telemedicine versus in-person visits: A mixed-methods study on 1226 patients. BMC Medical Informatics and Decision Making. 2023;23(1):261.\u003c/li\u003e\n\u003cli\u003eTurner A, Morris R, Rakhra D, Stevenson F, Hamilton F, Atherton H. Unintended consequences of online consultations: a qualitative study in UK primary care. British Journal of General Practice. 2022;72(715):e128-e37.\u003c/li\u003e\n\u003cli\u003eMiles O, West R, Nadarzynski T. Health chatbots acceptability moderated by perceived stigma and severity: a cross-sectional survey. Digital Health. 2021;7:20552076211063012.\u003c/li\u003e\n\u003cli\u003eNadarzynski T, Bayley J, Llewellyn C, Kidsley S, Graham CA. Acceptability of artificial intelligence-enabled chatbots, video consultations and live webchats as online platforms for sexual health advice. BMJ Sexual \u0026amp; Reproductive Health. 2020;46(3):210-7.\u003c/li\u003e\n\u003cli\u003eNadarzynski T, Puentes V, Pawlak I, Mendes T, Montgomery I, Bayley J, et al. Barriers and facilitators to engagement with artificial intelligence-based chatbots for sexual and reproductive health advice. Sexual Health. 2021;18(5):385-93.\u003c/li\u003e\n\u003cli\u003eHowcroft A, Bennett-Weston A, Khan A, Griffiths J, Gay S, Howick J. AI chatbots versus human healthcare professionals: A systematic review and meta-analysis of empathy in patient care. British Medical Bulletin. 2025;156(1):ldaf017.\u003c/li\u003e\n\u003cli\u003eAyers JW, Poliak A, Dredze M, Leas EC, Zhu Z, Kelley JB. Comparing physician and artificial intelligence chatbot responses to patient questions posted to a public social media forum. JAMA Internal Medicine. 2023;183(6):589-96.\u003c/li\u003e\n\u003cli\u003eChen D, Chauhan K, Parsa R, Liu ZA, Liu FF, Mak E. Patient perceptions of empathy in physician and artificial intelligence chatbot responses to patient questions about cancer. NPJ Digital Medicine. 2025;8(1):275.\u003c/li\u003e\n\u003cli\u003eMathes S, Seurig S, Bluhme F, Beyer K, Heizmann F, Wagner M. ChatGPT performance on 120 interdisciplinary allergology questions\u0026mdash;systematic evaluation with clinical error impact assessment for critical erroneous AI-guided chatbot advice. Journal of Allergy and Clinical Immunology: In Practice. 2025;13(6):1350-7.e4.\u003c/li\u003e\n\u003cli\u003eJo E, Song S, Kim JH, Lim S, Kim JH, Cha JJ. Assessing GPT-4\u0026rsquo;s performance in delivering medical advice: Comparative analysis with human experts. JMIR Medical Education. 2024;10:e51282.\u003c/li\u003e\n\u003cli\u003eWilhelm TI, Roos J, Kaczmarczyk R. Large language models for therapy recommendations across 3 clinical specialties: Comparative study. Journal of Medical Internet Research. 2023;25:e49324.\u003c/li\u003e\n\u003cli\u003eArmbruster J, Bussmann F, Rothhaas C, Titze N, Gr\u0026uuml;tzner PA, Freischmidt H. \u0026ldquo;Doctor ChatGPT, can you help me?\u0026rdquo; The patient\u0026rsquo;s perspective: Cross-sectional study. Journal of Medical Internet Research. 2024;26:e58831.\u003c/li\u003e\n\u003cli\u003eScaff SPS, Reis FJJ, Ferreira GE, Jacob MF, Saragiotto BT. Assessing the performance of AI chatbots in answering patients\u0026rsquo; common questions about low back pain. Annals of the Rheumatic Diseases. 2025;84(1):143-9.\u003c/li\u003e\n\u003cli\u003ePfohl SR, Cole-Lewis H, Sayres R. A toolbox for surfacing health equity harms and biases in large language models. Nature Medicine. 2024;30(12):3590-600.\u003c/li\u003e\n\u003cli\u003eHausmann LRM, Hannon MJ, Kresevic DM, Hanusa BH, Kwoh CK, Ibrahim SA. Impact of perceived discrimination in health care on patient\u0026ndash;provider communication. Medical Care. 2011;49(7):626-33.\u003c/li\u003e\n\u003cli\u003evon dem Knesebeck O, Dingoyan D, Makowski A, Klein J, Ludecke D. Intersectional inequalities in interpersonal discrimination in outpatient care according to sex, history of migration, and income in Germany. European Journal of Public Health. 2025;36(1):20-4.\u003c/li\u003e\n\u003cli\u003eBabaii A, Mohammadi E, Sadooghiasl A. The meaning of the empathetic nurse\u0026ndash;patient communication: A qualitative study. Journal of Patient Experience. 2021;8:1-9.\u003c/li\u003e\n\u003cli\u003eCampos CFC, Olivo CR, Martins MdA, Tempski PZ. Physicians\u0026rsquo; attention to patients\u0026rsquo; communication cues can improve patient satisfaction with care and perception of physicians\u0026rsquo; empathy. Clinics. 2024;79:100377.\u003c/li\u003e\n\u003cli\u003eS\u0026oslash;rensen K, Van den Broucke S, Fullam J, Doyle G, Pelikan J, Slonska Z, et al. Health literacy and public health: A systematic review and integration of definitions and models. BMC Public Health. 2012;12:80.\u003c/li\u003e\n\u003cli\u003eSzab\u0026oacute; P, B\u0026iacute;r\u0026oacute; \u0026Eacute;, K\u0026oacute;sa K. Readability and comprehensibility of printed patient education materials. Frontiers in Public Health. 2021;9:725840.\u003c/li\u003e\n\u003cli\u003eFaul F, Erdfelder E, Lang A-G, Buchner A. G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods. 2007;39(2):175-91.\u003c/li\u003e\n\u003cli\u003eBorde T. Satzung der Ethikkommission der Alice Salomon Hochschule Berlin [Regulations governing the Ethics Committee of Alice Salomon University of Applied Sciences Berlin]. Alice Salomon Hochschule Berlin; 2019. \u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bmc-health-services-research","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"bhsr","sideBox":"Learn more about [BMC Health Services Research](http://bmchealthservres.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/BHSR/default.aspx","title":"BMC Health Services Research","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Artificial intelligence , Digital health communication , Patient perspective , Health information , Empathy , Health services research","lastPublishedDoi":"10.21203/rs.3.rs-9252051/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-9252051/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cb\u003eBackground\u003c/b\u003e\u003c/p\u003e \u003cp\u003eDigital health communication is gaining importance as health care systems face increasing demand and structural constraints. Generative artificial intelligence (AI) tools such as ChatGPT are increasingly used for health information seeking; however, direct comparisons between AI-generated and human responses from the user perspective remain limited.\u003c/p\u003e\u003cp\u003e\u003cb\u003eMethods\u003c/b\u003e\u003c/p\u003e \u003cp\u003eA quantitative exploratory cross-sectional online survey using a paired-response design was conducted in Germany. Participants evaluated two responses to the same real-world health query: one written by a physiotherapist and one generated by ChatGPT-4. Outcomes included perceived empathy, comprehensibility, potential discrimination, and expert-rated clinical quality. Paired comparisons were analyzed using Wilcoxon signed-rank tests, and logistic regression models were applied to explore associations between participant characteristics and preference patterns.\u003c/p\u003e\u003cp\u003e\u003cb\u003eResults\u003c/b\u003e\u003c/p\u003e \u003cp\u003eThe analytical sample comprised 224 participants. Across most items, the AI-generated response was rated more favorably than the human response, particularly with regard to comprehensibility and empathy (all p \u0026lt; .001). Preference patterns were highly consistent across both domains, whereas perceptions of discriminatory content were rare and did not differ significantly. Expert evaluations also favored the AI-generated response in terms of clinical accuracy and completeness, although these findings should be interpreted cautiously given the exploratory design.\u003c/p\u003e\u003cp\u003e\u003cb\u003eConclusions\u003c/b\u003e\u003c/p\u003e \u003cp\u003eThe findings suggest that AI-generated communication may align closely with central user expectations in text-based health information contexts, particularly regarding clarity and supportive language. At the same time, perceived communicative strengths should be considered alongside ongoing concerns related to clinical appropriateness, safety, and equity. These results support a differentiated understanding of AI-supported communication as context-dependent rather than universally preferable. AI-generated responses may complement professional consultation by improving access to understandable health information, particularly in low-threshold informational settings. Careful integration into existing care structures will be essential to ensure that technological support enhances the quality of health care delivery without replacing professional expertise.\u003c/p\u003e","manuscriptTitle":"User evaluations of AI- and human-generated responses in digital health communication: A paired survey study","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-05-06 18:31:42","doi":"10.21203/rs.3.rs-9252051/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"editorInvitedReview","content":"","date":"2026-05-19T02:51:43+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-05-08T03:07:44+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"228792391090547738500524491368137190557","date":"2026-05-07T12:29:08+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"242093537984274129388832917470730155082","date":"2026-05-04T23:58:05+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-04-29T12:55:47+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"290585148816107187430373459491149088879","date":"2026-04-27T16:41:12+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2026-04-27T09:23:15+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2026-04-02T20:46:29+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2026-03-31T00:36:15+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2026-03-31T00:35:18+00:00","index":"","fulltext":""},{"type":"submitted","content":"BMC Health Services Research","date":"2026-03-28T10:44:29+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bmc-health-services-research","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"bhsr","sideBox":"Learn more about [BMC Health Services Research](http://bmchealthservres.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/BHSR/default.aspx","title":"BMC Health Services Research","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"91706cc6-e145-40a2-87c9-54d0973c3993","owner":[],"postedDate":"May 6th, 2026","published":true,"recentEditorialEvents":[{"type":"editorInvitedReview","content":"","date":"2026-05-19T02:51:43+00:00","index":69,"fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-05-08T03:07:44+00:00","index":68,"fulltext":""},{"type":"reviewerAgreed","content":"228792391090547738500524491368137190557","date":"2026-05-07T12:29:08+00:00","index":67,"fulltext":""},{"type":"reviewerAgreed","content":"242093537984274129388832917470730155082","date":"2026-05-04T23:58:05+00:00","index":60,"fulltext":""}],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2026-05-06T18:31:42+00:00","versionOfRecord":[],"versionCreatedAt":"2026-05-06 18:31:42","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-9252051","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-9252051","identity":"rs-9252051","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}